Hello! I’m setting up a kbin instance, and while it’s a little hard to tell thanks to the 0.19 breakage, it seems to be interoperating to some degree with most of the fediverse. I can post messages back and forth, at least.
With lemmy.world, though, it’s not working. I get messages like this in the log:
Get fail: https://lemmy.world/comment/6458819, https://lemmy.world/comment/6458819
Bots Temporarily Blocked
And, indeed, it seems like lemmy.world is configured to reject connections from user-agents containing the word “bot”. This is what happens when I access the same URL while manually setting user-agent to what kbin uses for client HTTP requests:
$ wget -nv --user-agent="kbinBot/0.1 (+https://my.domain/bot)" https://lemmy.world/comment/6458819
https://lemmy.world/comment/6458819:
2024-01-07 05:25:43 ERROR 412: Precondition Failed.
And, verifying that the word “bot” is the issue:
$ wget -nv --user-agent="kbinAgent/0.1 (+https://my.domain/agent)" https://lemmy.world/comment/6458819
2024-01-07 05:25:56 URL:https://lemmy.world/comment/6458819 [264000] -> "6458819.2" [1]
$ wget -nv https://lemmy.world/comment/6458819
2024-01-07 05:26:06 URL:https://lemmy.world/comment/6458819 [264006] -> "6458819.3" [1]
While I can understand that many bots are harmful so this is probably a necessary measure, it seems to me like this will break federation with all kbin instances (unless they figure out to change their user agent). No? Would it maybe be a good idea to add an exception to this for kbinBot? I may have misunderstood something, as I’m new to all this, just saying how it looks to me poking at it briefly.
Also, is it okay if I change my user-agent so that it doesn’t contain the forbidden word? Presumably there’s no administrative reason kbin and lemmy.world shouldn’t talk.
You must log in or # to comment.
We are temp blocking most bot traffic for now, due to it causing a lot of slowness on LW. Please don’t change your user agent string to get around it, that would be very bad.
We can turn it back on and see if things still slow to a crawl again.
We are also looking to load balancing the bot traffic better, in the future.
It’s nothing personal, just trying to look out for the human users on the site.
Any questions feel free to DM me.
You saw that this isn’t a “bot” in the normal sense of the word, but just a kbin instance doing federation, right?
I mean I’m fine with however you want to limit access; I’ll leave it as “kbinBot” so it’ll get blocked, if that’s really what you want, but completely preventing a kbin instance from federating with lemmy.world because of how the user-agent looks seems like maybe not the right thing to do. (Assuming that’s what’s going on – kbin.social identifies itself as kbinBot also, and its posts seem like they’re going through, so maybe I’m just misunderstanding something about the situation.)
We say “Bot” to everything not user/frontend interaction. It was just in the last time very much. Almost up to 2.7 million. If we didnt knew it was “just the federation” we thought we would get ddosed. After we actived the “Bot” blocking it went under 500 active connections per hour.
Hey, I added a exception for kbin and mbin bots, so you should be good now. It was never meant to be a permanent block, just technical testing. Also I did specifically allow kbin.social as well a few days ago, so they should have been fine.
Please let me know if you have any technical issues going forward 💗
-Jelloeater
Ayyy, that’s wonderful! Thank you, that’s awesome. It still doesn’t work (my log says things like “failed: Failed to get chain object https://lemmy.world/comment/6479326”), but presumably that’s a problem on the kbin side, and I’m happy digging into it and seeing what I can find. If I have any questions or anything I’ll reach out, and thanks again for the quick response + resolution.
