Apparently one of the lemmy.ml admins was overzealous in banning all User-Agent strings that contained the word “bot”. Bans were entered for all of the individual strings containing that word which were observed in their webserver logs, which impacted kbin’s reported agent of “kbinBot”.
The issue has been fixed, and I observed that one of my kbin posts to a lemmy.ml community was successfully pushed to the original instance.
Edit:
Here are all the links that I’ve found with the lemmy.ml admins discussing the issue:
- https://lemmy.world/comment/1082149 (appears to have been where they first noticed and fixed the issue)
- https://lemmy.ml/post/1920972 (posted a few hours later, provides additional context for what happened)
There are plenty of high volume, non-malicious bots that do. robots.txt is a thing for a reason, and we can see here that lemmy.ml has implemented it. Not all bots that ignore robots.txt are malicious though, just poorly designed. You can basically lump them into three categories:
Their logs told them they had a lot of traffic from stuff identifying itself as bots. Throwing that traffic out wouldn’t break lemmy but would help them deal with the capacity problems that all of the mainstream lemmy/kbin instances had to deal with shortly after the Reddit exodus began. They fucked up and tagged kbin in the process, which definitely would have been one of the highest volume ActivityPub consumers matching their criteria.