please see both the article as well as the comments:

https://www.techdirt.com/2025/07/14/tackling-the-ai-bots-that-threaten-to-overwhelm-the-open-web/

would expect to see more of these types of apps:
https://geekflare.com/consumer-tech/apps-to-sell-internet-data-and-earn-money/

Protect Your Site with a Blackhole for Bad Bots
https://perishablepress.com/blackhole-bad-bots/

One potential downside to this approach is that the IP address block list will likely grow, over time, to be very large. One solution to this problem is to periodically prune the list by removing the oldest IP addresses, or have them automatically expire after a certain period of time.

The Devuan git store has had address block blocks a long time, and it does work to some extent. But as you would know, it's a method that requires constant monitoring and tuning, and that one can accept longish periods of seriously degraded service.

I don't want to force myself on anyone, if my help is not wanted that's perfectly fine for me, but that can be said politely rather than arrogantly dismissing it as "childish noise", don't you think so?

Yea we tried blocking IP addresses, but we stopped at some 50000 addresses blocked

Well of course it didn't work for you  if you only blocked 50k addresses... a single /16 is already 65536 addresses, so 50k is chicken feed...

No problem, I'll  keep my "childish noise" that works great for my websites to myself...

Yeah it's not an intentional DDOS attack, but the practical effect of all these rouge bots scraping all they can get can be like a DDOS.

Because a ddos on a linux git site, one that's NOT in bed with microshit, redcrap, and angry kiddie land (gnomers), seems suspicious.

Anyone with a viable solution may contribute. All else is just childish noise.

So at the moment I reserve judgment.

That said, with regards to the DDOS by dodgy AI scrapers, I have that problem too on my websites and I solved it with IP address block lists implemented with ipset using the kernel firewall of linux. I don't block single IPs I block whole /24, /16 or even /12 network ranges of many hosting/cloud providers, as I found that these attacks predominantly come from very specific cheap/dodgy hosting/cloud providers, almost never from end customer (DSL/fiber/mobile) IP addresses.

If the admins of git.devuan.org are interested in my blocklist I can provide it, but only privately as sharing it publicly could induce the AI scrapers to adapt their strategies.

just use a user-agent that displays the actual o/s family (i.e. Linux).

From the torbrowser design docs:

Likely, if the dev's attitude is anything to go by:

AKA: "We only support ie6. Use a conforming browser. Anyone who doesn't conform is a weirdo and anubis will block them."

We rail against gratuitous javascript and browser fingerprinting, and we promote privacy... Except here, where apparently it's just fine.
We rail against forced systemd wayland adoption software-conformity, and advocate for user-choice... Except here, where apparently it's just fine.

This thing is blatantly speech-police for web browsers, and one small step away from a de-anonymisation engine. It uses all the same methods we love to hate when others do it. I'd make a slippery-slope comment, but you've heard them all already.

Could you mitigate the scrapers by rate-limiting by IP address? Eg addresses making more that 10 requests per minute get responses delayed so they only get 10 responses per minute. That would not be too bad for a human, but hurt scrapers trying to train an AI system.

(from reading back in my irc logs) Something similar was tried for a short time.  Unfortunately, the first "D" in DDoS (Distributed) meant that approach was not very effective.  After successfully slowing down the bots for a few days via IP-address blocking, they adapted to use a unique IP address for each request. Before anubis was added to the site, the web interface was pretty much unusable most of the time.

If that's all that is done, just use a user-agent that displays the actual o/s family (i.e. Linux).  There are several "valid" (default for a browser) user-agents that could be rotated through, without needing to report Windows as the o/s.  I know it's not a perfect solution for privacy, but the user-agent string matching the operating system probably isn't a huge deal.  I wonder what happens if your user-string says "MyBrowser 1.2.3.4" - will anubis block access entirely?

FWIW, I see more cpu activity going to other websites than I do with git.devuan.org. I tested cbsnews.com, cnn.com and youtube.com.

Sites that are obviously training an AI system should be fed AI generated drivel so they end up with an AI system that produces rubbish output.