From mboxrd@z Thu Jan 1 00:00:00 1970 Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail From: "Randy Brukardt" Newsgroups: comp.lang.ada Subject: Re: "Red" And The DoD Language Competition Date: Sat, 14 Sep 2024 01:27:22 -0500 Organization: A noiseless patient Spider Message-ID: References: <0001HW.2C93AE0500F5F854306B4D38F@news.individual.net> <87seu42t7x.fsf@nightsong.com> Injection-Date: Sat, 14 Sep 2024 08:27:26 +0200 (CEST) Injection-Info: dont-email.me; posting-host="252dd07fce3ba32f52749428b1b0da73"; logging-data="1414132"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19MeyAzG6b1XAFIr2Xe8/ZvfSRevrWXATU=" Cancel-Lock: sha1:HgdW2KXZ4LqEqPhmFLoVWgvWpG4= X-Newsreader: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.7246 X-RFC2646: Format=Flowed; Original X-MSMail-Priority: Normal X-Priority: 3 Xref: news.eternal-september.org comp.lang.ada:66370 List-Id: "Lawrence D'Oliveiro" wrote in message news:vc091i$ljiq$2@dont-email.me... > On Thu, 12 Sep 2024 19:16:50 -0700, Paul Rubin wrote: > >> I run into sites all the time that block the wget user agent, but that I >> can retrieve with curl. > > And I run into sites all the time that block the default wget user agent, > but that I can retrieve with wget. You're confused. The attackers aren't using Wget, but they are *claiming* to be WGet. As you point out, real WGet users tend to claim to be other things. So blocking WGet would be more likely to block the attackers than real users. (As you state, real users know how to get around the blocks, so the inconvinience for them is minor. Usually, the attackers don't change their attacks often, there's plenty of sites that don't protect themselves at all. So they are more effective against attackers.) And anyone that thinks that ad revenue is important is probably blocking all grabbers, and probably throttling everything else so that grabbing multiple pages is very slow (at human reading speeds). (At least 90% of the browser hits I see are obviously fake, and if I cared enough I would block all of them - it would just take a bit of programming to check if the behavior is similar to that of a live human. But I only block when something is causing performance problems, and generally by IP.) Randy. Randy.