Re: Howto read line from a stream

["Randy Brukardt" <randy@rrsoftware.com>]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:1dlvlys7r0tyh$.zyytlf2vt5zm.dlg@40tude.net...
> On Mon, 1 Jun 2009 18:30:31 -0500, Randy Brukardt wrote:
...
> I don't think it is a good idea, how to handle poorly designed and badly
> implemented protocols. We are living in a real world. It is silly to imply
> that an incoherent implementation should indicate a security problem. The
> problem is elsewhere, in the protocol itself. Anyways, I doubt that
> somebody could statistically confirm that the percentage of malformed line
> terminators is higher among unwanted postings.

Maybe in general, but in the case of spam, there is a very strong 
correlation, simply because the majority of all legitimate mail is sent by 
only a few mail clients (Outlook, Thunderbird, GMail, etc.) and these do in 
fact follow the protocol standards. So anything that doesn't follow it is 
much more likely to be junk. Indeed, spam filtering is 100% about apply 
statistical models, since there is really no way to predict what good mail 
will look like.

It's possible that there is more of this problem with other mail these days; 
I would think that spammers would be more careful these days if for no 
reason than such malformed mail never gets delivered. But there still is an 
amazing amount of malformed mail.

Note that I created and manage the spam filter (in Ada, of course) on our 
mail servers, and we average 100,000 spams per week (about 99% spam). So I 
see a lot of spam in a week, even now when 97% or so gets automatically 
blocked or deleted.

> P.P.S. There are lots of text-oriented protocols used in automotive. The
> device vendors just do not care to handle LFs and CRs as specified (if
> specified! (:-)). You buy a $10**5+ device and have to communicate with 
> it.
> Guess which side has to be fixed?

I would argue that is mainly a problem with the specifications, rather than 
the device. But I feel your pain; hardly anything in the real world quite 
follows the specifications. Generally, you have handle screwed up line 
endings, but remember that fact so that you can then quarantine the mail 
(the reason being that you still need to scan the mail to see if it would be 
automatically deleted for other reasons). So it's pretty much the worst of 
both worlds. Browsers and HTML are even worse; if they refused to display a 
lot of improper junk, filtering would not be so hard.

                                                         Randy.