Re: ANSI C and POSIX (was Re: C/C++ knocks the crap out of Ada)

[c2a192@ugrad.cs.ubc.ca (Kazimir Kylheku)]

In article <dewar.829011320@schonberg>, Robert Dewar <dewar@cs.nyu.edu> wrote:
>Peter said
>
>"How?  No offense meant, but any code which can be affected by this is flat
>out broken.  POSIX-style read is to be given a pointer to at least nbytes
>of space, for the information read.  Period."
>
>That's really confusing, the code in question DID give a buffer large
>enough to hold nbytes of data, where nbytes is the number of bytes 
>for "the information read". Maybe I don't understand, but reading the
>above sentence, it sounds like you would be surprised by the Linux
>behavior.
>
>Here is the exact case. We declare a buffer of 100 bytes. We read a
>1000 bytes from a file whose total length is 68 bytes. On all systems
>that we had experience with other than Linux, this worked fine, the
>first 68 bytes of the buffer is filled, and the remaining 32 bytes
>is unused. 

This is poor coding. You are _advertizing_ a buffer of size 1000, but passing a
pointer to a 100 byte buffer. It wouldn't even occur to me to do this, and
until now I have been completely oblivious to this difference between Linux and
other systems.

Unfortunately, I could not find anything in POSIX.1 that would explicitly
disallow this. The document is not very assertive in defining undefined
behavior. I'm going to check it once again in case I missed something.

I checked the manual pages for read() on several systems. Linux documents that
EFAULT results if the buffer pointed at by buf is outside of the address space
of the process. On other systems, it is claimed that EFAULT results if the buf
pointer is directed outside of the address space.

>I am not claiming this is "correct" code in some abstract sense. I
>certainly can't tell that it is wrong from the definitions I have
>of the read function. What I am claiming is that this worked on
>all systems we tried it on, and then failed on Linux. I am not saying
>Linux is wrong here, just that its behavior was surprising!

It's not surprising: you lied to the read() function. But you are right, you
can't tell this from the definition in POSIX.1 or from typical manual pages.
There are certain unwritten rules, though!

>The code in question made 100% sure that the data read would never
>exceed the buffer size, and I would have been hard pressed to
>determine that the code was incorrect. 

Really? What if another process appended data to the file before you got to the
end? A simple script running as the same user ID as the program would always
have the right permissions to do this, provided the file in question is not a
temporary file that was unlinked before being written.

What prevented the code from advertizing the true buffer size of 100?

Can we see this code, along with some of its context?

In any case, it doesn't sound like a case of ``defensive programming''.

>I am not sure that POSIX is relevant here, almost none of the systems on
>which we ran claimed POSIX compliance. Peter, can you post the POSIX
>wording on read, I don't have it at hand. Does it in fact make it
>clear that the Linux behavior is correct and that the program was
>wrong.

The POSIX.1 wording on read() doesn't get into anything that is remotely
helpful in this discussion at all. In fact, the whole document is only an
interface specifiction that basically tells you how these functions are called
and what are their semantics, and minimal error conditions.

>Let's suppose that the POSIX standard does in fact make it clear that
>the Linux behavior is correct. I still think the check is unwise

I checked the POSIX.1 document and it does not make it clear. The importance of
the EFAULT checks is downplayed, and I could not find a definite statement
in the document that passing illegal arguments even leads to undefined behavior.
There is a gaping lack of a comprehensive list of what constitutes non-portable
or illegal use of the functions, as far as I could tell.

>(note that the check is not against the actual size of the buffer
>given, this is of course impossible in C, it is against the end
>of the address range of the data area). It's a good example of the
>kind of principle I mentioned before. Since almost all systems allow
>the program I described above to work correctly, and it is manifestly
>safe programming evenif the check is not present, I think it would
>be a better choice for Linux not to do this extra check.

And allow kernel code to potentially make an illegal reference to a not present
page? I'd rather not have it that way just to fix the extremely rare buggy
program. Perhaps checking the bounds just for the actual data transferred might
be better. But this would complicate the kernel code.  It's easy to check the
buffer first, immediately upon entry into the read() system call, when it is
not known how many bytes will be read from the given file or device.  From
there, control can pass to any of a number of subsystems  via a function
pointer in the f_op structure of the file node.  Since the check is done at the
entry point, the individual read() functions of various filesystem modules and
device drivers can confidently use the entire buffer without having to do their
own check after determining the actual transfer size.  IMHO, requiring the
individual read functions to do their own checking would result in
unnecessary complication, when it it so easy and obvious to do a comprehensive
check in the central sys_read() dispatcher! When working on a new device
driver, I wouldn't want to mess around with details that should be taken care
for me.

If you want an example of behavior that is unequivocally ``different'', look no
further than the Linux select() call. Unlike what happens on other UNIXes, the
Linux select() system call modifies the timeval structure. Code that depends on
the structure not being modified will break. This tripped me up, and caused
some fairly prominent programs to behave differently (including some versions
of the Mosaic web browser, which used select() as a delay mechanism. The little
globe picture would spin like crazy, since Mosaic assumed that the delay values
were preserved for multiple invocations of select rather than decremented down
to nothing!)
--