Re: sorting large numbers of large records

[mheaney@on2.com (Matthew Heaney)]

Hyman Rosen <hyrosen@mail.com> wrote in message news:<jyFVa.715$7h6.251@nwrdny03.gnilink.net>...
> Keith Thompson wrote:
>  > If you're using GNU sort, the length of each line can be unlimited;
> 
> So there :-)
> 
> > Finally, I'm not sure what GNU sort (or any other Unix-like sort) does
> > with input too big to fit into memory;
> 
> As far as I know, UNIX sort has always been able to sort arbitrarily large
> files by doing what I suggested in an earlier message - sort pieces of the
> original into temporary files, then merge them together.

Actually, since these are fixed size records, you could do a sort by
simply exchanging record values on disk.  You have random access via
Direct_IO.

You could also map the entire file, to view it as one large array.

In either case you could use a generic algorithm (the one in Charles
would be adequate) to do the work.