Re: Optimization Question

["M. Kotiaho" <mylastname@mail.com>]

dvdeug@my-deja.com wrote:

> In article <94g431$ge3$1@nnrp1.deja.com>,
>   Robert Dewar <robert_dewar@my-deja.com> wrote:
> > In article <94ftfu$b59$1@nnrp1.deja.com>,
> >   dvdeug@my-deja.com wrote:
> >
> > <<questions about speeding up code snipped>>
> >
> > I am *amazed* that this is only ten times slower when the
> > I/O is done in such a perfectly gruesome manner (sequential
> > I/O instantiated on bytes).
> >
> > It is elementary that you want to read big chunks of a file
> > at a time. What GNAT does is to read the entire source of
> > a program in one read statement.
>
> Actually, it's only about 3-5 times slower, after doing more careful
> measurements. I think GNU strings might be too wrapped up in the details
> of object code to be a good general purpose strings program.
>
> Unfortunately, that "perfectly gruesome manner" is the only one that
> jumps out at me. I don't want to use Ada.Text_IO, because I want control
> of CR's and LF's. I don't see how to use Sequential I/O on larger
> chunks, as I may end up with a piece of file that doesn't fill a chunk.
> Direct I/O can't be faster than sequential I/O. And reading a 17GB file
> into memory doesn't seem like a good idea.
>
> --
> David Starner - dstarner98@aasaa.ofe.org
>
> Sent via Deja.com
> http://www.deja.com/

As has been pointed out elsewhere, you can use Streams to read "big chunks"
at a time.  As you pointed out, you don't want to read 17 GB into memory.
You can still speed things up by using, say, 10K chunks ... the Read
procedure
has a "Last" parameter that will tell you if you ran out of file before you
filled your
chunk.  You also might want to buffer your output, rather than outputting
the
strings as soon as they are located.

HTH,

M. Kotiaho