Re: reading a text file into a string

["Martin Dowie" <martin.dowie@baesystems.com>]

Randy Brukardt wrote:
> Anyway, my opinion these days is that spending a lot of effort making
> something run 2% faster is wasted effort. You're always better off
> changing to a different way of solving the problem. The most recent
> instance was in my web log analyzer. It was running too slow on the
> AdaIC site's logs, and I wasted a lot of time trying to improve it.
> But replacing the binary lookups (log N, N being around 200,000) by a
> hashed lookup (very similar to switching from Sorted _Sets to
> Hashed_Maps in the Containers library) improved the speed by a factor
> of 5 (a result I didn't expect, because I had to use an expensive
> hash function -- all of the cheap ones I tried didn't work well on
> the actual data -- and log N wasn't that large -- between 12 and 19
> on the data I tested with). Moral: make sure you've exhausted
> algorithmic improvements before even thinking about squeezing a few
> extra percent out of the code. And when you think you've exhaused
> algorithmic improvements, try again, because sometimes non-obvious
> things work! (We hadn't originally used a hash because of the need to
> write sorted reports. But it turned out that using a hash and a
> quicksort on the report was faster than keeping the data sorted.)

I'd whole heartedly second this advice. It reminds me of a recent case
where a colleague had a program that seemed to be taking forever. I can't
recall what data structure he was using but he worked out that at it's
current speed it was going to need something fast approximating to the
entire life of the universe so far to complete! He switched to a 'quadtree'
and "bingo" - it only took a few (tens of) hours!

Cheers

-- Martin