From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=BAYES_00,FORGED_GMAIL_RCVD, FREEMAIL_FROM autolearn=no autolearn_force=no version=3.4.4 X-Google-Thread: 103376,b5cd7bf26d091c6f X-Google-NewGroupId: yes X-Google-Attributes: gida07f3367d7,domainid0,public,usenet X-Google-Language: ENGLISH,ASCII-7-bit Path: g2news1.google.com!news4.google.com!feeder.news-service.com!feeder.news-service.com!85.214.198.2.MISMATCH!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail From: Natasha Kerensikova Newsgroups: comp.lang.ada Subject: Re: Reading the while standard input into a String Date: Mon, 6 Jun 2011 10:08:01 +0000 (UTC) Organization: A noiseless patient Spider Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Injection-Date: Mon, 6 Jun 2011 10:08:01 +0000 (UTC) Injection-Info: mx04.eternal-september.org; posting-host="Mda950WjNwNLAFOE7yJXQw"; logging-data="20369"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19HtCPLCJmuXM+e5CdS/Cia" User-Agent: slrn/0.9.9p1 (FreeBSD) Cancel-Lock: sha1:rzHWvD9/k43e9Ma/0c5cDs2i+W8= Xref: g2news1.google.com comp.lang.ada:19630 Date: 2011-06-06T10:08:01+00:00 List-Id: Hello, On 2011-06-06, Ludovic Brenta wrote: > Natasha Kerensikova wrote on comp.lang.ada: >> I have encountered what was a very easily solved problem in C: dump the >> whole contents of the standard input into some place in memory, in order >> to process it afterwards. > > Personally I think this is a fundamentally bad idea. Consider: > > $ my_program < /dev/random > > Do you consider it acceptable to use all memory and all swap space > before finally bailing out with a Storage_Error and not doing any > actual processing? Actually, in principle, yes. I mean, I want to be able to write code that behave that way as far as the "reading the whole standard input" goes, because that's this part I'm trying to learn about. Of course, I don't want a real program to behave that way, so I would probably add a "reasonable" limit (like 16 MB) that interrupt the loop, but that's beyond the scope of my question. And anyway, I don't trust any program to not use all memory and all swap space, so I have posix ulimits set accordingly. > I'd rather process the input one fixed-length chunk at a time and > discard each chunk after processing (possibly by recycling the in- > memory buffer for the next chunk). I would do so if online processing was an option. However in the program containing code similar to what I posted, I cannot do any processing before reaching the end of input, because the processed format allows forward references, so references have to be gathered in a first pass on the whole input, before doing a second pass where dereference can take place. > Note that reading one character at a time from a stream is quite fast > since streams normally use the operating system's buffered IO. Adding > a second layer of buffering inside your program is probably counter- > productive. That second layer of buffer was actually meant to reduce the number of Ada.Strings.Unbounded.Append calls (and the expensive reallocations that go with them). However I thought that something similar to C's fread(), which can use efficient bulk memory copy, would be better than a loop of single character read (even when buffered in memory). Thanks for your comments, Natasha