From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,d02eb5c33ac65d9 X-Google-Attributes: gid103376,public X-Google-ArrivalTime: 2003-03-11 15:37:44 PST Path: archiver1.google.com!news1.google.com!newsfeed.stanford.edu!news-spur1.maxwell.syr.edu!news.maxwell.syr.edu!news-out.visi.com!petbe.visi.com!uunet!ash.uu.net!sac.uu.net!nntp.TheWorld.com!not-for-mail From: Robert A Duff Newsgroups: comp.lang.ada Subject: Re: Array and memory usage Date: 11 Mar 2003 18:37:42 -0500 Organization: The World Public Access UNIX, Brookline, MA Message-ID: References: <1ec946d1.0303100713.7f7bcbb7@posting.google.com> NNTP-Posting-Host: shell01.theworld.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: pcls4.std.com 1047425862 16849 199.172.62.241 (11 Mar 2003 23:37:42 GMT) X-Complaints-To: abuse@TheWorld.com NNTP-Posting-Date: Tue, 11 Mar 2003 23:37:42 +0000 (UTC) User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.2 Xref: archiver1.google.com comp.lang.ada:35230 Date: 2003-03-11T18:37:42-05:00 List-Id: "Randy Brukardt" writes: > Sort of. Windows has three stages of memory allocation, address space > allocation, virtual page allocation, and physical page allocation. The > first two are (partially) under control of the programmer. > > Swap space is allocated when virtual pages are allocated (called > "commit"). If the pages aren't commited, an attempt to access them > causes a page fault. (That's the idea Matt was talking about). > > Memory that is allocated in a load image (even as address space) is > always commited, so that it can be accessed without a fault -- but that > requires swap space to be allocated (even if no physical memory will > ever be used). Thank you (and Matthew) *very* much for explaining this to me. Very helpful. I still don't see the *point* of this design, though. (Not the first time I've disagreed with a design "aspect" of Windows...) The hardware is going to fault if there is no *physical* page allocated for a given virtual address. That requires switching to supervisor mode (slow). Whether or not the user-mode process sees this fault is largely irrelevant (to efficiency). I see no reason why swap space should be allocated (on the disk) until it is needed -- when the OS wants to swap out a (dirty) page. Or if you don't buy that, then why not allocate swap space when *physical* memory is first allocated for a given virtual address. Clearly, if a given virtual page frame has never been touched, so no physical memory has ever been allocated for it, it can't be swapped out, so no swap space is needed. In other words, I see no reason for this weird "committed" state, where swap space is allocated on the disk but no physical page is allocated. Running out of memory is pretty unpredictable anyway (on virtual systems like we're talking about), so who cares exactly when it occurs? Also, I see no reason why user-mode code needs to be involved at all, for this trick. There are reasons why user-mode code might want to set the protection bits of parts of its address space, and handle traps/signals/whatever caused by accessing those parts, and that requires separate system calls. I've done that (on both Unix and Windows). But surely *this* trick (relying on "allocate on first use") ought to be completely invisible to user-mode code. Am I confused, or was the designer of Windwos confused? > The proper way to deal with this is to avoid allocating the address > space at compile-time; rather use dynamic allocation to do it. That is > easy to do in Ada by writing a storage pool that allocates the address > space immediately, then commits additional memory only as needed. > > In my application, I did it by using accesses to unconstrained arrays, > and having an "Expand" routine. When one became full, I called the > Expand routine, which determined if uncommitted address space followed > the acces, and if it did, commited the needed memory and adjusted the > array descriptor for the access type. Thus, all of the expansion was > done in place. Of course, adjusting the array descriptor is not a safe > or portable operation, and it might not even be possible on some > compilers. If the memory couldn't be expanded, then I would allocate the > needed larger array from the pool, copy it, and free the old array -- > that is, the standard way of expanding dynamic arrays. > > One warning: address space has to be allocated in 64K chunks. If you try > to use smaller chunks than that, it get rounded up to 64K. (I found that > out the hard way). Why 64K? When not the page size supported by the hardware (which is 2**12 bytes on 386-family). > Randy. Sorry if this post slightly off topic. At least I'm not ranting about Adam Smith. ;-) - Bob