* Truncating Direct IO files @ 2015-04-21 7:30 tonyg 2015-04-21 7:47 ` Dmitry A. Kazakov 2015-04-21 22:01 ` Randy Brukardt 0 siblings, 2 replies; 5+ messages in thread From: tonyg @ 2015-04-21 7:30 UTC (permalink / raw) I am recording records to disk. Sometimes I replace thes records with a smaller set of records. I could delete the file each time but would prefer to truncate the file. I can't see a way of doing this, can you? ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Truncating Direct IO files 2015-04-21 7:30 Truncating Direct IO files tonyg @ 2015-04-21 7:47 ` Dmitry A. Kazakov 2015-04-21 8:20 ` tonyg 2015-04-21 22:01 ` Randy Brukardt 1 sibling, 1 reply; 5+ messages in thread From: Dmitry A. Kazakov @ 2015-04-21 7:47 UTC (permalink / raw) On Tue, 21 Apr 2015 00:30:37 -0700 (PDT), tonyg wrote: > I am recording records to disk. Sometimes I replace thes records with a > smaller set of records. Do you rewrite all file then? In that case you could simply work with a copy and delete the original, in a sort of double-buffering manner. > I could delete the file each time but would prefer to truncate the file. I > can't see a way of doing this, can you? There seems no way doing that without OS. In my implementation of a persistent storage pool I simply maintain the list of free elements and reuse them later. BTW, I don't overwrite any file elements within single transaction. I don't know what you are doing, but consider a scenario when the application or the whole system crashes in the middle of file updating. Would that corrupt the data? -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Truncating Direct IO files 2015-04-21 7:47 ` Dmitry A. Kazakov @ 2015-04-21 8:20 ` tonyg 2015-04-21 9:17 ` Dmitry A. Kazakov 0 siblings, 1 reply; 5+ messages in thread From: tonyg @ 2015-04-21 8:20 UTC (permalink / raw) I use your tables package in nearly everything I do! I've been reading up on oop and tagged types and I'm going to bite the bullet and make another attempt at using your persistant storage. Should I stick to normal records rather than use variants for the persistant storage? And yes crashes would corrupt my system :) ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Truncating Direct IO files 2015-04-21 8:20 ` tonyg @ 2015-04-21 9:17 ` Dmitry A. Kazakov 0 siblings, 0 replies; 5+ messages in thread From: Dmitry A. Kazakov @ 2015-04-21 9:17 UTC (permalink / raw) On Tue, 21 Apr 2015 01:20:55 -0700 (PDT), tonyg wrote: > I use your tables package in nearly everything I do! I ma glad to hear that. > I've been reading up > on oop and tagged types and I'm going to bite the bullet and make another > attempt at using your persistant storage. These are different things. 1. There is an implementation of persistent objects that supports serialization and deserialization. The object may refer to other objects. The objects are reference counted and when the last reference disappears the object, if updated, gets serialized. This persistence layer can be backed by anything. E.g. a database or by a direct file I/O with some infrastructure sitting on top of it. 2. There is an implementation of a persistent memory pool where you can allocate and free memory chunks. It supports stream access if you need to store something larger than single direct I/O element (block). #1 can sit on top of #2. 3. There is virtual direct I/O layer that supports transactions and makes it safe against crashes. #2 can sit on top of #3. 4. There is an implementation of various B-trees (maps) including such that are similar to a relational table with many keys and many values. #4 can sit on top of #3. I cannot tell what of this could be useful for you, because I don't know what are you trying to do. > Should I stick to normal records rather than use variants for the persistant storage? See above. You can store them directly into the persistent memory pool allocating storage there. You will need some kind of serialization and desrialization in the form of stream I/O. The persistent memory pool has stream interface. If you think that serialization is wasting resources, consider portability. The same storage file should work on a big-endian and little-endian machine, right? That's is why you should never use Direct I/O with binary data or stream attributes. The library provides subroutines to serialize numeric types. Use them to keep stream I/O portable. You can use the persistent layer if things are complicated. E.g. objects dependent on each other. Scopes blurred etc. It is possible to have persistent objects in situ, e.g. a large container of things, graph, tree, image. Typically very large objects may be too expensive to deserialize. They may sit in the persistent storage permanently and be accessed via a proxy object which is serialized and deserialized instead. Yes, it is complicated, but nobody said it would not be. -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Truncating Direct IO files 2015-04-21 7:30 Truncating Direct IO files tonyg 2015-04-21 7:47 ` Dmitry A. Kazakov @ 2015-04-21 22:01 ` Randy Brukardt 1 sibling, 0 replies; 5+ messages in thread From: Randy Brukardt @ 2015-04-21 22:01 UTC (permalink / raw) "tonyg" <tonythegair@gmail.com> wrote in message news:8323246f-eb13-49e8-9e0c-5c5ec1002060@googlegroups.com... > > > I am recording records to disk. Sometimes I replace thes records with a > smaller set of records. > I could delete the file each time but would prefer to truncate the > file. I can't see a way of doing this, can you? Ada doesn't provide that, mainly because not all popular operating systems provide a truncation mechanism for files, and we don't want implementations to have to fall back on the very expensive copy the whole file and replace mechanism (obviously, if that is acceptable to you, you can write that yourself). It appears that Ada 95 stream files provide truncation on Close and Reset (and thus some implementations do implement that, but all surveyed implementations did it wrong), but many implementations don't truncate and Ada 2005 and later removed the wording that appeared to require it. So one cannot depend on that. Randy. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-04-21 22:01 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-04-21 7:30 Truncating Direct IO files tonyg 2015-04-21 7:47 ` Dmitry A. Kazakov 2015-04-21 8:20 ` tonyg 2015-04-21 9:17 ` Dmitry A. Kazakov 2015-04-21 22:01 ` Randy Brukardt
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox