From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=BAYES_00, REPLYTO_WITHOUT_TO_CC autolearn=no autolearn_force=no version=3.4.4 Path: eternal-september.org!reader01.eternal-september.org!reader02.eternal-september.org!news.eternal-september.org!mx02.eternal-september.org!feeder.eternal-september.org!aioe.org!.POSTED!not-for-mail From: "Dmitry A. Kazakov" Newsgroups: comp.lang.ada Subject: Re: Truncating Direct IO files Date: Tue, 21 Apr 2015 11:17:20 +0200 Organization: cbb software GmbH Message-ID: References: <8323246f-eb13-49e8-9e0c-5c5ec1002060@googlegroups.com> <1tufk7ruhyejt.1tyb01i9kotm1.dlg@40tude.net> <95d6f315-d21b-4106-961c-dcf11f8c04ad@googlegroups.com> Reply-To: mailbox@dmitry-kazakov.de NNTP-Posting-Host: enOx0b+nfqkc2k+TNpOejg.user.speranza.aioe.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Complaints-To: abuse@aioe.org User-Agent: 40tude_Dialog/2.0.15.1 X-Notice: Filtered by postfilter v. 0.8.2 Xref: news.eternal-september.org comp.lang.ada:25568 Date: 2015-04-21T11:17:20+02:00 List-Id: On Tue, 21 Apr 2015 01:20:55 -0700 (PDT), tonyg wrote: > I use your tables package in nearly everything I do! I ma glad to hear that. > I've been reading up > on oop and tagged types and I'm going to bite the bullet and make another > attempt at using your persistant storage. These are different things. 1. There is an implementation of persistent objects that supports serialization and deserialization. The object may refer to other objects. The objects are reference counted and when the last reference disappears the object, if updated, gets serialized. This persistence layer can be backed by anything. E.g. a database or by a direct file I/O with some infrastructure sitting on top of it. 2. There is an implementation of a persistent memory pool where you can allocate and free memory chunks. It supports stream access if you need to store something larger than single direct I/O element (block). #1 can sit on top of #2. 3. There is virtual direct I/O layer that supports transactions and makes it safe against crashes. #2 can sit on top of #3. 4. There is an implementation of various B-trees (maps) including such that are similar to a relational table with many keys and many values. #4 can sit on top of #3. I cannot tell what of this could be useful for you, because I don't know what are you trying to do. > Should I stick to normal records rather than use variants for the persistant storage? See above. You can store them directly into the persistent memory pool allocating storage there. You will need some kind of serialization and desrialization in the form of stream I/O. The persistent memory pool has stream interface. If you think that serialization is wasting resources, consider portability. The same storage file should work on a big-endian and little-endian machine, right? That's is why you should never use Direct I/O with binary data or stream attributes. The library provides subroutines to serialize numeric types. Use them to keep stream I/O portable. You can use the persistent layer if things are complicated. E.g. objects dependent on each other. Scopes blurred etc. It is possible to have persistent objects in situ, e.g. a large container of things, graph, tree, image. Typically very large objects may be too expensive to deserialize. They may sit in the persistent storage permanently and be accessed via a proxy object which is serialized and deserialized instead. Yes, it is complicated, but nobody said it would not be. -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de