config files proposal

config files proposal

[Stephen Leake]

I've posted a summary of the requirements and options for a config
file standard, at
http://users.erols.com/leakstan/Stephe/Ada/config_files.html 

I'm going to experiment with an XML implementation before I post a
draft spec; I suspect using XML for the file format will have an
impact on the spec. If I find XML too complex, I'm leaning towards the
Java property style.

Please comment on the requirements. 

Please don't say "just use the registry"; let's take that discussion
as settled :). The consensus is we need both the registry and a
simpler config file.

I'm on vacation starting Tuesday (back the following Monday), so I
won't be responding to comments till then. I'll try to catch up on all
of this thread when I get back, but feel free to send me email
directly at stephen_leake@acm.org.

-- 
-- Stephe

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> http://users.erols.com/leakstan/Stephe/Ada/config_files.html

I would suggest using Wide_String for keys. (API#8)

You need to specify where a file is written in the path. (API#6) apparently
only talks about reading files.

FF #1 has to deal with the differences between systems anyway. Character
sets, newlines, etc. Or at least mention what you mean.  That is, file
format #1 kind of conflicts with file format #2, if Text_IO writing on one
system isn't readable on another.

FF #3 I would disagree with. It limits having large items in the config file
format. Making key/value pairs terminated by a newline is specifying the
implementation. It doesn't make any sense to do such a thing in XML, for
example. Perhaps starting continuation lines with a space as is done in RFC
822 headers would make more sense. In addition, API#15 conflicts with this
requirement.

FF#4 should be written as "leading and trailing whitespace on keys and
values is semantically meaningless". 

Question: Are keys case-sensitive?

Question: What about where most of the configuration is stored in a
system-wide file, with individual users capable of overriding particular
entries?  Kind of like /etc/bashrc and ~/.bashrc on Linux?

Question: Does the file automatically flush changes as it goes out of scope?
(API#4)

Question: Is the file closed (w.r.t. the OS) between the end of the read and
the start of the write?

Question: Would it make sense to allow compound values to be written by
converting a stream of them to base64 or something?

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Darren New]

Darren New wrote:
> FF#4 should be written as "leading and trailing whitespace on keys and
> values is semantically meaningless".

Oh, and then you have to specify the quoting rules for values where leading
and/or trailing whitespace *is* significant.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > http://users.erols.com/leakstan/Stephe/Ada/config_files.html
> 
> I would suggest using Wide_String for keys. (API#8)

Hmm. Good idea. I'm not clear why I limited that. But I'm glad I made
it specific, so you could point out it was wrong :).

> You need to specify where a file is written in the path. (API#6)
> apparently only talks about reading files.

It should be written where it is found; I'll add that. Hmm, special
case if it is a new file; write it to the first directory in the
search path.

> FF #1 has to deal with the differences between systems anyway.
> Character sets, newlines, etc. Or at least mention what you mean.
> That is, file format #1 kind of conflicts with file format #2, if
> Text_IO writing on one system isn't readable on another.

Good point. I guess I need to say "files can be copied with normal
system-dependent changes (such as line endings)."

> FF #3 I would disagree with. It limits having large items in the
> config file format. 

Only if there is a maximum line length. I guess some text editors
might impose one, but Ada.Text_IO doesn't.

> Making key/value pairs terminated by a newline is specifying the
> implementation. 

Yes, but that's what FF is about anyway.

> It doesn't make any sense to do such a thing in XML, for example.

True.

> Perhaps starting continuation lines with a space as is done in RFC
> 822 headers would make more sense. 

Well, I guess there are many ways to deal with mult-line formats.

> In addition, API#15 conflicts with this requirement.

I don't see how. API#15 says "the client may provide a string image of
a composite type". How does that imply multiline values. Ah; the
client could embed a newline in the string. Guess I should forbid
that.

But I'm leaning towards eliminating API#15 anyway; given multi-level
sections, I realized there's no need for it. I had it in my Windows
ini package, because there is only one level of sections.

> FF#4 should be written as "leading and trailing whitespace on keys
> and values is semantically meaningless".

Yes, much better. Thanks.

> Question: Are keys case-sensitive?

Good point. I think I'll go with the Ada identifier model and say
"no". 


> Question: What about where most of the configuration is stored in a
> system-wide file, with individual users capable of overriding
> particular entries? Kind of like /etc/bashrc and ~/.bashrc on Linux?

I'm not sure what your question is. Oh, I see; you want me to request
key "foo", and if not found in ~/.bashrc then look in /etc/bashrc to
find "foo". Hmm.

I guess we could add an "append_read_only" operation, that would read
in a second file, and store it in a separate read-only tree. I like
that. 

> Question: Does the file automatically flush changes as it goes out
> of scope? (API#4)

Yes, I need to add that. Guess that makes it Controlled; I think it
would be anyway, to avoid memory leaks.

> Question: Is the file closed (w.r.t. the OS) between the end of the
> read and the start of the write?

Yes. Actually, I think the file is OS closed at the end of Open, after
the data is read in.

I'm planning on renaming the file to a backup name before writing it,
for glitch protection.

> Question: Would it make sense to allow compound values to be written
> by converting a stream of them to base64 or something?

I think composite values are best handled by multiple levels. Ie, if
you have a record:

type Foo is record
   a : integer;
   b : integer;
end record;

you write this as (assuming Java property style)

foo.a = 1
foo.b = 2

base64 would be good for values that don't need to be user editable. I
guess persistent storage for a large object (whether it is composite
or not) could be supported that way. I'm not clear why that would be
in a typical "config file". Hmm, maybe a pixel bitmap for an icon?

Thanks for your comments!

-- 
-- Stephe

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> > You need to specify where a file is written in the path. (API#6)
> > apparently only talks about reading files.
> 
> It should be written where it is found; I'll add that. Hmm, special
> case if it is a new file; write it to the first directory in the
> search path.

The problem here is if you load a system-wide config file, make changes, and
want to save it. You at least need a routine that says "pretend you got it
from *here*" or something; i.e., to change the name&path of the file between
reading and writing.
 
> > FF #3 I would disagree with. It limits having large items in the
> > config file format.
> 
> Only if there is a maximum line length. I guess some text editors
> might impose one, but Ada.Text_IO doesn't.

Hrm. Perhaps.
 
> > In addition, API#15 conflicts with this requirement.
> 
> I don't see how. API#15 says "the client may provide a string image of
> a composite type". How does that imply multiline values. Ah; the
> client could embed a newline in the string. Guess I should forbid
> that.

Right. Actually, I read it as "provides a way to read/write the value". I
hadn't read it as "... as a String value" but I guess someone more atuned to
Ada would read it that way.

> But I'm leaning towards eliminating API#15 anyway; given multi-level
> sections, I realized there's no need for it. I had it in my Windows
> ini package, because there is only one level of sections.

I think it depends on just how you build the API.

> > Question: Are keys case-sensitive?
> 
> Good point. I think I'll go with the Ada identifier model and say
> "no".

If "no" and you're allowing Unicode, then you have to worry about the
bizarre transformations that some languages have.
 
> I guess we could add an "append_read_only" operation, that would read
> in a second file, and store it in a separate read-only tree. I like
> that.

That's how I've always done it. That also solves the problem of "where do I
store changes if there was no file found when I read?"
 
> I'm planning on renaming the file to a backup name before writing it,
> for glitch protection.

Make that optional. Or write to a new file, and rename over the old one.
Some OSes may do this for you automatically. Other OSes supply specific
routines to do this properly (maintaining file attributes and ACLs, etc.)
 
> base64 would be good for values that don't need to be user editable. I
> guess persistent storage for a large object (whether it is composite
> or not) could be supported that way. I'm not clear why that would be
> in a typical "config file". Hmm, maybe a pixel bitmap for an icon?

That was actually my thought, yes. Funky how we both had the same example in
mind.
 
> Thanks for your comments!

Thanks for paying attention! :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Ted Dennison]

Stephen Leake <stephen_leake@acm.org> wrote in message news:<wkznydwcm7.fsf@acm.org>...
> Darren New <dnew@san.rr.com> writes:
> > You need to specify where a file is written in the path. (API#6)
> > apparently only talks about reading files.
> 
> It should be written where it is found; I'll add that. Hmm, special
> case if it is a new file; write it to the first directory in the
> search path.

Perhaps I'm misunderstanding, but this sounds to me like you are
saying:

  1)  Users have very little control over where their config files get
written to.
  2)  There will be only one config file per system.
  3)  The facility will be dependent on the very system-dependant
feature of directory search paths.


I don't like the first two ideas at all, since they needlessly
restrict the funtionality. I could easily see loads of people who
would otherwise love to use the facility, forced to roll their own due
to problems with 1 and 2. Clients of the facility should be able to
read and write configurations to and from the directories and file
names of their choice.

The third issue I think requires a good deal of discussion before
being accepted. For one thing, it will prevent anyone from making a
portable implementation of the config file facility. It should also be
noted that if this facility's behavior depends on an
implementation-defined facility, then you are basicly saying that
(part of) this facility's behavior is implementation-defined. That in
turn means programs that use this facility are going to have
portability issues. In my mind, that's more serious than having
non-portable config files.

> > FF #3 I would disagree with. It limits having large items in the
> > config file format. 
> 
> Only if there is a maximum line length. I guess some text editors
> might impose one, but Ada.Text_IO doesn't.

Since it requires that the user supply a fixed-length buffer,
Ada.Text_IO effectively *does* impose a limit. It just lets you decide
what that limit is. :-)

> > Making key/value pairs terminated by a newline is specifying the
> > implementation. 

I'd disagree with this statement. It is (partially) specifying a file
format, but in no way specifies how the config file facility
implements it. The file format *has* to be part of the specification,
it can't be left as an "implementation detail".

Presumably this is just a first cut to get the "easy ones" ironed out,
before we start to agree on a file format. Right?

If so, a lot of this discussion will be rendered moot when the file
format is decided upon.

> > Question: Are keys case-sensitive?
> 
> Good point. I think I'll go with the Ada identifier model and say
> "no". 

I agree wholeheartedly, for the same reason.

> I'm not sure what your question is. Oh, I see; you want me to request
> key "foo", and if not found in ~/.bashrc then look in /etc/bashrc to
> find "foo". Hmm.
> 
> I guess we could add an "append_read_only" operation, that would read
> in a second file, and store it in a separate read-only tree. I like
> that. 

Perhaps it would be better to allow for some kind of "chaining" of
configuration objects? That way items not found in the first
configuration might be found in later ones.

In most of my projects that use configuration files, there are a set
of hard-coded defaults for all configuration values. That way if no
config files exist, we still have a reasonable set of defaults
defined. A client creating a "default" configuration object that is
loaded with values from the program at initialization time and chained
on the back would be one way of implementing this.

> > Question: Is the file closed (w.r.t. the OS) between the end of the
> > read and the start of the write?
> 
> Yes. Actually, I think the file is OS closed at the end of Open, after
> the data is read in.

*That* sounds like an implementation detail to me. Its a good idea
perhaps, but still an implementation detail.

> I'm planning on renaming the file to a backup name before writing it,
> for glitch protection.

Ditto

> base64 would be good for values that don't need to be user editable. I
> guess persistent storage for a large object (whether it is composite
> or not) could be supported that way. I'm not clear why that would be
> in a typical "config file". Hmm, maybe a pixel bitmap for an icon?

Motif's UIL did these as text. You define the colormap for each ASCII
character, then use the defined ASCII characters to draw the bitmap.
In the text buffer it looked a bit like ASCII art. The only big
drawback I saw was that the "art" value is destroyed if your bitmap
gets wider than your text editor.


-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  http://www.telepath.com/dennison/Ted/TED.html

Re: config files proposal

[Preben Randhol]

On Sun, 02 Jun 2002 22:16:00 GMT, Stephen Leake wrote:
> Darren New <dnew@san.rr.com> writes:
> 
>> Stephen Leake wrote:
>> Question: Would it make sense to allow compound values to be written
>> by converting a stream of them to base64 or something?
> 
> I think composite values are best handled by multiple levels. Ie, if
> you have a record:
> 
> type Foo is record
>    a : integer;
>    b : integer;
> end record;
> 
> you write this as (assuming Java property style)
> 
> foo.a = 1
> foo.b = 2
> 
> base64 would be good for values that don't need to be user editable. I
> guess persistent storage for a large object (whether it is composite
> or not) could be supported that way. I'm not clear why that would be
> in a typical "config file". Hmm, maybe a pixel bitmap for an icon?

xpm is ASCII based.

If you start including base64 in the config files there will be a lot
of data corruption. If one need to use sth like that then the config
file should be binary so a user cannot edit it. The user of the package
can easily make more than one config file.

Example:

   User editable settings goes in settings.config
   Program settings (like coulours, variables) goes in program.data

Then program.data can be binary or whatever, while settings.config is
ASCII.

I don't think the package should know which OS it is running on. I mean
the developer should give it the propper Path to look in I think.

Preben

Re: config files proposal

[Darren New]

Ted Dennison wrote:
> > > Question: Are keys case-sensitive?
> >
> > Good point. I think I'll go with the Ada identifier model and say
> > "no".
> 
> I agree wholeheartedly, for the same reason.

I think they should be case sensitive unless there's a standard way of
converting uppercase unicode to lowercase unicode in Ada's libraries.
Remember that (for example) you'll probably want to store this stuff in a
Map, implying a hashtable perhaps, meaning you'll need to convert between
whatever is supplied and either upper or lower case, and do so in the same
way a native speaker of the language would do so.

Remember too that (for example) the lower-case version of some German words
don't even have the same length as the upper-case version. And I'm not even
sure there's general agreement on whether uppercase letters in French have
the accent included or not. And I don't even *know* if there is the
equivalent with different Chinese, Japanese, or Cyrilic characters, for
example.

While it's admirable to ignore the case of the letters, I think trying to
implement a general "to_lower" that'll handle Unicode properly is going to
be more of a nightmare than writing the entire rest of the config file
parser. Unless you restrict the keys to US-ASCII, that is.

Just something to consider. 
 
> Perhaps it would be better to allow for some kind of "chaining" of
> configuration objects? That way items not found in the first
> configuration might be found in later ones.

Yes, I think that's a good way to go also. At least, that's how I do it. :-)

I think there should also be some sort of tag at the front of each file
saying it's an Ada config file, even if only

 -- Ada Grace.Config file V1.0 -- Do not hand-edit.

so one could check you've found a proper file and know what version it is.

> > Yes. Actually, I think the file is OS closed at the end of Open, after
> > the data is read in.
> 
> *That* sounds like an implementation detail to me. Its a good idea
> perhaps, but still an implementation detail.

I think it's an important detail for systems where having a file open
changes the semantics for other users. If, for example, you leave it open
after you write it, it's possible I won't be able to read it. If you leave
it open after reading it, I may be dedicating resources on the file server
to keeping track of it (assuming it's centrally served, for example).
Closing it each time would make it easier to reason about multiple users
using the same config file, and the overhead of reopening it should be
minimal, given it's a config file.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Jean-Marc Bourguet]

Darren New <dnew@san.rr.com> writes:

> While it's admirable to ignore the case of the letters, I think trying to
> implement a general "to_lower" that'll handle Unicode properly is going to
> be more of a nightmare than writing the entire rest of the config file
> parser. Unless you restrict the keys to US-ASCII, that is.

Well unicode define canonical forms (4 of them: x2 for the case
issues, x2 for determining if things like "superscript 2" and 2 are
equivalent).  And you are right, the case independant canonical forms
are dependent on the locale.

But even for case dependant comparisons, using canonical form is
needed as some glyphs have multiple representations.

Yours,

-- 
Jean-Marc

Re: config files proposal

[Darren New]

Jean-Marc Bourguet wrote:
> But even for case dependant comparisons, using canonical form is
> needed as some glyphs have multiple representations.

OK. Does Ada define Unicode libraries that let you work with this? Does
GNAT? :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Preben Randhol]

On Mon, 03 Jun 2002 16:50:41 GMT, Darren New wrote:
> Jean-Marc Bourguet wrote:
>> But even for case dependant comparisons, using canonical form is
>> needed as some glyphs have multiple representations.
> 
> OK. Does Ada define Unicode libraries that let you work with this? Does
> GNAT? :-)

http://sourceforge.net/projects/ngeadal/

Preben

Re: config files proposal

[Antonio Duran]

Stephen Leake <stephen_leake@acm.org> wrote in message news:<wkelfpy887.fsf@acm.org>...
> I've posted a summary of the requirements and options for a config
> file standard, at
> http://users.erols.com/leakstan/Stephe/Ada/config_files.html 
> 
> I'm going to experiment with an XML implementation before I post a
> draft spec; I suspect using XML for the file format will have an
> impact on the spec. If I find XML too complex, I'm leaning towards the
> Java property style.
> 
> Please comment on the requirements. 
> 
> Please don't say "just use the registry"; let's take that discussion
> as settled :). The consensus is we need both the registry and a
> simpler config file.
> 
> I'm on vacation starting Tuesday (back the following Monday), so I
> won't be responding to comments till then. I'll try to catch up on all
> of this thread when I get back, but feel free to send me email
> directly at stephen_leake@acm.org.

In relation to config files there are some points that I would like to
comment. I don't read the entire thread so please, forgive me if some
or all of them were previously discussed.

1. In my experience, most of the time, config files are writen, edited
or modified by humans and read and interpreted by programs. With this
in mind, in my modest opinion, the importance of having an API for
writing to config files is secondary. What is important is having the
means that allow a human user to understand what is in the config
file. Configuration tasks are simpler if the config file is commented
and handling comments is difficult if you use an API call to write to
the file. When config file gets complicated enough one must write an
application to handle it (eg. Comanche to mantain Apache config
files).

2. Seems that there is a consensus in that a config file must store
key/value(s) pairs. I don't understand the requirement to use
Wide_String for keys. I'm from Spain and I usually write in Spanish
but when programming in Ada I must adhere to the Ada syntax, I can't
write Ada identifiers that are syntactically correct spanish words and
the Ada reserved words are english words. Supporting Wide_String could
be a cool feature for, for example, mantaining localized messages in
different languages, however: is this the purpose of config files?. I
think the purpose of config files is to provide parameters to a
program or application and not to be used as a repository for
localized versions of program messages. Of course, they could be used
in that way. I suggest that keys in config file meet the Ada syntax
for identifier and be case-insensitive.

3. I think that is important to handle more than one config file in a
single program. From my point of view, one of the most important
drawbacks of layouts such as the Windows Registry is that everything
is on a single place so applications that can access to a single part
can also write other parts. In complicated apps it could be desirable
to mantain a set of different config files one for each part of the
application. The ability for handling multiple config files in a
single program should be a requirement for a config file handling
package.

4. I don't understand very well the implications of "... specified by
generic parameter at compile time ..." (API req. 5). Does this mean
that the name or the path of the file must be hard coded?. If so I
think we've missed totally the point (should I recompile each time I
change the name or location of the config file?).

5. Regarding to the file syntax, long time ago I worked on a system
that runs on CAIS (Common APSE Interface Set). CAIS contains a List
Management package that provides great power for handling parameter
lists and those lists have a human readable text representation. That
text representation is, from my point of view, more Ada like and could
be used as a base syntax for config files. The example provided in the
Stephen requirements page would be (comments were not allowed in CAIS
Lists)

-- Config file TBD.

(
   -- ServerLayout Section.

   ServerLayout => (
      Identifier => "Default Layout",
      Screen => ("Default Screen", 0, 0),
      InputDevice => (
         "Generic Keyboard",
         "Configured Mouse",
         "Generic Mouse"
      ),
      Display => (
         Depth => 1,
         Modes => (
            (Width => 16#400#, Height => 768),
            (Width => 800, Height => 600) -- New mode added.
         )
      )
   ) -- End of ServerLayout section.
) -- End of config file TBD

There are, however, some drawbacks in this representation:

- A escaping schema is needed for unprintable characters in strings
(?).
- Nested '(' and ')' would add a high degree of complexity for the
human user/reader.

Antonio Duran

Re: config files proposal

[Darren New]

Antonio Duran wrote:
> 1. In my experience, most of the time, config files are writen, edited
> or modified by humans and read and interpreted by programs.

I think that's true on UNIX (esp. server programs) much more so than on
Windows. I can't remember the last time I edited configuration manually on
Windows.

> 2. ...

A good argument.

> 3. I think that is important to handle more than one config file in a
> single program.

Unquestionably.

> From my point of view, one of the most important
> drawbacks of layouts such as the Windows Registry is that everything
> is on a single place so applications that can access to a single part
> can also write other parts.

Errrr... if you ignore the ACLs on registry entries, sure. That's like
saying the problem with UNIX is it's all files, so if you can write to one
you can write to them all.

> In complicated apps it could be desirable
> to mantain a set of different config files one for each part of the
> application. The ability for handling multiple config files in a
> single program should be a requirement for a config file handling
> package.

I think it's also important to have system-wide, program-wide, and user-wide
config files.
 
-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Darren New]

And for the folks wanting to use XML for this, here's what the IETF is 
thinking about using XML. Just an interesting draft.

http://www.ietf.org/internet-drafts/draft-hollenbeck-ietf-xml-guidelines-04.txt

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Ted Dennison]

Darren New <dnew@san.rr.com> wrote in message news:<3CFB94A7.A455B8DD@san.rr.com>...
> I think they should be case sensitive unless there's a standard way of
> converting uppercase unicode to lowercase unicode in Ada's libraries.

We need to match up well with what's in the Ada standard I think. 

The Ada standard has an Ada.Strings.Maps.Constants.Lower_Case_Map and
an Ada.Strings.Wide_Maps.Wide_Constants.Lower_Case_Map. There is no
reason why we can't define things to say that keys will be fed through
the appropriate Lower_Case_Map before being matched.

If the language itself has some kind of problem with
Ada.Strings.Wide_Maps.Wide_Constants.Lower_Case_Map, I say that is the
language's problem, and outside the scope of a configuration item
facility to try to worry about fixing. As long as we define what we
are doing carfully this way, it is still explicit. If we stick to
using the standard library, things should at least behave for people
the way they have come to expect them to behave when using Ada.

-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  http://www.telepath.com/dennison/Ted/TED.html

Re: config files proposal

[Antonio Duran]

Darren New <dnew@san.rr.com> wrote in message news:<3CFCE218.585D56E2@san.rr.com>...

> I think that's true on UNIX (esp. server programs) much more so than on
> Windows. I can't remember the last time I edited configuration manually on
> Windows.
> 

Maybe I'm not lucky enough since I recently had to configure Apache,
PHP, and MySQL on my machine and had to deal with text based config
files but these applications are not Windows native (that is, only for
Windows) ;-). Of course, using a text editor rather than a specific
tool is more error prone and one must take care of specific syntax
issues (eg. editing the hosts file and that tricky tab at the
beginning of entries).

> Errrr... if you ignore the ACLs on registry entries, sure. That's like
> saying the problem with UNIX is it's all files, so if you can write to one
> you can write to them all.

Right, I've ignored ACLs on registry entries (maybe because my
experience is on Windows 98). While I recognize the advantanges of
having configuration data in a single place (common tools for handling
data) I'm also concerned with, as we say in Spain, "putting all the
eggs in the same basket". There is a risk of loosing all your
configuration data. This is not only a Windows problem, as I recall,
on AIX there is also a repository for config data (I think its name is
RODM or something like that). Moreover, the data storage format of
those repositories is propietary, tailored to a specific system, and,
in general, even if you backed up your config data is complicated to
restore them back after a system crash.

While is true that on Unix, if you loose the contents of your /etc
directory you're in big trouble is also true that it contains ordinary
files and restoring them (provided you backed up them) is easier.

> I think it's also important to have system-wide, program-wide, and user-wide
> config files.

Agree.

Thank you for your answer and best regards,

     Antonio Dur�n

Re: config files proposal

[Ted Dennison]

Darren New <dnew@san.rr.com> wrote in message news:<3CFCE218.585D56E2@san.rr.com>...
> Antonio Duran wrote:
> > 1. In my experience, most of the time, config files are writen, edited
> > or modified by humans and read and interpreted by programs.
> 
> I think that's true on UNIX (esp. server programs) much more so than on
> Windows. I can't remember the last time I edited configuration manually on
> Windows.

That's because of the way Windows is designed. With everything thrown
in one big database (the registry), the penalty for a user hosing the
configuration file is just too great to risk it. Every program *has*
to inlcude support for updating registry settings safely, lest they
risk their users hosing their entire system.

It doesn't sound like we are planning a "monolithic database"
approach, so this won't be an issue for us.

That said, I actually go into the registry manually myself lots, and I
know there are plenty of Microsoft support pages that specify
procedures involving manually updating the registry. The procedure for
setting up autologon in NT is an example.

> > 3. I think that is important to handle more than one config file in a
> > single program.
> 
> Unquestionably.

Then clearly we are *not* in the "one monolithic database" situation.

> I think it's also important to have system-wide, program-wide, and user-wide
> config files.

How "wide" they are should be up to the developer to set up and
support. However, the facility should allow for such setups. For
instance, if chaining is supported, then a Unix developer could put
the first file in ~, the second in the installation directory, and the
third in /usr/local somewhere.


-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  http://www.telepath.com/dennison/Ted/TED.html

Re: config files proposal

[Volkert Barr]

I have also set up a small document. This should be merged with the work 
of Mr. Stephen Leake. I tried to contact him, but he did not replyed. 
Not to mention - this paper is in a very early stage, so it has 
several errors ;-). 

Hints and suggestions should be mailed to me 
(barr@cs.tu-berlin.de) and not to c.l.a. I try to canalize 
them and send updates to c.l.a. for further discussion. 

See at: http://swt.cs.tu-berlin.de/~barr/config_package.pdf


Volkert

Re: config files proposal

[sk]

Hi, 

Volkert Barr <barr@cs.tu-berlin.de>
> ... of Mr. Stephen Leake. I tried to contact him, but he 
> did not replyed. ...

First message of this thread :

Subject: config files proposal
   Date: Sun, 02 Jun 2002 16:07:53 GMT
   From: Stephen Leake <stephen_leake@acm.org>

> I'm on vacation starting Tuesday (back the following Monday), 
> so I won't be responding to comments till then. 

-- 
-------------------------------------
-- Merge vertically for real address
-------------------------------------
s n p @ t . o
 k i e k c c m
-------------------------------------

Re: config files proposal

[Volkert Barr]

> First message of this thread :
> 
> Subject: config files proposal
>    Date: Sun, 02 Jun 2002 16:07:53 GMT
>    From: Stephen Leake <stephen_leake@acm.org>
> 
> > I'm on vacation starting Tuesday (back the following Monday),
> > so I won't be responding to comments till then.
sorry!

Re: config files proposal

[sk]

Hi,

Nothing to be sorry about. Just thought I would
try to save you the anxiety of trying to contact
someone who is not there.

-- 
-------------------------------------
-- Merge vertically for real address
-------------------------------------
s n p @ t . o
 k i e k c c m
-------------------------------------

Re: config files proposal

[Volkert]

Volkert Barr <barr@cs.tu-berlin.de> wrote in message news:<3CFF15E7.9A697B40@cs.tu-berlin.de>...
> I have also set up a small document. 

An updated version can be downloaded!
 
> Hints and suggestions should be mailed to me 
> (barr@cs.tu-berlin.de) and not to c.l.a. I try to canalize 
> them and send updates to c.l.a. for further discussion. 
> 
> See at: http://swt.cs.tu-berlin.de/~barr/config_package.pdf
> 
> 
> Volkert

Re: config files proposal

[Jason King]

A few thoughts on windows .ini style.
1)  The separator for value and key is '=' not ':='.  If we use an .ini 
file it should not break .ini file conventions.
2)  Standard .ini files use ';' as a comment to eol indicator, e.g.
; this is a comment
[section]
real1=val1
;real2=oldval2
real2=val2
Read/Write functions should ignore the lines that start with a ';'.

Volkert wrote:
> Volkert Barr <barr@cs.tu-berlin.de> wrote in message news:<3CFF15E7.9A697B40@cs.tu-berlin.de>...
> 
>>I have also set up a small document. 
> 
> 
> An updated version can be downloaded!
>  
> 
>>Hints and suggestions should be mailed to me 
>>(barr@cs.tu-berlin.de) and not to c.l.a. I try to canalize 
>>them and send updates to c.l.a. for further discussion. 
>>
>>See at: http://swt.cs.tu-berlin.de/~barr/config_package.pdf
>>
>>
>>Volkert
> 

Re: config files proposal

[Preben Randhol]

On Fri, 07 Jun 2002 04:01:49 -0500, Jason King wrote:
> A few thoughts on windows .ini style.
> 1)  The separator for value and key is '=' not ':='.  If we use an .ini 
> file it should not break .ini file conventions.
> 2)  Standard .ini files use ';' as a comment to eol indicator, e.g.
> ; this is a comment

Yes but on unix it is very common that # is the mark for comments.

Re: config files proposal

[Volkert]

Jason King <jhking@airmail.net> wrote in message news:<3D00767D.9020100@airmail.net>...
> A few thoughts on windows .ini style.
What if have done is a "Windows Ini like" definition. 

> 1)  The separator for value and key is '=' not ':='.  If we use an .ini 
> file it should not break .ini file conventions.
ok!

> 2)  Standard .ini files use ';' as a comment to eol indicator, e.g.
> ; this is a comment
i am with Prebens opinion here. 
 

Volkert

Re: config files proposal

[Ted Dennison]

Jason King <jhking@airmail.net> wrote in message news:<3D00767D.9020100@airmail.net>...
> A few thoughts on windows .ini style.
> 1)  The separator for value and key is '=' not ':='.  If we use an .ini 
> file it should not break .ini file conventions.

If I can take this as a comment that the above is closer to "true"
.ini style, if that's what he was trying to accomplish, then this is a
good comment.

However, I think it would be a big mistake to make rigid compatability
with Windows' .ini files a goal. For one thing, many if not most Ada
programs do not run on Windows. For another, that file format is in
thrall to Microsoft, and is probably only defined in Microsoft
copyrighted documents, if at all. Anything we come up with will have
to be something we can define with precision in the new LRM, without
having to worry about violating anyone's copyright, or having the
format change out from under us at the whim of one (non-Ada) vendor.

-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  http://www.telepath.com/dennison/Ted/TED.html

Re: config files proposal

[Darren New]

Preben Randhol wrote:
> 
> On Fri, 07 Jun 2002 04:01:49 -0500, Jason King wrote:
> > A few thoughts on windows .ini style.
> > 1)  The separator for value and key is '=' not ':='.  If we use an .ini
> > file it should not break .ini file conventions.
> > 2)  Standard .ini files use ';' as a comment to eol indicator, e.g.
> > ; this is a comment
> 
> Yes but on unix it is very common that # is the mark for comments.

If you're going to only allow alphanumberic keys, you might as well
accept ; # and -- as comments.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Preben Randhol]

On Fri, 07 Jun 2002 17:06:02 GMT, Darren New wrote:
> If you're going to only allow alphanumberic keys, you might as well
> accept ; # and -- as comments.

Yes thought I would like to have only: -- 

Re: config files proposal

[Darren New]

Preben Randhol wrote:
> 
> On Fri, 07 Jun 2002 17:06:02 GMT, Darren New wrote:
> > If you're going to only allow alphanumberic keys, you might as well
> > accept ; # and -- as comments.
> 
> Yes thought I would like to have only: --

Oh, I should also comment that if you're *not* going to follow the windows
.ini file format, chances are the default extension shouldn't be ".ini". :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Preben Randhol]

On Fri, 07 Jun 2002 17:33:53 GMT, Darren New wrote:
> Preben Randhol wrote:
>> 
>> On Fri, 07 Jun 2002 17:06:02 GMT, Darren New wrote:
>> > If you're going to only allow alphanumberic keys, you might as well
>> > accept ; # and -- as comments.
>> 
>> Yes thought I would like to have only: --
> 
> Oh, I should also comment that if you're *not* going to follow the windows
> .ini file format, chances are the default extension shouldn't be ".ini". :-)

The package shouldn't care about the filename, just the content.

I like the :

Program.Name := Glosa
Program.Version := 0.8

etc...

more than the 
[Program]
Name=Glosa
Version=0.8

style too :-)

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > > You need to specify where a file is written in the path. (API#6)
> > > apparently only talks about reading files.
> > 
> > It should be written where it is found; I'll add that. Hmm, special
> > case if it is a new file; write it to the first directory in the
> > search path.
> 
> The problem here is if you load a system-wide config file, make changes, and
> want to save it. You at least need a routine that says "pretend you got it
> from *here*" or something; i.e., to change the name&path of the file between
> reading and writing.

Good point. I guess I can check the write privs on the found file. 

Or provide two search paths; one read-only and one writeable.

> > > Question: Are keys case-sensitive?
> > 
> > Good point. I think I'll go with the Ada identifier model and say
> > "no".
> 
> If "no" and you're allowing Unicode, then you have to worry about the
> bizarre transformations that some languages have.

yes. And Glib.XML is case sensitive. Although Glib.XML also uses
String for Tags, not Wide_String. So I have to check the XML
definition; does it allow Wide_String for tags? Hmm.
http://www.w3.org/TR/2000/REC-xml-20001006#NT-Letter seems to imply 16
bit characters, but I'm not clear on whether it's UniCode or something
else. There's also the issue of what Wide_Text_IO actually writes.

> > base64 would be good for values that don't need to be user editable. I
> > guess persistent storage for a large object (whether it is composite
> > or not) could be supported that way. I'm not clear why that would be
> > in a typical "config file". Hmm, maybe a pixel bitmap for an icon?
> 
> That was actually my thought, yes. Funky how we both had the same example in
> mind.

Yes. However, a better solution for bitmaps is to keep the bitmap in a
file that a bitmap editor will recognize, and put the path to it
(possibly relative) in the config file. 

But we don't need to _forbid_ binary data in the config file, if it's
not hard to do.

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

dennison@telepath.com (Ted Dennison) writes:

> Stephen Leake <stephen_leake@acm.org> wrote in message
> news:<wkznydwcm7.fsf@acm.org>...
> > Darren New <dnew@san.rr.com> writes:
> > > You need to specify where a file is written in the path. (API#6)
> > > apparently only talks about reading files.
> > 
> > It should be written where it is found; I'll add that. Hmm, special
> > case if it is a new file; write it to the first directory in the
> > search path.
> 
> Perhaps I'm misunderstanding, but this sounds to me like you are
> saying:
> 
>   1)  Users have very little control over where their config files get
> written to.

Perhaps. The config file API allows the application to specify a
search path (possibly two; one read-only). It's up to the application
how much control to pass on to the user. I'd like the config file
package to support the widest range of scenarios (even ones I don't
like :).

>   2) There will be only one config file per system. 

Not true.

>   3) The facility will be dependent on the very system-dependant
>   feature of directory search paths.

No need for this to be system dependent. In my draft spec, I represent
a search path by a list of strings.

In any case, the body of a config file implementation may very well be
system dependent; Ada.Text_IO is probably not up to implementing this
spec.

> I don't like the first two ideas at all, since they needlessly
> restrict the funtionality. 

Right; that's why they aren't there :).

> I could easily see loads of people who would otherwise love to use
> the facility, forced to roll their own due to problems with 1 and 2.

Please check back when I post the draft spec; that will probably make
it clearer.

> Clients of the facility should be able to read and write
> configurations to and from the directories and file names of their
> choice.

Yes, that's true. But we also have to handle the situations that
Darren is bringing up. An application should be able to use both
system-wide and user-specific config files. Users should _not_ be able
to edit system-wide config files, but they _should_ be able to read
them, as defaults that are overridden by their user-specific files.

> The third issue I think requires a good deal of discussion before
> being accepted. For one thing, it will prevent anyone from making a
> portable implementation of the config file facility. 

Well, using GNAT.Directory_Operations makes it portable in one sense.

100 % portable across operating systems is not possible. Some sort of
accomodation must be made for file and directory access on different
operating systems.

> It should also be noted that if this facility's behavior depends on
> an implementation-defined facility, then you are basicly saying that
> (part of) this facility's behavior is implementation-defined. 

That is certainly undesirable. But I think that as long as terms like
"directory" and "file" are well-defined, we should be ok. Note that
the MAC tends to blur these a little; a "file" has a "resource-fork"
and a "data-fork" (I think those are the right terms :).

> That in turn means programs that use this facility are going to have
> portability issues. In my mind, that's more serious than having
> non-portable config files.

Yes. The config-file API is intended to be completely portable.

> > > FF #3 I would disagree with. It limits having large items in the
> > > config file format.
> > 
> > Only if there is a maximum line length. I guess some text editors
> > might impose one, but Ada.Text_IO doesn't.
> 
> Since it requires that the user supply a fixed-length buffer,
> Ada.Text_IO effectively *does* impose a limit. It just lets you decide
> what that limit is. :-)

As has been discussed before on cla, there are ways around this.

> > > Making key/value pairs terminated by a newline is specifying the
> > > implementation.
> 
> I'd disagree with this statement. It is (partially) specifying a file
> format, but in no way specifies how the config file facility
> implements it. The file format *has* to be part of the specification,
> it can't be left as an "implementation detail".

Exactly.

> Presumably this is just a first cut to get the "easy ones" ironed
> out, before we start to agree on a file format. Right?

Yes.

> If so, a lot of this discussion will be rendered moot when the file
> format is decided upon.

Yes.

> > I'm not sure what your question is. Oh, I see; you want me to request
> > key "foo", and if not found in ~/.bashrc then look in /etc/bashrc to
> > find "foo". Hmm.
> > 
> > I guess we could add an "append_read_only" operation, that would read
> > in a second file, and store it in a separate read-only tree. I like
> > that. 
> 
> Perhaps it would be better to allow for some kind of "chaining" of
> configuration objects? That way items not found in the first
> configuration might be found in later ones.

That's what I was trying to say. There needs to be more than two files
in the chain, but only one (the first one?) is writeable.

> In most of my projects that use configuration files, there are a set
> of hard-coded defaults for all configuration values. That way if no
> config files exist, we still have a reasonable set of defaults
> defined. A client creating a "default" configuration object that is
> loaded with values from the program at initialization time and
> chained on the back would be one way of implementing this.

Yes, that's what I'd like to support.

> > > Question: Is the file closed (w.r.t. the OS) between the end of
> > > the read and the start of the write?
> > 
> > Yes. Actually, I think the file is OS closed at the end of Open, after
> > the data is read in.
> 
> *That* sounds like an implementation detail to me. Its a good idea
> perhaps, but still an implementation detail.

Yes, it is.

> > base64 would be good for values that don't need to be user editable. I
> > guess persistent storage for a large object (whether it is composite
> > or not) could be supported that way. I'm not clear why that would be
> > in a typical "config file". Hmm, maybe a pixel bitmap for an icon?
> 
> Motif's UIL did these as text. You define the colormap for each ASCII
> character, then use the defined ASCII characters to draw the bitmap.
> In the text buffer it looked a bit like ASCII art. The only big
> drawback I saw was that the "art" value is destroyed if your bitmap
> gets wider than your text editor.

I'd like to have a general "binary data - not human editable" data
type. Maybe with an Ada Streams interface?

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> I think there should also be some sort of tag at the front of each file
> saying it's an Ada config file, even if only
> 
>  -- Ada Grace.Config file V1.0 -- Do not hand-edit.

Part of the point is to _allow_ hand editing :).

> so one could check you've found a proper file and know what version
> it is.

Yes, I'm leaning towards a standard implemntation of config file
versions. If only to hint to application builders that they should
consider versioning issues! It's easy to do in the root tag of an XML
document. 

> > > Yes. Actually, I think the file is OS closed at the end of Open,
> > > after the data is read in.
> > 
> > *That* sounds like an implementation detail to me. Its a good idea
> > perhaps, but still an implementation detail.
> 
> I think it's an important detail for systems where having a file open
> changes the semantics for other users. 

Not for config files. These are _explicitly_ not accessible by
parallel processes.

Hmm. Guess I have to allow read-only parallel access if I'm allowing
system-wide config files. Sigh. Why do things have to be so
complicated?

So yes, the file is closed at the end of an open operation.

> If, for example, you leave it open after you write it, it's possible
> I won't be able to read it. 

Arg. Who gets to write a system config file? This is why I wanted to
exclude parallel access. 

> If you leave it open after reading it, I may be dedicating resources
> on the file server to keeping track of it (assuming it's centrally
> served, for example). Closing it each time would make it easier to
> reason about multiple users using the same config file, and the
> overhead of reopening it should be minimal, given it's a config
> file.

Yeah, I think closing it after each open and flush should be required.

This will still depend on OS file system locking for preventing read
while someone has a file open for write. That is _not_ required by the
config file API, so we can use it on a primitive OS like VxWorks (hmm,
maybe VxWorks has file locking?) or DOS. But it is an added feature of
using a real OS.

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

dennison@telepath.com (Ted Dennison) writes:

> Darren New <dnew@san.rr.com> wrote in message
> news:<3CFB94A7.A455B8DD@san.rr.com>... 
> > I think they should be case sensitive unless there's a standard way of
> > converting uppercase unicode to lowercase unicode in Ada's libraries.
> 
> We need to match up well with what's in the Ada standard I think. 
> 
> The Ada standard has an Ada.Strings.Maps.Constants.Lower_Case_Map and
> an Ada.Strings.Wide_Maps.Wide_Constants.Lower_Case_Map. There is no
> reason why we can't define things to say that keys will be fed through
> the appropriate Lower_Case_Map before being matched.

Good point.

> If the language itself has some kind of problem with
> Ada.Strings.Wide_Maps.Wide_Constants.Lower_Case_Map, I say that is
> the language's problem, and outside the scope of a configuration
> item facility to try to worry about fixing. As long as we define
> what we are doing carfully this way, it is still explicit. If we
> stick to using the standard library, things should at least behave
> for people the way they have come to expect them to behave when
> using Ada.

According to my interpretation of recent posts by Robert Dewar (is
that qualified enough :), there is very little use of Wide_String. So
it's not clear whether Lower_Case_Map is "what people expect". But I
agree it would be a good place to start. 

Perhaps case sensitivity should be an option in the config file API.

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

antonio_duran@hotmail.com (Antonio Duran) writes:

> 1. In my experience, most of the time, config files are writen, edited
> or modified by humans and read and interpreted by programs. With this
> in mind, in my modest opinion, the importance of having an API for
> writing to config files is secondary. What is important is having the
> means that allow a human user to understand what is in the config
> file.

True.

> Configuration tasks are simpler if the config file is commented and
> handling comments is difficult if you use an API call to write to
> the file. 

Good point; I hadn't thought of that.

I think this makes XML a better idea; we can allow for a "comment" tag
that is saved in the memory image, and written back out.

> When config file gets complicated enough one must write an
> application to handle it (eg. Comanche to mantain Apache config
> files).

Yes. We need to support both modes of operation.

> 2. Seems that there is a consensus in that a config file must store
> key/value(s) pairs. I don't understand the requirement to use
> Wide_String for keys. I'm from Spain and I usually write in Spanish
> but when programming in Ada I must adhere to the Ada syntax, I can't
> write Ada identifiers that are syntactically correct spanish words
> and the Ada reserved words are english words. 

Yes, but there's no reason config files have to have the same
restrictions. 

> Supporting Wide_String could be a cool feature for, for example,
> mantaining localized messages in different languages, however: is
> this the purpose of config files?. 

You could use them that way. In this particular case, the Gnu
'get_text' facility is probably better. But if you have only a few
messages, and also other things to configure, a config file would be a
good place to do this.

> I think the purpose of config files is to provide parameters to a
> program or application and not to be used as a repository for
> localized versions of program messages. Of course, they could be
> used in that way. I suggest that keys in config file meet the Ada
> syntax for identifier and be case-insensitive.

Ok, thanks for your input. 

But, I'd like you to think again :). Can you come up with any
situation in which you would use Wide_String keys? Do you use
Wide_String for anything in your applications?

Here in the US, it's easy to think "ASCII solves all problems"; I'd
appreciate some input from someone for whom that is not true.

> 3. I think that is important to handle more than one config file in
> a single program. From my point of view, one of the most important
> drawbacks of layouts such as the Windows Registry is that everything
> is on a single place so applications that can access to a single
> part can also write other parts. In complicated apps it could be
> desirable to mantain a set of different config files one for each
> part of the application. The ability for handling multiple config
> files in a single program should be a requirement for a config file
> handling package.

Yes, I intend to support this. Please check the draft spec when I post
it, to see if it meets these needs.

> 4. I don't understand very well the implications of "... specified
> by generic parameter at compile time ..." (API req. 5). Does this
> mean that the name or the path of the file must be hard coded?. 

only the defaults; API req 7 allows the application to override these
defaults. 

> If so I think we've missed totally the point (should I recompile
> each time I change the name or location of the config file?).

The point was that some applications (typically from Gnu and/or Linux)
put config files in very standard places, like /usr/local/share. So
this default should be compiled in on Linux. However, other
applications will want to specify thier own places for config files,
so that has to be allowed as well.

> 5. Regarding to the file syntax, long time ago I worked on a system
> that runs on CAIS (Common APSE Interface Set). 

> <snip example>

> There are, however, some drawbacks in this representation:
> 
> - A escaping schema is needed for unprintable characters in strings
> (?).
> - Nested '(' and ')' would add a high degree of complexity for the
> human user/reader.

And it's really only readable for people who know Ada; we'd like this
to be useful to a wider audience.

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> And for the folks wanting to use XML for this, here's what the IETF is 
> thinking about using XML. Just an interesting draft.
> 
> http://www.ietf.org/internet-drafts/draft-hollenbeck-ietf-xml-guidelines-04.txt

Thanks, that looks very interesting. I'll read it later :).


-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

Volkert Barr <barr@cs.tu-berlin.de> writes:

> I have also set up a small document. This should be merged with the work 
> of Mr. Stephen Leake. I tried to contact him, but he did not replyed. 

I guess I should have mentioned on the web site that I was on vacation
:).

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Oh, I should also comment that if you're *not* going to follow the windows
> .ini file format, chances are the default extension shouldn't be ".ini". :-)

note that on Unix, the notion of "file extension" is not built in to
the file system as it is on Windows. So I think the API should make no
mention of "file extension".

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

dennison@telepath.com (Ted Dennison) writes:

> However, I think it would be a big mistake to make rigid compatability
> with Windows' .ini files a goal. For one thing, many if not most Ada
> programs do not run on Windows. 

Possibly true, but not really relevant. If it's a good format, it will
be useful on non-Windows systems.

Perhaps you are implying "and therefore many people will not be
familiar with this format". That is probably true. But it's easy to
figure out!

> For another, that file format is in thrall to Microsoft, and is
> probably only defined in Microsoft copyrighted documents, if at all.
> Anything we come up with will have to be something we can define
> with precision in the new LRM, without having to worry about
> violating anyone's copyright, or having the format change out from
> under us at the whim of one (non-Ada) vendor.

Hmm. I'm not a copyright lawyer, but it seems like we can simply
define a syntax that looks like ini files. As long as we don't
actually read Microsoft documents in the process, we should be ok.
That's what the MingW people did for the Windows headers files.

But we should probably ask a lawyer about that, as has been said about
similar topics here recently.

-- 
-- Stephe

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> Good point. I guess I can check the write privs on the found file.

Naw. 
 
> Or provide two search paths; one read-only and one writeable.

Or just require the full path to the config to be saved, and perhaps default
it to where it was loaded from. I think if you try to "guess" which config
file to clobber, you're going to guess wrong too often.
 
> > If "no" and you're allowing Unicode, then you have to worry about the
> > bizarre transformations that some languages have.
> 
> yes. And Glib.XML is case sensitive. Although Glib.XML also uses
> String for Tags, not Wide_String. So I have to check the XML
> definition; does it allow Wide_String for tags? Hmm.

XML is case sensitive and uses unicode.

> http://www.w3.org/TR/2000/REC-xml-20001006#NT-Letter seems to imply 16
> bit characters, but I'm not clear on whether it's UniCode or something
> else. 

UTF-8 or UTF-16, as far as I understand.

> There's also the issue of what Wide_Text_IO actually writes.

That too.

> Yes. However, a better solution for bitmaps is to keep the bitmap in a
> file that a bitmap editor will recognize, and put the path to it
> (possibly relative) in the config file.

Possibly. Depends on what you're trying to accomplish.
 
> But we don't need to _forbid_ binary data in the config file, if it's
> not hard to do.

Correct. But if you want it editable, you probably need to keep it text.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> 
> Darren New <dnew@san.rr.com> writes:
> 
> > I think there should also be some sort of tag at the front of each file
> > saying it's an Ada config file, even if only
> >
> >  -- Ada Grace.Config file V1.0 -- Do not hand-edit.
> 
> Part of the point is to _allow_ hand editing :).

Right. But it should still be marked "don't do that". Or maybe "Read manual
before hand-editing". ;-)

> > I think it's an important detail for systems where having a file open
> > changes the semantics for other users.
> 
> Not for config files. These are _explicitly_ not accessible by
> parallel processes.

Um... So, if my config file is open, and thus my backup program doesn't back
it up, is that good or bad? If a program is running that only reads config
files, should I be allowed to copy the config file it uses to a different
directory while it's running?

I'm not just talking about multiple users of the config file API trying to
write at the same time.

> Hmm. Guess I have to allow read-only parallel access if I'm allowing
> system-wide config files. Sigh. Why do things have to be so
> complicated?
> 
> So yes, the file is closed at the end of an open operation.

Yeah. Just so much eaiser. :-)
 
> > If, for example, you leave it open after you write it, it's possible
> > I won't be able to read it.
> 
> Arg. Who gets to write a system config file? This is why I wanted to
> exclude parallel access.

Well, *somebody* has to write it, yes? I'm just saying that not all the
world is UNIX, and not every OS defaults to letting you edit files that are
already open.
 
-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> Note that
> the MAC tends to blur these a little; a "file" has a "resource-fork"
> and a "data-fork" (I think those are the right terms :).

So does Windows. (Under NTFS at least.) It's just that nobody uses them.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> > Configuration tasks are simpler if the config file is commented and
> > handling comments is difficult if you use an API call to write to
> > the file.
> 
> Good point; I hadn't thought of that.
> 
> I think this makes XML a better idea; we can allow for a "comment" tag
> that is saved in the memory image, and written back out.

Or simply allow an association of a comment string with each config value.
It gets filled in when read, or left out when written. It doesn't take XML
to do this.

-- This is comment one
key1=value1
-- This is comment two
key2=value2

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Preben Randhol]

On Mon, 10 Jun 2002 01:33:28 GMT, Darren New wrote:
> Right. But it should still be marked "don't do that". Or maybe "Read manual
> before hand-editing". ;-)

No it should not read "Don't handedit" if you will allow the user to do
this. It should be the latter you said. The reason is that for the files
that you don't want a user to hand edit you should reserve the "Don't
edit". Then the user understands that it is serious.

Preben

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > Good point. I guess I can check the write privs on the found file.
> 
> Naw. 

Hmm. What if I find a "system" config file, that I _don't_ have write
priviledge for?

> > Or provide two search paths; one read-only and one writeable.
> 
> Or just require the full path to the config to be saved, and perhaps default
> it to where it was loaded from. I think if you try to "guess" which config
> file to clobber, you're going to guess wrong too often.

With clear rules about it, there's no "guessing". Here's my proposed
rule:

---
There are two search paths; one for files that are read only, one for
files that are writeable.

If the first config file is found on a read only path, the "flush"
operation creates a new file in the first directory of the writeable
path.
---

That gives total control to the client of the Config_Files package.

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > > Configuration tasks are simpler if the config file is commented and
> > > handling comments is difficult if you use an API call to write to
> > > the file.
> > 
> > Good point; I hadn't thought of that.
> > 
> > I think this makes XML a better idea; we can allow for a "comment" tag
> > that is saved in the memory image, and written back out.
> 
> Or simply allow an association of a comment string with each config value.
> It gets filled in when read, or left out when written. 

The point would be to save the comments in memory, and write them out
again. That way, the comments can guide someone hand-editing the file.

We might as well let the app write comments via the API as well.

> It doesn't take XML to do this.

True; Ada can do anything :).

-- 
-- Stephe

Re: config files proposal

[Georg Bauhaus]

Darren New <dnew@san.rr.com> wrote:
: 
: UTF-8 or UTF-16, as far as I understand.

<?xml version='1.0' encoding='...'?>
declares "external representation". External entities in UTF-8/16 must
be read, others may. UTF-8 has the advantage of being stateless and
compatible across wires. We often have
<?xml version='1.0' encoding='ISO-8859-1'?>, useless for anything
outside "west-latin" tradition, but good and correct otherwise.

-- georg

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> 
> Darren New <dnew@san.rr.com> writes:
> 
> > Stephen Leake wrote:
> > > Good point. I guess I can check the write privs on the found file.
> >
> > Naw.
> 
> Hmm. What if I find a "system" config file, that I _don't_ have write
> priviledge for?

Sorry. I forgot to finish that sentence. I meant  "Naw, that's not good
enough." Or would you want root changing the system configuration every time
she ran your program?

> > > Or provide two search paths; one read-only and one writeable.
> >
> > Or just require the full path to the config to be saved, and perhaps default
> > it to where it was loaded from. I think if you try to "guess" which config
> > file to clobber, you're going to guess wrong too often.
> 
> With clear rules about it, there's no "guessing". Here's my proposed
> rule:
> 
> ---
> There are two search paths; one for files that are read only, one for
> files that are writeable.
> 
> If the first config file is found on a read only path, the "flush"
> operation creates a new file in the first directory of the writeable
> path.
> ---
> 
> That gives total control to the client of the Config_Files package.

That would work, yes. That's a good compirimse. Especially if one can change
these paths at run-time, on a per-config-file basis. In that case, one could
simply change the writable path to a single directory to force a config file
to be written. I'd also suggest that it be legal to have a zero-length
writable path, leading to an exception if an attempt is made to write the
config file. (Or perhaps to have changes discarded?)

Altho I must admit I see no reason for a writable *path* as such. 

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Darren New]

Stephen Leake wrote:
> > Or simply allow an association of a comment string with each config value.
> > It gets filled in when read, or left out when written.
> 
> The point would be to save the comments in memory, and write them out
> again. That way, the comments can guide someone hand-editing the file.

Sorry. I meant "written out when written". Sheesh. OK, no more late-night
posting for me.

My intention was to say that one should be associating comments with keys,
not with arbitrary places in the INI file. Then, you wouldn't be restricting
yourself to a specific file format that can support (say) multiple
independent comments per key.

> We might as well let the app write comments via the API as well.

I think you need to be careful that you don't make the API too close to file
semantics. I.e., you want "here's a data structure, store it", rather than
"write this key/value, write this comment, write this other key/value, write
another comment, oh and another comment", I would think.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Ted Dennison]

Stephen Leake <stephen_leake@acm.org> wrote in message news:<wkr8jgrya2.fsf@acm.org>...
> I'd like to have a general "binary data - not human editable" data
> type. Maybe with an Ada Streams interface?

An intersting idea. But if you have a separate data type for binary
data, don't you open the door for other data types, eg: Float,
Integer, Duration, Date/Time, etc?


-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  http://www.telepath.com/dennison/Ted/TED.html

Re: config files proposal

[Ted Dennison]

Darren New <dnew@san.rr.com> wrote in message news:<3D03E1AD.80EB805D@san.rr.com>...
> Stephen Leake wrote:
> > > Configuration tasks are simpler if the config file is commented and
> > > handling comments is difficult if you use an API call to write to
> > > the file.
> > 
> > Good point; I hadn't thought of that.
> > 
> > I think this makes XML a better idea; we can allow for a "comment" tag
> > that is saved in the memory image, and written back out.
> 
> Or simply allow an association of a comment string with each config value.
> It gets filled in when read, or left out when written. It doesn't take XML
> to do this.
> 
> -- This is comment one
> key1=value1
> -- This is comment two
> key2=value2

True, but it complicates the parsing code significantly, while XML
could handle it easily with some kind of attribute-comment tag. What
you essentially have here is another attribute. One of my 2 things I
said we'd have to require out of the config language to make XML a
good idea was multiple tags per key. (The other was nested keys).

-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  http://www.telepath.com/dennison/Ted/TED.html

Re: config files proposal

[Ted Dennison]

Darren New <dnew@san.rr.com> wrote in message news:<3D04C9D0.7DFCED9D@san.rr.com>...
> My intention was to say that one should be associating comments with keys,
> not with arbitrary places in the INI file. Then, you wouldn't be restricting
> yourself to a specific file format that can support (say) multiple
> independent comments per key.

Yeah, but then you can't make group comments, like:
  -----------------------------------------------------------
  -- The next few items set up system timeout values. Valid 
  -- values range from 0 to 120. The units are in seconds. 
  -- A timeout value of 0 indicates no timeout.


-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  (temporarily down)

Re: config files proposal

[Darren New]

Ted Dennison wrote:
> Yeah, but then you can't make group comments, like:
>   -----------------------------------------------------------
>   -- The next few items set up system timeout values. Valid
>   -- values range from 0 to 120. The units are in seconds.
>   -- A timeout value of 0 indicates no timeout.

Sure. Any comment more than 76 characters wide gets wrapped on output.

> > -- This is comment one
> > key1=value1
> > -- This is comment two
> > key2=value2
> 
> True, but it complicates the parsing code significantly, while XML
> could handle it easily with some kind of attribute-comment tag.

Uh, it does? I don't *think* so. It seems quite straightforward to parse
such a thing.

My concern with the XML vs INI wrt comments is the possibility that it would
make the data structure rather complex. What API do you want to use to say
"I want section FOO to have two comments, then key1=value2, then another
comment"? All of a sudden, you're not manipulating a collection of keys and
values, but rather you're writing out streams of tokens. If each item in the
database is a 4-tuple of package, comment, key, value, then it's easy to
manipulate. If you start treating the database as a stream of values, where
you can have empty sections, sections with only comments, comments with no
keys associated, etc, you're going to have a much more complex API for no
real good reason other than to make it look like XML. :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Stephen Leake]

dennison@telepath.com (Ted Dennison) writes:

> Stephen Leake <stephen_leake@acm.org> wrote in message
> news:<wkr8jgrya2.fsf@acm.org>... 
> > I'd like to have a general "binary data - not human editable" data
> > type. Maybe with an Ada Streams interface?
> 
> An intersting idea. But if you have a separate data type for binary
> data, don't you open the door for other data types, eg: Float,
> Integer, Duration, Date/Time, etc?

Well, we already have "approved" interfaces for scalar types.

Yes, the user could abuse the binary interface to store a Date, rather
than adding an appropriate interface for Calendar.Time. That's not bad
as a quick work-around.

My motto for this is "I need the power to shoot myself in the foot,
and the discipline not too". 

-- 
-- Stephe

Re: config files proposal

[Volkert Barr]

Darren New wrote:
> 
> Ted Dennison wrote:
> > Yeah, but then you can't make group comments, like:
> >   -----------------------------------------------------------
> >   -- The next few items set up system timeout values. Valid
> >   -- values range from 0 to 120. The units are in seconds.
> >   -- A timeout value of 0 indicates no timeout.
> 
> Sure. Any comment more than 76 characters wide gets wrapped on output.
> 
> > > -- This is comment one
> > > key1=value1
> > > -- This is comment two
> > > key2=value2
> >
> > True, but it complicates the parsing code significantly, while XML
> > could handle it easily with some kind of attribute-comment tag.
> 
> Uh, it does? I don't *think* so. It seems quite straightforward to parse
> such a thing.
The problem here is not the parsing of an comment. The problem is to
what
kind of element the comment belongs. This information is needed, if i
want
my application to read, manipulate and write an configuration 
data from/back to a file without losing my comment information. Please,
i have 
set up a  small data model, which take this case into count (comment is
traditionally 
called description). Look at
http://swt.cs.tu-berlin.de/~barr/config_package.pdf.
If such a case should be handled, is is up to us to define a appropriate 
grammar.

Volkert

Re: config files proposal

[Darren New]

Volkert Barr wrote:
> > > True, but it complicates the parsing code significantly, while XML
> > > could handle it easily with some kind of attribute-comment tag.
> >
> > Uh, it does? I don't *think* so. It seems quite straightforward to parse
> > such a thing.
> The problem here is not the parsing of an comment.

Well, that was the problem being discussed, I think. :-)

> The problem is to
> what
> kind of element the comment belongs.

That's a good point. Sure, you'd want comments on sections and subsections
as well. That still doesn't need XML to implement. 

> http://swt.cs.tu-berlin.de/~barr/config_package.pdf.

Excellent start, I think!

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: config files proposal

[Volkert Barr]

Darren New wrote:
> 
> Volkert Barr wrote:
> > > > True, but it complicates the parsing code significantly, while XML
> > > > could handle it easily with some kind of attribute-comment tag.
> > >
> > > Uh, it does? I don't *think* so. It seems quite straightforward to parse
> > > such a thing.
> > The problem here is not the parsing of an comment.
> 
> Well, that was the problem being discussed, I think. :-)
i had not engough time to follow the thread, so i was a bit ot 
ouf sync...  ;-)

> > The problem is to
> > what
> > kind of element the comment belongs.
> 
> That's a good point. Sure, you'd want comments on sections and subsections
> as well. That still doesn't need XML to implement.
> 
> > http://swt.cs.tu-berlin.de/~barr/config_package.pdf.
> 
> Excellent start, I think!
I have made an new update! 

Volkert

Re: config files proposal

[Antonio Duran]

I would like to raise a question that could be of interest regarding
config files. It only must be considered as a discussion point and I
am aware of the problems that can convey.

Form my point of view, in some cases it could be necessary/interesting
to keep configuration information confidential and to provide the
means to check the integrity of a configuration file.

Should the config file handling requirements make provisions for this
circumstances?

    Antonio Duran

Re: config files proposal

[Ted Dennison]

Darren New <dnew@san.rr.com> wrote in message news:<3D076CCD.A1F63320@san.rr.com>...
> Volkert Barr wrote:
> > > > True, but it complicates the parsing code significantly, while XML
> > > > could handle it easily with some kind of attribute-comment tag.
> > >
> > > Uh, it does? I don't *think* so. It seems quite straightforward to parse
> > > such a thing.
> > The problem here is not the parsing of an comment.
> 
> Well, that was the problem being discussed, I think. :-)

No, it wasn't.

> > The problem is to
> > what
> > kind of element the comment belongs.
> 
> That's a good point. Sure, you'd want comments on sections and subsections
> as well. That still doesn't need XML to implement. 

You're almost there. What I was trying to point out is that XML
doesn't really support it at all (at least not without adding some
kind of "commented-block" tag. Yeeach.).


> > http://swt.cs.tu-berlin.de/~barr/config_package.pdf.

Out of the 3 options presented there, I think the ".ini" looks the
best by far. The others just have way too much syntactic cruft
obscuring the important information.


-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  (temporarily down)

Re: config files proposal

[Georg Bauhaus]

Volkert Barr <barr@cs.tu-berlin.de> wrote:
:  Look at
: http://swt.cs.tu-berlin.de/~barr/config_package.pdf.

Incidentally, for those who would like to have INI files,
here is an XSLT:

<!-- transform an XML configuration document into a Windows(TM)
     INI-style configuration text (may need refinements).

     Cannot handle the nested sections (are there nested sections in
     the INI file format?).

     See Volkert Barr's draft -->

<transform
  version="1.0"
  xmlns="http://www.w3.org/1999/XSL/Transform">

  <output method="text"/>

  <template match="/configuration">
    ;Configuration: <value-of select="@name"/>
    <apply-templates/>
  </template>

  <template match="description">
    ;<value-of select="normalize-space(.)"/> <!-- line breaking needed here -->
  </template>

  <template match="section">
    [<value-of select="@name"/>]
    <apply-templates/>
  </template>

  <template match="item">
    <value-of select="@*['key']"/>=<apply-templates/>
  </template>
     <!-- @*['key'] instead of @key to work around XSLT key() function  -->

 </transform>

With input copied from the example in the PDF, the output is


    ;Configuration: A_Configuration
  
    ;bla
  
    [A_Section]
    
    KeyX=ValueX
    
    [A_Subsection]
    
      
    ;bla bla
      Key1=Value1
      Key2=Value2
    
  
-- Georg 

Re: config files proposal

[Stephen Leake]

antonio_duran@hotmail.com (Antonio Duran) writes:

> Form my point of view, in some cases it could be necessary/interesting
> to keep configuration information confidential and to provide the
> means to check the integrity of a configuration file.

I'm not clear what you mean by "confidential". Clearly, the
application program that uses the config file must be able to read it.

If you mean that no other process/person should be able to read it,
that would be up to the operating system. If the OS provides access
controls to files, you can enforce confidentiality.

I think that issue has to be outside the scope of the Config_Files
API, since it depends heavily on the OS, and we want the Config_Files
API to be OS independent.


As for checking the integrity of the config file, that could be done
with an XML DTD and a validating parser. Come to think of it, that is
an advantage of using XML that I had not considered. Although, a
non-XML parser could also check integrity to the same degree.

Hmm. I'm not clear what you mean by "integrity". XML DTD's allow
specifying what values must be present, and to some extent what type
they are.

In other situations, "integrity" might mean a version control revision
number, or a checksum, or something similar.

-- 
-- Stephe

Re: config files proposal

[Georg Bauhaus]

Stephen Leake <stephen_leake@acm.org> wrote:
: I think this makes XML a better idea; we can allow for a "comment" tag
: that is saved in the memory image, and written back out.

Incidentally, comments aren't lost in XML and each comment
belongs to an element. Namely, the element in which it appears.
There is a comment() function available to XPath expressions,
and a comment element type in XSLT for the creation of comment
nodes in the output.

(In SGML, things are even fancier. Comments are more Ada like,
except at both ends -- like so -- and they can appear in more
places than in XML.)

-- Georg

Re: config files proposal

[Ted Dennison]

antonio_duran@hotmail.com (Antonio Duran) wrote in message news:<e1a50f2.0206130108.995ef66@posting.google.com>...
> Form my point of view, in some cases it could be necessary/interesting
> to keep configuration information confidential and to provide the
> means to check the integrity of a configuration file.
> 
> Should the config file handling requirements make provisions for this
> circumstances?

No. It sounds to me like you are arguing for an encryption/decryption
package. Then you could add use of it on certian keys as an option.
That might be a good discussion, but something as involved as
encryption wouldn't belong in a config file package.

Or are you just talking about "password" type data? Most OS's have
special calls to deal with that type of data. I suppose it would be
possible to write some kind of one-way encoding function in Ada.
However, that should still probably be in its own package.


-- 
T.E.D. 
Home     -  mailto:dennison@telepath.com (Yahoo: Ted_Dennison)
Homepage -  (temporarily down)

Re: config files proposal

[Antonio Duran]

Stephen Leake <stephen.a.leake.1@gsfc.nasa.gov> wrote in message news:<uu1o7uuw1.fsf@gsfc.nasa.gov>...
> antonio_duran@hotmail.com (Antonio Duran) writes:
> 
> > Form my point of view, in some cases it could be necessary/interesting
> > to keep configuration information confidential and to provide the
> > means to check the integrity of a configuration file.
> 
> I'm not clear what you mean by "confidential". Clearly, the
> application program that uses the config file must be able to read it.
> 
> If you mean that no other process/person should be able to read it,
> that would be up to the operating system. If the OS provides access
> controls to files, you can enforce confidentiality.
> 
> I think that issue has to be outside the scope of the Config_Files
> API, since it depends heavily on the OS, and we want the Config_Files
> API to be OS independent.
> 
> 
> As for checking the integrity of the config file, that could be done
> with an XML DTD and a validating parser. Come to think of it, that is
> an advantage of using XML that I had not considered. Although, a
> non-XML parser could also check integrity to the same degree.
> 
> Hmm. I'm not clear what you mean by "integrity". XML DTD's allow
> specifying what values must be present, and to some extent what type
> they are.
> 
> In other situations, "integrity" might mean a version control revision
> number, or a checksum, or something similar.

What I meant with confidentiality and integrity was to use
cryptographic technics to encrypt/decrypt config files for
confidentiality and hash/digest for checking integrity.

I know this is far beyond the scope since by using that the file
couldn't be modified by hand with a simple text editor. My intent was
to know if that kind of functionality could be of interest in some
circumstances.

Antonio Duran

Config_Files proposal

[Stephen Leake]

I've updated the requirements on my web page:

http://users.erols.com/leakstan/Stephe/Ada/Config_Files/config_files.html

Note that it moved into its own directory, and I did not preserve the
requirement numbers from the last version. I think I've captured the
discussion we've had here.

I've also posted an example implementation using Glib.XML. It does
_not_ implement all of the API requirements (in particular the search
path features and comments are not there); I'm focusing on file format
issues at the moment. Since Glib.XML only supports String, this
implementation does not use Wide_String. There is a test driver also,
giving examples of how the API is used. If you want to compile it
yourself, you'll need to get Glib.XML from the GtkAda distribution at
http://libre.act-europe.fr/GtkAda/.

Please comment on the spec, or anything else :).

Now I'll do an implemenation using the Java properties file format, to
compare the two.

If someone wants, they can do an implementation for the Windows ini
format, or some other format. But I think Windows ini is less useful
than the Java properties; the Java format supports multiple levels of
key heirarchy. And I think the other formats are more confusing to
edit and parse.

-- 
-- Stephe

Re: config files proposal

[Stephen Leake]

antonio_duran@hotmail.com (Antonio Duran) writes:

> What I meant with confidentiality and integrity was to use
> cryptographic technics to encrypt/decrypt config files for
> confidentiality and hash/digest for checking integrity.

Ok, that's clear.

> I know this is far beyond the scope since by using that the file
> couldn't be modified by hand with a simple text editor. 

Yes.

> My intent was to know if that kind of functionality could be of
> interest in some circumstances.

Not to me, but it's a good question.

-- 
-- Stephe

Config_Files proposal

[Stephen Leake]

I've posted another example spec and implementation, at 

http://users.erols.com/leakstan/Stephe/Ada/Config_Files/config_files.html

This second implementation uses the Java property style file format.

I also changed the spec some. In particular, I got rid of the
"Section" parameters; keys now just use dotted notation.

I'd like to have a semi-formal vote on which file format to go with.
Then I'll move this into Grace, and we can finish up all the details.

To vote, either post here with your preference, or send me email.
Voting closes in a week (June 25), but I'm not going to be picky about
time of day :).

I've listed some advantages and disadvantages of each proposed file
format on the web page. Feel free to tell me about advantages or
disadvantages I've missed.

Based on that list, I vote for the Java property style format. I also
think most people here agree it's a good format.

Note that we are only voting on the general file format. The remaining
details of the file format (like how to represent comments) and spec
will be worked out later.

-- 
-- Stephe

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> Note that we are only voting on the general file format. The remaining
> details of the file format (like how to represent comments) and spec
> will be worked out later.

I think this is kind of the wrong way around. Asking which of two completely
different file formats is better, and then trying to build an API based on
that, seems backwards.

That said, there's some things missing from the spec:

1) When you read three config files off the path, change a value that was in
the second one, and write it out again, where does it get written?

2) Can you have a "local" and a "global" config file, such that changes to
the local override the global, but it's possible to have only some of the
information in the local file? E.g., the system administrator says "the mail
server is a smtp-server.example.com, but each user fills in his own "from"
email address"?

3) If you enable #2, and then the user changes her email address, does it
also store back the mail server address, thereby preventing the
administrator from changing it?

4) If you have a list of items, such as a recent-file list, how are they
managed? Does it make sense to use [] notation or something?

5) Do comments go in the files? If so, how does that interact with the
previous questions?

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Dr. Michael Paus]

Stephen Leake wrote:

> I'd like to have a semi-formal vote on which file format to go with.
> Then I'll move this into Grace, and we can finish up all the details.
> 
> To vote, either post here with your preference, or send me email.
> Voting closes in a week (June 25), but I'm not going to be picky about
> time of day :).

I vote for the Java version. For the implementation you should try to
stay as close as possible to the Java version as well.

Michael

Re: Config_Files proposal

[Ted Dennison]

Stephen Leake <stephen.a.leake.1@gsfc.nasa.gov> wrote in message news:<uwuswy0qr.fsf@gsfc.nasa.gov>...
> I've posted another example spec and implementation, at 
> 
> http://users.erols.com/leakstan/Stephe/Ada/Config_Files/config_files.html
> 
> This second implementation uses the Java property style file format.
> 
> I also changed the spec some. In particular, I got rid of the
> "Section" parameters; keys now just use dotted notation.

I think either the Java or the X style would be acceptable. With a bit
of work, someone could probably sell me on the ini style too though.

> I'd like to have a semi-formal vote on which file format to go with.
> Then I'll move this into Grace, and we can finish up all the details.

I hate to do this, but I have to go on record as being strongly
against voting for things. Everything in Grace needs to be decided by
general consensus. The only reason there isn't an official policy on
this at the Grace website is because I haven't had time to put it up.

I'm not saying everyone has to agree that one of these styles is the
best of all possible styles. But there should be some kind of general
agreement that one of them is meritous and acceptable, if not
everyone's perfect option. If we get down to two that seem to be
equally good with no huge contraversy between them, then perhaps a
vote or coin flip or executive decision or something similar would be
in order.

BTW: I have recently appointed Stephen to lead the config file effort
for Grace (see http://savannah.gnu.org/projects/grace/ ). So if there
were an "executive decision" to be made in the context of the Grace
project, it would be up to him to make it.

This is a good example of why we need to go with this approach. We may
now have consensus against using XML, but there really hasn't been
much discussion here on the relative merits of the other approaches
(other than that some folks think INI's should be reseved for
Windows). If I were to just vote now between the other 3 options, it
would be based on looks. Another thread here has recently illustrated
quite well that this would be quite succeptable to parrochial biases.
:-)

So really a better question right now would be to ask if anyone has
any big objections to any of the approaches presented (other than XML,
to which the objections are already well recorded). Also, does anyone
have any strong reasons why they think one is much better than the
others?

Re: Config_Files proposal

[Ted Dennison]

"Dr. Michael Paus" <paus@ib-paus.com> wrote in message news:<3D102A4E.2010900@ib-paus.com>...
> I vote for the Java version. For the implementation you should try to
> stay as close as possible to the Java version as well.

Another good example of my problem with votes. *Why* do you feel this way?

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > Note that we are only voting on the general file format. The remaining
> > details of the file format (like how to represent comments) and spec
> > will be worked out later.
> 
> I think this is kind of the wrong way around. Asking which of two completely
> different file formats is better, and then trying to build an API based on
> that, seems backwards.

That isn't what I meant to say :). The API _is_ orthogonal to the file
format.

What I meant to say was that I'd like to decide now whether we will go
forward with Java properties style or XML (or possibly some other
format). Given that choice, there are still (minor) issues to decide
about the file format.

In addition to that decision, there are decisions about the package
spec to be made. Note that the file "grace-config_files_java.ads"
is a package spec that could easily have a body that uses an XML file
format. I should have used a different name :(.

> That said, there's some things missing from the spec:
> 
> 1) When you read three config files off the path, change a value that was in
> the second one, and write it out again, where does it get written?

I tried to answer that in requirement 10, but I can see it isn't very
clear.

The intent is that only one file be writable, and that is where all
values are written.

For example, if you open ~/.myconfig (writable), /usr/local/app/config
(readonly) and /etc/app/config (readonly), then any values you change
are written to ~/.myconfig.

> 2) Can you have a "local" and a "global" config file, such that
> changes to the local override the global, but it's possible to have
> only some of the information in the local file? E.g., the system
> administrator says "the mail server is a smtp-server.example.com,
> but each user fills in his own "from" email address"?

Yes, but any values the user changes will be written to the local
file. Same example as above.

The final doc for the Config_Files library will have examples of these
use cases.

> 3) If you enable #2, and then the user changes her email address,
> does it also store back the mail server address, thereby preventing
> the administrator from changing it?

Hmm. I was going to say "no, the user did not change that value". That
is my intent. But my draft implementation does not follow that intent;
it writes all the values to the writeable (local) file.

I'll declare that implementation broken, in this respect. I guess I
need a "value changed since read" marker, and only changed values get
written. That's not bad. Then I have to enhance the unit test to test
this. That will be harder :).

The requirements need to be clearer on this as well.

Thanks for pointing this out!

Hmm. Now I think we need the queries "what file was this value read from"
and "what file will this value be written to".

> 4) If you have a list of items, such as a recent-file list, how are
> they managed? 

Given the current requirements and spec, you need a unique key for
each scalar item; foo.bar.item.1, foo.bar.item.2, ... foo.bar.item.n.

> Does it make sense to use [] notation or something?

I'm leaving that for a future enhancement. The biggest problem I have
with it at the moment is that it requires some notion of a "list" of
values an arbitrary scalar type in the Config_Files spec. This could
probably be handled with a generic or a tagged type or an
unconstrained array. At the moment, we have no explicit lists in the
API, which keeps things simple (the search path is a list in the body,
but not in the visible part of the spec). Since the same effect can be
accomplished via numbered keys, I don't think it's worth pursuing at
this time.

Hmm, I'm wrong. There is a requirement to return "a list of keys". I
guess that will be a list of strings. When I implement that, I'll
think about this issue again :).

> 5) Do comments go in the files? If so, how does that interact with
> the previous questions?

The requirements say we must support comments. Neither of my
implementations do so. That's one of the "minor" details that remain
to be worked out.

Once again, thanks for your insightful comments.

-- 
-- Stephe

Re: Config_Files proposal

[Robert I. Eachus]

Ted Dennison wrote:

 
> So really a better question right now would be to ask if anyone has
> any big objections to any of the approaches presented (other than XML,
> to which the objections are already well recorded). Also, does anyone
> have any strong reasons why they think one is much better than the
> others?


Yes.  But the problem is not with the various file formats, it is with 
the approach.  At this point we should be debating a set of requirements 
and a strawman interface package specification.  If the result favors 
one or the other file format, so be it.  My expectation is that a good 
set of requirements and interface will allow more than one underlying 
implementation.  I'd love to see a result that maps to registry, .ini 
files, and .cfg files with no differences in usage outside the create 
and/or open parameters.

Re: Config_Files proposal

[Marin David Condic]

XML is significantly better than some of the other formats suggested here
for one factor: Other tools will be able to make use of it. If it comes to
displaying an initialization file or using the data in any other way, you
gain the ability to make them work in entirely new contexts without having
to do some form of data conversion.

All other things being equal, gaining an XML parser in the process would be
A Good Thing. Imagine if one day when everything is sunshine and rainbows
and pastel unicorns and Ada has some kind of standard means of displaying
web page stuff, your initialization file XML data slides right in there.
Imagine that once the tools are there to handle XML on input and output how
many other things might be built on top of Grace that could leverage this
capability.

IOW, use this as an excuse to build an XML DOM & SAX. (I'd like to see an OO
implementation of this using tagged records and closely following the W3C
DOM model, despite the very nice non-OO implementation done by XML/Ada.)

There's my $0.02 on it. :-)

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com


"Ted Dennison" <dennison@telepath.com> wrote in message
news:4519e058.0206190708.2ef205e4@posting.google.com...
>
> So really a better question right now would be to ask if anyone has
> any big objections to any of the approaches presented (other than XML,
> to which the objections are already well recorded). Also, does anyone
> have any strong reasons why they think one is much better than the
> others?

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> That isn't what I meant to say :). The API _is_ orthogonal to the file
> format.

I don't think the API covers everything you'd want yet.
 
> > That said, there's some things missing from the spec:
> >
> > 1) When you read three config files off the path, change a value that was in
> > the second one, and write it out again, where does it get written?
> 
> I tried to answer that in requirement 10, but I can see it isn't very
> clear.
> 
> The intent is that only one file be writable, and that is where all
> values are written.
> 
> For example, if you open ~/.myconfig (writable), /usr/local/app/config
> (readonly) and /etc/app/config (readonly), then any values you change
> are written to ~/.myconfig.

What about values you don't change? What about values that you change back
to match what's in /usr/local/app/config? What about values you don't change
but which came from ~/.myconfig?

There's no obvious place I saw in the .ADS that shows multiple file names
being offered for reading, such that they'd be linked in any way.

> Yes, but any values the user changes will be written to the local
> file. Same example as above.

OK. That isn't clear from the docs. At least not to me.
 
> > 3) If you enable #2, and then the user changes her email address,
> > does it also store back the mail server address, thereby preventing
> > the administrator from changing it?
> 
> Hmm. I was going to say "no, the user did not change that value". That
> is my intent. But my draft implementation does not follow that intent;
> it writes all the values to the writeable (local) file.
> 
> I'll declare that implementation broken, in this respect. I guess I
> need a "value changed since read" marker, and only changed values get
> written.

Still not good enough. If I change the user email address this session,
write it out, then change the window position next time, I don't want to
lose the email address when I write it.

I think you need to keep track, for each value, what file it was read from
and whether it changed. Then when you write it out, write out any values
that have changed as well as any values that originally came from the file
you're rewriting.

> Hmm. Now I think we need the queries "what file was this value read from"
> and "what file will this value be written to".

Something close to that, yes. :)

Oh. One other question. How often are the config files read? If someone
changes a config file while I'm running, do I see the change reflected? What
if I run two copies of the app at once, one of which I use to configure a
bunch of stuff, then write it back out, and the other which I move a window
but don't change anything else. Does it reread the "changed" file, make the
updates, and write it back out? 

I think if you're not careful, there's going to be very unintuitive
behavior.

> but not in the visible part of the spec). Since the same effect can be
> accomplished via numbered keys, I don't think it's worth pursuing at
> this time.

Fair enough.
 
> Hmm, I'm wrong. There is a requirement to return "a list of keys". I
> guess that will be a list of strings. When I implement that, I'll
> think about this issue again :).

I'd think an unconstrained array of strings would be best there. Of course,
if it's part of Grace, you have th elists.
 
> Once again, thanks for your insightful comments.

Hey, it's fun. :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Darren New]

"Robert I. Eachus" wrote:
> Yes.  But the problem is not with the various file formats, it is with
> the approach.  At this point we should be debating a set of requirements
> and a strawman interface package specification.  If the result favors
> one or the other file format, so be it.

Seconded! That's what I've been saying. Work out an API first, and the file
format will pretty much either fall out of it or be arbitrary where it
doesn't.

>  My expectation is that a good
> set of requirements and interface will allow more than one underlying
> implementation.  I'd love to see a result that maps to registry, .ini
> files, and .cfg files with no differences in usage outside the create
> and/or open parameters.

Seconded again.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Stephen Leake]

dennison@telepath.com (Ted Dennison) writes:

> I hate to do this, but I have to go on record as being strongly
> against voting for things. Everything in Grace needs to be decided by
> general consensus. The only reason there isn't an official policy on
> this at the Grace website is because I haven't had time to put it up.
> 
> I'm not saying everyone has to agree that one of these styles is the
> best of all possible styles. But there should be some kind of general
> agreement that one of them is meritous and acceptable, if not
> everyone's perfect option. If we get down to two that seem to be
> equally good with no huge contraversy between them, then perhaps a
> vote or coin flip or executive decision or something similar would be
> in order.

I think that's where we are. I guess I could have said "I'd like to
reach consensus on this". But I don't know how to actually do that in
a newsgroup forum. In person, I can go around the room and ask "do you
have anything else to add".

Asking for a vote is pretty simple, and most people know what it means.

> BTW: I have recently appointed Stephen to lead the config file
> effort for Grace (see http://savannah.gnu.org/projects/grace/ ). So
> if there were an "executive decision" to be made in the context of
> the Grace project, it would be up to him to make it.

Yes, I guess I never said that here. Thanks.

> This is a good example of why we need to go with this approach. We
> may now have consensus against using XML, but there really hasn't
> been much discussion here on the relative merits of the other
> approaches (other than that some folks think INI's should be reseved
> for Windows). If I were to just vote now between the other 3
> options, it would be based on looks. Another thread here has
> recently illustrated quite well that this would be quite succeptable
> to parrochial biases. :-)

Ok, I'll accept that.

> So really a better question right now would be to ask if anyone has
> any big objections to any of the approaches presented (other than
> XML, to which the objections are already well recorded). Also, does
> anyone have any strong reasons why they think one is much better
> than the others?

Ok. Good question. I guess I'm learning how to do consensus on a
newsgroup.

I went with the choice between Java and XML because there was
definitely consensus on multi-level keys (rules out ini), and I felt
there was sufficient complaint about "complex syntax" to rule out X
style.

Note that I did ask people to vote for any of the file formats.

If we end up with an overwhelming majority for one format, I'll take
that as consensus. If it's close, or there are only a few votes, I'll
be back :).

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

"Robert I. Eachus" <rieachus@attbi.com> writes:

> Ted Dennison wrote:
> 
>  
> > So really a better question right now would be to ask if anyone has
> > any big objections to any of the approaches presented (other than XML,
> > to which the objections are already well recorded). Also, does anyone
> > have any strong reasons why they think one is much better than the
> > others?
> 
> 
> Yes.  But the problem is not with the various file formats, it is with
> the approach.  At this point we should be debating a set of
> requirements and a strawman interface package specification. 

Well, I posted those a couple weeks ago, so I thought we did that. It
did not stimulate much discussion of the specific requirements or
package spec; all of the discussion was on the file format.

> If the result favors one or the other file format, so be it. 

It certainly went against XML. The choice among the other three is
less clear.

Do you find the rationale I've posted at
http://users.erols.com/leakstan/Stephe/Ada/Config_Files/config_files.html
to be insufficient? What's missing?

> My expectation is that a good set of requirements and interface will
> allow more than one underlying implementation. 

Yes. But we agreed at the very start that a standard file format was
required.

> I'd love to see a result that maps to registry, .ini files, and .cfg
> files with no differences in usage outside the create and/or open
> parameters.

Ok, but that's explicitly outside the scope of the Config_Files
project.

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> "Robert I. Eachus" wrote:
> > Yes.  But the problem is not with the various file formats, it is with
> > the approach.  At this point we should be debating a set of requirements
> > and a strawman interface package specification.  If the result favors
> > one or the other file format, so be it.
> 
> Seconded! That's what I've been saying. Work out an API first, and the file
> format will pretty much either fall out of it or be arbitrary where it
> doesn't.

Since we keep saying the API _must_ be independent of the file format,
I don't see how finishing the API can help decide what file format to
use.

Note that Robert said "requirements and package spec", not just
"package spec".

> >  My expectation is that a good set of requirements and interface
> > will allow more than one underlying implementation. I'd love to
> > see a result that maps to registry, .ini files, and .cfg files
> > with no differences in usage outside the create and/or open
> > parameters.
> 
> Seconded again.

Well, I thought we did that. Do I need to wait longer than "a couple
weeks"? 

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > That isn't what I meant to say :). The API _is_ orthogonal to the file
> > format.
> 
> I don't think the API covers everything you'd want yet.

Ok. Do you think that prevents us from agreeing on which file format
to go forward with?

It's a pain maintaining two example implementations. The stuff we are
talking about here is totally independent of file format.

> What about values you don't change? 

As I said below, only values changed since being read from the config
file will be written; unchanged values are not written.

> What about values that you change back to match what's in
> /usr/local/app/config? 

Well, I guess we could provide an "undo" command. I guess "undo" can
simply delete the value from the writeable tree, which will expose the
value in the read-only tree (assuming there is one).

Perhaps also a total "erase", which would discard all changes.

> What about values you don't change but which came from ~/.myconfig?

Hmm. Good point. I guess all values from a writeable file must be
written, not just changed ones.

> There's no obvious place I saw in the .ADS that shows multiple file
> names being offered for reading, such that they'd be linked in any
> way.

Yes, that is not implemented.

> > Yes, but any values the user changes will be written to the local
> > file. Same example as above.
> 
> OK. That isn't clear from the docs. At least not to me.

Right, it needs to be added.

> > > 3) If you enable #2, and then the user changes her email
> > > address, does it also store back the mail server address,
> > > thereby preventing the administrator from changing it?
> > 
> > Hmm. I was going to say "no, the user did not change that value". That
> > is my intent. But my draft implementation does not follow that intent;
> > it writes all the values to the writeable (local) file.
> > 
> > I'll declare that implementation broken, in this respect. I guess I
> > need a "value changed since read" marker, and only changed values get
> > written.
> 
> Still not good enough. If I change the user email address this session,
> write it out, then change the window position next time, I don't want to
> lose the email address when I write it.
> 
> I think you need to keep track, for each value, what file it was read from
> and whether it changed. Then when you write it out, write out any values
> that have changed as well as any values that originally came from the file
> you're rewriting.

Yes.

> Oh. One other question. How often are the config files read? 

When you call "open".

> If someone changes a config file while I'm running, do I see the
> change reflected? 

We explicitly rule out multiple parallel processes accessing the same
files. So that's erroneous usage, and you may format your hard disk :).

> What if I run two copies of the app at once, one of which I use to
> configure a bunch of stuff, then write it back out, and the other
> which I move a window but don't change anything else. Does it reread
> the "changed" file, make the updates, and write it back out?

Don't do that.

Seriously, you've set up a race condition, which is an indication of
bad design. If you want multiple instances of your app running, each
with its own persistent window positions, you need multiple config
files. That is precisely one of my use cases, and why I argued for
specifying the config file on the app command line.

Or, if you really want one value shared among processes, use the
registry, since it is designed for that. 

> I think if you're not careful, there's going to be very unintuitive
> behavior.

Which is why we ruled out multiple processes from the beginning.

> > Hmm, I'm wrong. There is a requirement to return "a list of keys". I
> > guess that will be a list of strings. When I implement that, I'll
> > think about this issue again :).
> 
> I'd think an unconstrained array of strings would be best there. Of course,
> if it's part of Grace, you have th elists.

Ah. I guess I can rely on other Grace components. Good point.

> > Once again, thanks for your insightful comments.
> 
> Hey, it's fun. :-)

Less fun from this end :). But still Ok.

Let me ask again; can we choose the general file format now, or do you
think we need to work on the API more first?

-- 
-- Stephe

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> Ok. Do you think that prevents us from agreeing on which file format
> to go forward with?

I'm not sure why you're implementing two bodies while the spec is still in
flux. :-)  Either file format you want to support is OK by me.
 
> > What about values you don't change?
> 
> As I said below, only values changed since being read from the config
> file will be written; unchanged values are not written.

Then you're screwed. If I load 20 values out of a config file, change one,
and save it, then I'll lose 19 config values?
 
> > What about values that you change back to match what's in
> > /usr/local/app/config?
> 
> Well, I guess we could provide an "undo" command. I guess "undo" can
> simply delete the value from the writeable tree, which will expose the
> value in the read-only tree (assuming there is one).

I don't think that's quite what I meant. I meant that default values
probably shouldn't be stored, even if you got them by assigning the default
to the key.

I think until you (we) work out the semantics of what's going on,
implementing multiple bodies is relatively unhelpful to progressing the
spec. Of course, if you're enjoying it and all, that's something else. :-)

> Seriously, you've set up a race condition, which is an indication of
> bad design. If you want multiple instances of your app running, each
> with its own persistent window positions, you need multiple config
> files. That is precisely one of my use cases, and why I argued for
> specifying the config file on the app command line.

That's not what I'm saying.

I'm saying I open my text editor for file XYZ.ads, and another for XYZ.adb,
and then I move XYZ.adb off to the side. Then, in XYZ.ads, I configure all
kinds of things to do the proper indenting for Ada, and then I close
XYZ.ads. Since I want XYZ.adb to see these changes, I close that, and reopen
it. Unfortunately, XYZ.adb didn't reload any changes I made in the indenting
configuration section, so all that work's gone.

Bad news.

Relatively easy to fix - when a config file is saved, it is first read off
the disk, then any values in the in-memory configuration that differ from
the just-loaded configuration and which have been changed since they were
loaded (or since the last save?) are copied to the just-loaded
configuration, then the just-loaded configuration is written back out.

> > I think if you're not careful, there's going to be very unintuitive
> > behavior.
> 
> Which is why we ruled out multiple processes from the beginning.

OK. I'd misinterpreted, thinking you meant only one process could read or
write at a time, rather than only one process could be using a configuration
at a time.
 
> Let me ask again; can we choose the general file format now, or do you
> think we need to work on the API more first?

I think you can pick the file format, sure. If you have "heirarchy of keys
mapped to string values", I don't see much more you need to know for that.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Georg Bauhaus]

Darren New <dnew@san.rr.com> wrote:
: "Robert I. Eachus" wrote:
:> Yes.  But the problem is not with the various file formats, it is with
:> the approach.  At this point we should be debating a set of requirements
:> and a strawman interface package specification.  If the result favors
:> one or the other file format, so be it.
: 
: Seconded! That's what I've been saying. Work out an API first, and the file
: format will pretty much either fall out of it or be arbitrary where it
: doesn't.

O.K., and I say, for the last time, that this consensus rules out
any configuration process requirements that are not as
close to computers as interfaces and file formats.
I'm hoping and confident that Ada applications are configured by people
competent enough to do well with properties files.
As Stephen has said, don't use Config_Files otherwise.

Cheers,
 Georg

Re: Config_Files proposal

[Ted Dennison]

Stephen Leake wrote:
> Ok. Good question. I guess I'm learning how to do consensus on a
> newsgroup.

With the Lists stuff, often I would just pick a choice, start arguing 
its merits, and wait to see if anyone screamed. If no one complains: 
bingo! :-)

You should probably at least post the merits and drawback of each 
approach, as you see it. Presumably others will do the same if they 
think you missed something.

> If we end up with an overwhelming majority for one format, I'll take
> that as consensus. If it's close, or there are only a few votes, I'll
> be back :).

The problem with a vote is there is no real weight on it, and no good 
idea of the rationale behind the numbers. The nightmare scenario is that 
the majority was just going with the flow to get things moving, while 
the minority voting against it had a damn good reason.

The other issue with getting consensus is that we want to make sure we 
don't leave %49 or %32 of everyone behind every time we make a decision. 
We need as near to a universal buy-in as we can get.

Re: Config_Files proposal

[Ted Dennison]

Stephen Leake wrote:

> I went with the choice between Java and XML because there was
> definitely consensus on multi-level keys (rules out ini), and I felt
> there was sufficient complaint about "complex syntax" to rule out X
> style.

That's a good point. The X style does look like a real pain to parse. 
It'd probably be nearly as tough as the XML style to explain over the 
phone too.

Re: Config_Files proposal

[Georg Bauhaus]

Marin David Condic <dont.bother.mcondic.auntie.spam@[acm.org> wrote:

[...] Yes, I'll second that! (A configuration process in a multi-
company setting with people from a very heterogenous set of professions
might profit from a standard that was inspired by data representation
issues for more than 30 years.)

: I'd like to see an OO
: implementation of this using tagged records and closely following the W3C
: DOM model, despite the very nice non-OO implementation done by XML/Ada.

From what I can see, there isn't too much to be done, since the OO
approach is already reflected in discriminated subtypes and corresponding
packages. DOM2 support seems quite complete, the IDL interfaces have
been mostly implemented in XML/Ada.

: There's my $0.02 on it. :-)
Add EUR 0.02. :-)
 
-- Georg

Re: Config_Files proposal

[Marin David Condic]

Well, if, for example, the decisions for the API lead to some kind of
complex tree-like structure, you might have a really hard time storing this
in some simpler file format like .CSV or as simple attribute/value pairs.

I suggested elsewhere that establishing a package spec now and leaving the
file format for later would be a wise thing. It starts some progress and
helps clarify the problems. Assumimg that the API is going to have something
pretty basic like "Load/Store" operations with a file name and a data
structure, you really don't have to settle the file format issues until
later & it creates the possibility of trying out many different ones.

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com


"Stephen Leake" <stephen.a.leake.1@gsfc.nasa.gov> wrote in message
news:u4rfzw2kx.fsf@gsfc.nasa.gov...
>
> Since we keep saying the API _must_ be independent of the file format,
> I don't see how finishing the API can help decide what file format to
> use.
>

Re: Config_Files proposal

[Marin David Condic]

Its not that the XML/Ada thing is "bad" - just that I would prefer one that
built the whole thing up with tagged records, dispatching, etc., rather than
using discriminated records. Its almost like it is saying "Oh, the DOM
specified by the W3C is very OO and could readily be recognized in *real* OO
languages like C++ or Java - complete with inheritance and dispatching and
all the cool OO stuff, but since Ada isn't *really* an OO language it has to
kind of kludge the DOM together with discriminated records and all that."

That, and I've always felt that discriminated records were a pain in the
posterior. Maybe its just me, but they always seem to cause trouble as you
run into language rules with them that don't let them do what you want to
do. Tagged records just seem like a much more elegant solution.

I've looked over the XML/Ada stuff just enough to know that trying to
umbundle the discriminated records and turn them into tagged records with
inheritance and dispatching and all that, would be something beyond
"trivial". I wouldn't see the point in even attempting to do so unless that
was the direction chosen for the future of the project by the guys who
currently own it. (Would you really want to see two divergent
implementations, or would it be better to have just one?)

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com


"Georg Bauhaus" <sb463ba@l1-hrz.uni-duisburg.de> wrote in message
news:aesjsv$7kv$1@a1-hrz.uni-duisburg.de...
>
> From what I can see, there isn't too much to be done, since the OO
> approach is already reflected in discriminated subtypes and corresponding
> packages. DOM2 support seems quite complete, the IDL interfaces have
> been mostly implemented in XML/Ada.
>

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > Ok. Do you think that prevents us from agreeing on which file format
> > to go forward with?
> 
> I'm not sure why you're implementing two bodies while the spec is still in
> flux. :-)  Either file format you want to support is OK by me.

Fair enough.

> > > What about values you don't change?
> > 
> > As I said below, only values changed since being read from the config
> > file will be written; unchanged values are not written.
> 
> Then you're screwed. If I load 20 values out of a config file, change one,
> and save it, then I'll lose 19 config values?

Yes, this is bad.

> > > What about values that you change back to match what's in
> > > /usr/local/app/config?
> > 
> > Well, I guess we could provide an "undo" command. I guess "undo" can
> > simply delete the value from the writeable tree, which will expose the
> > value in the read-only tree (assuming there is one).
> 
> I don't think that's quite what I meant. I meant that default values
> probably shouldn't be stored, even if you got them by assigning the default
> to the key.

Yes. The default is provided again with every call to "read", so it
does not need to be stored.

> I think until you (we) work out the semantics of what's going on,
> implementing multiple bodies is relatively unhelpful to progressing
> the spec. 

Well, I usually find problems with a spec when I get around to
implementing the body. I did these two implementations to see if there
were any problems with the spec that were related to the file format.
I don't think there are.

> Of course, if you're enjoying it and all, that's something else. :-)

I was trying to eliminate one of the variables in the problem. If we
agree on what the file format is, that's one less thing to think about.

I think we are ready to do that, and it will allow people to focus
solely on the spec, rather than on the XML vs ini etc.

> > Seriously, you've set up a race condition, which is an indication
> > of bad design. If you want multiple instances of your app running,
> > each with its own persistent window positions, you need multiple
> > config files. That is precisely one of my use cases, and why I
> > argued for specifying the config file on the app command line.
> 
> That's not what I'm saying.
> 
> I'm saying I open my text editor for file XYZ.ads, and another for XYZ.adb,
> and then I move XYZ.adb off to the side. Then, in XYZ.ads, I configure all
> kinds of things to do the proper indenting for Ada, and then I close
> XYZ.ads. Since I want XYZ.adb to see these changes, I close that, and reopen
> it. Unfortunately, XYZ.adb didn't reload any changes I made in the indenting
> configuration section, so all that work's gone.

Let's see. You have two instances of "text_edit", sharing one config
file. That is explicitly forbidden by requirement 1.

So if you want this behavior, you have to use the registry or gconf,
not Config_Files.

> Bad news.
> 
> Relatively easy to fix - when a config file is saved, it is first read off
> the disk, then any values in the in-memory configuration that differ from
> the just-loaded configuration and which have been changed since they were
> loaded (or since the last save?) are copied to the just-loaded
> configuration, then the just-loaded configuration is written back out.

This is a race condition. Suppose two processes happen to start a
"flush" operation "at the same time"? Whichever goes last wins.

That's why we have requirement 1. Config_Files don't do this.

If you feel that makes Config_Files totally useless, I guess we can
talk about removing requirement 1 and see if we can come up with
locking semantics to fix it. But I'd rather not, especially since the
registry and gconf have already done that.

In general, I suggest that there are other ways to fix an apparent
need for concurrent access to config files.

For the specific case of a text editor, you can use a text editor that
supports multiple documents; that provides automatic sharing of config
file settings between XYZ.adb and XYZ.ads.

> > > I think if you're not careful, there's going to be very
> > > unintuitive behavior.
> > 
> > Which is why we ruled out multiple processes from the beginning.
> 
> OK. I'd misinterpreted, thinking you meant only one process could read or
> write at a time, rather than only one process could be using a configuration
> at a time.

Ok, I guess I need to make that clearer. I thought one implied the
other :). Your example is more complex than I had in mind; I'll try to
include something like it in the final rationale document.

> > Let me ask again; can we choose the general file format now, or do
> > you think we need to work on the API more first?
> 
> I think you can pick the file format, sure. If you have "heirarchy of keys
> mapped to string values", I don't see much more you need to know for that.

But I want _you_ to pick a format, so we can get consensus. Hmm. You said
above either is ok; I guess that counts as consensus for the Java
property format.

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Ted Dennison <dennison@telepath.com> writes:

> Stephen Leake wrote:
> > Ok. Good question. I guess I'm learning how to do consensus on a
> > newsgroup.
> 
> With the Lists stuff, often I would just pick a choice, start arguing
> its merits, and wait to see if anyone screamed. If no one complains:
> bingo! :-)

That approach does have a certain appeal.

> You should probably at least post the merits and drawback of each
> approach, as you see it. Presumably others will do the same if they
> think you missed something.

Hmm. I did that on the web page. By "post", do you mean here on
comp.lang.ada? 

> > If we end up with an overwhelming majority for one format, I'll
> > take that as consensus. If it's close, or there are only a few
> > votes, I'll be back :).
> 
> The problem with a vote is there is no real weight on it, and no good
> idea of the rationale behind the numbers. The nightmare scenario is
> that the majority was just going with the flow to get things moving,
> while the minority voting against it had a damn good reason.

Yes. However, that's why I posted the rationale (or at least the start
of one). I hoped that the ensuing discussion would bring out all the
"good reasons". I wanted a vote to put a specific end to the
discussion; sometimes that motivates people to respond. But I could
just as easily have said "I will pick a format on Tuesday; post your
comments by then".

> The other issue with getting consensus is that we want to make sure
> we don't leave %49 or %32 of everyone behind every time we make a
> decision. We need as near to a universal buy-in as we can get.

Well, we could define a vote to be successful only if 80% agree.
Either way, we are just counting people's opinions.

My impression so far is that there is strong sentiment against XML as
being too complex to hand-edit. And there is strong sentiment against
ini because it doesn't have multi-level keys. Other than that, nobody
cares much. So it probably is easier for me to say "I'm picking Java
property format, unless anyone has serious objections".

That also avoids the issue of whether we get a quorum in the voting
process. 

Thanks for putting up with my learning curve :).

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:

> XML is significantly better than some of the other formats suggested here
> for one factor: Other tools will be able to make use of it. If it comes to
> displaying an initialization file or using the data in any other way, you
> gain the ability to make them work in entirely new contexts without having
> to do some form of data conversion.

True.

> All other things being equal, gaining an XML parser in the process
> would be A Good Thing. 

Yes, but I think you'll agree that all other things are not equal
today. And since I am _not_ proposing to write an XML parser, but only
to reuse an existing one, we are not "gaining" anything; those parsers
are already there.

> Imagine if one day when everything is sunshine and rainbows and
> pastel unicorns and Ada has some kind of standard means of
> displaying web page stuff, your initialization file XML data slides
> right in there. 

XML is not "web page stuff"; Netscape doesn't know how to display it.
It has to be processed into HTML first (unless I am really missing
something). It's just as easy to process a Config_File into HTML. If
the Config_File standard catches on, it would be reasonable to define
and implement a standard translation to XML and HTML and whatever else
comes along.

> Imagine that once the tools are there to handle XML on input and
> output how many other things might be built on top of Grace that
> could leverage this capability.

Yes, that would be nice. But it is way outside the scope of the
Config_File mandate.

Even if we had better XML tools in Grace today, I think I would not
use XML for Config_Files. It just doesn't meet the requirements;
hand-editing is important, and too hard in XML.

> IOW, use this as an excuse to build an XML DOM & SAX. (I'd like to
> see an OO implementation of this using tagged records and closely
> following the W3C DOM model, despite the very nice non-OO
> implementation done by XML/Ada.)

I don't think we need an excuse for that :).

-- 
-- Stephe

Re: Config_Files proposal

[Ted Dennison]

"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> wrote in message news:<aeqcl6$sgg$1@nh.pace.co.uk>...
> All other things being equal, gaining an XML parser in the process would be
> A Good Thing. Imagine if one day when everything is sunshine and rainbows

Won't happen. At best you'd get something that can only parse the
limited syntax that the config file happens to use.

If folks really want an XML parser in the language, that should be a
separate effort. Perhaps the one Gnat developed could be used, now
that the license has changed?

If not, I don't see any hope of putting XML in the language either.
The effort would just be too much. XML/Ada hasn't gotten to 1.0 yet
after more than a year of effort, and I believe that effort has been
supported by ACT (or ACT-Europe?)

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> Let's see. You have two instances of "text_edit", sharing one config
> file. That is explicitly forbidden by requirement 1.

Then requirement one should speak of concurrent use, rather than
simultaneous use. :-)

> This is a race condition. Suppose two processes happen to start a
> "flush" operation "at the same time"? Whichever goes last wins.

That would be simultaneous use. ;-)
 
> If you feel that makes Config_Files totally useless, I guess we can
> talk about removing requirement 1 and see if we can come up with
> locking semantics to fix it. But I'd rather not, especially since the
> registry and gconf have already done that.

I don't know that we need locking semantics. I just thought it would be easy
to apply the changes and merge them appropriately. For that matter, you're
going to have a similar problem if you have two threads in the same program
doing this.

Certainly it at least has to be made clear in the specs.

> > OK. I'd misinterpreted, thinking you meant only one process could read or
> > write at a time, rather than only one process could be using a configuration
> > at a time.
> 
> Ok, I guess I need to make that clearer. I thought one implied the
> other :). Your example is more complex than I had in mind; I'll try to
> include something like it in the final rationale document.

OK. I'm just trying to have ideas about problems before anyone puts a lot of
work into something, only to find out that the end-users find it too
unintuitive to use.

Something like "list API" is one thing, since only programmers who read the
instructions are going to see that. Something like a config file where the
user can set configurations is another thing. Perhaps it's worth having an
option to create a lock file of some sort.

> But I want _you_ to pick a format, so we can get consensus. Hmm. You said
> above either is ok; I guess that counts as consensus for the Java
> property format.

Yes. I've been pushing towards something where there are minimal syntactic
interdependencies between keys, rather than something like XML. I'm more
concerned about precise semantics than about the niggly details of the file
format, tho.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Jacob Sparre Andersen]

Darren New wrote:
> 
> Stephen Leake wrote:
> > Let's see. You have two instances of "text_edit", sharing one config
> > file. That is explicitly forbidden by requirement 1.
> 
> Then requirement one should speak of concurrent use, rather than
> simultaneous use. :-)

This discussion made me think of how my e-mail client works
(typically, I have five running instances of it). It somehow
keeps an eye on changes to the configuration file, so if one
instance has saved a changed configuration, the other
instances will (when they happen to feel like looking at the
configuration file) complain, and ask if I want to use the
in-memory or the disk configuration.

I don't know how trivial the implementation of this feature
is, but just keeping a copy of the time stamp of the file
with the in-memory configuration would help a bit, although
one also could use a file check-sum.

Jacob
-- 
People in cars cause accidents. Accidents in cars cause
people.

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > Let's see. You have two instances of "text_edit", sharing one config
> > file. That is explicitly forbidden by requirement 1.
> 
> Then requirement one should speak of concurrent use, rather than
> simultaneous use. :-)

Yes, that is better.

> > This is a race condition. Suppose two processes happen to start a
> > "flush" operation "at the same time"? Whichever goes last wins.
> 
> That would be simultaneous use. ;-)

Ok, neither simultaneous nor concurrent use is supported.

> > If you feel that makes Config_Files totally useless, I guess we
> > can talk about removing requirement 1 and see if we can come up
> > with locking semantics to fix it. But I'd rather not, especially
> > since the registry and gconf have already done that.
> 
> I don't know that we need locking semantics. I just thought it would be easy
> to apply the changes and merge them appropriately. For that matter, you're
> going to have a similar problem if you have two threads in the same program
> doing this.

Yes, but Ada programmers are supposed to be aware of things like that.
I guess it wouldn't hurt to mention it.

> Certainly it at least has to be made clear in the specs.

yes.

> > > OK. I'd misinterpreted, thinking you meant only one process
> > > could read or write at a time, rather than only one process
> > > could be using a configuration at a time.
> > 
> > Ok, I guess I need to make that clearer. I thought one implied the
> > other :). Your example is more complex than I had in mind; I'll try to
> > include something like it in the final rationale document.
> 
> OK. I'm just trying to have ideas about problems before anyone puts a lot of
> work into something, only to find out that the end-users find it too
> unintuitive to use.

Yes, and I do appreciate that.

> Something like "list API" is one thing, since only programmers who
> read the instructions are going to see that. Something like a config
> file where the user can set configurations is another thing. Perhaps
> it's worth having an option to create a lock file of some sort.

Only if we can define it well.

> > But I want _you_ to pick a format, so we can get consensus. Hmm.
> > You said above either is ok; I guess that counts as consensus for
> > the Java property format.
> 
> Yes. I've been pushing towards something where there are minimal syntactic
> interdependencies between keys, rather than something like XML. I'm more
> concerned about precise semantics than about the niggly details of the file
> format, tho.

Ok, I'll put you on the Java side.

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Jacob Sparre Andersen <sparre@nbi.dk> writes:

> Darren New wrote:
> > 
> > Stephen Leake wrote:
> > > Let's see. You have two instances of "text_edit", sharing one config
> > > file. That is explicitly forbidden by requirement 1.
> > 
> > Then requirement one should speak of concurrent use, rather than
> > simultaneous use. :-)
> 
> This discussion made me think of how my e-mail client works
> (typically, I have five running instances of it). It somehow
> keeps an eye on changes to the configuration file, so if one
> instance has saved a changed configuration, the other
> instances will (when they happen to feel like looking at the
> configuration file) complain, and ask if I want to use the
> in-memory or the disk configuration.
> 
> I don't know how trivial the implementation of this feature
> is, but just keeping a copy of the time stamp of the file
> with the in-memory configuration would help a bit, although
> one also could use a file check-sum.

Yes, there are ways to do it. 

I think we have three choices here:

1) Simply forbid simultaneous and/or concurrent access to a single
   config file (as the requirements currently state).

2) Allow simultaneous and/or concurrent access, and define the
   behavior to be similar to Jacob's email program.

3) Allow full concurrent access to keys, the way the registry does.

I chose 1) originally to keep things simple, and because it meets my
needs.

I think we should avoid 3), because it's way too hard (you get into
"change notification messages"), and because the registry and gconf
already do this nicely.

I could go for 2), if enough people think it is important.

Or we could agree to do 2) later, after everything else.

-- 
-- Stephe

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> 1) Simply forbid simultaneous and/or concurrent access to a single
>    config file (as the requirements currently state).

I think "forbid simultaneous access" is fine. But that's not what you're 
doing. You're doing "screw the user if there's simultaneous access." Kind of 
the way C "forbids" indexing off the end of an array. ;-)

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > 1) Simply forbid simultaneous and/or concurrent access to a single
> >    config file (as the requirements currently state).
> 
> I think "forbid simultaneous access" is fine. But that's not what
> you're doing. You're doing "screw the user if there's simultaneous
> access." Kind of the way C "forbids" indexing off the end of an array.
> ;-)

Well, that's true; I'm only "forbidding" in the requirements and the
user guide, not in the code. But what else can I do, without assuming
more about what the operating system can do?

I'd like to use either Ada.Text_IO, or some Gnat package to do all the
file management stuff. I don't think there is any support for locking
in there, but I'll admit I haven't really looked.

It would be nice if we only use Ada.Text_IO, because then it will be
more portable to other compilers.

-- 
-- Stephe

Re: Config_Files proposal

[Jacob Sparre Andersen]

Stephen Leake wrote:

> Ted Dennison <dennison@telepath.com> writes:

> > You should probably at least post the merits and drawback of each
> > approach, as you see it. Presumably others will do the same if they
> > think you missed something.
> 
> Hmm. I did that on the web page. By "post", do you mean here on
> comp.lang.ada?

I don't know what _Ted_ meant, but it is my experience that
you get much more discussion going, if you post the text on
the newsgroup, so it is trivial for people to quote and
comment it.

Jacob
-- 
"Navier-Stokes-ligningen er som Biblen,
 det er randbetingelserne der bestemmer det hele."

Re: Config_Files proposal

[Stephen Leake]

Jacob Sparre Andersen <sparre@nbi.dk> writes:

> Stephen Leake wrote:
> 
> > Ted Dennison <dennison@telepath.com> writes:
> 
> > > You should probably at least post the merits and drawback of each
> > > approach, as you see it. Presumably others will do the same if they
> > > think you missed something.
> > 
> > Hmm. I did that on the web page. By "post", do you mean here on
> > comp.lang.ada?
> 
> I don't know what _Ted_ meant, but it is my experience that
> you get much more discussion going, if you post the text on
> the newsgroup, so it is trivial for people to quote and
> comment it.

Ok. But when the text gets long enough, I find it more awkward. Maybe
my threshold for "long enough" is too low :).

-- 
-- Stephe

Re: Config_Files proposal

[Ted Dennison]

Darren New wrote:
> I think "forbid simultaneous access" is fine. But that's not what you're 
> doing. You're doing "screw the user if there's simultaneous access." 
> Kind of the way C "forbids" indexing off the end of an array. ;-)

I'm probably missing something here. How is this worse than the current 
situation with Ada.Text_IO?

Re: Config_Files proposal

[Ted Dennison]

Jacob Sparre Andersen wrote:
> Stephen Leake wrote:
> 
>>Ted Dennison <dennison@telepath.com> writes:
>>>You should probably at least post the merits and drawback of each
>>>approach, as you see it. Presumably others will do the same if they
>>>think you missed something.
>>
>>Hmm. I did that on the web page. By "post", do you mean here on
>>comp.lang.ada?
> 
> I don't know what _Ted_ meant, but it is my experience that
> you get much more discussion going, if you post the text on
> the newsgroup, so it is trivial for people to quote and
> comment it.


Quite true. However, I'll admit I missed that on the website, which was 
my bad. After looking it over more carefully, I also have to admit I 
have nothing to add to it (which I guess leads me towards the same 
conclusion Stephen has been getting to).

Re: Config_Files proposal

[Marin David Condic]

I'd like to understand how to do some form of file lock-out for simultaneous
access by different processes without immediately getting into some kind of
OS dependant code. I can imagine a variety of semaphore schemes that will
depend on the OS or at least semi-portable facilities, but if the objective
is to to it strictly within the bounds of Ada, I can't think of a way to do
that.

If the goal is pure, portable, run-anywhere Ada code, I'm betting that it
will come down to "Don't do simultaneous access - it would be Bad..."

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com


"Ted Dennison" <dennison@telepath.com> wrote in message
news:3D1289D2.9090107@telepath.com...
> Darren New wrote:
> > I think "forbid simultaneous access" is fine. But that's not what you're
> > doing. You're doing "screw the user if there's simultaneous access."
> > Kind of the way C "forbids" indexing off the end of an array. ;-)
>
> I'm probably missing something here. How is this worse than the current
> situation with Ada.Text_IO?
>

Re: Config_Files proposal

[Darren New]

Ted Dennison wrote:
> Darren New wrote:
> 
>> I think "forbid simultaneous access" is fine. But that's not what 
>> you're doing. You're doing "screw the user if there's simultaneous 
>> access." Kind of the way C "forbids" indexing off the end of an array. 
>> ;-)
> 
> 
> I'm probably missing something here. How is this worse than the current 
> situation with Ada.Text_IO?

If I have a text editor program and a compiler,  I can save the source from 
my editor window (using Text_IO to write it) and then compile it with the 
compiler (using Text_IO to read it). This works like the user expects.

What the user doesn't expect is to run two copies of the text editor, change 
the configuration in one, and have that configuration disappear simply 
because they closed the other window later.

Re: Config_Files proposal

[Robert I. Eachus]

Stephen Leake wrote:


> Well, I posted those a couple weeks ago, so I thought we did that. It
> did not stimulate much discussion of the specific requirements or
> package spec; all of the discussion was on the file format.


Ah, what was missing was not your draft requirements and interface, what 
was missing was the debate about the requirements.  That is finally 
starting to show up, so let me weigh in on the concurrent access issue. ;-)

There are really two separate sets of concerns here.  First, an 
application which expects users to edit the config file implicitly 
during program installation.  Later edits will be with a text editor, or 
a separate program that changes the config data.  For such uses, the 
semantics are pretty simple.  The program checks a potentially multiple 
set of config at initialization time, changes nothing, and edits while 
the program is running can either be left implementation dependent, or 
defined to be not seen.

A program which is going to change its config attributes during 
operation is a horse of a different color.  As I see it, the three 
choices are:

1) Opening the config for writing means that other instances of the 
program, or other programs that use the same file must either die with 
some reasonable behavior, or connect somehow to the first running 
instance.  Defining how to do the latter is well beyond the scope here.

2)  A program which can change the config file is required to follow a 
standard protocol. An example would be:

     a) If the config data is changed, the program uses a locking 
protocol of some sort to insure that there are no race conditions.  (For 
example, under Windows opening the file before the change works.)  The 
important thing from the specification view here is that the attempt to 
change a value or values can fail.  Whether the user view is an explicit 
  transaction protocol, or a multiple change commit protocol that can 
raise an exception is a detailed design decision.  (Although I favor the 
latter, with an exception occurance string that should be meaningful to 
the actual end user.)

     b) If the config file is changed by some other program or some 
other instance, the program will check for changes at defined points. 
Of course, these points will be defined by the program, but a good 
starting point would be to always check for changes before creating a 
new window or a new file, and of course before changing any config 
parameters.

     c)  If the program "rolls up" a set of config data from several 
locations, all values inherited from files other than the last will not 
be checked for changes while the program is running.  If you have 27 
files in the config path, only the last will be modified, or checked for 
changes while the program is running.

3) Chaos.

Re: Config_Files proposal

[Stephen Leake]

"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:

> Well, if, for example, the decisions for the API lead to some kind of
> complex tree-like structure, you might have a really hard time storing this
> in some simpler file format like .CSV or as simple attribute/value pairs.

But that would violate the requirement that the API _must_ be
independent of the file format, and would therefore be a bad API
decision.

> I suggested elsewhere that establishing a package spec now and
> leaving the file format for later would be a wise thing. It starts
> some progress and helps clarify the problems. 

If I generalize this to "make some decisions now", choosing the file
format now is also wise :).

> Assumimg that the API is going to have something pretty basic like
> "Load/Store" operations with a file name and a data structure, you
> really don't have to settle the file format issues until later & it
> creates the possibility of trying out many different ones.

Since we need to settle on one file format, and we've had about a
month of discussion on it, what more needs to be done to pick a file
format? 

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

R> Ted Dennison wrote:
> > Darren New wrote:
> >
> >> I think "forbid simultaneous access" is fine. But that's not what
> >> you're doing. You're doing "screw the user if there's simultaneous
> >> access." Kind of the way C "forbids" indexing off the end of an
> >> array. ;-)
> > I'm probably missing something here. How is this worse than the
> > current situation with Ada.Text_IO?
> 
> If I have a text editor program and a compiler,  I can save the source
> from my editor window (using Text_IO to write it) and then compile it
> with the compiler (using Text_IO to read it). This works like the user
> expects.
> 
> What the user doesn't expect is to run two copies of the text editor,
> change the configuration in one, and have that configuration disappear
> simply because they closed the other window later.

That's what happens when you run two copies of the text editor, change
the same source file in both, and save from both. One set of changes
is lost.

Why should config files be different?

Hmm, one answer is "because the config file is hidden". That makes
sense; we can expect the user to be aware of loading the same source
file into to copies of the text editor, but not immediately aware of
sharing one config file between the two copies of the text editor.

But I still think the right answer in this case is one of:

"use a better (multi-document) text editor".

"use the registry for resources shared by multiple processes".

"use separate config files for separate instances of applications"

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:

> I'd like to understand how to do some form of file lock-out for simultaneous
> access by different processes without immediately getting into some kind of
> OS dependant code. 

You can't; file locking is OS dependant.

However, that in itself is ok for Config_Files. There is no
_requirement_ that it be possible to implement Config_Files in an OS
independent way. It would be nice, of course.

> I can imagine a variety of semaphore schemes that will depend on the
> OS or at least semi-portable facilities, but if the objective is to
> to it strictly within the bounds of Ada, I can't think of a way to
> do that.

That is _not_ one of the objectives. The main objective is to produce
something useful; sometimes that requires OS dependence.

> If the goal is pure, portable, run-anywhere Ada code, I'm betting
> that it will come down to "Don't do simultaneous access - it would
> be Bad..."

Not the goal.

My objection to getting into file locking is twofold:

1) It's complex, and hard to get right.

2) The registry already does it, so use that if you need it.

I believe there are many applications for config files that don't need
file locking. But I'm willing to be convinced otherwise. 

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Ted Dennison <dennison@telepath.com> writes:

> Jacob Sparre Andersen wrote:
> > Stephen Leake wrote:
> >
> >>Ted Dennison <dennison@telepath.com> writes:
> >>>You should probably at least post the merits and drawback of each
> >>>approach, as you see it. Presumably others will do the same if they
> >>>think you missed something.
> >>
> >>Hmm. I did that on the web page. By "post", do you mean here on
> >>comp.lang.ada?
> > I don't know what _Ted_ meant, but it is my experience that
> > you get much more discussion going, if you post the text on
> > the newsgroup, so it is trivial for people to quote and
> > comment it.
> 
> 
> Quite true. However, I'll admit I missed that on the website, which
> was my bad. 

Or mine, for making a less-than-wizzy website :). I also realized that
I have not been advertising the web url very often, so here it is
again:

http://users.erols.com/leakstan/Stephe/Ada/Config_Files/config_files.html

> After looking it over more carefully, I also have to admit I have
> nothing to add to it (which I guess leads me towards the same
> conclusion Stephen has been getting to).

Ok, making progress :).

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

"Robert I. Eachus" <rieachus@attbi.com> writes:

> Stephen Leake wrote:
> 
> 
> > Well, I posted those a couple weeks ago, so I thought we did that. It
> > did not stimulate much discussion of the specific requirements or
> > package spec; all of the discussion was on the file format.
> 
> 
> Ah, what was missing was not your draft requirements and interface,
> what was missing was the debate about the requirements.  

Ok, I guess I'm being impatient :).

> That is finally starting to show up, so let me weigh in on the
> concurrent access issue. ;-)
> 
> There are really two separate sets of concerns here.  First, an
> application which expects users to edit the config file implicitly
> during program installation.  Later edits will be with a text editor,
> or a separate program that changes the config data.  For such uses,
> the semantics are pretty simple.  The program checks a potentially
> multiple set of config at initialization time, changes nothing, and
> edits while the program is running can either be left implementation
> dependent, or defined to be not seen.

Yes, that is an important use case that I should capture. 

> A program which is going to change its config attributes during
> operation is a horse of a different color. As I see it, the three
> choices are:
> 
> 1) Opening the config for writing means that other instances of the
> program, or other programs that use the same file must either die with
> some reasonable behavior, or connect somehow to the first running
> instance.  Defining how to do the latter is well beyond the scope here.

Yes. I think that conforms to the current "forbid simultaneous access"
requirement, but adds the requirement for an enforcement mechanism.

I don't think the GNAT library supports file locking of this type, but
it shouldn't be hard to do. It would make the package much safer.

> 2) A program which can change the config file is required to follow
> a standard protocol. An example would be:
> 
>      a) If the config data is changed, the program uses a locking
> protocol of some sort to insure that there are no race conditions.
> (For example, under Windows opening the file before the change works.)

Windows (and I assume other reasonable OSes) supports "open locked for
write", so that no other program can "open for read" while it is
locked. Then we need to define what the second program does when it
attempts the read; fail with an error message? or wait and timeout
with an error message?

> The important thing from the specification view here is that the
> attempt to change a value or values can fail. 

Read can also fail, if the file is locked for write.

> Whether the user view is an explicit transaction protocol, or a
> multiple change commit protocol that can raise an exception is a
> detailed design decision. (Although I favor the latter, with an
> exception occurance string that should be meaningful to the actual
> end user.)

I can see that this would work. I don't want to implement it :).

We could agree to do choice 1) above first, with this protocol as a
later enhancement.

>      b) If the config file is changed by some other program or some
> other instance, the program will check for changes at defined
> points. Of course, these points will be defined by the program, but
> a good starting point would be to always check for changes before
> creating a new window or a new file, and of course before changing
> any config parameters.

Ok, that makes sense. 

One alternative is to have the writer broadcast some sort of message,
which is way more complex. That's what the Windows registry does; I
assume Gnome gconf does something similar.

Without instant notification of changes it will be pretty easy to set
up situations that will confuse the user. But maybe we can live with
it. 

>      c) If the program "rolls up" a set of config data from several
> locations, all values inherited from files other than the last will
> not be checked for changes while the program is running. If you have
> 27 files in the config path, only the last will be modified, or
> checked for changes while the program is running.

Hmm. The current app can only change the "last" config file. But some
other app may have one of the other config files as its "writable"
file, so any of the 27 may change. I don't think we want to ignore
those changes.

> 3) Chaos.

Hmm. Are you saying that simply forbidding access by multiple
processes, without an enforcement mechanism, is a recipe chaos? I
guess that could be true. Users will ignore warnings in the
application documentation, since they won't read it :).

-- 
-- Stephe

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> That's what happens when you run two copies of the text editor, change
> the same source file in both, and save from both. One set of changes
> is lost.

Well, maybe in *your* editors, based on file systems as horrificly bad as
UNIX. ;-)

> Why should config files be different?

Because reading a config file and writing a config file are both invisible
operations, as far as the user is concerned.
 
> Hmm, one answer is "because the config file is hidden". That makes
> sense; we can expect the user to be aware of loading the same source
> file into to copies of the text editor, but not immediately aware of
> sharing one config file between the two copies of the text editor.

Exactly. And there are commands to save the files and to load the files.
 
> But I still think the right answer in this case is one of:
> "use a better (multi-document) text editor".
> "use the registry for resources shared by multiple processes".
> "use separate config files for separate instances of applications"

What's wrong with #4 - merge changes into the most recent config file,
rather than just overwriting it?

How about when you go to save the new config file, you check the
size/date/checksum/contents/whatever against what it was when you loaded it,
and you throw an exception if it's different, *unless* the overwrite flag is
set? Then those who want to merge the changes can do so "manually" in their
code?

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> Since we need to settle on one file format, and we've had about a
> month of discussion on it, what more needs to be done to pick a file
> format?

This is like asking "Well, since we've been arguing for a month about
whether we'll drive or fly on our vacation, what's wrong with picking a
vehicle and then deciding where to go?"

The file format cannot be independent of the API. You just have some very
loose and informal ideas of what the API ought to support, but without
making them concrete, solidifying the file format is premature. 

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > That's what happens when you run two copies of the text editor, change
> > the same source file in both, and save from both. One set of changes
> > is lost.
> 
> Well, maybe in *your* editors, based on file systems as horrificly bad as
> UNIX. ;-)
> 
> > Why should config files be different?
> 
> Because reading a config file and writing a config file are both invisible
> operations, as far as the user is concerned.
>  
> > Hmm, one answer is "because the config file is hidden". That makes
> > sense; we can expect the user to be aware of loading the same source
> > file into to copies of the text editor, but not immediately aware of
> > sharing one config file between the two copies of the text editor.
> 
> Exactly. And there are commands to save the files and to load the files.
>  
> > But I still think the right answer in this case is one of:
> > "use a better (multi-document) text editor".
> > "use the registry for resources shared by multiple processes".
> > "use separate config files for separate instances of applications"
> 
> What's wrong with #4 - merge changes into the most recent config file,
> rather than just overwriting it?

Because you need "real locks", and you need to guard against all the
possible race conditions. The registry already does that.

When _any_ other process may write to the config file at _any_ time,
it is not possible to define "the most recent config file". You have
to lock it to get exclusive access, then merge your changes.

What's wrong with the answers I gave (besides "I don't like them" :)?

> How about when you go to save the new config file, you check the
> size/date/checksum/contents/whatever against what it was when you
> loaded it, and you throw an exception if it's different, *unless*
> the overwrite flag is set? Then those who want to merge the changes
> can do so "manually" in their code?

If we want to support this scenario (multiple processes reading and
writing to shared config file), we need "real locks". We also need
"real notification" so apps know when the config files have changed.
"real locks" is doable on most reasonable OSes, it's just complicated.
"real notification" is much harder. The registry already does both.

In light of Robert Eachus's post, I need to play with implementing at
least an exclusive read/write lock on a config file. When I do that,
I'll see how hard it would be to do the "merge changes when write"
approach. Maybe that will be easy to add.

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > Since we need to settle on one file format, and we've had about a
> > month of discussion on it, what more needs to be done to pick a file
> > format?
> 
> This is like asking "Well, since we've been arguing for a month about
> whether we'll drive or fly on our vacation, what's wrong with picking a
> vehicle and then deciding where to go?"
> 
> The file format cannot be independent of the API. 

Well, ok, that's a definite change. Nobody has said that before, but I
have said the opposite many times.

> You just have some very loose and informal ideas of what the API
> ought to support, but without making them concrete, solidifying the
> file format is premature.

I guess I did not make an explicit requirement on the web page that
the API be independent of the file format, but I thought that was an
accepted principle.

Can we agree that the choice of file format is between Java properties
and XML? Given that, I believe the two example implementations I've
done show that the API is independent of the file format.

Granted the API is not finished, but the things we are still talking
about have no impact on the file format. Or am I wrong about that?

-- 
-- Stephe

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> Because you need "real locks", and you need to guard against all the
> possible race conditions. The registry already does that.

But you can eliminate a large number of race conditions by implementing a
merge. You can change it from "if you start the same program twice, you're
screwed" to "if you click the button on both instances in the length of time
it takes to write the file, you're screwed." I think that's reasonable. You
might not.
 
> What's wrong with the answers I gave (besides "I don't like them" :)?

Well, it's an opinion. 
 
> If we want to support this scenario (multiple processes reading and
> writing to shared config file), we need "real locks".

Sure, there's a race condition if you write the file between the time you
check if it has changed and the time you write it out. I think that's a much
smaller window than the window between the time you start the program and
the time you exit it.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> I guess I did not make an explicit requirement on the web page that
> the API be independent of the file format, but I thought that was an
> accepted principle.

Great. Now, my API wants to store binary, and I don't want to have to give
keys to every value, and I want to be able to store the file as part of a
larger file.

My point is that the file format makes certain assumptions. You're already
assuming heirarchical keys, each with a single textual value, that the order
of keys is independent, etc. You just haven't made that especially concrete.
I think it's a good idea to make it concrete before deciding on the file
format. But since you're doing the work and I'm just complaining, this is
merely a suggestion. :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > Because you need "real locks", and you need to guard against all the
> > possible race conditions. The registry already does that.
> 
> But you can eliminate a large number of race conditions by implementing a
> merge. You can change it from "if you start the same program twice, you're
> screwed" to "if you click the button on both instances in the length of time
> it takes to write the file, you're screwed." I think that's reasonable. 

I see.

> You might not.

Right, I don't. I tend to be very picky; a file locking protocol is
either "right", or it's "wrong"; no room for "mostly right" :).

> > What's wrong with the answers I gave (besides "I don't like them"
> > :)?
> 
> Well, it's an opinion. 

I was hoping for something technical, that would allow judging the
benefit against the cost of doing file locking right.

> > If we want to support this scenario (multiple processes reading
> > and writing to shared config file), we need "real locks".
> 
> Sure, there's a race condition if you write the file between the time you
> check if it has changed and the time you write it out. I think that's a much
> smaller window than the window between the time you start the program and
> the time you exit it.

It is smaller, but I'm willing to bet someone will hit it. For
example, when you shutdown the computer, all the active editor
instances will write to the config file at once. Maybe us "real"
programmers don't leave editors active when we shutdown, but a lot of
people do!

-- 
-- Stephe

Re: Config_Files proposal

[Stephen Leake]

Darren New <dnew@san.rr.com> writes:

> Stephen Leake wrote:
> > I guess I did not make an explicit requirement on the web page that
> > the API be independent of the file format, but I thought that was an
> > accepted principle.
> 
> Great. Now, my API wants to store binary, 

As base64; no problem.

> and I don't want to have to give keys to every value, 

hmm. I guess you mean you want a record or array type accessed by a
single key. I guess I'll agree; that has an impact on the file format.

However, that's not in the list of "things we are talking about". It
contradicts API requirement 21.

> and I want to be able to store the file as part of a larger file.

Say what? If the "larger file" has a sufficient protocol, you can do
this now.

> My point is that the file format makes certain assumptions. You're
> already assuming heirarchical keys, each with a single textual
> value, that the order of keys is independent, etc. You just haven't
> made that especially concrete. 

Well, that's been in the API requirements from the start, and not
seriously challenged.

> I think it's a good idea to make it concrete before deciding on the
> file format. But since you're doing the work and I'm just
> complaining, this is merely a suggestion. :-)

Actually, it is concrete; API requirements 18 thru 21. If you are
suggesting that these need to change, we can talk about that.


I guess I need to update the API requirements list, and show those
that we have consensus on, and those that are still under discussion.

-- 
-- Stephe

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> Right, I don't. I tend to be very picky; a file locking protocol is
> either "right", or it's "wrong"; no room for "mostly right" :).

Avoid Linux, then. Its file locking has starvation problems built into the
kernel. ;-)

> I was hoping for something technical, that would allow judging the
> benefit against the cost of doing file locking right.

Well, there ya have what I can give. :-) By "it's my opinion" I meant "it's
my opinion that it's worth doing the merge." Technically, it's not
especially difficult. Also technically, I guess one could really do this
manually if one wanted to, loading up a second "config" object and doing the
comparison oneself, so I guess I can withdraw my objections. I.e., one can
"code around" the problem, and with the proper iterators built into the
config API (allowing you to iterate over all keys, for example), one need
only code this once.

> It is smaller, but I'm willing to bet someone will hit it. For
> example, when you shutdown the computer, all the active editor
> instances will write to the config file at once. Maybe us "real"
> programmers don't leave editors active when we shutdown, but a lot of
> people do!

Good point. I hadn't thought of that. Hmmm...  Of course, there's no way to
catch the system shutting down out from under you in a portable way either,
so in reality it's more likely to lose *both* sets of changes. :-)

Ok. Nevermind. Rock on. You have convinced me with your superior logic. :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Darren New]

Stephen Leake wrote:
> I guess I need to update the API requirements list, and show those
> that we have consensus on, and those that are still under discussion.

I think that's reasonable. I guess all I'm saying is I'd like to see the API
requirements list complete (even if not "agreed on") before putting anything
concrete in the file. Otherwise, I think it's premature. If you say (for
example) that the order of keys don't matter, and you want to be able to
associate a comment with multiple keys, your file format might need to be
radically different, just because you added requirements that weren't there
before.

Maybe I'm just misinterpreting the API requirements or something, trying to
either read more or less into them than you intended. :-)

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: Config_Files proposal

[Georg Bauhaus]

Stephen Leake <stephen.a.leake.1@gsfc.nasa.gov> wrote:
: Granted the API is not finished, but the things we are still talking
: about have no impact on the file format. Or am I wrong about that?

Looks like an XML/Ada based child package implementation of the current
API is unaffected, with or without file locking.

-- Georg

Re: Config_Files proposal

[Georg Bauhaus]

Stephen Leake <stephen.a.leake.1@gsfc.nasa.gov> wrote:
: hmm. I guess you mean you want a record or array type accessed by a
: single key. I guess I'll agree; that has an impact on the file format.

Isn't this drifting a bit towards a convenient general file
based storage mechanism for structured values?  Too much in my
view for Config_Files maybe. (I would build wrapper subprograms to
update record or array values.)

-- Georg

Re: Config_Files proposal

[Randy Brukardt]

To stick my head into a debate that I probably should stay out of:

Stephan said:

>But I still think the right answer in this case is one of:
>
>"use a better (multi-document) text editor".
>
>"use the registry for resources shared by multiple processes".
>
>"use separate config files for separate instances of applications"


I think these are silly. I've always been dubious of this package, and
if this is the result, then there is no reason to bother, because it
would be totally useless.

Let's look at each of these "solutions"

>"use a better (multi-document) text editor".

This doesn't have any effect on the problem. On modern (multi-tasking)
systems, it is possible to run multiple copies of a program. If that
happens, you still have the problem. For instance, you can run multiple
copies of Word, even though there is no NEED to do so. And, as someone
pointed out, users won't read the manual about that anyway. If you
believe in solid applications (and if you don't, why are you using
Ada?), "user beware" is not an option.

>"use separate config files for separate instances of applications"

This doesn't seem to be much of a solution, either. You're requiring the
user to specify the "separate" file in some way. (You could use a PID or
the like for this, but then it hardly is a config file, as it would be
different for every invokation). Since this is likely to be a pain,
you'll have a default. And then you'll be back to the first situation,
because users won't read the manual, and they'll run multiple copies

>"use the registry for resources shared by multiple processes".

This of course works, but now you've lost the portability. Moreover,
since I don't believe that any carefully written apps could ever assume
single access, the Config_Files package could never be used. So why
bother to define it?

Summary: Requirement 1 has to go. For this package to be useful, it has
to make it easy to create bullet-proof configuration files, and more
portably than relying on the registry. (If you just want to use the
registry, use Claw.Registry, which is plenty portable on Windows.) Yes,
this is a lot of work. But if this was easy, there wouldn't be any need
for the package. (Just writing a value to a file hardly requires a fancy
package.)

                Randy Brukardt.

Re: Config_Files proposal

[Ted Dennison]

OK. I think I'm finally getting the problem here. This is my old issue 
with the emacs ada-mode case exceptions file from two years ago, come 
back to haunt me. :-(

Randy Brukardt wrote:
> I think these are silly. I've always been dubious of this package, and
> if this is the result, then there is no reason to bother, because it
> would be totally useless.

I think that's a bit extreme. At most, I think one could say that it 
would not make for a good solution in some cases. In particular, it 
would be a problem any time the program itself is both reading and 
writing config files, *and* there is the possibility of multiple copies 
of the program running simultaniously. Remember that we must include 
networked installations of a program under this umbrella.

This situation includes just about any desktop user app (eg: an Ada text 
editor). However, it happens to exclude every situation where I've found 
myself using a configuration file in the past (generally systems 
software). So I think calling it "useless" is overstating the case, 
possibly to the detriment of the point.

> Summary: Requirement 1 has to go. For this package to be useful, it has
> to make it easy to create bullet-proof configuration files, and more

I should point out here that the solution for this problem with Emacs 
ada-mode case exceptions is not "bullet-proof" either. However, it was 
close enough for me (the original complainer), and I have yet to hear 
anyone call ada-mode's case exceptions "useless" since then.

Say we do remove requirement 1. What do we replace it with? One 
possiblity is "nothing". But in that case we will make a portable 
reference implementation that doesn't bother with this case, nearly 
every vendor will just use the reference, and you will still not have 
this situation handled.

So what's the alternative you'd like? In other words, how does one solve 
this problem so that its impossible to loose changes due to simultanious 
edits?

Probably the way to think of this of this problem is in terms of 
revision-control systems.

We could only allow files to be updated with "delta" operations. But 
even then, you have the old revision-control problem of merging 
conflicting changes. I've yet to see a merging algorithm that didn't 
have situations that require human intervention. However, with our file 
format we probably couldn't detect these conflicts, so we'd just be back 
to the problem of wiped-out changes (but with a much smaller window for 
the wipeout to occur). This is sort of like an unlocked checkout in 
revision control systems. I think this is roughly the solution they used 
for the emacs ada-mode problem.

Another possibility would be some kind of explicit "edit" call that 
locks the file and reloads everything, and an "end edit" call that saves 
and unlocks it. But the client would still need to be very careful to 
display the read-in configurations for editing, not the ones that the 
program read in when it started. This is the "locked check-out" 
approach. The problem with this approach is that the locker may go AWOL, 
holding up everyone's changes for an indefinite period.

Re: Config_Files proposal

[Ted Dennison]

Stephen Leake wrote:
> Darren New <dnew@san.rr.com> writes:
> 
>>But you can eliminate a large number of race conditions by implementing a
>>merge. You can change it from "if you start the same program twice, you're
>>screwed" to "if you click the button on both instances in the length of time
>>it takes to write the file, you're screwed." I think that's reasonable. 

I think when they fixed the Emacs ada-mode case exceptions file writing 
algorithm, they did something like this. I was satisfied at the time, 
for the exact reason Darren mentions.

> 
> Right, I don't. I tend to be very picky; a file locking protocol is
> either "right", or it's "wrong"; no room for "mostly right" :).

I'd have to agree with this. For a single application like Emacs, it may 
be good enough to come close. For something that is going to be an 
official part of Ada, I don't think we should put in a feature that 
proports to solve a problem that it doesn't actually solve.

> It is smaller, but I'm willing to bet someone will hit it. For
> example, when you shutdown the computer, all the active editor
> instances will write to the config file at once. Maybe us "real"

Exactly. And when someone gets this problem, hoses their config file 
that they had setup *juuuust* right, and tracks the problem down, they 
are going to be quite ticked that the language advertised something as 
safe when it in fact wasn't.

> programmers don't leave editors active when we shutdown, but a lot of
> people do!

Actually, I would probably not have that happen to me, just because I 
never purposely shut my systems down. They only way they go down is the 
hard way. :-)

Re: Config_Files proposal

[Randy Brukardt]

Ted Dennison wrote in message <3D1D22BB.7000305@telepath.com>...
>OK. I think I'm finally getting the problem here. This is my old issue
>with the emacs ada-mode case exceptions file from two years ago, come
>back to haunt me. :-(
>
>Randy Brukardt wrote:
>> I think these are silly. I've always been dubious of this package,
and
>> if this is the result, then there is no reason to bother, because it
>> would be totally useless.
>
>I think that's a bit extreme. At most, I think one could say that it
>would not make for a good solution in some cases. In particular, it
>would be a problem any time the program itself is both reading and
>writing config files, *and* there is the possibility of multiple copies
>of the program running simultaniously. Remember that we must include
>networked installations of a program under this umbrella.
>
>This situation includes just about any desktop user app (eg: an Ada
text
>editor). However, it happens to exclude every situation where I've
found
>myself using a configuration file in the past (generally systems
>software). So I think calling it "useless" is overstating the case,
>possibly to the detriment of the point.


Well, "useless" is a bit extreme, but not much. You say that pretty much
all desktop apps couldn't use the package if Requirement 1 remains.
Moreover, most embedded applications can't use the package, because they
don't have a file system. So that leaves how many applications? Let's be
generous and say 10%. Why should we bother with a "standard" package
that can safely be used in only 10% of the applications? That hardly
needs to be standardized!

>> Summary: Requirement 1 has to go. For this package to be useful, it
has
>> to make it easy to create bullet-proof configuration files, and more
>
>I should point out here that the solution for this problem with Emacs
>ada-mode case exceptions is not "bullet-proof" either. However, it was
>close enough for me (the original complainer), and I have yet to hear
>anyone call ada-mode's case exceptions "useless" since then.
>
>Say we do remove requirement 1. What do we replace it with? One
>possiblity is "nothing". But in that case we will make a portable
>reference implementation that doesn't bother with this case, nearly
>every vendor will just use the reference, and you will still not have
>this situation handled.
>
>So what's the alternative you'd like? In other words, how does one
solve
>this problem so that its impossible to loose changes due to
simultanious
>edits?


You can't exactly (and probably shouldn't in every possible case), but
you can adopt rules that make it unlikely that trouble will occur. I
suggest:

1) If two applications write different, unrelated keys in the same file,
both changes will be reflected, and nothing else will be changed.

2) If an application writes a group of keys, it needs to be possible
that the application can insure that the entire group is updated as a
set.

These the first rule means that the only time a simulateous edit will be
lost is when two apps write the same key. There isn't much that we can
do about that. The second rule means that an app can insure that some
consistent set of keys is written, even if another app is trying to
write the same keys. (Imagine a set of keys representing a screen
position, getting written at shutdown of a bunch of copies of an editor.
We don't care which position is saved, but we certainly want to insure
that a single complete position is saved, rather than parts of several.

>Another possibility would be some kind of explicit "edit" call that
>locks the file and reloads everything, and an "end edit" call that
saves
>and unlocks it. But the client would still need to be very careful to
>display the read-in configurations for editing, not the ones that the
>program read in when it started. This is the "locked check-out"
>approach. The problem with this approach is that the locker may go
AWOL,
>holding up everyone's changes for an indefinite period.


This can be mitigated by using a controlled object "lock" for editing.
Then, if the locking app is aborted or has an unhandled exception, the
lock will get freed. That prevents program bugs from keeping the file
locked. (You'll still need some way to deal with OS termination of the
application's process; probably the lock will have to have a time limit
on it.)

                  Randy.

Re: Config_Files proposal

[Ted Dennison]

Randy Brukardt wrote:
> Well, "useless" is a bit extreme, but not much. You say that pretty much
> all desktop apps couldn't use the package if Requirement 1 remains.
> Moreover, most embedded applications can't use the package, because they
> don't have a file system. So that leaves how many applications? Let's be

That leaves non-embedded real-time apps, which accout for all but one of 
the projects I've ever worked on. Every single one of those projects 
(with one trivial exception) had a configuration file.

> generous and say 10%. Why should we bother with a "standard" package

I don't think we have enough information to make that estimate. If I 
were to do it based on my past experience, it would be roughly come out 
to %75 of all Ada projects. Further, it would be %100 of all Ada 
projects that use configuration files. If the mood struck me, I could 
even try and argue something close to these numbers, based on Ada's 
percieved heavier usage in real-time apps than in desktop apps.

Of course this is unscientific, but that's my point; we don't have 
scientific numbers.

>>So what's the alternative you'd like? In other words, how does one
> solve
>>this problem so that its impossible to loose changes due to
> simultanious
>>edits?
> 
> You can't exactly (and probably shouldn't in every possible case), but

Are saying that you think we should put something in the Ada spec that 
doesn't solve the problem it was introduced for; something that meerly 
lessens the probablility of a problem occuring, but leaves the 
probablility there? There are probably instances of such things in the 
existing Ada spec, but I don't like the idea much.

> you can adopt rules that make it unlikely that trouble will occur. I
> suggest:
> 
> 1) If two applications write different, unrelated keys in the same file,
> both changes will be reflected, and nothing else will be changed.
 >
> 2) If an application writes a group of keys, it needs to be possible
> that the application can insure that the entire group is updated as a
> set.
> 
> These the first rule means that the only time a simulateous edit will be
> lost is when two apps write the same key. There isn't much that we can
> do about that. 

I don't understand this thought. If we were to have users explicitly 
lock files for editing (and reread the values immediately after doing 
so), we could prevent that. If we don't do that, then there's no way to 
perform rule #2.

>>Another possibility would be some kind of explicit "edit" call that
>>locks the file and reloads everything, and an "end edit" call that
...
>>approach. The problem with this approach is that the locker may go
> AWOL,
>>holding up everyone's changes for an indefinite period.
> 
> This can be mitigated by using a controlled object "lock" for editing.

No it can't. Controlled-objects aren't system-wide. This kind of locking 
can only be done through OS calls.

Even if they were system-wide (which they aren't), there's nothing that 
says that if the OS kills your Ada process then your controlled objects 
will all get finalized nicely for you. On every Unix OS I've used, a 
"kill -9" on an Ada program will *not* do that.

Re: Config_Files proposal

[Randy Brukardt]

Ted Dennison wrote in message <3D20F3E3.8090104@telepath.com>...

Sigh. I *knew* I shouldn't get involved in this one. It would help a lot
of people would read what I wrote...


>Randy Brukardt wrote:
>> Well, "useless" is a bit extreme, but not much. You say that pretty
much
>> all desktop apps couldn't use the package if Requirement 1 remains.
>> Moreover, most embedded applications can't use the package, because
they
>> don't have a file system. So that leaves how many applications? Let's
be
>
>That leaves non-embedded real-time apps, which accout for all but one
of
>the projects I've ever worked on. Every single one of those projects
>(with one trivial exception) had a configuration file.

Huh? Obviously we have very different definitions of these things,
because there can be no such thing. As a first approximation, apps break
down into two categories - the kind that users explicitly launch on OSes
(which you somewhat inaccurately called desktop apps), and embedded apps
that aren't explicitly run. Whatever your projects have been either are
(non-traditional) embedded apps [which would be my guess], or "regular"
OS-hosted apps.

*All* user-lanuched apps need protection against multiple launches. You
better not fry the patient because someone started the program twice.
Admittedly, such protection could be done at a higher layer. But it
seems unusual to say the least to create a package which is designed to
fail in a likely case; after all, Ada is a safe language.

>> generous and say 10%. Why should we bother with a "standard" package
>
>I don't think we have enough information to make that estimate. If I
>were to do it based on my past experience, it would be roughly come out
>to %75 of all Ada projects. Further, it would be %100 of all Ada
>projects that use configuration files. If the mood struck me, I could
>even try and argue something close to these numbers, based on Ada's
>percieved heavier usage in real-time apps than in desktop apps.
>
>Of course this is unscientific, but that's my point; we don't have
>scientific numbers.


Since you're discussing a category that doesn't exist, its hard to
respond. Probably, your thinking of embedded application where there is
a file system. This seems to be a reasonably rare case in general,
although clearly there are specific instances of it. That would be the
only case where a package without write protection of some sort could be
useful. Obviously, your experience varies from mine a lot.

Anyway, if the package is defined without locking, it can only be useful
to Ted's type of applications (whatever they are). If the package is
defined with some sort of locking, it can be useful to most
applications. Moreover, an implementation specifically for Ted's
applications could leave out the locking, and still meet the
specification. So I find it hard to imagine that we would want to define
this stuff to be unsafe for the majority of applications, when it is not
hard to make it safe enough for most apps.

>>>So what's the alternative you'd like? In other words, how does one
solve
>>>this problem so that its impossible to loose changes due to
simultanious
>>>edits?
>>
>> You can't exactly (and probably shouldn't in every possible case),
but
>
>Are saying that you think we should put something in the Ada spec that
>doesn't solve the problem it was introduced for; something that meerly
>lessens the probablility of a problem occuring, but leaves the
>probablility there? There are probably instances of such things in the
>existing Ada spec, but I don't like the idea much.

No, I'm saying that if there are two changes to the same key at the same
time, which one gets stored is implementation-defined. I don't see much
benefit in trying to define what happens in that case; usually one or
the other is good enough anyway. If its not, you'll probably need to use
a first-class database, which is clearly out of scope.

But if two apps write different keys to the same file at the same time,
both changes should be reflected. A race condition is a bad thing.

You seem to worry about about the idea that writing the same key at the
same time doesn't have a defined result. But that's no different than
writing the same variable twice at the same time, and that certainly
doesn't have a defined result.

One of the biggest advantages of using a "canned" package like this (or
like Claw for that matter), rather than rolling your own, is simply that
the "canned" package is much more likely to handle unlikely but
dangerous cases correctly. Especially if it is widely used. It is rare
that the normal case is enough work that it is worth figuring out
someone else's package -- it is the handling of unusual conditions (like
locking) that can be the big benefit.

>> you can adopt rules that make it unlikely that trouble will occur. I
>> suggest:
>>
>> 1) If two applications write different, unrelated keys in the same
file,
>> both changes will be reflected, and nothing else will be changed.
> >
>> 2) If an application writes a group of keys, it needs to be possible
>> that the application can insure that the entire group is updated as a
>> set.
>>
>> These the first rule means that the only time a simulateous edit will
be
>> lost is when two apps write the same key. There isn't much that we
can
>> do about that.
>
>I don't understand this thought. If we were to have users explicitly
>lock files for editing (and reread the values immediately after doing
>so), we could prevent that. If we don't do that, then there's no way to
>perform rule #2.


A *user* could prevent that. But the default case should allow rule #1,
even if there is no explicit locking. Rule #1 should *always* hold - for
*any* write. Indeed, rule #1 here is much more important than rule #2.
With rule #1, you can arrange your own locking to implement rule #2.
Without rule #1, you can't assume much of anything.

This package should be reasonably bullet-proof -- a simple use should be
reasonably safe by default. That means it must do some locking
internally.

Note that if the package supports "groups" of keys (as had been
suggested elsewhere), we don't need rule #2 at all. We can just provide
operations for updating an entire group of keys at once, and leave the
locking implicit. That may be preferable.

>>>Another possibility would be some kind of explicit "edit" call that
>>>locks the file and reloads everything, and an "end edit" call that
>...
>>>approach. The problem with this approach is that the locker may go
AWOL,
>>>holding up everyone's changes for an indefinite period.
>>
>> This can be mitigated by using a controlled object "lock" for
editing.
>
>No it can't. Controlled-objects aren't system-wide. This kind of
locking
>can only be done through OS calls.

>
>Even if they were system-wide (which they aren't), there's nothing that
>says that if the OS kills your Ada process then your controlled objects
>will all get finalized nicely for you. On every Unix OS I've used, a
>"kill -9" on an Ada program will *not* do that.


You obviously didn't read any of the rest of what I wrote here (and that
you didn't quote), which covered all of that.

Let me quote myself:

"Then, if the locking app is aborted or has an unhandled exception, the
lock will get freed. That prevents program bugs from keeping the file
locked. (You'll still need some way to deal with OS termination of the
application's process; probably the lock will have to have a time limit
on it.)"

Of course the lock has to be at the OS level. But you don't want to hold
it any longer than you have to. A controlled object provides protection
against program bugs holding the lock too long, by automatically
releasing the OS lock. It doesn't provide the entire solution (OS
termination - "kill" to Unix-heads - still has to be handled with a time
limit), but it lets the time limit case be quite rare, so we can leave
the lock held for quite a while before forcibly releasing it. That makes
the lock much safer, because if you let the lock be freed too soon (when
an App really is using it), you have all of the problems again -- only
worse, because the programmer has an expectation of safety.

               Randy.

Re: Config_Files proposal

[Robert I. Eachus]

Randy Brukardt wrote:


> "Then, if the locking app is aborted or has an unhandled exception, the
> lock will get freed. That prevents program bugs from keeping the file
> locked. (You'll still need some way to deal with OS termination of the
> application's process; probably the lock will have to have a time limit
> on it.)"
> 
> Of course the lock has to be at the OS level. But you don't want to hold
> it any longer than you have to. A controlled object provides protection
> against program bugs holding the lock too long, by automatically
> releasing the OS lock. It doesn't provide the entire solution (OS
> termination - "kill" to Unix-heads - still has to be handled with a time
> limit), but it lets the time limit case be quite rare, so we can leave
> the lock held for quite a while before forcibly releasing it. That makes
> the lock much safer, because if you let the lock be freed too soon (when
> an App really is using it), you have all of the problems again -- only
> worse, because the programmer has an expectation of safety.


Exactly.  The Ada standard has to be easy to use but prevent the 
"stupid" cases.  There will always be a way outside the Ada interface to 
muck things up:  Text editors, registry editors, system crashes, hard 
drive failures, etc.

What we have to worry about is the case of two (often different) Ada 
programs accessing the same configuration file and making a mess of it. 
  In other words, we want to guarantee that as long as all access is 
through the standard Ada package interface, strange things don't happen. 
  If strange things happen in the external environment that foul up the 
configuration file that is not our problem.

As far as I am concerned an interface that provided two types of handles 
should be sufficient.  You either have (shared) read access or unshared 
read-write access to file elements.  Whether the granularity is per 
file, per object, or some definition of group is something worth 
discussing.  But we have to have consistancy.  If one implementation 
locks files (Unix) and another locks objects (Windows Registry) then 
applications are not portable.

Since I feel that simplicity is best, I like the idea of configuration 
file locking.  Normal access is to open the file read it, then close it. 
  If your program wants to update the file it re-opens it read write. 
Yes this will not provide notification if the file changes after a 
program starts, but I think that is reasonable. I just find it hard to 
imagine situations where two programs update different but intersecting 
attributes.  For example let us say that one program changes the font 
size used in an output window, and another sets the size of the window. 
  I certainly want these actions to be done in some consistant order, 
but IMHO it would be bad programming if only one of the programs 
computed the correct line-length information...