Extension queries

[ RfDs/CfVs | Other proposals ]

Poll standings

See below for voting instructions.
Systems
[ ] conforms to ANS Forth.
 Gforth
 Win32Forth
 PFE (Guido Draheim)
 iForth (Marcel Hendrix)
[ ] already implements the proposal in full since release [ ].
 4th 3.3d R2 (Hans Bezemer)
 iForth 3.0
[ ] implements the proposal in full in a development version.
 Gforth
[ ] implements the proposal in full non-minimally in a development version.
 Gforth
[ ] will implement the proposal in full in release [ ].
 Gforth 0.7
 4th 3.5b R2 (Hans Bezemer)
[ ] will implement the proposal in full non-minimally in release [ ].
 Gforth 0.7
[ ] will implement the proposal in full in some future release.
[ ] will implement the proposal in full non-minimally in some future release.
 Win32Forth
There are no plans to implement the proposal in full in [ ].
[ ] will never implement the proposal in full.
 PFE
Programmers
[ ] I have used (parts of) this proposal in my programs.
 Anton Ertl
[ ] I would use (parts of) this proposal in my programs if the systems
    I am interested in implemented it.
 David N. Williams
 Anton Ertl
 Alex McDonald
[ ] I would use (parts of) this proposal in my programs if the systems
    I am interested in implemented it non-minimally.
 David N. Williams
 Anton Ertl
[ ] I would use (parts of) this proposal in my programs if this
    proposal was in the Forth standard.
 David N. Williams
 Anton Ertl
 Alex McDonald
[ ] I would not use (parts of) this proposal in my programs.
 Hans Bezemer (prefers [DEFINED])
Informal results

Change history

2005-03-16
No change in the normative section. Made reference implementation case-insensitive. Added non-normative section on naming guidelines. Fixed typo.
2nd RfD 2005-01-17
No change to the normative section. New, non-trivial reference implementation. More discussion of "Why not let ENVIRONMENT? return a flag and true, like for wordset queries?" and "Why the "X:" prefix?". New sections "Won't there be too many extension names?", "How about defining a way to query for implementor extensions?", "Why not just ask for word names with [UNDEFINED]?"

Problem

How does a program know whether the system it runs on supports one of the extensions that ran through the RfD/CfV process, so that the program can implement the extension itself or work around its absence?

Proposal

If the string passed to ENVIRONMENT? starts with "X:", ENVIRONMENT? returns false if the system does not implement the extension indicated by the query string in full, or if there is no such extension that has gone to a CfV.

For an extension from the list of CfVs, take the linked-to filename, delete the ".html", and prepend "X:" to construct a query string for the extension.

If the system implements the extension, ENVIRONMENT? may return true (without additional values) or false.

Typical Use

S" X:deferred" ENVIRONMENT? 0= [IF]
... \ reference implementation of the deferred words proposal
[THEN]

Remarks

Why allow returning false when the system supports the extension?
Returning false when the system supports the extension will usually be safer than returning true when the system does not support the extension; in the former case the program will be slower, or have degraded features; in the latter case the program will usually fail in unpredictable ways.

Therefore, systems must not return true for extensions that have not yet gone to a CfV (the proposal for the extension could still change).

So, if a system happens to already support the extension, it will have to report false on queries for the extension at least from the time when the proposal goes to a CfV until the time that an update of the system with updated extension queries is released.

Moreover (and possibly more importantly), this feature means that systems whose implementors have never heard of (or ignore) RfDs and CfVs will work correctly for extension queries (as long as they don't support any queries starting with "X:" on their own), so a program written to cope with this specification will usually work correctly even on such systems.

Why not let ENVIRONMENT? return a flag and true, like for wordset queries?
This proposal is easier to use. What is the point of returning an extra flag? "Yes, we have heard of that extension, but no, we have not implemented it"? That's not a useful information to have; what should a program do with that information?

Mitch Bradley and Guido Draheim would prefer a wordset-query-like behaviour, i.e., an additional flag if ENVIRONMENT? returns true. Richard Borrell would prefer a single flag (i.e., the proposed behaviour).

With the wordset-query-like behaviour, the typical use above would look like: 

S" X:deferred" ENVIRONMENT? dup [IF]
  drop
[THEN]
0= [IF]
... \ reference implementation of the deferred words proposal
[THEN]

Here are some statistics about the use of ENVIRONMENT? in general and wordset queries in particular in ANS Forth programs: 

Program      Author          ENVIRONMENT?  wordset queries
brainless    David Kuehling  yes           no
brew-0_03z9  Robert Epprecht yes           no
brew_..._38  Robert Epprecht yes           floating-ext
CD16v11      Brad Eckert     no            no
anagrams     Wil Baden       no            no
pentomino    Bruce Hoyt      no            no
tscp         Ian Osgood      no            no
Gray4        Anton Ertl	     yes	   no
garbage-coll Anton Ertl	     yes	   no

I believe that one of the reason that wordset queries are used so rarely is that they are so cumbersome to use (that's certainly one of the reasons that kept me from using it).

Why the "X:" prefix?
This will hopefully ensure that there is no naming conflict with any existing environmental query of any system; it also reserves a part of the environmental query name space (by requiring a false result for anything that has not gone to a CfV), without consuming all of it.

If you know of any name conflict of the "X:" prefix with an existing system and have a better suggestion for a prefix, let me know.

Bruce McFarling has suggested changing the prefix such that the query string can be used as a filename on DOS/Windows. However the prefix can be cut away easily, leading to a filename compatible with DOS/Windows (if the extension name is compatible), as the reference implementation demonstrates.

Guido Draheim suggested using a suffix, as it has advantages with input completion. A prefix can also be used to profit with input completion, and overall this issue does not seem very important. Prefixes are more traditional for tagging names in programs (while suffixes are used for file names).

What about extension proposals that have not (yet) gone to a CfV?
If you want to introduce queries for them, do it with a different prefix.
Why not include extension proposals that have not (yet) gone to a CfV?
They may still change before they go to a CfV, so it would not be clear if the system and the querying program refer to the same proposal.
Won't there be too many extension names?
Guido Draheim thinks that we will see many backwards-compatible revisions of proposals, so the number of extension names that should be known around for a the extensions would be a problem (since the nth revision would satisfy the requirements of revision 1-n, and thus n names would have to be kept around).

One way to deal with this would be to use a consistent naming scheme for backwards-compatible extensions: deferred, deferred-2, deferred-3 etc. Then the system just needs to store the base name and the current revision number, and can check with a little code whether the queried-for extension is supported.

Keeping this naming scheme would be the responsibility of the person who maintains the list of CfVs. (It's not the responsibility of the system implementor and therefore not in the normative part of this proposal).

How about defining a way to query for implementor extensions?
Guido Draheim suggested this. With this one system implementor could formally define an extension, and another system could adopt that extension, and programs could query for this extension.

Doing this would require reserving some part of the ENVIRONMENT? name space for implementor-extension queries, with each query having an implementor part, and an extension name. A naming authority would assign implementor part names to implementors, and the implementor would assign extension part names. (Actually the implementor could use it for arbitrary queries, not just extension queries).

I am not convinced that there is enough demand for that. In any case, I would like to see it handled in a separate RfD/CfV.

Why not just ask for word names with [UNDEFINED]?

Hans Bezemer and Albert van der Horst favour this approach.

That only tells whether a word exists in the system, but not how it behaves.

It would also require asking separately for each word. And for on-demand loading, it would require organizing each word separately. Lots of effort for the programmer and the system implementor.

Guidelines for extension names
Extension names should limited to 29 characters (excluding the "X:" prefix), so they can be stored into a wordlist systems with 31-char names.

The names of the extension queries (excluding the "X:" prefix) and the file names of the reference implementations should be lower case, so that the reference implementation of extension queries works on case-sensitive file systems (most Forth systems and the reference implementation match extension-query names case-insensitively, but making that standard is another RfD).

The names of the extension queries (excluding the "X:" prefix) should only contain printable ASCII characters that are allowed in all common file systems; it's probably best to just use alphanumeric characters and the "-".

Implementation and Tests

Experience

All ANS Forth systems I know implement this proposal in a minimal way (answer all queries with false). None implement it in a non-minimal way. No programs have used the proposal yet.

Comments

Voting instructions

Fill out the appropriate ballot(s) below and mail it/them to me <anton@mips.complang.tuwien.ac.at>. Your vote will be published (including your name (without email address) and/or the name of your system) here. You can vote (or change your vote) at any time by mailing to me, and the results will be published here.

Note that you can be both a system implementor and a programmer, so you can submit both kinds of ballots.

Ballot for systems

If you maintain several systems, please mention the systems separately in the ballot. Insert the system name or version between the brackets. Multiple hits for the same system are possible (if they do not conflict).

Since most ANS Forth systems already support this proposal in full in a minimal way (i.e., they return false on every extension query), this ballot also asks for non-minimal support.

Note that for this proposal non-implementation means that you reserve the right to answer "X:" queries with true even if you do not implement the queried-for extension. If the system answers the query with false, it supports the proposal in full in a minimal way. 

[ ] conforms to ANS Forth.
[ ] already implements the proposal in full since release [ ].
[ ] implements the proposal in full in a development version.
[ ] implements the proposal in full non-minimally in a development version.
[ ] will implement the proposal in full in release [ ].
[ ] will implement the proposal in full non-minimally in release [ ].
[ ] will implement the proposal in full in some future release.
[ ] will implement the proposal in full non-minimally in some future release.
There are no plans to implement the proposal in full in [ ].
[ ] will never implement the proposal in full.
If you want to provide information on partial implementation, please do so informally, and I will aggregate this information in some way.

Ballot for programmers

Just mark the statements that are correct for you (e.g., by putting an "x" between the brackets). If some statements are true for some of your programs, but not others, please mark the statements for the dominating class of programs you write. 
[ ] I have used (parts of) this proposal in my programs.
[ ] I would use (parts of) this proposal in my programs if the systems
    I am interested in implemented it.
[ ] I would use (parts of) this proposal in my programs if the systems
    I am interested in implemented it non-minimally.
[ ] I would use (parts of) this proposal in my programs if this
    proposal was in the Forth standard.
[ ] I would not use (parts of) this proposal in my programs.
If you feel that there is closely related functionality missing from the proposal (especially if you have used that in your programs), make an informal comment, and I will collect these, too. Note that the best time to voice such issues is the RfD stage.
Anton Ertl