GNU AutoGen Conversion

Starting with 4.2.4 we are using GNU AutoGen to provide consistent options processing, easier enum/flag/descriptor maintenance, and eventually documentation (including html, man, and info pages, as appropriate) for several programs and files.

Related Topics: GNUAutoGenImprovements, GenLocInfo

Conversion Progress Status

The conversion status is as follows:

Item	Directory	.def	.man	.mdoc	.texi	.html	Reconcile with FreeBSD	Reconcile with DLM's html pages	Man Section	Notes
/dev refclock entries	?	Not started							4
.ntprc	?	Not started						N/A	5
ntp.audio	?	Not started						N/A	5
ntp.conf	ntpd								5	What about the simulator?
ntp.keys	ntpd								5
ntp-keygen	util								loc
ntp-wait	util								loc
ntpd	ntpd								loc
ntpdsim	ntpd								loc	Remember ntp.conf
ntpdate (old)	ntpdate	Will be deprecated							loc
ntpdate (new)	?	Not started							loc
ntpdc	ntpdc								loc
ntpq	ntpq								loc
ntpsnmpd	ntpsnmpd								loc
ntptime	util	Not started							loc
ntptrace	scripts	Not started							loc
sntp	sntp								loc
tickadj	util	Not started							loc
timetrim	util	Not started							loc
refclock drivers	html/drivers	Not started							?

There are other items in the scripts/ directory that are not in the above table. If you think more items should be there, please add them.

Note that we can use DLM's .html pages as the starting point for the .texi pages.

We can use FreeBSD's .mdoc pages as the starting point for the info in our .def pages.

In the .def pages:

• in the flag stanzas:
  • the descrip tags may contain no formatting tags
  • the doc tags may contain .texi tags
• the doc-section stanzas may contain tags - by default these are .texi tags and some of these stanzas are only used for .texi and .info files. Some are used for manual pages, and for those we specify ds-format = 'mdoc'; and use .mdoc tags.

Making sure it works

Start with DLM's .html page for a given document and also the corresponding man page from FreeBSD.

Add appropriate content to the .def file for the manual pages, and to the foo.texi file for the .html page we'll produce.

• Look at the resulting .man, .mdoc, invoke-XXX.texi (if generated), and .html files for things like unprocessed or incorrectly processed tags and punctuation not "cuddling" properly.
  • Should we be producing (and installing) .info files?
• Compare the formatted ouput of the .mdoc, .man, and .html files to the formatted output from DLM and FreeBSD.
• If you have texi2pdf, run it on the .texi file we produce, work on any errors that are found, and look at the resulting .pdf file.

Note that DLM's pages often contain "Millspeak" and sometimes typos. Sometimes his pages contain broken sentences or formatting problems.

We should leave the "Millspeak" alone, fix the typos, any broken sentences, and improve formatting.

autogen issues

• ntp.conf (?) has some problem that causes autogen to abort.

mdoc tag fonts and formatting

To see exactly what sort of fonts and formatting are needed, here is a link to a PDF file for the formatted mdoc man page:

• groff_mdoc.pdf: PDF of the formatted mdoc man page

mdoc2man needs

mdoc2texi needs

Item	State	From	Notes
.Sh		ntp.conf	What to do about the node and menu?
.Ss		ntp.conf	What to do about the node and menu?

Current Usage Notes

We are using autogen-5.11.9 as of May 2011.

This version seems to have useful support for mdoc tags, and since that is the canonical form we are using that tag format internally now. sntp has been (perhaps fully) converted.

This version also supports useful license content information. See http://autogen.sourceforge.net/doc/autogen_104.html#SCM-license_002dfull for more information.

We were using autogen-5.11.6 as of Feb 2011.

This release of autogen removes some of Kapila's scripts, and Bruce indicates they can be put back in when they are better tested.

Additionally, agman-mdoc-cmd.tpl, agman5.tpl and agman8.tpl have been removed. While agman1.tpl remains, the following is now the preferred template and method of generating manual pages.

autogen -DMAN_SECTION=n -Tagman-cmd opts.def

where n is 1, 5, or 8.

Old Notes

Capabilities we still need

• We need AutoGen support for man pages in other sections (4, 5, 8, at least).
• Supporting more formatting directives
• Supporting tables/definitions
• Supporting configuration file formats (syntax, etc.)

It might be really nice to allow folks to format the documentation section in any of a number of formats. We could then use the post-processing stuff described below to get the data into a canonical form and perform the conversion to the desired output format from there.

AutoGen version in the generated files

As of 5.8.6, the AutoGen templates do not include the version of AutoGen that was used to produce the output files.

Here are some reasons why Harlan wants the version of AutoGen to be in the generated files:

• The rest of the GNU auto* tools do it.
• It makes it obvious which version of AutoGen was used to generate a file. This is useful for:
  • detecting the need to possibly rebuild if a developer used a significantly different version from what we are using on the tarball generation machine
  • seeing how different versions of AutoGen behave

Bruce suggests adding:

 [= (. autogen-version) =]

to the templates. Note, this is the templates, not the .def files.

Harlan is concerned about upgrade issues and would rather not change the master template files.

make can do things like:

 FOO?= bar

and something similar to this would be useful to allow us to override the stock header format and get the value of autogen-version in there.

It is also possible that if we had pre- and post- triggers we could use one of them to change the header template after it was set to get the autogen-version into the generated file.

There is also a similar issue here as with the expansion of, for example, prog-name in the documentation (described below). In this case we have some information in the header we want formatted and it will need to be in a C-style comment for .[ch] files and prefixed with ./" for the man pages (and something else for the .texi files).

-- HarlanStenn - 26 Sep 2006

Note also, in the .def file you may:

export = `v=\`autogen --version |sed 's/.*Ver. *//'\`
    echo '#define AUTOGEN_BUILD_VERSION "'${v}'"'
`;

and with that you will have a fairly obvious AG version string in your generated .h file. It still does not explain why you would want to do this. An upgraded AutoGen is not going to alter the output from the option processing templates, unless the templates are updated or a generator bug is fixed. The latter is not likely and the former is tested with:

#define AO_TEMPLATE_VERSION 118784
#if (AO_TEMPLATE_VERSION < OPTIONS_MINIMUM_VERSION) \
 || (AO_TEMPLATE_VERSION > OPTIONS_STRUCT_VERSION)
# error option template version mismatches autoopts/options.h header
  Choke Me.
#endif

So, it is not clear to me the need for having the AutoGen version in your header. Nevertheless, you can do it without changing the current templates.

-- BruceKorb

Expanding prog-name in the documentation

The various bits of documentation will often refer to the prog-name, and it would be really nice if:

• we did not have to type the program name each time
• the program name was in the proper font/style for each output type
  • .Nm for man pages
  • @command{...} for .texi stuff (this is a relatively new .texi directive)

This is similar to the above issue of customizing the header format of generated files.

Related Topics: GSoC2008doc

Pre- and Post- processing

There are a variety of different strings that are set in a .def file that get used in various template generation mechanisms. These are documented in the "Information Attributes" section of autogen.info (Section 7.5.4 in the one I saw).

Harlan thinks it would be Very Useful to be able to use optional pre- and/or post- triggers when processing .def and .tpl files. OK, perhaps only post-processing would be useful.

The point is it might be really useful to use some sort of cookies in the strings in the .def files that are post-processed to expand them (and possibly re-flow the text, for example in the --help output).

If we are going to allow an optional post-processing of a string it can happen in at least 2 places:

• right after the string is used
  • this is probably lame, but it would allow for per-string processing directives
• after the output file is created we do the post-processing.

Perhaps the following choices are sufficient. If there is a foo.def file:

• foo.def.TEMPLATE.post is post-processing directives for TEMPLATE (curdir)
• foo.def.post is a post-processing file (curdir)
• def.TEMPLATE.post is a post-processing file for TEMPLATE
• def.post is a general post-processing file

Where should the last 2 cases "live"? Should they have curdir-scope or curdir+subdir scope? What about a -I sort of thing?