r58 - 2013-08-07 - 09:18:08 - HarlanStennYou are here: NTP >  Dev Web > DevelopmentIssues > GNUAutoGenConversion
NTP users are strongly urged to take immediate action to ensure that their NTP daemons are not susceptible to being used in distributed denial-of-service (DDoS) attacks. Please also take this opportunity to defeat denial-of-service attacks by implementing Ingress and Egress filtering through BCP38.

ntp-4.2.8p10 was released on 21 March 2017. It addresses 6 medum- and 5 low-severity security issues, 4 informational security topics, 15 bugfixes, and contains other improvements over 4.2.8p9.

Please see the NTP Security Notice for vulnerability and mitigation details.

Are you using Autokey in production? If so, please contact Harlan - he's got some questions for you.

GNU AutoGen Conversion

Starting with 4.2.4 we are using GNU AutoGen to provide consistent options processing, easier enum/flag/descriptor maintenance, and eventually documentation (including html, man, and info pages, as appropriate) for several programs and files.

Related Topics: GNUAutoGenImprovements, GenLocInfo

Conversion Progress Status

The conversion status is as follows:

Item Directory .def .man .mdoc .texi .html Reconcile with
Reconcile with
DLM's html pages
Man Section Notes
.ntprc ? Not started           N/A 5  
ntp.audio ? Not started           N/A 5  
ntp.conf ntpd DONE wip DONE wip wip     5 What about the simulator?
ntp.keys ntpd DONE wip DONE wip wip     5  
ntp-keygen util DONE   DONE wip wip     loc  
ntp-wait util DONE   DONE wip   N/A   loc  
ntpd ntpd DONE   DONE wip       loc  
ntpdsim ntpd DONE             loc Remember ntp.conf
ntpdate (old) ntpdate Will be deprecated             loc  
ntpdate (new) ? Not started             loc  
ntpdc ntpdc DONE   DONE         loc  
ntpq ntpq DONE   DONE wip       loc  
ntpsnmpd ntpsnmpd DONE   wip wip       loc  
ntptime util Not started             loc  
ntptrace scripts Not started             loc  
sntp sntp DONE wip DONE wip wip     loc  
tickadj util Not started             loc  
timetrim util Not started             loc  
/dev refclock entries ? Not started             4  
refclock drivers html/drivers Not started             ?  

There are other items in the scripts/ directory that are not in the above table. If you think more items should be there, please add them.

Note that we can use DLM's .html pages as the starting point for the .texi pages.

We can use FreeBSD's .mdoc pages as the starting point for the info in our .def pages.

In the .def pages:

  • in the flag stanzas:
    • the descrip tags may contain no formatting tags
    • the doc tags may contain .texi tags
  • the doc-section stanzas may contain tags - by default these are .texi tags and some of these stanzas are only used for .texi and .info files. Some are used for manual pages, and for those we specify ds-format = 'mdoc'; and use .mdoc tags.

Making sure it works

Start with DLM's .html page for a given document and also the corresponding man page from FreeBSD.

Add appropriate content to the .def file for the manual pages, and to the foo.texi file for the .html page we'll produce.

  • Look at the resulting .man, .mdoc, invoke-XXX.texi (if generated), and .html files for things like unprocessed or incorrectly processed tags and punctuation not "cuddling" properly.
    • Should we be producing (and installing) .info files?
  • Compare the formatted ouput of the .mdoc, .man, and .html files to the formatted output from DLM and FreeBSD.
  • If you have texi2pdf, run it on the .texi file we produce, work on any errors that are found, and look at the resulting .pdf file.

Note that DLM's pages often contain "Millspeak" and sometimes typos. Sometimes his pages contain broken sentences or formatting problems.

We should leave the "Millspeak" alone. We should fix the typos, any broken sentences, and improve formatting.

autogen issues

  • ntp.conf (?) has some problem that causes autogen to abort.

mdoc tag fonts and formatting

To see exactly what sort of fonts and formatting are needed, here is a link to a PDF file for the formatted mdoc man page:

mdoc2man needs

Item State From
.Pa   sntp
.Li   sntp

mdoc2texi needs

Item State From Notes
.Oc   invoke-ntp.conf.texi  
.Oo   invoke-ntp.conf.texi  
.Sh wip ntp.conf What to do about the node and menu?
.Ss wip ntp.conf What to do about the node and menu?

Current Usage Notes

We are using autogen-5.18.x as of July 2013.

We are using autogen-5.16.2 as of September 2012.

We are using autogen-5.11.9 as of May 2011.

This version seems to have useful support for mdoc tags, and since that is the canonical form we are using that tag format internally now. sntp has been (perhaps fully) converted.

This version also supports useful license content information. See http://autogen.sourceforge.net/doc/autogen_104.html#SCM-license_002dfull for more information.

We were using autogen-5.11.6 as of Feb 2011.

This release of autogen removes some of Kapila's scripts, and Bruce indicates they can be put back in when they are better tested.

Additionally, agman-mdoc-cmd.tpl, agman5.tpl and agman8.tpl have been removed. While agman1.tpl remains, the following is now the preferred template and method of generating manual pages.

autogen -DMAN_SECTION=n -Tagman-cmd opts.def

where n is 1, 5, or 8.

Old Notes

Capabilities we still need

  • We need AutoGen support for man pages in other sections (4, 5, 8, at least).
  • Supporting more formatting directives
  • Supporting tables/definitions
  • Supporting configuration file formats (syntax, etc.)

It might be really nice to allow folks to format the documentation section in any of a number of formats. We could then use the post-processing stuff described below to get the data into a canonical form and perform the conversion to the desired output format from there.

AutoGen version in the generated files

As of 5.8.6, the AutoGen templates do not include the version of AutoGen that was used to produce the output files.

Here are some reasons why Harlan wants the version of AutoGen to be in the generated files:

  • The rest of the GNU auto* tools do it.
  • It makes it obvious which version of AutoGen was used to generate a file. This is useful for:
    • detecting the need to possibly rebuild if a developer used a significantly different version from what we are using on the tarball generation machine
    • seeing how different versions of AutoGen behave

Bruce suggests adding:

 [= (. autogen-version) =]

to the templates. Note, this is the templates, not the .def files.

Harlan is concerned about upgrade issues and would rather not change the master template files.

make can do things like:

 FOO?= bar

and something similar to this would be useful to allow us to override the stock header format and get the value of autogen-version in there.

It is also possible that if we had pre- and post- triggers we could use one of them to change the header template after it was set to get the autogen-version into the generated file.

There is also a similar issue here as with the expansion of, for example, prog-name in the documentation (described below). In this case we have some information in the header we want formatted and it will need to be in a C-style comment for .[ch] files and prefixed with ./" for the man pages (and something else for the .texi files).

-- HarlanStenn - 26 Sep 2006

Note also, in the .def file you may:

export = `v=\`autogen --version |sed 's/.*Ver. *//'\`
    echo '#define AUTOGEN_BUILD_VERSION "'${v}'"'

and with that you will have a fairly obvious AG version string in your generated .h file. It still does not explain why you would want to do this. An upgraded AutoGen is not going to alter the output from the option processing templates, unless the templates are updated or a generator bug is fixed. The latter is not likely and the former is tested with:

#define AO_TEMPLATE_VERSION 118784
# error option template version mismatches autoopts/options.h header
  Choke Me.

So, it is not clear to me the need for having the AutoGen version in your header. Nevertheless, you can do it without changing the current templates.

-- BruceKorb

Expanding prog-name in the documentation

The various bits of documentation will often refer to the prog-name, and it would be really nice if:

  • we did not have to type the program name each time
  • the program name was in the proper font/style for each output type
    • .Nm for man pages
    • @command{...} for .texi stuff (this is a relatively new .texi directive)

This is similar to the above issue of customizing the header format of generated files.

Related Topics: GSoC2008doc

Pre- and Post- processing

There are a variety of different strings that are set in a .def file that get used in various template generation mechanisms. These are documented in the "Information Attributes" section of autogen.info (Section 7.5.4 in the one I saw).

Harlan thinks it would be Very Useful to be able to use optional pre- and/or post- triggers when processing .def and .tpl files. OK, perhaps only post-processing would be useful.

The point is it might be really useful to use some sort of cookies in the strings in the .def files that are post-processed to expand them (and possibly re-flow the text, for example in the --help output).

If we are going to allow an optional post-processing of a string it can happen in at least 2 places:

  • right after the string is used
    • this is probably lame, but it would allow for per-string processing directives
  • after the output file is created we do the post-processing.

Perhaps the following choices are sufficient. If there is a foo.def file:

  • foo.def.TEMPLATE.post is post-processing directives for TEMPLATE (curdir)
  • foo.def.post is a post-processing file (curdir)
  • def.TEMPLATE.post is a post-processing file for TEMPLATE
  • def.post is a general post-processing file

Where should the last 2 cases "live"? Should they have curdir-scope or curdir+subdir scope? What about a -I sort of thing?

Topic attachments
I Attachment Action Size Date Who Comment
pdfpdf groff_mdoc.pdf manage 218.0 K 2012-12-15 - 10:29 HarlanStenn PDF of the formatted mdoc man page
Edit | WYSIWYG | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r58 < r57 < r56 < r55 < r54 | More topic actions
Dev.GNUAutoGenConversion moved from Dev.GNUNopAutoGenConversion on 2005-07-11 - 01:46 by HarlanStenn - put it back
SSL security by CAcert
Get the CAcert Root Certificate
This site is powered by the TWiki collaboration platform
IPv6 Ready
Copyright & 1999-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding the site? Send feedback