24-April-1998
Please email error reports to www-html-editor@w3.org.
DELIM
GENERAL SGMLREF
SHORTREF SGMLREF
must be changed to:
DELIM
GENERAL SGMLREF
HCRO "&#x" -- 38 is the number for ampersand --
SHORTREF SGMLREF
And the initial <!SGML "ISO 8879:1986" must be changed to <!SGML "ISO 8879:1986 (WWW)".
accept-charset %Charsets; #IMPLIED -- list of supported charsets --archive attribute is defined in the DTD as taking a value of type %URI". This is incorrect: the value may be a space-separated list of URIs (as indicated in the definition of the attribute and in the DTDs comment).
Further down the page, the specification reads, "There must be exactly one "User-agent" field per record." In fact, there can be more than one User-Agent field in the robots.txt file, just not more than one per record.
For information about search robots, please consult, for example: