draft-mcquistin-augmented-ascii-diagrams-11.txt | draft-mcquistin-augmented-ascii-diagrams-12.txt | |||
---|---|---|---|---|
Network Working Group S. McQuistin | Network Working Group S. McQuistin | |||
Internet-Draft V. Band | Internet-Draft V. Band | |||
Intended status: Experimental D. Jacob | Intended status: Experimental D. Jacob | |||
Expires: 10 March 2023 C. S. Perkins | Expires: 14 September 2023 C. S. Perkins | |||
University of Glasgow | University of Glasgow | |||
6 September 2022 | 13 March 2023 | |||
Describing Protocol Data Units with Augmented Packet Header Diagrams | Describing Protocol Data Units with Augmented Packet Header Diagrams | |||
draft-mcquistin-augmented-ascii-diagrams-11 | draft-mcquistin-augmented-ascii-diagrams-12 | |||
Abstract | Abstract | |||
This document describes a machine-readable format for specifying the | This document describes a machine-readable format for specifying the | |||
syntax of protocol data units within a protocol specification. This | syntax of protocol data units within a protocol specification. This | |||
format is comprised of a consistently formatted packet header | format is comprised of a consistently formatted packet header | |||
diagram, followed by structured explanatory text. It is designed to | diagram, followed by structured explanatory text. It is designed to | |||
maintain human readability while enabling support for automated | maintain human readability while enabling support for automated | |||
parser generation from the specification document. This document is | parser generation from the specification document. This document is | |||
itself an example of how the format can be used. | itself an example of how the format can be used. | |||
skipping to change at page 1, line 38 ¶ | skipping to change at page 1, line 38 ¶ | |||
Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
Drafts is at https://datatracker.ietf.org/drafts/current/. | Drafts is at https://datatracker.ietf.org/drafts/current/. | |||
Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
This Internet-Draft will expire on 10 March 2023. | This Internet-Draft will expire on 14 September 2023. | |||
Copyright Notice | Copyright Notice | |||
Copyright (c) 2022 IETF Trust and the persons identified as the | Copyright (c) 2023 IETF Trust and the persons identified as the | |||
document authors. All rights reserved. | document authors. All rights reserved. | |||
This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
Provisions Relating to IETF Documents (https://trustee.ietf.org/ | Provisions Relating to IETF Documents (https://trustee.ietf.org/ | |||
license-info) in effect on the date of publication of this document. | license-info) in effect on the date of publication of this document. | |||
Please review these documents carefully, as they describe your rights | Please review these documents carefully, as they describe your rights | |||
and restrictions with respect to this document. Code Components | and restrictions with respect to this document. Code Components | |||
extracted from this document must include Revised BSD License text as | extracted from this document must include Revised BSD License text as | |||
described in Section 4.e of the Trust Legal Provisions and are | described in Section 4.e of the Trust Legal Provisions and are | |||
provided without warranty as described in the Revised BSD License. | provided without warranty as described in the Revised BSD License. | |||
Table of Contents | Table of Contents | |||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 | |||
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 | 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 | |||
2.1. Limitations of Current Packet Format Diagrams . . . . . . 4 | 2.1. Limitations of Current Packet Format Diagrams . . . . . . 4 | |||
2.2. Formal languages in standards documents . . . . . . . . . 7 | 2.2. Formal languages in standards documents . . . . . . . . . 7 | |||
3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7 | 3. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 7 | |||
4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 10 | 3.1. Defining Protocol Data Units . . . . . . . . . . . . . . 8 | |||
4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 10 | 3.2. Cross-Referencing Previously Defined Fields . . . . . . . 13 | |||
4.2. PDUs That Cross-Reference Previously Defined Fields . . . 13 | 3.3. Specifying Enumerated Types . . . . . . . . . . . . . . . 14 | |||
4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 16 | 3.4. Splitting Fields . . . . . . . . . . . . . . . . . . . . 15 | |||
4.4. PDUs with Constraints on Field Values . . . . . . . . . . 17 | 3.5. Extending Sub-Structures . . . . . . . . . . . . . . . . 15 | |||
4.5. PDUs with Constraints on Field Sizes . . . . . . . . . . 18 | 3.6. Storing Data for Parsing . . . . . . . . . . . . . . . . 18 | |||
4.6. PDUs That Extend Sub-Structures . . . . . . . . . . . . . 21 | 3.7. Connecting Structures with Functions . . . . . . . . . . 18 | |||
4.7. Storing Data for Parsing . . . . . . . . . . . . . . . . 22 | 3.8. Specifying Protocols . . . . . . . . . . . . . . . . . . 19 | |||
4.8. Connecting Structures with Functions . . . . . . . . . . 23 | 3.9. Importing Definitions from Other Documents . . . . . . . 20 | |||
4.9. Specifying Enumerated Types . . . . . . . . . . . . . . . 24 | 4. Design Principles . . . . . . . . . . . . . . . . . . . . . . 20 | |||
4.10. Specifying Protocol Data Units . . . . . . . . . . . . . 25 | 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 | |||
4.11. Importing PDU Definitions from Other Documents . . . . . 26 | 6. Security Considerations . . . . . . . . . . . . . . . . . . . 23 | |||
5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 26 | 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 | |||
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 | 8. Informative References . . . . . . . . . . . . . . . . . . . 23 | |||
7. Security Considerations . . . . . . . . . . . . . . . . . . . 26 | Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 25 | |||
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 27 | A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 25 | |||
9. Informative References . . . . . . . . . . . . . . . . . . . 27 | A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 26 | |||
Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 29 | Appendix B. Tooling & source code . . . . . . . . . . . . . . . 26 | |||
A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 29 | Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 | |||
A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 29 | ||||
Appendix B. Tooling & source code . . . . . . . . . . . . . . . 29 | ||||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 30 | ||||
1. Introduction | 1. Introduction | |||
Packet header diagrams have become a widely used format for | Packet header diagrams have become a widely used format for | |||
describing the syntax of binary protocols. In otherwise largely | describing the syntax of binary protocols. In otherwise largely | |||
textual documents, they allow for the visualisation of packet | textual documents, they allow for the visualisation of packet | |||
formats, reducing human error, and aiding in the implementation of | formats, reducing human error, and aiding in the implementation of | |||
parsers for the protocols that they specify. | parsers for the protocols that they specify. | |||
Figure 1 gives an example of how packet header diagrams are used to | Figure 1 gives an example of how packet header diagrams are used to | |||
skipping to change at page 7, line 44 ¶ | skipping to change at page 7, line 41 ¶ | |||
formal languages, including ABNF [RFC5234], ASN.1 [ASN1], C, CBOR | formal languages, including ABNF [RFC5234], ASN.1 [ASN1], C, CBOR | |||
[RFC7049], JSON, the TLS presentation language [RFC8446], YANG models | [RFC7049], JSON, the TLS presentation language [RFC8446], YANG models | |||
[RFC7950], and XML. While this broad range of languages may be | [RFC7950], and XML. While this broad range of languages may be | |||
problematic for the development of tooling to parse specifications, | problematic for the development of tooling to parse specifications, | |||
these, and other, languages serve a range of different use cases. | these, and other, languages serve a range of different use cases. | |||
ABNF, for example, is typically used to specify text protocols, while | ABNF, for example, is typically used to specify text protocols, while | |||
ASN.1 is used to specify data structure serialisation. This document | ASN.1 is used to specify data structure serialisation. This document | |||
specifies a structured language for specifying the parsing of binary | specifies a structured language for specifying the parsing of binary | |||
protocol data units. | protocol data units. | |||
3. Design Principles | 3. Augmented Packet Header Diagrams | |||
The use of structures that are designed to support machine | ||||
readability might potentially interfere with the existing ways in | ||||
which protocol specifications are used and authored. To the extent | ||||
that these existing uses are more important than machine readability, | ||||
such interference must be minimised. | ||||
In this section, the broad design principles that underpin the format | ||||
described by this document are given. However, these principles | ||||
apply more generally to any approach that introduces structured and | ||||
formal languages into standards documents. | ||||
It should be noted that these are design principles: they expose the | ||||
trade-offs that are inherent within any given approach. Violating | ||||
these principles is sometimes necessary and beneficial, and this | ||||
document sets out the potential consequences of doing so. | ||||
The central tenet that underpins these design principles is a | ||||
recognition that the standardisation process is not broken, and so | ||||
does not need to be fixed. Failure to recognise this will likely | ||||
lead to approaches that are incompatible with the standards process, | ||||
or that will see limited adoption. However, the standards process | ||||
can be improved with appropriate approaches, as guided by the | ||||
following broad design principles: | ||||
Most readers are human: Primarily, standards documents should be | ||||
written for people, who require text and diagrams that they can | ||||
understand. Structures that cannot be easily parsed by people | ||||
should be avoided, and if included, should be clearly delineated | ||||
from human-readable content. | ||||
Any approach that shifts this balance -- that is, that primarily | ||||
targets machine readers -- is likely to be disruptive to the | ||||
standardisation process, which relies upon discussion centered | ||||
around documents written in prose. | ||||
Writing tools are diverse: Standards document writing is a | ||||
distributed process that involves a diverse set of tools and | ||||
workflows. The introduction of machine-readable structures into | ||||
specifications should not require that specific tools are used to | ||||
produce standards documents, to ensure that disruption to existing | ||||
workflows is minimised. This does not preclude the development of | ||||
optional, supplementary tools that aid in the authoring machine- | ||||
readable structures. | ||||
The immediate impact of requiring specific tooling is that | ||||
adoption is likely to be limited. A long-term impact might be | ||||
that authors whose workflows are incompatible might be alienated | ||||
from the process. | ||||
Canonical specifications: As far as possible, machine-readable | ||||
structures should not replicate the human readable specification | ||||
of the protocol within the same document. Machine-readable | ||||
structures should form part of a canonical specification of the | ||||
protocol. Adding supplementary machine-readable structures, in | ||||
parallel to the existing human readable text, is undesirable | ||||
because it creates the potential for inconsistency. | ||||
As an example, program code that describes how a protocol data | ||||
unit can be parsed might be provided as an appendix within a | ||||
standards document. This code would provide a specification of | ||||
the protocol that is separate to the prose description in the main | ||||
body of the document. This has the undesirable effect of | ||||
introducing the potential for the program code to specify | ||||
behaviour that the prose-based specification does not, and vice- | ||||
versa. | ||||
Expressiveness: Any approach should be expressive enough to capture | ||||
the syntax and parsing process for the majority of binary | ||||
protocols. If a given language is not sufficiently expressive, | ||||
then adoption is likely to be limited. At the limits of what can | ||||
be expressed by the language, authors are likely to revert to | ||||
defining the protocol in prose: this undermines the broad goal of | ||||
using structured and formal languages. Equally, though, | ||||
understandable specifications and ease of use are critical for | ||||
adoption. A tool that is simple to use and addresses the most | ||||
common use cases might be preferred to a complex tool that | ||||
addresses all use cases. | ||||
It may be desirable to restrict expressiveness, however, to | ||||
guarantee intrinsic safety, security, and computability properties | ||||
of both the generated parser code for the protocol, and the parser | ||||
of the description language itself. In much the same way as the | ||||
language-theoretic security ([LANGSEC]) community advocates for | ||||
programming language design to be informed by the desired | ||||
properties of the parsers for those languages, protocol designers | ||||
should be aware of the implications of their design choices. The | ||||
expressiveness of the protocol description languages that they use | ||||
to define their protocols can force such awareness. | ||||
Broadly, those languages that have grammars which are more | ||||
expressive tend to have parsers that are more complex and less | ||||
safe. As a result, while considering the other goals described in | ||||
this document, protocol description languages should attempt to be | ||||
minimally expressive, and either restrict protocol designs to | ||||
those for which safe and secure parsers can be generated, or as a | ||||
minimum, ensure that protocol designers are aware of the | ||||
boundaries their designs cross, in terms of computability and | ||||
decidability [SASSAMAN]. | ||||
Minimise required change: Any approach should require as few changes | ||||
as possible to the way that documents are formatted, authored, and | ||||
published. Forcing adoption of a particular structured or formal | ||||
language is incompatible with the IETF's standardisation process: | ||||
there are very few components of standards documents that are non- | ||||
optional. | ||||
4. Augmented Packet Header Diagrams | ||||
The design principles described in Section 3 can largely be met by | ||||
the existing uses of packet header diagrams. These diagrams aid | ||||
human readability, do not require new or specialised tools to write, | ||||
do not split the specification into multiple parts, can express most | ||||
binary protocol features, and require no changes to existing | ||||
publication processes. | ||||
However, as discussed in Section 2.1 there are limitations to how | ||||
packet header diagrams are used that must be addressed if they are to | ||||
be parsed by machine. In this section, an augmented packet header | ||||
diagram format is described. | ||||
The concept is first illustrated by example. This is appropriate, | ||||
given the visual nature of the language. In future drafts, these | ||||
examples will be parsable using provided tools, and a formal | ||||
specification of the augmented packet diagrams will be given in | ||||
Appendix A. | ||||
4.1. PDUs with Fixed and Variable-Width Fields | As discussed in Section 2.1 there are limitations to how packet | |||
header diagrams are used that must be addressed if they are to be | ||||
parsed by machine. In this section, an augmented packet header | ||||
diagram format is described. The principles that underpin the design | ||||
of this format are discussed in Section 4. | ||||
The simplest PDU is one that contains only a set of fixed-width | The concept is illustrated by example, with accompanying explanatory | |||
fields in a known order, with no optional fields or variation in the | descriptions. This is appropriate, given the visual nature of the | |||
packet format. | language. In future drafts a formal specification of the format will | |||
be given in Appendix A. | ||||
Some packet formats include variable-width fields, where the size of | Our examples are drawn from the specifications of TCP [RFC9293], STUN | |||
a field is either derived from the value of some previous field, or | [RFC8489], and QUIC [QUIC-TRANSPORT]. These examples are | |||
is unspecified and inferred from the total size of the packet and the | illustrative only of the Augmented Packet Header Diagram format that | |||
size of the other fields. | we define in this document, and they do not necessarily reflect the | |||
current state of the specifications they are taken from. For | ||||
example, the published QUIC specification [RFC9000] does not use | ||||
packet header diagrams to describe the syntax of the protocol. | ||||
To ensure that there is no ambiguity, a PDU description can contain | 3.1. Defining Protocol Data Units | |||
only one field whose length is unspecified. The length of a single | ||||
field, where all other fields are of known (but perhaps variable) | ||||
length, can be inferred from the total size of the containing PDU. | ||||
A PDU description is introduced by the exact phrase "A/An _______ is | A PDU description is introduced by the exact phrase "A/An _______ is | |||
formatted as follows" within a paragraph. Optionally, this phrase | formatted as follows" within a paragraph. Optionally, this phrase | |||
can include a note or comment, delimited by commas, immediately | can include a note or comment, delimited by commas, immediately | |||
following the PDU name. That is, "A/An _______, with an optional | following the PDU name. That is, "A/An _______, with an optional | |||
comment, is formatted as follows" can be used to introduce a PDU | comment, is formatted as follows" can be used to introduce a PDU | |||
description. The introductory phrase is followed by the PDU | description. The introductory phrase is followed by the PDU | |||
description itself, as a packet diagram within an <artwork> element | description itself, as a packet diagram within an <artwork> element | |||
(itself optionally within a <figure> element) in the XML | (itself optionally within a <figure> element) in the XML | |||
representation, starting with a header line to show the bit width of | representation, starting with a header line to show the bit width of | |||
the diagram. The description of the fields follows the diagram, as | the diagram. The description of the fields follows the diagram, as | |||
an XML list (either <dl> or hanging <list>), after a paragraph that | an XML list (either <dl> or hanging <list>), after a paragraph that | |||
begins with the text "where:". | begins with the text "where:". | |||
PDU names must be unique, both within a document, and across all | PDU names must be unique, both within a document, and across all | |||
documents that are linked together (i.e., using the structured | documents that are linked together (i.e., using the structured | |||
language defined in Section 4.11). | language defined in Section 3.9). | |||
Each field is defined by a structured text definition and a prose | Each field is defined by a structured text definition and a prose | |||
description. The structured text definition comprises the field name | description. The structured text definition comprises the field name | |||
and an optional short name in parenthesis. These are followed by a | and an optional short name in parenthesis. These are followed by a | |||
colon, the field length, an optional presence expression (described | colon, the field length, an optional expression constraining the | |||
in Section 4.2), and, optionally, a terminating period. Text | value of the field, an optional presence constraint, and, optionally, | |||
following the terminating period is not parsed, and this space can be | a terminating period. Text following the terminating period is not | |||
used for optional notes or comments. Field names cannot be the same | parsed, and this space can be used for optional notes or comments. | |||
as a previously defined PDU name, and must be unique within a given | Field names cannot be the same as a previously defined PDU name, and | |||
structure definition. The structured text definition is given either | must be unique within a given structure definition. The structured | |||
in a <dt> tag (if using a <dl>) or as the "hangText" (if using a | text definition is given either in a <dt> tag (if using a <dl>) or as | |||
hanging <list>) of a <t> element. The field's prose description is | the "hangText" (if using a hanging <list>) of a <t> element. The | |||
given in the following <dd> element or within the same <t> element. | field's prose description is given in the following <dd> element or | |||
Prose descriptions may include structured text (e.g., as defined in | within the same <t> element. Prose descriptions may include | |||
Section 4.7). | structured text (e.g., as defined in Section 3.6). | |||
For example, this can be illustrated using the IPv4 Header Format | ||||
[RFC791]. An IPv4 Header is formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
|Version| IHL | DSCP |ECN| Total Length | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Identification |Flags| Fragment Offset | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Time to Live | Protocol | Header Checksum | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Source Address | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Destination Address | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Options ... | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| : | ||||
: Payload : | ||||
: | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Version (V): 4 bits. This is a fixed-width field, whose full label | ||||
is shown in the diagram. The field's width -- 4 bits -- is given | ||||
in the label of the description list, separated from the field's | ||||
label by a colon. | ||||
Internet Header Length (IHL): 4 bits. This is a shorter field, whose | ||||
full label is too large to be shown in the diagram. A short label | ||||
(IHL) is used in the diagram, and this short label is provided, in | ||||
brackets, after the full label in the description list. | ||||
Differentiated Services Code Point (DSCP): 6 bits. This is a fixed- | ||||
width field, as previously discussed. | ||||
Explicit Congestion Notification (ECN): 2 bits. This is a fixed- | ||||
width field, as previously discussed. | ||||
Total Length (TL): 2 bytes. This is a fixed-width field, as | ||||
previously discussed. Where fields are an integral number of | ||||
bytes in size, the field length can be given in bytes rather than | ||||
in bits. | ||||
Identification: 2 bytes. This is a fixed-width field, as previously | ||||
discussed. | ||||
Flags: 3 bits. This is a fixed-width field, as previously discussed. | ||||
Fragment Offset: 13 bits. This is a fixed-width field, as previously | ||||
discussed. | ||||
Time to Live (TTL): 1 byte. This is a fixed-width field, as | ||||
previously discussed. | ||||
Protocol: 1 byte. This is a fixed-width field, as previously | ||||
discussed. | ||||
Header Checksum: 2 bytes. This is a fixed-width field, as previously | ||||
discussed. | ||||
Source Address: 32 bits. This is a fixed-width field, as previously | ||||
discussed. | ||||
Destination Address: 32 bits. This is a fixed-width field, as | ||||
previously discussed. | ||||
Options: (IHL-5)*32 bits. This is a variable-length field, whose | ||||
length is defined by the value of the field with short label IHL | ||||
(Internet Header Length). Constraint expressions can be used in | ||||
place of constant values: the grammar for the expression language | ||||
is defined in Appendix A.1. Constraints can include a previously | ||||
defined field's short or full label, where one has been defined. | ||||
Short variable-length fields are indicated by "..." instead of a | ||||
pipe at the end of the row. | ||||
Payload: TL - ((IHL*32)/8) bytes. This is a multi-row variable- | ||||
length field, constrained by the values of fields TL and IHL. | ||||
Instead of the "..." notation, ":" is used to indicate that the | ||||
field is variable-length. The use of ":" instead of "..." | ||||
indicates the field is likely to be a longer, multi-row field. | ||||
However, semantically, there is no difference: these different | ||||
notations are for the benefit of human readers. | ||||
4.2. PDUs That Cross-Reference Previously Defined Fields | ||||
Binary formats often reference sub-structures that have been defined | ||||
earlier in the specification. For example, in RTP [RFC3550], the | ||||
Contributing Source Identifiers in an RTP Data Packet are defined as | ||||
comprising a list of Source Identifier elements. A Source Identifier | ||||
is formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| SSRC | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
SSRC: 32 bits. This is a fixed-width field, as described previously. | ||||
The following example shows how a Source Identifier can be referenced | ||||
in the description of an RTP Data Packet. It also shows how the | ||||
presence of some fields in a format may be dependent on the values of | ||||
an earlier field. | ||||
An RTP Data Packet is formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| V |P|X| CC |M| PT | Sequence Number | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Timestamp | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Synchronization Source identifier | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| [Contributing Source identifiers] | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Header Extension | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Payload : | ||||
: : | ||||
: | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Padding | Padding Count | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Version (V): 2 bits. This is a fixed-width field, as described | ||||
previously. | ||||
Padding (P): 1 bit. This is a fixed-width field, as described | ||||
previously. | ||||
Extension (X): 1 bit. This is a fixed-width field, as described | ||||
previously. | ||||
CSRC count (CC): 4 bits. This is a fixed-width field, as described | ||||
previously. | ||||
Marker (M): 1 bit. This is a fixed-width field, as described | ||||
previously. | ||||
Payload Type (PT): 7 bits. This is a fixed-width field, as described | ||||
previously. | ||||
Sequence Number (PT): 16 bits. This is a fixed-width field, as | ||||
described previously. | ||||
Timestamp (PT): 32 bits. This is a fixed-width field, as described | ||||
previously. | ||||
Synchronization Source identifier: 1 Source Identifier. This is a | ||||
field whose structure is a previously defined PDU format (Source | ||||
Identifier). To indicate this, the width of the field is | ||||
expressed in terms of cross-referenced structure. When used in | ||||
constraint expressions, PDU names refer to the length of that PDU | ||||
structure. | ||||
Contributing Source identifiers: CC Source Identifier. Where a field | ||||
is comprised of a sequence of previously defined structures, | ||||
square brackets can be used to indicate this in the diagram. The | ||||
length of the sequence can be defined using the constraint | ||||
expression grammar as described earlier. Where the length is | ||||
unknown, the type of each element of the sequence must be given in | ||||
square brackets. | ||||
In this example, both a PDU name (Source Identifier) and a field | ||||
name (CC) are used in the constraint expression. The PDU name | ||||
refers to the length of the PDU, while the field name refers to | ||||
the value of the field. This is possible because field names | ||||
cannot be the same as previously defined PDU names. | ||||
Header Extension: 32 bits; present only when X == 1. This is a field | ||||
whose presence is predicated on an expression given using the | ||||
constraint expression grammar described earlier. Optional fields | ||||
can be of any previously defined format (e.g., fixed- or variable- | ||||
width). Optional fields are indicated by the presence of "; | ||||
present only when [expr]." at the end of the definition term | ||||
(i.e., the text contained within the <dt> tag or "hangText" | ||||
attribute). | ||||
[Note that this example deviates from the format as described in | ||||
[RFC3550]. As specified in that document, the Header Extension | ||||
would be a cross-referenced structure. This is not shown here for | ||||
brevity.] | ||||
Payload. The length of the Payload is not specified, and hence needs | ||||
to be inferred from the total length of the packet and the lengths | ||||
of the known fields. There can only be one field of unspecified | ||||
size in a PDU. Fields where the length is not specified may also | ||||
denote this with the phrase "variable length" in place of the | ||||
length definition. | ||||
Padding: PC bytes; present only when (P == 1) && (PC > 0). Note | ||||
that PC is defined below. This is a variable size field, with size | ||||
dependent on a later field in the packet. Fields can only depend | ||||
on the value of a later field if they follow a field with | ||||
unspecified size. | ||||
The comment (i.e., "Note that PC is defined below") following the | ||||
expression is not parsed. | ||||
Padding Count (PC): 1 byte; present only when P == 1. This is a | ||||
fixed-width field, as previously discussed. | ||||
4.3. PDUs with Non-Contiguous Fields | ||||
In some binary formats, fields are striped across multiple non- | ||||
contiguous bits. This is often to allow for backwards compatibility | ||||
with previous definitions of the same fields in earlier documents: | ||||
striping in this way allows for careful use of the possible range of | ||||
values. | ||||
This format is illustrated using the STUN Message Type | ||||
[draft-ietf-tram-stunbis-21]. A STUN Message Type is formatted as | ||||
follows: | ||||
0 1 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
|M|M|M|M|M|C|M|M|M|C|M|M|M|M| | ||||
|B|A|9|8|7|1|6|5|4|0|3|2|1|0| | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Method (M): 12 bits (split field). This field is comprised of | ||||
multiple sub-fields (M0 through MB) as shown in the diagram. That | ||||
these sub-fields should be concatenated, after parsing, into a | ||||
single field is indicated by their being labelled using the 'M' | ||||
short field name followed by a single hexadecimal digit, with the | ||||
least significant bit labelled with 0, and subsequent bits | ||||
labelled in sequence. | ||||
Class (C): 2 bits (split field). This field follows the same format | ||||
as M described above. | ||||
4.4. PDUs with Constraints on Field Values | ||||
A PDU may be defined not only by the layout and type of its fields, | A PDU may be defined not only by the layout and type of its fields, | |||
but also by the value of those fields. For example, field values may | but also by the value of those fields. For example, field values may | |||
be constrained to be of a known exact value or to be within a range. | be constrained to be of a known exact value or to be within a range. | |||
More generally, our format enables a boolean expression to be | The "Data Offset" and "Reserved" fields in the example below make use | |||
attached to a field, which must be true for the PDU to be parsed | of value constraints. More generally, our format enables a boolean | |||
successfully. | expression to be attached to a field, which must be true for the PDU | |||
to be parsed successfully. | ||||
This format is illustrated using the QUIC Long Header Packet format | ||||
[QUIC-TRANSPORT]. A Long Header is formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+ | ||||
|1|1| T | R | P | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Version | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| DCID Len | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Destination Connection ID (DCID) ... | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| SCID Len | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Source Connection ID (SCID) ... | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Header Form (HF): 1 bit; HF == 1. This is a fixed-width field, | ||||
constrained to be a of an known, exact value. At most one field | ||||
value constraint may be given, and if provided, it must be given | ||||
as a boolean expression, separated by a semi-colon in the field | ||||
definition name (i.e., the text contained within the <dt> tag or | ||||
"hangText" attribute). If present, a value constraint must follow | ||||
the name, short name, and length of the field, but appear before | ||||
any presence constraint, if applicable. The order of the field | ||||
must be the same in both the diagram and description list. | ||||
Fixed Bit (FB): 1 bit; FB == 1. This is a fixed-width field, with a | ||||
value constraint, as previously described. | ||||
Long Packet Type (T): 2 bits. This is a fixed-width field as | ||||
previously described. | ||||
Reserved Bits (R): 2 bits. This is a fixed-width field as previously | ||||
described. | ||||
Packet Number Length (P): 2 bits. This is a fixed-width field as | ||||
previously described. | ||||
Version: 32 bits. This is a fixed-width field as previously | ||||
described. | ||||
DCID Len (DLen): 1 byte; DLen <= 20. This is a fixed-width field, | ||||
with a value constraint, as previously described. Note that the | ||||
constraint language is not limited to equality; it is defined | ||||
fully in Appendix A.1. | ||||
Destination Connection ID: DLen bytes. This is a variable-width | ||||
field as previously described. | ||||
SCID Len (SLen): 1 byte; SLen <= 20. This is a fixed-width field, | In addition, a PDU may contain fields that have a size that is | |||
with a value constraint, as previously described. | specified in terms of the value of another field. Our constraint | |||
syntax can be used to specify the length of fields in known units (of | ||||
bits, bytes, or other structures). In the example below, the | ||||
"Options" field is defined in units of "TCP Option" structures, and | ||||
this is indicated by square brackets in the diagram and description | ||||
list. | ||||
Source Connection ID: SLen bytes. This is a variable-width field as | If the units are of variable-width, then it may not be possible to | |||
previously described. | specify the length of the sequence. However, it is still necessary | |||
to be able to constrain the overall width of the field. To support | ||||
this, our constraint syntax includes a "size" function that evaluates | ||||
to the width, in bits, of the given named field. The "Options" field | ||||
in the example below makes use of this syntax to constrain the size | ||||
of the field. | ||||
4.5. PDUs with Constraints on Field Sizes | Finally, the presence of a field in a PDU may depend on the value of | |||
other fields in that PDU. As shown by the "Options" field in the | ||||
example below, a constraint expression can be attached to each field, | ||||
where that field is only present in the PDU when the expression is | ||||
true. | ||||
A PDU may contain fields that have a size that is specified in terms | We define an ABNF grammar for constraint expressions in Appendix A.1. | |||
of the value of another field. So far, our constraint syntax can be | This grammar is used across value, size, and presence constraint | |||
used to specify the length of fields in known units (of bits, bytes, | expressions. | |||
or other structures). If the units are of variable-width, then it | ||||
may not be possible to specify the length of the sequence. However, | ||||
it is still necessary to be able to constraint the overall width of | ||||
the field. To support this, our constraint syntax includes a "size" | ||||
function that evaluates to the width, in bits, of the given named | ||||
field. | ||||
This syntax is illustrated using the TCP Header format [RFC9293]. A | These elements can be illustrated using the TCP Header format | |||
TCP Header is formatted as follows: | [RFC9293]. A TCP Header is formatted as follows: | |||
0 1 2 3 | 0 1 2 3 | |||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| Source Port | Destination Port | | | Source Port | Destination Port | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| Sequence Number | | | Sequence Number | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| Acknowledgment Number | | | Acknowledgment Number | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
skipping to change at page 19, line 29 ¶ | skipping to change at page 10, line 29 ¶ | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| [Options] | | | [Options] | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| : | | : | |||
: Payload : | : Payload : | |||
: | | : | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
where: | where: | |||
Source Port: 16 bits. This is a fixed-width field as previously | Source Port: 16 bits. This is a fixed-width field, whose full label | |||
described. | is shown in the diagram. The field's width -- 16 bits -- is given | |||
in the label of the description list, separated from the field's | ||||
label by a colon. | ||||
Destination Port: 16 bits. This is a fixed-width field as previously | Destination Port: 2 bytes. This is a fixed-width field as previously | |||
described. | described. Where fields are an integral number of bytes in size, | |||
the field length can be given in bytes rather than in bits. | ||||
Sequence Number: 32 bits. This is a fixed-width field as previously | Sequence Number: 32 bits. This is a fixed-width field as previously | |||
described. | described. | |||
Acknowledgment Number: 32 bits. This is a fixed-width field as | Acknowledgment Number: 32 bits. This is a fixed-width field as | |||
previously described. | previously described. | |||
Data Offset (DOffset): 4 bits; DOffset >= 5. This is a fixed-width | Data Offset (DOffset): 4 bits; DOffset >= 5. This is a fixed-width | |||
field, with a value constraint, as previously described. | field, with a constraint on its value. At most one field value | |||
constraint may be given per field, and if provided, it must be | ||||
given as a boolean expression, separated by a semi-colon in the | ||||
field definition name (i.e., the text contained within the <dt> | ||||
tag or "hangText" attribute). If present, a value constraint must | ||||
follow the name, short name, and length of the field, but appear | ||||
before any presence constraint, if applicable. The order of the | ||||
field must be the same in both the diagram and description list. | ||||
Reserved (Rsrvd): 4 bits; Rsrvd == 0. This is a fixed-width field, | Reserved (Rsrvd): 4 bits; Rsrvd == 0. This is a fixed-width field, | |||
with a value constraint, as previously described. | with a value constraint, as previously described. This is a | |||
shorter field, whose full label is too large to be shown in the | ||||
diagram. A short label (Rsrvd) is used in the diagram, and this | ||||
short label is provided, in brackets, after the full label in the | ||||
description list. | ||||
Control bits: Optionally, field description lists can be nested. If | Control bits: Optionally, field description lists can be nested. If | |||
the XML element (either a <dd> or a <t>) containing the | the XML element (either a <dd> or a <t>) containing the | |||
description of a field contains an XML list (either <dl> or | description of a field contains an XML list (either <dl> or | |||
hanging <list>) as its last element, then this nested list will be | hanging <list>) as its last element, then this nested list will be | |||
parsed for fields, and the outer description will be ignored. In | parsed for fields, and the outer description will be ignored. In | |||
this example, "Control bits" describes the group of 8 single bit | this example, "Control bits" describes the group of 8 single bit | |||
fields that are described in the list that follows; it is these | fields that are described in the list that follows; it is these | |||
single bit fields that will form part of the structure. | single bit fields that will form part of the structure. | |||
skipping to change at page 20, line 42 ¶ | skipping to change at page 11, line 51 ¶ | |||
Checksum: 16 bits. This is a fixed-width field as previously | Checksum: 16 bits. This is a fixed-width field as previously | |||
described. | described. | |||
Urgent Pointer: 16 bits. This is a fixed-width field as previously | Urgent Pointer: 16 bits. This is a fixed-width field as previously | |||
described. | described. | |||
Options: [TCP Option]; size(Options) == (DOffset-5)*32; present | Options: [TCP Option]; size(Options) == (DOffset-5)*32; present | |||
only when DOffset > 5. This is a variable-width field that is | only when DOffset > 5. This is a variable-width field that is | |||
comprised of a sequence of TCP Option sub-structures. TCP Option | comprised of a sequence of TCP Option sub-structures. TCP Option | |||
is an enumerated type, to be defined in Section 4.9. As defined, | is an enumerated type, to be defined in Section 3.3. As defined, | |||
the TCP Option type can be either 2 or 3 bytes, depending on the | the TCP Option type can be either 2 or 3 bytes, depending on the | |||
option type. As a result, it is not possible to specify the | option type. As a result, it is not possible to specify the | |||
number of TCP Option structures that the Option field will | number of TCP Option structures that the Option field will | |||
contain. However, the overall size of the field can be | contain. However, the overall size of the field can be | |||
constrained. The "size(Options) == (DOffset-5)*32" makes use of | constrained. The "size(Options) == (DOffset-5)*32" makes use of | |||
the "size" function. This evaluates to the size, in bits, of the | the "size" function. This evaluates to the size, in bits, of the | |||
named field. The argument passed to the "size" field must be the | named field. The argument passed to the "size" field must be the | |||
name of the field being defined, or of a previously defined field. | name of the field being defined, or of a previously defined field. | |||
The "DOffset" field contains the number of 32-bit words that are | The "DOffset" field contains the number of 32-bit words that are | |||
present in the TCP Header. By default, with no TCP options, this | present in the TCP Header. By default, with no TCP options, this | |||
is 5. As a result, the size of the Options field is constrained | is 5. As a result, the size of the Options field is constrained | |||
to the value of DOffset, less 5, and multiplied to get the value | to the value of DOffset, less 5, and multiplied to get the value | |||
in bits. | in bits. | |||
Payload. This is a variable-width field as previously described. | The presence of the "Options" field is predicated on an | |||
expression. Optional fields are indicated by the presence of "; | ||||
present only when [expr]." at the end of the definition term | ||||
(i.e., the text contained within the <dt> tag or "hangText" | ||||
attribute). | ||||
4.6. PDUs That Extend Sub-Structures | Payload. This is a multi-row variable-length field, denoted in the | |||
diagram by the ":" notation in the field's border. The length of | ||||
the Payload is not specified, and hence needs to be inferred from | ||||
the total length of the packet and the lengths of the known | ||||
fields. There can only be one field of unspecified size in a PDU | ||||
Fields where the length is not specified may also denote this with | ||||
the phrase "variable length" in place of the length definition. | ||||
The simplest PDU is one that contains only a set of fixed-width | ||||
fields in a known order, with no optional fields or variation in the | ||||
packet format. | ||||
Some packet formats include variable-width fields (e.g., the | ||||
"Options" field in the example above), where the size of a field is | ||||
either derived from the value of some previous field, or is | ||||
unspecified and inferred from the total size of the packet and the | ||||
size of the other fields. | ||||
To ensure that there is no ambiguity, a PDU description can contain | ||||
only one field whose length is unspecified. The length of a single | ||||
field, where all other fields are of known (but perhaps variable) | ||||
length, can be inferred from the total size of the containing PDU. | ||||
For example, the "Payload" field in the example above is unspecified; | ||||
its length can be determined by subtracting the length of the other | ||||
fields from the total size of the PDU. | ||||
3.2. Cross-Referencing Previously Defined Fields | ||||
Binary formats often reference sub-structures that have been defined | ||||
earlier in the specification. For example, in TCP [RFC9293], the | ||||
SACK Range Option (a TCP option type, as will be discussed in | ||||
Section 3.3) is defined in terms of of SACK blocks. A SACK Block is | ||||
formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Left Edge | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Right Edge | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Left Edge: 4 bytes. This is a fixed-width field, as described | ||||
previously. | ||||
Right Edge: 4 bytes. This is a fixed-width field, as described | ||||
previously. | ||||
The SACK Block sub-structure is then used in the definition of the | ||||
SACK Range Option. | ||||
A SACK Range Option is formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| 5 | Length | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| [Blocks] | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Option Kind (Kind): 1 byte; Kind == 5. This is a fixed-width field, | ||||
as described previously. | ||||
Option Length (Length): 1 byte. This is a fixed-width field, as | ||||
described previously. | ||||
Blocks: (Length-2)/8 SACK Blocks. Where a field is comprised of a | ||||
sequence of previously defined structures, square brackets can be | ||||
used to indicate this in the diagram. The length of the sequence | ||||
can be defined using the constraint expression grammar as | ||||
described earlier. Where the length is unknown, the type of each | ||||
element of the sequence must be given in square brackets. | ||||
In this example, both a PDU name (SACK Block) and a field name | ||||
(Length) are used in the constraint expression. This is possible | ||||
because field names cannot be the same as previously defined PDU | ||||
names. | ||||
3.3. Specifying Enumerated Types | ||||
In addition to the use of the sub-structures, it is desirable to be | ||||
able to define a type that may take the value of one of a set of | ||||
alternative structures. | ||||
The alternative structures that comprise an enumerated type are | ||||
identified using the exact phrase "The <enumerated type name> is one | ||||
of: <list of structure names>" where the list of structure names is a | ||||
comma separated list (with the last element, if there is more than | ||||
one element, preceded by 'or'), each optionally preceded by "a" or | ||||
"an". The structure names must be defined within the document or a | ||||
linked document. Optionally, this phrase can include a note or | ||||
comment, delimited by commas, immediately following the enumerated | ||||
type name. That is, "The <enumerated type name>, with an optional | ||||
comment, is one of: <list of structure names>" can be used to define | ||||
an enumerated type. In both cases, the colon is optional; for | ||||
example, "The <enumerated type name> is one of <list of structure | ||||
names>" is valid. | ||||
Where an enumerated type has only two variants, an alternative phrase | ||||
can be used: "The <enumerated type name> is either a <variant 1 name> | ||||
or <variant 2 name>". The names of the variants must be defined | ||||
within the document or a linked document. An optional note or | ||||
comment can be included with this alternative phrasing: "The | ||||
<enumerated type name>, with an optional comment, is either a | ||||
<variant 1 name> or <variant 2 name>" can be used. | ||||
An EOL Option is formatted as follows: | ||||
0 | ||||
0 1 2 3 4 5 6 7 | ||||
+-+-+-+-+-+-+-+-+ | ||||
| 0 | | ||||
+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Option Kind (Kind): 1 byte; Kind == 0. This is a fixed-width field, | ||||
with a value constraint, as previously described. | ||||
A TCP Option is either an EOL Option or a SACK Range Option. | ||||
3.4. Splitting Fields | ||||
In some binary formats, fields are striped across multiple non- | ||||
contiguous bits. This is often to allow for backwards compatibility | ||||
with previous definitions of the same fields in earlier documents: | ||||
striping in this way allows for careful use of the possible range of | ||||
values. | ||||
This format is illustrated using the STUN Message Type [RFC8489]. A | ||||
STUN Message Type is formatted as follows: | ||||
0 1 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
|M|M|M|M|M|C|M|M|M|C|M|M|M|M| | ||||
|B|A|9|8|7|1|6|5|4|0|3|2|1|0| | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Method (M): 12 bits (split field). This field is comprised of | ||||
multiple sub-fields (M0 through MB) as shown in the diagram. That | ||||
these sub-fields should be concatenated, after parsing, into a | ||||
single field is indicated by their being labelled using the 'M' | ||||
short field name followed by a single hexadecimal digit, with the | ||||
least significant bit labelled with 0, and subsequent bits | ||||
labelled in sequence. | ||||
Class (C): 2 bits (split field). This field follows the same format | ||||
as M described above. | ||||
3.5. Extending Sub-Structures | ||||
A PDU may not only use or reference existing sub-structures, but they | A PDU may not only use or reference existing sub-structures, but they | |||
may extend them, adding new fields, or enforcing different or | may extend them, adding new fields, or enforcing different or | |||
additional constraints. | additional constraints. | |||
Where a sub-structure is extended, the diagram may show the sub- | Where a sub-structure is extended, the diagram may show the sub- | |||
structure as a block, labelled with the sub-structure name. It may | structure as a block, labelled with the sub-structure name. It may | |||
also be desirable to show the sub-structure diagram in full; in this | also be desirable to show the sub-structure diagram in full; in this | |||
case, the fields must be given in the same order and be of the same | case, the fields must be given in the same order and be of the same | |||
length. New field constraints can be shown. Similarly, in the | length. New field constraints can be shown. Similarly, in the | |||
description list, those fields inherited without change (i.e., with | description list, those fields inherited without change (i.e., with | |||
no change to their constraints) do not need to be repeated. Those | no change to their constraints) do not need to be repeated. Those | |||
with different or additional constraints must be described, and the | with different or additional constraints must be described, and the | |||
order of the fields in the description list must match that of the | order of the fields in the description list must match that of the | |||
sub-structure and the containing structure. | sub-structure and the containing structure. | |||
This format is illustrated using the QUIC Retry Packet format | This can be illustrated using QUIC [QUIC-TRANSPORT]. A Long Header | |||
[QUIC-TRANSPORT]. A Retry Packet is formatted as follows: | is formatted as follows: | |||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+ | ||||
|1|1| T | R | P | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Version | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| DCID Len | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Destination Connection ID (DCID) ... | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| SCID Len | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| Source Connection ID (SCID) ... | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Header Form (HF): 1 bit; HF == 1. This is a fixed-width field, with | ||||
a value constraint, as previously described. | ||||
Fixed Bit (FB): 1 bit; FB == 1. This is a fixed-width field, with a | ||||
value constraint, as previously described. | ||||
Long Packet Type (T): 2 bits. This is a fixed-width field as | ||||
previously described. | ||||
Reserved Bits (R): 2 bits. This is a fixed-width field as previously | ||||
described. | ||||
Packet Number Length (P): 2 bits. This is a fixed-width field as | ||||
previously described. | ||||
Version ID (VID): 32 bits. This is a fixed-width field as previously | ||||
described. | ||||
DCID Len (DLen): 1 byte; DLen <= 20. This is a fixed-width field, | ||||
with a value constraint, as previously described. | ||||
Destination Connection ID (DCID): DLen bytes. This is a fixed-width | ||||
field, with a length constraint, as previously described. | ||||
SCID Len (SLen): 1 byte; SLen <= 20. This is a fixed-width field, | ||||
with a value constraint, as previously described. | ||||
Source Connection ID (SCID): SLen bytes. This is a variable-width | ||||
field as previously described. | ||||
The syntax for extending sub-structures can be illustrated with the | ||||
QUIC Retry Packet format [QUIC-TRANSPORT]. A Retry Packet is | ||||
formatted as follows: | ||||
0 1 2 3 | 0 1 2 3 | |||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| : | | : | |||
: Long Header : | : Long Header : | |||
: | | : | | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
| Retry Token ... | | Retry Token ... | |||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |||
skipping to change at page 22, line 19 ¶ | skipping to change at page 18, line 5 ¶ | |||
Retry Token. This is a variable-length field as previously defined. | Retry Token. This is a variable-length field as previously defined. | |||
Retry Integrity Tag: 128 bits. This is a fixed-width field as | Retry Integrity Tag: 128 bits. This is a fixed-width field as | |||
previously defined. | previously defined. | |||
As shown, the Long Header packet sub-structure is included. The | As shown, the Long Header packet sub-structure is included. The | |||
Retry Packet enforces a new value constraint on the Long Packet Type | Retry Packet enforces a new value constraint on the Long Packet Type | |||
(T) field. | (T) field. | |||
4.7. Storing Data for Parsing | 3.6. Storing Data for Parsing | |||
The parsing process may require data from previously parsed | The parsing process may require data from previously parsed | |||
structures. This means that data needs to be stored persistently | structures. This means that data needs to be stored persistently | |||
throughout the process. This data needs to be identified. | throughout the process. This data needs to be identified. | |||
That the value of a particular field be stored upon parsing is | That the value of a particular field be stored upon parsing is | |||
indicated by the exact phrase "On receipt, the value of <field name> | indicated by the exact phrase "On receipt, the value of <field name> | |||
is stored as <stored name>." being present at the end of the | is stored as <stored name>." being present at the end of the | |||
description of a field (i.e., at the end of the <dd> or <t> element.) | description of a field (i.e., at the end of the <dd> or <t> element.) | |||
skipping to change at page 23, line 5 ¶ | skipping to change at page 18, line 35 ¶ | |||
where: | where: | |||
Long Header (LH): 1 Long Header; LH.T == 0. This is field is a sub- | Long Header (LH): 1 Long Header; LH.T == 0. This is field is a sub- | |||
structure, with a constraint, as previously defined. On receipt, | structure, with a constraint, as previously defined. On receipt, | |||
the value of LH.DCID is stored as Initial DCID. | the value of LH.DCID is stored as Initial DCID. | |||
In this example, the value of the DCID field of the Long Header sub- | In this example, the value of the DCID field of the Long Header sub- | |||
structure is stored as Initial DCID. | structure is stored as Initial DCID. | |||
4.8. Connecting Structures with Functions | 3.7. Connecting Structures with Functions | |||
The parsing or serialisation of some binary formats cannot be fully | The parsing or serialisation of some binary formats cannot be fully | |||
described without the use of functions. These functions take | described without the use of functions. These functions take | |||
arguments (values from another structure), perform some computation, | arguments (values from another structure), perform some computation, | |||
and generate a new structure. | and generate a new structure. | |||
Given the goal of fully capturing the parsing or serialisation of | Given the goal of fully capturing the parsing or serialisation of | |||
binary protocols, it is necessary to include the signature of these | binary protocols, it is necessary to include the signature of these | |||
helper functions. | helper functions. | |||
skipping to change at page 23, line 28 ¶ | skipping to change at page 19, line 10 ¶ | |||
the function. This is immediately followed by a set of brackets | the function. This is immediately followed by a set of brackets | |||
containing a comma separated list of the function's parameters, | containing a comma separated list of the function's parameters, | |||
formatted as "<parameter name>: <parameter type>". This is followed | formatted as "<parameter name>: <parameter type>". This is followed | |||
by "->" and the return type of the function, followed by a colon. | by "->" and the return type of the function, followed by a colon. | |||
The body of the function is not captured, owing to the complexity of | The body of the function is not captured, owing to the complexity of | |||
both capturing and translating arbitrary code. As a result, it can | both capturing and translating arbitrary code. As a result, it can | |||
be described in whichever format is most suitable for the document | be described in whichever format is most suitable for the document | |||
and its readership. | and its readership. | |||
Those values that are stored persistently, as defined in Section 4.7, | Those values that are stored persistently, as defined in Section 3.6, | |||
are accessible by functions. | are accessible by functions. | |||
As an example, the "apply_protection" function is defined as: | As an example, the "apply_protection" function is defined as: | |||
func apply_protection(to: Unprotected Packet) | func apply_protection(to: Unprotected Packet) | |||
-> Protected Packet: | -> Protected Packet: | |||
apply packet protection to payload | apply packet protection to payload | |||
apply header protection to first_byte and packet_number | apply header protection to first_byte and packet_number | |||
construct appropriate Protected Packet based on first_byte | construct appropriate Protected Packet based on first_byte | |||
return Protected Packet | return Protected Packet | |||
skipping to change at page 24, line 8 ¶ | skipping to change at page 19, line 39 ¶ | |||
must take a single parameter, of the same type as PDU B, and return a | must take a single parameter, of the same type as PDU B, and return a | |||
PDU B. | PDU B. | |||
To indicate that a PDU can be serialised to another by way of a | To indicate that a PDU can be serialised to another by way of a | |||
function, the sentence "A/An <PDU name A> is serialised to a <PDU | function, the sentence "A/An <PDU name A> is serialised to a <PDU | |||
name B> using the <function name> function" is used. This indicates | name B> using the <function name> function" is used. This indicates | |||
that a PDU B is generated by passing PDU A into the named function. | that a PDU B is generated by passing PDU A into the named function. | |||
The function must take a single parameter, of the same type as PDU A, | The function must take a single parameter, of the same type as PDU A, | |||
and return a PDU B. | and return a PDU B. | |||
4.9. Specifying Enumerated Types | 3.8. Specifying Protocols | |||
In addition to the use of the sub-structures, it is desirable to be | ||||
able to define a type that may take the value of one of a set of | ||||
alternative structures. | ||||
The alternative structures that comprise an enumerated type are | ||||
identified using the exact phrase "The <enumerated type name> is one | ||||
of: <list of structure names>" where the list of structure names is a | ||||
comma separated list (with the last element, if there is more than | ||||
one element, preceded by 'or'), each optionally preceded by "a" or | ||||
"an". The structure names must be defined within the document or a | ||||
linked document. Optionally, this phrase can include a note or | ||||
comment, delimited by commas, immediately following the enumerated | ||||
type name. That is, "The <enumerated type name>, with an optional | ||||
comment, is one of: <list of structure names>" can be used to define | ||||
an enumerated type. In both cases, the colon is optional; for | ||||
example, "The <enumerated type name> is one of <list of structure | ||||
names>" is valid. | ||||
Where an enumerated type has only two variants, an alternative phrase | ||||
can be used: "The <enumerated type name> is either a <variant 1 name> | ||||
or <variant 2 name>". The names of the variants must be defined | ||||
within the document or a linked document. An optional note or | ||||
comment can be included with this alternative phrasing: "The | ||||
<enumerated type name>, with an optional comment, is either a | ||||
<variant 1 name> or <variant 2 name>" can be used. | ||||
An EOL Option is formatted as follows: | ||||
0 | ||||
0 1 2 3 4 5 6 7 | ||||
+-+-+-+-+-+-+-+-+ | ||||
| 0 | | ||||
+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Option Kind (Kind): 1 byte; Kind == 0. This is a fixed-width field, | ||||
with a value constraint, as previously described. | ||||
A Window Scale Factor Option is formatted as follows: | ||||
0 1 2 3 | ||||
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
| 3 | Length | Window Scale | | ||||
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ||||
where: | ||||
Option Kind (Kind): 1 byte; Kind == 3. This is a fixed-width field, | ||||
with a value constraint, as previously described. | ||||
Option Length (Length): 1 byte; Length == 3. This is a fixed-width | ||||
field, with a value constraint, as previously described. | ||||
Window Scale Factor: 1 byte. This is a fixed-width field, as | ||||
previously described. | ||||
A TCP Option is either an EOL Option or a Window Scale Factor Option. | ||||
4.10. Specifying Protocol Data Units | ||||
A document will set out different structures that are not, on their | A document will set out different structures that are not, on their | |||
own, protocol data units. To capture the parsing or serialisation of | own, protocol data units. To capture the parsing or serialisation of | |||
a protocol, it is necessary to be able to identify or construct those | a protocol, it is necessary to be able to identify or construct those | |||
packets that are valid PDUs. As a result, it is necessary for the | packets that are valid PDUs. As a result, it is necessary for the | |||
document to identify those structures that are PDUs. | document to identify those structures that are PDUs. | |||
The PDUs that comprise a protocol are identified using the exact | The PDUs that comprise a protocol are identified using the exact | |||
phrase "This document describes the <protocol name> protocol. The | phrase "This document describes the <protocol name> protocol. The | |||
<protocol name> protocol uses <list of PDU names>" where the list of | <protocol name> protocol uses <list of PDU names>" where the list of | |||
PDU names is a comma separated list (with the last element, if there | PDU names is a comma separated list (with the last element, if there | |||
is more than one element, preceded by 'and'), each optionally | is more than one element, preceded by 'and'), each optionally | |||
preceded by "a" or "an". As a short-form, the variation "This | preceded by "a" or "an". As a short-form, the variation "This | |||
document describes the <protocol name>, which uses <list of PDU | document describes the <protocol name>, which uses <list of PDU | |||
names>" can be used as an alternative. The PDU names must be | names>" can be used as an alternative. The PDU names must be | |||
structure names defined in the document or a linked document. The | structure names defined in the document or a linked document. The | |||
PDU names are pluralised in the list. A document must contain | PDU names are pluralised in the list. A document must contain | |||
exactly one instance of this phrase. | exactly one instance of this phrase. | |||
This document describes the Example protocol. The Example protocol | This document describes the Example protocol. The Example protocol | |||
uses Long Headers, STUN Message Types, IPv4 Headers, RTP Data | uses Long Headers, STUN Message Types, and TCP Headers. | |||
Packets, and TCP Headers. | ||||
4.11. Importing PDU Definitions from Other Documents | 3.9. Importing Definitions from Other Documents | |||
Protocols are often specified across multiple documents, either | Protocols are often specified across multiple documents, either | |||
because the specification of a protocol's data units has changed over | because the specification of a protocol's data units has changed over | |||
time, or because of explicit extension points contained in the | time, or because of explicit extension points contained in the | |||
protocol's original specification. To allow a document to make use | protocol's original specification. To allow a document to make use | |||
of a previous PDU definition, it is possible to import PDU | of a previous PDU definition, it is possible to import PDU | |||
definitions (written in the format described in this document) from | definitions (written in the format described in this document) from | |||
other documents. | other documents. | |||
A PDU definition is imported using the exact phrase "A/An ________ is | A PDU definition is imported using the exact phrase "A/An ________ is | |||
formatted as described in <document identifier>". The document | formatted as described in <document identifier>". The document | |||
identifier must refer, unambiguously, to an existing document. An | identifier must refer, unambiguously, to an existing document. An | |||
Internet-Draft is identified by its name. RFCs are identified by | Internet-Draft is identified by its name. RFCs are identified by | |||
"RFC" followed by their number. | "RFC" followed by their number. | |||
5. Open Issues | 4. Design Principles | |||
* Need a simple syntax for defining a list of identical objects, and | The use of structures that are designed to support machine | |||
a way of referring to the size of the enclosing packet. The | readability might potentially interfere with the existing ways in | |||
format cannot currently represent RFC 6716 section 3.2.3, and | which protocol specifications are used and authored. To the extent | |||
should be able to (the underlying type system can do so). | that these existing uses are more important than machine readability, | |||
such interference must be minimised. | ||||
* Need some discussion about the checks that the tooling might | In this section, the broad design principles that underpin the format | |||
perform, and the implications of those checks. For example, the | described by this document are given. However, these principles | |||
tooling checks for consistency between the diagram and the | apply more generally to any approach that introduces structured and | |||
description list of fields, ensuring that fields match by name and | formal languages into standards documents. | |||
width. -01 of this draft had a field that mismatched because of | ||||
case: is this something that the tooling should identify? More | ||||
broadly, what is the trade-off between the rigour that the tooling | ||||
can enforce, and the flexibility desired/needed by authors? | ||||
* Need to describe the rules governing the import of PDU definitions | It should be noted that these are design principles: they expose the | |||
from other documents. | trade-offs that are inherent within any given approach. Violating | |||
these principles is sometimes necessary and beneficial, and this | ||||
document sets out the potential consequences of doing so. | ||||
6. IANA Considerations | The central tenet that underpins these design principles is a | |||
recognition that the standardisation process is not broken, and so | ||||
does not need to be fixed. Failure to recognise this will likely | ||||
lead to approaches that are incompatible with the standards process, | ||||
or that will see limited adoption. However, the standards process | ||||
can be improved with appropriate approaches, as guided by the | ||||
following broad design principles: | ||||
Most readers are human: Primarily, standards documents should be | ||||
written for people, who require text and diagrams that they can | ||||
understand. Structures that cannot be easily parsed by people | ||||
should be avoided, and if included, should be clearly delineated | ||||
from human-readable content. | ||||
Any approach that shifts this balance -- that is, that primarily | ||||
targets machine readers -- is likely to be disruptive to the | ||||
standardisation process, which relies upon discussion centered | ||||
around documents written in prose. | ||||
Writing tools are diverse: Standards document writing is a | ||||
distributed process that involves a diverse set of tools and | ||||
workflows. The introduction of machine-readable structures into | ||||
specifications should not require that specific tools are used to | ||||
produce standards documents, to ensure that disruption to existing | ||||
workflows is minimised. This does not preclude the development of | ||||
optional, supplementary tools that aid in the authoring machine- | ||||
readable structures. | ||||
The immediate impact of requiring specific tooling is that | ||||
adoption is likely to be limited. A long-term impact might be | ||||
that authors whose workflows are incompatible might be alienated | ||||
from the process. | ||||
Canonical specifications: As far as possible, machine-readable | ||||
structures should not replicate the human readable specification | ||||
of the protocol within the same document. Machine-readable | ||||
structures should form part of a canonical specification of the | ||||
protocol. Adding supplementary machine-readable structures, in | ||||
parallel to the existing human readable text, is undesirable | ||||
because it creates the potential for inconsistency. | ||||
As an example, program code that describes how a protocol data | ||||
unit can be parsed might be provided as an appendix within a | ||||
standards document. This code would provide a specification of | ||||
the protocol that is separate to the prose description in the main | ||||
body of the document. This has the undesirable effect of | ||||
introducing the potential for the program code to specify | ||||
behaviour that the prose-based specification does not, and vice- | ||||
versa. | ||||
Expressiveness: Any approach should be expressive enough to capture | ||||
the syntax and parsing process for the majority of binary | ||||
protocols. If a given language is not sufficiently expressive, | ||||
then adoption is likely to be limited. At the limits of what can | ||||
be expressed by the language, authors are likely to revert to | ||||
defining the protocol in prose: this undermines the broad goal of | ||||
using structured and formal languages. Equally, though, | ||||
understandable specifications and ease of use are critical for | ||||
adoption. A tool that is simple to use and addresses the most | ||||
common use cases might be preferred to a complex tool that | ||||
addresses all use cases. | ||||
It may be desirable to restrict expressiveness, however, to | ||||
guarantee intrinsic safety, security, and computability properties | ||||
of both the generated parser code for the protocol, and the parser | ||||
of the description language itself. In much the same way as the | ||||
language-theoretic security ([LANGSEC]) community advocates for | ||||
programming language design to be informed by the desired | ||||
properties of the parsers for those languages, protocol designers | ||||
should be aware of the implications of their design choices. The | ||||
expressiveness of the protocol description languages that they use | ||||
to define their protocols can force such awareness. | ||||
Broadly, those languages that have grammars which are more | ||||
expressive tend to have parsers that are more complex and less | ||||
safe. As a result, while considering the other goals described in | ||||
this document, protocol description languages should attempt to be | ||||
minimally expressive, and either restrict protocol designs to | ||||
those for which safe and secure parsers can be generated, or as a | ||||
minimum, ensure that protocol designers are aware of the | ||||
boundaries their designs cross, in terms of computability and | ||||
decidability [SASSAMAN]. | ||||
Minimise required change: Any approach should require as few changes | ||||
as possible to the way that documents are formatted, authored, and | ||||
published. Forcing adoption of a particular structured or formal | ||||
language is incompatible with the IETF's standardisation process: | ||||
there are very few components of standards documents that are non- | ||||
optional. | ||||
5. IANA Considerations | ||||
This document contains no actions for IANA. | This document contains no actions for IANA. | |||
7. Security Considerations | 6. Security Considerations | |||
Poorly implemented parsers are a frequent source of security | Poorly implemented parsers are a frequent source of security | |||
vulnerabilities in protocol implementations. Structuring the | vulnerabilities in protocol implementations. Structuring the | |||
description of a protocol data unit so that a parser can be | description of a protocol data unit so that a parser can be | |||
automatically derived from the specification can reduce the | automatically derived from the specification can reduce the | |||
likelihood of vulnerable implementations. | likelihood of vulnerable implementations. | |||
As described in Section 3, the expressiveness of a protocol | As described in Section 4, the expressiveness of a protocol | |||
description language has implications for the safety, security, and | description language has implications for the safety, security, and | |||
computability properties of the parser for the protocol description | computability properties of the parser for the protocol description | |||
language itself, and on the generated parser code for the protocols | language itself, and on the generated parser code for the protocols | |||
described using it. The language-theoretic security ([LANGSEC]) | described using it. The language-theoretic security ([LANGSEC]) | |||
community explores the security implications of programming language | community explores the security implications of programming language | |||
design; the principles developed in that community should guide the | design; the principles developed in that community should guide the | |||
development of protocol description languages. | development of protocol description languages. | |||
8. Acknowledgements | 7. Acknowledgements | |||
The authors would like to thank Marc Petit-Huguenin for extensive | The authors would like to thank Marc Petit-Huguenin for extensive | |||
feedback on the draft, including work on formalising the constraint | feedback on the draft, including work on formalising the constraint | |||
syntax as given in Appendix A.1. | syntax as given in Appendix A.1. | |||
Wesley Eddy provided valuable feedback on the description format | Wesley Eddy provided valuable feedback on the description format | |||
through adopting it in [RFC9293]. | through adopting it in [RFC9293]. | |||
The authors would like to thank David Southgate for preparing a | The authors would like to thank David Southgate for preparing a | |||
prototype implementation of some of the ideas described here. | prototype implementation of some of the ideas described here. | |||
This work has received funding from the UK Engineering and Physical | This work has received funding from the UK Engineering and Physical | |||
Sciences Research Council under grant EP/R04144X/1. | Sciences Research Council under grant EP/R04144X/1. | |||
9. Informative References | 8. Informative References | |||
[RFC8357] Deering, S. and R. Hinden, "Generalized UDP Source Port | [RFC8357] Deering, S. and R. Hinden, "Generalized UDP Source Port | |||
for DHCP Relay", RFC 8357, March 2018, | for DHCP Relay", RFC 8357, March 2018, | |||
<https://www.rfc-editor.org/info/rfc8357>. | <https://www.rfc-editor.org/info/rfc8357>. | |||
[QUIC-TRANSPORT] | [QUIC-TRANSPORT] | |||
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed | Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed | |||
and Secure Transport", Work in Progress, Internet-Draft, | and Secure Transport", Work in Progress, Internet-Draft, | |||
draft-ietf-quic-transport-27, 21 February 2020, | draft-ietf-quic-transport-27, 21 February 2020, | |||
<http://www.ietf.org/internet-drafts/draft-ietf-quic- | <https://www.ietf.org/archive/id/draft-ietf-quic- | |||
transport-27.txt>. | transport-27.txt>. | |||
[RFC9000] Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed | ||||
and Secure Transport", RFC 9000, May 2021, | ||||
<https://www.rfc-editor.org/info/rfc9000>. | ||||
[RFC6958] Clark, A., Zhang, S., Zhao, J., and Q. Wu, "RTP Control | [RFC6958] Clark, A., Zhang, S., Zhao, J., and Q. Wu, "RTP Control | |||
Protocol (RTCP) Extended Report (XR) Block for Burst/Gap | Protocol (RTCP) Extended Report (XR) Block for Burst/Gap | |||
Loss Metric Reporting", RFC 6958, May 2013, | Loss Metric Reporting", RFC 6958, May 2013, | |||
<https://www.rfc-editor.org/info/rfc6958>. | <https://www.rfc-editor.org/info/rfc6958>. | |||
[RFC7950] Bjorklund, M., "The YANG 1.1 Data Modeling Language", | [RFC7950] Bjorklund, M., "The YANG 1.1 Data Modeling Language", | |||
RFC 7950, August 2016, | RFC 7950, August 2016, | |||
<https://www.rfc-editor.org/info/rfc7950>. | <https://www.rfc-editor.org/info/rfc7950>. | |||
[RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol | [RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol | |||
skipping to change at page 28, line 30 ¶ | skipping to change at page 24, line 43 ¶ | |||
[RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object | [RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object | |||
Representation (CBOR)", RFC 7049, October 2013, | Representation (CBOR)", RFC 7049, October 2013, | |||
<https://www.rfc-editor.org/info/rfc7049>. | <https://www.rfc-editor.org/info/rfc7049>. | |||
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. | [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. | |||
Jacobson, "RTP: A Transport Protocol for Real-Time | Jacobson, "RTP: A Transport Protocol for Real-Time | |||
Applications", RFC 3550, July 2003, | Applications", RFC 3550, July 2003, | |||
<https://www.rfc-editor.org/info/rfc3550>. | <https://www.rfc-editor.org/info/rfc3550>. | |||
[draft-ietf-tram-stunbis-21] | [RFC8489] Petit-Huguenin, M., Salgueiro, G., Rosenberg, J., Wing, | |||
Petit-Huguenin, M., Salgueiro, G., Rosenberg, J., Wing, | ||||
D., Mahy, R., and P. Matthews, "Session Traversal | D., Mahy, R., and P. Matthews, "Session Traversal | |||
Utilities for NAT (STUN)", Work in Progress, Internet- | Utilities for NAT (STUN)", RFC 8489, February 2020, | |||
Draft, draft-ietf-tram-stunbis-21, 21 March 2019, | <https://www.rfc-editor.org/info/rfc8489>. | |||
<http://www.ietf.org/internet-drafts/draft-ietf-tram- | ||||
stunbis-21.txt>. | ||||
[RFC791] Postel, J., "Internet Protocol", RFC 791, September 1981, | [RFC791] Postel, J., "Internet Protocol", RFC 791, September 1981, | |||
<https://www.rfc-editor.org/info/rfc791>. | <https://www.rfc-editor.org/info/rfc791>. | |||
[RFC793] Postel, J., "Transmission Control Protocol", RFC 793, | [RFC793] Postel, J., "Transmission Control Protocol", RFC 793, | |||
September 1981, <https://www.rfc-editor.org/info/rfc793>. | September 1981, <https://www.rfc-editor.org/info/rfc793>. | |||
[LANGSEC] LANGSEC, "LANGSEC: Language-theoretic Security", | [LANGSEC] LANGSEC, "LANGSEC: Language-theoretic Security", | |||
<http://langsec.org>. | <http://langsec.org>. | |||
[SASSAMAN] Sassaman, L., Patterson, M. L., Bratus, S., and A. | [SASSAMAN] Sassaman, L., Patterson, M. L., Bratus, S., and A. | |||
Shubina, "The Halting Problems of Network Stack | Shubina, "The Halting Problems of Network Stack | |||
Insecurity", ;login: -- December 2011, Volume 36, Number | Insecurity", ;login: -- December 2011, Volume 36, Number | |||
6, <https://www.usenix.org/publications/login/december- | 6, <https://www.usenix.org/publications/login/december- | |||
2011-volume-36-number-6/halting-problems-network-stack- | 2011-volume-36-number-6/halting-problems-network-stack- | |||
insecurity>. | insecurity>. | |||
[RFC9293] Eddy, W., "Transmission Control Protocol", RFC 9293, | [RFC9293] Eddy, W., "Transmission Control Protocol", RFC 9293, | |||
August 2022, <https://www.rfc-editor.org/info/rfc9293>. | August 2022, <https://www.rfc-editor.org/info/rfc9293>. | |||
[RFC768] Postel, J., "User Datagram Protocol", RFC 768, August | ||||
1980, <https://www.rfc-editor.org/info/rfc768>. | ||||
Appendix A. ABNF specification | Appendix A. ABNF specification | |||
A.1. Constraint Expressions | A.1. Constraint Expressions | |||
constant = %x31-39 *(%x30-39) ; natural numbers without leading 0s | constant = %x31-39 *(%x30-39) ; natural numbers without leading 0s | |||
short-name = ALPHA *(ALPHA / DIGIT / "-" / "_") | short-name = ALPHA *(ALPHA / DIGIT / "-" / "_") | |||
name = short-name *(" " short-name) | name = short-name *(" " short-name) | |||
sp = [" "] ; optional space in expression | sp = [" "] ; optional space in expression | |||
bool-expr = "(" sp bool-expr sp ")" / | bool-expr = "(" sp bool-expr sp ")" / | |||
"!" sp bool-expr / | "!" sp bool-expr / | |||
skipping to change at page 29, line 36 ¶ | skipping to change at page 26, line 8 ¶ | |||
name / short-name "." short-name / | name / short-name "." short-name / | |||
"size(" short-name ")" / | "size(" short-name ")" / | |||
constant | constant | |||
op = "+" / "-" / "*" / "/" / "%" / "^" | op = "+" / "-" / "*" / "/" / "%" / "^" | |||
length = expr sp unit / "[" sp name sp "]" / "variable length" | length = expr sp unit / "[" sp name sp "]" / "variable length" | |||
unit = %s"bit" / %s"bits" / %s"byte" / %s"bytes" / name | unit = %s"bit" / %s"bits" / %s"byte" / %s"bytes" / name | |||
A.2. Augmented packet diagrams | A.2. Augmented packet diagrams | |||
Future revisions of this draft will include an ABNF specification for | Future revisions of this draft will include an ABNF specification for | |||
the augmented packet diagram format described in Section 4. Such a | the augmented packet diagram format described in Section 3. Such a | |||
specification is omitted from this draft given that the format is | specification is omitted from this draft given that the format is | |||
likely to change as its syntax is developed. Given the visual nature | likely to change as its syntax is developed. Given the visual nature | |||
of the format, it is more appropriate for discussion to focus on the | of the format, it is more appropriate for discussion to focus on the | |||
examples given in Section 4. | examples given in Section 3. | |||
Appendix B. Tooling & source code | Appendix B. Tooling & source code | |||
The source for this draft is available from https://github.com/ | The source for this draft is available from https://github.com/ | |||
glasgow-ipl/draft-mcquistin-augmented-ascii-diagrams. | glasgow-ipl/draft-mcquistin-augmented-ascii-diagrams. | |||
The source code for tooling that can be used to parse this document | The source code for tooling that can be used to parse this document | |||
is available from https://github.com/glasgow-ipl/ips-protodesc-code. | is available from https://github.com/glasgow-ipl/ips-protodesc-code. | |||
This tooling supports the automatic generation of Rust parser code | This tooling supports the automatic generation of Rust parser code | |||
from protocol descriptions written in the Augmented Packet Header | from protocol descriptions written in the Augmented Packet Header | |||
End of changes. 49 change blocks. | ||||
606 lines changed or deleted | 449 lines changed or added | |||
This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |