draft-mcquistin-augmented-ascii-diagrams-03.txt   draft-mcquistin-augmented-ascii-diagrams-04.txt 
Network Working Group S. McQuistin Network Working Group S. McQuistin
Internet-Draft V. Band Internet-Draft V. Band
Intended status: Experimental D. Jacob Intended status: Experimental D. Jacob
Expires: 10 September 2020 C. S. Perkins Expires: 26 October 2020 C. S. Perkins
University of Glasgow University of Glasgow
9 March 2020 24 April 2020
Describing Protocol Data Units with Augmented Packet Header Diagrams Describing Protocol Data Units with Augmented Packet Header Diagrams
draft-mcquistin-augmented-ascii-diagrams-03 draft-mcquistin-augmented-ascii-diagrams-04
Abstract Abstract
This document describes a machine-readable format for specifying the This document describes a machine-readable format for specifying the
syntax of protocol data units within a protocol specification. This syntax of protocol data units within a protocol specification. This
format is comprised of a consistently formatted packet header format is comprised of a consistently formatted packet header
diagram, followed by structured explanatory text. It is designed to diagram, followed by structured explanatory text. It is designed to
maintain human readability while enabling support for automated maintain human readability while enabling support for automated
parser generation from the specification document. This document is parser generation from the specification document. This document is
itself an example of how the format can be used. itself an example of how the format can be used.
skipping to change at page 1, line 38 skipping to change at page 1, line 38
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on 10 September 2020. This Internet-Draft will expire on 26 October 2020.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
skipping to change at page 2, line 15 skipping to change at page 2, line 15
as described in Section 4.e of the Trust Legal Provisions and are as described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Simplified BSD License. provided without warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Limitations of Current Packet Format Diagrams . . . . . . 4 2.1. Limitations of Current Packet Format Diagrams . . . . . . 4
2.2. Formal languages in standards documents . . . . . . . . . 7 2.2. Formal languages in standards documents . . . . . . . . . 7
3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7 3. Design Principles . . . . . . . . . . . . . . . . . . . . . . 7
4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 9 4. Augmented Packet Header Diagrams . . . . . . . . . . . . . . 10
4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 10 4.1. PDUs with Fixed and Variable-Width Fields . . . . . . . . 10
4.2. PDUs That Cross-Reference Previously Defined Fields . . . 12 4.2. PDUs That Cross-Reference Previously Defined Fields . . . 13
4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 15 4.3. PDUs with Non-Contiguous Fields . . . . . . . . . . . . . 15
4.4. Importing PDU Definitions from Other Documents . . . . . 15 4.4. PDUs with Constraints on Field Values . . . . . . . . . . 16
5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 16 4.5. PDUs That Extend Sub-Structures . . . . . . . . . . . . . 17
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 4.6. Storing Data for Parsing . . . . . . . . . . . . . . . . 18
7. Security Considerations . . . . . . . . . . . . . . . . . . . 16 4.7. Connecting Structures with Functions . . . . . . . . . . 19
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 4.8. Specifying Enumerated Types . . . . . . . . . . . . . . . 20
9. Informative References . . . . . . . . . . . . . . . . . . . 17 4.9. Specifying Protocol Data Units . . . . . . . . . . . . . 21
Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 18 4.10. Importing PDU Definitions from Other Documents . . . . . 21
A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 18 5. Open Issues . . . . . . . . . . . . . . . . . . . . . . . . . 22
A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 19 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22
Appendix B. Source code repository . . . . . . . . . . . . . . . 19 7. Security Considerations . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23
9. Informative References . . . . . . . . . . . . . . . . . . . 23
Appendix A. ABNF specification . . . . . . . . . . . . . . . . . 24
A.1. Constraint Expressions . . . . . . . . . . . . . . . . . 24
A.2. Augmented packet diagrams . . . . . . . . . . . . . . . . 25
Appendix B. Source code repository . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 25
1. Introduction 1. Introduction
Packet header diagrams have become a widely used format for Packet header diagrams have become a widely used format for
describing the syntax of binary protocols. In otherwise largely describing the syntax of binary protocols. In otherwise largely
textual documents, they allow for the visualisation of packet textual documents, they allow for the visualisation of packet
formats, reducing human error, and aiding in the implementation of formats, reducing human error, and aiding in the implementation of
parsers for the protocols that they specify. parsers for the protocols that they specify.
Figure 1 gives an example of how packet header diagrams are used to Figure 1 gives an example of how packet header diagrams are used to
skipping to change at page 10, line 31 skipping to change at page 10, line 51
A PDU description is introduced by the exact phrase "A/An _______ is A PDU description is introduced by the exact phrase "A/An _______ is
formatted as follows:" at the end of a paragraph. This is followed formatted as follows:" at the end of a paragraph. This is followed
by the PDU description itself, as a packet diagram within an by the PDU description itself, as a packet diagram within an
<artwork> element in the XML representation, starting with a header <artwork> element in the XML representation, starting with a header
line to show the bit width of the diagram. The description of the line to show the bit width of the diagram. The description of the
fields follows the diagram, as an XML <dl> list, after a paragraph fields follows the diagram, as an XML <dl> list, after a paragraph
containing the text "where:". containing the text "where:".
PDU names must be unique, both within a document, and across all PDU names must be unique, both within a document, and across all
documents that are linked together (i.e., using the structured documents that are linked together (i.e., using the structured
language defined in Section 4.4). language defined in Section 4.10).
Each field of the description starts with a <dt> tag comprising the Each field of the description starts with a <dt> tag comprising the
field name and an optional short name in parenthesis. These are field name and an optional short name in parenthesis. These are
followed by a colon, the field length, an optional presence followed by a colon, the field length, an optional presence
expression (described in Section 4.2), and a terminating period. The expression (described in Section 4.2), and a terminating period. The
following <dd> tag contains a prose description of the field. Field following <dd> tag contains a prose description of the field. Field
names cannot be the same as a previously defined PDU name, and must names cannot be the same as a previously defined PDU name, and must
be unique within a given structure definition. be unique within a given structure definition.
For example, this can be illustrated using the IPv4 Header Format For example, this can be illustrated using the IPv4 Header Format
skipping to change at page 15, line 43 skipping to change at page 16, line 15
Method (M): 12 bits. This field is comprised of multiple sub-fields Method (M): 12 bits. This field is comprised of multiple sub-fields
(M0 through MB) as shown in the diagram. That these sub-fields (M0 through MB) as shown in the diagram. That these sub-fields
should be concatenated, after parsing, into a single field is should be concatenated, after parsing, into a single field is
indicated by their being labelled using the 'M' short field name indicated by their being labelled using the 'M' short field name
followed by a single hexadecimal digit, with the least significant followed by a single hexadecimal digit, with the least significant
bit labelled with 0, and subsequent bits labelled in sequence. bit labelled with 0, and subsequent bits labelled in sequence.
Class (C): 2 bits. This field follows the same format as M described Class (C): 2 bits. This field follows the same format as M described
above. above.
4.4. Importing PDU Definitions from Other Documents 4.4. PDUs with Constraints on Field Values
A PDU may be defined not only by the layout and type of its fields,
but also by the value of those fields. For example, field values may
be constrained to be of a known exact value or to be within a range.
More generally, our format enables a boolean expression to be
attached to a field, which must be true for the PDU to be parsed
successfully.
This format is illustrated using the QUIC Long Header Packet format
[QUIC-TRANSPORT]. A Long Header is formatted as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+
|1|1| T | R | P |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DCID Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Connection ID (DCID) ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SCID Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Connection ID (SCID) ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
where:
Header Form (HF): 1 bit; HF == 1. This is a fixed-width field,
constrained to be a of an known, exact value. At most one field
value constraint may be given, and if provided, it must be given
as a boolean expression, separated by a semi-colon in the field
definition name (i.e., the text contained within the <dt> tag).
If present, a value constraint must follow the name, short name,
and length of the field, but appear before any presence
constraint, if applicable.
The order of the field must be the same in both the diagram and
description list.
Fixed Bit (FB): 1 bit; FB == 1. This is a fixed-width field, with a
value constraint, as previously described.
Long Packet Type (T): 2 bits. This is a fixed-width field as
previously described.
Reserved Bits (R): 2 bits. This is a fixed-width field as previously
described.
Packet Number Length (P): 2 bits. This is a fixed-width field as
previously described.
Version: 32 bits. This is a fixed-width field as previously
described.
DCID Len (DLen): 1 byte; DLen <= 20. This is a fixed-width field,
with a value constraint, as previously described. Note that the
constraint language is not limited to equality; it is defined
fully in Appendix A.1.
Destination Connection ID: DLen bytes. This is a variable-width
field as previously described.
SCID Len (SLen): 1 byte; SLen <= 20. This is a fixed-width field,
with a value constraint, as previously described.
Source Connection ID: SLen bytes. This is a variable-width field as
previously described.
4.5. PDUs That Extend Sub-Structures
A PDU may not only use or reference existing sub-structures, but they
may extend them, adding new fields, or enforcing different or
additional constraints.
Where a sub-structure is extended, the diagram may show the sub-
structure as a block, labelled with the sub-structure name. It may
also be desirable to show the sub-structure diagram in full; in this
case, the fields must be given in the same order and be of the same
length. New field constraints can be shown. Similarly, in the
description list, those fields inherited without change (i.e., with
no change to their constraints) do not need to be repeated. Those
with different or additional constraints must be described, and the
order of the fields in the description list must match that of the
sub-structure and the containing structure.
This format is illustrated using the QUIC Retry Packet format
[QUIC-TRANSPORT]. A Retry Packet is formatted as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :
: Long Header :
: |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Retry Token ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ +
| |
+ Retry Integrity Tag +
| |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
where:
Long Header (LH): 1 Long Header; LH.T == 3. This field is a
previously defined sub-structure. Its constraints can access
fields in that sub-structure. In this example, the T field of the
Long Header must be equal to 3.
Retry Token This is a variable-length field as previously defined.
Retry Integrity Tag: 128 bits. This is a fixed-width field as
previously defined.
As shown, the Long Header packet sub-structure is included. The
Retry Packet enforces a new value constraint on the Long Packet Type
(T) field.
4.6. Storing Data for Parsing
The parsing process may require data from previously parsed
structures. This means that data needs to be stored persistently
throughout the process. This data needs to be identified.
That the value of a particular field be stored upon parsing is
indicated by the exact phrase "On receipt, the value of <field name>
is stored as <stored name>." being present at the end of the
description of a field (i.e., at the end of the <dd> element.)
An Initial Packet is formatted as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :
: Long Header :
: |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
where:
Long Header (LH): 1 Long Header; LH.T == 0. This is field is a sub-
structure, with a constraint, as previously defined. On receipt,
the value of LH.DCID is stored as Initial DCID.
In this example, the value of the DCID field of the Long Header sub-
structure is stored as Initial DCID.
4.7. Connecting Structures with Functions
The parsing or serialisation of some binary formats cannot be fully
described without the use of functions. These functions take
arguments (values from another structure), perform some computation,
and generate a new structure.
Given the goal of fully capturing the parsing or serialisation of
binary protocols, it is necessary to include the signature of these
helper functions.
Function signatures are described in <artwork> elements. They are
constructed as the word "func", followed by a space, then the name of
the function. This is immediately followed by a set of brackets
containing a comma separated list of the function's parameters,
formatted as "<parameter name>: <parameter type>". This is followed
by "->" and the return type of the function, followed by a colon.
The body of the function is not captured, owing to the complexity of
both capturing and translating arbitrary code. As a result, it can
be described in whichever format is most suitable for the document
and its readership.
Those values that are stored persistently, as defined in Section 4.6,
are accessible by functions.
As an example, the "apply_protection" function is defined as:
func apply_protection(to: Unprotected Packet)
-> Protected Packet:
apply packet protection to payload
apply header protection to first_byte and packet_number
construct appropriate Protected Packet based on first_byte
return Protected Packet
In this example, 'Unprotected Packet' and 'Protected Packet' are
existing types.
To indicate that a structure is created from another by way of a
function: "A/An <structure type A> is constructed from a <structure
type B> using the <function name>(<parameters>) function.". This
indicates that a structure of type A is generated using the named
function. The parameters in the function call are named fields from
structure type B. They must match the function signature, both in
number and type.
4.8. Specifying Enumerated Types
In addition to the use of the sub-structures, it is desirable to be
able to define a type that may take the value of one of a set of
alternative structures.
The alternative structures that comprise an enumerated type are
identified using the exact phrase "The <enumerated type name> is one
of: <list of structure names>" where the list of structure names is a
comma separated list (with the last element, if there is more than
one element, preceded by 'or'), each optionally preceded by "a" or
"an". The structure names must be defined within the document or a
linked document.
Where an enumerated type has only two variants, an alternative phrase
can be used: "The <enumerated type name> is either a <variant 1 name>
or <variant 2 name>". The names of the variants must be defined
within the document or a linked document.
A PING Frame is formatted as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
where:
Frame Type (FT): 1 * Variable-Length Integer Encoding; FT == 1.
Frame type, set to 1 for PING frames.
A HANDSHAKE_DONE Frame is formatted as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 30 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
where:
Frame Type (FT): 1 * Variable-Length Integer Encoding; FT == 30.
Frame type, set to 30 for HANDSHAKE_DONE frames.
A Frame is either a PING Frame or a HANDSHAKE_DONE Frame.
4.9. Specifying Protocol Data Units
A document will set out different structures that are not, on their
own, protocol data units. To capture the parsing or serialisation of
a protocol, it is necessary to be able to identify or construct those
packets that are valid PDUs. As a result, it is necessary for the
document to identify those structures that are PDUs.
The PDUs that comprise a protocol are identified using the exact
phrase "This document describes the <protocol name> protocol, The
<protocol name> protocol uses <list of PDU names>" where the list of
PDU names is a comma separated list (with the last element, if there
is more than one element, preceded by 'and'), each optionally
preceded by "a" or "an". The PDU names must be structure names
defined in the document or a linked document. The PDU names are
pluralised in the list. A document must contain exactly one instance
of this phrase.
The Example protocol uses Long Headers, STUN Message Types, IPv4
Headers, and RTP Data Packets.
4.10. Importing PDU Definitions from Other Documents
Protocols are often specified across multiple documents, either Protocols are often specified across multiple documents, either
because the specification of a protocol's data units has changed over because the specification of a protocol's data units has changed over
time, or because of explicit extension points contained in the time, or because of explicit extension points contained in the
protocol's original specification. To allow a document to make use protocol's original specification. To allow a document to make use
of a previous PDU definition, it is possible to import PDU of a previous PDU definition, it is possible to import PDU
definitions (written in the format described in this document) from definitions (written in the format described in this document) from
other documents. other documents.
A PDU definition is imported using the exact phrase "A/An ________ is A PDU definition is imported using the exact phrase "A/An ________ is
skipping to change at page 17, line 25 skipping to change at page 23, line 25
9. Informative References 9. Informative References
[RFC8357] Deering, S. and R. Hinden, "Generalized UDP Source Port [RFC8357] Deering, S. and R. Hinden, "Generalized UDP Source Port
for DHCP Relay", RFC 8357, March 2018, for DHCP Relay", RFC 8357, March 2018,
<https://www.rfc-editor.org/info/rfc8357>. <https://www.rfc-editor.org/info/rfc8357>.
[QUIC-TRANSPORT] [QUIC-TRANSPORT]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", Work in Progress, Internet-Draft, and Secure Transport", Work in Progress, Internet-Draft,
draft-ietf-quic-transport-20, 23 April 2019, draft-ietf-quic-transport-27, 21 February 2020,
<http://www.ietf.org/internet-drafts/draft-ietf-quic- <http://www.ietf.org/internet-drafts/draft-ietf-quic-
transport-20.txt>. transport-27.txt>.
[RFC6958] Clark, A., Zhang, S., Zhao, J., and Q. Wu, "RTP Control [RFC6958] Clark, A., Zhang, S., Zhao, J., and Q. Wu, "RTP Control
Protocol (RTCP) Extended Report (XR) Block for Burst/Gap Protocol (RTCP) Extended Report (XR) Block for Burst/Gap
Loss Metric Reporting", RFC 6958, May 2013, Loss Metric Reporting", RFC 6958, May 2013,
<https://www.rfc-editor.org/info/rfc6958>. <https://www.rfc-editor.org/info/rfc6958>.
[RFC7950] Bjorklund, M., "The YANG 1.1 Data Modeling Language", [RFC7950] Bjorklund, M., "The YANG 1.1 Data Modeling Language",
RFC 7950, August 2016, RFC 7950, August 2016,
<https://www.rfc-editor.org/info/rfc7950>. <https://www.rfc-editor.org/info/rfc7950>.
 End of changes. 11 change blocks. 
21 lines changed or deleted 295 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/