draft-perkins-avt-rapid-rtp-sync-02.txt   draft-perkins-avt-rapid-rtp-sync-03.txt 
Network Working Group C. Perkins Network Working Group C. Perkins
Internet-Draft University of Glasgow Internet-Draft University of Glasgow
Updates: RFC3550 T. Schierl Updates: RFC3550 T. Schierl
(if approved) Fraunhofer HHI (if approved) Fraunhofer HHI
Intended status: Standards Track February 13, 2009 Intended status: Standards Track March 9, 2009
Expires: August 17, 2009 Expires: September 10, 2009
Rapid Synchronisation of RTP Flows Rapid Synchronisation of RTP Flows
draft-perkins-avt-rapid-rtp-sync-02.txt draft-perkins-avt-rapid-rtp-sync-03.txt
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 1, line 34 skipping to change at page 1, line 34
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on August 17, 2009. This Internet-Draft will expire on September 10, 2009.
Copyright Notice Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents in effect on the date of
(http://trustee.ietf.org/license-info) in effect on the date of publication of this document (http://trustee.ietf.org/license-info).
publication of this document. Please review these documents Please review these documents carefully, as they describe your rights
carefully, as they describe your rights and restrictions with respect and restrictions with respect to this document.
to this document.
Abstract Abstract
This memo outlines how RTP multimedia sessions are synchronised, and This memo outlines how RTP multimedia sessions are synchronised, and
discusses how rapidly such synchronisation can occur. We show that discusses how rapidly such synchronisation can occur. We show that
most RTP sessions can be synchronised immediately, but that the use most RTP sessions can be synchronised immediately, but that the use
of video switching multipoint conference units (MCUs) or large source of video switching multipoint conference units (MCUs) or large source
specific multicast (SSM) groups can greatly increase the initial specific multicast (SSM) groups can greatly increase the initial
synchronisation delay. This increase in delay can be unacceptable to synchronisation delay. This increase in delay can be unacceptable to
some applications that use layered and/or multi-description codecs. some applications that use layered and/or multi-description codecs.
skipping to change at page 3, line 7 skipping to change at page 2, line 22
reduce the initial synchronisation delay for SSM sessions. A new reduce the initial synchronisation delay for SSM sessions. A new
feedback packet is defined for use with the Extended RTP Profile for feedback packet is defined for use with the Extended RTP Profile for
RTCP-based Feedback (RTP/AVPF), allowing video switching MCUs to RTCP-based Feedback (RTP/AVPF), allowing video switching MCUs to
rapidly request resynchronisation. Two new RTP header extensions are rapidly request resynchronisation. Two new RTP header extensions are
defined to allow rapid synchronisation of late joiners, and guarantee defined to allow rapid synchronisation of late joiners, and guarantee
correct timestamp based decoding order recovery for layered codecs in correct timestamp based decoding order recovery for layered codecs in
the presence of clock skew. the presence of clock skew.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Synchronisation of RTP Flows . . . . . . . . . . . . . . . . . 4 2. Synchronisation of RTP Flows . . . . . . . . . . . . . . . . . 3
2.1. Initial Synchronisation Delay . . . . . . . . . . . . . . 5 2.1. Initial Synchronisation Delay . . . . . . . . . . . . . . 4
2.1.1. Unicast Sessions . . . . . . . . . . . . . . . . . . . 6 2.1.1. Unicast Sessions . . . . . . . . . . . . . . . . . . . 5
2.1.2. Source Specific Multicast (SSM) Sessions . . . . . . . 6 2.1.2. Source Specific Multicast (SSM) Sessions . . . . . . . 5
2.1.3. Any Source Multicast (ASM) Sessions . . . . . . . . . 7 2.1.3. Any Source Multicast (ASM) Sessions . . . . . . . . . 6
2.1.4. Discussion . . . . . . . . . . . . . . . . . . . . . . 7 2.1.4. Discussion . . . . . . . . . . . . . . . . . . . . . . 6
2.2. Synchronisation for Late Joiners . . . . . . . . . . . . . 8 2.2. Synchronisation for Late Joiners . . . . . . . . . . . . . 7
3. Reducing RTP Synchronisation Delays . . . . . . . . . . . . . 9 3. Reducing RTP Synchronisation Delays . . . . . . . . . . . . . 8
3.1. Rapid Resynchronisation Request . . . . . . . . . . . . . 9 3.1. Rapid Resynchronisation Request . . . . . . . . . . . . . 8
3.2. In-band Delivery of Synchronisation Metadata . . . . . . . 10 3.2. In-band Delivery of Synchronisation Metadata . . . . . . . 9
3.3. Signalling . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3. Signalling . . . . . . . . . . . . . . . . . . . . . . . . 10
4. Application to Decoding Order Recovery in Layered Codecs . . . 12 4. Application to Decoding Order Recovery in Layered Codecs . . . 11
4.1. Problem description . . . . . . . . . . . . . . . . . . . 12 4.1. Problem description . . . . . . . . . . . . . . . . . . . 11
4.2. Use of RTP Header Extensions for Synchronisation . . . . . 12 4.2. Use of RTP Header Extensions for Synchronisation . . . . . 11
4.3. Timestamp based decoding order recovery . . . . . . . . . 13 4.3. Timestamp based decoding order recovery . . . . . . . . . 12
5. Security Considerations . . . . . . . . . . . . . . . . . . . 16 5. Security Considerations . . . . . . . . . . . . . . . . . . . 15
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 17 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 17 8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16
8.1. Normative References . . . . . . . . . . . . . . . . . . . 17 8.1. Normative References . . . . . . . . . . . . . . . . . . . 16
8.2. Informative References . . . . . . . . . . . . . . . . . . 17 8.2. Informative References . . . . . . . . . . . . . . . . . . 16
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction 1. Introduction
When using RTP to deliver multimedia content it's often necessary to When using RTP to deliver multimedia content it's often necessary to
synchronise playout of audio and video components of a presentation. synchronise playout of audio and video components of a presentation.
This is achieved using information contained in RTP Control Protocol This is achieved using information contained in RTP Control Protocol
(RTCP) Sender Report (SR) packets [1]. These are sent periodically, (RTCP) Sender Report (SR) packets [1]. These are sent periodically,
and the components of a multimedia session cannot be synchronised and the components of a multimedia session cannot be synchronised
until sufficient RTCP SR packets have been received for each flow to until sufficient RTCP SR packets have been received for each flow to
allow the receiver to establish mappings between the media clock used allow the receiver to establish mappings between the media clock used
skipping to change at page 9, line 11 skipping to change at page 8, line 11
given sufficient reports. Collecting sufficient RTCP SR data to given sufficient reports. Collecting sufficient RTCP SR data to
perform this estimation, however, may require several reports, perform this estimation, however, may require several reports,
further increasing the synchronisation delay. further increasing the synchronisation delay.
These delays are likely an issue for tuning in to an ongoing These delays are likely an issue for tuning in to an ongoing
multicast RTP session, or for video switching MCUs. multicast RTP session, or for video switching MCUs.
3. Reducing RTP Synchronisation Delays 3. Reducing RTP Synchronisation Delays
Two backwards compatible RTP extensions are defined to reduce the Two backwards compatible RTP extensions are defined to reduce the
possible synchronisation delay: a repid resynchronisation request possible synchronisation delay: a rapid resynchronisation request
message, and RTP header extensions that can convey synchronisation message, and RTP header extensions that can convey synchronisation
metadata in-band. metadata in-band.
3.1. Rapid Resynchronisation Request 3.1. Rapid Resynchronisation Request
The general format of an RTP/AVPF transport layer feedback message is The general format of an RTP/AVPF transport layer feedback message is
shown below. shown below.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
skipping to change at page 9, line 36 skipping to change at page 8, line 36
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source | | SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Feedback Control Information (FCI) : : Feedback Control Information (FCI) :
: : : :
A new feedback message type, RTCP-SR-REQ, is defined with FMT = XXX. A new feedback message type, RTCP-SR-REQ, is defined with FMT = XXX.
(the next available FMT is 5?) This MAY be sent to indicate that a (the next available FMT is 5?) This MAY be sent to indicate that a
receiver is unable to synchronise media streams, and desires that the receiver is unable to synchronise media streams, and desires that the
media source send an RTCP SR packet as soon as possible (within the media source send an RTCP SR packet as soon as possible (within the
constraints of RTCP the early feedback rules). On receipt of this, constraints of the RTCP early feedback rules). On receipt of this,
the media source SHOULD generate an RTCP SR packet as soon as the media source SHOULD generate an RTCP SR packet as soon as
possible within the RTCP early feedback rules. That RTCP SR packet possible within the RTCP early feedback rules. That RTCP SR packet
MAY be sent as a non-compound RTCP packet, if this has been MAY be sent as a non-compound RTCP packet, if this has been
negotiated. negotiated.
The Feedback Control Information (FCI) part of the packet is emtpy. The Feedback Control Information (FCI) part of the packet is emtpy.
The SSRC of packet sender indicates the member that is unable to The SSRC of packet sender indicates the member that is unable to
synchronise media streams, while the SSRC of media source indicates synchronise media streams, while the SSRC of media source indicates
the sender of the media it is unable to synchronise. The lenght MUST the sender of the media it is unable to synchronise. The lenght MUST
equal 2. equal 2.
skipping to change at page 11, line 32 skipping to change at page 10, line 32
| .... | | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
An NTP timestamp format timestamp MAY be included on any RTP packets An NTP timestamp format timestamp MAY be included on any RTP packets
the sender chooses, but it is RECOMMENDED when performing timestamp the sender chooses, but it is RECOMMENDED when performing timestamp
based decoding order recovery for layered codecs transported in based decoding order recovery for layered codecs transported in
multiple RTP flows, as discussed in Section 4. This header extension multiple RTP flows, as discussed in Section 4. This header extension
MAY be also sent on the RTP packets corresponding to a video random MAY be also sent on the RTP packets corresponding to a video random
access point, and on the associated audio packets, to allow rapid access point, and on the associated audio packets, to allow rapid
synchronisation for late joiners and in video switching scenarios. synchronisation for late joiners and in video switching scenarios.
Note that the inclusion of an RTP header extension will reduce the
efficiency of RTP header compression, if it is used.
In all cases, irrespective of whether in-band NTP timestamp format In all cases, irrespective of whether in-band NTP timestamp format
timestamps are included or not, regular RTCP SR packets MUST be sent timestamps are included or not, regular RTCP SR packets MUST be sent
to provide backwards compatibility with receivers that synchronize to provide backwards compatibility with receivers that synchronize
RTP flows according to [1]. The sender reports are also required to RTP flows according to [1]. The sender reports are also required to
receive the upper 8 bit of the Seconds of the NTP timestamp format receive the upper 8 bit of the Seconds of the NTP timestamp format
timestamp not included in the NTP header extension. timestamp not included in the NTP header extension.
3.3. Signalling 3.3. Signalling
skipping to change at page 12, line 26 skipping to change at page 11, line 26
4.1. Problem description 4.1. Problem description
One option for decoding order recovery in layered codecs is to use One option for decoding order recovery in layered codecs is to use
the NTP (sample presentation) timestamps to reorder data of the same the NTP (sample presentation) timestamps to reorder data of the same
layered media transported in different RTP flows. For a timestamp- layered media transported in different RTP flows. For a timestamp-
based decoding order recovery process, it is crucial to allow exact based decoding order recovery process, it is crucial to allow exact
alignment of media frames respectively samples using the NTP timing alignment of media frames respectively samples using the NTP timing
information. In the presence of clock skew, it may not be possible information. In the presence of clock skew, it may not be possible
to derive exact matching NTP timestamps using the NTP wallclock in to derive exact matching NTP timestamps using the NTP wallclock in
each RTP flow's RTCP sender reports. This is due to the fact that each RTP flow's RTCP sender reports. This is due to the fact that
RTCP sender reports are not send at the same point of time in the RTCP sender reports are not sent at the same point of time in the
multiple RTP flows transporting data of the same layered media. If multiple RTP flows transporting data of the same layered media. If
the RTCP SR packets are not send at the same time, they therefore do the RTCP SR packets are not send at the same time, they therefore do
not contain the same NTP wallclock timestamp. If there is a skew not contain the same NTP wallclock timestamp. If there is a skew
present in the clock used for NTP wallclock timestamp generation, present in the clock used for NTP wallclock timestamp generation,
using different wallclock timestamps for the same sampling instance using different wallclock timestamps for the same sampling instance
in the RTP flow inevitably leads to non-matching NTP timestamps in the RTP flow inevitably leads to non-matching NTP timestamps
generated from RTP timestamps and wallclock timestamp in the multiple generated from RTP timestamps and wallclock timestamp in the multiple
RTP flows. In order to allow a common and straight forward RTP flows. In order to allow a common and straight forward
timestamp-based decoding order recovery process, it is important to timestamp-based decoding order recovery process, it is important to
guarantee exact matching of NTP timestamps. Thus in the presence of guarantee exact matching of NTP timestamps. Thus in the presence of
non-perfect clocks, which should be the normal case, an additional non-perfect clocks, which should be the normal case, an additional
mechanism shall be used. If synchronously in all flows inserting mechanism shall be used. An exact inter-flow alignment of NTP
header extensions as defined in Section 3.2, an exact inter-flow timestamps can be guaranteed, if the NTP header extension is always
alignment of NTP timestamps can be guaranteed with the prerequisite inserted at the same timing position in all the RTP flows in question
that only the NTP timestamps are used when synchronously present in and if those NTP header extensions are used to update the NTP-RTP
all the RTP flows in question. relation in all RTP flows at the same point of time.
4.2. Use of RTP Header Extensions for Synchronisation 4.2. Use of RTP Header Extensions for Synchronisation
The NTP header extension SHOULD be used with a layered, multi- The NTP header extension SHOULD be used with a layered, multi-
description, or multi-view codec, to provide exact matching of NTP description, or multi-view codec, to provide exact matching of NTP
timestamps between layers, descriptions, or views trasnported in timestamps between layers, descriptions, or views transported in
different RTP flows to allow timestamp-based decoding order recovery. different RTP flows to allow timestamp-based decoding order recovery.
If the NTP header extension is inserted for RTP flows transporting If the NTP header extension is inserted for RTP flows transporting
samples or parts of samples of the same layered media, the NTP header samples or parts of samples of the same layered media, the NTP header
extension SHALL be included at least once in each of the RTP flows of extension SHALL be included at least once in each of the RTP flows of
the same media for the sampling time instance of an insertion of a the same media for the sampling time instance of an insertion of a
NTP header extension and such synchronously inserted NTP header NTP header extension and such synchronously inserted NTP header
extensions SHALL contain the same NTP timestamp. The frequency of extensions SHALL contain the same NTP timestamp. The frequency of
inserting NTP header extensions in the RTP flows is up to the sender. inserting NTP header extensions in the RTP flows is up to the sender.
Note: If the decoding order of RTP flows is given by any means (as Note: If the decoding order of RTP flows is given by any means (as
e.g., by mechanism defined in [6]), the NTP timestamp provided by the e.g., by mechanism defined in [6]), the NTP timestamp provided by the
header extension allows to collect data of the same sample from the header extension allows to collect data of the same sample from the
RTP flows, forming the sample decoding order. RTP flows, forming the sample decoding order.
It is RECOMMENDED that the receiver uses for timestamp-based decoding It is RECOMMENDED that the receiver uses for timestamp-based decoding
order recovery the NTP timestamps provided in the RTP NTP header order recovery the NTP timestamps provided in the RTP NTP header
extensions only, if such extensions are present for the RTP flows. extensions only, if such extensions are present for the RTP flows.
Section 4 gives further details about the timestamp-based decoding Section 4 gives further details about the timestamp-based decoding
order recovery. order recovery.
Note: The NTP header insertion as described above allows the receiver Note: Using the RTP header extensions described above allows the
to find the corresponding sample of the layered media or parts receiver to find the corresponding sample of the layered media, or
thereof in all the RTP flows at the point of the NTP header extension parts thereof, in all RTP flows at the instant the RTP header
insertion. This guarantees that any clock skew present in the NTP extension is inserted into the flows. This guarantees that any clock
timestamp generation process based on RTCP sender reports is avoided, skew present in the NTP timestamp generation process based on RTCP
thus this approach allows directly comparing NTP timestamps of the sender reports is avoided, and so allows direct comparison of NTP
RTP flows. Furthermore, this approach solves the possible problem of timestamps across multiple RTP flows. Furthermore, this approach
clock skews identified for the NI-T mode as defined in [7]. Such an solves the possible problem of clock skews identified for the NI-T
NTP header extension insertion is only effective for clock skew mode as defined in [7]. To ensure the absence of clock skew, a
elimination, if such insertion is applied in all RTP flows of the header extension containing the NTP timestamp MUST be inserted into
layered media at the same time and if the receiver uses such the RTP flows comprising a layered media stream at the same instant
synchronously sent NTP timestamps for the decoding order recovery in each RTP flow. This may require the insertion of extra packets in
process only. This may require the insertion of extra packets in
some of the RTP flows, since in layered video codecs not all sampling some of the RTP flows, since in layered video codecs not all sampling
instances may be present in all the flows. If such a header instances may be present in all the flows. If such a header
extension is included in all flows at a sampling time instance, the extension is included in all flows at a sampling time instance, the
NTP timestamps for samples following in decoding order the NTP header NTP timestamps for samples following in decoding order the NTP header
insertion point can be constructed using the RTP timestamps and insertion point can be constructed using the RTP timestamps and
identical reference NTP timestamps in the NTP header extension in all identical reference NTP timestamps in the NTP header extension in all
RTP flows. It should be noted that the frequency of inserting the RTP flows. It should be noted that the frequency of inserting the
NTP header extension is crucial in presence of clock skew, since the NTP header extension is crucial in presence of clock skew, since the
points of insertion may be the only points for a receiver to start points of insertion may be the only points for a receiver to start
the decoding order recovery. the decoding order recovery.
skipping to change at page 17, line 14 skipping to change at page 16, line 14
6. IANA Considerations 6. IANA Considerations
(tbd - this needs to register the new RTP/AVPF transport layer (tbd - this needs to register the new RTP/AVPF transport layer
feedback packet type) feedback packet type)
7. Acknowledgements 7. Acknowledgements
This memo has benefitted from discussions with numerous members of This memo has benefitted from discussions with numerous members of
the IETF AVT working group, including Magnus Westerlund, Randell the IETF AVT working group, including Magnus Westerlund, Randell
Jesup, and Jonathan Lennox. The header extension format of Variant A Jesup, Jonathan Lennox, Gerard Babonneau, Ingemar Johansson, and Roni
in Section 3.2 was suggested by Dave Singer, matching a similar Even. The header extension format of Variant A in Section 3.2 was
mechanism specified by ISMA. suggested by Dave Singer, matching a similar mechanism specified by
ISMA.
8. References 8. References
8.1. Normative References 8.1. Normative References
[1] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, [1] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson,
"RTP: A Transport Protocol for Real-Time Applications", STD 64, "RTP: A Transport Protocol for Real-Time Applications", STD 64,
RFC 3550, July 2003. RFC 3550, July 2003.
[2] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, [2] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
 End of changes. 13 change blocks. 
57 lines changed or deleted 58 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/