TCP Hollywood at IFIP Networking 2016
19 May 2016
/ tcp-hollywood
Stephen McQuistin presented our paper
TCP Hollywood: An Unordered, Time-Lined, TCP for Networked Multimedia
Applications at the
IFIP Networking 2016 conference in Vienna, Austria, on 19 May 2016
(slides).
This paper describes the design of TCP Hollywood, and presents analysis
showing when it will improve performance compared to standard TCP.
Ossification of the transport-layer forces networked multimedia
applications to use TCP or UDP, but neither protocol is well suited to
the task. UDP contributes minimal latency, making it the recommended
transport to meet the strict latency bounds of real-time applications,
but provides limited support to applications, and is commonly blocked
by enterprise firewalls. TCP prefers reliability to timeliness, and its
congestion control tends to drive up queueing delay, but is often the
only transport that can pass through middleboxes on the path.
Accordingly, and despite its many problems, TCP is rapidly becoming the
de facto transport for multimedia traffic. To improve transport for
these applications, we present TCP Hollywood, an unordered, time-lined,
TCP variant designed to support real-time multimedia traffic while
being widely deployable.
TCP Hollywood is wire compatible with standard TCP, but eliminates
two sources of transport-induced latency, and provides reliability
semantics that better suit real-time multimedia applications.
Specifically, TCP Hollywood: 1) removes head-of-line blocking at the
receiver and delivers received data to the application immediately,
irrespective of ordering; and 2) relaxes reliability to respect time
lines provided by the application, so only data that will arrive in
time is retransmitted, otherwise retransmissions carry new data. The
combination of both design elements reduces latency and introduces
message-oriented semantics, allowing TCP Hollywood to express
inter-dependencies between messages. Crucially, TCP Hollywood is
wire-compatible with TCP, and incrementally deployable on the public
Internet.
Head-of-line blocking occurs with TCP when packet loss occurs. Because
TCP provides a reliable, ordered, byte stream service, any lost packets
must be retransmitted before the following data can be delivered to the
application. This can take a long time, leading to a stall in the media
play-out of a real-time application while waiting for retransmission.
TCP Hollywood addresses this in two ways. Firstly, by adding a framing
layer, using COBS encoding implemented in an user-space intermediary
layer that sits over a modified TCP stack, and provides a message-oriented
delivery service. This allows complete messages received after a packet
loss to be delivered to the application out of order, without waiting
for retransmission.
Secondly, TCP Hollywood allows inconsistent retransmissions.
That is, data that cannot be delivered before the application deadline
is replaced by new data, sent in a TCP segment with the same sequence
number as the lost data to complete the TCP sequence number space. We
extend the Sockets API to allow applications to specify deadlines for
delivery of messages, allowing the kernel to determine, based on RTT
and deadline, when a message has expired and when to send new data in
a retransmitted segment. We also allow dependencies between messages to
be signalled, to avoid sending data that depends on a previous message
that was known to be lost. Finally, because TCP Hollywood provides a
message delivery service, rather than a byte stream, it's straight
forward to provide demultiplexing of messages onto separate channels.
The structure of an TCP Hollywood implementation is shown in the figure
below. It comprises an intermediary layer that provides the message
segmentation and channel handling, plus extensions to the kernel TCP
stack that implement inconsistent retransmission, expiry of messages
that will not arrive in time, and dependency tracking.
The benefits of TCP Hollywood for reducing latency depend on the
relative values of the network round-trip time, Trtt,
the media framing interval, and the play-out delay. A real-time
application will have a bound on the arrival time of data if it is
to be played out in order, and with the correct timing. We denote
this bound as Tmax. Taking into account this bound, the
amount of media encoded into each message Tframing, and
Trtt, we derive an upper bound on the play-out delay at the
receiver of Tplayout ≤ Tmax − Tframing − Trtt/2
if the media is to arrive in time to be useful. We denote this as
the application deadline. We also know the retransmission
time for a lost segment, being equal to the sum of the round-trip
time, the duration of the media encoded in a segment, and the time for
the following three segments to arrive in order to generate a triple
duplicate ACK. This gives a lower-bound on the play-out delay, if the
retransmission is to be useful. As the figure below shows, this gives
a limited region where the play-out delay is small enough that the
application deadline is met, but large enough for a retransmission to
arrive, and a potentially larger region where retransmissions arrive
too late to be useful. It is this region of wasted TCP retransmissions
that we target with inconsistent retransmissions.
We expand on this analysis
in the
paper. Assuming a typical 150ms delay bound for voice telephony,
inconsistent retransmissions can prevent stalls on intercontinental
calls running over TCP Hollywood, although they offer limited benefit
for more local calling.
Our NOSSDAV 2016
paper discusses other scenarios where inconsistent retransmissions
can have practical benefit for video streaming.
TCP Hollywood ought to be entirely compatible with TCP. The only
on-the-wire visible difference between a TCP Hollywood flow and a
standard TCP flow appears within the payload data carried by
inconsistent retransmissions. Recall from Section III-A that
inconsistent retransmissions carry new payload data inside of segments
with previously transmitted sequence numbers. This modification is
invisible to receivers and middleboxes that only process TCP/IP
headers, but is visible to middleboxes that use deep packet inspection
if they compare the contents of a retransmitted packet with the
original data. Depending on the configuration such behaviour may
disrupt the connection. For example, a firewall may interpret
inconsistent retransmissions as belonging to a man-on-the-side attack,
and reset the connection.
We conducted experiments with a live deployment of TCP Hollywood to
obtain an initial assessment on whether such middleboxes exist, and
what impact they have. A TCP Hollywood server was setup on a public IP
address, and configured to always send inconsistent retransmissions in
lieu of the original data, so that all retransmissions contained new
data with the same sequence numbers. The server was configured to
listen on ports 80, 4001, and 5001. Port 80 is used by web traffic, and
can be expected to be affected by middleboxes such as "transparent"
caches and firewalls. We expect ports 4001 and 5001 to be less likely
to be subject to interference by middleboxes, since they are not used
by popular applications.
The evaluation was conducted using clients in 14 different locations in
the UK, connecting to a server located at the University of Glasgow.
The clients connected via eight different fixed-line residential ISPs
(Andrews & Arnold, BT, Demon, EE, Eclipse, Sky, TalkTalk, and Virgin),
and four mobile operators (EE, O2, Three, and Vodafone). All of the
fixed-line residential ISPs successfully delivered the inconsistent
retransmissions. In contrast only one out of the four mobile operators
delivered inconsistent retransmissions. The three remaining mobile
operators delivered the original segments instead, while the server saw
no corresponding segment loss. The observed behaviour is consistent
with a transparent split-connection TCP performance enhancing proxy
cache that intercepts and responds to ACKs from the client on behalf of
the server. On two of the three providers, this caching behaviour was
seen on both port 80 and port 4001, while the other provider appeared
to operate a cache on port 80 only.
Crucially, TCP Hollywood continued to operate whether or not the
middlebox was present in the network. At no time did connections
suffer a reset, and the use of the TCP Hollywood extensions did not
affect connectivity or performance. Our sample size is small, but this
experiment provides evidence that TCP Hollywood continues to deliver
messages and eliminate head-of-line blocking, even when inconsistent
retransmissions are absent. In the worst-case, performance is the same
as TCP without our extensions.