Architecture and Requirements for Transport Services

1.1. Background

The architecture of the Transport Services System is based on the survey of services provided by IETF transport protocols and congestion control mechanisms [RFC8095], and the distilled minimal set of the features offered by transport protocols [RFC8923]. These documents identified common features and patterns across all transport protocols developed thus far in the IETF.¶

Since transport security is an increasingly relevant aspect of using transport protocols on the Internet, this document also considers the impact of transport security protocols on the feature-set exposed by Transport Services [RFC8922].¶

One of the key insights to come from identifying the minimal set of features provided by transport protocols [RFC8923] was that features either require application interaction and guidance (referred to in that document as Functional or Optimizing Features), or else can be handled automatically by an implementation of the Transport Services System (referred to as Automatable Features). Among the identified Functional and Optimizing Features, some are common across all or nearly all transport protocols, while others present features that, if specified, would only be useful with a subset of protocols, but would not harm the functionality of other protocols. For example, some protocols can deliver messages faster for applications that do not require messages to arrive in the order in which they were sent. This functionality needs to be explicitly allowed by the application, since reordering messages would be undesirable in many cases.¶

1.2. Overview

This document describes the Transport Services System in three sections:¶

• Section 2 describes how the Transport Services API model differs from that of traditional socket-based APIs. Specifically, it offers asynchronous event-driven interaction, the use of messages for data transfer, and the flexibility to use different transport protocols and paths without requiring major changes to the application.¶

• Section 3 explains the fundamental requirements for a Transport Services System. These principles are intended to make sure that transport protocols can continue to be enhanced and evolve without requiring significant changes by application developers.¶

• Section 4 presents the Transport Services Implementation and defines the concepts that are used by the API [I-D.ietf-taps-interface] and described in the implementation guidelines [I-D.ietf-taps-impl]. This introduces the Preconnection, which allows applications to configure Connection Properties.¶

1.3. Specification of Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

1.4. Glossary of Key Terms

This subsection provides a glossary of key terms related to the Transport Services architecture. It provides a short description of key terms that are later defined in this document.¶

• Application: An entity that uses the transport layer for end-to-end delivery of data across the network [RFC8095].¶

• Cached State: The state and history that the Transport Services Implementation keeps for each set of the associated Endpoints that have been used previously.¶

• Candidate Path: One path that is available to an application and conforms to the Selection Properties and System Policy during racing.¶

• Candidate Protocol Stack: One Protocol Stack that can be used by an application for a Connection during racing.¶

• Client: The peer responsible for initiating a Connection.¶

• Clone: A Connection that was created from another Connection, and forms a part of a Connection Group.¶

• Connection: Shared state of two or more Endpoints that persists across Messages that are transmitted and received between these Endpoints [RFC8303]. When this document (and other Transport Services documents) use the capitalized "Connection" term, it refers to a Connection object that is being offered by the Transport Services system, as opposed to more generic uses of the word "connection".¶

• Connection Context: A set of stored properties across Connections, such as cached protocol state, cached path state, and heuristics, which can include one or more Connection Groups.¶

• Connection Group: A set of Connections that share properties and caches.¶

• Connection Property: A Transport Property that controls per-Connection behavior of a Transport Services implementation.¶

• Endpoint: An entity that communicates with one or more other endpoints using a transport protocol.¶

• Endpoint Identifier: An identifier that specifies one side of a Connection (local or remote), such as a hostname or URL.¶

• Equivalent Protocol Stacks: Protocol Stacks that can be safely swapped or raced in parallel during establishment of a Connection.¶

• Event: A primitive that is invoked by an Endpoint [RFC8303].¶

• Framer: A data translation layer that can be added to a Connection to define how application-layer Messages are transmitted over a Protocol Stack.¶

• Local Endpoint: The local Endpoint.¶

• Local Endpoint Identifier: A representation of the application's identifier for itself that it uses for a Connection.¶

• Message: A unit of data that can be transferred between two Endpoints over a Connection.¶

• Message Property: A property that can be used to specify details about Message transmission, or obtain details about the transmission after receiving a Message.¶

• Parameter: A value passed between an application and a transport protocol by a primitive [RFC8303].¶

• Path: A representation of an available set of properties that a Local Endpoint can use to communicate with a Remote Endpoint.¶

• Peer: An Endpoint application party to a Connection.¶

• Preconnection: an object that represents a Connection that has not yet been established.¶

• Preference: A preference to prohibit, avoid, ignore, prefer, or require a specific Transport Feature.¶

• Primitive: A function call that is used to locally communicate between an application and an Endpoint, which is related to one or more Transport Features [RFC8303].¶

• Protocol Instance: A single instance of one protocol, including any state necessary to establish connectivity or send and receive Messages.¶

• Protocol Stack: A set of Protocol Instances that are used together to establish connectivity or send and receive Messages.¶

• Racing: The attempt to select between multiple Protocol Stacks based on the Selection and Connection Properties communicated by the application, along with any Security Parameters.¶

• Remote Endpoint: The peer that a local Endpoint can communicate with when a Connection is established.¶

• Remote Endpoint Identifier: A representation of the application's identifier for a peer that can participate in establishing a Connection.¶

• Rendezvous: The action of establishing a peer-to-peer Connection with a Remote Endpoint.¶

• Security Parameters: Parameters that define an application's requirements for authentication and encryption on a Connection.¶

• Server: The peer responsible for responding to a Connection initiation.¶

• Socket: The combination of a destination IP address and a destination port number [RFC8303].¶

• System Policy: The input from an operating system or other global preferences that can constrain or influence how an implementation will gather Candidate Paths and Protocol Stacks and race the candidates during establishment of a Connection.¶

• Selection Property: A Transport Property that can be set to influence the selection of paths between the Local and Remote Endpoints.¶

• Transport Feature: A specific end-to-end feature that the transport layer provides to an application.¶

• Transport Property: A property that expresses requirements, prohibitions and preferences [RFC8095].¶

• Transport Service: A set of transport features, without an association to any given framing protocol, that provides a complete service to an application.¶

• Transport Services Implementation: This consists of all objects and protocol instances used internally to a system or library to implement the functionality needed to provide a transport service across a network, as required by the abstract interface.¶

• Transport Services System: The Transport Services Implementation and the Transport Services API.¶

2.1. Event-Driven API

Originally, the Socket API presented a blocking interface for establishing connections and transferring data. However, most modern applications interact with the network asynchronously. Emulation of an asynchronous interface using the Socket API can use a try-and-fail model: If the application wants to read, but data has not yet been received from the peer, the call to read will fail. The application then waits and can try again later.¶

In contrast to the Socket API, all interactions using the Transport Services API are expected to be asynchronous. The API is defined around an event-driven model (see Section 4.1.6), which models this asynchronous interaction. Other forms of asynchronous communication could also be available to applications, depending on the platform implementing the interface.¶

For example, when an application that uses the Transport Services API wants to receive data, it issues an asynchronous call to receive new data from the Connection. When delivered data becomes available, this data is delivered to the application using asynchronous events that contain the data. Error handling is also asynchronous, resulting in asynchronous error events.¶

This API also delivers events regarding the lifetime of a connection and changes in the available network links, which were not previously made explicit in the Socket API.¶

Using asynchronous events allows for a more natural interaction model when establishing connections and transferring data. Events in time more closely reflect the nature of interactions over networks, as opposed to how the Socket API represents network resources as file system objects that may be temporarily unavailable.¶

Separate from events, callbacks are also provided for asynchronous interactions with the Transport Services API that are not directly related to events on the network or network interfaces.¶

2.2. Data Transfer Using Messages

The Socket API provides a message interface for datagram protocols like UDP, but provides an unstructured stream abstraction for TCP. While TCP has the ability to send and receive data as a byte-stream, most applications need to interpret structure within this byte-stream. For example, HTTP/1.1 uses character delimiters to segment messages over a byte-stream [RFC9112]; TLS record headers carry a version, content type, and length [RFC8446]; and HTTP/2 uses frames to segment its headers and bodies [RFC9113].¶

The Transport Services API represents data as messages, so that it more closely matches the way applications use the network. A message-based abstraction provides many benefits, such as:¶

• providing additional information to the Protocol Stack;¶

• the ability to associate deadlines with messages, for applications that care about timing;¶

• the ability to control reliability, which messages to retransmit when there is packet loss, and how best to make use of the data that arrived;¶

• the ability to automatically assign messages and connections to underlying transport connections to utilize multi-streaming and pooled connections.¶

Allowing applications to interact with messages is backwards-compatible with existing protocols and APIs because it does not change the wire format of any protocol. Instead, it provides the Protocol Stack with additional information to allow it to make better use of modern transport services, while simplifying the application's role in parsing data. For protocols that inherently use a streaming abstraction, framers (Section 4.1.5) bridge the gap between the two abstractions.¶

2.3. Flexible Implementation

The Socket API for protocols like TCP is generally limited to connecting to a single address over a single interface (IP source address). It also presents a single stream to the application. Software layers built upon this API often propagate this limitation of a single-address single-stream model. The Transport Services architecture is designed:¶

• to handle multiple candidate endpoints, protocols, and paths;¶

• to support candidate protocol racing to select the most optimal stack in each situation;¶

• to support multipath and multistreaming protocols;¶

• to provide state caching and application control over it.¶

A Transport Services Implementation is intended to be flexible at connection establishment time, considering many different options and trying to select the most optimal combinations by racing them and measuring the results (see Section 4.2.1 and Section 4.2.2). This requires applications to specify identifiers for the Local and Remote Endpoint that are higher-level than IP addresses, such as a hostname or URL, which are used by a Transport Services Implementation for resolution, path selection, and racing. An implementation can further implement fallback mechanisms if connection establishment of one protocol fails or performance is detected to be unsatisfactory.¶

Information used in connection establishment (e.g. cryptographic resumption tokens, information about usability of certain protocols on the path, results of racing in previous connections) are cached in the Transport Services Implementation. Applications have control over whether this information is used for a specific establishment, in order to allow tradeoffs between efficiency and linkability.¶

Flexibility after connection establishment is also important. Transport protocols that can migrate between multiple network-layer interfaces need to be able to process and react to interface changes. Protocols that support multiple application-layer streams need to support initiating and receiving new streams using existing connections.¶

2.4. Coexistence

While the architecture of the Transport Services System is designed as an enhanced replacement for the Socket API, it need not replace it entirely on a system or platform; indeed, coexistence has been recommended for incremental deployability [RFC8170]. The architecture is therefore designed such that it can run alongside (or, indeed, on top of) an existing Socket API implementation; only applications built to the Transport Services API are managed by the system's Transport Services Implementation.¶

3.1. Provide Common APIs for Common Features

Any functionality that is common across multiple transport protocols SHOULD be made accessible through a unified set of calls using the Transport Services API. As a baseline, any Transport Services API SHOULD allow access to the minimal set of features offered by transport protocols [RFC8923]. If that minimal set is updated or expanded in the future, the Transport Services API ought to be extended to match.¶

An application can specify constraints and preferences for the protocols, features, and network interfaces it will use via Properties. Properties are used by an application to declare its preferences for how the transport service should operate at each stage in the lifetime of a connection. Transport Properties are subdivided into Selection Properties, which specify which paths and Protocol Stacks can be used and are preferred by the application; Connection Properties, which inform decisions made during connection establishment and fine-tune the established connection; and Message Properties, set on individual Messages.¶

It is RECOMMENDED that the Transport Services API offers properties that are common to multiple transport protocols. This enables a Transport Services System to appropriately select between protocols that offer equivalent features. Similarly, it is RECOMMENDED that the Properties offered by the Transport Services API are applicable to a variety of network layer interfaces and paths, which permits racing of different network paths without affecting the applications using the API. Each is expected to have a default value.¶

It is RECOMMENDED that the default values for Properties are selected to ensure correctness for the widest set of applications, while providing the widest set of options for selection. For example, since both applications that require reliability and those that do not require reliability can function correctly when a protocol provides reliability, reliability ought to be enabled by default. As another example, the default value for a Property regarding the selection of network interfaces ought to permit as many interfaces as possible.¶

Applications using the Transport Services API need to be designed to be robust to the automated selection provided by the Transport Services System. This automated selection is constrained by the properties and preferences expressed by the application and requires applications to explicitly set properties that define any necessary constraints on protocol, path, and interface selection.¶

3.2. Allow Access to Specialized Features

There are applications that will need to control fine-grained details of transport protocols to optimize their behavior and ensure compatibility with remote systems. It is therefore RECOMMENDED that the Transport Services API and the Transport Services Implementation permit more specialized protocol features to be used.¶

A specialized feature could be needed by an application only when using a specific protocol, and not when using others. For example, if an application is using TCP, it could require control over the User Timeout Option for TCP [RFC5482]; these options would not take effect for other transport protocols. In such cases, the API ought to expose the features in such a way that they take effect when a particular protocol is selected, but do not imply that only that protocol could be used. For example, if the API allows an application to specify a preference to use the User Timeout Option, communication would not fail when a protocol such as UDP is selected.¶

Other specialized features, however, can also be strictly required by an application and thus further constrain the set of protocols that can be used. For example, if an application requires support for automatic handover or failover for a connection, only Protocol Stacks that provide this feature are eligible to be used, e.g., Protocol Stacks that include a multipath protocol or a protocol that supports connection migration. A Transport Services API needs to allow applications to define such requirements and constrain the options available to a Transport Services Implementation. Since such options are not part of the core/common features, it will generally be simple for an application to modify its set of constraints and change the set of allowable protocol features without changing the core implementation.¶

To control these specialized features, the application can declare its preference – whether the presence of a specific feature is prohibited, should be avoided, can be ignored, is preferred, or is required in the pre-establishment phase. An implementation of a Transport Services API would honor this preference and allow the application to query the availability of each specialized feature after a successful establishment.¶

3.3. Select Between Equivalent Protocol Stacks

A Transport Services Implementation can attempt and select between multiple Protocol Stacks based on the Selection and Connection Properties communicated by the application, along with any Security Parameters. The implementation can only attempt to use multiple Protocol Stacks when they are "equivalent", which means that the stacks can provide the same Transport Properties and interface expectations as requested by the application. Equivalent Protocol Stacks can be safely swapped or raced in parallel (see Section 4.2.2) during connection establishment.¶

The following two examples show non-equivalent Protocol Stacks:¶

• If the application requires preservation of message boundaries, a Protocol Stack that runs UDP as the top-level interface to the application is not equivalent to a Protocol Stack that runs TCP as the top-level interface. A UDP stack would allow an application to read out message boundaries based on datagrams sent from the remote system, whereas TCP does not preserve message boundaries on its own, but needs a framing protocol on top to determine message boundaries.¶

• If the application specifies that it requires reliable transmission of data, then a Protocol Stack using UDP without any reliability layer on top would not be allowed to replace a Protocol Stack using TCP.¶

The following example shows Equivalent Protocol Stacks:¶

• If the application does not require reliable transmission of data, then a Protocol Stack that adds reliability could be regarded as an Equivalent Protocol Stack as long as providing this would not conflict with any other application-requested properties.¶

A Transport Services Implementation can race different security protocols, e.g., if the System Policy is explicitly configured to consider them equivalent. A Transport Services implementation SHOULD only race Protocol Stacks where the transport security protocols within the stacks are identical. To ensure that security protocols are not incorrectly swapped, a Transport Services Implementation MUST only select Protocol Stacks that meet application requirements ([RFC8922]). A Transport Services Implementation MUST NOT automatically fall back from secure protocols to insecure protocols, or to weaker versions of secure protocols. A Transport Services Implementation MAY allow applications to explicitly specify which versions of a protocol ought to be permitted, e.g., to allow a minimum version of TLS 1.2 in case TLS 1.3 is not available.¶

A Transport Services Implementation MAY specify security properties relating to how the system operates (e.g., requirements, prohibitions, and preferences for the use of DNS Security Extensions (DNSSEC) or DNS over HTTPS (DoH)).¶

3.4. Maintain Interoperability

It is important to note that neither the Transport Services API [I-D.ietf-taps-interface] nor the guidelines for implementation of the Transport Service System [I-D.ietf-taps-impl] define new protocols or protocol capabilities that affect what is communicated across the network. A Transport Services System MUST NOT require that a peer on the other side of a connection uses the same API or implementation. A Transport Services Implementation acting as a connection initiator is able to communicate with any existing Endpoint that implements the transport protocol(s) and all the required properties selected. Similarly, a Transport Services Implementation acting as a Listener can receive connections for any protocol that is supported from an existing initiator that implements the protocol, independent of whether the initiator uses the Transport Services System or not.¶

A Transport Services Implemenation makes decisions that select protocols and interfaces. In normal use, a given version of a Transport Services System SHOULD result in consistent protocol and interface selection decisions for the same network conditions given the same set of Properties. This is intended to provide predictable outcomes to the application using the API.¶

3.5. Support Monitoring

The Transport Services API increases the layer of abstraction for applications, and it enables greater automation below the API. Such increased abstraction comes at the cost of increased complexity when application programmers, users or system administrators try to understand why any issues and failures may be happening. Transport Services systems should therefore offer monitoring functions that provide relevant debug and diagnostics information. For example, such monitoring functions could indicate the protocol(s) in use, the number of open connections per protocol, and any statistics that these protocols may offer.¶

4.1. Transport Services API Concepts

Fundamentally, a Transport Services API needs to provide Connection objects (Section 4.1.2) that allow applications to establish communication, and then send and receive data. These could be exposed as handles or referenced objects, depending on the chosen programming language.¶

Beyond the Connection objects, there are several high-level groups of actions that any Transport Services API needs to provide:¶

• Pre-establishment (Section 4.1.3) encompasses the properties that an application can pass to describe its intent, requirements, prohibitions, and preferences for its networking operations. These properties apply to multiple transport protocols, unless otherwise specified. Properties specified during pre-establishment can have a large impact on the rest of the interface: they modify how establishment occurs, they influence the expectations around data transfer, and they determine the set of events that will be supported.¶

• Establishment (Section 4.1.4) focuses on the actions that an application takes on the Connection objects to prepare for data transfer.¶

• Data Transfer (Section 4.1.5) consists of how an application represents the data to be sent and received, the functions required to send and receive that data, and how the application is notified of the status of its data transfer.¶

• Event Handling (Section 4.1.6) defines categories of notifications that an application can receive during the lifetime of a Connection. Events also provide opportunities for the application to interact with the underlying transport by querying state or updating maintenance options.¶

• Termination (Section 4.1.7) focuses on the methods by which data transmission is stopped, and connection state is torn down.¶

The diagram below provides a high-level view of the actions and events during the lifetime of a Connection object. Note that some actions are alternatives (e.g., whether to initiate a connection or to listen for incoming connections), while others are optional (e.g., setting Connection and Message Properties in pre-establishment) or have been omitted for brevity and simplicity.¶

     Pre-establishment     :       Established             : Termination
     -----------------     :       -----------             : -----------
                           :                               :
 +-- Local Endpoint        :           Message             :
 +-- Remote Endpoint       :    Receive() |                :
 +-- Transport Properties  :       Send() |                :
 +-- Security Parameters   :              |                :
 |                         :              |                :
 |               InitiateWithSend()       |        Close() :
 |   +---------------+   Initiate() +-----+------+ Abort() :
 +---+ Preconnection |------------->| Connection |-----------> Closed
     +---------------+ Rendezvous() +------------+         :
    Listen() |             :           |     |             :
             |             :           |     v             :
             v             :           | Connection        :
     +----------+          :           |   Ready           :
     | Listener |----------------------+                   :
     +----------+  Connection Received                     :
                           :                               :

In this diagram, the lifetime of a Connection object is divided into three phases: pre-establishment, the Established state, and Termination.¶

Pre-establishment is based around a Preconnection object, that contains various sub-objects that describe the properties and parameters of desired Connections (Local and Remote Endpoints, Transport Properties, and Security Parameters). A Preconnection can be used to start listening for inbound connections, in which case a Listener object is created, or can be used to establish a new connection directly using Initiate (for outbound connections) or Rendezvous (for peer-to-peer connections).¶

Once a Connection is in the Established state, an application can send and receive Message objects, and receive state updates.¶

Closing or aborting a connection, either locally or from the peer, can terminate a connection.¶

4.2. Transport Services Implementation

This section defines the key architectural concepts for the Transport Services Implementation within the Transport Services System.¶

The Transport Services System consists of the Transport Services Implementation and the Transport Services API. The Transport Services Implementation consists of all objects and protocol instances used internally to a system or library to implement the functionality needed to provide a transport service across a network, as required by the abstract interface.¶

• Path: Represents an available set of properties that a Local Endpoint can use to communicate with a Remote Endpoint, such as routes, addresses, and physical and virtual network interfaces.¶

• Protocol Instance: A single instance of one protocol, including any state necessary to establish connectivity or send and receive Messages.¶

• Protocol Stack: A set of Protocol Instances (including relevant application, security, transport, or Internet protocols) that are used together to establish connectivity or send and receive Messages. A single stack can be simple (a single transport protocol instance over IP), or it can be complex (multiple application protocol streams going through a single security and transport protocol, over IP; or, a multi-path transport protocol over multiple transport sub-flows).¶

• Candidate Path: One path that is available to an application and conforms to the Selection Properties and System Policy, of which there can be several. Candidate Paths are identified during the gathering phase (Section 4.2.1) and can be used during the racing phase (Section 4.2.2).¶

• Candidate Protocol Stack: One Protocol Stack that can be used by an application for a Connection, for which there can be several candidates. Candidate Protocol Stacks are identified during the gathering phase (Section 4.2.1) and are started during the racing phase (Section 4.2.2).¶

• System Policy: The input from an operating system or other global preferences that can constrain or influence how an implementation will gather candidate paths and Protocol Stacks (Section 4.2.1) and race the candidates during establishment (Section 4.2.2). Specific aspects of the System Policy either apply to all Connections or only certain ones, depending on the runtime context and properties of the Connection.¶

• Cached State: The state and history that the implementation keeps for each set of associated Endpoints that have been used previously. This can include DNS results, TLS session state, previous success and quality of transport protocols over certain paths, as well as other information. This caching does not imply that the same decisions are necessarily made for subsequent connections, rather, it means that cached state is used by a Transport Services Implementation to inform functions such as choosing the candidates to be raced, selecting appropriate transport parameters, etc. An application SHOULD NOT rely on specific caching behaviour, instead it ought to explicitly request any required or desired properties via the Transport Services API.¶