DETNET Q. Xiong Internet-Draft A. Liu Intended status: Informational ZTE Corporation Expires: 28 August 2024 25 February 2024 Gap Analysis for Enhanced DetNet draft-xiong-detnet-enhanced-detnet-gap-analysis-03 Abstract From charter and milestones, the enhanced Deterministic Networking (DetNet) is required to provide the enhancement of flow identification and packet treatment for data plane to achieve the DetNet QoS in large-scale networks. This document discusses the characteristics of scaling deterministic networks and analyzes the gaps of the existing technologies especially applying the DetNet data plane as per RFC8938. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 28 August 2024. Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. Xiong & Liu Expires 28 August 2024 [Page 1] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Conventions used in this document . . . . . . . . . . . . . . 3 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 3 3. Characteristics of Scaling Deterministic Networks . . . . . . 3 3.1. Large-scale Dynamic Flows . . . . . . . . . . . . . . . . 3 3.1.1. Flows with Different T-Spec . . . . . . . . . . . . . 3 3.1.2. Flows with Different Levels of Applications . . . . . 4 3.1.3. Flows with Different SLAs . . . . . . . . . . . . . . 4 3.2. Large-scale Network Topology . . . . . . . . . . . . . . 4 3.2.1. Large Number of Hops and Complex Topology within a DetNet Domain . . . . . . . . . . . . . . . . . . . . 4 3.2.2. Long Distance links within a DetNet Domain . . . . . 5 3.2.3. Topology across Multiple Domains . . . . . . . . . . 5 3.2.4. Topology across Heterogeneous Networks . . . . . . . 5 4. Gap Analysis for Enhanced DetNet . . . . . . . . . . . . . . 5 4.1. Gap Analysis of Providing Flows Identification . . . . . 5 4.2. Gap Analysis of Providing Deterministic Latency . . . . . 6 4.2.1. Gap Analysis of Explicit Routes . . . . . . . . . . . 6 4.2.2. Gap Analysis of Resources Allocation . . . . . . . . 7 4.2.3. Gap Analysis of Queuing Mechanisms . . . . . . . . . 7 5. Security Considerations . . . . . . . . . . . . . . . . . . . 8 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 8 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 8. Normative References . . . . . . . . . . . . . . . . . . . . 8 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 1. Introduction As per [RFC8655], it defined the overall architecture for Deterministic Networking (DetNet) , which provides a capability for real-time applications with extremely low data loss rates and bounded latency within a network domain. It has three goals: minimum and maximum end-to-end latency from source to destination, bounded jitter (packet delay variation), packet loss ratio and upper bound on out- of-order packet delivery. To achieve the above objectives, multiple techniques need to be used in combination, including explicit routes, Xiong & Liu Expires 28 August 2024 [Page 2] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 service protection and resource allocation defined by DetNet. As defined in [RFC8938], the DetNet data plane describes how application flows, or App-flows are carried over DetNet networks and it is provided by the DetNet service and forwarding sub-layers with DetNet-related data plane functions and mechanisms. The enhanced DetNet is required to provide the enhancement of flow identification and packet treatment for data plane to achieve the DetNet QoS in large-scale networks. [I-D.ietf-detnet-scaling-requirements] has described the enhanced DetNet data plane requirements for scaling deterministic networks. As per [I-D.zhao-detnet-enhanced-use-cases], various deterministic applications are co-existed with different SLAs guarantees in scaling networks. It is required to analyse the characteristics of the scaling networks and applicability for existing DetNet technologies. This document discusses the characteristics of scaling deterministic networks and analyzes the gaps of the existing technologies especially applying the DetNet data plane as per [RFC8938]. 2. Conventions used in this document 2.1. Terminology The terminology is defined as [RFC8655] and [RFC8938]. 2.2. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 3. Characteristics of Scaling Deterministic Networks 3.1. Large-scale Dynamic Flows 3.1.1. Flows with Different T-Spec As described in [RFC8557], deterministic forwarding can only apply to flows with such well-defined Traffic Specification (T-Spec) characteristics as periodicity and burstiness. As defined in DetNet architecture [RFC8655], the traffic characteristics of an App-flow can be CBR (constant bit rate) or VBR (variable bit rate) of L1, L2 and L3 layers (VBR takes the maximum value when reserving resources). But the current scenarios and technical solutions only consider CBR flow, without considering the coexistence of VBR and CBR, the burst Xiong & Liu Expires 28 August 2024 [Page 3] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 and aperiodicity of flows. The operations such as shaping or scheduling have not been specified. Even TSN mechanisms are based on a constant and forecastable traffic characteristics. It will be more complicated in a large-scale network where much more flows coexist and the traffic characteristics is more dynamic. A huge number of flows with different DetNet QoS requirements is dynamically concurrent and the state of each flow cannot be maintained. It is required to offer reliable delivery and SLA guarantee for dynamic flows. For example, periodic flow and aperiodic flow (including micro burst flow, etc.), CBR and VBR flow, flow with different periods or phases, etc. When the network needs to forward these deterministic flows at the same time, it must solve the problems of time micro bursts, queue processing and aggregation of multiple flows. 3.1.2. Flows with Different Levels of Applications In scaling networks, [I-D.ietf-detnet-scaling-requirements] has described the enhanced requirements for DetNet enhanced data plane including the deterministic latency guarantees and it also mentioned the enhanced DetNet should support different levels of application requirements which is an important requirement for the DetNet deployment. Moreover, mutiple services and traffic flows with different bounded latency requirements may be also co-existed in the same application. 3.1.3. Flows with Different SLAs In scaling networks, multiple flows may demand different set of SLAs and it may define more than one DetNet QoS levels according to different application scenarios as per [I-D.xiong-detnet-differentiated-detnet-qos]. These flows should be transmitted and forwarded with different DetNet QoS forwarding behaviors. 3.2. Large-scale Network Topology 3.2.1. Large Number of Hops and Complex Topology within a DetNet Domain In scaling networks, the topology may consists of a large number nodes and links which may lead to burst accumulation when a flow may traverse a path with a large number of hops. And it may also impact the controlling of end-to-end delay and jitter with the increasing of transmission hops. And the topology may be complex including star, ring, mesh, and their combinations can possibly be hierarchical. It may lead to the difficulty with path computation. Xiong & Liu Expires 28 August 2024 [Page 4] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 3.2.2. Long Distance links within a DetNet Domain In scaling networks, high speed, long-distance transmission and asymmetric links may also co-exists and affects the bounded latency such as increasing transmission latency, jitter and packet loss in large-scale networks. 3.2.3. Topology across Multiple Domains In scaling networks, the flows may be transmitted through the topology across multiple domains within a single administrative control or a closed group of administrative control as per [RFC8655]. The interworking of mechanisms within different domains, end-to-end path computation and resources scheduling with bounded latency constraint should be considered. 3.2.4. Topology across Heterogeneous Networks In scaling networks, the network topology may across heterogeneous networks and the DetNet domains or nodes may be interconnected with different sub-network technologies such as FlexE tunnels, TSN sub- network, IP/MPLS/SRv6 tunnels and so on. It is required to support the inter-domain deterministic metric and routes to achieve the end- to-end bounded latency. 4. Gap Analysis for Enhanced DetNet As defined in [RFC8938], the DetNet data plane describes how application flows, or App-flows are carried over DetNet networks and it is provided by the DetNet service and forwarding sub-layers with DetNet-related data plane functions and mechanisms. This section analyzes the DetNet technical gaps when applying the DetNet data plane as per RFC8938 in large-scale networks. [I-D.xiong-detnet-large-scale-enhancements] has proposed the overall framework of DetNet enhancements for scaling deterministic networks based on the gaps. 4.1. Gap Analysis of Providing Flows Identification In [RFC8938], the DetNet data plane can provide the DetNet-Specific Metadata such as Flow-ID for both the service and forwarding sub- layers. The flow-based state information is required to be maintained for per-flow processing rules. For example, the resource reservation configuration is required for each flow. DetNet as per [RFC8938] provides the capability to aggregate the individual flows to downscale the operations of flow states. However, it still requires large amount of control signaling to establish and maintain DetNet flows. It may be challenging for network operations with a Xiong & Liu Expires 28 August 2024 [Page 5] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 large number of deterministic flows and network nodes in large-scale networks. It may consider the aggregation based on the flow classification to futher improve the scalability. And the flow identification is required to be dynamic and simplified to ensure the aggregated flows have compatible DetNet flow-specific characteristics. 4.2. Gap Analysis of Providing Deterministic Latency As described in [RFC8655], the primary goals are to achieve the DetNet QoS to provide minimum and maximum end-to-end latency and bounded jitter, low packet loss ratio and an upper bound on out-of- order packet delivery. But the data plane [RFC8938] particularly focuses on the DetNet service sub-layer which provides a set of Packet Replication, Elimination, and Ordering Functions (PREOF) functions to provide end-to-end service assurance. It mainly provides the capabilities for DetNet to guarantee the reliability. The DetNet forwarding sub-layer provides corresponding forwarding assurance with IETF existing functions using resource allocations and explicit routes. But these functions can not provide the deterministic latency (bounded latency, low packet loss and in-order delivery) assurance in large-scale networks. The following sections mainly discuss the gap analysis for the forwarding sub-layer functions to provide deterministic latency assurance. 4.2.1. Gap Analysis of Explicit Routes Traditional routes only have reachability. As per [RFC8938], explicit optimized paths with allocation of resources should be provided to achieve the DetNet QoS. But the deterministic requirements such as end-to-end delay and jitter are only used as path computation constraints. Multiple network metrics which are measured and distributed by the routing system should be taken into consideration. In large-scale networks, it may be challenging to compute the best path to meet all of the requirements. In multi-domain scenarios, the inter-domain deterministic routes need to be established and provisioned. Especially when interconnecting with sub-networks, the selection of intra-domain paths acrossing cooperating domains should consider the bounded latency in each domain and the stitching of the paths. Xiong & Liu Expires 28 August 2024 [Page 6] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 Moreover, the paths vary with the real-time change of the network topology. On the basic of the resources, the steering path and routes for deterministic flows should be programmed before the flows coming and able to provide SLA capability. And the routes should be considered to be established in distributed and centralized control Plane. As described in [RFC8557], the packet replication and elimination service protection should be provided to achieve the low packet loss ratio. It will copy the flows and spread the data over multiple disjoint forwarding paths. The bounded latency and jitter of each path should be meet service deterministic requirement. And the difference of latency within these paths should be limited. So the replication and elimination deterministic routes with configured latency and jitter policy should be taken into consideration. It is required to generate two disjoint paths with very close delay to form 1+1 protection and perform concurrent transmission and dual reception, and make replication and elimination on the egress PE. 4.2.2. Gap Analysis of Resources Allocation As per [RFC8938], the forwarding sub-layer uses buffer resources for packet queuing, as well as reservation and allocation of bandwidth capacity resources. The reservation of the bandwidth can not guarantee the deterministic latency. In large-scale networks, the bandwidth, buffer and scheduling resources are combined with queuing mechanisms to guarantee the deterministic latency. The deterministic resources may be include the resources that can guarantee the deterministic latency such as the nodes, links, interfaces, buffers, bandwidth, queuing and scheduling mechanisms and so on. The planning, reservation and allocation of deterministic resources should be taken into consideration in DetNet data plane. 4.2.3. Gap Analysis of Queuing Mechanisms As per [RFC8938], the forwarding sub-layer provides the QoS-related functions needed by the DetNet flow including the use of queuing techniques. But the queuing techniques which are defined in existing IETF technologies can not guarantee the bounded latency such as Active Queue Management(AQM). And the queuing mechanisms which are defined in IEEE802.1 TSN can not be directly applied in large-scale networks such Time Aware Shaping [IIEEE802.1Qbv] and Cyclic Queuing and Forwarding [IEEE802.1Qch] with time synchronization. Enhancement of queuing mechanisms have been discussed in DetNet such as cyclic-scheduling queuing mechanism (e.g. Tagged Cyclic Queuing and Forwarding (TCQF)[I-D.eckert-detnet-tcqf], and Cycle Specified Queuing and Forwarding (CSQF) Xiong & Liu Expires 28 August 2024 [Page 7] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 [I-D.chen-detnet-sr-based-bounded-latency]), deadline-scheduling queuing mechanism (e.g. Earlist Deadline Forwarding (EDF) [I-D.peng-detnet-deadline-based-forwarding]), timeslot-scheduling queuing mechanism (e.g. Timeslot Queuing and Forwarding (TQF) [I-D.peng-detnet-packet-timeslot-mechanism]), and asynchronous queuing mechanism (e.g. Work Conserving Stateless Core Fair Queuing (C-SCORE) [I-D.joung-detnet-stateless-fair-queuing]). The queuing- based requirements in DetNet enhanced data plane has been described in [I-D.ietf-detnet-scaling-requirements]. The function of multiple queuing mechanisms and related DetNet-Specific metadata should be defined in DetNet data plane as per [I-D.xiong-detnet-data-fields-edp]. 5. Security Considerations TBA 6. Acknowledgements TBA 7. IANA Considerations TBA 8. Normative References [I-D.chen-detnet-sr-based-bounded-latency] Chen, M., Geng, X., Li, Z., Joung, J., and J. Ryoo, "Segment Routing (SR) Based Bounded Latency", Work in Progress, Internet-Draft, draft-chen-detnet-sr-based- bounded-latency-03, 7 July 2023, . [I-D.eckert-detnet-tcqf] Eckert, T. T., Li, Y., Bryant, S., Malis, A. G., Ryoo, J., Liu, P., Li, G., Ren, S., and F. Yang, "Deterministic Networking (DetNet) Data Plane - Tagged Cyclic Queuing and Forwarding (TCQF) for bounded latency with low jitter in large scale DetNets", Work in Progress, Internet-Draft, draft-eckert-detnet-tcqf-05, 5 January 2024, . [I-D.ietf-detnet-scaling-requirements] Liu, P., Li, Y., Eckert, T. T., Xiong, Q., Ryoo, J., zhushiyin, and X. Geng, "Requirements for Scaling Xiong & Liu Expires 28 August 2024 [Page 8] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 Deterministic Networks", Work in Progress, Internet-Draft, draft-ietf-detnet-scaling-requirements-05, 20 November 2023, . [I-D.joung-detnet-stateless-fair-queuing] Joung, J., Ryoo, J., Cheung, T., Li, Y., and P. Liu, "Latency Guarantee with Stateless Fair Queuing", Work in Progress, Internet-Draft, draft-joung-detnet-stateless- fair-queuing-01, 19 October 2023, . [I-D.peng-detnet-deadline-based-forwarding] Peng, S., Du, Z., Basu, K., cheng, Yang, D., and C. Liu, "Deadline Based Deterministic Forwarding", Work in Progress, Internet-Draft, draft-peng-detnet-deadline- based-forwarding-08, 14 December 2023, . [I-D.peng-detnet-packet-timeslot-mechanism] Peng, S., Liu, P., Basu, K., Liu, A., Yang, D., and G. Peng, "Timeslot Queueing and Forwarding Mechanism", Work in Progress, Internet-Draft, draft-peng-detnet-packet- timeslot-mechanism-05, 14 December 2023, . [I-D.xiong-detnet-data-fields-edp] Xiong, Q., Liu, A., Gandhi, R., and D. Yang, "Data Fields for DetNet Enhanced Data Plane", Work in Progress, Internet-Draft, draft-xiong-detnet-data-fields-edp-01, 10 July 2023, . [I-D.xiong-detnet-differentiated-detnet-qos] Xiong, Q., Zhao, J., Du, Z., Zeng, Q., and C. Liu, "Differentiated DetNet QoS for Deterministic Services", Work in Progress, Internet-Draft, draft-xiong-detnet- differentiated-detnet-qos-00, 23 October 2023, . [I-D.xiong-detnet-large-scale-enhancements] Xiong, Q., Du, Z., Zhao, J., and D. Yang, "Enhanced DetNet Data Plane (EDP) Framework for Scaling Deterministic Networks", Work in Progress, Internet-Draft, draft-xiong- Xiong & Liu Expires 28 August 2024 [Page 9] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 detnet-large-scale-enhancements-03, 10 July 2023, . [I-D.zhao-detnet-enhanced-use-cases] Zhao, J., Xiong, Q., and Z. Du, "Enhanced Use cases for Scaling Deterministic Networks", Work in Progress, Internet-Draft, draft-zhao-detnet-enhanced-use-cases-00, 23 October 2023, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8557] Finn, N. and P. Thubert, "Deterministic Networking Problem Statement", RFC 8557, DOI 10.17487/RFC8557, May 2019, . [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", RFC 8578, DOI 10.17487/RFC8578, May 2019, . [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", RFC 8655, DOI 10.17487/RFC8655, October 2019, . [RFC8938] Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S. Bryant, "Deterministic Networking (DetNet) Data Plane Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020, . [RFC8956] Loibl, C., Ed., Raszuk, R., Ed., and S. Hares, Ed., "Dissemination of Flow Specification Rules for IPv6", RFC 8956, DOI 10.17487/RFC8956, December 2020, . [RFC8964] Varga, B., Ed., Farkas, J., Berger, L., Malis, A., Bryant, S., and J. Korhonen, "Deterministic Networking (DetNet) Data Plane: MPLS", RFC 8964, DOI 10.17487/RFC8964, January 2021, . Xiong & Liu Expires 28 August 2024 [Page 10] Internet-Draft Gap Analysis for Enhanced DetNet February 2024 [RFC9023] Varga, B., Ed., Farkas, J., Malis, A., and S. Bryant, "Deterministic Networking (DetNet) Data Plane: IP over IEEE 802.1 Time-Sensitive Networking (TSN)", RFC 9023, DOI 10.17487/RFC9023, June 2021, . [RFC9024] Varga, B., Ed., Farkas, J., Malis, A., Bryant, S., and D. Fedyk, "Deterministic Networking (DetNet) Data Plane: IEEE 802.1 Time-Sensitive Networking over MPLS", RFC 9024, DOI 10.17487/RFC9024, June 2021, . Authors' Addresses Quan Xiong ZTE Corporation No.6 Huashi Park Rd Wuhan Hubei, 430223 China Email: xiong.quan@zte.com.cn Aihua Liu ZTE Corporation China Email: liu.aihua@zte.com.cn Xiong & Liu Expires 28 August 2024 [Page 11]