Internet-Draft TreeDN August 2024
Giuliano, et al. Expires 22 February 2025 [Page]
Workgroup:
MOPS
Internet-Draft:
draft-ietf-mops-treedn-07
Published:
Intended Status:
Informational
Expires:
Authors:
L. Giuliano
Juniper Networks
C. Lenart
Verizon
R. Adam
GEANT

TreeDN- Tree-based CDNs for Live Streaming to Mass Audiences

Abstract

As Internet audience sizes for high-interest live events reach unprecedented levels and bitrates climb to support 4K/8K/Augmented Reality (AR), live streaming can place a unique type of stress upon network resources. TreeDN is a tree-based CDN architecture designed to address the distinctive scaling challenges of live streaming to mass audiences. TreeDN enables operators to offer Replication-as-a-Service (RaaS) at a fraction the cost of traditional, unicast-based CDNs- in some cases, at no additional cost to the infrastructure. In addition to efficiently utilizing network resources to deliver existing multi-destination traffic, this architecture also enables new types of content and use cases that previously were not possible or economically viable using traditional CDN approaches. Finally, TreeDN is a decentralized architecture and a democratizing technology in the way that it makes content distribution more accessible to more people by dramatically reducing the costs of replication.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 22 February 2025.

Table of Contents

1. Problem Statement

Live streaming to mass audiences can impose unique demands on network resources. For example, live sporting events broadcast over the Internet to end users has much lower tolerance for long playout buffers than typical on-demand video streaming. Viewers of live sporting events have long been conditioned by broadcast television to expect to see the content in real time, with only very short buffers for broadcast delays to prevent profanity and other objectionable content from making on the air (the "seven-second delay" [BROADCAST-DELAY]). With micro-betting, even this 5-10 second delay can be too long. By comparison, when watching on-demand movies, an extra one- or two-minute playout buffer tends to be perfectly acceptable for viewers. If playout buffers for live sports are that long, viewers run the risk of being alerted to the game winning score from text messages from friends or cheers from the bar across the street, minutes before they view it themselves.

Another unique characteristic of live streaming is join rate. While on-demand video streaming can consume massive amounts of network resources, the viewing rates tend to be smooth and predictable. Service Providers observe gradual levels of traffic increases over the evening hours corresponding to prime-time viewing habits. By comparison, viewing rates of live video streams can more closely resemble a step function with much less predictability as mass audiences of viewers tune in to watch the game at the same time.

Previous efforts at more efficient network replication of multi-destination traffic have experienced mixed success in terms of adoption. IP multicast is widely deployed on financial networks, video distribution networks, L3VPN networks and certain enterprises. But most of these deployments are restricted to "walled-garden" networks. Multicast over the global Internet has failed to gain traction, as only a very small portion of the Internet is multicast-enabled at this time.

TreeDN is a tree-based CDN architecture that is the result of the evolution of network-based replication mechanisms based on lessons learned from what has and has not worked well in the past. TreeDN addresses the fundamental issues of what has hindered multicast from adoption on the global Internet and enables service providers the opportunity to deliver new Replication-as-a-Service (RaaS) offerings to content providers, while more efficiently utilizing network resources by eliminating duplicated traffic, and thus, improving the experience of end users. TreeDN accomplishes this with the combination of a simplified model of native multicast along with network overlays to reach receivers on unicast-only parts of the Internet.

By more efficiently supporting multi-destination traffic, TreeDN is an architecture that can enable new types of content, such as Augmented Reality (AR) live streaming to mass audiences, that previously weren't possible or economically viable on the Internet due to the inefficiencies of unicast.

2. Applicability

While the primary use case mentioned throughout this document is live streaming of multimedia content (audio, video, AR, real-time telemetry data), the TreeDN architecture can provide efficient delivery for any content that needs to be replicated and delivered to multiple destinations. For example, large software file updates (eg, OS upgrades) that need to be delivered to many end users in a very short window of time can cause significant strain on network resources. Using TreeDN, this use case can be handled much more efficiently by the network.

3. Multicast Challenges in the Past

The following issues have been the some of the primary challenges for deployment of IP multicast over the global Internet. This is not intended to be an exhaustive list, but to rather to provide context to the solution and how it addresses these primary challenges.

TreeDN is the evolution of network-based replication based on lessons learned over decades and is designed to address the problems listed above.

4. TreeDN Architecture

TreeDN leverages a simplified model for multicast deployment combined with network overlays to deliver traffic to receiving hosts on unicast-only networks. With network overlays, a service can be achieved and delivered to end users while recognizing and tolerating the practical realities of what is possible over a network as diverse as the global Internet. That is, the replication service is available to users and applications across the global Internet regardless of what protocols may exist in the underlying networks that constitute the underlay.

                        TreeDN Provider
                +-------------------------------+
                |                               |
                |   Native Multicast On-Net     |
+----------+    |         (PIM-SSM)             |
| Content/ |----+                               |
| Mcast    |    |                               |
| Source   |    |                   +-----------+
+----------+    +---|-------|-------| AMT Relay |  +--------------+
                    |       |       +----|------+  | Unicast-Only |
                   +-+     +-+           .         |    Network   |
                   +-+     +-+           ..........|........      |
                 Native Content        AMT Tunnel  +-------.------+
                    Receivers                              .
                                                  AMT     +-+
                                                  Gateway +-+
                                                           |
                                                       Content
                                                       Receiver
Figure 1: TreeDN Provider Example

4.1. TreeDN Overlays

One overlay technology that TreeDN leverages is Automatic Multicast Tunneling (AMT) [RFC7450]. With AMT, end hosts on unicast-only networks (AMT Gateways) can dynamically build tunnels to routers on the multicast-enabled part of the network (AMT Relays) and receive multicast streams. The AMT Gateway is a thin software client which typically sits on the receiving end host and initiates the tunnel at an AMT Relay, which is a tunnel server that typically sits at the border of the multicast network. AMT allows any end host on the Internet to receive multicast content regardless of whether their local provider supports multicast (aka, "off-net receivers"), which addresses the "All or Nothing" Problem. Links and devices that do not support multicast are simply tunneled over- they no longer present a barrier to the overall replication service for end users. Those networks that do deploy and support multicast, as well as the content providers that serve up multicast content, are able to enjoy the benefits of efficient replication and delivery. Further, these benefits can serve as incentives for operators who do not yet support multicast to enable it on their networks, a key benefit of incremental deployment described in section 4.3 of [RFC9049]. Once the cost of carrying duplicated unicast tunnels is perceived by those operators to exceed the cost of deploying multicast, they are more likely to enable multicast on their networks. In this way, TreeDN effectively supports incremental deployment in a way that was not previously possible with traditional (non-overlay) multicast networking. Finally, AMT also addresses the "Chicken and Egg" Problem, as all end hosts on the global Internet that have access to an AMT Relay are capable of becoming audience members.

To support receiving on both native and non-native networks, receiving hosts can first attempt to join the traffic natively and, if no multicast traffic is received, fallback to AMT. This fallback mechanism can be handled by the application layer.

In addition to AMT, other overlay technologies like Locator/ID Separation Protocol (LISP) [RFC9300] can be utilized to deliver content from multicast-enabled networks to end hosts that are separated by portions of the network (at the last/middle/first mile) that do not support multicast.

4.2. TreeDN Native On-Net

Networks that support multicast provide the native on-net component of TreeDN. The primary requirement of the native on-net is to support Source-Specific Multicast (SSM) [RFC4607]. PIM-SSM, which is merely a subset of PIM-SM, is the multicast routing protocol typically used in SSM. However, any multicast routing protocol capable of supporting SSM can be used as a TreeDN native on-net, such as mLDP, Global Table Multicast (GTM) [RFC7716] and BGP-based Multicast [I-D.ietf-bess-bgp-multicast], or even BGP-MVPN [RFC6513] for those operators who carry the global routing table in a VRF. Likewise, any data plane technology that supports SSM, including BIER [RFC8279] and SR-P2MP [I-D.ietf-spring-sr-replication-segment] can be used.

The key benefit of SSM as the native on-net component of TreeDN is that it radically simplifies the control plane needed to support replication in the network. This simplification comes by moving source discovery from the network layer to some sort of out-of-band mechanism, usually in the application layer. In SSM, the receiver uses Internet Group Management Protocol, Version 3 (IGMPv3) [RFC3376] for IPv4 or Multicast Listener Discovery Version 2 (MLDv2) [RFC3810] for IPv6 to specify both the source and group address of the multicast stream. This allows the last hop router to immediately join the multicast stream along the shortest-path tree (SPT) without the need for shared trees. This benefit addresses the "It's Too Complex" Problem. By eliminating the need for network-based source discovery, most of the complexity of multicast is then eliminated, which reduces the cost of deploying and operating a multicast network. Further rationale for this SSM-only approach can be found in Any-Source Multicast (ASM) Deprecation [RFC8815].

5. Replication-as-a-Service (RaaS)

Content providers have traditionally used CDNs to distribute content that needs to be delivered to large audiences, essentially outsourcing the task of replication to CDN providers. Most CDNs utilize unicast delivery, as multicast is not an option due to its lack of general availability on the global Internet. TreeDN is a CDN architecture that leverages tree-based replication to more efficiently utilize network resources to deliver simultaneous multi-destination traffic. By leveraging overlay networking to address the "All or Nothing" and "Chicken and Egg" Problems and SSM to address the "It's Too Complex" Problem, TreeDN avoids the practical issues that previously prevented multicast from being a viable option for CDN providers.

TreeDN has several advantages over traditional unicast-based CDN approaches. First, the TreeDN functionality can be delivered entirely by the existing network infrastructure. Specifically, for operators with routers that support AMT natively, multicast traffic can be delivered directly to end users without the need for specialized CDN devices, which typically are servers that need to be racked, powered, cooled and connected to ports on routers that could otherwise have been consumed by paying customers. In this way, SPs can offer new RaaS functionality to content providers at potentially zero additional cost in new equipment.

Additionally, TreeDN is an open architecture that leverages mature, IETF-specified and widely implemented network protocols. TreeDN also requires far less coordination between the content provider and the CDN operator. That is, there are no storage requirements for the data, nor group-key management issues since a TreeDN provider merely forwards packets. A TreeDN provider simply needs to have enough accounting data (eg, traffic data, number of AMT tunnels, etc) to properly bill customers for the service. By contrast, traditional unicast-based CDNs often incorporate proprietary, non-interoperable technologies and require significant coordination between the content provider and the CDN to handle such things as file storage, data protection and key-management.

TreeDN introduces a deployment model that requires new considerations for transport layer mechanisms that are frequently relied upon by traditional unicast-based CDNs. A discussion on these considerations and differences can be found in section 7.

6. Decentralization/Democratization of Content Sourcing

TreeDN is an inherently decentralized architecture. This reduces the cost for content sourcing, as any host connected to a multicast-enabled network, or on a source-capable overlay, can send out a single data stream that can be reached by an arbitrarily large audience. By effectively reducing to zero the marginal cost of reaching each additional audience member, from the perspective of the source, TreeDN democratizes content sourcing on the Internet.

8. TreeDN Deployments

EUMETCast Terrestrial is a service from EUMETSAT that delivers meteorological satellite data to end users for purposes such as operational monitoring of climate and detection of global climate changes. EUMETCast Terrestrial connects to the GEANT network, which provides TreeDN services to deliver this real-time data natively to end users on multicast-enabled networks as well as to end users on unicast-only networks via a global deployment of AMT relays. Details of the EUMETCast Terrestrial service over the GEANT TreeDN network are described in [EUMETCast-TERRESTRIAL-over-AMT]. Additional details on how this deployment uses encryption, authorization, reliability and unicast feedback channels for end-to-end file delivery monitoring can be found in [EUMETSAT-TERRESTRIAL].

The Multicast Menu is a web-based portal that lists and can launch active multicast streams that are available on a global TreeDN network of various research and educations networks. Details of the this TreeDN network, as well as the Multicast Menu, are described in [Multicast-Menu].

The RARE network is a global testbed interconnecting several national research and education networks (NRENs) via routers running BIER. AMT relays are deployed to deliver multicast traffic from sources on the RARE network to receivers on unicast-only networks across the Internet. Details of the RARE network are described in [BIER-AMT-Deployment].

9. Operational Considerations

TreeDN is essentially the synthesis of SSM plus overlay networking technologies like AMT. As such, any existing tools to manage, operate and troubleshoot a PIM-SSM domain and AMT deployment can be used to manage a TreeDN deployment. Protocol error handling for PIM-SSM can be found in [RFC4607] and in section 4.8 of [RFC7761] and for AMT in [RFC7450].

One potential operational benefit of a multicast-based approach like TreeDN over traditional, unicast-based CDNs is the visibility that multicast state provides in the routing infrastructure. That is, multicast routers maintain a forwarding cache of multicast flows that usually includes the source address, group address, incoming/outgoing interfaces and forwarding rate. Generally speaking, such flow state information is not typically available in core networks for unicast, so additional tools outside the routing infrastructure are usually required for monitoring CDN performance and troubleshooting issues like packet loss location. Of course, this benefit comes at a cost of additional state being maintained in the routers for multicast.

Additionally, since multicast leverages reverse-path forwarding (RPF), the source of the content can potentially have a greater influence over the path taken through the network from source to native receivers/AMT relays. That is, the BGP peer advertising the reachability of the source's subnet can do so in ways that can prefer a particular path through the network for multicast distribution that are not as easy to accomplish with traditional, destination-based unicast routing.

10. Security Consideration

Since TreeDN is essentially the synthesis of SSM plus overlay networking technologies like AMT, the TreeDN architecture introduces no new security threats that are not already documented in SSM and the overlay technologies that comprise it. In particular, Section 6 of [RFC7450] candidly notes that AMT, like UDP, IGMP and MLD, provides no mechanisms for ensuring message delivery or integrity, nor does it provide confidentiality, since sources/groups joined through IGMP/MLD could be associated with the particular content being requested.

[RFC4609] and [RFC8815] describes the additional security benefits of using SSM instead of ASM.

11. IANA Considerations

This document has no IANA actions.

12. Acknowledgements

Many thanks to those who have contributed to building and operating the first TreeDN network on the Internet, including Pete Morasca, William Zhang, Lauren Delwiche, Natalie Landsberg, Wayne Brassem, Jake Holland, Andrew Gallo, Casey Russell, Janus Varmarken, Csaba Mate, Frederic Loui, Max Franke, Todor Moskov, Erik Herz, Bradley Cao, Katie Merrill, Karel Hendrych, Haruna Oseni and Isabelle Xiong. The writing of this document to describe the TreeDN architecture was inspired by a conversation with Dino Farinacci and Mike McBride. Thanks also to Jeff Haas, Vinod Kumar, Ron Bonica, Jeffrey Zhang and Eric Vyncke for their thoughtful reviews and suggestions, Chris Lemmons for his detailed shepherd review and Stephen Farrell, Magnus Westerlund, Reese Enghardt, Jurgen Schonwalder, Carlos Pignataro, Erik Kline, Gunter Van de Velde, Warren Kumari and Zaheduzzaman Sarker for their last call reviews.

13. References

13.1. Normative References

[RFC3376]
Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A. Thyagarajan, "Internet Group Management Protocol, Version 3", RFC 3376, DOI 10.17487/RFC3376, , <https://www.rfc-editor.org/info/rfc3376>.
[RFC3810]
Vida, R., Ed. and L. Costa, Ed., "Multicast Listener Discovery Version 2 (MLDv2) for IPv6", RFC 3810, DOI 10.17487/RFC3810, , <https://www.rfc-editor.org/info/rfc3810>.
[RFC4607]
Holbrook, H. and B. Cain, "Source-Specific Multicast for IP", RFC 4607, DOI 10.17487/RFC4607, , <https://www.rfc-editor.org/info/rfc4607>.
[RFC6388]
Wijnands, IJ., Ed., Minei, I., Ed., Kompella, K., and B. Thomas, "Label Distribution Protocol Extensions for Point-to-Multipoint and Multipoint-to-Multipoint Label Switched Paths", RFC 6388, DOI 10.17487/RFC6388, , <https://www.rfc-editor.org/info/rfc6388>.
[RFC7450]
Bumgardner, G., "Automatic Multicast Tunneling", RFC 7450, DOI 10.17487/RFC7450, , <https://www.rfc-editor.org/info/rfc7450>.
[RFC7761]
Fenner, B., Handley, M., Holbrook, H., Kouvelas, I., Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, , <https://www.rfc-editor.org/info/rfc7761>.

13.2. Informative References

[Algorhyme]
"Algorhyme", Wikipedia , n.d., <https://en.wikipedia.org/wiki/Radia_Perlman#Spanning_Tree_Protocol>.
[BIER-AMT-Deployment]
"BIER + AMT Deployment in GEANT/RARE Network", IETF112 Proceedings , n.d., <https://datatracker.ietf.org/meeting/112/materials/slides-112-mboned-bier-amt-depolyment-in-geantrare-network-00>.
[BROADCAST-DELAY]
"Broadcast Delay", Wikipedia , n.d., <https://en.wikipedia.org/wiki/Broadcast_delay>.
[DVB-MABR]
"Adaptive media streaming over IP multicast", DVB Document A176 Rev.3 (Fourth edition) , n.d., <https://dvb.org/wp-content/uploads/2022/01/A176r3_Adaptive-Media-Streaming-over-IP-Multicast_Interim-Draft-TS-103-769-v121_March_2023.pdf>.
[EUMETCast-TERRESTRIAL-over-AMT]
"EUMETCast Terrestrial over AMT", IETF115 Proceedings , n.d., <https://datatracker.ietf.org/meeting/115/materials/slides-115-mboned-eumetcast-over-amt>.
[EUMETSAT-TERRESTRIAL]
"EUMETSAT Terrestrial Service", IETF110 Proceedings , n.d., <https://datatracker.ietf.org/meeting/110/materials/slides-110-mboned-eumetsat-multicast-over-the-mbone-00>.
[I-D.ietf-bess-bgp-multicast]
Zhang, Z. J., Giuliano, L., Patel, K., Wijnands, I., Mishra, M. P., and A. Gulko, "BGP Based Multicast", Work in Progress, Internet-Draft, draft-ietf-bess-bgp-multicast-08, , <https://datatracker.ietf.org/doc/html/draft-ietf-bess-bgp-multicast-08>.
[I-D.ietf-ipsecme-g-ikev2]
Smyslov, V. and B. Weis, "Group Key Management using IKEv2", Work in Progress, Internet-Draft, draft-ietf-ipsecme-g-ikev2-13, , <https://datatracker.ietf.org/api/v1/doc/document/draft-ietf-ipsecme-g-ikev2/>.
[I-D.ietf-spring-sr-replication-segment]
Voyer, D., Filsfils, C., Parekh, R., Bidgoli, H., and Z. J. Zhang, "SR Replication segment for Multi-point Service Delivery", Work in Progress, Internet-Draft, draft-ietf-spring-sr-replication-segment-19, , <https://datatracker.ietf.org/doc/html/draft-ietf-spring-sr-replication-segment-19>.
[I-D.jholland-quic-multicast]
Holland, J., Pardue, L., and M. Franke, "Multicast Extension for QUIC", Work in Progress, Internet-Draft, draft-jholland-quic-multicast-05, , <https://datatracker.ietf.org/doc/html/draft-jholland-quic-multicast-05>.
[MAUD]
"Multicast-Assisted Unicast Delivery", IBC2023 Tech Papers , n.d., <https://www.ibc.org/technical-papers/ibc2023-tech-papers-multicast-assisted-unicast-delivery/10235.article>.
[Multicast-Menu]
"Offnet Sourcing with the Multicast Menu", IETF114 Proceedings , n.d., <https://datatracker.ietf.org/meeting/114/materials/slides-114-mboned-offnet-sourcing-with-the-multicast-menu-01>.
[RFC4609]
Savola, P., Lehtonen, R., and D. Meyer, "Protocol Independent Multicast - Sparse Mode (PIM-SM) Multicast Routing Security Issues and Enhancements", RFC 4609, DOI 10.17487/RFC4609, , <https://www.rfc-editor.org/info/rfc4609>.
[RFC5740]
Adamson, B., Bormann, C., Handley, M., and J. Macker, "NACK-Oriented Reliable Multicast (NORM) Transport Protocol", RFC 5740, DOI 10.17487/RFC5740, , <https://www.rfc-editor.org/info/rfc5740>.
[RFC6513]
Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, , <https://www.rfc-editor.org/info/rfc6513>.
[RFC7716]
Zhang, J., Giuliano, L., Rosen, E., Ed., Subramanian, K., and D. Pacella, "Global Table Multicast with BGP Multicast VPN (BGP-MVPN) Procedures", RFC 7716, DOI 10.17487/RFC7716, , <https://www.rfc-editor.org/info/rfc7716>.
[RFC8085]
Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, , <https://www.rfc-editor.org/info/rfc8085>.
[RFC8279]
Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., Przygienda, T., and S. Aldrin, "Multicast Using Bit Index Explicit Replication (BIER)", RFC 8279, DOI 10.17487/RFC8279, , <https://www.rfc-editor.org/info/rfc8279>.
[RFC8777]
Holland, J., "DNS Reverse IP Automatic Multicast Tunneling (AMT) Discovery", RFC 8777, DOI 10.17487/RFC8777, , <https://www.rfc-editor.org/info/rfc8777>.
[RFC8815]
Abrahamsson, M., Chown, T., Giuliano, L., and T. Eckert, "Deprecating Any-Source Multicast (ASM) for Interdomain Multicast", BCP 229, RFC 8815, DOI 10.17487/RFC8815, , <https://www.rfc-editor.org/info/rfc8815>.
[RFC9000]
Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, , <https://www.rfc-editor.org/info/rfc9000>.
[RFC9049]
Dawkins, S., Ed., "Path Aware Networking: Obstacles to Deployment (A Bestiary of Roads Not Taken)", RFC 9049, DOI 10.17487/RFC9049, , <https://www.rfc-editor.org/info/rfc9049>.
[RFC9300]
Farinacci, D., Fuller, V., Meyer, D., Lewis, D., and A. Cabellos, Ed., "The Locator/ID Separation Protocol (LISP)", RFC 9300, DOI 10.17487/RFC9300, , <https://www.rfc-editor.org/info/rfc9300>.
[Trees]
"Trees", Poetry Foundation , n.d., <https://www.poetryfoundation.org/poetrymagazine/poems/12744/trees>.

Appendix A. Netverses

With inspiration from (and apologies to) Radia Perlman [Algorhyme] and Joyce Kilmer [Trees], the following poem is not intended to provide any normative or informative technical value on TreeDN beyond (mild) amusement for the reader who made it this far in the document:

I think that I shall never see
A CDN more lovely than a tree.

A tree whose crucial property
Is efficient mass-audience delivery.

Using SSM for simplified operation
Of native branches that eliminate duplication.

A tree extended by AMT,
Enabling unicast-only receivers full delivery.

A tree that scales to reach millions of places
To viably support the highest of bitrate use cases.

A CDN is built by folks like me,
But only end users can generate enough demand to necessitate a tree.

Authors' Addresses

Lenny Giuliano
Juniper Networks
2251 Corporate Park Drive
Herndon, VA 20171,
United States of America
Chris Lenart
Verizon
22001 Loudoun County Parkway
Ashburn, VA 20147,
United States of America
Rich Adam
GEANT
City House
126-130 Hills Road
Cambridge
CB2 1PQ
United Kingdom