Cloud Hosting

Systems Approach Edge routers have been an essential part of the internet for decades, connecting access networks enterprise LANs, mobile and broadband networks to the global backbone.

These devices often have cryptic names MPLS VPN Provider Edge routers, S/P-Gateways in the case of cellular networks, and Broadband Network Gateways (BNG) in the case of fiber networks but they are, at their core, IP (L3) packet forwarders, sometimes augmented with features to support the business logic required by commercial access providers. But the world is changing, and the form and function of the edge router is changing with it.

To account for modern cloud technology, especially the rush to the edge, we expect it to be less common to think in terms of edge networks connecting to backbone networks. Instead, we will think in terms of local edge clouds connecting to global hyperscalers. Devices will request service from an edge cloud, which will sometimes forward requests to remote clouds (see for example, Cloudflare Workers and Fly.io), continuing the trend of true end-to-end connections being the exception.

The edge router will increasingly be realized as a disaggregated collection of virtual functions rather than by a physical box

L3 connectivity is still there, of course, but it will increasingly be an implementation detail. And as this transition happens, the L3 data plane will be subsumed into the switching fabric of the edge cloud, with the associated control plane (whether IETF-specified, 3GPP-specified, BBF-specified, or proprietary) implemented by microservices running in the cloud (at the edge or centralized).

That is, the edge router will increasingly be realized as a disaggregated collection of virtual functions rather than by a physical box, with control in the cloud and with the dataplane running on specialized infrastructure for speed and scale. In this sense, we see the paradigm introduced by SDN logically centralized control with distributed forwarding making its way to the edge.

SD-WANs are a current example of applying an SDN architecture to the edge, and more recently, cloud-delivered SASE (Secure Access Service Edge) services blend layers of security into the solution. But the pattern is much the same L3 packet forwarding in the data plane coupled with a rich cloud-based control plane with significant (functional) overlap with cloud native implementations of access gateways.

And with most of todays SD-WAN offerings being vertically integrated and proprietary, we would argue that the benefits of SDN (such as the ability for network operators to customize the functionality) are only partly delivered in these solutions today.

Once you stop thinking in terms of edge routers as special devices and start to view routing as yet another edge function, its a small step to realize that todays diverse set of edge routers are all fundamentally the same, and that it is possible to build a generalized (and disaggregated) edge routing capability that accommodates them all. This function can be centrally orchestrated and deployed, with functional elements running in multiple edges where case-specific packet processing needs to take place.

Easier said than done, of course, but it strikes me as a likely outcome, and worth a little forethought. The key insight is that all the scenarios outlined above have a similar structure, with L3 forwarding in the data plane augmented with support for:

Secure tunnels requiring encapsulation/decapsulation

Differentiated Service requiring Q-in-Q tagging and class-based queues

Billing & Accounting requiring per-flow counters

Policy Enforcement requiring access control rules

Observability requiring in-band network telemetry

And a microservice-based control plane that implements:

Authentication triggering changes to data plane tunnels

Subscriber Management triggering updates to per-flow counters and queues

Mobility & Routing triggering forwarding changes according to resource availability

Session & Policy Management triggering changes access control rules

Diagnostics & Anomaly Detection triggering changes to in-band network telemetry

All of the data plane features can be realized in P4-programming forwarding pipelines (more on that in a moment), where the triggering relationship in the list of control functions helps us understand how to craft a converged control/data-plane interface something that P4-Runtime (P4RT) supports.

An example of the generalized data plane already exists, and we describe it in our SDN book. Its the fabric.p4 program that implements the forwarding pipeline for ONFs SD-Fabric, which (a) implements L3 forwarding for the leaf-spine switching fabric you would find in an edge cloud, and (b) can be extended to connect different access network technologies (5Gs UPF and a PON-based BNG) to the internet.

The current implementation is a bit crude (it uses #ifdef), but the idea is clear: its possible to build an L3 forwarding pipeline that can be extended with access-specific plugins.

Popping up a level, one can imagine iterating on fabric.p4 until you have an extensible edge cloud data plane suitable for all of the use cases outlined above. The P4RT-generated interface could then support multiple control plane tenants, for example, allowing a 3GPP-defined core and an SD-WAN controller to independently set queue parameters, define encapsulation/decapsulation labels, install forwarding rules, and so on.

Converging on a shared data plane, but accepting that multiple control planes will co-exist, is a good starting point. But converging on the control plane is likely within reach as well, where we can expect a converged data plane to catalyze that process.

In my mind, its primarily a matter of aligning incentives for the various domains. Its already the case that the BBF is working towards a converged access network control plane that aligns with the 3GPP-defined mobile core, largely because Telcos have an incentive to make that happen.

Another good example is Magma, which defines a unified control plane and a programmable data plane for both RAN-based and Wi-Fi-based wireless networks. As enterprises start to roll out private 5G, the push to unify how they are managed will only increase.

The SD-WAN use case is more of a wild card. On the one hand, SD-WAN is surprisingly similar to SD-RAN in the functionality it needs from an edge router. On the other hand, SD-WAN offerings so far have resisted disaggregation. Of course the same was true of telco access networks, until recently.

Operators gain the ability to customize the functionality rather than just accepting the bundle that comes from the router vendor

If we accept that unification of edge routing is possible, a reasonable next question is: is it desirable? I would argue that the value will come first from disaggregation, as we have already seen in other environments such as the cloud data center.

Once the control plane is disaggregated from the data plane, innovation can happen more easily in both, and the operators of these devices gain the ability to customize the functionality rather than just accepting the bundle that comes from the router vendor.

And secondly, there is an opportunity to take a more holistic view of the edge, which offers the chance for applying consistent network policies that are independent of the access technology. But this is a topic for another post.

View post:
The internet's edge routers are all so different. What if we unified them with software? - The Register