STATEFUL NETWORK SLICE SELECTION USING SLICE SELECTOR AS CONNECTION TERMINATION PROXY
Some embodiments provide a method for establishing multiple virtual service networks over multiple datacenters. The method configures, for each virtual service network of the plurality of virtual service networks, a set of machines distributed across the datacenters to implement an ordered set of network services for the virtual service network. The method configures multiple service network selectors executing within the datacenters to receive a data message, select one of the virtual service networks for the data message based on analysis of contents of the data message, determine a location within the datacenters for a machine implementing a first network service of the ordered set of network services for the selected virtual service network, and transmit the data message to the machine implementing the first network service.
Communications service provider networks receive and process many types of traffic from many different types of devices, especially moving forward. For example, these networks will have traffic from mobile phones, Internet of Things (IoT) devices, self-driving automobiles, home computers, etc. Ideally, this traffic should be treated differently by the network based on the type of application (e.g., streaming video, web browsing, telephone calls, etc.), the type of device (e.g., data traffic for self-driving automobiles need extremely low latency), and other differentiators. While 4G and 5G standards have introduced a certain level of traffic differentiation, more adaptable network slicing, that can be generalized to other types of networks, is desirable.
BRIEF SUMMARYSome embodiments provide methods for establishing a virtual service network across a set of datacenters. The set of datacenters across which the virtual service network is established may include, e.g., one or more public clouds, a software-defined wide area network (SD-WAN) that spans public and private clouds, a telecommunications service provider access network (e.g., spanning a combination of the radio access network, edge clouds, and core clouds), or other types of datacenters. The virtual service network of some embodiments includes multiple network slices each of which provides different network services to data messages assigned to the network slice.
In some embodiments, when a device (e.g., a mobile endpoint device in the telecommunications context) transmits a data message onto such a network, a network slice selector initially processes the data message. The network slice selector assigns the data message to one of the network slices of the virtual service network and handles service chaining operations to ensure that the data message is processed by the correct set of network services for the assigned slice. In different embodiments, this network slice selector may be implemented by a virtual machine (VM), a containerized function, a software forwarding element (e.g., a flow-based forwarding element) operating within a VM, within a container or within virtualization software of a host computer, a set of modules executing outside of a forwarding element (e.g., between a VM and a port of a forwarding element) within virtualization software of a host computer, a hardware forwarding element (e.g., a programmable switch), or other implementations.
In some cases, many network slice selectors are configured to implement a virtual service network. In the telecommunications service provider example, some embodiments configure a network slice selector for each cellular tower, base station, or other aspect of the access network. The telecommunications service provider access network of some embodiments includes edge clouds for each cellular tower, and configures at least one network slice selector at each such edge cloud. In other examples (e.g., for SD-WAN traffic entirely contained within a set of connected datacenters), distributed network slice selectors are configured such that the network slice selection for a data message sent from a VM occurs at the same host computer as the source of the data message (though outside of the source VM) or at a designated device (e.g., a specific nearby switch or router, a dedicated VM).
Each network slice of a virtual service network, in some embodiments, includes one or more network services such as firewalls, load balancers, network address translation, metering (e.g., for billing purposes), virtual private network (VPN) gateways, radio access network (RAN) functions (e.g., distributed unit and centralized unit functions), evolved packet core (EPC) functions (e.g., home subscriber server, serving gateway, packet data network gateway, mobility management entity), or other types of network functions. These network functions may be implemented as virtual network functions (VNFs), physical network functions (PNFs), and/or cloud network functions (CNFs) in different embodiments.
When a network slice selector assigns a data message to a network slice, the slice selector is responsible in some embodiments for performing the service chaining to ensure that the data message traverses the network services of the assigned slice in the correct order. In some embodiments, the slice selector transmits the data message to the first network service (e.g., the VM, container, or other data compute node that implements the network service) and maintains context information for that data message. Upon the first network service completing its processing of the data message, the first network service returns the data message to the slice selector. The slice selector then uses the maintained context information to transmit the data message to the next network service, and so on. In some embodiments, when the full network slice is implemented across multiple datacenters, a similar service chaining module operates at each datacenter to handle the service chaining for the slice within its own datacenter. These service chaining modules may be implemented in the same manner as the network slice selectors in some embodiments (e.g., as VMs, as forwarding elements in VMs or virtualization software). A service chaining module of some embodiments receives a data message as the data message ingresses to the datacenter, identifies the slice for the data message (e.g., based on context information provided with the data message by the network slice selector or service chaining module of the previous datacenter), and provides the data message to the next network service within the datacenter. Other embodiments use distributed service chaining rather than returning data messages to a designated slice selector or service chaining module in each datacenter (e.g., by adding tags to the packet headers to indicate the order of services in a selected network slice).
In some embodiments, a controller hierarchy configures various entities within the one or more datacenters to implement a virtual service network. A high-level controller (referred to herein as a virtual service network (VSN) controller) receives configuration data for the virtual service network from a user (e.g., a telecommunications provider, a datacenter tenant) through an interface (e.g., a set of REST APIs, a graphical interface, a command line interface). This VSN controller coordinates sets of other controllers that configure the entities in the datacenters in which the VSN is implemented. In some embodiments, each datacenter has its own suite of lower-level controllers. These controllers may include compute controllers (e.g., for configuring VMs that implement the VNFs), network controllers (e.g., for configuring forwarding elements to transmit data messages between the slice selector(s) and the network services), storage controllers, and SDN controllers (e.g., for configuring the slice selectors and/or gateways that transmit data messages between the datacenters).
Network slice selectors may assign data messages to slices using different techniques in different embodiments. Slice selection may be based on a combination of layer 2 to layer 4 (L2-L4) headers and/or by performing deep packet inspection (e.g., to classify traffic based on data in the layer 5 to layer 7 (L5-L7) headers. For example, slice selection may be based simply on the source device by using the source network layer (e.g., IP) address, or may be based on the type of traffic and/or destination network domain by looking at the higher layer (L5-L7) headers. In some embodiments, the network slice selector integrates with other control plane components to collect additional information about a connection (e.g., regarding the user session, device type, or other data) and uses this information as part of the slice selection process (e.g., using only this collected information or combining this information with the L2-L4 and/or L5-L7 packet header data). In some embodiments, the network slice selector maintains state for mapping connections to network slices so that deep packet inspection does not need to be performed on each data message of a connection. In addition, for some connections, only certain data messages contain the L5-L7 header information required for performing the slice selection.
When performing network slice selection using deep packet inspection, in certain cases the initial data message for a connection may not include the L5-L7 header information that the slice selector needs to correctly identify the slice. For example, a connection between an endpoint device (e.g., a mobile device such as a smart phone or tablet, a laptop or desktop computer, an IoT device, a self-driving automobile, a smart camera belonging to a security system, or other device) and a network domain (e.g., a web domain such as www.netflix.com, www.google.com, etc.) often begins with a set of connection initiation messages such as a TCP handshake. After completion of the handshake, the device then sends, e.g., an http get message that includes the network domain. Subsequent data messages sent between the device and the network domain may not include such information.
In some embodiments, the network slice selector acts as a proxy to terminate the connection initiation messages without sending these messages across the virtual service network to the intended destination. That is, the network slice selector exchanges the appropriate set of connection initiation messages with the endpoint device, such that the endpoint device behaves as though the connection has been properly set up with the destination domain. The network slice selector stores data about these connection initiation messages for subsequent use. Upon receiving a data message from the endpoint device (e.g., an http get message), the network slice selector performs deep packet inspection to examine the L5-L7 headers and select one of the network slices for the connection. The network slice selector then exchanges the connection initiation messages with the destination network domain, this time acting as the device rather than the destination domain. These connection initiation messages are transmitted via the selected network slice, such that all of the network services on the selected network slice process the connection initiation messages. Once this connection is established with the destination domain, the slice selector forwards the data message to the destination domain as well, via the selected network slice. The slice selector also stores connection information (e.g., a connection 5-tuple) mapping the connection to the selected network slice, for use in processing data messages. For subsequent data messages received for the connection, the network slice selector uses the mapping of the connection to the slice to assign the data message to the selected slice without the need to perform deep packet inspection. This avoids performing deep packet inspection on every data message for the connection, which may not be possible as the required information might not be in the L5-L7 headers of all of the data messages belonging to the connection. In some embodiments, an egress gateway that processes data messages after completion of the virtual service network also stores the connection-to-slice mapping information, and uses this to assign data messages sent from the destination domain to the endpoint device to the correct network slice (with the network services performed in the opposite order).
Some embodiments, rather than terminating the connection initiation messages at the slice selector, pass these messages through to a default network slice initially, then replay the messages over the correct network slice for the connection after the network slice is selected. In this procedure, the slice selector assigns the initial connection initiation messages to the default slice while storing these messages (or the data required to recreate the messages). In addition, the egress gateway (i.e., a gateway that transmits the messages out to the destination (e.g., to the Internet) and receives reverse-direction messages from the destination domain) stores information regarding the connection initiation messages as well. Upon receiving the initial data message, the network slice selector performs deep packet inspection to select the correct slice. Before transmitting the data messages onto the selected slice, however, the slice selector replays the connection initiation messages over the network slice. These messages are not sent from the egress gateway out to the destination domain; rather, the connection initiation messages are sent only between the slice selector and the egress gateway. Once these messages have been replayed, the network services of the selected slice will have processed the connection initiation messages and thus are prepared for the subsequent data message traffic between the endpoint device and the destination network domain.
In the case of stateful slice selection, as mentioned, subsequent data messages are sent using the state stored by the network slice selector, both for resource/time savings and because many of the subsequent data messages do not have the information in their L5-L7 headers necessary for deep packet inspection to be useful. However, mobile devices (e.g., smart phones, tablets, self-driving automobiles) may move from one geographic range served by a first slice selector to another geographic range served by a second slice selector (e.g., when moving from one cell tower to another or when moving from a WiFi network to a cellular network) while maintaining one connection. Different embodiments use different techniques to ensure that the state is maintained, without requiring action on the part of the endpoint device.
In some embodiments, the second slice selector (the slice selector for the region to which the mobile device moves) forwards all data messages for the connection to the first slice selector (the slice selector for the region in which the mobile device was located when the connection was initiated). That is, the second slice selector receives data indicating that the first slice selector is the location of the slice mapping state for the connection, and thus forwards the data traffic for the connection to the first slice selector. In different embodiments, the first slice selector either (i) pushes this state location information directly to the second slice selector or (ii) pushes the state location information to a network controller (e.g., the aforementioned VSN controller), from which the second slice selector retrieves the state location information.
In other embodiments, the second slice selector receives the state (i.e., the mapping of the connection to the network slice) and is thus able to forward the data messages for the connection to the network slice without involving the first network slice selector. In different embodiments, the second slice selector may receive the state directly from the first slice selector or from a network controller (e.g., the aforementioned VSN controller). In some such embodiments, the first slice selector pushes the state either (i) directly to the second slice selector (e.g., before the device has moved to the geographic region of the second slice selector) or (ii) to the network controller, from which the second slice selector retrieves the state. In other such embodiments, the first slice selector pushes location information for the state to the network controller, and the second slice selector retrieves this location information from the network controller, then uses this location information to retrieve the state from the first slice selector.
In the case that the first slice selector pushes the state information directly to the second slice selector, in some embodiments the first slice selector pushes all of its slice mappings to slice selectors for neighboring geographical regions, in case mobile devices that initiate connections within the geographical region of the first slice selector move to any of the neighboring geographical regions. In other such embodiments, the first slice selector uses location data of the mobile device (if that data is made available) to push the state information to slice selectors for neighboring geographical regions to which the device is likely to move.
In some embodiments, a virtual service network is sliced hierarchically. That is, slices of a virtual service network are themselves virtual service networks with a slice selector and multiple network slices. For example, in telecommunications networks, a mobile network operator (MNO) owns the physical infrastructure of the access and core networks (i.e., the RAN and EPC infrastructure), and traffic from devices that subscribe to that MNO are processed by that infrastructure. In addition, the MNO may lease that infrastructure to one or more mobile virtual network operators (MVNOs) that also have subscriber devices using the same infrastructure. Those MVNOs, in some cases, also lease their virtual infrastructure to additional MVNOs or other entities. In addition, hierarchical layers of slice selection can be implemented over networks for additional reasons besides different telecommunications service providers.
In the above telecommunications provider example, a first slice selector configured by the MNO might assign data messages to network slices based on the source device (e.g., by source network address). Thus, data messages from source devices associated with the MNO are sent to another virtual service network configured by the MNO, while data messages from source devices associated with different MVNOs are sent to virtual service networks configured by the respective MVNOs. In some embodiments, a second slice selector for each virtual service network performs additional slice selection based on various aspects of the data message headers. If an MVNO leases their virtual infrastructure to one or more additional MVNOs, then this second slice selector might also assign data messages to network slices based on a more fine-grained network address analysis (e.g., if the first MVNO is assigned a pool of IP addresses, and divides this pool between its own devices and devices for another MVNO). In other cases, the second level slice selector may perform stateful slice selection based on deep packet inspection, such as that described above.
In some embodiments, within a virtual service network (a collection of multiple network slices with different ordered sets of network services), a network slice selector is always the first entity to process a data message. In other embodiments, after the first network slice selector selects one of the slices, this slice (which is a virtual service network) may include network services applied to the data messages before the second network slice selector selects slices within that virtual service network. Similarly, in some embodiments, network services may be applied to data messages for all network slices within a virtual service network, after the different services are applied for a given slice.
While the above examples of networks that use slice selection and virtual service networks are telecommunications provider networks (both for hierarchical slice selection as well as single-level slice selection), it should be understood that the virtual service networks can be configured for other types of networks as well. For instance, within datacenters or for networks that span across multiple datacenters, virtualization software (e.g., in the host computers that host the VMs or other endpoints of the networks) can be configured to implement slice selection for data messages sent by the network endpoints. One type of network spanning multiple datacenters is using a set of one or more public clouds to connect enterprise datacenters (e.g., a headquarters and one or more remote branch offices); in such a network, network services for a network slice may be implemented in the enterprise datacenters as well as within the public cloud(s).
The preceding Summary is intended to serve as a brief introduction to some embodiments of the invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The Detailed Description that follows and the Drawings that are referred to in the Detailed Description will further describe the embodiments described in the Summary as well as other embodiments. Accordingly, to understand all the embodiments described by this document, a full review of the Summary, Detailed Description and the Drawings is needed. Moreover, the claimed subject matters are not to be limited by the illustrative details in the Summary, Detailed Description and the Drawing, but rather are to be defined by the appended claims, because the claimed subject matters can be embodied in other specific forms without departing from the spirit of the subject matters.
The novel features of the invention are set forth in the appended claims. However, for purpose of explanation, several embodiments of the invention are set forth in the following figures.
In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.
Some embodiments provide methods for establishing a virtual service network across a set of datacenters. The set of datacenters across which the virtual service network is established may include, e.g., one or more public clouds, a software-defined wide area network (SD-WAN) that spans public and private clouds, a telecommunications service provider access network (e.g., spanning a combination of the radio access network, edge clouds, and core clouds), or other types of datacenters. The virtual service network of some embodiments includes multiple network slices each of which provides different network services to data messages assigned to the network slice.
In some embodiments, when a device (e.g., a mobile endpoint device in the telecommunications context) transmits a data message onto such a network, a network slice selector initially processes the data message. The network slice selector assigns the data message to one of the network slices of the virtual service network and handles service chaining operations to ensure that the data message is processed by the correct set of network services for the assigned slice. In different embodiments, this network slice selector may be implemented by a virtual machine (VM), a containerized function, a software forwarding element (e.g., a flow-based forwarding element) operating within a VM, within a container or within virtualization software of a host computer, a set of modules executing outside of a forwarding element (e.g., between a VM and a port of a forwarding element) within virtualization software of a host computer, a hardware forwarding element (e.g., a programmable switch), or other implementations.
In some cases, many network slice selectors are configured to implement a virtual service network. In the telecommunications service provider example, some embodiments configure a network slice selector for each cellular tower, base station, or other aspect of the access network. The telecommunications service provider access network of some embodiments includes edge clouds for each cellular tower, and configures at least one network slice selector at each such edge cloud. In other examples (e.g., for SD-WAN traffic entirely contained within a set of connected datacenters), distributed network slice selectors are configured such that the network slice selection for a data message sent from a VM occurs at the same host computer as the source of the data message (though outside of the source VM) or at a designated device (e.g., a specific nearby switch or router, a dedicated VM or container).
As shown, the virtual service network 100 includes numerous (N) network slices 105-115. Each of these network slices represents a network service path (i.e., an ordered set of network services performed on data messages assigned to the slice). These network services can include firewalls, load balancers, network address translation, metering (e.g., for billing purposes) functions, VPN gateways, radio access network (RAN) functions (e.g., distributed unit and centralized unit functions), evolved packet core (EPC) functions (e.g., home subscriber server, serving gateway, packet data network gateway, mobility management entity), or other types of network functions.
In different embodiments, the network slices of a virtual service network may serve different purposes. Some embodiments slice a network based on the source device (e.g., using the source network address or information identifying the type of device) or subscriber information (e.g., by interfacing with authentication, authorization, and accounting systems or policy systems), while other embodiments slice a network based on the type of traffic (e.g., by performing deep packet inspection). Each network slice can have a prescribed quality of service (QoS) service-level agreement (SLA). For example, a network slice for self-driving automobiles might have extremely low latency requirements, a network slice for streaming video might have high bandwidth requirements, and an IoT slice might have less strict bandwidth or latency requirements for a single device but have a massive connectivity expectation.
These network services may be implemented as virtualized network functions (VNFs), physical network functions (PNFs), and/or cloud native network functions (CNFs) in different embodiments. VNFs are network services that are implemented in virtualized data compute nodes, such as virtual machines. This enables, for instance, the same network service configuration for a particular slice to be implemented in numerous edge clouds (e.g., along with the numerous slice selectors). CNFs are network services implemented in cloud-native data compute nodes, such as specific types of containers. Lastly, PNFs are network services implemented by a physical device (e.g., a specific firewall or load balancer device). In general, PNFs are more usefully located in centralized datacenters rather than edge clouds, so that the same physical device does not need to be replicated for each edge cloud.
In this example, the first network slice 105 includes two VNFs A and B as well as a PNF C. The second network slice 110 is entirely virtual, with three VNFs B, D, and E. The last network slice 115 includes the same three network services as slice 105 (VNFs A and B as well as PNF C) followed by a CNF F. In some embodiments, the same VM can implement a VNF for multiple different network slices. In this example, one VM might implement the same VNF B for all three of the illustrated network slices 105-115. If this VNF is located within the edge clouds, then a single VM may be instantiated in each edge cloud (e.g., for each slice selector). In other embodiments, however, a separate VNF (e.g., a separate VM or other data compute node) is instantiated for each VNF, even if the VNF configuration is the same for multiple slices. Thus, in this example, three different VNFs are instantiated for VNF B for each of the slices 105-115. Thus, if this VNF is located within the edge clouds, then each edge cloud would have three different VMs for VNF B.
Because of the manner in which devices access the network 100, some embodiments have numerous slice selectors 120-130. Devices may access a telecommunications service provider network through base stations (e.g., cell towers), wireless access points, wired hookups (e.g., within a home), or other means. For provider networks, the slice selectors of some embodiments are implemented close to the devices, so that the slice selection can occur before data traffic crosses most of the network. For instance, in the case of 5G wireless networks with multi-access edge computing, some embodiments configure a slice selector for each distributed unit (DU). Other embodiments configure a slice selector for each centralized unit (CU), which receives traffic from multiple DUs. In this case, each slice selector has an associated geographic range (i.e., that of its associated DU or CU).
In such situations, such as that shown in
This example shows that multiple devices can attach to a given slice selector at any particular time. In the example, a smart refrigerator and a laptop are attached to the first slice selector 120, a tablet device is attached to the second slice selector 125, and an autonomous car and a mobile phone are attached to the last slice selector 130. In different embodiments, the network slice selectors may be implemented by a virtual machine (VM), a software forwarding element (e.g., a flow-based forwarding element) operating within a VM or within virtualization software of a host computer, a set of modules executing outside of a forwarding element (e.g., between a VM and a port of a forwarding element) within virtualization software of a host computer, a physical device (e.g., a dedicated hardware forwarding element, a physical host computer), a container application (e.g., a Kubernetes system running a network service mesh), or other implementations.
In some embodiments, traffic from the device 220 initially passes through the radio access network (RAN), which is not shown in this figure. Some embodiments implement network slicing prior to the RAN (i.e., on the device side of the RAN), while in this example network slicing occurs after the RAN. Next, the data traffic arrives at the slice selector 225 (in the edge cloud 205), which analyzes the traffic and assigns the traffic to the network slice 200.
When the slice selector 225 assigns a data message to the network slice 200, the slice selector 225 is responsible in some embodiments for performing service chaining to ensure that the data message traverses the network services of the assigned slice (i.e., the VNFs A-D) in the correct order. In some embodiments, the slice selector 225 transmits the data message to the first network service (i.e., the VM that implements VNF A in the same edge cloud 205) and maintains context information for that data message. When VNF A completes its processing of the data message, the VNF returns the data message to the slice selector 225. If additional network services for the slice are also implemented in the edge cloud 225 (which is not the case for the slice 200), then the slice selector 225 would use the maintained context information to transmit the data message to the next network service, and so on.
In this case, the second network service VNF B is implemented in the core cloud 210. In some embodiments, the network slice selector 225 transmits the data message to a service chaining module at the core cloud (e.g., via wide area network (WAN) gateways that interconnect the clouds 205-215). In some embodiments, when the full network slice is implemented across multiple datacenters, a similar service chaining module operates at each datacenter to handle the service chaining for the slice within its own datacenter (e.g., in both the core cloud 210 and the public cloud 215). These service chaining modules may be implemented in the same manner as the network slice selectors in some embodiments (e.g., as VMs, as forwarding elements in VMs or virtualization software, as containers). Once the last network service is completed, in some embodiments an egress gateway 230 sends the data message to its destination via the Internet.
Next, the slice selector 225 identifies that the next network service for the selected slice 200 is located in the core cloud 210, and thus transmits the data message to the service chaining module 310 that operates in the core cloud 210 (shown by the encircled 4) via WAN gateways (that are not shown in the figure for simplicity). In some embodiments, the service chaining module 310 uses a learning operation (e.g., MAC learning) to store the source of these data messages, so that reverse-direction traffic is sent to the slice selector 225 in the correct edge cloud 205 (i.e., as opposed to the edge cloud 207).
The service chaining module 310 in the core cloud 210 receives the data message as the data message ingresses to the core cloud 210 (after processing by a WAN gateway) and identifies the slice for the data message (e.g., based on context information provided with the data message by the slice selector 310, a stored slice mapping for the connection, or other factors). This service chaining module 310 provides the data message to the network services within the core cloud 210, in this case to VNF B and then to VNF C. As shown, the service chaining module sends the data message to VNF B (shown by the encircled 5), receives the data message back from VNF B (shown by the encircled 6), sends the message to VNF C (shown by the encircled 7), and receives the data message back from VNF C (shown by the encircled 8).
After the data message is processed by VNF C, the data message is transmitted by the service chaining module 310 to another service chaining module 315 (shown by the encircled 9) in the public cloud 215 (e.g., via WAN gateways interconnecting the core cloud 210 and the public cloud 215). The service chaining module 310 operates similarly to the service chaining module 310 in the core cloud 210 in some embodiments, using a learning mechanism to store information for processing return traffic. This service chaining module 310 within the public cloud 215 sends the data message to VNF D (shown by the encircled 10), which performs its network service and returns the data message to the service chaining module 315.
Lastly, the service chaining module 315 determines that the network slice processing is complete for the data message, and sends it to the egress gateway 230, which transmits the data message to its destination via the Internet. While this example shows connectivity between an endpoint device and an Internet domain, in the case of other virtual service networks the destination may instead be located within the public cloud or another datacenter connected via the WAN. The egress gateway 230 of some embodiments stores information mapping the connection to the network slice 200, so that reverse-direction traffic (i.e., data messages from the public Internet domain) are assigned to the same slice (with the network functions performed in the reverse direction). In other embodiments, the egress gateway 230 assigns data messages in a non-stateful manner (e.g., using the destination network address of the data messages). The egress gateway may be implemented together with the service chaining module in some embodiments (or with the original slice selector for virtual service networks that only span a single datacenter).
The slice selectors, network services (e.g., VNFs, CNFs, PNFs), as well as the various forwarding elements that handle transmission of data messages between these entities (e.g., software forwarding elements that tunnel data messages between host machines, WAN gateways) require configuration. In some embodiments, a centralized controller allows a user (e.g., a network administrator) to provide configuration for an entire VSN, and then a controller hierarchy configures the various entities within the one or more datacenters to implement this VSN.
The VSN controller 410 coordinates the centralized storage and distribution of this information to the other controllers in the hierarchy. In some embodiments, a suite of controllers 415 in each of the datacenters receives the VSN configuration data from the VSN controller 410 and configures the entities in the datacenters to implement the VSN. In some embodiments, each datacenter has its own suite of these lower-level controllers. These controller suites may be the same sets of controllers in each datacenter (e.g., a suite of controllers provided by a single company), or different sets of controllers (e.g., a different set of controllers for private edge and core clouds as compared to the public clouds).
The controller suite 415 in the first datacenter 420 includes a software-defined networking (SDN) controller 425, a compute controller 430, and a network controller 435. It should be understood that different embodiments may include additional controllers or may combine the functionality of multiple controllers into a single controller. For instance, some embodiments include an orchestrator that acts as a layer between the VSN controller 410 and the other controllers in the controller suite 415 (e.g., an openstack controller), or combine the SDN controller 425 features with those of the network controller 435. In addition, some embodiments include a storage controller for managing storage relevant to the VSN within the datacenter.
The SDN controller 425 configures the slice selector 440. In this example, a single slice selector 440 operates in the datacenter 420 (e.g., as a VM or within a VM on the host computer 445, in virtualization software of the host computer 445), though it should be understood that in other embodiments the slice selector 440 is implemented in a distributed manner within the datacenter. In some embodiments, the SDN controller 425 configures the slice selector with flow entries or other configuration data to assign data messages to the flows correctly and to perform service chaining operations to ensure that data messages are sent to the correct network services in the correct order within the datacenter 420. In addition, in datacenters that host network services but not the slice selectors (e.g., core clouds, public and/or private clouds for the telecommunications provider example), the SDN controllers of some embodiments configure the service chaining modules as well as the egress gateways (which may perform slice selection for reverse-direction data messages).
The compute controller 430 is responsible for instantiating and configuring the VNFs (e.g., as VMs in this example). In some embodiments, the VMs are instantiated on host computers 450 by the compute controller 430, which configures the VMs to implement the specified network service. In some embodiments, the compute controller 430 uses templates for firewalls, load balancers, or other network services for instantiating the VMs, then provides the specific configuration data for the network service as specified by the network administrator to the VM. In addition, the compute controller 430 of some embodiments is also responsible for configuring any CNFs and/or PNFs implemented in the datacenter 420.
The network controller 435 configures forwarding elements (e.g., the software forwarding element 455 or other types of forwarding elements such as programmable hardware forwarding elements) to implement the network connectivity between the network services and the slice selector 440. This configuration includes forwarding according to, e.g., a logical forwarding plane that connects the various entities of a slice (the slice selector and the network services), as well as performing encapsulation on data messages to tunnel those data messages between the entities within the datacenter. In addition to the software forwarding elements 455 (e.g., virtual switches operating in virtualization software) shown on the host computers 450, in some embodiments a similar software forwarding element executes in the host computer 445 to forward and encapsulate/decapsulate data messages to and from the slice selector 440. In some embodiments (e.g., when the slice selector is implemented in a distributed manner within the software forwarding elements or between the software forwarding elements and the VMs), the network controller 435 also receives the slice selector configuration and configures the appropriate network entities to implement the slice selector.
In addition to these controllers in the controller suite 415, some embodiments also include one or more WAN SDN controllers 460. The WAN SDN controller 460 is responsible for interconnecting the datacenters as needed, and configures WAN gateways 465 in each of the datacenters to do so. These WAN gateways may interconnect the datacenters using MPLS, SD-WAN, or other technologies for inter-datacenter communications. In many cases, not all of the datacenters will need direct communication. For instance, in the telecommunications example, the edge clouds may not need to communicate with each other, as data traffic is not sent between edge clouds but rather between an edge cloud and a core cloud.
In some embodiments, rather than communicating directly with the controllers in the controller suite 415 and the WAN SDN controller 460, the VSN controller 410 provides data to an agent in each datacenter and an agent for the WAN SDN controller 460. These agents are responsible for translating data from the VSN controller 410 (which may be provided in a uniform format for all controllers) into data that the various controller suites can use. In some embodiments, the VSN controller 410 pushes data in a policy format to the local agents, which translate this into data that instructs the various SDN controllers, compute controllers, and/or network controllers, to configure the datacenter components according to those policies. This allows the VSN controller 410 to use a single format to communicate with various different types of controller suites (e.g., different public cloud controllers, enterprise datacenter controller suites). Similarly, for the WAN SDN controller 460, the agent would convert the policies into WAN configuration instructions.
As mentioned above, network slice selectors may assign data messages to slices using different techniques in different embodiments. Slice selection may be based on packet header information, including layer 2 to layer 4 (L2-L4) headers and/or by performing deep packet inspection (e.g., to classify traffic based on data in the layer 5 to layer 7 (L5-L7) headers). For example, slice selection may be based simply on the source device by using the source network layer (e.g., IP) address, or may be based on the type of traffic and/or destination network domain by looking at the upper layer (L5-L7) headers.
In addition, in some embodiments the network slice selector integrates with other control plane components to collect additional information about a connection (e.g., regarding the user session, device type, or other data) and uses this information as part of the slice selection process (e.g., using only this collected information or combining this information with the L2-L4 and/or L5-L7 packet header data). Examples of such control plane components include Authentication, Authorization, and Accounting (AAA) protocols (e.g., Remote Authentication Dial-in User Service (RADIUS)), the Policy Control and Charging Rules Function (PCRF), or other such components that can provide device and/or user data to the slice selector.
In some embodiments, the network slice selector maintains state for mapping connections to network slices so that deep packet inspection does not need to be performed on each data message of a connection. In addition, for some connections, only certain data messages contain the L5-L7 header information required for performing the slice selection.
When performing network slice selection using deep packet inspection, in certain cases the initial data message for a connection may not include the L5-L7 header information that the slice selector needs to correctly identify the slice. For example, a connection between an endpoint device (e.g., a mobile device such as a smart phone or tablet, a laptop or desktop computer, an IoT device, a self-driving automobile, a smart camera belonging to a security system) and a network domain (e.g., a web domain such as www.netflix.com, www.google.com, etc.) often begins with a set of connection initiation messages such as a TCP handshake. After completion of the handshake, the device then sends, e.g., an http get message that includes the network domain. Subsequent data messages sent between the device and the network domain may not include such information.
Different embodiments use different techniques to identify the correct network slice for a connection while ensuring that (i) the connection is initiated correctly between the client (e.g., an endpoint device) and server (e.g., a web domain) and (ii) all of the messages are transmitted on the correct network slice, even if that network slice cannot be selected based on the first message. In some embodiments, the network slice selector acts as a proxy to terminate the connection initiation messages without sending these messages across the virtual service network to the intended destination. In other embodiments, the slice selector passes the connection initiation messages through to a default network slice initially, then replays the messages over the correct network slice for the connection after the network slice is selected.
As shown, the endpoint device 505 (e.g., a smart phone, a self-driving automobile, an IoT device) initially transmits a TCP SYN message directed to the network domain 520 (e.g., using a destination IP address for the network domain). Such a message may not have useful header information (e.g., L5-L7 header information) for the slice selector 510 to use in order to assign the connection being initiated to one of the network slices. As mentioned, in some embodiments the network slices are application-specific and/or device-specific, configured to meet latency, bandwidth, jitter, or other requirements of different applications or device types. Streaming video, self-driving vehicles, IoT devices, and other devices/applications all have different requirements that can be met by different network slices with different SLAs.
Rather than forwarding the connection initiation message (the SYN message) through the network to the Internet (and eventually the network domain 520), the slice selector 510 performs soft termination of the connection initiation. The slice selector 510 stores a record of the SYN message (i.e., the message itself or enough data to recreate the message) and replies to the endpoint device 505 with a SYN-ACK message. This SYN-ACK message is formatted as though the network domain received the SYN message and replied with the SYN-ACK message. This causes the endpoint device 505 to send an ACK message and behave as though the connection has been set up with the network domain 520. The slice selector 505 stores data about all of the connection initiation messages for subsequent use.
Based on the appearance of the connection having been setup properly, the endpoint device 505 then sends a data request message (or other message). This message will typically have L7 information (or other higher-layer information, such as a TLS server name indication) required by the slice selector to correctly assign the connection to a network slice. For instance,
The slice selector 510 performs deep packet inspection on the data message (in this case, the data request) to identify the correct network slice for the connection. In some embodiments, this involves inspecting the L5-L7 headers (e.g., the http data shown in
The slice selector 510 then exchanges the connection initiation messages with the destination network domain, this time acting as the device rather than the destination domain. These connection initiation messages are transmitted via the selected network slice, such that all of the network services on the selected network slice process the connection initiation messages. As shown, the slice selector 510 transmits the SYN message to the network domain 520 via the selected network slice and the egress gateway 515. The egress gateway 515, in some embodiments, also stores connection to slice mapping state so that return traffic from the network domain 520 can also be transmitted via the selected network slice. The network domain 520 returns a SYN-ACK message, which the egress gateway 515 transmits via the selected network slice. This message does not reach the endpoint device 510, as the slice selector 510 recognizes the SYN-ACK message and returns the ACK message to the network domain (via the selected network slice). At this point, the connection is fully initiated between the endpoint device 505 and the network domain 520, as the entire 3-way handshake has been fully exchanged.
The slice selector 510 can then transmit the data request (i.e., the initial data message on which deep packet inspection was performed) to the network domain 520 via the selected network slice. Subsequent data messages for the connection are also exchanged between the endpoint device 505 and the network domain 520, and the slice selector 510 uses the stored state to assign this traffic to the selected network slice without performing deep packet inspection (as does the egress gateway 515 for return traffic). This avoids performing deep packet inspection on every data message for the connection, which may not be possible as the required information might not be in the L5-L7 headers of all of the data messages belonging to the connection.
As shown, the process 900 begins by receiving (at 905) a connection initiation message from an endpoint device (e.g., a phone, a self-driving automobile, a tablet, an IoT device). This message may be a TCP SYN message, the initial message of a TLS handshake, a QUIC protocol message over UDP, or other connection initiation message. The process then performs (at 910) the rest of the connection handshake with the endpoint device while storing copies of the handshake messages (or data needed to recreate the handshake messages). For example, in the TCP 3-way handshake example, the slice selector exchanges SYN-ACK and ACK messages with the endpoint device, acting as the server in this handshake. For optimized single-message connection initiation (e.g., QUIC protocol), this operation is not required, and the slice selector either performs deep packet inspection on the initial message or on a subsequent message without performing a handshake with the endpoint device.
Once the connection has been setup on the endpoint device, the process 900 receives (at 915) an initial data message for the connection from the endpoint device. This may be an http get message such as that shown in
Next, the process 900 transmits (at 930) the connection handshake messages to the destination network domain via the selected network slice in order to establish the connection between the endpoint device and the network domain. In this exchange of messages, the slice selector acts as the endpoint device rather than the network domain, allowing the network domain server to setup the connection. This also allows the network services of the selected network slice to process the connection initiation messages, so that these services will be prepared for the subsequent data messages (e.g., a firewall would often be configured to reject data messages for a TCP connection for which it had not processed the original 3-way handshake messages).
The process 900 also transmits (at 935) the initial data message to the destination network domain via the selected network slice. In addition, the process 900 receives and transmits (at 940) subsequent data messages for the connection via the selected network slice using the stored connection mapping state. The stored state allows the slice selector to assign each data message for the connection from the endpoint device to the selected network slice without the need to perform deep packet inspection (which, as mentioned, may not even be possible for many of the subsequent data messages).
As mentioned, rather than terminating the connection initiation messages at the slice selector, some embodiments pass these messages through to the destination on a default network slice initially, then replay the messages over the correct network slice for the connection after the network slice is selected.
As shown, the endpoint device 1005 (e.g., a smart phone, a self-driving automobile, an IoT device) initially transmits a TCP SYN message directed to the network domain 1020 (e.g., using a destination IP address for the network domain). Such a message does not have useful L5-L7 header information for the slice selector 1010 to use in order to assign the connection being initiated to one of the network slices, as described above. In this case, rather than terminating the handshake and responding to the endpoint device, the slice selector 1010 assigns the TCP SYN message to the default network slice and transmits the message through this network to the network domain 1020 (via the egress gateway 1015). In addition, both the slice selector 1010 and the egress gateway 1015 store records of the SYN message (i.e., the message itself or enough data to recreate the message). The network domain 1020 replies with a SYN-ACK message, which the egress gateway 1015 assigns to the default slice, and the slice selector treats the ACK message from the endpoint device 1005 similarly.
With the connection setup between the endpoint device 1005 and the network domain 1020, the endpoint device 1005 then sends a data request message (or other message). This message will typically have L5-L7 information required by the slice selector to correctly assign the connection to an application-specific or device-specific network slice.
The slice selector 1010 performs deep packet inspection on the data message (in this case, the data request) to identify the correct network slice for the connection. In some embodiments, this involves inspecting the L5-L7 headers (e.g., the http data shown in
Before transmitting the data messages onto the selected slice, however, the slice selector replays the connection initiation messages over the selected network slice. The various network services of the selected slice will not have yet processed the connection initiation messages, and thus would be likely to reject data messages for the connection if these data messages were sent without replaying the handshake. Thus, as shown, the slice selector 1010 uses its stored data to exchange the TCP 3-way handshake messages with the egress gateway 515, which has also stored information for these messages. The SYN, SYN-ACK, and ACK messages are transmitted between the slice selector 1010 and the egress gateway 1015 via the selected network slice. These messages are not transmitted outside of the virtual service network (i.e., to either the endpoint device or the public Internet), because the connection has already been established between the endpoint device 1005 and the network domain 1020. The egress gateway 1015, in some embodiments, also stores the connection to slice mapping state so that return traffic from the network domain 1020 can be transmitted via the selected network slice.
Once these messages have been replayed, the network services of the selected slice will have processed the connection initiation messages and thus are prepared for the subsequent data message traffic between the endpoint device and the destination network domain. As shown, the slice selector 1010 transmits the data request (i.e., the initial data message on which deep packet inspection was performed) to the network domain 1020 via the selected network slice. Subsequent data messages for the connection are also exchanged between the endpoint device 1005 and the network domain 1020, and the slice selector 1010 uses the stored state to assign this traffic to the selected network slice without performing deep packet inspection (as does the egress gateway 1015 for return traffic). This avoids performing deep packet inspection on every data message for the connection, which may not be possible as the required information might not be in the L5-L7 headers of all of the data messages belonging to the connection (e.g., as shown by the data message 800 of
As shown, the process 1100 begins by receiving (at 1105) a connection initiation message from an endpoint device (e.g., a phone, a self-driving automobile, a tablet, an IoT device). This message may be a TCP SYN message, the initial message of a TLS handshake, a QUIC protocol message over UDP, or other connection initiation message. The process then transmits (at 1110) the connection initiation messages (i.e., the initial message and any subsequent messages) to the destination network domain via a default network slice to establish the connection between the endpoint device and the network domain. Similarly, for return messages of the connection initiation handshake sent from the endpoint domain, the slice selector receives these messages via the default slice and transmits them to the endpoint device.
During the connection initiation handshake, the process stores (at 1115) data regarding the handshake messages (i.e., copies of the messages, or data needed to recreate the messages). For example, in the case of a TCP 3-way handshake, the slice selector stores the data in order to send SYN and ACK messages. For single-message connection initiation (e.g., QUIC protocol), this operation is not required, and the slice selector of some embodiments performs deep packet inspection on the initial message so as to avoid the need to use the default slice.
Once the connection has been setup on the endpoint device, the process 1100 receives (at 1120) an initial data message for the connection from the endpoint device. This may be an http get message such as that shown in
Next, the process 1100 replays (at 1135) the connection handshake messages with the egress gateway via the selected network slice. In this exchange of messages, the slice selector acts as the client (i.e., the endpoint device) and the egress gateway acts as the server (i.e., the network domain). In the TCP 3-way handshake example, the slice selector sends a SYN message, receives a SYN-ACK message from the egress gateway, and sends an ACK message. This serves the purpose of allowing the network services of the selected network slice to process the connection initiation messages, so that these services will be prepared for the subsequent data messages (e.g., a firewall would often be configured to reject data messages for a TCP connection for which it had not processed the original 3-way handshake messages).
The process 1100 also transmits (at 1140) the initial data message to the destination network domain via the selected network slice. In addition, the process 900 receives and transmits (at 1145) subsequent data messages for the connection via the selected network slice using the stored connection mapping state. The stored state allows the slice selector to assign each data message for the connection from the endpoint device to the selected network slice without the need to perform deep packet inspection (which, as mentioned, may not even be possible for many of the subsequent data messages).
As shown, the process 1200 begins by receiving (at 1205) a set of connection handshake messages from an endpoint device via a default network slice and transmitting return messages from the destination network domain for the connection to the endpoint device via a default network slice. The egress gateway receives the messages sent by the endpoint device and transmits these messages to their destination in addition to receiving the return traffic and sending this onto the default network slice towards the slice selector (and eventually the endpoint device). In addition, the process 1200 stores (at 1210) data regarding the handshake messages (i.e., copies of the messages, or data needed to recreate the messages). For example, in the case of a TCP 3-way handshake, the egress gateway stores the data in order to send the SYN-ACK message. single-message connection initiation (e.g., QUIC protocol), this operation is not required, and the slice selector of some embodiments performs deep packet inspection on the initial message so as to avoid the need to use the default slice or perform handshake replay.
Once the connection has been setup between the endpoint device and the network domain, the process 1200 receives (at 1215) a replay of the connection initiation message from the slice selector via the selected network slice. The slice selector, at this point, would have received a data message from the endpoint device, used deep packet inspection to assign the connection to a particular one of the possible network slices, and then begun the replay process by sending the initial connection initiation message to the egress gateway. The process 1200 performs (at 1220) the complete replay of the connection handshake with the slice selector via the selected network slice using the stored data for these messages. That is, the egress gateway recognizes the initial connection initiation message as corresponding to a stored set of handshake message data and uses that stored set of data to perform the replay. For instance, in the TCP 3-way handshake example, the egress gateway receives as SYN message, sends the SYN-ACK message to the slice selector, and receives the ACK message. This enables the network services of the selected network slice to process the full set of handshake messages and be prepared for the rest of the data belonging to the connection.
The process 1200 stores (at 1225) state mapping the connection (e.g., using the 5-tuple) to the selected network slice. This information may be stored in a connection mapping table such as that shown in
Other embodiments may use other techniques to build up the state that maps connections to network slices. In some embodiments, the slice selector integrates (e.g., over a control plane channel) with an external component that provides a populated connection-to-slice mapping table to the slice selector, which the slice selector uses to perform stateful slice selection (thereby avoiding the requirement for the slice selector to perform deep packet inspection). The external component, in different embodiments, may provide an entirely pre-populated connection-to-slice mapping table or gradually populate the table with updates as endpoint devices initiate connections. As an example, the slice selectors of some embodiments can integrate with the 5G Network Slice Selection Function (NSSF), allowing the NSSF to define the connection-to-slice mapping. In some such embodiments, the NSSF provides the mapping state to the slice selector, which uses the state to select the correct slice for data packets. That is, an offline external component provides the connection-to-slice mapping state and the slice selector enforces this state in the data plane.
Stateful slice selection, in which an initial data message is inspected to select a network slice for a connection and subsequent data messages are assigned to the network slice based on state stored by the slice selector, works so long as the same slice selector (and egress gateway) process all of the data traffic for a connection. However, in a distributed network (e.g., a telecommunications service provider access network) with numerous slice selectors associated with different geographic ranges, mobile devices (e.g., smart phones, tablets, self-driving automobiles) may move from one geographic range served by a first slice selector to another geographic range served by a second slice selector (e.g., when moving from one base station to another, between groups of base stations that provide traffic to the same centralized unit, when moving from a WiFi network to a cellular network) while maintaining a connection. Different embodiments use different techniques to ensure that the state is maintained, without requiring action on the part of the endpoint device.
In some embodiments, the second slice selector (the slice selector for the region to which the mobile device moves) forwards all data messages for the connection to the first slice selector (the slice selector for the region in which the mobile device was located when the connection was initiated). That is, the second slice selector receives data indicating that the first slice selector is the location of the slice mapping state for the connection, and thus forwards the data traffic for the connection to the first slice selector.
When the mobile device 1300 initiates a connection (which may be only one of multiple connections initiated by the device (e.g., in a single PDU session)), the first slice selector 1320 assigns the connection to the slice 1335, one of several slices of a virtual service network implemented over the access network. As shown, the network slice 1335 includes three VNFs A-C before transmitting data through an egress gateway (not shown) to the Internet. The first slice selector 1320, after performing deep packet inspection to select the network slice, stores state data mapping the connection (in this case, a TCP connection between IP1 and IP2) to the selected network slice. As mentioned above, this state data may be stored as a flow entry (or set of flow entries), as an entry in a connection table, or in another manner. For subsequent traffic from the mobile device 1300 that belongs to this connection, the slice selector 1320 assigns the traffic to the selected network slice 1335 (other connections from the device 1300 may be assigned to other slices). Return traffic for the connection is received from the Internet at the egress gateway, which uses similar stored state to assign this traffic to the same network slice 1335. This return traffic is processed by the VNFs of the network slice 1335 in the reverse order, and then sent from the slice selector 1300 to the mobile device 1300.
In the second stage, however, the mobile device 1300 has moved to the second geographic region 1325, and thus no longer connects to the first slice selector 1320 (i.e., the mobile device 1300 is connected to a different base station that provides traffic to the second slice selector 1330 rather than the first slice selector 1320). The second slice selector 1330 does not have the connection-to-slice mapping state to assign this data traffic from the device 1300 to the correct network slice, and in many cases the data messages will not include the necessary data in the L5-L7 headers for the slice selector 1330 to assign the connection to the network slice. As such, the second slice selector 1330 forwards this traffic to the first slice selector 1320, which uses its stored state information to assign the traffic to the selected network slice 1335. New connections started by the device 1300 while in the second geographic region 1325 will be assigned to the correct slice by the second slice selector 1330.
For the second slice selector 1330 to transmit the data traffic to the first slice selector 1320, in some embodiments the second slice selector 1330 sends the packet via a routable network between the two slice selectors. That is, in such embodiments a routable network exists between the two edge clouds at which the slice selectors are implemented, which can be used to transmit data traffic between the two slice selectors. In other embodiments, the data traffic can be sent through a core cloud (if the two edge clouds connect to the same core cloud) or other WAN connection, or through the VSN controller (though this solution is not optimal if a large amount of traffic is transmitted between slice selectors).
Reverse-direction (return) traffic for the ongoing connection is treated differently in different embodiments, because the slice selector does not need the connection state in some embodiments to process return traffic and send this return traffic to the device 1300. However, in many cases, at least one of the network services is stateful and implemented in the same location (e.g., the same edge cloud) as the slice selector, and thus the return traffic needs to be sent to that edge cloud in order for the same implementation of those network services (i.e., the VM in the first edge cloud with the first slice selector 1320 rather than a VM in the second edge cloud with the second slice selector 1330). The first slice selector 1320 then forwards this return traffic to the second slice selector 1330 in order for the second slice selector 1330 to forward the data to the mobile device 1300 (e.g., through the RAN). In some embodiments, the service chaining module in the core cloud uses its learning function (e.g., a MAC learning feature) to automatically transmit the return traffic to the first slice selector 1320 from which it received the traffic originating at the mobile device 1300. In addition, in some embodiments, the first slice selector 1320 uses a similar learning function when receiving traffic for the connection from the second slice selector 1330, so that it automatically forwards the return traffic onto the network between the two slice selectors (which results in that traffic returning to the second slice selector 1330. For instance, when there is a routable network between the two slice selectors, the first slice selector 1320 stores the MAC address of the router from which it received the traffic from the second slice selector 1330, so that return traffic can be forwarded to this router using the stored MAC address. Other embodiments use a separate ingress gateway function on the slice (i.e., before the first network service), that is responsible for sending return traffic to the correct slice selector
In order for the second slice selector 1330 to forward the data traffic for a particular connection to the first slice selector 1320, the second slice selector needs to receive data indicating that the first slice selector 1320 has the state information for the connection. In different embodiments, the first slice selector either (i) pushes the state location information to a network controller (e.g., the aforementioned VSN controller), from which the second slice selector retrieves the state location information or (ii) pushes the state location information to the second slice selector.
In addition, the first slice selector 1420 pushes information to the network controller 1425 specifying that the first slice selector is the location of the slice mapping state for this connection. This network controller, in some embodiments, is a VSN controller that provides VSN configuration data to the controllers at multiple datacenters in which the VSN is implemented. Specifically, in some embodiments, the first slice selector 1420 provides the slice mapping state location data to one of the controllers local to its datacenter (e.g., the SDN controller that configures the slice selector), which in turn passes the state location data to the VSN controller so that it can be accessed by slice selectors at other datacenters.
In the second stage 1410, the mobile device 1435 has moved to a second geographic range 1450 associated with the second slice selector 1430. Upon receiving a data message from the device 1435 for an ongoing connection that the second slice selector 1430 does not recognize, this slice selector 1430 sends a request to the controller 1425 (e.g., by making such a request to one of the controllers local to its datacenter, which in turn sends the request to the VSN controller). The controller 1425 stores this state location information 1400, and thus returns the information 1400 to the second slice selector 1430 (e.g., via the controller local to the datacenter of the second slice selector 1430).
Based on this state location information, in the third stage 1415, the second slice selector 1430 is able to forward the data message for this connection (as well as subsequent data messages for the connection) to the first slice selector 1420, which can forward the data onto the selected network slice 1445. In some embodiments, datacenter-to-datacenter connections (i.e., routable networks) exist between edge clouds, while in other embodiments this traffic is passed from one slice selector to another through core clouds or other networks.
In other embodiments, the slice selector through which a connection was initiated pushes the state location information to other slice selectors (e.g., geographically neighboring slice selectors) such that those other slice selectors have the state location information available if the mobile device that initiated the connection moves into a new geographic region.
In addition, the first slice selector 1515 pushes information to the second slice selector 1520 specifying that the first slice selector 1515 is the location of the slice mapping state for this connection. Different embodiments transmit the state location information in different ways. In some embodiments, this information is transmitted through the data network (e.g., via a routable datacenter-to-datacenter network, through an edge cloud) as for the data traffic sent between the two slice selectors (but as control plane data between control plane interfaces of the slice selectors), while in other embodiments the state location information is pushed to a controller (i.e., as shown in
In the second stage 1510, the mobile device 1525 has moved to a second geographic range 1540 associated with the second slice selector 1520. Upon receiving data traffic from the device 1525 for an ongoing connection, the second slice selector 1520 can map that data traffic to the state location data that it already stores and forward the data messages to the first slice selector 1515, which forwards the data onto the selected network slice 1535. In some embodiments, datacenter-to-datacenter connections (i.e., routable networks) exist between edge clouds, while in other embodiments this traffic is passed from one slice selector to another through core clouds or other networks.
Rather than data for a connection always being forwarded to the original slice selector through which a mobile device initiated the connection, other embodiments provide the slice mapping state for the connection to other slice selectors to which the mobile device moves. The second slice selector (i.e., the slice selector into the range of which the mobile device moves) receives the slice mapping state for the connection and is thus able to forward the data messages for the connection to the network slice without involving the first network slice selector (through which the connection was initiated).
When the mobile device 1600 initiates a connection (which may be only one of multiple connections initiated by the device (e.g., in a single PDU session)), the first slice selector 1620 assigns the connection to the slice 1635, one of several slices of a VSN implemented over the access network. As shown, the network slice 1635 includes three VNFs A-C before transmitting data through an egress gateway (not shown) to the Internet. The first slice selector 1620, after performing deep packet inspection to select the network slice, stores state data mapping the connection (in this case, a TCP connection between IP1 and IP2) to the selected network slice. As mentioned above, this data may be stored as a flow entry (or set of flow entries), as an entry in a connection table, or in another manner. For subsequent traffic from the mobile device 1600 that belongs to this connection, the slice selector 1620 assigns the traffic to the selected network slice 1635 (other connections from the device 1600 may be assigned to other slices). Return traffic for the connection is received from the Internet at the egress gateway, which uses similar stored state to assign this traffic to the same network slice 1635. This return traffic is processed by the VNFs of the network slice 1635 in the reverse order, and then sent from the slice selector 1600 to the mobile device 1600
In the second stage, however, the mobile device 1600 has moved to the second geographic region 1625, and thus no longer connects to the first slice selector 1620 (i.e., the mobile device 1600 is connected to a different base station that provides traffic to the second slice selector 1630 rather than the first slice selector 1620). In this case, rather than forwarding data from the mobile device 1600 to the first slice selector 1620, the first slice selector 1620 has provided the slice mapping state for the connection to the second slice selector 1630. As such, the second slice selector 1630 can forward this data directly to the network slice 1635 selected for the connection, without the need to perform any deep packet inspection.
In some embodiments, one or more of the network services for the slice is stateful and is implemented in the edge clouds along with the slice selectors. If the services are stateless, then when the traffic moves to the second slice selector 1630, the instances of those services in the new edge cloud can process the traffic without any issues. However, when a network service in the edge cloud is stateful, then some embodiments transfer the state from the instance of the service in the edge cloud with the first slice selector 1620 to the instance of the network service in the edge cloud with the second slice selector 1630. Another option utilized by some embodiments is to migrate the network service instance from the first edge cloud to the second edge cloud. However, if the network service instance is processing traffic for numerous connections, then this option has downsides of interrupting the other connections. In some other embodiments, if any of the network services for the selected slice are stateful and implemented in the edge clouds with the slice selectors, then slice mapping state for the connection is not provided to the second slice selector, which instead forwards data traffic to the first slice selector as shown above in
In different embodiments, the second slice selector 1630 may receive the state directly from the first slice selector or from a network controller (e.g., the aforementioned VSN controller). In some such embodiments, the first slice selector pushes the state either (i) directly to the second slice selector (e.g., before the device has moved to the geographic region of the second slice selector) or (ii) to the network controller, from which the second slice selector retrieves the state. In other such embodiments, the first slice selector pushes location information for the state to the network controller, and the second slice selector retrieves this location information from the network controller, then uses this location information to retrieve the state from the first slice selector.
In addition, the first slice selector 1720 pushes the connection to slice mapping state 1700 to the network controller 1725, so that other slice selectors can retrieve this state if needed. This network controller, in some embodiments, is a VSN controller that provides VSN configuration data to the controllers at multiple datacenters in which the VSN is implemented. Specifically, in some embodiments, the first slice selector 1720 provides the slice mapping state 1700 to one of the controllers local to its datacenter (e.g., the SDN controller that configures the slice selector), which in turn passes the state to the VSN controller so that it can be accessed by slice selectors at other datacenters.
In the second stage 1710, the mobile device 1735 has moved to a second geographic range 1750 associated with the second slice selector 1730. Upon receiving a data message from the device 1735 for an ongoing connection that the second slice selector 1730 does not recognize, this slice selector 1730 sends a request to the controller 1725 (e.g., by making such a request to one of the controllers local to its datacenter, which in turn sends the request to the VSN controller). The controller 1725 stores the slice mapping state 1700 for the connection specified in this request, and thus returns the state 1700 to the second slice selector 1730 (e.g., via the controller local to the datacenter of the second slice selector 1730).
Based on this slice mapping state, in the third stage 1715, the second slice selector 1730 is able to process the data message received from the mobile device 1735 (as well as subsequent data messages for this connection) and forward this data message onto the selected network slice (i.e., the slice specified in the slice mapping state for the connection).
In other embodiments, the slice selector through which a connection was initiated only provides state location information to the controller, allowing other slice selectors to retrieve the state location information and use that to retrieve the slice mapping state directly from the first slice selector.
In addition, the first slice selector 1825 pushes state location information 1840 to the network controller 1830 specifying that the first slice selector is the location of the slice mapping state for this connection. This network controller, in some embodiments, is a VSN controller that provides VSN configuration data to the controllers at multiple datacenters in which the VSN is implemented. Specifically, in some embodiments, the first slice selector 1825 provides the slice mapping state location data 1840 to one of the controllers local to its datacenter (e.g., the SDN controller that configures the slice selector), which in turn passes the state location data to the VSN controller so that it can be accessed by slice selectors at other datacenters.
In the second stage 1810, the mobile device 1845 has moved to a second geographic range 1860 associated with the second slice selector 1835. Upon receiving a data message from the device 1845 for an ongoing connection that the second slice selector 1835 does not recognize, this slice selector 1835 sends a request to the controller 1830 (e.g., by making such a request to one of the controllers local to its datacenter, which in turn sends the request to the VSN controller). The controller 1830 stores this state location information 1840, and thus returns the information 1840 to the second slice selector 1835 (e.g., via the controller local to the datacenter of the second slice selector 1835).
Based on this state location information, in the third stage 1815, the second slice selector 1830 sends a request to the first slice selector 1825 for the slice mapping state for the connection. This request specifies the connection (e.g., by 5-tuple) in some embodiments, and is formatted in a specific manner recognized by the first slice selector 1825 as a request for slice mapping state. In response, the first slice selector 1825 sends the slice mapping state 1800 for the connection to the second slice selector 1835. In some embodiments, routable datacenter-to-datacenter connections exist between edge clouds, while in other embodiments the request and subsequent response is passed from one slice selector to another through core clouds or other networks.
In the fourth stage 1820, the second slice selector 1835 is able to process the data message received from the mobile device 1845 (as well as subsequent data messages for this connection) and forward this data message onto the selected network slice 1855 (i.e., the slice specified in the slice mapping state for the connection).
In still other embodiments, the slice selector through which a connection was initiated pushes slice mapping state to other slice selectors (e.g., geographically neighboring slice selectors) such that those other slice selectors have the slice mapping state for the connection available if the mobile device that initiated the connection moves into a new geographic region.
In addition, the first slice selector 1915 pushes the slice mapping state 1900 for the connection to the second slice selector 1920, indicating that the connection is assigned to the network slice 1935. Different embodiments transmit the slice mapping state in different ways. In some embodiments, the state is transmitted through the data network (e.g., via a routable network between datacenters, through an edge cloud), while in other embodiments the state is pushed to a controller (i.e., as shown in
In the second stage 1910, the mobile device 1925 has moved to a second geographic range 1940 associated with the second slice selector 1920. Upon receiving data traffic from the device 1925 for the ongoing connection, the second slice selector 1920 is able to process the data message received from the mobile device 1925 (as well as subsequent data messages for this connection) and forward this data message onto the selected network slice 1935 (i.e., the slice specified in the slice mapping state for the connection).
In a number of the above examples, the first slice selector pushes the slice mapping state (or the state location information) to the second controller. In some embodiments, the first slice selector pushes all of its slice mapping state (or the state location information for those connections) to slice selectors for neighboring geographical regions, in case mobile devices that initiate connections within the geographical region of the first slice selector move to any of the neighboring geographical regions. In other such embodiments, the first slice selector uses location data of the mobile device (if that data is available) to push the state information to slice selectors for neighboring geographical regions to which the device is likely to move.
Some embodiments only push the slice mapping state (or state location information) to directly neighboring regions (i.e., regions that partially overlap or abut the region in which a connection is initiated), while other embodiments push the state to additional regions (e.g., all regions, regions that neighbor all of the neighboring regions of the region in which a connection is initiated). In some embodiments, the slice selector is configured with a list of all of the regions to which it pushes slice mapping state (or state location information), and pushes this state directly to the slice selectors for those other regions (e.g., by transmitting the information through a connection between datacenters). Once a mobile device moves to a different region and the slice selector for that region processes the data traffic for a connection from the mobile device using the slice mapping state, in some embodiments the slice selector for the new region also pushes the state to the slice selectors for its neighboring regions, in case the mobile device continues to move.
The slice selectors of other embodiments push the state to a central controller (e.g., the VSN controller) that automatically distributes the state to the slice selectors for neighboring regions, in which case the slice selector does not need to be configured with a list of slice selectors to which to push its state, as this is handled at the controller.
As mentioned, some embodiments use more precise location data for a mobile device to intelligently push slice mapping state (or state location information) to specific neighboring regions.
All of the above examples illustrate a single virtual service network implemented over a physical infrastructure (e.g., a telecommunications service provider access network). In some embodiments, however, a virtual service network is sliced hierarchically. That is, slices of a virtual service network are themselves virtual service networks with a slice selector and multiple network slices.
For example, in a telecommunications service provider network of some embodiments, a mobile network operator (MNO) owns the physical infrastructure 2200 of the access and core networks (i.e., the RAN and EPC infrastructure), and configures the slice selector 2205 to process traffic from devices that subscribe to the MNO. In addition, the MNO may lease the physical infrastructure to one or more mobile virtual network operators (MVNOs) that also have subscriber devices using the same infrastructure. Those MVNOs, in some cases, also lease their virtual infrastructure to additional MVNOs or other entities. In the example of
For example, the slice selector 2205 configured by the MNO assigns data messages to either VSN 2210 or VSN 2215 based on the source device (e.g., by source network address). Thus, data messages from source devices associated with the MNO are sent to the VSN 2210 while data messages from source devices associated with the MVNO is sent to the VSN 2215, which is configured by the MVNO. If additional MVNOs lease the infrastructure as well, then the slice selector 2205 would have additional VSNs from which to select (with each MVNO able to configure the slice selector and sets of network services for the slices of its own VSN).
Each of the VSNs 2210 and 2215 has its own respective slice selector 2225 and 2230 as well. In the example, each of these slice selectors 2225 and 2230 chooses between two possible network slices, but it should be understood that just as the provider infrastructure may have numerous VSNs from which the top-level slice selector 2205 chooses, each of the VSNs will often include numerous slices. In some embodiments, these slice selectors 2210 and 2215 for the tenant VSNs perform additional slice selection based on various aspects of the data message headers. For example, while the top-level slice selector 2205 selects VSNs based on the source device network address in some embodiments, the lower-level slice selectors 2210 and 2215 might assign data messages to slices in the stateful manner described above (e.g., using deep packet inspection to assign connections to slices in an application-aware manner).
Just as a single level of slice selector may be implemented in different ways (e.g., as a flow-based forwarding element operating within a VM or virtualization software, as a programmable physical forwarding element, as a separate set of modules executing between a VM and a port of a software forwarding element), different embodiments implement the multiple levels of slice selectors 2325 and 2330 in different ways. When the form factor for the slice selector is a VM or a forwarding element executing within a VM, some embodiments use separate VMs for each instance of the provider slice selector 2325 and each instance of the tenant slice selector 2330 (and any other tenant slice selectors). This allows, e.g., the provider admin to configure the VM and forwarding elements for the provider slice selector 2325 separately from the VMs and forwarding elements for each of the tenant slice selectors.
In this case, when the access network receives a data message, the message (after any preliminary processing, e.g., through the RAN) is first sent to the provider slice selector 2325. After the provider slice selector forwarding element selects one of the tenant VSNs (or the provider's own VSN, which is effectively another tenant VSN), the provider slice selector 2325 sends the data message to the slice selector 2330 for the selected tenant VSN in the same edge cloud (i.e., in this example, the edge cloud 2305). In some embodiments, the provider slice selector 2325 uses service chaining techniques to send the data message to the tenant slice selector 2330, while in other embodiments the provider slice selector 2325 is finished processing the data message at this point, and is simply configured to send the data message to the appropriate tenant slice selector (e.g., slice selector 2330).
This tenant slice selector 2330 receives the data message, performs slice selection and service chaining for its selected slice (i.e., in the same manner shown in
In the example of
In some embodiments, rather than implementing the different levels of slice selectors separately, the lower-level (tenant) slice selectors are implemented in the same VM and/or forwarding element as the top-level (provider) slice selector. For instance, in some such embodiments, a first set of flow entries implement the provider slice selector and separate sets of flow entries implement each of the tenant slice selectors. Which of these separate sets of flow entries are evaluated (i.e., which of the tenant slice selectors evaluates a data message) depends on which of the first set of flow entries is matched by the first slice selector (i.e., to which tenant VSN the data message is assigned).
In a service insertion model for the slice selectors, in which the slice selection is performed as a service associated with a port of a software forwarding element, then some embodiments perform both top-level (provider) slice selection and lower-level (tenant) slice selection as separate services one after another. That is, a data message is intercepted initially by the provider slice selector, and then based on which tenant VSN is chosen, the data message is intercepted by one of the tenant slice selectors.
The separate controller instances 2405-2415 are accessed by logins with different administrative privileges in some embodiments (i.e., using role-based access control). That is, the VSN controller interface (CLI, GUI, APIs) of some embodiments provides different logins to a provider administrator account and separate accounts for each tenant administrator. These different accounts are able to provide different sets of configuration data to the VSN controller to configure the respective VSNs. For instance, the provider VSN controller 2405 allows the provider to configure the top-level slice selector, the chaining between any services at the top level and the selected tenant VSN, and the physical network and gateways between datacenters in some embodiments. In addition, this provider VSN controller 2405 includes functions in some embodiments to manage the life cycle of the provider VSN (e.g., instantiate the slice selector, gateways, and other components), and/or to monitor and optimize the provider VSN entities and the various tenant VSNs.
Each of the tenant VSN controllers 2410 and 2415, in some embodiments, allows the respective tenant to separately configure their respective VSNs. By using different logins for the administrators, the controller 2400 isolates a tenant administrator to only configure their own VSN, and not any of the other tenant VSNs or the provider VSN. In some embodiments, each tenant administrator is able to configure their respective slice selector, the network services of their various slices, the chaining between the slice selector and the network services, and other aspects of the tenant's configuration. In addition, the tenant VSN controllers 2410 and 2415 each include functions in some embodiments to manage the life cycle of the tenant VSN and the various network services and/or to monitor and optimize the various network services (and slices as a whole).
As described above with respect to
In the above example of
The first tenant VSN 2510 includes a slice selector 2520 that selects between two slices 2525 and 2530, with different sets of network services for each of these two slices. In addition, as shown, irrespective of to which slice a data message is assigned, the data message is sent to VNF E for processing after completion of either slice 2525 or slice 2530. For example, irrespective of the type of data being sent over a connection, the tenant might want a single metering service to process all of its connections for billing purposes. In this case, irrespective of to which slice a data message is assigned, the slice selector (or service chaining module in the final datacenter in which the slice is implemented) sends the data message to the VM implementing VNF E. Other embodiments, however, do not allow such network services outside of the slices, as it becomes more difficult to control QoS parameters for all of the connections if these connections are all sent to the same VNF. In this case, separate instances of VNF E would be included within each of the slices 2525 and 2530.
When the provider slice selector 2505 assigns a data message to the second tenant VSN 2515, in this example the provider slice selector first sends the data message to an instance of the network service VNF A, then to the slice selector 2535 for the second tenant VSN 2515. This allows the provider to configure network services to apply to all data traffic sent to a particular tenant VSN (e.g., for billing purposes, to provide RAN or EPC functions, or for other purposes). Other embodiments, however, require that the lower-level slice selector is the first entity to which a data message is sent after the upper-level slice selector, concentrating all of the network services into the lowest layer of network slices.
The above examples illustrate two levels of slice selection—e.g., an MNO that owns infrastructure and one or more MVNOs that lease that infrastructure from the MNO. In many cases, an MVNO will also sub-lease their virtual infrastructure to one or more additional MVNOs, and a third level of slice selection is used. In this case, the MNO slice selector might assign data messages to tenant VSNs based on source network address analysis, with the first level of MVNO slice selector then assigning its data messages to sub-tenant VSNs based on a more fine-grained network address analysis (e.g., all source devices with IP addresses in a/24 subnet are assigned to a first level of tenant VSN, which is then sub-divided between source multiple/28 subnets). The third level of slice selection may then perform stateful slice selection based on deep packet inspection, such as that described above.
The second tenant VSN 2615, on the other hand, has a slice selector 2625 that assigns data messages to either (i) a third-level VSN 2630 or (ii) a network slice 2635 with a set of VNFs that are part of the configuration for the VSN 2615. For instance, if the VSN 2615 is managed by a first MVNO, then the VSN 2630 might be managed by a second MVNO that leases a portion of the virtual infrastructure while the network slice 2635 is for data traffic to and from devices that subscribe to the first MVNO. The VSN 2630 has a third-level slice selector 2640 configured by the second MVNO that chooses between two slices 2645 and 2650 (e.g., in an application-aware manner, based on L2-L4 headers, or using other factors).
In addition, if the first MVNO was to perform application-aware slice selection, then the slice selector 2625 might choose between the VSN 2630 and multiple different network slices for the different applications. In this case, the slice selector 2625 might be configured to send data traffic that matches a particular set of source network addresses to the VSN 2630 and then slice data traffic for its other source network addresses based on the application layer data. In a flow-based forwarding element implementation of the slice selector 2625, the flow entry for the VSN 2630 (matching based on source address) would have a higher priority than the flow entries for the other network slices, so that data traffic for the second MVNO that otherwise matches the application-aware flow entries would not be sent to one of the network slices.
Other embodiments, however, do not allow a slice selector to choose between a VSN and a network slice that is not further subdivided. In this case, the slice selector 2625 would be configured to select between the VSN 2630 and another VSN, also configured by the first MVNO, with a slice selector that chooses between the slice 2635 and any other slices configured for the first MVNO.
While the above examples of VSNs that use slice selection are telecommunications provider networks (both for hierarchical slice selection as well as single-level slice selection), it should be understood that the virtual service networks can be configured for other types of networks as well. For instance, within datacenters or for networks that span across multiple datacenters, virtualization software (e.g., in the host computers that host the VMs or other endpoints of the networks) or VMs can be configured to implement slice selection for data messages sent by the network endpoints.
In some embodiments, the first guest VM 2705 transmits a data message that is processed by a slice selector 2715 operating in the same host computer 2720 as the guest VM 2705. This slice selector 2715 in some embodiments is implemented by a software forwarding element executing in the host computer (e.g., a software virtual switch executing in the virtualization software). In other embodiments, the slice selector is implemented as part of a service insertion layer between the guest VM 2705 and a software forwarding element (not shown). This service insertion layer, in some embodiments, uses characteristics of the data message (e.g., source and/or destination addresses, L5-L7 header information determined from deep packet inspection, or other factors) to assign the data message to a particular slice.
In the example, the data traffic is assigned to a first slice with three services implemented as VNFs 2725-2735 in three VMs on different host computers. At least one other network slice (i.e., a different ordered set of network services) is implemented separately within the datacenter (as shown by the dashed lines), such that different traffic from the first guest VM 2705 (including, in some cases, different traffic to the same second guest VM 2710) is processed by this different set of network services. In some embodiments with slice selectors and/or service chaining modules implemented on host computers with all of the guest VMs, data traffic from various different source VMs are sent along the same service path (i.e., to the same VNFs in the same order) from different origin host computers.
Some embodiments use a similar service chaining method as that described above for telecommunications service provider networks, with data messages returning to the slice selector 2715 after each service in the network slice completes its processing. In this case, the traffic does not follow a linear flow through the chain of services as shown in the figure, but rather repeatedly returns to the host computer 2720.
Other embodiments use distributed service chaining such that a forwarding element on the same host computer as the first VNF 2725 automatically forwards the data traffic to the second VNF 2730 rather than returning the data traffic to the slice selector 2715. Some such embodiments accomplish this by automatically forwarding traffic received from the interface to which one VNF connects to the next VNF in the service chain, while in other such embodiments the slice selector appends a tag or set of tags to the data messages that is used by forwarding elements along the service chain to forward the messages to the next VNF in the service chain.
When the data traffic is received at the host 2745 of the destination guest VM 2710, a reverse slice selector 2740 stores connection information for the traffic before providing the data to the guest VM 2710. The reverse slice selector 2740 is similar to the egress gateway in
In some embodiments, the slice selection and reverse slice selection functions are performed by the same component in the hosts 2720 and 2745 (e.g., a software forwarding element, the service insertion layer, or other component). The slice selector and reverse slice selector are distributed in such embodiments, with the component configured on each host computer (e.g., each host computer with a VM that attaches to the VSN) to perform both slice selection and reverse slice selection functions.
When a VM initiates a connection (i.e., acting as a client) and sends traffic for which there is no stored connection mapping, this component performs slice selection (using L2-L4 header fields, deep packet inspection, or other factors) and sends the traffic to the network services of the slice in the configured order (in this case, VNF 2725, then VNF 2730, then VNF 2735). When a host receives incoming traffic for a new connection (i.e., with the VM on that host acting as a server), the component acts as a reverse slice selector and stores the slice mapping data for the connection. When the VM returns traffic for a connection initiated elsewhere, this component acts as a reverse slice selector, using stored slice mapping data for the connection and sending the traffic to the network services of the selected slice in the reverse order (in this case, VNF 2735, then VNF 2730, then VNF 2735).
In the example, data traffic is sent from a VM 2820 executing on a host computer in the main office 2805 to a VM 2825 executing on a host computer in the branch office 2810. As in the previous example of
The data traffic from the VM 2820 is assigned to a first network slice with four network services implemented as VNFs. As shown, the first VNF 2835 is implemented in the main office datacenter 2805. For instance, some embodiments might use a firewall to filter data traffic within the initial datacenter before sending the data traffic through the WAN to other datacenters. After processing by the initial VNF 2835, the data traffic sent via WAN gateways (e.g., MPLS gateways, SD-WAN gateways) to the public cloud, where additional network services 2840-2850 for the selected network slice are implemented. After processing by the three network services 2840-2850, the data traffic is transmitted via another set of WAN gateways to the branch office, where the data is delivered to a host of the destination VM 2825. A reverse slice selector 2860 in this host (e.g., implemented by a software forwarding element, service insertion layer, or other component) stores the connection to slice mapping information for use in assigning return traffic to the same network slice.
In some embodiments, as described above, the slice selector 2830 handles service chaining (at least until the data traffic transmitted to another datacenter, at which point a service chaining module in that datacenter handles the service chaining). As such, the slice selector 2830 not only determines to which network slice a data message is assigned, but also determines the location (e.g., a VM, container, or physical device in the current datacenter, another datacenter) of the next network service in the selected network slice. For example, in the example shown in
It should also be noted that in the example of
As shown in the figure, a guest VM 2905 in a host computer in a first public cloud 2910 transmits a data message to a public Internet destination. This destination might be, e.g., a user endpoint device (e.g., with the VM 2905 acting as a server in response to a data request from the client device) or another destination in a public or private cloud. In this case, the data message is sent to a slice selector 2915 implemented on a different host computer from the VM 2905, which assigns the data message to a first network slice and transmits the data message to the first two network services 2920 and 2925 within the first datacenter. The slice selector 2915, in this example, is centralized (within the datacenter 2910) rather than distributed on all of the host computers that host guest VMs attached to the VSN. In some embodiments, the guest VM 2905 is configured to use the slice selector 2915 as its IP gateway, so that all traffic is initially sent to the slice selector 2915 for processing. Different embodiments may use a single slice selector for a datacenter (e.g., as a VM or a forwarding element executing in a VM) as shown in this figure, or a distributed slice selector as shown in
As described above, in different embodiments, the data message is either returned to the slice selector 2915 or distributed service chaining is used and the data message is passed directly from the VNF 2920 to the VNF 2925 (i.e., through forwarding elements on these host computers, but without returning to the host on which the slice selector 2915 is implemented).
Next, because the third and fourth network services 2930 and 2935 for the selected network slice are implemented on a host computer in a second public cloud 2940, the data message is transmitted through WAN gateways to this datacenter 2940. There, a service chaining module (or distributed service chaining layer) sends the data message to the network services 2930 and 2935, which are implemented on the same host computer in this case. Finally, after the network slice is completed, the data message is sent to the public Internet via an egress gateway 2945. This egress gateway operates similarly to the egress gateway 230 in some embodiments, storing connection to slice mapping state for use in assigning reverse direction traffic received from the Internet.
In some embodiments, the VSN is configured in one or more datacenters (e.g., as shown in
The bus 3005 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 3000. For instance, the bus 3005 communicatively connects the processing unit(s) 3010 with the read-only memory 3030, the system memory 3025, and the permanent storage device 3035.
From these various memory units, the processing unit(s) 3010 retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments.
The read-only-memory (ROM) 3030 stores static data and instructions that are needed by the processing unit(s) 3010 and other modules of the electronic system. The permanent storage device 3035, on the other hand, is a read-and-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 3000 is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 3035.
Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device. Like the permanent storage device 3035, the system memory 3025 is a read-and-write memory device. However, unlike storage device 3035, the system memory is a volatile read-and-write memory, such a random-access memory. The system memory stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention's processes are stored in the system memory 3025, the permanent storage device 3035, and/or the read-only memory 3030. From these various memory units, the processing unit(s) 3010 retrieve instructions to execute and data to process in order to execute the processes of some embodiments.
The bus 3005 also connects to the input and output devices 3040 and 3045. The input devices enable the user to communicate information and select commands to the electronic system. The input devices 3040 include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices 3045 display images generated by the electronic system. The output devices include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as a touchscreen that function as both input and output devices.
Finally, as shown in
Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.
While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.
As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms display or displaying means displaying on an electronic device. As used in this specification, the terms “computer readable medium,” “computer readable media,” and “machine readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.
This specification refers throughout to computational and network environments that include virtual machines (VMs). However, virtual machines are merely one example of data compute nodes (DCNs) or data compute end nodes, also referred to as addressable nodes. DCNs may include non-virtualized physical hosts, virtual machines, containers that run on top of a host operating system without the need for a hypervisor or separate operating system, and hypervisor kernel network interface modules.
VMs, in some embodiments, operate with their own guest operating systems on a host using resources of the host virtualized by virtualization software (e.g., a hypervisor, virtual machine monitor, etc.). The tenant (i.e., the owner of the VM) can choose which applications to operate on top of the guest operating system. Some containers, on the other hand, are constructs that run on top of a host operating system without the need for a hypervisor or separate guest operating system. In some embodiments, the host operating system uses name spaces to isolate the containers from each other and therefore provides operating-system level segregation of the different groups of applications that operate within different containers. This segregation is akin to the VM segregation that is offered in hypervisor-virtualized environments that virtualize system hardware, and thus can be viewed as a form of virtualization that isolates different groups of applications that operate in different containers. Such containers are more lightweight than VMs.
Hypervisor kernel network interface modules, in some embodiments, is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive/transmit threads. One example of a hypervisor kernel network interface module is the vmknic module that is part of the ESXi™ hypervisor of VMware, Inc.
It should be understood that while the specification refers to VMs, the examples given may be any type of DCNs, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, the example networks might include combinations of different types of DCNs in some embodiments.
While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. In addition, a number of the figures (including
Claims
1-23. (canceled)
24. For a network slice selector, a method comprising:
- from an electronic device, receiving a connection initiation message for a connection between the electronic device and a network domain;
- exchanging a set of connection initiation messages with the electronic device to set up the connection;
- receiving a data message belonging to the connection from the electronic device;
- selecting one of a plurality of network slices to which the electronic device has access as the network slice for the connection based on analysis of the received data message; and
- forwarding the data message and subsequent data messages belonging to the connection onto the selected network slice to be delivered to the network domain.
25. The method of claim 24, wherein:
- the connection initiation message comprises a transmission control protocol (TCP) SYN message; and
- exchanging the set of connection initiation messages comprises sending a SYN-ACK message from the network slice selector to the electronic device and receiving an ACK message from the electronic device.
26. The method of claim 25 further comprising, after selecting the network slice and prior to forwarding the data message, forwarding the SYN message and the ACK message onto the selected network slice to be delivered to the network domain.
27. The method of claim 24, wherein the selected network slice comprises a set of network services to be applied to data messages forwarded onto the selected network slice.
28. The method of claim 27, wherein the selected network slice is a first network slice and the set of network services is a first set of network services, wherein the plurality of network slices comprises at least a second network slice that comprises a second set of network services to be applied to data messages forwarded onto the second network slice.
29. The method of claim 27, wherein at least a subset of the network services are implemented as virtualized network functions (VNFs) that execute in at least two different datacenters.
30. The method of claim 24, wherein forwarding the data message onto the selected network slice comprises:
- forwarding the data message to a first network service of the selected network slice;
- receiving the data message from the first network service after the first network service processes the data message; and
- forwarding the data message to a second network service of the selected network slice.
31. The method of claim 24, wherein the network slice selector executes in a first datacenter, wherein forwarding the data message onto the selected network slice comprises:
- forwarding the data message to a first network service of the selected network slice that executes in the first datacenter;
- receiving the data message from the first network service after the first network service processes the data message; and
- forwarding the data message to a service chaining module that executes in a second datacenter, wherein the service chaining module forwards the data message to a second network service that executes in the second datacenter.
32. The method of claim 24, wherein the network slice selector is a flow-based forwarding element executing in a virtual machine.
33. The method of claim 24, wherein the network slice selector executes in a container.
34. A non-transitory machine readable medium storing a network slice selector for execution by at least one processing unit, the network slice selector comprising sets of instructions for:
- receiving, from an electronic device, a connection initiation message for a connection between the electronic device and a network domain;
- exchanging a set of connection initiation messages with the electronic device to set up the connection;
- receiving a data message belonging to the connection from the electronic device;
- selecting one of a plurality of network slices to which the electronic device has access as the network slice for the connection based on analysis of the received data message; and
- forwarding the data message and subsequent data messages belonging to the connection onto the selected network slice to be delivered to the network domain.
35. The non-transitory machine readable medium of claim 34, wherein:
- the connection initiation message comprises a transmission control protocol (TCP) SYN message; and
- the set of instructions for exchanging the set of connection initiation messages comprises a set of instructions for sending a SYN-ACK message from the network slice selector to the electronic device and receiving an ACK message from the electronic device.
36. The non-transitory machine readable medium of claim 35, wherein the network slice selector further comprises a set of instructions for forwarding the SYN message and the ACK message onto the selected network slice to be delivered to the network domain, after selecting the network slice and prior to forwarding the data message.
37. The non-transitory machine readable medium of claim 34, wherein the selected network slice comprises a set of network services to be applied to data messages forwarded onto the selected network slice.
38. The non-transitory machine readable medium of claim 37, wherein the selected network slice is a first network slice and the set of network services is a first set of network services, wherein the plurality of network slices comprises at least a second network slice that comprises a second set of network services to be applied to data messages forwarded onto the second network slice.
39. The non-transitory machine readable medium of claim 37, wherein at least a subset of the network services are implemented as virtualized network functions (VNFs) that execute in at least two different datacenters.
40. The non-transitory machine readable medium of claim 34, wherein the set of instructions for forwarding the data message onto the selected network slice comprises sets of instructions for:
- forwarding the data message to a first network service of the selected network slice;
- receiving the data message from the first network service after the first network service processes the data message; and
- forwarding the data message to a second network service of the selected network slice.
41. The non-transitory machine readable medium of claim 34, wherein the network slice selector executes in a first datacenter, wherein the set of instructions for forwarding the data message onto the selected network slice comprises sets of instructions for:
- forwarding the data message to a first network service of the selected network slice that executes in the first datacenter;
- receiving the data message from the first network service after the first network service processes the data message; and
- forwarding the data message to a service chaining module that executes in a second datacenter, wherein the service chaining module forwards the data message to a second network service that executes in the second datacenter.
42. A system comprising:
- a set of host computers executing network services for a plurality of network slices; and
- a computing device executing a network slice selector that: receives, from an electronic device, a connection initiation message for a connection between the electronic device and a network domain; exchanges a set of connection initiation messages with the electronic device to set up the connection; receives a data message belonging to the connection from the electronic device; selects one of the plurality of network slices to which the electronic device has access as the network slice for the connection based on analysis of the received data message; and forwards the data message and subsequent data messages belonging to the connection to the selected network slice to be delivered to the network domain.
43. The system of claim 42, wherein the network slice selector forwards the data message onto the selected network slice by:
- forwarding the data message to a first network service of the selected network slice;
- receiving the data message from the first network service after the first network service processes the data message; and
- forwarding the data message to a second network service of the selected network slice.
Type: Application
Filed: Oct 22, 2022
Publication Date: Feb 9, 2023
Inventors: Marc-Andre Bordeleau (Shawinigan), Raja Kommula (Cupertino, CA), Jeremy Tidemann (Urbana, IL), Constantine Polychronopoulos (Saratoga, CA), Edward Choh (Richmond), Ojas Gupta (Mountain View, CA), Georgios Oikonomou (Patras), Robert Kidd (Champaign, IL)
Application Number: 17/971,590
