NSX vSphere components

NSX uses the management plane, control plane, and data plane models. The components are represented diagrammatically in the following diagram:

The management plane

The management plane contains the NSX Manager and vCenter Server. It is important to know that each NSX Manager should be registered with only one vCenter Server. The NSX Manager provides a management UI and API for NSX. We will be discussing NSX Manager and vCenter Server integration during NSX Manager installation and configuration modules. Once the integration is done, NSX Manager can be managed from a vSphere web client, which acts as a single pane of glass for configuring and securing the vSphere infrastructure. Immediate benefit is network administrators no longer need to switch between multiple management consoles. All network services can be configured and monitored from a single interface.

The control plane

The control plane primarily consists of NSX Controllers and the control VM, which allows us to perform distributed routing. The control plane also allows multicast-free VXLAN networks, which was a limitation in earlier vCloud networking and security versions. Controllers maintain ARP, VTEP (VXLAN tunnel endpoint), and MAC table. The NSX logical router control virtual machine and VMware NSX Controller are virtual machines that are deployed by VMware NSX Manager. The User World Agent (UWA) is composed of the ntcpad and vsfwd daemons on the ESXi host. Communication related to NSX between the NSX Manager instance or the NSX Controller instances and the ESXi host happen through the UWA. NSX Controller clusters are deployed in ODD number fashion and the maximum number of supported controllers is three. Since every controller in a control cluster is active at the same time, it ensures that the control plane is intact even if there is a controller failure. Controllers talk to each other to be in sync through a secured SSL channel. Controllers use a slicing technology to divide the workload among other controllers. Have a look at the following figure, which is a three-node controller cluster, in which slicing technology is dividing the workload across the controllers:

It is important to understand that there are two types of applications running on each of these controllers:

• VXLAN: Enables extension of a Layer-2 IP subnet anywhere in the fabric, irrespective of the physical network design.
• Logical router: Routing between IP subnets can be done in a logical space without traffic touching the physical router. This routing is performed directly in the hypervisor kernel with minimal CPU/memory overhead. This functionality provides an optimal data path for routing traffic within the virtual infrastructure.

The functionality of these applications is to learn and populate controller tables, and also distribute learned routes to underlying ESXi hosts. Lastly, the control plane and data plane configuration will be intact even during the failure of a management plane-this is the real power of software-defined networking.

Three-node controller clusters

In a large-scale distributed system with n number of servers, it is extremely difficult to ensure that one specific server can perform a write operation to a database or that only one server is the master that processes all writes. The fundamental problem is we do not have a simple way through which process execution can be done. How do we resolve this? All we need is to promote one server as master, and have some consensus with other servers. Paxos is a distributed consensus protocol published in 1989. The algorithm also ensures we have a leader election whenever there is a server failure. Paxos distinguishes the roles of proposer, acceptor, and learner, where a process (server/node) can play one or more roles simultaneously. The following are a few vendors who are using the Paxos algorithm extensively for the same reason:

• VMware NSX Controller uses a Paxos-based algorithm within an NSX Controller cluster
• Amazon Web Services uses the Paxos algorithm extensively to power its platform
• Nutanix implements the Paxos algorithm to ensure strict consistency is maintained in cassandara (for storing cluster metadata)
• Apache Mesos uses the Paxos algorithm for its replicated log coordination
• Google uses the Paxos algorithm for providing the Chubby lock service for loosely coupled distributed systems
• The Windows fabric used by many of the Azure services makes use of the Paxos algorithm for replication between nodes in a cluster

NSX Controllers are deployed in a three-node clustered fashion to ensure we are getting the highest level of resiliency since the controllers are running a fault-tolerant, distributed consensus algorithm called Paxos.

Controller roles

The NSX Controller provides the control plane functions for routing and logical switching functions. Each controller node is running a set of roles that defines the type of task the controller node can run. There are total of five roles running in a controller node; they are as follows:

• API
• Persistence server
• Logical manager
• Switch manager
• Directory server

While each of these roles needs a different master, it is important to understand that the leader is the responsible controller for allocating the tasks to other controllers.

The following figure depicts the responsibilities of various roles:

As we know, three node controllers form a control cluster. We will have a look at the role election per controller. Each role has a master controller node and only if a master controller node for a given role fails there would be a cluster-wide election for a new master role. This is one of the prime reasons a three-node cluster is a must in an enterprise environment, to avoid any split-brain situation which might eventually end up with data inconsistencies and the whole purpose of the control plane would be defeated. In the following figure, we have a three-node control cluster running and each controller is running a master role:

• Controller 1: Directory server master role running
• Controller 2: Persistence server master role running
• Controller 3: API and switch manager master role running

The data plane

NSX logical switches, ESXi hypervisor, distributed switches, and NSX edge devices are all data plane components. Once the management plane is up and running, we can deploy control plane and data plane software and components. Behind the scenes, these three VMware Installation Bundles (VIB) get pushed to the underlying ESXi hypervisor:

1. VXLAN VIB
2. Distributed routing VIB
3. Distributed firewall VIB

Up to now, we have discussed the management, control, and data plane components in the NSX world; in the upcoming modules, we will have a closer look at the installation part and design specification for each layer.