NSX-T Data Center Logical

Routing

© 2019 VMware, Inc.

Importance
In NSX-T Data Center, logical routing provides an optimized and scalable way of handling east-
west and north-south traffic. A solid understanding of NSX-T Data Center logical routing
architecture, routing components, and routing features is essential for building an efficient and
secure layer 3 network infrastructure.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-2

Module Lessons
• Lesson: Logical Routing Overview
• Lesson: NSX Edge and Edge Clusters
• Lesson: Configuring Tier-0 and Tier-1 Gateways
• Lesson: Configuring Static and Dynamic Routing
• Lesson: ECMP and High Availability

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-3

Logical Routing Overview

© 2019 VMware Inc. All rights reserved.

Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Explain the function and features of logical routing
• Describe the architecture of NSX-T Data Center two-tier routing
• Identify the components that work together in logical routing
• Differentiate between north-south and east-west routing
• Describe the functions of and interactions between service and distributed gateways
• Recognize the various types of gateway interfaces

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-5

Logical Routing Use Cases
In NSX-T Data Center, logical routing is used in many ways:
• Support for single or multitenant deployment models
• Complete separation of tenants and networks
• Solution for cloud environments with containerized workloads and multi-hypervisors
• Optimized routing path and simplified routing in virtual networks
• Distributed routing and centralized services in data centers
• Ability to extend logical networks to physical environments

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-6

Prerequisites for Logical Routing
For logical routing to work, certain requirements must be met:
• The NSX Management Cluster must be formed and available.
• Transport zones and N-VDS should be created.
• Hypervisors must be prepared as NSX-T Data Center transport nodes and added to the
management plane.
• Transport nodes must be attached to appropriate transport zones.
• An N-VDS must be created on each transport node.
• NSX Edge nodes must be deployed and preconfigured according to the requirements.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-7

Logical Routing in NSX-T Data Center
NSX-T Data Center gateways provide:
• Centralized north-south routing
• Distributed east-west routing
• Multitenant support
• Centralized stateful services, such as NAT or
load balancing

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-8

Gateway Components: Distributed Router and Service Router
Distributed Router (DR): Service Router (SR):
• Provides basic packet- • Provides on and off ramp
forwarding functionalities gateway services, including
• Spans all transport nodes north-south routing
(hypervisors and edge • Provides routing and
transport nodes) centralized services, such
• Runs as a kernel module in as NAT, load balancing,
the ESXi hypervisor and as and so on
an OVS file in the KVM • Functions provided by the
• Provides distributed east- NSX Edge node
west routing functionality
• Provides local routing on
the hypervisors

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5-9

Gateway: Distributed Router (1)
A single distributed router can span multiple ESXi and KVM hosts. A distributed router can
provide local routing between different segments on a host.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 10

Gateway: Distributed Router (2)
A distributed router can provide distributed east-west routing functionality across hosts.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 11

Gateway: Service Router
The service router (SR) is instantiated on the
NSX Edge node.
The service router has the following functions
and characteristics:
• Provides on and off ramp gateway services,
including north-south routing
• Provides routing and centralized services,
such as NAT, load balancing, and so on
• Functions provided by the NSX Edge node

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 12

Interaction between Distributed and Service Routers
A distributed gateway is replicated on all transport nodes that belong to the same transport, and it
runs local routing instances.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 13

About Edge Nodes
Edge nodes are appliances with pools of capacity for hosting any services that are not
distributed. Other characteristics of edge nodes include:
• Form factor choices include VM and bare metal server.
• Both OVA and ISO flavors are available for deployment.
• Edge node sizes are available in small, medium, and large.
• Edge nodes support active-active and active-standby configurations for resiliency.
• They leverage DPDK technology for fast packet processing.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 14

Logical Routing: Multitier Topology
A tiered architecture has
several key features and
functions:
• Supports tenant isolation
• Includes separate controls for
different administrative
domains
• Eliminates physical
dependency when new
tenants are introduced
The Tier-0 Gateway is also
referred to as provider, and the
Tier-1 Gateway as tenant.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 15

Tier-0 and Tier-1 Gateways
NSX-T Data Center supports different gateway tiers (logical routers) for different purposes.

Tier-0 Gateway Tier-1 Gateway

Generally owned and configured by the provider Owned and configured by the tenant
or infrastructure administrator
Supports static or dynamic routing (BGP) on Does not require or use dynamic protocols
uplinks toward the upstream physical gateway
Supports equal-cost multipath (ECMP) routing to Does not support ECMP and must connect to a Tier-0
upstream physical gateways Gateway for external connectivity
Offers a gateway service between logical and Offers default gateway services to local logical
physical networks (north-south) networks (east-west)
Provides segment interconnection and separation
Always requires an NSX Edge cluster Requires an NSX Edge cluster only if services are
used

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 16

Logical Router Interfaces
Types of interfaces used by gateways:
• Uplink interfaces (1) connect Tier-0
Gateways to upstream physical devices.
• Downlink interfaces (2) connect segments
(logical switches) to gateways.
• RouterLink ports (3) connect Tier-0 and Tier-
1 Gateways.
• An intra-tier transit link (4) is an internal link
between the distributed and service routers
on a gateway.
• The centralized service port (CSP) (5) is a
special interface for VLAN-based services
and partner service redirection. CSP is also a
downlink (2).

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 17

Centralized Service Port
CSP supported on NSX Edge nodes enables you to connect VLAN-backed switches to Tier-1 or
Tier-0 Gateways on a downlink port.
CSP connects to nondistributed services available on VLAN networks, including connectivity to
the physical infrastructure, NAT, load balancing, DHCP server, and so on.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 18

Single-Tier Deployment Example
In a single-tier deployment, segments are connected directly to Tier-0.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 19

Multitier Topology Examples
In a multitier deployment, segments are connected to Tier-1.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 20

Tier-0 Gateway Uplink Connections
Each Tier-0 service router running on NSX Edge nodes can have multiple uplinks to the physical
world. The Tier-0 deployment can be active-active or active-standby. ECMP can be enabled.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 21

Review of Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Explain the function and features of logical routing
• Describe the architecture of NSX-T Data Center two-tier routing
• Identify the components that work together in logical routing
• Differentiate between north-south and east-west routing
• Describe the functions of and interactions between service and distributed gateways
• Recognize the various types of gateway interfaces

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 22

NSX Edge and Edge Clusters

© 2019 VMware Inc. All rights reserved.

Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Describe the main functions and features of NSX Edge
• Explain NSX Edge architecture
• Recognize benefits of NSX Edge clusters
• Describe the various types of routing topologies
• Explain the north-south packet-flow process
• Describe edge node deployment methods and configuration process

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 24

NSX Edge Functions
NSX Edge has several functions:
• Serves as a resource for the routing
components of user-created gateways on
hosts, providing connectivity to external
networks
• Hosts gateways offering various networking
services, such as NAT, load balancing, and
so on
• Offers DPDK-based VM or bare metal form
factors for high performance
• Terminates overlay network tunnels
• Uses Linux OS-based control plane
• Uses separate routing tables for
management and overlay traffic

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 25

NSX Edge VM Form Factor and Sizing Options

The form factors supported by NSX Edge are:

• VM on ESXi host
• Bare metal
For NSX Edge nodes deployed as VMs on hypervisors, several deployment sizes are available.

Size Memory vCPU Disk Space VM Hardware

Version
Small 4 GB 2 200 GB 11 or later (vSphere
6 or later)
Medium 8 GB 4 200 GB 11 or later (vSphere
6 or later)
Large 32 GB 8 200 GB 11 or later
(vSphere 6 or
later)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 26

NSX Edge Bare Metal Hardware Requirements
NSX Edge bare metal supports only specific CPU types. You can find the CPU requirements in
the NSX-T Data Center Installation Guide.
The NSX Edge bare metal node has memory, CPU, and disk requirements.

Form Factor Memory CPU Cores Disk CPU Capability

Bare metal 32 GB 8 200 GB AES-NI1 GB Huge
Page support

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 27

Logical Routing Topology (1)
The edge transport nodes are in the management cluster, separated from the cluster running the
workloads.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 28

Logical Routing Topology (2)
The transport nodes run the DR instances individually, separated from the edge nodes running
the SR instances.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 29

Logical Routing Topology (3)
The VM communicates with external entities through the edge nodes. The edges nodes in
management clusters participate in the data path for the external traffic.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 30

NSX Edge Cluster Guidelines
An NSX Edge cluster is a group of homogeneous nodes with common properties. When you
configure an edge cluster, the following guidelines apply:
• Maximum of 10 edge nodes are supported in a cluster: Use one cluster to provide eight-way
ECMP paths northbound and another cluster to provide centralized services for tenants.
• An edge transport node can be added to only one edge cluster.
• Maximum of 16 clusters can be configured in NSX-T Data Center 2.4.
• When configuring any kind of service through a service router, an edge cluster is required.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 31

NSX Edge Node Deployment Prerequisites
Requirement Description
Supported deployment media OVA or OVF, preboot execution environment (PXE), ISO
with or without PXE.
Supported platforms ESXi hypervisors for VM. Bare metal has specific hardware
requirements.
PXE The password for root and admin users must be encrypted
with SHA-512.
Host name The host name must not contain invalid characters.
VMware Tools Preinstalled version in the VM. Removing and replacing are
not supported.
NSX TCP and IP ports Verify that required ports are open.
IP addresses Plan IPv4/IPv6 addressing scheme accordingly.
OVF template Verify user privileges to install OVF templates. Install
vSphere Client integration plug-in in browser.
Network Time Protocol (NTP) service The same NTP service should be used by all the edge
nodes in an edge cluster.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 32

Deploying NSX Edge Nodes from the Simplified UI
You can deploy edge transport nodes from the NSX Manager simplified UI.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 33

Using vCenter Server to Deploy NSX Edge Nodes
If you prefer an interactive edge installation, you can deploy edge nodes by using a UI-based VM
management tool, such as vSphere Client connected to vCenter Server.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 34

Using the OVF Tool to Deploy NSX Edge Nodes
For the scripted NSX Edge installation, you can
use the OVF tool (4.0 or above) command-line
utility.
With both standalone host and vCenter Server
installations, you use the appropriate
command-line switches.
The NSX Edge OVF has specific properties
defined. They must be referred to as prop in
ovftool with --allowExtraConfig switch.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 35

Installing NSX Edge on Bare Metal
You can install NSX Edge for NSX-T Data
Center on bare metal by using an ISO file:
1. Verify that the system BIOS mode is set to
Legacy BIOS.
2. Create a bootable disk with the NSX Edge
ISO file on it.
3. Boot the physical machine from the disk.
4. Select Automated install.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 36

Using PXE to Deploy NSX Edge Nodes from an ISO File
Using PXE, you can install NSX Edge nodes on bare metal or as a VM.
This procedure automatically configures networking settings, such as IP address, gateway,
network mask, NTP, and DNS.
The PXE boot process is made up of several components, including DHCP, HTTP, and TFTP
servers.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 37

Joining NSX Edge with the Management Plane
Installing NSX Edge node by any method other than NSX Manager does not automatically join
the NSX Edge to the management plane.
To join an NSX Edge node with the management plane, follow these steps:
1. Open an SSH session to the NSX Manager appliance and retrieve the SSL thumbprint by
entering get certificate api thumbprint at the command prompt.
2. Open an SSH session to the edge node and run the join management-plane command.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 38

Verifying the Edge Transport Node Status
In the simplified UI, the Edge Transport Nodes page lists different states of the edge nodes
known by NSX Manager.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 39

Enabling Edge Node SSH Service
By default, the SSH service is disabled on edge nodes. You can enable SSH to access the CLI
for management and troubleshooting purposes.
You can start and enable SSH only through the console by using the CLI.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 40

Postdeployment Verification Checklist
After deployment, you can verify the connectivity of the NSX Edge nodes in several ways:
 If you enabled SSH, ensure that you can SSH to the newly deployed edge nodes.
 Check that you can ping your NSX Edge node.
 Ensure that the edge nodes can ping their corresponding default gateway.
 Confirm that the NSX Edge nodes can ping the hypervisor hosts that are in the same network
as the NSX Edge nodes.
 Ensure that the NSX Edge nodes can reach their configured DNS server and NTP server.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 41

Creating an Edge Cluster
You can deploy an edge cluster from the NSX Manager simplified UI.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 42

Mapping NSX Edge Node Interfaces (1)
An edge node deployment requires various
interface types and assignments:
• On the vSphere distributed switch or
standard switch, you must allocate at least
two vmnics to the NSX Edge node for VM
form factor.
• The first interface must be defined for
management access (eth0 or em0), using
one vNIC or physical interface.
• Data path interfaces (named as fp-ethX) are
allocated by the N-VDS module for overlay
tunneling and uplink connections using the
remaining vNICs.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 43

Mapping NSX Edge Node Interfaces (2)
The data path interfaces can be defined when
adding the edge transport node:
• The number of TEP interfaces is based on
the Uplink Profile selection.
• N-VDS logical uplink-x interfaces must be
assigned to a port group in vSphere.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 44

Verifying NSX Edge Node Interfaces Mapping
After you map the NSX Edge node interfaces, you can verify the mapping on the N-VDS tab.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 45

Edge Node VM Deployment Options
You can deploy an NSX Edge VM on an NSX prepared ESXi host:
• If the ESXi transport node has multiple virtual switches running, for example, one standard or
distributed switch (from vSphere) and one N-VDS from NSX, then the NSX Edge vNICs can be
attached to the standard switch or distributed switch.
• In the same scenario, the NSX Edge node vNICs can be attached to the N-VDS as well.
• If the ESXi host transport node uses only the N-VDS, the NSX Edge vNICs are attached to
VLAN logical switches. The host TEP and the NSX Edge TEP must be in different subnets.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 46

Lab: Deploying and Configuring NSX Edge Nodes
Deploy NSX Edge nodes and configure them as transport nodes
1. Prepare for the Lab
2. Deploy Two Edge Nodes from the NSX Manager Simplified UI
3. Enable SSH on the Edge Nodes
4. Configure an Edge Cluster

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 47

Review of Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Describe the main functions and features of NSX Edge
• Explain NSX Edge architecture
• Recognize benefits of NSX Edge clusters
• Describe the various types of routing topologies
• Explain the north-south packet-flow process
• Describe edge node deployment methods and configuration process

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 48

Configuring Tier-0 and Tier-1 Gateways

© 2019 VMware Inc. All rights reserved.

Learner Objectives

After completing this lesson, you should be able to meet the following objectives:
• Identify routing configuration tasks
• Configure a Tier-0 Gateway
• Configure a Tier-1 Gateway
• Test end-to-end connectivity provided by Tier-0 and Tier-1 Gateways

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 50

Gateway Configuration Tasks

To achieve full network connectivity, you must perform gateway

configuration tasks:
• For the provider-level (Tier-0) configuration:
— Create the Tier-0 Gateway and its segments.
— Configure static routing or BGP routing on the Tier-0
Gateway.
• For the tenant-level (Tier-1) configuration: Create the Tier-1
Gateway and its segments.
• Configure segments and connections to gateways.
• Configure connectivity between Tier-0 and Tier-1 Gateways.
• Enable route advertisement and redistribution.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 51

Configuring a Tier-0 Gateway: Step 1
Step 1: In the NSX Manager simplified UI, create uplink segments to associate with the Tier-0
Gateway uplinks.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 52

Configuring a Tier-0 Gateway: Step 2
Step 2: Create a Tier-0 Gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 53

Configuring a Tier-0 Gateway: Step 3
Step 3: Configure gateway interfaces to associate with the previously created (Step 1) uplink
segments.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 54

Configuring a Tier-0 Gateway: Step 4
Step 4: Configure static routing (as required) on the Tier-0 Gateways.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 55

Configuring a Tier-0 Gateway: Step 5
Step 5: Configure static routes and next-hop addresses for remote networks (as required).

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 56

Reviewing the Tier-0 Gateway Configuration
After a Tier-0 Gateway is successfully created, it is listed on the Tier-0 Gateway page.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 57

Configuring a Tier-1 Gateway: Step 1
Step 1: Create a Tier-1 Gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 58

Configuring a Tier-1 Gateway: Step 2
Step 2: Add interfaces on the Tier-1 Gateway to connect to segments (tenant networks). Click the
SEGMENTS tab.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 59

Testing East-West Connectivity
With gateway ports configured on the Tier-1 Gateway, VMs on various subnets (segments)
attached to the Tier-1 Gateway can reach each other.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 60

Configuring a Tier-1 Gateway: Step 3
Step 3: Connect the Tier-1 Gateway to the Tier-0 Gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 61

Configuring a Tier-1 Gateway: Step 4
Step 4: Enable route advertisement so tenant networks can be propagated to Tier-0 Gateways.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 62

Testing North-South Connectivity
VMs on the tenant networks can now communicate with external workloads.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 63

Routing Topologies
For NSX-T Data Center tiered routing with or
without services, you can use two typologies:
• Single tier:
— Only the Tier-0 Gateway is included in this
topology.
— Downlinks and services are provided.
— The Tier-0 Gateway is connected to
external networks.
• Multitier:
— Both Tier-0 and Tier-1 Gateways are
included in this topology.
— Depending on needs, the Tier-1 Gateway
might run the service instance (SR).
— The Tier-0 Gateway is connected to
external networks.
© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 64
Single-Tier Topology
East-west routing is performed by the distributed router in the Tier-0 Gateway.
The service router (SR) establishes external connectivity and also enables any stateful services.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 65

Single-Tier Routing: Egress to Physical Network (1)
A packet needs to be sent from the source (Src) VM [Link] to the destination (Dst) VM
[Link]. The packet is first forwarded to the gateway [Link].

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 66

Single-Tier Routing: Egress to Physical Network (2)
The gateway checks its routing table to make a routing decision.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 67

Single-Tier Routing: Egress to Physical Network (3)
To send the packet to a remote host (TEP [Link]), the source host (TEP [Link])
encapsulates the packet with a GENEVE header. The original packet is intact.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 68

Single-Tier Routing: Egress to Physical Network (4)
The encapsulated packet travels through the overlay tunnel and arrives at the edge node.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 69

Single-Tier Routing: Egress to Physical Network (5)
The edge node decapsulates the packet and makes a routing decision. To reach the destination
network [Link]/24, the default route [Link]/0, with the next-hop [Link], is used.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 70

Single-Tier Routing: Egress to Physical Network (6)
The edge node sends the packet to its upstream physical gateway, which, in turn, routes the
packet to its destination [Link].

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 71

Single-Tier Routing: Ingress from Physical Network (7)
The return packet sourced from VM [Link] is destined for VM [Link]. The source VM
[Link] sends the packet to its default gateway, which routes the packet to the edge node.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 72

Single-Tier Routing: Ingress from Physical Network (8)
The edge node SR checks its routing table and decides that the packet should be forwarded to
the next-hop [Link], which is on a remote host.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 73

Single-Tier Routing: Ingress from Physical Network (9)
The edge node SR checks and passes the packet to the DR.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 74

Single-Tier Routing: Ingress from Physical Network (10)
The edge node DR refers to its ARP and MAC tables and decides that the packet’s next-hop is
the remote host (TEP [Link]).

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 75

Single-Tier Routing: Ingress from Physical Network (11)
The edge node encapsulates the packet with a GENEVE header.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 76

Single-Tier Routing: Ingress from Physical Network (12)
The encapsulated packet is sent across the overlay tunnel.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 77

Single-Tier Routing: Ingress from Physical Network (13)
The receiving host decapsulates the packet and routes it to its destination (VM [Link]).

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 78

Multitier Topology (1)
East-west routing is performed by the distributed router in the Tier-1 Gateway. The Tier-0
Gateway provides services and external connectivity.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 79

Multitier Topology (2)
If the Tier-1 Gateway is not configured with any services, the service router component is not
instantiated on the Tier-1 Gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 80

Multitier Topology (3)
When stateful services are configured on the Tier-1 Gateway, the service router component is
instantiated on the Tier-1 Gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 81

Multitier Routing: Egress to Physical Network Example
This topology demonstrates an example of routing through multiple tiered gateway routers.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 82

Multitier Routing: Egress to Physical Network (1)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 83

Multitier Routing: Egress to Physical Network (2)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 84

Multitier Routing: Egress to Physical Network (3)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 85

Multitier Routing: Egress to Physical Network (4)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 86

Multitier Routing: Egress to Physical Network (5)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 87

Multitier Routing: Egress to Physical Network (6)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 88

Multitier Routing: Egress to Physical Network (7)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 89

Multitier Routing: Egress to Physical Network (8)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 90

Multitier Routing: Egress to Physical Network (9)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 91

Multitier Routing: Egress to Physical Network (10)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 92

Multitier Routing: Egress to Physical Network (11)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 93

Multitier Routing: Egress to Physical Network (12)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 94

Multitier Routing: Egress to Physical Network (13)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 95

Multitier Routing: Egress to Physical Network (14)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 96

Multitier Routing: Egress to Physical Network (15)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 97

Multitier Routing: Egress to Physical Network (16)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 98

Multitier Routing: Egress to Physical Network (17)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 99

Lab: Configuring the Tier-1 Gateway
Create a Tier-1 Gateway and configure gateway ports
1. Prepare for the Lab
2. Create a Tier-1 Gateway
3. Create Gateway Ports on Segments
4. Test East-West L3 Connectivity

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 100

Review of Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Identify routing configuration tasks
• Configure a Tier-0 Gateway
• Configure a Tier-1 Gateway
• Test end-to-end connectivity provided by Tier-0 and Tier-1 Gateways

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 101

Configuring Static and Dynamic Routing

© 2019 VMware Inc. All rights reserved.

Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Distinguish between static and dynamic routing
• Configure static routes on the Tier-0 Gateway
• Describe Tier-0 Gateway capabilities and supported features
• Recognize BGP features supported by the Tier-0 Gateway
• Configure BGP on the Tier-0 Gateway
• Recognize BGP route advertisement filters

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 103

Static and Dynamic Routing
Static routing: Dynamic routing:
• Static route configuration is performed • Dynamic route configuration enables
manually by administrators. gateways to exchange information about the
• The configuration process enables the fine- network.
tuning of route selection. • Routing protocols are used to share
• Route changes cannot be made dynamically. information about networks.
• Limited scalability is due to administrative • Routers inform neighbor gateways when a
overhead. network change occurs.
• Failover planning is possible: • Dynamic routing protocol categories:

— Network administrators must design and — Interior gateway protocols (IGPs)

account for all network failure scenarios. — Exterior gateway protocols (EGPs)
— Route redundancy must be configured
manually.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 104

Tier-0 Gateway Capabilities
Tier-0 Gateway supports:
• Static routing toward upstream physical
gateways
• Dynamic routing (BGP):
— External BGP (eBGP) sessions with
upstream physical gateways
— Internal BGP (iBGP) sessions with other
Tier-0 Gateways in the same AS
• The Bidirectional Forwarding Detection
(BFD) protocol for fast failover
• ECMP toward physical gateways

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 105

Configuring Static Routes on a Tier-0 Gateway (1)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 106

Configuring Static Routes on a Tier-0 Gateway (2)

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 107

Viewing the Static Route Configuration

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 108

BGP on Tier-0
External BGP is used to establish neighbor
relationships between Tier-0 and upstream
physical gateways with different AS.
Network prefixes are exchanged between the
BGP peers.
The BGP dynamic neighbor enables peering to
a group of remote neighbors.
Four-byte autonomous system number (ASN) is
supported.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 109

Routing Features Supported by the Tier-0 Gateway
The Tier-0 Gateway supports
several routing features:
• eBGP (single and multihop)
• Inter-SR iBGP and next-hop-
self
• Route Aggregation
• Community Lists
• IP Prefix Lists
• Route Maps
• Allow AS-In

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 110

Configuring Dynamic Routing on Tier-0 Gateways: Step 1
Step 1: Enable BGP and set the local AS.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 111

Configuring Dynamic Routing on Tier-0 Gateways: Step 2
Step 2: Configure BGP neighbors.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 112

Configuring a Tier-0 Gateway: Step 3
Step 3: Configure route redistribution.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 113

Verifying BGP Configuration of Tier-0 Gateway on Edge Nodes
You use the edge node CLI to verify NSX Edge BGP connections.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 114

BFD on a Tier-0 Gateway
BFD is a protocol that can detect forwarding
path failures:
• Provides fast detection of node (edge or
physical gateway) or uplink failure
• Protects both static routes and BGP peers
• Establishes multiple BFD sessions if multiple
links exist between two systems
• Can be enabled per BGP neighbor or
globally per gateway

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 115

Enabling BFD on a Tier-0 Gateway

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 116

About IP Prefix Lists
An IP prefix list contains IP networks with subnet masks that are permitted or denied, based on
the match condition. IP prefix lists are used in BGP filters or route maps with the in or out
direction specified.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 117

Configuring an IP Prefix List
You can configure the IP prefix list by allowing or denying network prefixes:
• Allow (Permit) [Link]/8 network prefixes so that they can be advertised out.
• Deny [Link]/24 network prefixes with le and ge settings for the network mask: those with
greater than or equal to 26 bits and less than or equal to 30 bits.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 118

About Route Maps (1)
A route map defines which routes from the specified routing protocol can be redistributed into the
target routing process.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 119

About Route Maps (2)
A route map consists of matching criteria and BGP attributes.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 120

Using Route Maps in BGP Route Advertisements
Route maps are supported globally or on a per-neighbor (BGP peer) basis.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 121

BGP Feature: Allow AS-In
By default, BGP drops received routes that
contain their own ASN to avoid loops.
For a single customer with two sites
interconnected to the same ISP, routes
received from a BGP peer can contain the
same ASN.
The BGP allowas-in configuration option
can be used to accept those routes.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 122

BGP Feature: Multipath Relax
BGP multipath relax enables ECMP across
different neighboring ASNs if all other attributes
are equal.
• To support load balancing, the same prefix
can be advertised from multiple BGP
gateways.
• From the perspective of other eBGP
neighbors, this prefix includes BGP paths
with different AS_PATH attribute values but
the same AS_PATH attribute lengths.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 123

Internal BGP Support
Internal BGP (iBGP) is
supported between the Tier-0
Gateway and the upstream
router.
Route map supports set
local-preference and set
next-hop-self options.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 124

About Inter-SR Routing
Service routers (SRs) automatically exchange
routing information through iBGP peers inside
the same Tier-0 Gateway.
Inter-SR routing works in the following ways:
• Increases the resiliency by avoiding a traffic
blackhole if only a single uplink is faulty
• Synchronizes eBGP and static routes
• Is only applicable for active-active Tier-0
Gateways

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 125

Inter-SR Routing (2)
Inter-SR routing uses an automatically
generated internal interface. The interface has
the following characteristics:
• IP address range: 169.254.0.x
• No TTL decrement for traffic going over this
interface
The auto-plumbed iBGP peer has the following
features and functions:
• It cannot be changed.
• One iBGP neighbor is automatically created
for carrying IPv4 and IPv6 routes.
• Connected and user-defined static routes
can be redistributed in BGP.
• Inter-SR routing is not supported when
running iBGP northbound.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 126

Inter-SR Routing Example (1)
EN1 is used to route traffic from the blue VM to the Internet.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 127

Inter-SR Routing Example (2)
The link between EN1 and the upstream router fails.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 128

Inter-SR Routing Example (3)
Because iBGP is established between EN1 and EN2, traffic from the blue VM is rerouted to the
Internet, avoiding the failed link.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 129

Lab: Configuring the Tier-0 Gateway
Create a Tier-0 Gateway and configure north-south end-to-end connectivity
1. Prepare for the Lab
2. Create Uplink Segments
3. Create a Tier-0 Gateway
4. Connect the Tier-0 and Tier-1 Gateways
5. Test the End-To-End Connectivity

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 130

Review of Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Distinguish between static and dynamic routing
• Configure static routes on the Tier-0 Gateway
• Describe Tier-0 Gateway capabilities and supported features
• Recognize BGP features supported by the Tier-0 Gateway
• Configure BGP on the Tier-0 Gateway
• Recognize BGP route advertisement filters

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 131

ECMP and High Availability

© 2019 VMware Inc. All rights reserved.

Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Explain the purpose of ECMP routing
• Configure ECMP routing using the NSX Manager simplified UI
• Identify the active-active and active-standby modes for high availability
• Recognize failure conditions, the failover process, and failback modes

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 133

About Equal-Cost Multipath Routing
ECMP routing has several features and
functions:
• It increases the north-south communication
bandwidth by combining multiple uplinks.
• ECMP routing performs traffic load
balancing.
• ECMP routing provides fault tolerance for
any failed paths.
• A maximum of eight ECMP paths are
supported.
• Hashing is based on 2-tuple IP source and
destination addresses.
• ECMP routing is only available on Tier-0
Gateways.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 134

Enabling ECMP
ECMP is automatically enabled on Tier-0 Gateways, when BGP is enabled. If required, ECMP
can be disabled from the BGP configuration section on the Tier-0 Gateways page.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 135

Edge Node High Availability
Multiple edge nodes can be pooled in a cluster for scale out and redundancy.
The high availability feature supports active-active and active-standby modes.
For example, Tier-0 provides ECMP in active-active mode and stateful services in active-standby
mode on the same pair of edge nodes.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 136

Tier-0 Gateway Active-Active Mode
Active-active mode is the default high
availability mode for Tier-0 Gateways.
Tier-1 Gateways are not supported in this
mode.
Logical routing is active on more than one edge
node at a time.
The active-active mode supports:
• Scale-out high availability
• ECMP routing
• Stateless services (reflexive NAT)
A maximum of eight active-active nodes are
supported per gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 137

Tier-0 Gateway Active-Standby Mode
Active-standby is the default high availability
mode for Tier-1 Gateways. Tier-0 Gateways
also support this mode.
Logical routing is active on only one edge node
at a time. Similarly, SR is active on only one
edge node at a time.
Centralized stateful services provided in the
active-standby mode include:
• SNAT/DNAT
• Load balancer
• Edge firewall
• DHCP server
• VPN

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 138

Failure Conditions and Failover Process (1)
The BFD protocol detects forwarding path
failures.
It provides low-overhead detection of faults
even on physical media that does not support
failure detection of any kind, such as Ethernet.
BFD keepalives are sent on both management
and tunnel interfaces.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 139

Failure Conditions and Failover Process (2)
If a standby gateway fails to receive keepalives on both management and tunnel interfaces, the
gateway becomes active.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 140

Failure Conditions and Failover Process (3)
Dynamic routing protocol peer sessions are established on the uplinks. If an active gateway loses
all its BGP neighbors and a standby gateway is available, the active gateway steps down and
becomes the standby gateway.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 141

Edge Node Failback Modes
You can select different fail over
options:
• Preemptive: If the preferred
node fails and then recovers,
it takes over its peer and
becomes the active node.
The peer changes its state to
standby.
• Non Preemptive: If the
preferred node fails and then
recovers, it checks whether
its peer is active. If so, the
preferred node does not take
over its peer and stays in the
standby mode.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 142

Lab: Verifying Equal Cost Multipathing Configurations
Enable Equal Cost Multipathing on gateways
1. Prepare for the Lab
2. Verify the BGP Configuration
3. Verify that Equal-Cost Multipathing is Enabled
4. Verify the Result of the ECMP Configuration

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 143

Review of Learner Objectives
After completing this lesson, you should be able to meet the following objectives:
• Explain the purpose of ECMP routing
• Configure ECMP routing using the NSX Manager simplified UI
• Identify the active-active and active-standby modes for high availability
• Recognize failure conditions, the failover process, and failback modes

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 144

Key Points (1)
• The NSX-T Data Center routing function meets the needs of service providers and tenants.
• Static route configuration is performed manually by an administrator.
• Dynamic route configuration enables gateways to exchange information about the network.
• NSX logical routing commonly implements a two-tiered topology.
• Tier-1 Gateways have downlink ports to connect to NSX logical switches and uplink ports to
connect to NSX Tier-0 Gateways.
• A gateway consists of two optional parts: a distributed gateway and one or more service
gateways.
• You can deploy an NSX Edge node through the NSX Manager UI, the OVF tool, and an ISO
file in a PXE environment.
• Joining NSX Edge nodes with the management plane ensures that NSX Manager and the NSX
Edge nodes can communicate with one another.

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 145

Key Points (2)
• A multinode NSX Edge cluster helps ensure that at least one NSX Edge node is always
available.
• EBGP is the interchange of autonomous system IP addresses within a particular host section
of IP addresses.
• An IP prefix list contains one or more IP addresses that are assigned access permissions for
route advertisement.
• ECMP routing increases the north-south communication bandwidth by adding an uplink to the
Tier-0 Gateway and configuring it for each NSX Edge node in an NSX Edge cluster.
• Multiple edge nodes can be pooled in a cluster for scale out and redundancy.
• High availability supports two modes: active-active and active-standby.
Questions?

© 2019 VMware, Inc. VMware NSX-T: Install, Configure, Manage | 5 - 146