Accelerating Open Networking with

Edge-Core

© 2015 Edgecore Networks. All rights reserved. Subject to errors and misprints. | [Link]
Open Networking
What is Open Networking?
Applications

Controller

Open Software

Open Hardware
Open Networking Architecture
Cloud Orchestration
Network & Distributed Services
Control Applications

Linux Provisioning
Monitoring, Management
SDN Controller

Open
Linux Linux Source
User Agents Network Legacy Open
Plug-ins Switch
Space Protocols Quagga Protocols OS

Linux OS

Universal SwitchOS Installer (ONIE) Bare-Metal

Switch
MerchantProprietary
Traditional Silicon
HW
Network Product
Edge-Core Hardware Offerings
• 1G, 10G, 40G, 100G Bare Metal Switches
– Broadcom switching ASIC
– Freescale PowerPC or Intel x86 CPU
– Open Network Install Environment (ONIE)
• Installer / Boot loader for the switch OS
 Data Center Switches Hardware

1GbE 10G SFP+ 40G QSFP+ 100G QSFP28

AS5712-54X (Intel Atom C2538)

AS4600-54T AS6712-32X (Intel Atom C2538)) AS7700-32X
48 x 10G SFP+, 6 x 40G QSFP+
48 x 1G Base-T, 32x40G QSFP+ 32 x 100G QSFP28
4 x 10G SFP
+ 2 modules
AS5710-54X (Freescale P2041)
48 x 10G SFP+, 6 x 40G QSFP+ AS6700-32X (Free scale P2041)
32x40G QSFP+

AS5610-52X (Freescale P2041) AS6701-32X (Free scale P2020)

AS4610-54P*
48 x 10G SFP+, 4 x 40G QSFP+ 32x40G QSFP+
48 x 1G Base-T,
4 x 10G SFP
+ 2 modules
AS5600-52X Freescale P2020)
48 x 10G SFP+, 4 x 40G QSFP+
Wedge-16X

Wedge-16X
16x40G, BRCM Trident II
Intel Rangely Micro Server
• OpenBMC
• FBOSS Agent
Edge-Core Contribution to OCP
Full Range of Open Switch Designs
– First OCP-Approved Switch: 10G TOR

– New Contribution 40G Cost-Optimized for TOR, Spine

– New Contribution 100G Industry’s First Open 100G Design

– OCP-Approved Open Rack Switch Adapter

OCP
OSRP 1RU
Edge-Core Software Offerings
• In house developed EdgeCOS

• 3rd party software (No Vendor Lock In!)

– Cumulus Linux
– Big Switch Big Tap / Big Cloud Fabric
– Pica8 PicOS
– Broadcom Turnkey
– IP-infusion
– Pluribus
– Tier 1 partner’s in-hours software (HP, FB & others..)
Software Options

 HW:Bare Metal Switch with ONIE

 OS: Cumulus Linux
 Management and Automation: Puppet, Chef, CFEngine
Cumulus  Monitoring: Collectd, Monit, Ganglia, NetSNMP, Nagios/Icinga, Graphite
 HW:Bare Metal Switch with ONIE
 OS: Switch Light OS
Big Switch  Controller: Big Switch Controller
 Application: Big-Tap, Big cloud fabric
 HW:White Box Switch
 OS:PicOS with Standard L2/L3
 Openflow 1.3 (OVS)
 Controller: 3rd party Ryu, Big Switch, Floodlight, Trema, NOX
 Management and Automation: Puppet, Chef, CFEngine
 Software Options

 HW:White Box Switch

Broadcom  Standard L2/L3
Turnkey  Openflow 1.3 (Accton or OFDPA)
 Controller: 3rd party
IP-Infusion  HW: Bare Metal Switch with ONIE
 Standard L2 +L3+ MPLS
Pluribus  Server Switch
 Open source software
 HW: Bare Metal Switch with ONIE
 Standard L2 +MLAG /L3+ OSFP/BGP +ECMP, VxLAN
 Openflow 1.3 (OFDPA), VxLAN, MPLS L2 VPN
 Controller: 3rd party Ryu, Opendaylight (currently), other in the future
(ONOS etc)
Data Center Switches Hardware and Software Compatibility List

1GbE 10G SFP+ 40G QSFP+ 100G QSFP28

AS5712-54X
AS4600-54T AS6712-32X AS7712-32X

AS5710-54X
AS6700-32X

AS5610-52X AS6701-32X
AS4610-54P

AS5600-52X
 Open Networking Software Free Distribution

Available for OCP hardware

10G Open HW platform
• AS5710-54X
• AS5712-54X
40G Open HW platform
• AS6712-32X
100G Open HW platform
• AS7712-32X
• AS7512-32X (Cavium)
ONF Atrium open source
SDN Distribution
Edge-COS Software Platform
OFDPA (Openflow Data Plane Abstraction)
 Broadcom’s implementation of OpenFlow 1.3
 Abstraction of OpenFlow 1.3 to fit the HW silicon
 Supports OF 1.3 flexibility on ASIC based Switches
 Translate OF features into hardware environment
 Multi-tables, multi-features
AS5710-54X / AS6700-32X AS5600-52X Pica 8

ACL Policy Flow Table 4K 2K Depend on chipset

Ingress Port Flow Table 1 1 X
VLAN Flow Table 4094 4094 X
Termination MAC Flow Table 512 512 X
Bridging Flow Table 288K 128K X
Unicast Routing Flow Table 112K IPv4/, 56K IPv6 16K/8K X

Multicast Routing Flow Table 104K IPv4/52K IPv6 4K/2K X

OpenFlow OFDPA Abstraction Switch Pipeline

30

10 20 40 60

50
More Flow Table Entries
Switching
Bridging
Switch MAC MAC Eth VLAN IP IP IP TCP TCP
Action Flow Table
Port src dst type ID Src Dst Prot sport dport
(Max 288K)
* * [Link].. * * * * * * * port6

VLAN
VLAN
Switch MAC MAC Eth VLAN IP IP IP TCP TCP Flow Table
Action
Port src dst type ID Src Dst Prot sport dport (4K)
port6,
* * * * vlan1 * * * * * port7,port9

Routing
Unicast Routing
Switch MAC MAC Eth VLAN IP IP IP TCP TCP
Action Flow Table
Port src dst type ID Src Dst Prot sport dport
(Max 112K)
* * * * * * [Link] * * * port6

Flow Switching
Switch MAC MAC Eth VLAN IP IP IP TCP TCP
Action
Port src dst type ID Src Dst Prot sport dport
port3 00:2e.. 00:1f.. 0800 vlan1 [Link] [Link] 4 17264 80 port6
ACL Policy
Flow Table
Firewall
Switch MAC MAC Eth VLAN IP IP IP TCP TCP
Forward
Port src dst type ID Src Dst Prot sport dport
* * * * * * * * * 22 drop
Edge-Core SDN Solution
• Bare Metal Switch
• EdgeCOS switch OS
– OpenFlow 1.3 with Broadcom OF-DPA support
• SDN Controller (OpenDayLight)
• SDN Applications (OF-DPA compliant)
• OpenStack integration
Flooding Domain
No Flooding Domain
SPINE 1 SPINE 2
Tenant 1
Tenant 2

TOR 1 TOR 2 TOR 3

VM A VM B VM C VM D VM E VM F

Server 1 Server 2 Server 3

19
Flooding Domain (Performance & Security)- cont.
RabbitMQ

REST API ODL Controller + APP Flooding Domain Openflow

SPINE 1 SPINE 2

Openstack Controller
Tenant 1
+ Network node
VLAN 2000 Tenant 2
DA [Link] 5001
SA [Link]
L2 IP UDP VxLAN L2 IP Payload
VLAN 2000

TOR 1 TOR 2 TOR 3

DA Broadcast
SA MAC A
L2 IP Payload

VM A VM B VM C VM D VM E VM F

Neutron OVS Agent Server 1 Server 2 Server 3 Neutron OVS Agent

VTEP IP 192.168. VTEP IP [Link]
Openstack
Compute Node
Switch Fabric with ECMP
Without ECMP
SPINE 1 SPINE 2
Tenant 1
Tenant 2
Tenant 3

TOR 1 TOR 2 TOR 3

VM A VM B VM C VM D VM E VM F

Server 1 Server 2 Server 3

21
Switch Fabric with ECMP (Load balance) - cont.

ODL Controller + APP

With ECMP
SPINE 1 SPINE 2
Tenant 1

Openstack Controller Tenant 2

+ Network node Tenant 3
L3 ECMP

TOR 1 TOR 2 TOR 3

VM A VM B VM C VM D VM E VM F

Server 1 Server 2 Server 3

22
 Demo Topology
DUT1 (SPINE1) DUT2 (SPINE2)
AOS6700 1 AOS6700 Management PC
1 32
32 Craft port and
16 16
serial console of
all DUTs

49
49 54 54
54 49
DUT3 (TOR1,[Link]/24) DUT4 (TOR2,[Link]/24) DUT5 (TOR3,
AS5700 AS5700 AS5700 [Link]/24)
2 1 2
2 L2SW
Tenant 1(demo)
Tenant 2 (admin)

192.168..1.35 VM G
VM A VM D [Link] VM H
192.168..1.33 192.168..1.36
VM B 192.168..2.17
VM E [Link]
VM C 192.168..5.27
VM F [Link]
Compute Node 1 Compute Node 2 Compute Node 3 OPENSTACK OPENSTACK ODL and APP
Controller Network
Network1 VMA,D,G,H Network2 VMB,E [Link] [Link] [Link]
Network3 VMC,F

23
ODL Controller – Config

Configure DUT1 Role As SPINE

Configure DUT2 Role As SPINE

Configure DUT3 As TOR With

[Link]/24
Configure DUT4 As TOR With
[Link]/24
Configure DUT5 As TOR With
[Link]/24

24
ODL Controller - Topology

25
Open Networking Ecosystem
 Edge-Core
3rd party  Hardware
 Preloaded Switch OS
Software  SDN Controller
Vendors  Applications
 3rd Party Software Partners
 Switch OS
 SDN Controllers
Open  Applications
Source Edge-Core  Open Source Community
Community  SDN Controller
 Open Source SW “Distribution”
 Applications