Simple inter-tenant communication rules with Cisco ACI
In Cisco ACI, there is the notion of tenant. It’s the top logical construct that is housing all underneath tenant components: private networks, bridge domains, subnets and EPG’s. Tenants are by default unable to communicate with each other without involving the external network. This premise is also true for other SDN solutions, such as VMware NSX, where the demarcation between the tenant zone and the physical or external network is the NSX Edge gateway. It's important to keep this segmentation to guarantee security, but sometimes multiple tenant constructs belong to the same entity or customer. In this case, what would be the technical solution to enable communication between two or multiple tenants for a subset of their networks?
There are essentially two approaches to solving this challenge: Use virtual functions or use traditional network semantics to bring common capabilities like route leaking and implement them within this tenancy model.
NFV or network virtualization solution
In this particular case, I'm assuming that overlapping address spaces is not required, or NAT would have to be added into the picture. Let's keep it simple for the moment. When Tenant A wants to talk to Tenant B, there are several management points where multiple security rules and routing configuration need to be changed to allow end-to-end communication. All edge routing and security functions are provided by virtual components within virtual appliances. Unless an intelligent central policy repository is available, it's impossible to guess where changes are required and which devices are involved in Tenant A to Tenant B communications. This is getting worse as the number of networks in each tenant is growing.
In other words we're back to a "box-by-box" management when advanced network functions are required. This is because the control-plane between tenants is not automatically orchestrated and configured as a whole. Each virtual appliance provides a specific service, and has a specific identity on the network, only relevant to a particular tenant. NFV and network virtualization have these limitations, their functions stay "traditional" network functions. They can rely on automation and orchestration to provision and configure certain aspects of the network, but not the entire network. Consequently you can't really expect the same flexibility when it comes to advanced, yet common configuration scenario such as tenant route leaking.
If we take the following picture as an example, making Tenant A talk to Tenant B would require the following actions:
- Change security rules of the FW of Tenant A to allow traffic from Tenant B.
- Change security rules of the FW of Tenant B to allow traffic from Tenant A.
- Change security rules of the Aggregation FW to allow traffic between Tenant A and B.
- Configure a dynamic routing protocol between tenants (static is still possible, but not really manageable at scale).
Leaking between tenants at the container level, the simple approach
In the ideal world, you don't only want SDN solutions to provide automation and overlay networks, you also want traditional constructs such as VRF and ACL to be applicable and translated to this new model. A tenant doesn't necessarily have a mapping to a classical network construct, but what if we could create VRF's within a tenant. This would enable route leaking with the tools everyone already knows. This is how Cisco ACI implements inter-tenant communication, without external resources required. No need for extra x86 capacity or multiple network hop between tenants. This is a core function of ACI, as illustrated below:
The picture highlights some fundamentals ACI constructs and rules:
- Both Tenant A and Tenant B have one VRF (minimum is one VRF per tenant).
- EPG A, B and C belong to Tenant A. An EPG represents a collection of end hosts with similar policy requirements. In a simplistic, network centric approach, this may correspond to a VLAN.
- EPG D, E and F belong to Tenant B.
- EPG's need contracts to talk together. This is because ACI is a zero trust model where you have to explicitly allow communication between EPG's. A contract contains one or more filters, which are 5 tuple ACL's.
- You can export contracts between tenants. This is how we achieve route leaking. Exporting a contract does two things: it exports and imports routes where required and enforce contract security rules. In the above example, C1 and C2 allow SSH communication between EPG A and D, and between EPG B and E, which belong to different tenants.
Here is an output of the configuration after Tenant B contract has been exported to Tenant A
Leaf-1# show ip route vrf Tenant_A:ctx IP Route Table for VRF "Tenant_A:ctx" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '%
' in via output denotes VRF 192.168.10.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 03:06:30, static 192.168.10.1/32, ubest/mbest: 1/0, attached *via 192.168.10.1, vlan93, [1/0], 03:06:30, local, local 192.168.11.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 03:06:30, static 192.168.11.1/32, ubest/mbest: 1/0, attached *via 192.168.11.1, vlan93, [1/0], 03:06:30, local, local 192.168.12.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 00:38:41, static 192.168.13.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 00:38:41, static Leaf-1# show ip route vrf Tenant_B:ctx IP Route Table for VRF "Tenant_B:ctx" '*' denotes best ucast next-hop '**' denotes best mcast next-hop '[x/y]' denotes [preference/metric] '% ' in via output denotes VRF 192.168.10.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 00:13:43, static 192.168.11.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 01:05:46, static 192.168.12.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 00:38:16, static 192.168.12.1/32, ubest/mbest: 1/0, attached *via 192.168.12.1, vlan102, [1/0], 00:47:21, local, local 192.168.13.0/24, ubest/mbest: 1/0, attached, direct, pervasive *via 10.0.136.64%overlay-1, [1/0], 01:04:03, static 192.168.13.1/32, ubest/mbest: 1/0, attached *via 192.168.13.1, vlan102, [1/0], 00:48:12, local, local
You can see now that Tenant A VRF knows the routes to reach subnets 192.168.13.0/24 and 192.168.12.0/24, which corresponds to Tenant B EPG D and EPG E subnets. In a similar way, Tenant B knows the routes to reach subnets 192.168.10.0/24 and 192.168.11.0/24, which are Tenant A EPG A and EPG B subnets. On top of that, we have restricted the communication to SSH just by creating an SSH "rule" in the contract. Although from a pure networking side, all appropriate routes are installed, the policy can be more granular and enforce security rules within the ACI contract. This allows us to manage both network and security aspects within a single abstracted policy. This is a very simple way to create SDN topologies while maintaining flexibility.
The key thing to understand here, is that ACI is implementing a high-level intent language, removing traditional network bottlenecks and shortcomings which may still be present in other SDN solutions. This includes addressing concerns around virtual appliances HA capabilities, performance and operational costs. These edge virtual appliances can completely be removed from the picture, and more importantly the configuration is now driven by policies and deployed in the entire network, regardless of whether it's virtual or physical.