Here are the subnets in use on this network:
| Subnet | Endpoint A | Endpoint A Interface | Endpoint B | Endpoint B Interface |
|---|---|---|---|---|
| 10.254.254.1 | ISP-1 | Lo0 | N/A | N/A |
| 10.254.254.2 | ISP-2 | Lo0 | N/A | N/A |
| 100.64.1.254 | Cust-A | Lo0 | N/A | N/A |
| 100.64.2.254 | Cust-A | Lo0 | N/A | N/A |
| 10.0.0.0/30 | ISP-1 | Gig-E 1/0 | Cust-A | Gig-E 1/0 |
| 10.0.0.4/30 | ISP-1 | Gig-E 2/0 | Cust-B | Gig-E 1/0 |
| 10.0.0.8/30 | ISP-1 | Gig-E 3/0 | ISP-2 | Gig-E 3/0 |
| 10.0.0.12/30 | ISP-2 | Gig-E 1/0 | Cust-B | Gig-E 2/0 |
| 100.64.1.0/26 | Cust-A | Fast-E 0/0 | Knoppix-32 | Eth 1/0 |
| 100.64.2.0/26 | Cust-B | Fast-E 0/0 | CentOS7_1 | Eth 1/0 |
Obviously, we would typically want traffic to flow across the high-speed link rather than the low-speed link. However, BGP doesn't consider bandwidth when determining the "best" path from one host to another:
As you can see, BGP has selected a route via the low-speed circuit from the host Knoppix-32 PC to the CentOS7_1 web server in Cust_B's network. To solve this problem, it's easy enough to set a weight on the outbound link to force traffic to use the circuit connected to ISP-1. All we have to do is set a sufficiently high metric on the route we want to take:
Cust-A
router bgp 65512
neighbor 10.0.0.1 weight 30
Since higher weights take priority over lower weights, this will force outbound traffic to use ISP-1 rather than ISP-2. However, that only has an effect on our outbound traffic. BGP may still provide a route from Cust-B back to us through ISP-2 (the low-bandwidth circuit). This potentially causes two problems: first, we'd rather have our traffic go through the faster circuit (for obvious reasons); and second, this can cause "asymmetric routing." Some applications and network devices (stateful firewalls, for example) really don't like asymmetric routing. Unfortunately, trying to troubleshoot a problem caused by asymmetric routing can be a real PITA, and no, not the tasty kind :( To force other networks to prefer the path via ISP-1, we will adjust BGP's "MED" ("Multi Exit Discriminator"), one of the metrics that BGP uses to calculate the "best" route between endpoints. First, on our router, we'll create an access list to identify our internal networks:
Cust-A(config)#ip access-list standard BGP_Internal_Nets
Cust-A(config-std-nacl)#permit 100.64.1.0 0.0.0.63
Cust-A(config-std-nacl)#permit host 100.64.1.254
Next, we create a route map:
Cust-A(config)#route-map BGP_MED 10
Cust-A(config-route-map)#match ip addr BGP_Internal_Nets
Cust-A(config-route-map)#set metric 110
Finally, we apply the route map to the LESS-PREFERRED neighbor (ISP-2) in our BGP configuration:
Cust-A(config)#router bgp 65512
Cust-A(config-router)#neighbor 172.16.0.1 route-map BGP_MED out
Cust-A(config-router)#exit
Cust-A(config)#exit
Cust-A#clear ip bgp 65511
Unlike weight, a lower MED is preferable to a higher MED, and therefore, by advertising a higher-than-default MED to ISP-2's BGP process, we are effectively telling it to prefer an alternate route to our network.
After BGP re-converges, we should see that both ISP-1 and ISP-2 are using the higher-bandwidth link via ISP-1 to reach 100.64.1.x:
ISP-1#sho ip bgp | inc 65512
* 10.0.0.0/30 10.0.0.2 0 0 65512 i
*> 100.64.1.0/26 10.0.0.2 0 0 65512 i
*> 100.64.1.254/32 10.0.0.2 0 0 65512 i
* 172.16.0.0/30 10.0.0.2 0 0 65512 i
ISP-1#
...and...:
ISP-2(config)#do sho ip bgp | inc 65512
*>i100.64.1.0/26 10.0.0.2 0 100 0 65512 i
* 172.16.0.2 110 0 65512 i
*>i100.64.1.254/32 10.0.0.2 0 100 0 65512 i
* 172.16.0.2 110 0 65512 i
ISP-2(config)#
Perfect! Both routers are now advertising a preferred route via ISP-1, just as we wanted (">" indicates a preferred route). You can verify this by a traceroute from CentOS7_1:
By setting the MED in our BGP config, we have redundant links to our ISP, but will still prefer the high-bandwidth circuit unless there is a problem. I'll leave testing fail-over as an exercise for the reader ;)
