JNCIA Lesson 8 -- Link Aggregation

In the lesson on Spanning Tree, we addressed the issue of redundancy, but what if your users are consuming more bandwidth than a single Gig-E uplink port can provide? STP/RSTP/MSTP allows you to connect multiple links between switches, but only ONE path can be active at any time, so you are still limited to 1Gbps between switches (assuming a Gig-E switch, of course).

Fortunately, there is a standardized protocol, known as "Link Aggregation Control Protocol" (LACP) or sometimes simply "LAG" (Link AGgregation) for creating bundled Ethernet links between switches. That is, in much the same way that a T1 line is a bundle of individual 64kbps lines that have been concatenated together to provide 1.544Mbps of bandwidth, there is also a way to bundle several individual Ethernet channels together to create a single, virtual channel with a bandwidth equal to the sum of the physical Ethernet links in the bundle. For example, if we were to take two Gig-E ports on our EX3200 switches and aggregate them together using LACP, we would have a 2Gbps (2 x 1Gbps) uplink between the switches. Even better, having multiple Ethernet ports in a LAG group provides not only increased bandwidth between switches, but also provides redundancy. If a single channel in the LAG group fails, the LAG bundle continues to carry traffic, albeit at a reduced rate.

Fortunately, it is very simple to create a LAG group between ports on the EX3200 switch, taking only four, easy steps:

1. Create the virtual aggregated interface for the LAG group using the "set chassis aggregated-devices..." command (this will create a new interface, ae0, on the switch);
2. Delete unit 0 in the interfaces that you want to bundle together;
3. Use the "set interfaces...ether-options 802.3ad..." command to associate the physical interfaces that you want to aggregate with the virtual aggregated interface that you created in step 1; and
4. Set the configuration options for the virtual aggregated interface with the "set interfaces...aggregated-ether-options..." and "set interfaces...family ethernet-switching" commands.

Let's set up a LAG group between ge-0/0/22 and ge-0/0/23 on two of our Juniper switches:

root@main3200# set chassis aggregated-devices ethernet device-count 1

[edit]
root@main3200# delete interfaces ge-0/0/22 unit 0

[edit]
root@main3200# delete interfaces ge-0/0/23 unit 0

[edit]
root@main3200# set interfaces ge-0/0/22 ether-options 802.3ad ae0

[edit]
root@main3200# set interfaces ge-0/0/23 ether-options 802.3ad ae0

[edit]
root@main3200# set interfaces ae0 aggregated-ether-options lacp active

[edit]
root@main3200# set interfaces ae0 unit 0 family ethernet-switching

[edit]
root@main3200# commit
commit complete

[edit]
root@main3200#
You now have a working LAG bundle between the switches, but if you try to ping from a host connected to one switch to a host connected on the other, you will find your pings failing with a "destination unreachable" error. Since you have multiple VLANs on the two switches, you will need to tag the VLANs on the aggregated Ethernet port, ae0, before hosts on the two switches can communicate with each other:

root@main3200# edit interfaces ae0 unit 0 family ethernet-switching

[edit interfaces ae0 unit 0 family ethernet-switching]
root@main3200# set port-mode trunk

[edit interfaces ae0 unit 0 family ethernet-switching]
root@main3200# set vlan members 20

[edit interfaces ae0 unit 0 family ethernet-switching]
root@main3200# set native-vlan-id default

[edit interfaces ae0 unit 0 family ethernet-switching]
root@main3200# commit
commit complete

[edit interfaces ae0 unit 0 family ethernet-switching]
root@main3200#


Now, connect a pair of Ethernet cables between the switches -- one from ge-0/0/22 on one switch to ge-0/0/22 on the second switch, and likewise for ge-0/0/23 -- and let's see if we can ping from a host on one switch to a host on the second switch:

me@myllt:~$ ping -c4 192.168.10.8
PING 192.168.10.8 (192.168.10.8) 56(84) bytes of data.
64 bytes from 192.168.10.8: icmp_req=1 ttl=64 time=0.795 ms
64 bytes from 192.168.10.8: icmp_req=2 ttl=64 time=0.502 ms
64 bytes from 192.168.10.8: icmp_req=3 ttl=64 time=0.519 ms
64 bytes from 192.168.10.8: icmp_req=4 ttl=64 time=0.501 ms

--- 192.168.10.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.501/0.579/0.795/0.125 ms
me@myllt:~$


Sweet! We have connectivity :) Now, let's see what happens when we pull an Ethernet cable:

me@myllt:~$ ping 192.168.10.8
PING 192.168.10.8 (192.168.10.8) 56(84) bytes of data.
64 bytes from 192.168.10.8: icmp_req=1 ttl=64 time=0.948 ms
64 bytes from 192.168.10.8: icmp_req=2 ttl=64 time=0.312 ms
64 bytes from 192.168.10.8: icmp_req=3 ttl=64 time=0.484 ms
64 bytes from 192.168.10.8: icmp_req=4 ttl=64 time=0.500 ms
64 bytes from 192.168.10.8: icmp_req=5 ttl=64 time=0.493 ms
64 bytes from 192.168.10.8: icmp_req=6 ttl=64 time=0.492 ms
64 bytes from 192.168.10.8: icmp_req=7 ttl=64 time=0.506 ms
64 bytes from 192.168.10.8: icmp_req=9 ttl=64 time=0.496 ms
64 bytes from 192.168.10.8: icmp_req=10 ttl=64 time=0.489 ms
64 bytes from 192.168.10.8: icmp_req=11 ttl=64 time=0.492 ms
64 bytes from 192.168.10.8: icmp_req=12 ttl=64 time=0.493 ms
64 bytes from 192.168.10.8: icmp_req=13 ttl=64 time=0.481 ms
64 bytes from 192.168.10.8: icmp_req=14 ttl=64 time=0.492 ms
64 bytes from 192.168.10.8: icmp_req=15 ttl=64 time=0.505 ms
^C
--- 192.168.10.8 ping statistics ---
15 packets transmitted, 14 received, 6% packet loss, time 13998ms
rtt min/avg/max/mdev = 0.312/0.513/0.948/0.129 ms
me@myllt:~$


Cool -- we only dropped one packet while the Junipers rerouted all of the traffic onto the good Ethernet link, and unlike STP/RSTP/MSTP, when both links are active, we can use all of the available bandwidth, rather than just a single 1Gbps link at a time.

Lesson 11 -- Etherchannel, Link Aggregation and What Other Vendors Mean by Trunking

After spending some time verifying Ethernet interface status, watching packet counters on Ethernet interfaces, and so on, you and George finally realize that the reason some of the users at the main office are complaining about poor performance on the network is because the link between the two 2924XL switches is over capacity. These are only 10/100M switches, and with all of the servers attached to the first switch, users on the second switch are simply flooding the link between the two switches. There are several ways to solve the problem, of course. For example, you could buy newer, more capable switches. This would probably be a good idea for several reasons, not the least of which is that a 2924 is well beyond obsolete. However, IT does not have enough money in the budget to upgrade the network right now, so you will need to find another solution. The second option would be to move some of the servers to ports on the second switch to help balance the load between the two switches. However, some of the servers are seeing a heavier load than some of the others. For example, your Samba file and print server is running at a very consistent, very heavy load. On the other hand, the DNS servers, while serving a lot of incoming requests and replies, are not transferring nearly as much data as the file server. Consequently, while distributing servers across the two switches would probably help, it probably wouldn't completely resolve the problem.

However, there is an industry-standard way of combining two physical ports on a switch (or router) to create a single logical port. This is called "Link Aggregation," often shortened to "LAG" or "LAG groups." This is sometimes also called "bonding," and, at least in Brocade's world, "trunking" (I just discovered the other day that Juniper also uses Cisco's terminology and calls VLAN tagging, "trunking", sigh...) You can also combine even more ports, if need be, to create even greater capacity. Even better, this also provides redundancy between the switches. If one port fails, or a network cable is disconnected, the LAG group will remain up, since there is still a physical connection between the two switches.

We'll combine two physical 100Mbps ports to create a single, 200Mbps logical port, and see if that solves the problem. Here's how you do it:

On the second switch:

lab2924b# conf t
lab2924b(config-if)# int fa0/2
lab2924b(config-if)# shut
lab2924b(config-if)# no spanning-tree
lab2924b(config)# int fa0/1
lab2924b(config-if)# no span
lab2924b(config)# int fa0/1
lab2924b(config-if)# port group 1 distribution destination
lab2924b(config-if)# int fa0/2
lab2924b(config-if)# port group 1 distribution dest
lab2924b(config-if)# no shut
lab2924b(config-if)# exit
lab2924b(config)# exit


...then do the same thing on the first switch:

lab2924a# conf t
lab2924a(config)# int fa0/2
lab2924a(config-if)# shut
lab2924a(config-if)# no span
lab2924a(config)# int fa0/1
lab2924a(config-if)# no span
lab2924a(config)# int fa0/1
lab2924a(config-if)# port group 1 distribution destination
lab2924a(config-if)# int fa0/2
lab2924a(config-if)# port group 1 distribution destination
lab2924a(config-if)# no shut
lab2924a(config-if)# exit
lab2924a(config)# exit


Notice how turning off spanning tree kicks you out of interface configuration mode? Yeah...I edited out the part where I tried to enter additional configuration commands after not realizing I was no longer in interface configuration mode :)

Now, let's verify that we are trunking -- excuse me, bonding -- ports properly:

lab2924b# sho port group 1
Group  Interface              Transmit Distribution
-----  ---------------------  ---------------------
    1  FastEthernet0/1        destination address
    1  FastEthernet0/2        destination address
lab2924b# sho etherchannel summary
Flags:  d - default     D - down
        I - in use

Group Ports
----- -----
1     Fa0/1(Id) Fa0/2(I)
lab2924b#


...and on the second switch:

lab2924a# sho port group 1
Group  Interface              Transmit Distribution
-----  ---------------------  ---------------------
    1  FastEthernet0/1        destination address
    1  FastEthernet0/2        destination address
lab2924a# sho etherchannel summary
Flags:  d - default     D - down
        I - in use

Group Ports
----- -----
1     Fa0/1(Id) Fa0/2(I)
lab2924a#


Note: As I mentioned at the top of this post, Cisco 2924XL switches are very old. In fact, not only are they obsolete, the model that replaced the 2924 is also obsolete. I've learned a lot of useful things with the two 2924 switches that I am using to study for my CCNA, but quite honestly, if you want to pass the current CCNA exam, you need to learn a completely different technique for creating Etherchannel/LACP bonds between switches than the one presented here. I could repeat the lab above with what I have read are the proper commands for modern switches, but there's another guy who already did a great job of explaining how to configure LAG, so I really don't see the point. Just follow this link for a really, really great write-up.