Showing posts with label Spanning Tree. Show all posts
Showing posts with label Spanning Tree. Show all posts

Monday, November 7, 2016

Advanced Cisco Configuration: Access Ports, Trunk Ports and Native VLANs

I ran into a very interesting problem today that, in hindsight, was rather obvious. Suppose we have a router with a switchport module in one of its open slots connected to a managed switch. Let's use FastEthernet0/15 for the switchport module interface connected to FastEthernet0/0 on the standalone switch. Fa0/15 on the router's switchport module is configured as an access port on some particular VLAN, say VLAN 10, and Fa0/0 on the standalone switch is configured as a trunk port on some different VLAN, say 238. The problem came to my attention because the log files on the router were filling up with errors along the lines of this:
01:48:19: %CDP-4-NATIVE_VLAN_MISMATCH: Native VLAN mismatch discovered on FastEthernet0/15...

At first glance, I was somewhat confused by this because the CDP error message suggested that the Native VLAN's on the two switchports didn't match (the standalone switch had the native VLAN left at its default, or VLAN 1). I didn't have access to the switch, so I couldn't look at its configuration, but I knew that a trunk port on the switch could NOT communicate with the access port on the router's switch module. What, then, were the techs who originally set this up trying to accomplish? With the VLAN mismatch, there shouldn't have been any traffic flowing from the switch to the router, right?

Well, no.

After mocking things up in GNS3 as well as in hardware with a spare 3825 and a Catalyst 2950 at my desk, I was able to figure out what was actually happening. In short, the managed switch was sending Ethernet frames with a VLAN tag out Fa0/0, which the router was dutifully discarding, since it was expecting untagged frames. However, any traffic on VLAN 1 -- the default native VLAN -- on the switch was being sent out of Fa0/0 as untagged frames...which the router happily accepted as belonging to VLAN 10 when they entered Fa0/15. CDP wasn't happy about this, since it could tell that the native VLAN on the managed switch wasn't the same as the VLAN configured on router's switchport, but since configuring a port as an access port means, "accept any untagged Ethernet frames coming in to this port, and place them in VLAN 10," traffic was still flowing between the switch and the router.

Needless to say, this is bad practice and is very insecure. If you don't match up the native VLAN's between Layer-2 devices, it is very easy to allow traffic across your network that you didn't intend. Even worse, VLAN 1 is the default management VLAN on Cisco switches, so essentially, you are allowing traffic that is supposed to be in one VLAN into the management VLAN on the standalone switch (!). This is why CDP raises an alarm about misconfigurations such as this: in all likelihood, something is happening that you did not intend. In fact, when I mocked this up on a pair of 3640 routers with NM16-ESW switch modules in GNS3, spanning-tree on the 3640 disabled the port until I disabled spanning tree on the affected VLAN:
00:21:54: %SPANTREE-7-RECV_1Q_NON_TRUNK: Received 802.1Q BPDU on non trunk FastEthernet0/15 VLAN106.
00:21:54: %SPANTREE-7-BLOCK_PORT_TYPE: Blocking FastEthernet0/15 on VLAN106. Inconsistent port type.
R1(config)#no spanning-tree vlan 106

Interestingly enough, the router didn't complain when I set up the connected interfaces as access ports in different VLAN's -- it was only once the router received an Ethernet frame with an 802.1Q VLAN tag that spanning-tree disabled the port on me. Even more interesting, the physical Cisco 3825 router with a 16-port switch module and 2950 switch did not disable any ports with a similar configuration (perhaps Spanning-Tree was not running on the 3825?).

Tuesday, November 12, 2013

JNCIA Lesson 6 -- Spanning Tree, Rapid Spanning Tree and Multiple Spanning Tree

In our network design, we have two EX3200 switches at the main office, connected through a single copper cross-over cable. A savvy network administrator would probably look at our design and ask, "What would happen if either Ethernet port, or more likely, the Ethernet cable itself were to fail?" The link from our main3200 switch to our main2-3200 switch is indeed a single point of failure on our network. Many a junior network admin have had the sudden realization that such a design is a problem waiting to happen, and in their eagerness to show how pro-active and fore-sighted they are, have then run a second Ethernet cable between the two switches in order to create a redundant connection between them.

If the junior admin was lucky, (s)he probably just spent a while wondering why one of the redundant Ethernet ports suddenly went off-line (had a customer do that to me once, on a Metro-Ethernet ring, in fact). If the junior admin was not so lucky, however, (s)he created a broadcast storm, crashing the entire Layer-2 network with (her)his well-intentioned, but naive, blunder:



In actual practice, the diagrams above are an over-simplification: if Switch A was actually sending a broadcast, it would have forwarded the frame out the ports to *both* Switch B and Switch C, and both Switch B and Switch C would have forwarded the broadcast frame to each other, resulting in multiple copies of the same frame circulating through the network simultaneously. In fact, if there are four or more switches in the network connected in a full-mesh network, then with every cycle there are more and more frames circulating through the network until the switches can no longer keep up with traffic (with three or less switches, the number of frames circulating won't grow, since the switches will not forward a broadcast back out the interface upon which it was received).

In our example with the junior admin, (her)his instincts were good; the network, as originally designed, was vulnerable to failure, and it would be a good idea to design a redundant path between switches, if there were a way to prevent such loops from occurring. Fortunately, this is exactly the type of problem that the Spanning Tree protocol (STP), and it's later refinements, Rapid Spanning Tree (RSTP) and Muliple Spanning Tree (MSTP), were designed to solve.

The method through with STP prevents loops is quite simple: essentially, the protocol just shuts down redundant network ports until/unless the switches in the tree detect a failure on an active port. In our example above, and assuming that Switch A was the "root bridge" (we'll define that term shortly), Spanning Tree (or RSTP or MSTP) would put one of the ports on either Switch B or Switch C into a "blocking" state, thus breaking the loop. Now, if Switch A sends a broadcast frame to Switches B and C, they will each forward the packet to devices connected to their switch ports, but NOT to each other.

This (simplified) explanation of Spanning Tree raises an important question: how do the switches determine if there is a loop, or if there is a network port which needs to be shut down? When a switch (or group of switches) is first powered on, it sends a query out each of its enabled switch ports, called a "Bridge Protocol Data Unit." Routers and PCs ignore the BPDU, but other switches examine the BPDU to check for loops in the layer-2 topology. Some of the data transferred during this process is used to elect a "root bridge" -- the switch that will be the pinnacle of the spanning tree network. As we've already stated, spanning tree will shut down ports to break loops in the layer-2 network, but the root bridge is special: all enabled ports on the root bridge always remain active. On any switches with a single port connected to the root bridge, those ports will also always remain active, and are known as "root ports." If there are multiple ports to the root bridge, the switch will select a single port as a root port, and shut down the rest. Any ports that spanning tree uses to reach other switches, but which do not provide a path for that switch to reach the root bridge will are called "designated ports." Confused yet? Let's look a drawing to try to make it a little clearer:



As you can see, all of the switch ports that are directly connected to Switch A are root ports, all of the switch ports in a forwarding state that are NOT directly connected to Switch A are designated ports, and the ports in a blocking state are non-designated ports.

One other question that might come to mind at this point is, "how do the switches determine which switch is the root bridge?" Spanning Tree uses two metrics for electing a root bridge: first, the admin can set a value known as "root bridge priority" when configuring spanning tree, and lowest priority wins; second, if two or more switches have the same priority, then the switch with the lowest MAC address wins. In JunOS, the root bridge priority and MAC address are concatenated into a single value known as the "root ID" which can be shown with the "show spanning-tree bridge" command:

root@main3200# run show spanning-tree bridge

STP bridge parameters
Context ID                          : 0
Enabled protocol                    : RSTP
<...snip...>
  Local parameters
    Bridge ID                       : 32768.3c:8a:b0:99:df:c1
<...snip...>
root@main3200#


As you can see, the root ID on this switch is 32768.3c:8a:b0:99:df:c1. Suppose we had two other switches in a full-mesh, layer-2 network, and the other switches had root ID's of 32768.3c:8a:b0:99:e5:41 and 32768.3c:8a:b0:9a:22:01. Which switch would be elected root bridge? Let's connect these three switches together and find out! I'll connect ge-0/0/23 on the first switch to ge-0/0/22 on the second switch, connect ge-0/0/23 on the second switch to ge-0/0/22 on the third switch, then connect ge-0/0/23 on the third switch to ge-0/0/22 on the first switch. Once the switches are wired up to create a loop at layer-2, we'll re-run the "show spanning-tree bridge" command and see which switch was actually elected to be the root bridge:

root@main3200# run show spanning-tree bridge | match "Root ID"
  Root ID                           : 32768.3c:8a:b0:99:df:c1

[edit]
root@main3200#


root@branch1-3200# run show spanning-tree bridge | match "Root ID"
  Root ID                           : 32768.3c:8a:b0:99:df:c1

[edit]
root@branch1-3200#


root@branch2-3200# run show spanning-tree bridge | match "Root ID"
  Root ID                           : 32768.3c:8a:b0:99:df:c1

[edit]
root@branch2-3200#


Since all three switches have the same priority (JunOS' default of 32768), the election was determined by the lowest MAC address, which belongs to main3200.

If we wanted to know which ports were shut down, we would use the "show spanning-tree interface" command, but here, I have to make an admission: I couldn't actually find enough cross-over cables to connect all three switches in a full-mesh network, so I had to disconnect branch2-3200 for the following portion of the lab:

root@main3200# run show spanning-tree interface

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role
                         port ID        bridge ID          Cost
ge-0/0/22.0    128:535      128:535  32768.3c8ab099dfc1     20000  FWD    DESG
ge-0/0/23.0    128:536      128:536  32768.3c8ab099dfc1     20000  FWD    DESG

[edit]
root@main3200#


root@branch1-3200# run show spanning-tree interface

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role
                         port ID        bridge ID          Cost
ge-0/0/22.0    128:535      128:536  32768.3c8ab099dfc1     20000  BLK    ALT
ge-0/0/23.0    128:536      128:535  32768.3c8ab099dfc1     20000  FWD    ROOT

[edit]
root@branch1-3200#


If you notice, ge-0/0/22 on branch1-3200 is blocked by Spanning Tree, which shows that, when a link between two switches is determined to be redundant, if the path-cost back to the root bridge is equal, the lowest port number will be shut down (however, you can manually assign a port-cost to bias for or against a particular port, if you want).

RSTP is enabled on Juniper EX-series switches by default, but if you need to manually configure Spanning Tree, Rapid Spanning Tree or Multiple Spanning Tree, it's a very easy process. All three protocols are configured in exactly the same way, with just a couple of exceptions, which we'll cover in just a minute. Consequently, we'll only run through the configuration steps once, using RSTP. If you are using STP or MSTP, then rather than using "edit protocols rstp" (or "set protocols rstp ..."), you will use "edit protocols stp" or "edit protocols mstp" as appropriate.

First, we'll set the bridge priority to be lower on branch1-3200, to make it the root bridge instead of main3200:

root@branch1-3200# edit protocols rstp

[edit protocols rstp]
root@branch1-3200# set bridge-priority 20k

[edit protocols rstp]
root@branch1-3200#


Next, we will disable RSTP on ge-0/0/12.0 and commit the config:
root@branch1-3200# set interface ge-0/0/12.0 disable

[edit protocols rstp]
root@branch1-3200# commit
commit complete

[edit protocols rstp]
root@branch1-3200#


Now, one of the ports on main3200 should be in the blocking state, rather than on branch1-3200:

root@main3200# run show spanning-tree interface

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role
                         port ID        bridge ID          Cost
ge-0/0/22.0    128:535      128:536  20480.3c8ab099e541     20000  BLK    ALT  
ge-0/0/23.0    128:536      128:535  20480.3c8ab099e541     20000  FWD    ROOT

[edit protocols rstp]
root@main3200#


...and sure enough, ge-0/0/22.0 on main3200 is now in the blocking state. Let's adjust the priority to put ge-0/0/23 in the blocking state, instead:

root@main3200# set interface ge-0/0/22 cost 1000

[edit protocols rstp]
root@main3200# commit
commit complete

[edit protocols rstp]
root@main3200# run show spanning-tree interface

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role
                         port ID        bridge ID          Cost
ge-0/0/22.0    128:535      128:536  20480.3c8ab099e541      1000  FWD    ROOT
ge-0/0/23.0    128:536      128:535  20480.3c8ab099e541     20000  BLK    ALT  

[edit protocols rstp]
root@main3200#


One of the problems with spanning tree is the time it takes for the network to "converge" -- that is, the time it takes for the switches to elect a root bridge, determine if there are any loops in the topology, and then to shut down any redundant links. In STP, convergence can take almost a full minute, which is at least part of the reason RSTP was created. While RSTP is already quicker to converge than STP, it can converge even quicker if point-to-point links are declared as such with the "set protocols rstp interface ge-0/0/<whatever> mode point-to-point" command (Edit: if you are confused here, you are not alone. I'm not sure what a "point-to-point" link even means in a layer-2 network, but if I find out, I'll notate it here).

MSTP also differs from standard STP configuration in that you must add two more lines to the config:

root@main3200# set protocols mstp configuration-name jncia-example-1

[edit]
root@main3200# set protocols mstp msti 1 vlan [ 1 2 4 ]

[edit]
root@main3200# set protocols mstp msti 2 vlan [ 3 5 6 20 ]

[edit]
root@main3200#
<...etc...>


The only "gotcha" when putting VLANs into an MSTP instance (MSTI) is that every VLAN in a given instance must share a common topology.

Sunday, October 6, 2013

Appendix A -- More on Spanning Tree

Spanning Tree was enabled by default on the 2924 switches I have in my lab, but sometimes the network admin needs to tweak the settings a little. For example, the network admin may want to force a particular switch to be the root bridge -- wouldn't it make sense to have the most powerful switch in your network be the root bridge rather than a small, inexpensive, low-powered access switch in a wire closet in the hut next door to your main building on your campus?

Unfortunately, I have become quite frustrated with the documentation on Spanning Tree in the several CCNA test prep books I have purchased over the years (I started working on a CCNA over eight years ago, but circumstances changed, and I never took the exam). I don't know about you, but for me, I don't retain much by simply reading technical documentation. Since I can't afford to NOT know the correct answers when I take the CCNA exam this coming week, I decided the best way to get the straight scoop was to configure, tweak, and observe Spanning Tree on my switches.

Root Bridge Election: Let's start with manipulating the root bridge election process. This was an area where I got really frustrated with the documentation, so let's just play with it on the switches and see what happens. Here goes:

lab2924a#sho span

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 0003.e3e4.f887
  Configured hello time 2, max age 20, forward delay 15
  We are the root of the spanning tree
  Topology change flag not set, detected flag not set, changes 1
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 13, path cost 0
   Timers: message age 0, forward delay 0, hold 0
   BPDU: sent 1416, received 8

Interface Fa0/2 (port 14) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 14, path cost 0
   Timers: message age 0, forward delay 0, hold 0
   BPDU: sent 1383, received 2


...and lab2924b:

lab2924b#sho span

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 00d0.58dd.d186
  Configured hello time 2, max age 20, forward delay 15
  Current root has priority 32768, address 0003.e3e4.f887
  Root port is 13, cost of root path is 19
  Topology change flag not set, detected flag not set, changes 3
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 13, path cost 0
   Timers: message age 3, forward delay 0, hold 0
   BPDU: sent 39, received 3092

Interface Fa0/2 (port 14) in Spanning tree 1 is BLOCKING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 14, path cost 0
   Timers: message age 6, forward delay 0, hold 0
   BPDU: sent 1, received 2916


The rest of the ports on the two switches are just "leaf nodes" -- that is, they connect to routers and laptops, rather to switches, and therefore, they are incapable of creating loops in the layer-2 network. From the output above, we can see that lab2924a is the root bridge, that fa0/1 and fa0/2 on lab2924a are designated ports, that fa0/1 on lab2924b is a root port and that fa0/2 on lab2924b is a blocking port. What happens if I pull the Ethernet cable on fa0/1 on one of the switches? Let's find out! Here's what I see on lab2924a:

lab2924a#sho span

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 0003.e3e4.f887
  Configured hello time 2, max age 20, forward delay 15
  We are the root of the spanning tree
  Topology change flag set, detected flag set, changes 2
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 30, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is down
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 13, path cost 0
   Timers: message age 0, forward delay 0, hold 0
   BPDU: sent 1734, received 8

Interface Fa0/2 (port 14) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 14, path cost 0
   Timers: message age 0, forward delay 0, hold 0
   BPDU: sent 1705, received 4


For almost a full minute, however, I couldn't see anything from lab2924b (since I was directly connected to 2924a), which underscores a big problem with STP -- the network will recover from failure of a redundant link, but users will definitely notice the outage while spanning tree reconverges. Eventually, however, my telnet session recovered, and here is what I saw on lab2924b:

lab2924b#sho span

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 00d0.58dd.d186
  Configured hello time 2, max age 20, forward delay 15
  Current root has priority 32768, address 0003.e3e4.f887
  Root port is 14, cost of root path is 19
  Topology change flag set, detected flag not set, changes 4
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is down
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 00d0.58dd.d186
   Designated port is 13, path cost 19
   Timers: message age 0, forward delay 0, hold 0
   BPDU: sent 39, received 3474

Interface Fa0/2 (port 14) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 14, path cost 0
   Timers: message age 1, forward delay 0, hold 0
   BPDU: sent 3, received 3340


Now we see that fa0/1 is down on both switches, and fa0/2 is forwarding on both switches once again. No surprises, right?

From the output above, we can see that both switches have a priority of 32768. If both switches have the same priority, then the choice of root bridge is decided by MAC address. lab2924a has MAC 0003.e3e4.f887 and lab2924b has MAC 00d0.58dd.d186. 0x0003 < 0x00d0 so apparently the low MAC address wins the election process when the priority is the same. But what if I wanted lab2924b to be the root bridge? Would I set the STP priority on lab2924b higher or lower than lab2924a? Let's try it and find out:

lab2924b#conf t
lab2924b(config)#spanning-tree priority 4096
lab2924b(config)#exit
lab2924b#sho span

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 4096, address 00d0.58dd.d186
  Configured hello time 2, max age 20, forward delay 15
  We are the root of the spanning tree
  Topology change flag set, detected flag set, changes 6
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 1, topology change 24, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 4096, address 00d0.58dd.d186
   Designated bridge has priority 4096, address 00d0.58dd.d186
   Designated port is 13, path cost 0
   Timers: message age 0, forward delay 0, hold 0
   BPDU: sent 48, received 3744

Interface Fa0/2 (port 14) in Spanning tree 1 is LISTENING
   Port path cost 19, Port priority 128
   Designated root has priority 4096, address 00d0.58dd.d186
   Designated bridge has priority 4096, address 00d0.58dd.d186
   Designated port is 14, path cost 0
   Timers: message age 0, forward delay 3, hold 0
   BPDU: sent 10, received 4070


Wow...that was fast! Since STP takes almost a minute to move the ports to the forwarding state, I didn't expect lab2924b to assume its new role as root bridge quite so quickly, but it did. You can see that fa0/2 is still in the listening state as I ran the "sho span" command, but it already knows that it's the root bridge. Edit: I later reset lab2924b's priority back to 32768, and it seemed to take a few seconds to update its status, so it's not quite instantaneous -- say about 10 or 15 seconds, maybe?

How about lab2924a?

lab2924a#sho span

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 0003.e3e4.f887
  Configured hello time 2, max age 20, forward delay 15
  Current root has priority 4096, address 00d0.58dd.d186
  Root port is 13, cost of root path is 19
  Topology change flag set, detected flag not set, changes 4
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 128
   Designated root has priority 4096, address 00d0.58dd.d186
   Designated bridge has priority 4096, address 00d0.58dd.d186
   Designated port is 13, path cost 0
   Timers: message age 2, forward delay 0, hold 0
   BPDU: sent 1869, received 36

Interface Fa0/2 (port 14) in Spanning tree 1 is BLOCKING
   Port path cost 19, Port priority 128
   Designated root has priority 4096, address 00d0.58dd.d186
   Designated bridge has priority 4096, address 00d0.58dd.d186
   Designated port is 14, path cost 0
   Timers: message age 10, forward delay 0, hold 0
   BPDU: sent 2087, received 30


Yep, it has relinquished its role as root bridge already, and has fa0/2 in a blocking state. So apparently, a low priority and/or a low MAC address makes a switch a root bridge, and the lowest numbered redundant interface on a switch becomes the root port. Easy :)

Here's something even cooler. In our scenario, we have two switches with two redundant links. However, we are only using fa0/1 to carry traffic between the two switches. In effect, while fa0/2 is providing redundancy, it is essentially wasted bandwidth until a failure occurs. Wouldn't it be great if we could distribute the load between these two ports without setting up LAG or Etherchannel? As it turns out, we can.

You see, Spanning Tree runs a separate instance for each VLAN, which is often described as "Per-VLAN Spanning Tree" or "PVST." It turns out that we can tweak the priority of each VLAN on the FastEthernet port on each switch, making fa0/1 the designated/root port for some VLANs and fa0/2 the designated/root port for other VLANs. To keep it simple, I set this up with odd numbered VLANs on fa0/1 and even numbered VLANs on fa0/2. In a real environment, it would probably be better to do some traffic analysis and find out which VLANs typically have equal amounts of traffic and try to balance load intelligently. At any rate, here's how you do it:

lab2924b(config)#int fa0/1
lab2924b(config-if)#spanning-tree vlan 1 port-priority 64
lab2924b(config-if)#spanning-tree vlan 3 port-priority 64
lab2924b(config-if)#spanning-tree vlan 5 port-priority 64
lab2924b(config-if)#spanning-tree vlan 7 port-priority 64
lab2924b(config-if)#int fa0/2
lab2924b(config-if)#spanning-tree vlan 2 port-priority 64
lab2924b(config-if)#spanning-tree vlan 4 port-priority 64
lab2924b(config-if)#spanning-tree vlan 6 port-priority 64
lab2924b(config-if)#exit
lab2924b(config)#exit
lab2924b#sho spanning-tree vlan 2

Spanning tree 2 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 00d0.58dd.d180
  Configured hello time 2, max age 20, forward delay 15
  Current root has priority 32768, address 0003.e3e4.f880
  Root port is 14, cost of root path is 19
  Topology change flag set, detected flag not set, changes 9
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0

Interface Fa0/1 (port 13) in Spanning tree 2 is BLOCKING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f880
   Designated bridge has priority 32768, address 0003.e3e4.f880
   Designated port is 13, path cost 0
   Timers: message age 1, forward delay 0, hold 0
   BPDU: sent 8, received 3274

Interface Fa0/2 (port 14) in Spanning tree 2 is FORWARDING
   Port path cost 19, Port priority 64
   Designated root has priority 32768, address 0003.e3e4.f880
   Designated bridge has priority 32768, address 0003.e3e4.f880
   Designated port is 14, path cost 0
   Timers: message age 2, forward delay 0, hold 0
   BPDU: sent 5, received 3376

lab2924b#sho span vlan 1

Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, address 00d0.58dd.d186
  Configured hello time 2, max age 20, forward delay 15
  Current root has priority 32768, address 0003.e3e4.f887
  Root port is 13, cost of root path is 19
  Topology change flag not set, detected flag not set, changes 7
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0

Interface Fa0/1 (port 13) in Spanning tree 1 is FORWARDING
   Port path cost 19, Port priority 64
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 13, path cost 0
   Timers: message age 2, forward delay 0, hold 0
   BPDU: sent 347, received 6622

Interface Fa0/2 (port 14) in Spanning tree 1 is BLOCKING
   Port path cost 19, Port priority 128
   Designated root has priority 32768, address 0003.e3e4.f887
   Designated bridge has priority 32768, address 0003.e3e4.f887
   Designated port is 14, path cost 0
   Timers: message age 4, forward delay 0, hold 0
   BPDU: sent 309, received 6948


Important! I didn't show it in the config above, but I repeated the same configuration on lab2924a -- this doesn't seem to work if the priority is changed on only one switch!

Notice how fa0/1 is forwarding on VLAN 1, but is blocking on VLAN 2, and vice versa for fa0/2? You are now utilizing both 100M Ethernet ports on your switch while still avoiding loops!

Monday, September 30, 2013

Lesson 10 -- Spanning Tree

As the company has grown, management has hired more employees, and your poor old 2924 switch finally ran out of ports. However, it's worked reasonably well, so you found another one on E-Bay, and installed it in your main office server rack. You connected an Ethernet patch cable from Fa0/1 on the first switch to Fa0/1 on the second switch, then copied the config using the same trick you used on the routers: copy the config to a TFTP server, modify just the lines that were different (like hostname and IP address) and download the new config onto the new hardware. You also hired a well-meaning, but very, very green assistant to help you with the IT work. He's enthusiastic, but...

Anyway, you leave for lunch one day, and when you return, you find your assistant -- we'll call him George for now -- in the server room, laptop connected through a console cable to the switch, and frowning in thought as he scratches his head.

"What's the matter, George?" you ask.

George shrugs, taking a few seconds to compose his thoughts before replying. "While you were at lunch, I received a phone call from the manager of Customer Service, who was asking if there was anything we could do to speed up the Internet for some of the new employees on the new 2924 switch we installed last week. They said access to the file server was kind of sluggish, too. So, I came in here to the server room and connected a second Ethernet cable between the two switches, but it didn't seem to make any difference. I see both ports are up, but I can't figure out why they aren't seeing any improvement in speed*."

You chuckle, then have George run a couple of commands on his laptop:

lab2924b# conf t
lab2924b(config-if)# int fast0/1
lab2924b(config-if)# shut
lab2924b(config-if)# no shut
lab2924b(config-if)# exit
lab2924b(config)# exit
lab2924b# sho span


Spanning tree 1 is executing the IEEE compatible Spanning Tree protocol
Bridge Identifier has priority 32768, address 00d0.58dd.d180
Configured hello time 2, max age 20, forward delay 15
Current root has priority 32768, address 0003.e3e4.f880
Root port is 13, cost of root path is 19
Topology change flag not set, detected flag not set, changes 2
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0


Interface Fa0/1 (port 13) in Spanning tree 1 is LISTENING
Port path cost 19, Port priority 128
Designated root has priority 32768, address 0003.e3e4.f880
Designated bridge has priority 32768, address 0003.e3e4.f880
Designated port is 13, path cost 0
Timers: message age 2, forward delay 0, hold 0
BPDU: sent 4, received 185
^C


George repeats the "sho span" a few times with the same results, then...:

lab2924b# sho span
<...snip...>
Interface Fa0/1 (port 13) in Spanning tree 1 is LEARNING
Port path cost 19, Port priority 128
Designated root has priority 32768, address 0003.e3e4.f880
Designated bridge has priority 32768, address 0003.e3e4.f880
Designated port is 13, path cost 0
Timers: message age 2, forward delay 11, hold 0
BPDU: sent 4, received 187
^C
lab2924b# sho span
<...snip...>
Interface Fa0/1 (port 13) in Spanning tree 1 is FORWARDING
Port path cost 19, Port priority 128
Designated root has priority 32768, address 0003.e3e4.f880
Designated bridge has priority 32768, address 0003.e3e4.f880
Designated port is 13, path cost 0
Timers: message age 1, forward delay 0, hold 0
BPDU: sent 5, received 194


George is thoroughly confused now. "What is with all of the 'listening,' 'learning' and 'forwarding' jazz?"

"Patience, grasshopper, and all will be explained. First run a few more commands..."

lab2924b# sho span int fa0/2
Interface Fa0/2 (port 14) in Spanning tree 1 is BLOCKING
Port path cost 19, Port priority 128
Designated root has priority 32768, address 0003.e3e4.f880
Designated bridge has priority 32768, address 0003.e3e4.f880
Designated port is 14, path cost 0
Timers: message age 1, forward delay 0, hold 0
BPDU: sent 1, received 4
^C


"You see, lucky for you, our switches are running a protocol called 'Spanning Tree.' Spanning tree is a way of making sure there are no loops when you connect multiple switches together. Suppose the switches were forwarding packets through both network interfaces where you've got the network cables connected. What would happen the first time a host on the network sent a Layer-2 broadcast message?"

George furrowed his brows. "I don't know, I guess."

"Think it through. Say my laptop on port 9 sends a broadcast. What does the 2924a switch do with that frame?"

"Oh, that's easy,?" George replied, brightening. "The switch will forward it out every port."

"Including port 1?" you ask.

Again George nods. "Yep."

"Which is connected to the 2924b switch," you add.

"Yep," George agrees again.

"And what does the 2924b switch do with the broadcast frame?"

"Since it's a broadcast, it sends it out every port except the one it received the frame on."

"Including port 2, where you connected the second Ethernet cable between the two switches?" you prompt.

The lightbulb of understanding finally fires in George's brain, and you suppress a laugh -- barely -- as his jaw suddenly falls to the floor. "Ohhhh..."

"Mmmm-hmmm."

"So the broadcast frame just continues to circle repeatedly between both switches, then."

"Exactly. It's called a 'broadcast storm' and, especially on older switches like these 2924's, it can bring your network to a screeching halt. It will easily max out the CPU on the switches so that you can't even log in to fix the problem. Sometimes the only way to stop a broadcast storm is to turn off one of the switches, or start pulling Ethernet cables until you see the switch start responding to telnet or pings again. Here is another command that is helpful for troubleshooting problems with Spanning Tree."

lab2924b# sho span brie
<...snip...>
Port                    Designated
Name Port ID Prio Cost Sts Cost Bridge ID Port ID
------- ------- ---- ---- --- ---- -------------- -------
Fa0/1 128.13 128 19 FWD 0 0003.e3e4.f880 128.13
Fa0/2 128.14 128 19 BLK 0 0003.e3e4.f880 128.14
Fa0/3 128.15 128 19 FWD 19 00d0.58dd.d180 128.15
Fa0/4 128.16 128 19 BLK 19 00d0.58dd.d180 128.16
Fa0/5 128.17 128 19 BLK 19 00d0.58dd.d180 128.17
Fa0/6 128.18 128 19 BLK 19 00d0.58dd.d180 128.18
Fa0/20 128.34 128 19 BLK 19 00d0.58dd.d180 128.34
Fa0/21 128.35 128 19 BLK 19 00d0.58dd.d180 128.35
Fa0/22 128.36 128 19 BLK 19 00d0.58dd.d180 128.36
Fa0/23 128.37 128 19 BLK 19 00d0.58dd.d180 128.37
<...snip...>


"This shows you which ports are forwarding, shown as 'FWD' or blocking, shown as 'BLK.' Now let's see what we can do to troubleshoot that bandwidth problem..."

Post-Script: Lest you think that this is an exaggerated tale, yes, this really happened to me. I was working for a service provider at the time, and a network admin working for one of my customers -- also a service provider -- called at the very end of the business day to request that I turn up a second port on one of my switches in a Metro Ethernet ring. The alarm bells went off in my head, but I foolishly ignored them, and, rather than inquiring about his plans for a second port on the same switch, I simply turned the port up and called it a day. When I returned to work the following day, I had a series of messages in my voice mail, wondering if I had turned the port up, if I had configured it properly, etc. I called my peer at the other service provider back, and started troubleshooting the problem with him. I finally figured out that he was trying to configure a redundant port between his switch and my switch, and really had to suppress a laugh when he admitted that he couldn't figure out why his port was going down when he plugged in his network cable. "Ummm...spanning tree?" I prompted as gently as I could. "Oh..." was his reply.