In our network design, we have two EX3200 switches at the main office, connected through a single copper cross-over cable. A savvy network administrator would probably look at our design and ask, "What would happen if either Ethernet port, or more likely, the Ethernet cable itself were to fail?" The link from our main3200 switch to our main2-3200 switch is indeed a
single point of failure on our network. Many a junior network admin have had the sudden realization that such a design is a problem waiting to happen, and in their eagerness to show how pro-active and fore-sighted they are, have then run a second Ethernet cable between the two switches in order to create a redundant connection between them.
If the junior admin was lucky, (s)he probably just spent a while wondering why one of the redundant Ethernet ports suddenly went off-line (had a customer do that to me once, on a Metro-Ethernet ring, in fact). If the junior admin was
not so lucky, however, (s)he created a
broadcast storm, crashing the entire Layer-2 network with (her)his well-intentioned, but naive, blunder:
In actual practice, the diagrams above are an over-simplification: if Switch A was actually sending a broadcast, it would have forwarded the frame out the ports to *both* Switch B and Switch C, and both Switch B and Switch C would have forwarded the broadcast frame to each other, resulting in multiple copies of the same frame circulating through the network simultaneously. In fact, if there are four or more switches in the network connected in a
full-mesh network, then with every cycle there are more and more frames circulating through the network until the switches can no longer keep up with traffic (with three or less switches, the number of frames circulating won't grow, since the switches will not forward a broadcast back out the interface upon which it was received).
In our example with the junior admin, (her)his instincts were good; the network, as originally designed,
was vulnerable to failure, and it would be a good idea to design a redundant path between switches, if there were a way to prevent such loops from occurring. Fortunately, this is exactly the type of problem that the Spanning Tree protocol (STP), and it's later refinements, Rapid Spanning Tree (RSTP) and Muliple Spanning Tree (MSTP), were designed to solve.
The method through with STP prevents loops is quite simple: essentially, the protocol just shuts down redundant network ports until/unless the switches in the tree detect a failure on an active port. In our example above, and assuming that Switch A was the "root bridge" (we'll define that term shortly), Spanning Tree (or RSTP or MSTP) would put one of the ports on either Switch B or Switch C into a "blocking" state, thus breaking the loop. Now, if Switch A sends a broadcast frame to Switches B and C, they will each forward the packet to devices connected to their switch ports, but
NOT to each other.
This (simplified) explanation of Spanning Tree raises an important question: how do the switches determine if there is a loop, or if there is a network port which needs to be shut down? When a switch (or group of switches) is first powered on, it sends a query out each of its enabled switch ports, called a "Bridge Protocol Data Unit." Routers and PCs ignore the BPDU, but other switches examine the BPDU to check for loops in the layer-2 topology. Some of the data transferred during this process is used to elect a "root bridge" -- the switch that will be the pinnacle of the spanning tree network. As we've already stated, spanning tree will shut down ports to break loops in the layer-2 network, but the root bridge is special: all enabled ports on the root bridge
always remain active. On any switches with a
single port connected to the root bridge, those ports will also always remain active, and are known as "root ports." If there are multiple ports to the root bridge, the switch will select a single port as a root port, and shut down the rest. Any ports that spanning tree uses to reach other switches, but which do not provide a path for that switch to reach the root bridge will are called "designated ports." Confused yet? Let's look a drawing to try to make it a little clearer:
As you can see, all of the switch ports that are directly connected to Switch A are root ports, all of the switch ports in a forwarding state that are
NOT directly connected to Switch A are designated ports, and the ports in a blocking state are non-designated ports.
One other question that might come to mind at this point is, "how do the switches determine which switch is the root bridge?" Spanning Tree uses two metrics for electing a root bridge: first, the admin can set a value known as "root bridge priority" when configuring spanning tree, and lowest priority wins; second, if two or more switches have the same priority, then the switch with the lowest MAC address wins. In JunOS, the root bridge priority and MAC address are concatenated into a single value known as the "root ID" which can be shown with the "show spanning-tree bridge" command:
root@main3200# run show spanning-tree bridge
STP bridge parameters
Context ID : 0
Enabled protocol : RSTP
<...snip...>
Local parameters
Bridge ID : 32768.3c:8a:b0:99:df:c1
<...snip...>
root@main3200#
As you can see, the root ID on this switch is 32768.3c:8a:b0:99:df:c1. Suppose we had two other switches in a full-mesh, layer-2 network, and the other switches had root ID's of 32768.3c:8a:b0:99:e5:41 and 32768.3c:8a:b0:9a:22:01. Which switch would be elected root bridge? Let's connect these three switches together and find out! I'll connect ge-0/0/23 on the first switch to ge-0/0/22 on the second switch, connect ge-0/0/23 on the second switch to ge-0/0/22 on the third switch, then connect ge-0/0/23 on the third switch to ge-0/0/22 on the first switch. Once the switches are wired up to create a loop at layer-2, we'll re-run the "show spanning-tree bridge" command and see which switch was actually elected to be the root bridge:
root@main3200# run show spanning-tree bridge | match "Root ID"
Root ID : 32768.3c:8a:b0:99:df:c1
[edit]
root@main3200#
root@branch1-3200# run show spanning-tree bridge | match "Root ID"
Root ID : 32768.3c:8a:b0:99:df:c1
[edit]
root@branch1-3200#
root@branch2-3200# run show spanning-tree bridge | match "Root ID"
Root ID : 32768.3c:8a:b0:99:df:c1
[edit]
root@branch2-3200#
Since all three switches have the same priority (JunOS' default of 32768), the election was determined by the lowest MAC address, which belongs to main3200.
If we wanted to know which ports were shut down, we would use the "show spanning-tree interface" command, but here, I have to make an admission: I couldn't actually find enough cross-over cables to connect all three switches in a full-mesh network, so I had to disconnect branch2-3200 for the following portion of the lab:
root@main3200# run show spanning-tree interface
Spanning tree interface parameters for instance 0
Interface Port ID Designated Designated Port State Role
port ID bridge ID Cost
ge-0/0/22.0 128:535 128:535 32768.3c8ab099dfc1 20000 FWD DESG
ge-0/0/23.0 128:536 128:536 32768.3c8ab099dfc1 20000 FWD DESG
[edit]
root@main3200#
root@branch1-3200# run show spanning-tree interface
Spanning tree interface parameters for instance 0
Interface Port ID Designated Designated Port State Role
port ID bridge ID Cost
ge-0/0/22.0 128:535 128:536 32768.3c8ab099dfc1 20000 BLK ALT
ge-0/0/23.0 128:536 128:535 32768.3c8ab099dfc1 20000 FWD ROOT
[edit]
root@branch1-3200#
If you notice, ge-0/0/22 on branch1-3200 is blocked by Spanning Tree, which shows that, when a link between two switches is determined to be redundant, if the path-cost back to the root bridge is equal, the lowest port number will be shut down (however, you can manually assign a port-cost to bias for or against a particular port, if you want).
RSTP is enabled on Juniper EX-series switches by default, but if you need to manually configure Spanning Tree, Rapid Spanning Tree or Multiple Spanning Tree, it's a very easy process. All three protocols are configured in exactly the same way, with just a couple of exceptions, which we'll cover in just a minute. Consequently, we'll only run through the configuration steps once, using RSTP. If you are using STP or MSTP, then rather than using "edit protocols rstp" (or "set protocols rstp ..."), you will use "edit protocols stp" or "edit protocols mstp" as appropriate.
First, we'll set the bridge priority to be lower on branch1-3200, to make it the root bridge instead of main3200:
root@branch1-3200# edit protocols rstp
[edit protocols rstp]
root@branch1-3200# set bridge-priority 20k
[edit protocols rstp]
root@branch1-3200#
Next, we will disable RSTP on ge-0/0/12.0 and commit the config:
root@branch1-3200# set interface ge-0/0/12.0 disable
[edit protocols rstp]
root@branch1-3200# commit
commit complete
[edit protocols rstp]
root@branch1-3200#
Now, one of the ports on main3200 should be in the blocking state, rather than on branch1-3200:
root@main3200# run show spanning-tree interface
Spanning tree interface parameters for instance 0
Interface Port ID Designated Designated Port State Role
port ID bridge ID Cost
ge-0/0/22.0 128:535 128:536 20480.3c8ab099e541 20000 BLK ALT
ge-0/0/23.0 128:536 128:535 20480.3c8ab099e541 20000 FWD ROOT
[edit protocols rstp]
root@main3200#
...and sure enough, ge-0/0/22.0 on main3200 is now in the blocking state. Let's adjust the priority to put ge-0/0/23 in the blocking state, instead:
root@main3200# set interface ge-0/0/22 cost 1000
[edit protocols rstp]
root@main3200# commit
commit complete
[edit protocols rstp]
root@main3200# run show spanning-tree interface
Spanning tree interface parameters for instance 0
Interface Port ID Designated Designated Port State Role
port ID bridge ID Cost
ge-0/0/22.0 128:535 128:536 20480.3c8ab099e541 1000 FWD ROOT
ge-0/0/23.0 128:536 128:535 20480.3c8ab099e541 20000 BLK ALT
[edit protocols rstp]
root@main3200#
One of the problems with spanning tree is the time it takes for the network to "converge" -- that is, the time it takes for the switches to elect a root bridge, determine if there are any loops in the topology, and then to shut down any redundant links. In STP, convergence can take almost a full minute, which is at least part of the reason RSTP was created. While RSTP is already quicker to converge than STP, it can converge even quicker if point-to-point links are declared as such with the "set protocols rstp interface ge-0/0/<whatever> mode point-to-point" command (
Edit: if you are confused here, you are not alone. I'm not sure what a "point-to-point" link even means in a layer-2 network, but if I find out, I'll notate it here).
MSTP also differs from standard STP configuration in that you must add two more lines to the config:
root@main3200# set protocols mstp configuration-name jncia-example-1
[edit]
root@main3200# set protocols mstp msti 1 vlan [ 1 2 4 ]
[edit]
root@main3200# set protocols mstp msti 2 vlan [ 3 5 6 20 ]
[edit]
root@main3200#
<...etc...>
The only "gotcha" when putting VLANs into an MSTP instance (MSTI) is that every VLAN in a given instance must share a common topology.
