15454 Multishelf Networking Question

[jnlawrence76]

I have a concern with the way a vendor setup the network for our two 15454's.  Unfortunately this vendor is acting like a car dealer and doesn't want to answer/respond to our questions about the setup.  Below is my concern/question that I sent to our vendor to no avail and hoping someone on here can help out.  I have also attached a simple diagram of the physical setup:

I have put a diagram together of the current physical setup and it shows the Travis ONS is going from its Slot 11 to Switch 2 Port 0/2 (VLAN2) and Slot 7 is going to Switch 1 port 0/2 (VLAN2).  Both ports which are on VLAN 2.  Then you have the Main ONS Slot 7 going to Switch 1 port 0/1(VLAN1) and Slot 11 going to Switch 2 port 0/1(VLAN1) which is a trunking port with no native VLAN set so it defaults to VLAN1.  Not sure why this port is setup to trunk unless it was meant to be used to connect to our network and the Slot 11 and 7 of the Main ONS was supposed to plug into another port.

So right now I see that the Main ONS is only talking on VLAN 1 and the Travis ONS is only talking on VLAN 2.  Is this how it should be setup?  Also, does there have to be multiple VLANS as of now I don't see why we would need to have two VLANs setup.  We normally use VLAN 2 for management purposes and would like to continue to do that.  And we have heard talk of a heartbeat between the two ONS.  What VLAN/Port is the heartbeat taking place on if there is actually one occurring?

Thanks in advance

[aareed]

Based on your description it appears that you have questions about a 15454 MSTP system in Multi-Shelf Mode, using a pair of external switches to provided communication between the Primary Shelf (Node Controller) and the Secondary Shelf (Shelf Controller).

Looking at your diagram, it appears to be configured correctly.

The Multi-shelf configuration uses VLANS 1 & 2 for communication.

VLAN 1 provides communication external to the shelf and VLAN 2 is for communication between the shelves.

Port 1 on your switch should be connected to your Primary Shelf/Node Controller TCC2 cards (slot 7 and 11 for each respective switch).  These ports should be setup for trunking to handle VLANS 1 & 2,

Port 22 provides inter-switch communication and should also be setup for trunking VLANS 2 & 2.

Ports 2-9 should be setup with VLAN 2 only and will be connected to your secondary shelves (Travis).

There should also be a port setup with VLAN 1 only that provides your DCN access external to the switch which should connnect to your LAN.

More details on the configuration can be found in the user documentation:

http://www.cisco.com/en/US/docs/optical/hardware/15454install/guide/hig_15454.html#wp547312

[jnlawrence76]

The Travis ONS is not considered a secondary/backup ONS. Both are active to each respective building that they are named after.  With that said, I would think that Travis and Main should both have a port plugged into VLAN 1 and VLAN 2 rather than Main having both ports plugged into VLAN 1 and Travis plugged into VLAN 2.  Am I just not understanding how the ONS works?  As you can see in the diagram Slot 7 on both Travis and Main are Active and Slot 11 on both Travis and Main are in Standby.

You will have to excuse my lack of knowledge on the ONS.

[aareed]

Jeremy,

If your shelf is setup as a Multi-Shelf, which the description you have provided indicates, then the TRAVIS shelf is a Secondary shelf with the MAIN acting as the Node Controller.  This doesn't mean that one shelf is redundant of the other, but rather that the two shelves are consolidated as a single node.

Typically this would occur if you have your mux/demux components on the primary shelf and transponders on additional shelves, or if you have a split node in which you have the "sides" of the MSTP network physically separated into two different units.

If this is not the case, then can you explain the functionality of the MAIN and TRAVIS shelves and attach inventories of each shelf.

Also, since you have them tied together via ethernet, I am assuming that they are within close proximity of each other.

Please describe your total network.  Is it a DWDM (MSTP) netork or is it a TDM (SONET/SDH).

What is your understanding of why the two shelves are interconnected using this external switch.  I am assuming this is an MSTP Multi-shelf configuration, but if it is not, then I need to understand your network a little better in order to advise you.

Aaron Reed

Carrier Escalation - Cisco Optical TAC

[jnlawrence76]

Aaron,

Thanks for your feedback.  After talking to a few people I was told this would show up as one single Node.  My question now is what is the purpose of VLAN 2?  We normally use VLAN 2 as a management VLAN.  When I have tried to put the ports that are using VLAN 2 on another VLAN I get a critical error on one of the two ONS's.  Is it possible to change the ports to another VLAN or does the ONS's have to see these ports on VLAN 2?

[aareed]

Jeremy,

Depending on what version of software you are running and if you are only using the 15454 M12 chassis (17 slot) and not the M2 or M6 Chassis, you can select which VLANs to use when you configure the MSTP shelf for mult-shelf.

See the following 9.2 MSTP Documentation:

http://www.cisco.com/en/US/docs/optical/15000r9_2/dwdm/procedure/guide/454d92_turnupnode.html#wp398430

Step 1 Complete the "DLP-G46 Log into CTC" task on page 3-30 at the node that you want to configure as a multishelf node.

Step 2 If you want to set up the login node as the node controller, complete the following steps. If not, continue with Step 3.

a. In node view (single-node mode) or multishelf view (multishelf mode), click the Provisioning > General > Multishelf Config tabs.

b. Click Enable as Node Controller.

c. From the LAN Config drop-down list, complete one of the following:

•Choose Ethernet Switch if MS-ISC-100T cards or the Catalyst 2950 or the Catalyst 3560 switches are already installed and configured. Choose the public VLAN ID and private VLAN ID for the ONS 15454 multishelf node. In case of an ONS 15454 M6, the public VLAN ID and private VLAN ID are static (1 and 2 respectively).

---

Note Public VLAN ID is used by the node controller to communicate with the external network. Private VLAN ID is used by the node controller to communicate with the subtending shelves.

---

The default is VLAN IDs 1 & 2 and it is assumed that you are using dedicated switches for the Multi-shelf configuration.  If you are using the switches for other purposes, then it is assumed that you are not using VLANs 1 or 2.

If you are not using the M2 or M6 chassis, you should be able to go back and reprovision your multi-shelf configuration in CTC to use different VLANs, and of course make the corresponding changes on your switches as well.

Aaron Reed

Carrier Escalation - Cisco Optical TAC

[jnlawrence76]

Aaron,

I found we are running version 9.00-008I-17.17.  With that said, could we change the VLANs we are using on this version?  It seems as soon as I change the ports that are on VLAN two, the ONS goes into a Major warning state so it seems that it must see that those ports are on VLAN 2, however I have no idea where I can change which VLAN it's looking for (if that is even possible) on the ONS.

Currently we are using two older 2950's but would perfer to just plug into our two 6500 core switches but still have the concern about the VLANs as we want all management of the ONS to be done on VLAN 2.

Thanks again for your quick response to this issue.

[aareed · Accepted solution]

Jeremy,

Release 9.0 does not support renaming VLANs for multishelf configuration.  However, even if it did, if you have dedicated 2950's I would recommend leaving your system setup as is to prevent problems down the road with someone accidentally configuring the same VLAN for a different purpose on that switch which will open your 15454 DCN up to unwanted/possibly malicious traffic.

The default configuration using a 2950 should included a non-trunked DCN  port which hand's off the VLAN 1 traffic to your LAN, which from there you should be able to map into whatever VLAN you like without touching the isolated VLAN 1 & 2 of the Multi-shelf configuration.

Here is a link to the 9.0 user documentation that describes setting up a node from Single Shelf to Multi-Shelf Mode:

http://www.cisco.com/en/US/docs/optical/15000r9_0/dwdm/procedure/guide/454d90_turnupnode.html#wp446954

Here is a link that explains how the 2950's should be configured:

NTP-G159 Configure a Cisco Catalyst 2950 for a Multishelf Node

http://www.cisco.com/en/US/docs/optical/15000r9_0/dwdm/procedure/guide/454d90_installshelf.html#wp470774

and here is a link that describes how the MSTP nodes should be connected to the 2950's:

NTP-G158 Connect a Multishelf Node and Subtending Shelves to a Catalyst 2950

http://www.cisco.com/en/US/docs/optical/15000r9_0/dwdm/procedure/guide/454d90_installshelf.html#wp454073

Note When Cisco Catalyst 2950s are used in a multishelf node configuration, the Cisco Catalyst 2950 must be dedicated to multishelf LAN traffic. Cisco Catalyst 2950 ports not used by the multishelf configuration cannot be used for any other task.

Ports 23 & 24 provide your DCN connection out to your LAN

Aaron Reed

Carrier Escalation - Cisco Optical TAC

[jnlawrence76]

Thanks for your help Aaron.  You have answered my questions!

[jnlawrence76]

One additional question. What version do we need to be at that would allow us to rename the VLANs?  Is this a higher or lower version of the software?  If lower, what bugs would we encounter if we decided to lower the code?

[aareed]

Changing the Vlan IDs for the Multi-shelf switch configuration is supported in release 9.2, however I would recommend against it.  You are better off with the dedicated 2950's.

Aaron Reed

Carrier Escalation - Cisco Optical TAC

[jnlawrence76]

What if we keep the 2950s, upgrade to 9.2, can we change the vlans still?

[aareed]

You can but there is no need to change the VLANs.  The 2950s are a stand alone LAN the sole purpose of providing ethernet connectivity between the Main Shelf (Node Controller) and Subtended Shelves (Shelf Controllers).  They are not intended to be part of a larger LAN, and should not be used in that fashion.  Using the 2950's is just an optional implementation that customers an use instead of purchasing a pair of 15454-MS-ISC cards which are dedicated switches that can be installed in the 15454 chassis as a pair of blades (working/protect).

The VLANs used on the 2950 are not part of (and should not be) a larger LAN system.  The interfaces (DCN) from the 2950 that tie them to the LAN are not tagged with a VLAN.

[jnlawrence76]

Excuse my lack of knowledge on the DCN port you are talking about, but how should the 2950 ports be configured to be set as a DCN port.  I believe it's supposed to be port 23 and 24, but in our config on both they are set as shown below:

interface FastEthernet0/23
switchport mode access
!
interface FastEthernet0/24
switchport mode access

Doesn't seem to be anything special.  How should the port on my core be configured when connecting to one of these ports in order for use to be able to manage our ONS from our network?

[aareed]

Jeremy,

The traffic egressing the 2950 DCN port (23 or 24) is untagged because these ports are setup as Access Ports.  So you would need to configure your core device as an Access port also, then add whatever management VLAN you want to use to it (VLAN 2) in your case.

So the 2950 ports 23 & 24 are configured as:

http://www.cisco.com/en/US/docs/optical/15000r9_0/dwdm/procedure/guide/454d90_installshelf.html#wpxref85792

Step 6 Enter the following commands to configure Ports 23 and 24. Replace port with the port number (23 through 24) and repeat for each port.

(Switch(config)#interface FastEthernet0/port

(Switch(config-if)#switchport mode access

and then on your core device that attahes to ports 23 or 24 you would expect to see a configuration similar to:

(Switch(config)#interface FastEthernet0/port

(Switch(config-if)#switchport access vlan 2

(Switch(config-if)#switchport mode access

which would take the untagged VLAN and add the VLAN ID 2 Tag to it.

So essentially your managment VLAN 2 gets stripped of it's VLAN ID when it egresses your CORE switch, and would be retagged with VLAN ID 1 (because it is default) on the 2950, where it would get passed to the Primary/Node Controller Shelf which manages the shelves within the 15454 multi-shelf.