Network Design

[Palmen000]

 I am going to design and build an LAN network under a tunnel underground with long distance between the switches.

I will have 2 Catalyst switches and 8 Industrial IE3000, and they will be connected with fiber.

For now I am planning on use Layer-2 switching, and if I do that will design the network according to "Suggestion 2".

The network will have 3 VLANs

1 MGMT VLAN

1 RTU VLAN

1 CCTV VLAN

Any suggestion or ideas about the design when it comes to STP, portsecurity, and so on?

[Alessio Andreoli]

PVSTP

Switchport security with violation and 802.1X authentication policy and max one mac address at the time or shutdown plus trap

very good SNMP config - V3 if possible

Use VTPv2 - Catalyst to be primary servers

Tacacs+ access and no console recovery

All not connected port disabled (shutdown)

I also would add a private VLAN for GUEST network access if i were you.

If you want to do something fancy you could even split the fiber and use a internal couple to interconnect the two catalysts so to form a pyramid :

                                                          CAT1 --- CAT2

       IND1        IND2     IND3   ..................................................... IND8

each of which could be double connected to the catalysts for redundancy. this way you will never have STP issues.

Set the catalysts to be VTP servers AND the industrial ones to never become anything else than client -- not even trying to negotiate :)

Good Luck

Hope this helps

[Filip Knezevic]

I don't think you need to worry too much with the simple topology as this one. If needed STP will block the port between last two switches. You can use RSTP for faster convergence.

Port security is really important if somebody can compromise the switches. 

I like to put spanningtree bpduguard enable on access ports. It will block BPDUs to not enter your network per that port and potentially create spanning tree issues. But do not put it on trunk.

[Robert Parten]

You have an incredibly simple topology so you don't need to consider too much in the way of design; however, below are some of the practices I implement in scenarios like yours:

1. Disable VTP - You want to administratively configure all VLANs and tagging on trunk ports. As you network grows, it should not be an excuse to turn on VTP, instead it should be a reason to look to automate or use tools to manage your infrastructure
2. Rapid-PVST should be the default spanning-tree protocol and just fine for environments with a small to moderate number of VLANs deployed. It is only when you grow to closer than to 100 VLANs, and above, should you consider something like MST.
3. Set BPDUGUARD and BPDUFILTER global for portfast enabled ports. You can use other STP enhancements as well, it won't hurt as long as you understand what you're doing. You can leave the timers alone.
4. Determine your spanning-tree primary and secondary root from the get-go, this way you have predictability during a failover
5. For the link connecting your switches together, and the links from those switches into your network, you may want to look at setting your link-debounce timers to 0 for faster convergence.
6. If you can afford more than one fiber connection between the switches, go for it and place them into a port-channel, use LACP if available in your code or static Etherchannel is fine too
7. Loopguard, instead of UDLD, is something to consider too, while it doesn't prevent against wiring (layer 1) issues, it is assumed that once your fiber cables are installed there is little chance of them suffering from a unidirectional issue and loop-guard has better features and is more granular.

Everyone else made great recommendations too, using SNMPv3 if you can, disable telnet, use the strongest SSH cipher you can, disable HTTP/HTTPS on the switches, and keep good logs.

[Filip Knezevic]

Glad you agree Robert :). Just my thoughts too.

Security is overrated after all :), I see telnet dominating every production network I saw :P.

Palmen, you can rate our incredibly useful comments as helpful. :)

[Nathan Spitzer]

Robert - BPDUFILTER is the single most dangerous command you can ever enter on a switch. The number of occasions where it is EVER appropriate is vanishingly small.

Since it causes BPDU's to not be transmitted on that interface and not received on that interface  it disables BPDUGuard. Putting BPDUFilter on an interface effectively turns spanning tree off on that interface and guarantees that should someone loop that port by accident it will result in a total site outage and be very difficult to find.

This is a case of "been there, done that, had the 2 hour Datacenter outage" when someone added that command to "just get it to work"

[Filip Knezevic]

Most dangerous thing after VTP. VTP is really bad :)

[Alessio Andreoli]

VTP Dangerous? Ok to telnet? Security overrated? What have you smoked? Nice comment on BPDU filtering. Can be destabilizing.

[Filip Knezevic]

I was just kidding about security.

About the VTP:
https://digitalshan.wordpress.com/2011/06/25/why-is-vtp-dangerous/

That's how I understood the issues VTP can do. I labbed it also.

And if you disagree, you can maybe explain if I'm wrong instead of being offensive...

[Alessio Andreoli]

I was kidding too. I know vtp and it actually is not dangerous at all. What probably you are pointing out (I can't see your link right now) is the consequence of a vtp server version misconfiguration which overrides all the vtp domain with the wrong server info. This is a human mistake. Not a danger. Please accept my apologies if the joke wasn't clear enough.

To be clear, before physically connecting a switch in a VTP domain you need :

a) setting vtp to transparent mode
b) setting it to client
c) configuring domain, password and version

If you do this nothing will happen to your layer 2 infrastructure.

Take care
Ale😏

[Filip Knezevic]

No problem.

Yes, a human mistake, but it can be really a huge one :). Imagine that it changes the MGMT vlan on 200 switches... That would surely hurt.

After all, 'According to the survey, 66 percent of respondents cited human error in the configuration of network devices as the most common cause of outages.'

In our network we don't have use for it anyway. We have transport switches with 3000-4000 vlans withc changes being made daily.

Too bad Palmen abandoned his own thread.

[umer zubairi]

Hi,

The design looks okay if you've reasonable number of CCTV cameras connected to the network as your up-links are only 1G and traffic has to traverse all over the fiber ring.

Below are the key points to get it running in a proper way:

• Just use the default STP which is rapid PVST.
• Yes, you can run different VLANs and for that you just have to configure trunk ports between switches to carry multiple vlans.

On IE3000 switches:

• create the only  Mgmt SVI (interface vlan) and keep only the vlans for other devices.
• Run the command ip-default gateway with the mgmt interface vlan ip address that's configured on the Catalyst switches.

On Catalyst switches:

• Use stacking to avoid the bridging loop and Layer 3 IP Base license to ensure inter-vlan routing between different LAN segments.
• Configure the SVIs (Interface vlans) for all the vlans including the mgmt vlan in the topology.

Good Luck!

Please rate the answer and mark it 'Correct' if it answers your question.

[Peter Marshall]

Great suggestions from everyone. Regarding VTP, if your orgainization uses it, then make sure your devices meet the standard. If you organization does not use it, then either disable VTP, or set the VTP mode to transparent. That should end the great debate about VTP - at least for this thread. Regarding the drawing, are you going to daisy-chain or home-run the fiber back to the core switches? I strongly advise against the daisy-chain. All host/edge/access ports should have this in their configuation (there may be minor variations based on your IOS): switchport mode access switchport access vlan [vlan] switchport nonegotiate spanning-tree portfast spanning-tree bpduguard enable spanning-tree guard root If a port is not used, shut it down and turn off PoE if it's a PoE switch: power inline never Last, but not least, ask for help building an underground network. There may be building codes, rules, regulations, and you may need permits. My guess is someone else is handling that. You will have to decide singlemode or multimode for your fiber. Check out places like lanshack.com so you can make decisions about distances and speeds. This link has a good chart: http://www.lanshack.com/Fiber-Type-vs-Speed-and-Distance.aspx Those industrial switches, the power, and the network cabling need protection. Companies such as Pentair and L-Com show NEMA enclosures that protect the switch and the cables from critters and water. Your cable installer should know the right ones to use. Here’s an example: http://www.l-com.com/nema-enclosures-14x12x7-inch-120-vac-weatherproof-enclosure-fan-heat-controller Sounds like a great project. I hope you haven’t abandoned this thread & I’d like to hear how it turns out. Best of luck.

[Lntumba]

Hi,

Don't break your brain go into config mode try command (config)#ip ?  if you can see

  route                 Establish static routes

  routing               Enable IP routing

this means your switch is Layer 3 but it doesn't exist make sure you are on Layer 2 switch. 

[BannerMOTD]

Doesn't VTPv3 prevent overriding of the VLAN database? Why not just use VTPv3?

[Alessio Andreoli]

Hi Delta,

people are just not familiar with this solution. You can still override everything but it is way harder to do it because you need to authenticate first.

VTPv3 is a good solution for the classic enterprise ans branch switching design but then again, is matter of personal choice other than architectural circumstances.

Take care,

Alessio