Management Network Design

[Mark Mattix]

Hello, I'm looking for recommendations for the best way to design a management network for one of my locations. Some devices have built in management interfaces like my 2960s and other devices like my 3925E do not have specific interfaces so I planned to use gi0/3 as a management port.

My plan is to create a management vlan on my L3 core switch and then connect all of the other components to this management vlan and only allow SSH into the vlan. 

What methods can I use to insure data traffic doesn't attempt to use the management port on a device like the 3925E which will have a another interface for data traffic? Should I be doing this a different way? Thanks for the advice!

[Mark Malone · Accepted solution]

For devices that didn't have MGMT ports we used same as you a spare port or a vlan on a switch specific for mgmt. , then we used extensive acls in and out to prevent traffic trying to use other interfaces , there is a feature on ios-xr called mpp but its still only in pipeline in ios-xe we requested a feature release through Cisco but are still waiting on it to come inline but I have been told its not long away now but were waiting nearly a year , its on the config but doesn't work right , TAC informed us its not fully supported that's why we ended up using acls as it was the only other sure way of preventing it , it doesn't exist in ios

we then connected all out mgmt. ports back to a separate mgmt. switch where the svi vlan was defined , we connected as much as we could physically through the mgmt. ports but budget and distance prevented us connecting everything as the campus was so big that's why we used a vlan too , so its not full oob its a mix but traffic is still segregated , we used oob switches in conjunction with this for serial connections and then used a 3g sim on these so we could access if both wan connections went down for remote offices , couple of hundred bought these devices but for the main campus offices the 48ports were same price as 3850 but you can get 8 ports ones too that were cheap

make sure all your mgmt. protocols are sourced for the interface too

ip ssh source-interface GigabitEthernet0
logging source-interface GigabitEthernet0

ip tacacs source-interface GigabitEthernet0

ntp source GigabitEthernet0

[Mark Mattix]

Thanks a lot for that information Mark. When you state a source for SSH does this mean management traffic will only be allowed out that interface, like Gi0? Or does this mean if I attempted to run an SSH session on the device it would source that session from what I specified?

Would you be able to post an example of the ACL's you used for traffic going in and out? I appreciate it!

[Mark Malone]

with ssh its inbound you need to worry about making sure it goes in through the mgmt. interface and not any of your other up interfaces, setting the ssh source interface will force any ssh traffic outbound through that interface when you initiate from the router itself but in our case that was rare as we have mgmt. jump servers we get to our network devices from  , yes I can post an example tomorrow there basic enough extended acls but I will have to change around the subnets so there is no sensitive information

just as Jon noted brought something back to me, on our nexus and some of the ios-xe platforms like 3650s and 3850s when you use the mgmt. interface you will have to use a vrf and you cant change its name either and everything then that needs to be sourced will also need the vrf in it so its part of the mgmt. vrf routing table

like an example under the vty set the access-list 154 in vrf-also or with AAA and tacacs the tacacs server most also contain the mgmt vrf in the AAA too and but I can share some examples if you have any devices like that too

[Mark Mattix]

Ok, thanks again Mark!

[Mark Malone]

so the way we did the acls was 1 for inbound 157 and 1 acl for outbound 158 then applied to the mgmt. interface or vlan  , ssh was only inbound an we allowed in from our jump serveres to the mgmt. subnet which as example below is 172.16.48.0/23 , so prime could ssh to the devices, our monitoring device could also access them and of course our jumpservers so we could get to cli , we had no outbound rule for ssh apart from set the source of the ssh , for other traffic like netflow and https we had rules in and out

interface GigabitEthernet0
 ip access-group 157 in
 ip access-group 158 out

access-list 157 remark #### SSH #### v20160512
access-list 157 permit tcp host 172.19.80.241 172.16.48.0 0.0.1.255 eq 22
access-list 157 permit tcp host 172.28.17.210 172.16.48.0 0.0.1.255 eq 22
access-list 157 permit tcp host 172.34.160.22 172.16.48.0 0.0.1.255 eq 22
access-list 157 permit tcp host 172.17.164.101 172.16.48.0 0.0.1.255 eq 22
access-list 157 permit tcp host 172.29.60.162 172.16.48.0 0.0.1.255 eq 22
access-list 157 permit tcp host 172.25.154.53 172.16.48.0 0.0.1.255 eq 22

[Mark Mattix]

Ok, that makes sense, thanks a lot for sharing this!

[Top-mmorpg]

Thanks, i've got the same problem to.

[Mark Mattix]

Mark, can you notice anything I might be doing wrong with this ACL?

interface ve 1
ip access-group Management-IN in
ip access-group Management-OUT out

ip access-list extended Management-IN
permit tcp 192.168.1.0 0.0.255.255 any eq 443
deny ip any any log

ip access-list extended Management-OUT
permit tcp any eq 443 192.168.1.0 0.0.255.255
deny ip any any

I can connect to my server with https when the acl is not applied but lose connection when I apply it. I have additional statements that are similar but for port 80 as well. Thanks

[Jon Marshall]

Where is the server in relation to the SVI ?

[Mark Mattix]

The server I'm trying to control access to is directly on the 10.10.1.0 network. I'm applying my ACL in and out on the 10.10.1.0 SVI.

[Mark Mattix]

I guess I should also state that this device is actually a Brocade L3 switch, I'm assuming they work the same though...

[Jon Marshall]

I'm not following your acl.

Does the server have a 192.168.1.x IP ?

If so in both your acls you have the port in the wrong place. 

Jon

[Mark Mattix · Accepted solution]

Sorry I edited my prior comment, my client device is on 192.168.1.0 and the server is on 10.10.1.0. These ACLs are being applied to the Server 10.10.1.0 SVI.

According to this:

ip access-list extended Management-IN
permit tcp 192.168.1.0 0.0.255.255 any eq 443
deny ip any any log

Doesn't this mean my client from 192.168.1.0 sourcing from any port going to my server on 10.10.1.0 listening on 443 should be allowed? Do you think I would need the reverse statement as well to allow the server to come into the SVI and back out to the client?  Thank you!

[Jon Marshall]

You have applied the acls in the wrong direction so just swap them around. 

Jon