Access to management port over other VLAN

[Tosj Reiling]

Hi all!

I'm having some issues configuring our new network infrastructure.

We have 4 Cisco WS-C3850-24P-L switches in 2 stacks.

One is for our LAN core and the other for our SERVER core.

We have 2 PFSense firewalls with CARP failover connected in LAGG LAPC to the stacks and configured the VLANS for the LAN segment.

Right now I'm trying to configure the distribution and core switches in a way that their management ports are connected to his own interface in the PFSense that I can create firewall rules to allow or deny access.

At the moment I have the issue that I can't ping/SSH the management ports when I'm on another VLAN (VLAN22) even though I have an allow all rule in the PFSense.

PFSenses and servers are in a 10.32.1.0/24 network.

VLAN22 is in a 10.32.22.0/24 network

All the management ports on the switch are in a 10.32.3.0/24 network and are up and connected with their management port on the "SWITCH"-switch which is directly connected to the PFSense on a 10.32.3.0/24 network.

The distriubution switches are all WS-C2960X-24PS-L switches

I've included a mockup below to visualize the connections.

[paul driver]

Hello

Do the rtrs know how to reach the lan cores subnets?, Do they have valid routes towards them?
Also do the Lan cores have valid routse towards the rtrs?

How are the cores/fw and rtrs communicating , static or dynamic routing? 

[Tosj Reiling]

Hi Paul

Thank you for the response!

The routers are connected to the PFSenses and are WAN interfaces with static IP's.

All my clients have connection the internet and are able to ping clients between the VLANS and I can also ping the PFSense interfaces/gateways.

When I'm on the "Switch" client I can connect to all the management IP's but when I'm on VLAN22 (Operations) I can't access the management IP's..

I've checked on the LAN core with "show IP route" and no routing is enabled.

[paul driver]

Hello

---

@Tosj Reiling wrote:

Hi Paul

Thank you for the response!

The routers are connected to the PFSenses and are WAN interfaces with static IP's.

All my clients have connection the internet and are able to ping clients between the VLANS and I can also ping the PFSense interfaces/gateways.

When I'm on the "Switch" client I can connect to all the management IP's but when I'm on VLAN22 (Operations) I can't access the management IP's..

I've checked on the LAN core with "show IP route" and no routing is enabled.

---

When you say it fails on vlan22 is that from a port assigned to vlan22 on the "Switch" or from any switch in vlan 22 or from any switch in any other vlan apart from the server vlan?

---

I've checked on the LAN core with "show IP route" and no routing is enabled.

---

Are the PFsence FW's performing your inter-vlan routing for the site then ?

From the PFsence can you ping vlan22 sourced from 10.32.3.0 vlan?

what does the arp table show?

[Tosj Reiling]

That's a port on the Operations switch which is in access mode with VLAN22, the Operations switch is trunked to the LAN core with VLAN22 allowed. The LAN core is trunked to the PFSenses, on the PFSense I've created the VLANs.

So a client (10.32.22.101) connected to the Operations Switch on the Operations Interface in the PFSense is able to ping a client (10.32.3.101) connected to the "Switch"-switch on the Switch Interface in the PFSense.

Also from both clients I'm able to ping the DHCP server (10.32.1.10) on the Server Core.

When I'm on the client (10.32.3.101) on the Switch interface I'm able to SSH into the management ports of the other switches (10.32.3.10 - 16) but I can't ping or SSH them from the Operations VLAN (10.32.22.0/24)



I wanted to use the PFSense to control the routing and traffic.



The ARP from the client (10.32.3.101) connected to the "Switch" interface gives me the following:

Interface: 10.32.3.101 --- 0x6

Internet Address Physical Address Type
10.32.3.1 b4-96-91-39-58-79 dynamic
10.32.3.255 ff-ff-ff-ff-ff-ff static
244.0.0.22 01-00-5e-00-00-16 static
244.0.0.251 01-00-5e-00-00-fb static
244.0.0.252 01-00-5e-00-00-fc static
239.255.255.250 01-00-5e-7f-ff-fa static
255.255.255.255 ff-ff-ff-ff-ff-ff static

[Jon Marshall]

Just to add to Paul's questions, what is the default gateway set to on the switches ie. it needs to be the management VIP on the firewalls. 

Jon

[Tosj Reiling]

I've set the default gateway on the switches to the PFSense CARP virtual IP of that interface
10.32.1.2 for the Server interface
10.32.3.2 for the Switch interface
10.32.22.2 for the Operations interface (VLAN22)
etc..

[Jon Marshall]

Are you using the dedicated management interface on the 3850s ? 

If so try adding this - 

"ip route vrf Mgmt-vrf 0.0.0.0 0.0.0.0 10.32.3.2"

Jon

[Tosj Reiling]

Yes I'm using the dedicated management ports on the 3850's and also the 2960x's
Where do I add this ip route? On the "Switch"-switch, Operations-switch or the LAN-core switch?

[Jon Marshall]

Pick one of the 3850s you cannot connect to and add it to the switch and then see if you can then connect 

Jon

[Tosj Reiling]

I've added this ip route to the LAN core and now I'm able to SSH from the client on VLAN22 into the management port (10.32.3.11) of the LAN Core switch, but I can't SSH into the other management ports (10.32.3.12) like the "Camera"-switch on the Camera Interface (192.168.67.0/24) on the PFSense

I've connected all the distribution switches with their dedicated management port onto the "Switch"-switch for easier management. All the management ports are in the 10.32.3.0/24 range.

[Jon Marshall]

You need to add the same route to all your 3850s. 

I am not sure the 2960 switches support VRFs though so you would need to manage them inband ie. just trunk the management vlan to them. 

Jon

[Tosj Reiling · Accepted solution]

Hey Jon

Thank you for your help, I've looked furher into the ip route and the management VRF's and found this link:

https://community.cisco.com/t5/routing/catalyst-2960-x-management-port-routing/m-p/3709763/highlight/true#M302092

I've enabled ip routing on the 2960's and added the ip route to the "Switch" gateway:

ip route 0.0.0.0 0.0.0.0 10.32.3.2

Thanks all!