Cisco homelab VLAN troubleshooting

[fraley.b12]

Hello everyone I am fairly new to advanced networking concepts. I do some networking for my company but most of it is not setting up new devices or networks.

Anyways, I was able to get my hands on a Cisco ISR 2921 and a Cisco 3560-x PoE L3 switch from my work. I have deployed both to my "Homelab" in the hopes of using it to become more familiar with more in-depth networking concepts and study for my CCNA at some point in the future. I have deployed both of these devices in my network using this configuration from a very helpful Youtube video I watched. My network map was essentially the same as the videos @6:00 minutes.

So anyway, the problem - I have a couple of servers running Proxmox that I want to put on VLAN 10, I have confirmed that IP addresses on VLAN 10 are able to reach out to the internet but I am unable contact them from my local LAN. The VLANs can all ping my LAN but not vice-versa. I am assuming that is kind of the point of the VLANs but I would like to be able to manage my Proxmox servers and the IPMI interfaces (although those would be on a separate VLAN as well) from my home network.

Some troubleshooting and guidance allowed me to setup a static route from my LAN router (DDWRT) and now traffic is actually being routed to the VLANs but I get a weird looping issue that I could not figure out. When I fire up a ping from my LAN I get TTL expired in transit:

ping 10.10.10.1

Pinging 10.10.10.1 with 32 bytes of data:
Reply from 192.168.100.1: TTL expired in transit.
Reply from 192.168.100.1: TTL expired in transit.
Reply from 192.168.100.1: TTL expired in transit.
Reply from 192.168.100.1: TTL expired in transit.

Ping statistics for 10.10.10.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

And from a traceroute I can see that the packets are being bounced back and fourth between the interface on the switch and the "external" interface on the router.

tracert 10.10.10.1

Tracing route to 10.10.10.1 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.1.1
  2    <1 ms    <1 ms    <1 ms  192.168.1.2
  3     5 ms     8 ms     1 ms  192.168.1.2
  4    <1 ms    <1 ms    <1 ms  192.168.100.1
  5     1 ms     1 ms     1 ms  192.168.1.2
  6    <1 ms    <1 ms    <1 ms  192.168.100.1
  7     1 ms     1 ms     1 ms  192.168.1.2
  8     1 ms     1 ms    <1 ms  192.168.100.1
  9     1 ms     1 ms     1 ms  192.168.1.2
 10     1 ms     1 ms     1 ms  192.168.100.1

So I tried even removing the Cisco ISR and changing switch interface from 192.168.100.2 to 192.168.1.2. Then changing the default route on the switch from 192.168.100.1 to 192.168.1.1. But I am still getting the same looping issue. So I will probably put the ISR back unless there is a good enough reason not to keep it.

But does anyone have ideas on what trouble shooting I could try next? I found a couple of similar issues on the support forums but the resolutions were not applicable for my situation. I can even post both of the configs on the router and switch if need be. Really any help would be greatly appreciated.

[balaji.bandi]

Looks good process. now all your LAN stuff working..

So your setup working as below now :

ISP --- DDWRT --- Cisco 3560X --- Devices on VLANS.

in this case whatever IP address DDWRT was given as DHCP, that IP work for the internet. the rest will fail  since you do not have a NAT rule for other VLAN work (this is just an assumption)

Now its time to bring back your Router setup as below: So router can do the NAT for you, so all VLAN able to get internet (I Hope this make sense )

ISP---DDWRT Router---CISCO2921-----Switch---Lan device?

Note: since you have done many changes for the network work, keep saving the config and out of the box to save your time

[fraley.b12]

ISP --- DDWRT --- Cisco 3560X --- Devices on VLANS.

That setup is correct.

in this case whatever IP address DDWRT was given as DHCP, that IP work for the internet. the rest will fail  since you do not have a NAT rule for other VLAN work (this is just an assumption)

Not sure if this helps or not, but I have DHCP being handled on the switch at a VLAN level. Are you referring to the IP address my ISP has handed down to DDWRT?

And if I do bring back the router, should I use the same initial config?

[balaji.bandi]

See case here :

ISP giving IP address X - you connected to DDWRT

DDWRT  - Y IP giving on switch - so if the Y IP you used, you able to get Internet, that means DDWRT doing NAT here. ( we are not sure DDWRT have capabilities to do NAT A/B/C ip address.

So in this case You introducing here Cisco router and Config Y IP our side ( Lan side A/B/C) so you router do NAT - and all LAN IP address range can get internet.

And if I do bring back the router, should I use the same initial config?

at this stage, i can not confirm what configuration you have there, since days you have changed many configs.

High level :

1. Router hold all Layer 3 VLAN information and DHCP

2. switch connect as Layer 2 to Router with Trunk configuration (port connected to Router to Switch).

3. You do Access VLAN respected ports, So the Cisco router can do all the work for you.

You can still use the example :

example: Router on a stick ( is this work for you ?)

https://www.firewall.cx/cisco-technical-knowledgebase/cisco-routers/336-cisco-router-8021q-router-stick.html

[fraley.b12]

SP giving IP address X - you connected to DDWRT

DDWRT  - Y IP giving on switch - so if the Y IP you used, you able to get Internet, that means DDWRT doing NAT here. ( we are not sure DDWRT have capabilities to do NAT A/B/C ip address.

 

So in this case You introducing here Cisco router and Config Y IP our side ( Lan side A/B/C) so you router do NAT - and all LAN IP address range can get internet.

That does make sense.

At the risk of complicating things further, I unplugged my switch to get ready to add the router back in, and I should like to mention I have run "write mem" several times. When I plugged it back in, the config changed but I am not sure why. It looks like the gateway of last resort has been removed.

It was:

Homelab-Switch#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 192.168.1.1 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 192.168.1.1
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.10.20.0/24 is directly connected, Vlan20
L        10.10.20.1/32 is directly connected, Vlan20
      192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.1.0/24 is directly connected, GigabitEthernet0/48
L        192.168.1.2/32 is directly connected, GigabitEthernet0/48

Now it is this:

Homelab-Switch#show ip route␍␊
[15:48:02:193] Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP␍␊
[15:48:02:289]        D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area ␍␊
[15:48:02:369]        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2␍␊
[15:48:02:449]        E1 - OSPF external type 1, E2 - OSPF external type 2␍␊
[15:48:02:513]        i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2␍␊
[15:48:02:609]        ia - IS-IS inter area, * - candidate default, U - per-user static route␍␊
[15:48:02:705]        o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP␍␊
[15:48:02:785]        + - replicated route, % - next hop override␍␊
[15:48:02:849] ␍␊
[15:48:02:849] Gateway of last resort is not set␍␊
[15:48:02:897] ␍␊
[15:48:02:897]       10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks␍␊
[15:48:02:961] C        10.10.10.0/24 is directly connected, Vlan10␍␊
[15:48:03:025] L        10.10.10.1/32 is directly connected, Vlan10␍␊
[15:48:03:089] C        10.10.20.0/24 is directly connected, Vlan20

Now when I try to run the `IP route 0.0.0.0 0.0.0.0 192.168.1.1` form the global conf t, the change does not appear to be persistent. Any ideas why?! I swear I feel like I'm taking crazy pills.

[balaji.bandi]

if the config is not saved, May config reg issue - (not sure what is going on here until we see below information)

Can you post the below information after changing the config and working -

show run

show version

Show version show you what config reg :

if this is  - then you have an issue :

Configuration register is 0x2142

change to   ( make sure you copy the running-config to notepad, if not you loose all)

Configuration register is 0x2102

https://www.cisco.com/c/en/us/support/docs/routers/10000-series-routers/50421-config-register-use.html

[fraley.b12 · Accepted solution]

So I apologize for the radio silence but I finally figured it out.

After doing some digging as to why I have not been able to set the gateway of last resort on the switch. I found a helpful forum post about how you can only set it if the next hop was available. Which I thought was odd seeing as I had the VLANs configured and the interface connected to a LAN interface on my LAN router. So I ran 

sh int brief

and it came back that my configured interface was not up. Which is odd seeing as I know the interface is not shut and the cable was plugged in. I double-checked the interface on the switch only to realize that I did not have the cable connected to the correct interface. After switching the cable I set the gateway of last resort and it stuck!

Now onto getting VLANs access to my LAN and the internet. So I setup static routes in my DD-WRT router to all of the VLANs. One thing that I forgot to enable was NAT. So I enabled NAT on the static routes on my DD-WRT router and boom all of my VLANs can now access my LAN and the internet.

The static route config in DD-WRT is attached for those interested.

So I was able to setup VLANs on my L3 switch. Connect it via a dedicated interface on the both the switch and an interface on my LAN router. Set-up static routes on my LAN router and give my LAN access to the VLANs and give my VLANs access to my LAN and the internet. All of this I was able to do without the Cisco ISR.

Thank you for the help @balaji.bandi you have been incredibly helpful throughout this entire process.

[balaji.bandi]

Glad you able to resolve the issue

Now onto getting VLANs access to my LAN and the internet. So I setup static routes in my DD-WRT router to all of the VLANs. One thing that I forgot to enable was NAT. So I enabled NAT on the static routes on my DD-WRT router and boom all of my VLANs can now access my LAN and the internet.

This was we suggested from the beginning of the post since we do not have visibility of what DDWRT can do and can not, now we know it has capabilities. routing and NAt (most of the ISP router does these features nowadays).

So I was able to setup VLANs on my L3 switch. Connect it via a dedicated interface on the both the switch and an interface on my LAN router. Set-up static routes on my LAN router and give my LAN access to the VLANs and give my VLANs access to my LAN and the internet. All of this I was able to do without the Cisco ISR.

yes, this was suggested a lack of feature visibility of DDWRT, it make sense, also some time  ISR router has throughput limitation.

finally, you made it what you intended to achieve.