cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
536
Views
2
Helpful
9
Replies

Connecting Nexus 9K to ISR C1111 [Not Routing]

TheGoob
Level 4
Level 4

Hello

 

I have a working ISR with 6 Static WAN NAT to 6 Network/vlan connectivity. All is good. I am running DHCP Servers for each vlan and assigning  an Interface with a vlan. On my PC I plug in to each Interface 1 at a time and it grabs correct Network LAN IP and translates to correct WAN IP. So as far as NAT and ACL (outbound) all is good.

I want  to integrate my Nexus 9K into the mix and utilize its 10G interfaces for LAN connectivity and not the 1G interface speeds of the ISR.

This is a 2-fold situation; gonna post what I did and find its errors through that and also ask what would be the correct implementation of doing what I want.

It is my belief that even though everything is connected to the Nexus 10G interfaces, data will be reduced to 1G because it routes back through the ISR/DHCP Serves. So what I did was

1.) On ISR I created a TRUNK Port (vlan 2-7)

2.) On the DHCP Servers (6 of them) I left the default-router 192.168.1.1, 192.168.1.2 and so on but I changed the vlan Interfaces to 192.168.1.2, 192.168.2.2 and so on.

3.) On Nexus I created 6 vlans (Well, 2-7 and then existing 1) and 6 vlan Interfaces… 192.168.1.1, 192.168.2.1 etc. I did  this so that even though the ISR is handing out the DHCP Pools, the default Router IP’s are the Nexus vlan Interface IP’s so all data routes local on Nexus/10G.

I can see when specifically  the link that the ISR shows explicitly interface up and vlan 2-7 up, so it made its connection to the Nexus. 

The issue is, every time I connect the PC to each of the 6 vlans it grabs the correct IP for the vlan I am on, so again I think the routing is right, but I can not hit the Internet, or even ping outside the network. I can ping each other vlan just not the Internet. 
So what I tried was forget the TRUNK, let me run 6 Ethernet cables from ISR to 6 assigned vlan interfaces in Nexus; same thing! Can ping locally just not Internet. 
With this said, on Nexus or ISR I had no static routes, I assumed I did not need them cause of the L3 Interfaces. I did try with them but still no go. Either I am missing some routing issue or my method is not correct which was why I’m curious about how would it be done to “extend ‘ 6 vlans from ISR to Nexus but allow Nexus to do the routing cause I wanna utilize the 10G.

 

I hope this is not too confusing. I suppose it could be better answered backwards, asking about how this would be implemented. Instead of what I did and why it doesn’t work. 

1 Accepted Solution

Accepted Solutions


@TheGoob wrote:

Hello. 
So easiest response first.. Even though 6 vlans, 6 subnets and 6 wans… Using 1 0.0.0.0 0.0.0.0 192.168.1.2 will work for all Networks because the source IP is translated at ISR level?

Correct.

As far as speed goes. This is the thing. Currently I have 500Mbps Internet.. So yes, downloading from Internet will always be max limited to 440-480 Mbps. What I was wanting to achieve was, I guess for fun/knowledge sake was to allow my various servers and subnets as well as PC and streaming LAN NAS Plex, to all have 10Gbps for transferring files on the LAN across the networks. I have  10 GBPs NICS across 6 devices and wanted to utilize it as such. It really helps when deploying gigabyte plus files and compiling isos as well as unraring and extracting files across the LAN at 10G and not 1G.

So it is not a necessity in terms of my Internet download speed. It is something I want to achieve because I can.

 

Yes, inter-vlan routing is achieved using 6 interface vlan on the Nexus acting as default gateway for each of the 6 vlans. Vlans communicate between them at the speed they are connected to the Nexus switch 10 Gbps.

Since I posted I had put together a scenario to deploy.

Nexus, through guestshell, I will run 6 DHCP Servers and keep the ‘lan routing’ on the Nexus. Each vlan interface will have the 192.168.1.1, 192.168.2.1 and so on IP. On the FPR I will create 6 vlans and assign an IP to each such as 192.168.1.2, 192.168.2.2 and so on in relation to its vlan/subnet on the Nexus. 
1.) Even though there is an IP from each vlan of the Nexus on the ISR, do I still need static routes on the ISR to find the subnets on the Nexus?

No, because on the ISR you have defined the 6 interface vlan and the ISR is connected to the Nexus using a trunk which carries all vlans.

2.) You had already mentioned using the one NEXUS to FPR route, such as 0.0.0.0 0.0.0.0 192.168.1.2 as you said, for ALL vlans to access out in the Internet and get translated on the FPR to their correct WAN address based on NAT. Does this still hold true? 


Yes, with one correction - you mean ISR instead of FPR I think.

Regards, LG
*** Please Rate All Helpful Responses ***

View solution in original post

9 Replies 9

liviu.gheorghe
Spotlight
Spotlight

Hello @TheGoob ,

 

I have a working ISR with 6 Static WAN NAT to 6 Network/vlan connectivity. All is good. I am running DHCP Servers for each vlan and assigning  an Interface with a vlan. On my PC I plug in to each Interface 1 at a time and it grabs correct Network LAN IP and translates to correct WAN IP. So as far as NAT and ACL (outbound) all is good.

I want  to integrate my Nexus 9K into the mix and utilize its 10G interfaces for LAN connectivity and not the 1G interface speeds of the ISR.

This is a 2-fold situation; gonna post what I did and find its errors through that and also ask what would be the correct implementation of doing what I want.

It is my belief that even though everything is connected to the Nexus 10G interfaces, data will be reduced to 1G because it routes back through the ISR/DHCP Serves. So what I did was

1.) On ISR I created a TRUNK Port (vlan 2-7)

2.) On the DHCP Servers (6 of them) I left the default-router 192.168.1.1, 192.168.1.2 and so on but I changed the vlan Interfaces to 192.168.1.2, 192.168.2.2 and so on.

3.) On Nexus I created 6 vlans (Well, 2-7 and then existing 1) and 6 vlan Interfaces… 192.168.1.1, 192.168.2.1 etc. I did  this so that even though the ISR is handing out the DHCP Pools, the default Router IP’s are the Nexus vlan Interface IP’s so all data routes local on Nexus/10G.

I can see when specifically  the link that the ISR shows explicitly interface up and vlan 2-7 up, so it made its connection to the Nexus. 

There are two types of traffic that will hit the ISR:

  • DHCP traffic - which is very very light - 3 messages per host requesting an IP every lease interval which is configurable
  • Internet traffic - which begs the question: What kind of internet bandwidth do you have from your ISP? If you have more than 1 Gbps, then your question is legitimate, if not you can leave it at 1 Gbps and that is it.

In case you have more than 1 Gbps from your ISP, you have on the ISR C1111 8 Layer 2 GigabitEthernet ports. You can configure a Etherchannel link between the ISR and Nexus using for example 4 ports, which will basically give you a 4 Gbps link between the two devices. Make all port members in the Etherchannel the same config on both ends (MTU, native vlan, speed, duplex, allowed vlans), configure the Etherchannel a L2 connection (switchport) and trunk and you have 4 Gbps to the Internet.

The issue is, every time I connect the PC to each of the 6 vlans it grabs the correct IP for the vlan I am on, so again I think the routing is right, but I can not hit the Internet, or even ping outside the network. I can ping each other vlan just not the Internet. 
So what I tried was forget the TRUNK, let me run 6 Ethernet cables from ISR to 6 assigned vlan interfaces in Nexus; same thing! Can ping locally just not Internet. 
With this said, on Nexus or ISR I had no static routes, I assumed I did not need them cause of the L3 Interfaces. I did try with them but still no go. Either I am missing some routing issue or my method is not correct which was why I’m curious about how would it be done to “extend ‘ 6 vlans from ISR to Nexus but allow Nexus to do the routing cause I wanna utilize the 10G.

 

In this case you need a static default route on your Nexus, pointing towards your ISR - you only need one route in only one vlan:

ip route 0.0.0.0 0.0.0.0 192.168.1.2

in this way, for every destination that is known to the Nexus, it will be forwarded to the ISR and from there to the Internet.


Hope this helps. 

Regards, LG
*** Please Rate All Helpful Responses ***

TheGoob
Level 4
Level 4

Hello. 
So easiest response first.. Even though 6 vlans, 6 subnets and 6 wans… Using 1 0.0.0.0 0.0.0.0 192.168.1.2 will work for all Networks because the source IP is translated at ISR level?

As far as speed goes. This is the thing. Currently I have 500Mbps Internet.. So yes, downloading from Internet will always be max limited to 440-480 Mbps. What I was wanting to achieve was, I guess for fun/knowledge sake was to allow my various servers and subnets as well as PC and streaming LAN NAS Plex, to all have 10Gbps for transferring files on the LAN across the networks. I have  10 GBPs NICS across 6 devices and wanted to utilize it as such. It really helps when deploying gigabyte plus files and compiling isos as well as unraring and extracting files across the LAN at 10G and not 1G.

So it is not a necessity in terms of my Internet download speed. It is something I want to achieve because I can.

 

Since I posted I had put together a scenario to deploy.

Nexus, through guestshell, I will run 6 DHCP Servers and keep the ‘lan routing’ on the Nexus. Each vlan interface will have the 192.168.1.1, 192.168.2.1 and so on IP. On the FPR I will create 6 vlans and assign an IP to each such as 192.168.1.2, 192.168.2.2 and so on in relation to its vlan/subnet on the Nexus. 
1.) Even though there is an IP from each vlan of the Nexus on the ISR, do I still need static routes on the ISR to find the subnets on the Nexus?

2.) You had already mentioned using the one NEXUS to FPR route, such as 0.0.0.0 0.0.0.0 192.168.1.2 as you said, for ALL vlans to access out in the Internet and get translated on the FPR to their correct WAN address based on NAT. Does this still hold true? 

This just occurred to me, what is the Link method between ISR and NEXUS? Would I do a vlan 2-7 TRUNK on both or 6 cables vlan 2 ISR to vlan 2 Nexus and so on or just 1 from vlan 2 to vlan 2 for routing purposes using the static routes. 


@TheGoob wrote:

Hello. 
So easiest response first.. Even though 6 vlans, 6 subnets and 6 wans… Using 1 0.0.0.0 0.0.0.0 192.168.1.2 will work for all Networks because the source IP is translated at ISR level?

Correct.

As far as speed goes. This is the thing. Currently I have 500Mbps Internet.. So yes, downloading from Internet will always be max limited to 440-480 Mbps. What I was wanting to achieve was, I guess for fun/knowledge sake was to allow my various servers and subnets as well as PC and streaming LAN NAS Plex, to all have 10Gbps for transferring files on the LAN across the networks. I have  10 GBPs NICS across 6 devices and wanted to utilize it as such. It really helps when deploying gigabyte plus files and compiling isos as well as unraring and extracting files across the LAN at 10G and not 1G.

So it is not a necessity in terms of my Internet download speed. It is something I want to achieve because I can.

 

Yes, inter-vlan routing is achieved using 6 interface vlan on the Nexus acting as default gateway for each of the 6 vlans. Vlans communicate between them at the speed they are connected to the Nexus switch 10 Gbps.

Since I posted I had put together a scenario to deploy.

Nexus, through guestshell, I will run 6 DHCP Servers and keep the ‘lan routing’ on the Nexus. Each vlan interface will have the 192.168.1.1, 192.168.2.1 and so on IP. On the FPR I will create 6 vlans and assign an IP to each such as 192.168.1.2, 192.168.2.2 and so on in relation to its vlan/subnet on the Nexus. 
1.) Even though there is an IP from each vlan of the Nexus on the ISR, do I still need static routes on the ISR to find the subnets on the Nexus?

No, because on the ISR you have defined the 6 interface vlan and the ISR is connected to the Nexus using a trunk which carries all vlans.

2.) You had already mentioned using the one NEXUS to FPR route, such as 0.0.0.0 0.0.0.0 192.168.1.2 as you said, for ALL vlans to access out in the Internet and get translated on the FPR to their correct WAN address based on NAT. Does this still hold true? 


Yes, with one correction - you mean ISR instead of FPR I think.

Regards, LG
*** Please Rate All Helpful Responses ***

TheGoob
Level 4
Level 4

Well that was it. Everything works as it should. All 6 vlans on Nexus have their correct WAN IP and can also co,,unicate across vlans at 10G. Amazing. Just amazing.

 

Next step is to remove 3 vlans from configuration and add in the FPR for vlans 3-6. I suppose same concept... Though I will need to figure this one out. I will create a NEW thread 'Connecting FPR between ISR and Nexus'

BUT I will mess around with it first. I only like asking AFTER I have failed or have legit questions.

BUT so I do not waste time.... Being that the FPR will "house" 3 vlans and 3 static wan ips, do I do the NAT and ACL's on FPR or how does all that work now? Currently ACL's and NAT's for all 6 are on ISR.. Just was not sure being ISR will be first to touch internet.. In my mind I will assume to remove ACL and NAT for those 3 WAN/LAN networks from ISR and do them on FPR.. But I clearly need a link from ISR to FPR.. I also do not think I need all 6 vlans on ISR, so I will remove 3 vlans. Maybe make a vlan of it's own, vlan 8, and use that for the link between ISR and FPR. Also I will need to make a static route so ISR knows where to find the 3 vlans, through the FPR. Hmmmmm

TheGoob
Level 4
Level 4

Alright, something is weird. No matter what I do i can not connect to the internet via wifi. My PS5 keeps saying obtaining ip took too long and my phones etc aren’t connecting. Not saying anything is at fault or wrong, but I feel the DHCP server (being hosted on guestshell) is delaying too long or taking too long to hand out ip addresses.

Also, I do have spamming-tree portfast edge enabled. 

 

Well, how do you connect to wireless? 

Where is the Wireless Access Point connected?

What is the configuration of the interface in which the AP is connected?

Regards, LG
*** Please Rate All Helpful Responses ***

TheGoob
Level 4
Level 4

I have a TPLink Mesh thing. When using FPR as my only router, the TPLink based plugged into the switch and ran as access point. When changing over to ISR and doing dhcp on nexus through the guestshell all i  was move the cord from fpr to nexus. The WiFi light is green so the device is connected but the time it takes to hand out an ip is too long. It’s weird. So by my tv is one of the mesh devices. My PS5 sees the connection tries to connect but times out. So on back of little mesh is an interface. I plug in direct and ps5 says same thing but if you let it sit for a minute it’ll eventually grab an ip but does not wifi. I can’t explain it but it’s like the nexus dhcp server isn’t sending out ips fast enough for ps5 to think there is connectivity…. I manually input an ip, doesn’t accept it. I don’t know .

TheGoob
Level 4
Level 4

To test my theory, I am gonna put the DHCP Servers back onto the ISR and see if it works.

TheGoob
Level 4
Level 4

Moved everything back to ISR for DHCP Server... Left vlan Interfaces on nexus so I sure hope to heck it will still route acoss the Nexus @ 10G and not route through IDR 1G..But being they are SVI's, I assume LAN traffic stays on Nexus.

Anyway, now PS5 and everything runs smooth and fast. It has something to do with the Nexus handing out IP's which it natively does not. Did I set it up wrong? Sure, why not, but I can not see how. It functioned 100%, just slow to hand out IP's. This is the guide I followed.

https://github.com/ndelecro/nx-os-programmability/tree/master/Guest_Shell/PXE_Server