6 Interface vlan, 6 Static Routes... Uses what it wants when it wants.

[TheGoob]

Nexus 9k. Seem to fix one thing, then the next thing stops working.

interface Vlan2
  no shutdown
  ip address 192.168.1.1/24
  ip dhcp relay address 192.168.1.1

interface Vlan3
  no shutdown
  ip address 192.168.2.1/24
  ip dhcp relay address 192.168.2.1

interface Vlan4
  no shutdown
  ip address 192.168.3.1/24
  ip dhcp relay address 192.168.3.1

interface Vlan5
  no shutdown
  ip address 192.168.4.1/24
  ip dhcp relay address 192.168.4.1

interface Vlan6
  no shutdown
  ip address 192.168.6.1/24
  ip dhcp relay address 192.168.6.1

ip route 0.0.0.0/0 192.168.1.2
ip route 0.0.0.0/0 192.168.2.2
ip route 0.0.0.0/0 192.168.3.2
ip route 0.0.0.0/0 192.168.4.2
ip route 0.0.0.0/0 192.168.6.2

Currently has this output. I reboot, it changes... I delete all routes and put them back in, another vlan chooses to work.

 ping 8.8.8.8 source-interface vlan 2
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=119 time=25.817 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=25.343 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=119 time=25.34 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=119 time=25.074 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=119 time=25.779 ms

--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 25.074/25.47/25.817 ms
switch(config)# ping 8.8.8.8 source-interface vlan 3
PING 8.8.8.8 (8.8.8.8): 56 data bytes
Request 0 timed out
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 packets received, 100.00% packet loss
switch(config)# ping 8.8.8.8 source-interface vlan 4
PING 8.8.8.8 (8.8.8.8): 56 data bytes
Request 0 timed out
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 packets received, 100.00% packet loss
switch(config)# ping 8.8.8.8 source-interface vlan 6
PING 8.8.8.8 (8.8.8.8): 56 data bytes
Request 0 timed out
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 packets received, 100.00% packet loss
switch(config)# ping 8.8.8.8 source-interface vlan 5
PING 8.8.8.8 (8.8.8.8): 56 data bytes
Request 0 timed out
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 packets received, 100.00% packet loss

[MHM Cisco World]

If all host have VLAN in NSK as GW 

One VLAN need to unique and connect NSK to FPR.

And we use this for defualt router toward fpr.

MHM

[TheGoob]

Are you suggesting that I create an Interface SVI or vlan SVI separate from the 1-6 vlans I have…

On Nexus make it unique, like 192.168.9.1 and on FPR make it 192.168.1.2? How does FPR know to decode all 6 Nexus vlan networks ? Or does it “just know”. 
On FPR is it set as vlan routed? Do I need to make ACLs or NAT?

if so I can do all that on my own, just not sure to what depth this will get. 

[MHM Cisco World · Accepted solution]

FPR will have NAT for each VLAN in nexus except the one that use to connect FPR to Nexus 
FPR will have static route for each VLAN in nexus toward the IP of VLAN using connect FPR to nexus 
FPR will have ACL for each VLAN in Nexus
MHM

[TheGoob]

I will indeed post my topology, I just want to do 1 thought at a time.

With what you mentioned, I would need to remove each vlan on the FPR, or else the 6 static routes through this new "connect fpr" will conflict and not work cause they say "already has route [via the 6 vlans]", So I need to remove the 6 vlans on FPR?

[balaji.bandi]

ping 8.8.8.8 source-interface vlan 2

This shows only VLAN 2 have ACL allow and NAT in your Firewall  (FPR)

So add networks which not working in to object network Group for NAT and Associated ACL in FPR to work.

[TheGoob]

They all do, every vlan and network. This only stopped when routing through the Nexus now. If I changed the Nexus from its current L3 vlan interface setup and went back to L2, each vlan assigned ports have everything as it should. This all started when I create 6 interface vlans. 

[TheGoob]

On FPR

EVERY vlan has a NAT

   x.x.x.177 Dynamic 192.168.1.0

   x.x.x.178 Dynamic 192.168.2.0

And so on for 6 vlan/WAN 

I then have the OUTBOUND trust ACL

   all 6 zones, associated to their vlans, have OUTBOUND open

[balaji.bandi]

This only stopped when routing through the Nexus now

i have also asked other post how is your network diagram looks like ?

If I changed the Nexus from its current L3 vlan interface setup and went back to L2

where is other L3 switch in the diagram, what is that IP address ?

i still do not get it - why do you need so many default routes ? what .2 for all the VLAN interface configured ?

ip route 0.0.0.0/0 192.168.1.2
ip route 0.0.0.0/0 192.168.2.2
ip route 0.0.0.0/0 192.168.3.2
ip route 0.0.0.0/0 192.168.4.2
ip route 0.0.0.0/0 192.168.6.2

If the user IP address vlan pointing to nexus switch as gateway , then you need to have static from FPR point back to nexus switch.

(for better clarify we need to see your network connection diagram)

[TheGoob]

The FPR has 6 Interfaces, again in alignment with Nexus, 6 vlans. The Nexus is running DHCP Servers for each vlan (each vlan also has its own static wan ip). Each vlan on Nexus has its own route back to its familiar vlan ip on FPR for Internet access, thus the 6 static routes.

[balaji.bandi]

(for better clarify we need to see your network connection diagram)

[TheGoob]

Got it working.

6 wan ip's, 6 vlans, FPR to Nexus, Got Nexus set up at a dhcp server for each vlan. Every vlan cans ee each other, connect to each other and connect to the Internet, and best of all, what started it all in 5 different posts in 5 different formats, i transfer from vlan to vlan 700+/- MBps

[TheGoob]

Quick follow up question. Do I need any special NAT for outgoing email now that I am being routed through a vlan not directly connected to the Host sending email? When FPR vlan 2 connected to NEXUS vlan 2 I could send and receive email fine. Now that there is no vlan 2 on FPR and is being routed though an Interface not its subnet using static route, all of a sudden outgoing does not work. 
I am running the DHCP Server on the NEXUS whereas it was on the FPR, I am wondering if I need a more intricate NAT rule .