routing between vlans

[qus83]

Hi Guys

I have an issue with vlans connection,

the costumer has two vlans (50, 51) and he asked to make new one in same way as the current vlans

so I have made new on vlan53 in same way as vlan50 & vlan51 as mentioned in below config.

in vlan50 they have many virtual servers and they add new virtual servers to vlan53 and they asked to make these vlans see each other

but the problem is that I can't make routing because its c2960x stack switches

and I have no idea how to do that on fortigate firwall

shell I have to contact with ISP?????

or make rule(routing , nating,vpn) in firewall to allow the connection?

I have tried to figure it out many times but I couldn't solve it and now i'm out of thoughts 

I hope someone can help me to figure it out 

below the config. and picture  

vlan 50
 name Groun
!
vlan 51
 name GLeem
!
vlan 53
 name Glees
!
vlan 52
 name Test
!
interface Port,channel1
 switchport access vlan 50
 switchport mode access
 spanning,tree portfast
 spanning,tree bpduguard enable
!
interface Port,channel2
 switchport trunk allowed vlan 1,50-53
 switchport mode trunk
 spanning,tree portfast trunk
 spanning,tree bpduguard enable
!
interface Port,channel3
 switchport access vlan 50
 switchport mode access
 spanning,tree portfast
 spanning,tree bpduguard enable
!
interface Port,channel4
 switchport trunk allowed vlan 1,50-53
 switchport mode trunk
 spanning,tree portfast trunk
 spanning,tree bpduguard enable
!
interface Port,channel5
 switchport access vlan 50
 switchport mode access
 spanning,tree portfast
 spanning,tree bpduguard enable
!
interface Port,channel6
 switchport trunk allowed vlan 1,50-53
 switchport mode trunk
 spanning,tree portfast trunk
 spanning,tree bpduguard enable
!
interface GigabitEthernet1/0/1
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 1 mode on
!
interface GigabitEthernet1/0/2
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
 channel-group 2 mode on
!
interface GigabitEthernet1/0/3
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 3 mode on
!
interface GigabitEthernet1/0/4
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
 channel-group 4 mode on
!
interface GigabitEthernet1/0/5
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 5 mode on
!
interface GigabitEthernet1/0/6
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
 channel-group 6 mode on
!
interface GigabitEthernet1/0/7
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 7 mode on
!
interface GigabitEthernet1/0/8
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 8 mode on
!
interface GigabitEthernet1/0/9
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 9 mode on
!
interface GigabitEthernet1/0/10
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 10 mode on
!
interface GigabitEthernet1/0/11
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 11 mode on
!
interface GigabitEthernet1/0/12
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 12 mode on
!
interface GigabitEthernet1/0/13
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/14
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/15
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/16
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/17
 description Support port i VLAN 50
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/24
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
!
interface GigabitEthernet2/0/1
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 1 mode on
!
interface GigabitEthernet2/0/2
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
 channel-group 2 mode on
!
interface GigabitEthernet2/0/3
 switchport access vlan 50
 switchport mode access
 channel-group 3 mode on
!
interface GigabitEthernet2/0/4
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk

interface GigabitEthernet2/0/4
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
 channel-group 4 mode on
!
interface GigabitEthernet2/0/5
 switchport access vlan 50
 switchport mode access
 channel-group 5 mode on
!
interface GigabitEthernet2/0/6
 switchport trunk allowed vlan  1,50-53
 switchport mode trunk
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
 channel-group 6 mode on
!
interface GigabitEthernet2/0/7
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 7 mode on
!
interface GigabitEthernet2/0/8
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 8 mode on
!
interface GigabitEthernet2/0/9
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 9 mode on
!
interface GigabitEthernet2/0/10
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 10 mode on
!
interface GigabitEthernet2/0/11
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 11 mode on
!
interface GigabitEthernet2/0/12
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
 channel-group 12 mode on

interface GigabitEthernet2/0/13
 switchport access vlan 50
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet2/0/14
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet2/0/15
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface GigabitEthernet2/0/16
 switchport access vlan 53
 switchport mode access
 spanning-tree portfast
 spanning-tree bpduguard enable
!
interface Vlan1
 ip address 192.168.1.254 255.255.255.0
!
interface Vlan52
 ip address 192.168.52.254 255.255.255.0
!
interface Vlan50
 ip address 192.168.50.254 255.255.255.0
!
interface Vlan51
 ip address 192.168.51.254  255.255.255.0
!
interface Vlan53
 ip address 192.168.53.254   255.255.255.0
!
ip default-gateway 192.168.50.1
ip http server
ip http secure-server

ip route 0.0.0.0 0.0.0.0 10.10.10.1
ip route 8.8.8.8 255.255.255.255 192.168.1.1
ip route 192.168.0.0 255.255.0.0 192.168.1.1
ip route 192.168.0.0 255.255.0.0 192.168.50.1
ip route 192.168.0.0 255.255.0.0 192.168.51.1
ip route 192.168.0.0 255.255.0.0 192.168.53.1
ip route 192.169.0.0 255.255.0.0 192.168.1.1
ip route 192.169.0.0 255.255.0.0 192.168.50.1
ip route 192.169.0.0 255.255.0.0 192.168.51.1
ip route 192.169.0.0 255.255.0.0 192.168.53.1
ip route 200.55.120.21 255.255.255.255 192.168.50.1
ip route 400.60.20.22 255.255.255.255 192.168.1.1
ip route 192.168.25.0 255.255.255.0 192.168.1.1
ip route 192.168.25.0 255.255.255.0 192.168.50.1
ip route 192.168.25.0 255.255.255.0 192.168.51.1
ip route 192.168.25.0 255.255.255.0 192.168.53.1
!

[chrihussey]

If I understand the diagram correctly, this is the LAN 1 & 2 site connected to the router which has fiber to the MPLS cloud. The fact that you have static routes configured tells me that you some routing capabilities. However, the static routes make little sense in that the same networks have four different destinations and the default route destination is not a valid next hop for the switch.

Is the default gateway for the devices on the LANs the firewall, the router or the switch itself?

Can you post the entire switch config as well as the output of "sh ver" and "sh ip route"?

Thank you

[qus83]

Hi

Thanks for your reply

sh ver

cisco WS-C2960X-24TD-L (APM86XXX) processor (revision L0) with 524288K bytes of memory.
Processor board ID FCW1941B5RL
Last reset from power-on
5 Virtual Ethernet interfaces
1 FastEthernet interface
52 Gigabit Ethernet interfaces
4 Ten Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       :
Motherboard assembly number     : 73-16690-03
Power supply part number        : 341-0529-02
Motherboard serial number       : FOC19418EVM
Power supply serial number      : LIT19341J5K
Model revision number           : L0
Motherboard revision number     : A0
Model number                    : WS-C2960X-24TD-L
Daughterboard assembly number   : 73-14200-03
Daughterboard serial number     : FOC19420HCJ
System serial number            : FCW1941B5RL
Top Assembly Part Number        : 68-100468-01
Top Assembly Revision Number    : A0
Version ID                      : V05
CLEI Code Number                : CMML610ARD
Daughterboard revision number   : A0
Hardware Board Revision Number  : 0x18

Switch Ports Model                     SW Version            SW Image
------ ----- -----                     ----------            ----------
*    1 28    WS-C2960X-24TD-L          15.2(2)E5             C2960X-UNIVERSALK9-M
     2 28    WS-C2960X-24TD-L          15.2(2)E5             C2960X-UNIVERSALK9-M

Switch 02
---------
Switch Uptime                   : 1 year, 37 weeks, 19 hours, 0 minutes
Base ethernet MAC Address       :
Motherboard assembly number     : 73-16690-03
Power supply part number        : 341-0529-02
Motherboard serial number       : FOC19418FA1
Power supply serial number      : LIT19341JKL
Model revision number           : L0
Motherboard revision number     : A0
Model number                    : WS-C2960X-24TD-L
Daughterboard assembly number   : 73-14200-03
Daughterboard serial number     : FOC19420JDT
System serial number            : FCW1941B5SF
Top assembly part number        : 68-100468-01
Top assembly revision number    : A0
Version ID                      : V05
CLEI Code Number                : CMML610ARD
Daughterboard revision number   : A0

Configuration register is 0xF

 sh ip route

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 10.10.10.1 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 10.10.10.1
      8.0.0.0/32 is subnetted, 1 subnets
S        8.8.8.8 [1/0] via 192.168.1.1
      192.168.0.0/8 is variably subnetted, 12 subnets, 3 masks
S        192.168.0.0/16 [1/0] via 192.168.53.1
                      [1/0] via 192.168.51.1
                      [1/0] via 192.168.50.1
                      [1/0] via 192.168.1.1
C        192.168.1.0/24 is directly connected, Vlan1
L        192.168.1.254/32 is directly connected, Vlan1
C        192.168.52.0/24 is directly connected, Vlan15
L        192.168.52.254/32 is directly connected, Vlan15
C        192.168.50.0/24 is directly connected, Vlan16
L        192.168.50.254/32 is directly connected, Vlan16
C        192.168.51.0/24 is directly connected, Vlan17
L        192.168.51.254/32 is directly connected, Vlan17
C        192.168.53.0/24 is directly connected, Vlan18
L        192.168.53.254/32 is directly connected, Vlan18
S        192.169.0.0/16 [1/0] via 192.168.53.1
                      [1/0] via 192.168.51.1
                      [1/0] via 192.168.50.1
                      [1/0] via 192.168.1.1
      200.0.0.0/32 is subnetted, 1 subnets
S        200.55.120.21[1/0] via 192.168.50.1
      10.10.0.0/8 is variably subnetted, 3 subnets, 2 masks
S        400.60.20.22[1/0] via 192.168.1.1
C        10.10.10.0/24 is directly connected, FastEthernet0
L       10.10.10.10/32 is directly connected, FastEthernet0
S     192.168.25.0/24 [1/0] via 192.168.53.1
                      [1/0] via 192.168.51.1
                      [1/0] via 192.168.50.1
                      [1/0] via 192.168.1.1

and the running config. that I posted is the entire switch config.

I wish you can help me

[chrihussey]

Can you confirm my earlier interpretation is correct?

"If I understand the diagram correctly, this is the LAN 1 & 2 site connected to the router which has fiber to the MPLS cloud."

Is the default gateway for the devices on the LANs the firewall, the router or the switch itself?

[qus83]

Yes that is right

And for firewall config we have static and dynamic route (ospf)

And the default gateway is confirmed in switch it self

[chrihussey]

So the understanding is that the VLAN interfaces on the switch are the default gateways for the devices on the respective VLANs. I would assume then the router/firewall is the .1 hosts in the static routes and I assume you are trunking to from the switch to the router/firewall.

1- If so, the switch should be performing routing and the hosts in VLAN 53 should at least be able to get to the the hosts in the other local VLANs. Does that work?

2- If you are trunking to the router/firewall, then a corresponding interface or network route needs to be created in that device.

3- If things are operating in this fashion, it is sub-optimal as all traffic needs to be redirected by the switch to the router/firewall since all interfaces are active on both.

 Also, (again) the default route points to an invalid next hop. Not exactly sure how things are indeed working if it is configured as explained.

Finally, if the switch is doing the routing, it would be much cleaner if it connected to the router/firewall on a single network and all traffic and routing went across this single network. Static routes would be needed in the router/firewall and probably only a default route would be needed in the switch. Aside from that, you could also run a dynamic routing protocol.

Hope this helps

[Micheline Murphy]

Hello and good morning--I agree that you've got far too many static routes.  I would recommend going to a dynamic routing protocol of some sort.  It will be much more scalable in the event that you have to add more VLANs.

I'd use something simple like OSPF.  Enable OSPF on all of the SVIs, and maybe on the L3 physical interface.  Looks like you have only one physical interface, Fa0.  If it's connected to another switch/router within your control, then turn up OSPF on the other device.  Create a 0.0.0.0/0 route that points to the device that links directly to your ISP. 

If Fa0 is connected directly to your ISP, then the 0.0.0.0/0 remains on the switch with the SVIs.  OSPF will just be in charge of directing traffic between your SVIs. 

Here's a good link for configuring OSPF.  OSPF has a lot of bells and whistles, but the basic turnup is pretty simple.  For your situation, the only knob I would think you might want to turn is to advertise a summary route from your SVI router northbound for all of the SVIs rather than a bunch of individual subnets. IP Routing: OSPF Configuration Guide, Cisco IOS Release 15M&T

Hope this helps, MM

[paul driver]

Hello

Without getting to indeph, It seems everything is fine for all the previous vlans apart from this new vlan 53 you've introduced, So have you made sure the FortiGate Fw has a route back towards this new subnet?

[qus83]

Hi

thanks for your replay

i have made static route same as other vlans in fortigate firewall with same gateway for all vlans

but i don't know maybe there is something missing need to be configured in firewall to allow the connection?  i'm not so good with firewall still learning i hope you can help me

[paul driver]

Hello

Is there any access-list or fw rule that need to be amended?

Go through the exiting configuration on the FW pertaining to one of the other vlans and see if you are missing something?

[qus83]

Hi Again

well I have checked the FortiGate there is no access list that configured inside firewall

is it possible to just call the ISP and guide them to do routing in the routers that located in there side ??

or we don't need that just adjust in firewall???

[paul driver]

Hello
It does seem to suggest that somewhere maybe within your ISP that this new subnet isn’t being seen. And maybe a simple static route pointing back into your network or a single NAT statement is all that is required.
 
My reasoning is you’ve checked your FW and the configuration on the L3 switch in relation to the exiting working vlans are the same.
 
Wouldn’t hurt giving them a call and querying.

On a side note: 
Cleaning up your own static route configuration as stated by others on this post would be a good idea,
 
You have duplicate static pointing to different next-hops
ip route 192.168.0.0 255.255.0.0 192.168.51.1
ip route 192.168.0.0 255.255.0.0 192.168.53.1
ip route 192.168.0.0 255.255.0.0 192.168.1.1

ip route 192.169.0.0 255.255.0.0 192.168.1.1
ip route 192.169.0.0 255.255.0.0 192.168.51.1
ip route 192.169.0.0 255.255.0.0 192.168.53.1

ip route 192.168.25.0 255.255.255.0 192.168.1.1
ip route 192.168.25.0 255.255.255.0 192.168.51.1
ip route 192.168.25.0 255.255.255.0 192.168.53.1

Lastly your default static route seems to not be pointing to any valid L3 interface , unless its recursive which i guess this wouldn’t be intentional as i see nothing for 10.10.10.1.
 
See as though you have a default-gateway pointing to the next-hop on vlan 50 suggest pointing your default route this way also and remove the static routes pertaining to this vlan 50.

no ip route 0.0.0.0 0.0.0.0 10.10.10.1
ip route 0.0.0.0 0.0.0.0 vlan 50 192.168.50.1
no  ip route 192.168.0.0 255.255.0.0 192.168.50.1
no ip route 192.169.0.0 255.255.0.0 192.168.50.1
no ip route 200.55.120.21 255.255.255.255 192.168.50.1
no ip route 192.168.25.0 255.255.255.0 192.168.50.1

[qus83]

Hi

Thanks a lot for your explanation

the ip add 10.10.10.1 is pointing to our network

and 10.10.10.10 is the ip add for customer's stack switches

what I did in FortiGate :

1- in policy&objects ---> Address

2- policy & objects---- addresses----address ------ server1(192.168.53.63)294.148

3- policy & objects---- addresses----address ------ server2 (192.168.53.62)294.148

4- policy & objects---- address----address group------access_to_hosts-53---add (server1,server2 )

5- policy & objects----ipv4 policy---  21 / 294.148(port38)..>294.277(port38) --include  access_to_hosts-50(for vlan50) goes to  2 destination ip--- I added the group range that I made for vlan53 servers goes to same destination ip

6-  policy & objects----ipv4 policy---  20 / 294.277(port38)..>294.148(port38) the reverse for point 5

7- last thing I did , made nat rule between group for vlan50 and vlan53 in same interface source and destination  294.148(port38)

then I logged into one of the vlan50 servers and tried to ping to vlan53 servers , failed (timeout)

I can't find the ip 192.168.50.1 , 192.168.51.1 ,192.168.52.1 , 192.168.53.1 might be configured in the routers that located in ISP

please correct me if I made something wrong ?