Layer 2 communication vs. Layer 3

[Federico Coto Fajardo]

Hi All,

I have the following scenario:
Internal LAN - 4506 - inside ASA, DMZ ASA - 2950 - Servers
4506 Gig 6/3 connects to 2950 Fas 0/1

The internal network has a default gateway pointing to the 4506.
The 4506, has a default gateway to inside ASA
ASA DMZ is on the same subnet as the Servers on the 2950

So, I would assume that traffic from my machine intended to a Server, will go through my DG (4506), then
through the ASA and finally through the 2950 to the Servers.

But what is happening is that traffic from my machine goes through my DG (4506), but then straight through the 2950 to
the Servers (bypassing the ASA)

From STP point of view, the 4506 is the Root Bridge.
The root port on 2950 is the direct connection to the 4506.
I think this is why, the connection does not go through the ASA.

What I'm trying to understand is the following....
The fact that the 4506 has a DG pointing to the ASA, does not take precedence over the fact that there's a direct
connection between 4506 and 2950?

Layer 2 follow the path directly between Switches
Layer 3 follow the path through the ASA

Which takes precedence and why?

Layer 2 takes precedence but I don't understand why, since my laptop is on a total different subnet from the Servers that
I want to reach.

I think the solution is to manipulate the cost, so that Layer 2 will prefer the path through the ASA (so Layer 2 and Layer
3 will have the same path).
But could someone explain to me why this is happening?

Thank you All,

Federico.

[Ganesh Hariharan]

Hi All,
I have the following scenario:
Internal LAN - 4506 - inside ASA, DMZ ASA - 2950 - Servers
4506 Gig 6/3 connects to 2950 Fas 0/1
The internal network has a default gateway pointing to the 4506. 
The 4506, has a default gateway to inside ASA
ASA DMZ is on the same subnet as the Servers on the 2950
So, I would assume that traffic from my machine intended to a Server, will go through my DG (4506), then
through the ASA and finally through the 2950 to the Servers.
But what is happening is that traffic from my machine goes through my DG (4506), but then straight through the 2950 to
the Servers (bypassing the ASA)
From STP point of view, the 4506 is the Root Bridge. 
The root port on 2950 is the direct connection to the 4506.
I think this is why, the connection does not go through the ASA.
What I'm trying to understand is the following....
The fact that the 4506 has a DG pointing to the ASA, does not take precedence over the fact that there's a direct
connection between 4506 and 2950?

Layer 2 follow the path directly between Switches
Layer 3 follow the path through the ASA
Which takes precedence and why?

Layer 2 takes precedence but I don't understand why, since my laptop is on a total different subnet from the Servers that 
I want to reach.
I think the solution is to manipulate the cost, so that Layer 2 will prefer the path through the ASA (so Layer 2 and Layer 
3 will have the same path). 
But could someone explain to me why this is happening?
Thank you All,
Federico.

Hi Federico,

You said in your post 4506 and 2950 are directly connected via trunk or access port and it should follow the routing logic as packet is going from different source to destination.

HTH

Ganesh.H

[Federico Coto Fajardo]

Hi Ganesh,

The connection from the LAN to the Servers should go through the ASA (looking at the routing).

But, there's a direct connection from the 4506 to the 2950 via access ports on VLAN 100 (VLAN where all the servers reside).

So, I know the traffic is going from the 4506 directly to the 2950 to the servers (not following the routing, but following the STP path).

Is this the right behavior and what will be the solution?

My goal is to make the communication through the ASA.

Thank you!

Federico.

[Jon Marshall]

coto.fusionet wrote:
Hi Ganesh,
The connection from the LAN to the Servers should go through the ASA (looking at the routing).
But, there's a direct connection from the 4506 to the 2950 via access ports on VLAN 100 (VLAN where all the servers reside).
So, I know the traffic is going from the 4506 directly to the 2950 to the servers (not following the routing, but following the STP path).
Is this the right behavior and what will be the solution?
My goal is to make the communication through the ASA.
Thank you!
Federico.

Federico

It is not following the STP L2 path.

If the 4506 and the 2950 share a common vlan then all that is happening is that your host sends it's traffic to the 4506. The 4506 then looks at the destination address of the packet. If the destination address is on a directly connected network then it won't use a statically configured route, it will simply use the directly connected interface and route the packets that way.

Do you have an interface on the 4506 for vlan 100 ? If so that is why packets go straight to the servers instead of going via the ASA - nothing to do with STP. It's to do with routing.

If you do have a vlan 100 interface on the 4506 and you are sending packets from your host to a server in vlan 100 then it will always bypass the ASA. The way to fix this is to remove the vlan 100 interface off the 4506 switch so it cannot be routed directly or better yet disconnect the 2950 from the 4500 so the only path is to go via the ASA.

However before you do any of that could you explain your topology a little more ie. you say -

ASA DMZ is on the same subnet as the Servers on the 2950

what do you actually mean by that ie. is the subnet for the servers the ASA dmz or not ?

Jon

[Federico Coto Fajardo]

Hi Jon,

I totally agree with you but the deal is that the 4506 and the 2950 indeed share a common VLAN, but the 4506 has no
IP address belonging to the Servers VLAN.
In other words, the Servers are in the 172.16.125.0/24 and there's no 172.16.125.x IP on the 4506.

Because of this, the 4506 should send traffic to its default gateway (ASA), to reach the Server's VLAN.

I am attaching the configuration from the 4506.

The 2950 has a management IP on the 172.16.125.x and all ports on VLAN 100.
The 2950 has a trunk to the ASA.

The server's default gateway is 172.16.125.1 (which is the ASA's DMZ).

So the communication should be through the ASA following the Layer 3 path.

But, I see the Root Port on the 2950 for VLAN 100 being the Fas 0/1 (which is the directly connected to the 4506).

So, I still don't understand why the communication does not flow through the ASA, if there server's VLAN is not
directly connected to the 4506.
The default gateway of the 4506 is the ASA's inside 172.16.11.5

As you can see from the 4506's configuration, there's no IP on the server's VLAN, so it should send the traffic to the ASA to reach this network, but the fact that the 4506 is sharing a direct connection on VLAN 100 with the 2950 is breaking this.

Please let me know.

Thank you.

Federico.

[Jon Marshall]

Federico

This doesn't make any sense. If the 4500 has no L3 SVI for vlan 100 then when it does the route lookup for 172.16.125.0/24 it will see it has to send it to 172.16.11.5 which is the ASA firewall.

Shouldn't make any different about the STP root for vlan 100 because first the packet has to be routed to vlan 100.

How are you determining that packets are bypassing the ASA - have you done a traceroute ? if not can you.

Jon

[Federico Coto Fajardo]

I am determining that the packets are bypassing the ASA because of two reasons:

1. A traceroute that shows that the packets from my machine goes to the DG (4506), and then directly to the server.
2. If I disconnect the direct connection between the 4506 and the 2950 (my final goal), the communication breaks
from the internal LANs to the servers.

So, if the packets were really going through the ASA, it should not matter that I disconnect the link between the
4506 and the 2950.

I agree it does not make sense, and that's why I posted this threat.

This is how I found about the problem.

We determined that the 4506 had a direct connection to the 2950 bypassing the ASA and this was a security breach from
CDP.
Something changed, and the port from the 2950 that connects directly to the 4506 was in err-disable state, and there
was no communication to the servers from the internal LAN.
We found that the port from the 2950 that connects directly to the 4506 was configured as an access port, with portfast
and BPDU Guard.
This is the reason that the port was err-disabled, because it started receiving BPDUs from the 4506 in that port (with the
BPDU Guard feature enabled).
What I have done so far is:

Disable portfast and BPDU Guard on the direct link from both Switches.
But my final goal is to completely remove that connection, and leave all traffic going through the ASA, but when
I attempt to do that, the communication from the LAN to the servers break.

Weird.

Federico.

[Jon Marshall]

What is the source address of your host ?

What is the destination address of the server you are trying to reach.

Can you also post "sh ip arp | include 172.16.125"

Jon

[Federico Coto Fajardo]

I have tested from two different sources:

Machine 1:  172.16.138.37 (reachable through 172.16.15.5 on VLAN 6)

Machine 2:  172.16.24.15 (VLAN 8 directly connected to the 4506)

The only path to the DMZ network (Servers) from the internal LAN is through the 4506.

The destination address of the server we are trying to reach is 172.16.125.40.

All Servers are in the 172.16.125.0/24 with default gateway 172.16.125.1 (ASA's DMZ)

From the 4506:

D       172.16.138.32/27 [90/1253120] via 172.16.15.5, 2d10h, Vlan6

I don't why there is an existing direct link between the 4506 and the 2950 (should not be), but now can't get rid of it.

This direct link is an access connection on VLAN 100.

I don't see an output for ARP for the 172.16.125, why could this be? (I have a continuos PING for it)

Thank you,

Federico.

[gurkamal01]

Hi Federico,

If do trace and you dont reach the ASA in the result you may have an ACL restricting access form inside to the server Vlan on the ASA device.

Regards

Gurkamal

[KARUPPUCHAMY MALAIYANDI]

Hi,

I am sure , the traffic is going via your ASA.

1. A traceroute that shows that the packets from my machine goes to the DG (4506), and then directly to the server.   -- usually when we are doing traceroute ASA/PIX ip will not display.if you want to find out more then login the ASA via ASDM then check that..

2. If I disconnect the direct connection between the 4506 and the 2950 (my final goal), the communication breaks
from the internal LANs to the servers.  -- Because you are running client server vtp mode,If vtp server is not reachable, obviously the vlan 100 information will be remove in the access switch because the access switch is running as vtp client.

If you want to test more, make both the switch as vtp transparent mode and remove your direct connectivity from 4506 to 2950.you will get ping response..

Thanks & Regards

Karuppu

[Federico Coto Fajardo]

I would have totally agree with you, but now that I check, the 2950 is in VTP transparent mode:

sw-dmz#sh vtp stat
VTP Version                     : 2
Configuration Revision          : 0
Maximum VLANs supported locally : 250
Number of existing VLANs        : 8
VTP Operating Mode              : Transparent
VTP Domain Name                 :
VTP Pruning Mode                : Disabled
VTP V2 Mode                     : Disabled
VTP Traps Generation            : Disabled
MD5 digest                      : 0xF8 0xA8 0x2D 0xF4 0x4C 0xFB 0x75 0x6F
Configuration last modified by 172.16.125.10 at 4-26-93 04:00:02

172.16.125.10 (is the management IP for the 2950)

Also, the only direct connection between the 4506 and the 2950 is an access port (which I disabled portfast and BPDU Guard to avoid the port from setting itself to err-disabled when receiving BPDUs).

From the 2950:

sw-dmz#sh run int fas 0/1

interface FastEthernet0/1
description Conexion sw1-4506 giga6/3
switchport access vlan 100
switchport mode access

sw-dmz#sh cdp neigh
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
sw1-4506
                 Fas 0/1            128         R S I     WS-C4506  Gig 6/3

sw-dmz#sh ip int br 
Vlan100                    172.16.125.10   YES unset  up                    up

From the 4506:

sw1-4506#sh cdp neigh
sw-dmz
                 Gig 6/3           150              S I   WS-C2950T Fas 0/1

sw1-4506#sh run int gig 6/3
interface GigabitEthernet6/3
description sw-dmz Fa0/1
switchport access vlan 100
switchport mode access
qos trust dscp
end

In summary....
Both switches have a single direct connection (bypassing the ASA), via a port configured as an access port (therefore no VTP information could pass through).
The 2950 is in VTP transparen mode (even though the 4506 is indeed the VTP server).

I thought that this will be the only explanation, but that's not the case...

Federico.

[Jon Marshall]

Federico

Can you do a quick test ?

Can you deny traffic on the firewall from just your host and then we can verify once and for all whether traffic is actually bypassing the ASA. If you don't have an acl on the ASA inside interface then make sure your acl looks like -

access-list inside_access_in deny ip host   host

access-list inside_access_in permit ip any any

Jon

[Federico Coto Fajardo]

Jon,

You are right.

If I enter an ACE to the ACL of the inside of the ASA blocking traffic from my machine to the Server, I can't PING it anymore (so, you were right and the traffic is definitely going through the ASA).

But now I'm more confused....

If the traffic is passing through the ASA, why the connection breaks when I disconnect the link between the 4506 and the 2950?

Here's a working trace from my machine:

C:\Users\fcoto>tracert 172.16.125.40

Tracing route to 172.16.125.40 over a maximum of 30 hops

  1     1 ms     1 ms     1 ms  172.16.24.2
  2     1 ms     3 ms     5 ms  172.16.125.40

That's why I thought is going directly without going to the ASA (but you guys proove me wrong)...

Now, I don't understand why the traffic doesn't just flow normally if I disconnect the connection between switches.

Could it has something to do with the fact that they share VLAN 100 (even though the 2950 is a VTP transparent server)?

Federico.

[Jon Marshall · Accepted solution]

Federico

From your 4500  "sh vlan" output -

100  vlan-dmz                         active    Gi6/3, Gi6/48

you have 2 ports connected to vlan 100 on the 4500. Are you absolutely sure that the server you are trying to get to is connected to the 2950 and not the 4500 ?

Jon