bridging 2 vlans in same switch with transparent firewall...

[Alejandro Olivan Alvarez]

Hi people.... first post! and I've still half way in CCNA R&S... so plesa dont be too hard :-)

OK...I'm playing with my eBay CISCO crap doing my happy Labs, but I'm stuck trying to understand what's the root cause (and eventually if there is a solution) to the following trouble scenario...

- I got a catalyst switch, 3 Vlans, 3 subnets, with a single trunk port port to a 2801 (10, 20 and 30), doing legacy "router on a stick" scenario ... OK working.

- A pix firewal (runing ASA image) in transparent mode is attached to one VLAN 10 in its internal interface, the external interface connects to ISP DSL modem .... OK working

- I got the 2801 with subinterface in VLAN10 getting ip from ISP router DHCP throug Pix/ASA ... OK working

- Devices connected to VLAN10 ports get IP from DHCP on ISP router and surf the web. The catalyst SVI interface belongs to VLAN10 has IP assigned and works perfect.

- 2801 sub interfaces on VLANS 20 and 30 are statically assigned, 2801 provides DHCP service to those 2 VLANs, acts as gateway for those subnets and do NAT them, traffic goes through VLAN10 to ISP router and Internet.... OK working!

But now, I want to try the following:

I split VLAN10 into two VLANs, lets say 10 and 11.

I interconnect through the transparent PIX/ASA both VLANs by connecting cables directly on access ports VLAN10 to inside, VLAN11 to outside

I connect the ISP router to an access port at VLAN11.

Result:

Everything keeps going almost perfect... I can have 2 layer 2 segments, a "secure" one and an "insecure" one, bridged together through ASA/PIX, now I can even connect devices directly to ISP router by pluging them to VLAN 11... almost perfect.

Trouble / Symptomps:

- ping to to Internet or hosts in VLAN 11 segment, "insecure" side, Internet segment, etc. from VLAN 20 or 30 works, but it does intermittently: a ping to 8.8.8.8 goes a while... it drops a while... it returns a while... endlessly.

- SVI in VLAN 10 can be pinged and telneted from VLANs 20 and 30, but it cannot ping internet, althoug traffic goes through VLAN10 to PIX/ASA to Internet.

- Moving SVI to VLAN11 makes it to reach Internet inmediately, but EVERYTHING ELSE STOPS reaching Internet, complete disaster.

Now I want to understand what is happening here... I have readed and searched similar questions but no clue... I guess something related to STP, a loop or something but I do not see a loop enywhere... I simply do not understand this behaviour...

Could some one help me?

(NOTE: I know description is not precisse, I tried to simplify and make explain short as possible... If necessary, I can paste configs or draw diagram)

Thank you in advance.

Best regards.

[milan.kulik]

Hi,

it's not clear what do you mean by "...- SVI in VLAN 10 can be pinged and telneted from VLANs 20 and 30".

Is there an SVI in VLAN 10 configured on your switch? What is its IP address?

You are saying you are running a router on the stick on your router with a subinterface configured in VLAN10 getting IP from ISP router DHCP throug Pix/ASA.

So if you simply configure an L2 VLAN10 on your switch without any L3 SVI interface, your topology should work, even if you connect your ISP router to VLAN11 and interconnect VLAN 10 and 11 through a transparent FW (if the FW is really transparent passing through all Ethernet frames) via ports configured as access ones on your switch.

Best regards,

Milan

[Alejandro Olivan Alvarez]

Hi... thank you for your help... sorry, I didnt explained well

I'll try to explain:

1st clarification

The switch has configured a management Interface... I probably confused terms and I called it SVI probably wrongly.... so... in other words:

The switch has a Vlan10 interface that can be pinged from hosts connected in Vlans 20 and 30. That Vlan10 interface has an IP in the Vlan10 layer3 subnet range. so, hosts can reach it through the 2801 that correctly performs inter VLAN routing as I learned.

2nd clarification

Yes, the router has 3 subinterfaces, since its physical interface is configured as a trunk, they look something like fa0/0:10 fa0/0:20 fa0/0:30 (I'm not in console... probably is not exact nomenclature!). Because of the working trunk, those Interfaces do "connect" to their respective layer2 domain vlans at the switch.

The subinterface :10 gets IP from DHCP server present at vlan10: DHCP server from ISP router, since there is layer2 connectivity through the PIX/ASA to ISP router (in the pre-trouble scenario, ISP router is wired to external transparent PIX/ASA iface directly, but at the trouble-scenario it instead passes new layer2 "hop", the new vlan11, since ISP router plugs to a port in VLAN11 ) ... note that vlan10 and vlan11 reside in the same switch but share the same l3 subnet address space.

Subinterfaces :20 and :30 have static IPs and are ,respectively, default-gateway IPs and do NAT for hosts in Vlan20 and 30 layer2 spaces... I think no trouble with them...

3rd, about your comment:

You got to the point... but you understand the underlying theory why it happens and me not:

If I configure the Virtual Vlan10 Interface, (IP from vlan10 subnet)... I got that intermittency in Internet access from hosts in vlans 20 and 20, the switch cannot ping internet, but hosts in vlan 20 and 30 still can ping and manage the switch through Vlan10 interface.

If I do a shutdown and no ip address at Vlan10 interface and configure instead the vlan10 subnet ip address at Vlan11 interface, the network Internet access crash, no ping from hosts at Vlan20 0 30 to Vlan11 interface, but switch Vlan11 iface reaches the directly connected at vlan11 ISP router and surfs Internet. 

By your comment I guess that Vlan interfaces (SVIs ?) are for some black magic interfering on the process... you suggest to remove SVI... I'll try :-D

But then... should I remove all VLan interfaces? or could I instead configure a Vlan20 or Vlan30 interface for managent? the fact that "almost works" is really intrigue!!! why that intermitency? I'm enormously curious on what's happening ... networking is great!!!

Thank you very much!!!

[milan.kulik]

Hi,

I guess there might be several issues causing you problem:

a) SVI in VLAN10 - you can easily test by removing it

b) the FW not being 100% transparent - I'd try to remove the FW and interconnect VLAN10 and 11 via a cross cable connected to access ports within those VLANs.  If that works, I'd start blaming the FW.

c) NAT - I suppose you are using public IP addresses within VLAN10 and you are NATing the outgoing traffic to your router VLAN10 subinterface IP address using the NAT overload option?

Best regards,

Milan  

[Alejandro Olivan Alvarez]

Hi!

a) sure I'll do! :-D

b) Good Idea! I'll try using a crossover cable...

c) No.... all subnets are private, this is all crap home cheap labs :-p.  ISP Router is a DSL home router, but yes, it is doing NAT.... so traffic from hosts at vlans 20 and 30 do pass double NAT (first at 2801 with overload as you pointed, then at ISP router) whereas 2801 itself along with hosts connected to vlan 10 (and vlan11 when in trouble scenario) do get IP from ISP DHCP and have ISP router as default gateway, so single NAT to Internet

You're right.... there is an overload in acl... (I take some notes yesterday, so i can paste here)

!
interface FastEthernet0/0
no ip address
duplex auto
speed auto
!
interface FastEthernet0/0.1
description Native VLAN
encapsulation dot1Q 1 native
!
interface FastEthernet0/0.30
description Example Natted VLAN 30 
encapsulation dot1Q 30
ip address 192.168.112.97 255.255.255.224
ip nat inside
ip virtual-reassembly
!
interface FastEthernet0/0.20
description Example Natted VLAN 20
encapsulation dot1Q 20
ip address 192.168.112.129 255.255.255.224
ip nat inside
ip virtual-reassembly
!
interface FastEthernet0/0.10
description ISP Router reaching VLAN
encapsulation dot1Q 10
ip address dhcp
ip nat outside
ip virtual-reassembly
!

....

!
ip nat inside source list 101 interface FastEthernet0/0.10 overload
!

access-list 101 permit ip any any

Sorry the weird addressing...I force myself to use extensive subnetting to stop my brain thinking always on /24 and get comfortable with subneting. 

I but have to try, points a) and b) and your yesterday's suggestions calmly ... sure I'll post results! I'm counting the minutes to put hands on my  Lab toys again!

[Alejandro Olivan Alvarez]

UPDATE:

As pomised... I post yesterday's experiment results :-D

tried your idea of bypassing the firewall witk a crossover cable, vlan port to vlan port, and the result was spannig-tree getting furious and disabling all ports, full lab crash... wow!  I readed around that transparent firewall by default lets arp go through in both directions, but it is obvious there are more things going on under the hood beyond arp, that, casually, because the firewall intervention, prevented spanning tree infuriation...

Also, I tried, removing all SVIs (shutdown) first, or assigning management IP to an SVI from VLAN 20 or 30 second, with same result of interminttency + non SVI reachability on the first case (same as before), and intermitence but with reachability of management interface (so, one litle improvement) in the second one.

I got the feeling that, beyond CCNA literature I got on hand, that talks about handling the redundant wiring between physically independent devices preventing potential loops, there is something more advanced, more deep , that covers and explains what happens here, why the catalyst behaves that way, why the bridging through firewall prevents spanning tree and direct wiring does not.... I still have long way to walk...

regards!

[milan.kulik]

Hi,

which switch model are you running exactly?

And what is your STP configuration?

Are you running per-VLAN STP?

Connecting two access ports assigned to different VLANs via a cross cable should NOT kill STP!

Can you paste your switch config?

Best regards,

Milan

[Alejandro Olivan Alvarez]

Hi... of course... I'll do as soon as I get hands on my lab crap.

I didnt pasted configs before because it looks too aggresive to my taste (like if just wanting someone to correct and solve my problem), on the contrary this is not bussiness, this are labs... and 'm more interested on understanding what is happening, why, and how to manage them through IOS cli.

The only thing I can advance is that the switch it is a catalyst2950 24 port (I know is unsupported... my lab is made of what people calls "eBay crap", but I have learned a lot this way, man on hardware, beyond books). Before configuration, the switch was put to system defaults, so, spanning-tree is at system defaults whatever they are.

If I'm running per-VLAN STP I 'm not aware of it!!!

I didn't explained well again (my english is not very good) in respect to STP, when I wired with crossover, STP was not killed, on the contrary, it apparently became very active!!! It reacted crippling my Lab :-D

Now, thank you very much for your interest and help!

[Alejandro Olivan Alvarez]

Hi again!

There it go... this is the last version of my config.... There seems to be spanning tree stuff at the start of it that I never pointed attention at... I have to understand what it does.

Building configuration...

Current configuration : 3148 bytes
!
version 12.1
no service single-slot-reload-enable
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname catalyst2950-24
!
enable secret 5 XXXXXXXXXXXXXXXXXXXXXXXX
!
clock timezone CST 1
clock summer-time CDT recurring
ip subnet-zero
ip domain-name example.com
ip name-server 8.8.8.8
ip name-server 8.8.4.4
!
spanning-tree portfast default
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
!
interface FastEthernet0/1
switchport mode trunk
no ip address
!
interface FastEthernet0/2
switchport access vlan 10
no ip address
!
interface FastEthernet0/3
switchport access vlan 10
no ip address
!
interface FastEthernet0/4
switchport access vlan 10
no ip address
!
interface FastEthernet0/5
switchport access vlan 10
no ip address
keepalive 3
!
interface FastEthernet0/6
switchport access vlan 10
no ip address
keepalive 3
!
interface FastEthernet0/7
switchport access vlan 10
no ip address
keepalive 3
!
interface FastEthernet0/8
switchport access vlan 10
no ip address
keepalive 3
!
interface FastEthernet0/9
switchport access vlan 20
no ip address
keepalive 3
!
interface FastEthernet0/10
switchport access vlan 20
no ip address
keepalive 3
!
interface FastEthernet0/11
switchport access vlan 20
no ip address
keepalive 3
!
interface FastEthernet0/12
switchport access vlan 20
no ip address
keepalive 3
!
interface FastEthernet0/13
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/14
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/15
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/16
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/17
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/18
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/19
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/20
switchport access vlan 30
no ip address
keepalive 3
!
interface FastEthernet0/21
switchport access vlan 30
no ip address
!
interface FastEthernet0/22
switchport access vlan 30
no ip address
!
interface FastEthernet0/23
switchport access vlan 30
no ip address
!
interface FastEthernet0/24
switchport access vlan 30
no ip address
keepalive 3
!
interface Vlan1
no ip address
no ip route-cache
shutdown
!
interface Vlan20
no ip address
no ip route-cache
shutdown
!
interface Vlan10
ip address 192.168.1.252 255.255.255.0
no ip route-cache
!
ip default-gateway 192.168.1.1
ip http server
!
snmp-server engineID local 800000090300002156E22641
snmp-server community public RO
snmp-server community private RW
snmp-server location Spain
snmp-server contact alex@alexolivan.com
banner motd ^C CISCO labs are fun... ^C
!
line con 0
exec-timeout 0 0
password cisco
login
line vty 0 4
password cisco
login
line vty 5 15
password cisco
login
!
ntp clock-period 17179849
ntp server 213.98.15.138
end

Best regards.

[milan.kulik]

Hi,

which poort is connected to which device?

I suppose Fa0/1 is the trunk connected to your router?

I don't see any ports in VLAN11 in your config.

Were there ant syslog messages visible on the switch when you interconnected those two access ports via a cross cable? Which ports have you used that time?

BR,

Milan

[Alejandro Olivan Alvarez]

hi... what a mistake... I didn't realize that I should paste the problematic config!!!... instead, I pasted the one I have as a "working one"...

I understand what you mean... so

 !
interface FastEthernet0/5
switchport access vlan 11
no ip address
keepalive 3
!
interface FastEthernet0/6
switchport access vlan 11
no ip address
keepalive 3
!
interface FastEthernet0/7
switchport access vlan 11
no ip address
keepalive 3
!
interface FastEthernet0/8
switchport access vlan 11
no ip address
keepalive 3
!

And for the "picture"of connections of the non-working Lab at the catalyst:

2801 is plug at fa0/1, just as you guessed, trunking.

firewall internal port is wired to fa0/2, at vlan10

ISPRouter is plug to fa0/5, first port at vlan 11

firewall external port is wired to fa0/6, second port at vlan11

As you see, I just innocently though of those 7 ports as I could "split once again" them creating two independent switches... Ports 2,3,4 are "secure" side, while ports 5,6,7 and 8 are a "ISP side"/"unsecure side". Note that ports 2 and 6 are the "bridging" ones connected to the transparent firewall.

When I used the crossover cable, as you suggested, I removed the firewall from the scenario... so, the cable went from fa/02 to fa0/6, directly bridging both vlans.

I didn't remember seeing messages on the console, but since as I plugged and unplugged several times during the lab, the console was cluttered, and I don't remember clearly... I will therefore repeat the experiment and report precise info.

Prepare but to a probable "no, no messages", since I remember clearly that I issued several sh spanning-tree commands trying to get info, and that way I saw all ports were down... again, when I repeat the lab I will carefully copy those show reports.

Last, as a noob, I'm aware that the big difference with an experienced technician is that the later knows what and where to look during the problem, Me instead, I get confused and I don't know how to react... So, is there in such scenario some sh command  you consider I should run and study the results? if so, I could do it an note the results too!

Thank you very much for your time!

[milan.kulik]

Hi,

well, I guess

switchport mode access

might be the command missing on all your switch ports except the trunk FastEthernet0/1.

It's possible the ports are negotiating to start trunking when interconnected via a cross cable or your transparent FW passing DTP frames through!!

You can use

sh int fa0/x switchport

command to check the access/trunk port status.

If this does not fix the issue, I'd try to get rid off DHCP and configure a static IP on the router subinterfacenterface FastEthernet0/0.10. I also don't see the reason for using "keepalive 3" command on some of your ports.

But I believe the main issue are the "switchport mode access" missing commands.

BR,

Milan

[Alejandro Olivan Alvarez]

Hi again!

OK... You hit the nail!:

Exactly as you guessed, the missing switchport mode access  on access ports was the main issue, which, as you stated (I was wrong thinking that a Vlan access port will implicitly set and keep the port to access mode) the ports were negotiating a trunk:

With a direct cable, they suceeded, and Vlans went down as looped

Through the PIX/ASA they missed to do so: arp traffic is by default allowed, but VTP, 801q and bpdu frames are not.... Since the PIX is configured to defend outside to inside, there is an allow any any inside to outside, so, my guess on intermittency is that trunk negotiation being only partially successfull because on returning frames outside to inside of trunk negotiation being dropped... but this is a guess (I should check to confirm and learn)

What I have being unable to success is on SVI connectivity:

Now, but, assigning SVI to Vlan20 or Vlan30 interfface works perfect, but, when assigning SVI to Vlan10 or Vlan11 interface there are problems:

I have tried adding static mac address table entries , but to no avail... it seems the switch does not like to have its IP address shared in two different vlans.

Now but... thanks to you, I have took a lesson and have learn to no underestimate the importancy of the switchport mode access command. Labs works 99%

I will try to investigate if I can guess what happens with Vlan10 SVI Iface not wanting to ping Vlan11 port connected hosts...

Thank you very much for your help!!!!

[milan.kulik]

So now you've got VLAN10 and 11 interconnected via access ports (corss cable or transparent FW) and

interface Vlan10
ip address 192.168.1.252 255.255.255.0

configured on your switch?

And some PC with IP address 192.168.1.x (and mask 255.255.255.0) connected to an access port within VLAN11?

And the switch is not able to Ping the PC?

When you issue

sh arp

command after that unsuccesful Ping from the switch, do you see anything for that  192.168.1.x IP address in the output?

BR,

Milan                                                                                                                                                                   

[Alejandro Olivan Alvarez]

Hi again...

No, it is unable to ping.

it can ping hosts that are connected to ports in the VLAN to which the SVI has address configured.

By issuing sh arp, I see same MACs appearing on each VLAN, since, effectively they're reachable in every vlan through the bridge, macs of every host connected to vlan 10 and 11 appear... and everyone can ping everyone, but the switch SVI interface... if SVI is Vlan11 interface it can ping the gateway and surf Inet, but it can neither ping vlan11 connected hosts. If SVI is Vlan10 interface, it can ping everything but nothing on vlan11.

My tests included the PIX/ASA... So, I will repeat SVI ping tests, but using just crossover cable, I will note sh arp command results in either case, and report again!!!

Best regards!!!