04-30-2008 12:18 PM - edited 03-05-2019 10:42 PM
Our School District has 25 buildings and each have it's own vlan assigned to it. Recently one of our building vlans, 111, just stopped communicating with another, 157, but does communicate with the remaining 23 vlans. Vlan 157 is not able to communicate with vlan 111 but can communicate with the other 23 vlans.
We have a 6509 sup720 running CatOS with 3500 series edge switches.
Any suggestions on where to start looking would be greatly appreciated.
Cathy Perry
WWCSD Tech Group
05-02-2008 10:29 AM
Cathy
Don't thank me just yet as there is nothing obviously wrong in your configs.
One thing
set vlan 111 9/14 (Bldg Server)
set vlan 157 9/9-10 (IE filtering Server & Bldg Server)
Is the Bldg server the same server in both set commands. ie does it have 2 NIC's, one in vlan 111 and one in vlan 157.
If so can you print off the routing table off this server
Jon
05-02-2008 10:38 AM
"Jon, there have been no configuration changes to the 6500 in at least a month."
You sure about that??
This is from the output of the sho run on the MSFC that you just sent us:
MSFC#sho run
Building configuration...
Current configuration : 9610 bytes
!
! Last configuration change at 22:55:39 EDT Tue Apr 29 2008
! NVRAM config last updated at 22:26:23 EDT Tue Apr 29 2008
The configuration change was made the night before you started this thread. Sound interesting to you? Not only was a config change made, it was saved, too. So, if the change is what caused this problem, rebooting the switch may not help you.
VICTOR
05-02-2008 11:22 AM
Victor,
Thank you for bringing this to my attention. This change had slipped my mind since it did not correct the problem. There had been no other changes in the switch since April 4.
The problem had started before this configuration change. When making the change did not correct the problem I decided to start this thread.
Hope this clears up the confusion.
Thank you
Cathy
05-02-2008 11:18 AM
Jon,
These are 3 separate servers
vlan 111 9/14 goes to Stevenson MS server
vlan 157 9/10 goes to Admams MS server
vlan 157 9/9 goes to Adams IE filter server.
Myself and our Network Administrator have checked DNS/DHCP Mgmt to ensure each has the proper gateway loaded if this is what you are asking for. Vlan 111 has a gateway of 10.91.72.2 Vlan 157 has a gateway of 10.91.48.2.
Cathy
05-02-2008 12:14 PM
Cathy
Have checked your configs and there is still nothing obvious.
Are both servers in the 157 vlan unable to connect to vlan 111 server.
What do your arp/mac-address tables/cef tables look like when you try to communicate between the 2 vlans.
I'll have a think over the weekend and see if anything else comes to mind.
Jon
05-02-2008 12:54 PM
Cathy,
The link below can help you with troubleshooting cef issues on the 6509 you have running hybrid mode. They do not specify the mitigation actions but at last this should give you an idea how to look at cef tables and pinpoint inconsistencies.
http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00804c5472.shtml
-serg
05-07-2008 07:58 AM
Do you have copy of the configuration before the change was made?
05-07-2008 10:23 AM
TJ,
Not sure which change you are referring to. I have several configurations of both the Sup720 and MSFC. If you are referring to the change I made on 4/29/08 all I did was remove a No Ip Unreachables config from vlan 111 in the MSFC.
I always take a fresh copy of the configs before I make any changes to them. Except when it is what I believe to be a small change as this one.
Please let me know which configs you would like to see.
Thank you
Cathy Perry
05-02-2008 02:02 PM
Jon,
Yes, both servers in vlan 157 are not able to ping to vlan 111 server.
As far as arp/mac-address tables/cef tables go I don't have that level of experience / knowledge to know what I would be looking at.
Here's a thought:
As I have been reading the different posts I have been pouring over configurations from different points in time from the 6500 and two things continue to stick out to me:
PortInstanceCost on the Sup 720
and
Control Plane on the MSFC.
Is there any remote possiblity that either of these could be the cause?
These are both in previous configs I have looked through but not the current configuration. I haven't mentioned them since these vlans could communicate with each other for about 3 1/2 weeks after the last time we touched the configs on the sup720.
Thanks and have a great weekend.
Cathy
05-02-2008 07:31 AM
5) Can you post a "sh ip route" from the 6500.
âsh ip routeâ did not give me the response I expected to see. What information
should I see when I run this for you?
6) Can you post the interface vlan
sh ip route is off of the MSFC, not the supervisor.
And I think Jon is asking for the interface configurations for both vlans from the MSFC, not the module.
05-02-2008 10:36 AM
"there have been no configuration changes to the 6500 in at least a month."
You sure about that??
This is from the output of the sho run on the MSFC that you just sent us:
MSFC#sho run
Building configuration...
Current configuration : 9610 bytes
!
! Last configuration change at 22:55:39 EDT Tue Apr 29 2008
! NVRAM config last updated at 22:26:23 EDT Tue Apr 29 2008
The configuration change was made the night before you started this thread. Sound interesting to you? Not only was a config change made, it was saved, too. So, if the change is what caused this problem, rebooting the switch may not help you.
VICTOR
!
05-01-2008 12:07 PM
Cathy, can you clear arp and if that does not help, reload the 6509?
have you tried that?
Sorry guys for interjecting, I do not see that the setup is that complicated and the notion that the issue just happened by itself gave me an idea of a software glitch that could be fixed by reloading a switch.
Agree with Jon, you guys in a better position to see a feasibility for the reload. Sometimes it's a quick fix.
One more thing is to look through the logs of 6509 to make sure there is not critical errors there....
-serg
05-01-2008 12:16 PM
Jon,
I did run the clear arp command last night which did not help.
I like the idea of a reload except it is very difficult to schedule with night classes in the district. I will see what we can do.
Cathy
05-01-2008 12:19 PM
Cathy
Different person posted about clearing arp and reloading.
Problem with reloading is if it fixes it you don't actually know what fixed it so if it happens again you are non the wiser and you have to reload again. Depends on how much downtime you can get on your switch. And bear in mind that a reload can sometimes bring a different set of problems.
But not saying you shouldn't do it. You are in the best position to judge.
Jon
05-01-2008 12:41 PM
Cathy,
Check the trunk links to make sure that both of the vlans in question are allowed on the trunks. Also check the VTP configuration revisions to make sure they are consistent. Run the show vlan command on each switch, make sure both vlans are on both switches. It does not appear to be a routing (layer 3) problem since they are functioning properly with other vlans and you have ruled out an ACL causing the issue. Appears to be a layer 2 issue.
good luck
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide