Hello guys,
i turn to you 'coz i'm tired to investigate this issue and any help will be appreciated. So let me introduce my network and then to the issue that i have.
Imagine two multilayer switches (Cisco 3550, Cisco 3650)[x1],[x2], another switch (router) 7606 (X1) (sub720 pfc3bxl) and one more Juniper M7i router (X2).
Here is the scenario (I'll try to graph some ASCI table)
.................(X1)......................
....................||........................
...[SW1]==[SW2]==[SW3].....
....................|.........................
.................(X2)......................
Legend:
==: 2x1gbps interfaces members of channel group
||: Another port-channel to the router
|: One gbps link to 2nd router
.: just to solve the problem with (\t)
(The two routers are connected to SW2, but the posting form ignores the \t (tab))
Now the interesting part. Few days ago i've notice that very fast peak/load of traffic is appear/disappear, which is coming from nowhere!(i mean it), to be more specific i'll show you some output from SW2:
SW2# show interfaces summary | i (Port-channel12|Port-channel10|GigabitEthernet2/0/3|Port-channel9)
* GigabitEthernet2/0/3 210921000 62606 427674000 69053 0
* Port-channel9 -------> 710577000 114088 479390000 96853 0
* Port-channel10 177279000 44811 184909000 43224 0
* Port-channel12 23194000 5184 13995000 4514 0
SW2# show interfaces summary | i (Port-channel12|Port-channel10|GigabitEthernet2/0/3|Port-channel9)
* GigabitEthernet2/0/3 213796000 64173 573820000 88916 0
* Port-channel9 -------> 847973000 132931 499757000 109976 0
* Port-channel10 195319000 51409 232962000 52896 0
* Port-channel12 42034000 7642 15156000 4850 0
SW2# show interfaces summary | i (Port-channel12|Port-channel10|GigabitEthernet2/0/3|Port-channel9)
* GigabitEthernet2/0/3 256471000 77067 447029000 76463 0
* Port-channel9 -------> 707196000 110862 416789000 91718 0
* Port-channel10 162892000 42874 194286000 44114 0
* Port-channel12 35055000 6373 12639000 4044 0
This is the output from the same command executed on the sw2 three consecutive times.
Now get your attention to traffic that comes from Port-channel9. As you can see, on the first execution of command the traffic was ~710mbps and 114kpps. Second time this traffic grows up to ~847mbps and 132kpps. If you follow the output, you will notice that the traffic goes out through Gi2/0/3.Third time (2-3 sec after 2nd execution) you can see that this traffic is dissapear.Just because the summary show me where this traffic comes from (input from SW1 PO9) i started to observe the traffic that goes out from SW[1] /Port-Channel9/ and guess what ... there is _NO SUCH TRAFFIC_. That's right, there is just an usual part of ~710mbps and it's not increase/decrease more than +/- ~5mbps all the time!!! This is not all! I logged into my junper and starting monitor on incoming interfaces... So you maybe already guess what ... yeah ... the traffic is not comming on the junper's interface! It looks like the traffic is looped into the SW2. There is no alarms, no STP errors, no routing protocol flaps ... NOTHING. Those 150mbps of traffic just appear to disappear again and again . This cause problems because my connection to juniper router is up to 900mbps and there is no-way to transmit this 150+ mbps through this port which cause serious packet loss and framedrops. So i'm run out of ideas and you are my last hope.
Does anyone have an idea what may cause that to happen?
Thank you in advance for your effort !
Kind Regards,
Danail Petrov
