How can I pass another complete network over my network

[dajohnso]

I have a network that connects two buildings with switch A and switch B. These two switches are connected with a pair of trunks using 1G dishes. I have about 10 vlans configures in my environment and everything works fine for my equipment and networks. I have a need to connect a 3rd parties network (switch C and D) over my network but I dont want to intermix the two networks. I have verified they have cisco switches as well and none of the vlans overlap (except we both have native vlan 1, I know, I'm already considering changing to a different native vlan). How can I bridge their switches using my link? I tried turning off lldp and connecting their trunk to my switch on an unused VLAN and that didnt work. I changes my ports to trunks and only allowed vlans 1,400,600,1002-1005 and I was able to ping from C to D (over A & B) with different IP blocks then I use in my network but when I added a vlan to switch C it did not show up on switch D vtp but they do when they are directly connected? So in short how can I connect C & D over my A & B so that C&D do not show up as cdp nei in my environment and  C&D do not show my switches in cdp nei and so that all traffic and admin of C can reach D? I don't need to (or want to) be able to access anything on C or D and I certainly do not want C&D to see anything in my network. I can make any changes to my environment needed but I can only make suggestions for changes on the vendor network. Would it help if I moved everything in vlan 1 to vlan 2 and made it my native/management vlan? I want the connection between C&D to be as transparent as possible almost like they are directly connected. (I would rather NOT put them on one set of dishes and me on the other, I like having the redundant pair of dishes even though I do not need 2Gbps throughput, they are on a  port channel group for load balance and redundancy)

[Joseph W. Doherty]

We've come pretty far, with confirmation that QinQ is a possible solution, although on your 4948s, there appears to be a bug, that doesn't seem to also be on your 4948Es.

From some comments, @dajohnso has made, if the need for this capability looks like it may grow far beyond the OP description, and because QinQ does appear to have some inherent "gotchas", I'm wondering whether any other technologies, would be better long term solutions.

Especially with the 4948Es, they may, even as old as they are, may offer some such features not often found in the lower model Catalyst switches.

[dajohnso]

OK, so I have received my 2nd 4948E and ready to configure and swap my core 4948's to eliminate the bug affecting vlan 1. Now I have a choice between 2 versions of IOS. My CCIE moved away a few years ago to another state so I no longer have access to download IOS software so I am stuck with the versions I have. Here are my choices... 

cat4500e-entservicesk9-mz.152-2.E6

cat4500e-ipbasek9-mz.152-4.E6

I reviewed the bug fixes and although there were several hundred bug fixes between release 15.2(2) and 15.2(4) I am not sure if I am better off with (2) or (4). I have always leaned toward enteservices when possible. I compared the two but didn't see anything that should affect me? 

And to keep this on topic, I am speaking specifically related to my origional question of passing the vendor traffic over my (2) core switches.

somewhat of a mute point now...I just purchased a4948E on ebay with enterservice 15.2(4)E9. There is only 1 version newer (E10)...so I think I'm set.

[dajohnso]

ok, so I have replicated my core on the new 4948E devices (ios cat4500e-entservicesk9-mz.152-2.E6) and everything works as expected (including vlan 1 !!!). No known issues at this time. I can't thank you guys enough, this was a very informative process and I really learned so much. So happy to realize a lot for the issues were IOS version specific. Although I have understood the tunnel in theory, this was my first implementation.  FYI, this IOS version 15.2 also fixed the issue of port channel groups not working properly if I mixed fiber and copper. Testing on the 4948E cores I was able to mix and match fiber/copper without issue. It even let me put the fibers in PO1 and copper in Po6 and allowed BOTH to be connected simultaneously and used BOTH to pass traffic (i.e. it did not block one of the paths).

[dajohnso]

ok, so I replaced my Core switches in production yesterday and today. Yesterday I did switch B, I copied the entire config from B and loaded it on A and moved all the connection over, no issue.

Today I started the process of replacing switch A. I copied the entire config onto the new 4948E, changed the vlan 1 IP for the old switch and changed its name adding "-old" to existing hostname. Created a trunk to connect the new switch to the old one so the new switch had access to the network and the internet so as I moved users to the new switch they would only be down as long as it took for the into to come up.  I verified the "cdp nei" and ping between switches and everything looked good. I started moving users over. No issues. When I got to the Internet uplink I immediately noticed that ping times when from 4ms to 3800ms !!! After the main uplink was moved to the new switch I saw the link go down a few seconds, come back up, pass about 3 packets at 3-6ms, and then all additional pings came in at 3000+ms, I moved it back to the same port n the old switch and immediately it dropped back to 4ms? Moved it to the 4948E again, and a few packets at 4ms and then 3000+ms every time (and some lost packets). I cant understand why? Both switch A and B have the same IOS (cat4500e-entservicesk9-mz.152-2.E6), Additionally, I have (2) internets installed in my environment, vlan 400 is a Verizon FTTI network, and vlan 200 is RCN (cable broadband). So I moved my laptop to the vlan 200 and tired moving that connection, and again, as soon as I moved the uplink to the new 4948E the replies went to 4000ms, moved it back to the old 4948 and back to 4ms. I am at a loss.  So its happening in both VLANS? To be clear, the only thing connect to the old 4948 now is just the uplink and a trunk to the new 4948E. Any ideas? I ordered another 4948E, waiting for it to come in so I can try swapping units.

UPDATE: Just checked, switch B CPU utilization is 4%, switch A is 75%? I also noticed switch A takes considerably longer to boot than B.

here are the two highest utilization tasks.

79 616949 2248766 274 2.47% 2.69% 2.60% 0 Cat4k Mgmt HiPri
80 15004998 2025681 7407 70.95% 67.15% 67.12% 0 Cat4k Mgmt LoPri

[dajohnso]

upgraded to cat4500e-entservicesk9-mz.152-4.E5.bin, will post tomorrow if it had any impact.

[dajohnso]

OK, its been a while but I upgraded all my related switches to 4948E and IOS 15.2(4)E10. I had issues with the "vendor" networks I was passing over the core switches? It looks like ping works fine for several vlans that are being passed over the network but all of the cameras streaming over the link did not work. Looking at the traffic (capture) it looks like ARP requests on the vendor networks are not always making it over my network? Am I missing something? Here are the configs for the relevant ports.

interface Port-channel6
description trunk to parking
switchport
switchport mode trunk
!

interface GigabitEthernet1/47
description trunk parking-1
switchport mode trunk
channel-group 6 mode on
!
interface GigabitEthernet1/48
description trunk parking-1
switchport mode trunk
channel-group 6 mode on
!

interface GigabitEthernet1/36
description to Vendor Trunk
switchport access vlan 701
switchport mode dot1q-tunnel
no cdp enable
l2protocol-tunnel cdp
l2protocol-tunnel lldp
l2protocol-tunnel stp
l2protocol-tunnel vtp
!

I have the same config for switch A and B, the "vendor" network (trunk from their switch) is plugged into my port gi1/36. Also, in my diagram above I show dual point to point dishes, I replaced this with a direct fiber link just to make sure it wasn't a dish related issue. When the two "vendor" cores are connected together they have no issues, when I connect their cores to my port gi1/36 the cameras will not stream anymore. Vendor has stated that all the cameras use IP/TCP (and not UDP). All cameras have static IP's and talk back to a streaming server also static IP. The cameras are in building 1 and traverse the link to the server in building 2.  Works fine when they are on their fiber but when they move to my port gi1/36 they can ping all the cameras but streaming does not work.

[Joseph W. Doherty]

Just wondering, whether perhaps ARPs, and/or their replies, are extensively delayed or dropped.

[dajohnso]

I didn't do the capture, the vendor did. Their response to me was it did not look like the ARP requests were being transmitted over the tunnel so I have to assume they were dropped or so delayed they weren't in the capture period. To be clear, when thier switch is connected via fiber they have no issues, when I moved them to my dot1q tunnel port it stopped working even though my two switches were connected via fiber. I have no other issues in my network and all clients work fine except for the vendor CCTV network after they are moved over.

[Joseph W. Doherty]

Very curious, because a ping, which works for your vendor (?), also would ARP across the tunnel.

Conceptionally, all should work fine, practically (and from some of your direct experience), these Cisco 4800 switches seem to have issues using this feature.

Unfortunately, devices being EoL, short circuits vendor support.

[Jens Albrecht]

Are the IP cameras and the streaming server located in the same Vlan/IP subnet?
Beside the regular ARP requests there are some really old legacy protocols like Reverse-ARP and Proxy-ARP that might cause problems.

You can do a "show int gi1/36 controller" to check on your side for drops and possible reasons.

[dajohnso]

606604 packets input, 120627368 bytes, 0 no buffer
Received 42475 broadcasts (31621 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
5950182 packets output, 1260450927 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

I didn't see anything stand out.