Virtual NIC issue on VOIP system and SG500X

[Mike Horowitz]

We have a VOIP system with 1 physical NIC attached to our SG500X. It has a virtual MAC address along with the physical MAC address. It has 2 IP addresses tied to it.

I can ping the physical IP address, but can only ping the virtual address from the same vlan as the virtual nic. However, if I clear the arp table, a few pings will go through until they all time out again. This is causing problems on my phone system, and I need to establish this communication. I have tried putting in static MAC and ARP entries, but that does not work. Any ideas? Thank you in advance!

[Tom Watts]

Mike, I did not receive an email. My email is towatts@cisco.com

I will be available around 8:15pm EST if you can.

Thank you.

-Tom
Please rate helpful posts

[Mike Horowitz]

Hi Tom,

After making your changes, and playing a bit, new information came to light. I am able to ping one virtual NIC on one of the systems, but not the other. I can ping that one from the same building (different vlan), but not from the second building. I was also told of other network delays accessing the servers, and pings to servers are very erratic and inconsistent. I am looking at the possibility of one of the switches being a little faulty. I am not sure how to fully diagnose this though, since it is transmitting data.

Mike

[Tom Watts]

Hi Mike, can you make a packet capture going to each NIC?

I'm very interested in seeing the source and destination MAC addresses, also, can you perform a trace route from each NIC side going to one another? I'd like to see the hops and TTL

-Tom
Please rate helpful posts

[Mike Horowitz]

Hi Tom,

Packet capture will ahve to wait - the pc I use for that has to be replaced. Here is a traceroute from my pc in the 10.50 subnet to the voip system in the 10.140 subnet:

C:\Users\mhorowitz>tracert 10.140.1.10

Tracing route to 10.140.1.10 over a maximum of 30 hops

  1     1 ms     9 ms     4 ms  10.50.1.1

  2     1 ms     4 ms     4 ms  10.100.1.1

  3    <1 ms    <1 ms     1 ms  10.140.1.10

Trace complete.

Here is traceroute from second site to voip at first site:

C:\Documents and Settings\mhorowitz>tracert 10.40.1.10

Tracing route to 10.40.1.10 over a maximum of 30 hops

  1    <1 ms     4 ms     4 ms  192.168.2.1

  2     1 ms     4 ms     4 ms  10.100.1.2

  3     2 ms     2 ms     2 ms  10.40.1.10

Trace complete.

Another good visual is the following erratic ping to a server from one site to the next:

C:\Users\mhorowitz>ping mhw-fs6 -t

Pinging mhw-fs6.local.mhwltd.com [192.168.2.16] with 32 bytes of data:

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time=145ms TTL=126

Reply from 192.168.2.16: bytes=32 time=69ms TTL=126

Reply from 192.168.2.16: bytes=32 time=268ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time=67ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time=3ms TTL=126

Reply from 192.168.2.16: bytes=32 time=2ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Reply from 192.168.2.16: bytes=32 time=195ms TTL=126

Reply from 192.168.2.16: bytes=32 time<1ms TTL=126

Ping statistics for 192.168.2.16:

    Packets: Sent = 17, Received = 17, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

    Minimum = 0ms, Maximum = 268ms, Average = 44ms

And this is an example of a ping from my workstation to a server at the same site. Server is a domain controller, and does not do much as it is only for about 20 users:

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time=248ms TTL=127

Reply from 10.30.1.4: bytes=32 time=156ms TTL=127

Reply from 10.30.1.4: bytes=32 time=24ms TTL=127

Reply from 10.30.1.4: bytes=32 time=127ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time=236ms TTL=127

Reply from 10.30.1.4: bytes=32 time=143ms TTL=127

Reply from 10.30.1.4: bytes=32 time=34ms TTL=127

Reply from 10.30.1.4: bytes=32 time=141ms TTL=127

Reply from 10.30.1.4: bytes=32 time=29ms TTL=127

Reply from 10.30.1.4: bytes=32 time=28ms TTL=127

Reply from 10.30.1.4: bytes=32 time=226ms TTL=127

Reply from 10.30.1.4: bytes=32 time=133ms TTL=127

Reply from 10.30.1.4: bytes=32 time=1ms TTL=127

Reply from 10.30.1.4: bytes=32 time=8ms TTL=127

Reply from 10.30.1.4: bytes=32 time=66ms TTL=127

Reply from 10.30.1.4: bytes=32 time=23ms TTL=127

Reply from 10.30.1.4: bytes=32 time=222ms TTL=127

Reply from 10.30.1.4: bytes=32 time=128ms TTL=127

Reply from 10.30.1.4: bytes=32 time=20ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

Reply from 10.30.1.4: bytes=32 time=115ms TTL=127

Reply from 10.30.1.4: bytes=32 time=8ms TTL=127

Reply from 10.30.1.4: bytes=32 time=114ms TTL=127

Reply from 10.30.1.4: bytes=32 time=407ms TTL=127

Reply from 10.30.1.4: bytes=32 time=6ms TTL=127

Reply from 10.30.1.4: bytes=32 time=205ms TTL=127

Reply from 10.30.1.4: bytes=32 time=111ms TTL=127

Reply from 10.30.1.4: bytes=32 time=4ms TTL=127

Reply from 10.30.1.4: bytes=32 time=109ms TTL=127

Reply from 10.30.1.4: bytes=32 time=403ms TTL=127

Reply from 10.30.1.4: bytes=32 time<1ms TTL=127

[Mike Horowitz]

Tom,

You were on to something with the Green Ethernet. I disabled it on the SG200 switches, and it looks like things improved with speed and pings. However, the original problem in this post still exists - I can now ping only 1 of the VOIP virtual NICS, and only from my workstation (not others). Still very wierd.''Mike

[Mike Horowitz]

Hi Tom,

Attached is a wireshark of the port of one phone system. ICMP requests from the phones to the NEC box drop. There are also many malformed packets, but I'm not sure if that is accurate. Let me know if you see anything. Thanks again.

Mike

[Mike Horowitz]

Hi Tom,

After working with TAC, we determined it is a bug in the SG500X with it changing the MAC address between vlan's. Bug ID is CSCub82382. I hope this will be resolved soon as I may have to redo network set up in the mean time (not an easy task). Thanks for your help.

Mike

[Tom Watts]

Hi Mike, thanks for the update. I had heard some possible problems when the switch will send a port MAC instead of the system MAC when going between the vlans. It is good to know that IS the cause.

I did send you an email a couple weeks back as well. (Unfortunately I've been out of work since, being ill).

I will look forward to looking up the ID and checking the exact details.

-Tom
Please rate helpful posts

[Mike Horowitz]

Hi Tom,

i hope everything is OK with you. I was given contact info for Marcelo in level 2, but I have not heard anything new from him. I am trying to find out if this issue will be fixed in the next ios update, and when that  will be.

I just ordered and configured 2 HP switches to replace the Cisco's with, and am in the process of reconfiguring my network because of it. I am looking at this as a temporary fix, so I am counting on this issue being resolved quickly.

Mike