topic Hi Paul, in Wireless

Devices "hiding" behind a bridge

Paul Taylor — Sun, 04 Jul 2021 05:17:17 GMT

At intermittent times, we will have a single device start to "hide" behind a 1242 bridge.

In the past, my team would reload or reboot the bridge, and everything would come back up.

But the 90 seconds of lost data were causing problems.

So, I began to dig deeper.

I found the the WB was always available.

So I would ping the hidden device from the WB, and it would ping.

Next, I showed the ARP table, and got the MAC address of the hidden device.

Then, I would start a ping from my PC, and leave it running. Constant "Destination Unreachables" from the router.

Next, I showed the bridging table for F0, and sometimes the device would be there, sometimes not.

Regardless, I would add a forwarding bridging entry manualy, and exit conf.

Showing the briding table would show the device, with a "P"ermanent age, but still no pinging.

Finally, I would remove the forwarding briding entry, and pings would start within a couple of seconds.

During these periods, outbound UDP packets will continue to be forwarded from the PC.

So, my managers want to convert everything to UDP, which is a potential disaster in my opinion.

But if I can't get this fixed, I have no alternative.

Can anyone suggest where to look?

Here is a config:

Crane01wb#show run

Building configuration...

Current configuration : 1924 bytes

version 12.3

no service pad

service timestamps debug datetime msec

service timestamps log datetime msec localtime show-timezone

service password-encryption

hostname Crane01wb

enable secret 5 RandomValuesHere

ip subnet-zero

no aaa new-model

dot11 ssid somessid

authentication open

dot11 arp-cache optional

power inline negotiation prestandard source

username admin privilege 15 password 7 RandomHexValuesHere

bridge irb

interface Dot11Radio0

no ip address

no ip route-cache

encryption key 1 size 128bit 7 RandomHexValuesHere transmit-key

encryption mode wep mandatory

ssid lanewwlan01

speed basic-1.0 basic-2.0 basic-5.5 6.0 9.0 basic-11.0 12.0 18.0 24.0 36.0 48.0 54.0

station-role workgroup-bridge

antenna receive right

antenna transmit right

antenna gain 0

bridge-group 1

bridge-group 1 spanning-disabled

interface Dot11Radio1

no ip address

no ip route-cache

shutdown

encryption key 1 size 128bit 7 randomHexVBaluesHere transmit-key

encryption mode wep mandatory

ssid somessid

dfs band 3 block

speed basic-6.0 9.0 basic-12.0 18.0 basic-24.0 36.0 48.0 54.0

channel dfs

station-role root

bridge-group 1

bridge-group 1 subscriber-loop-control

bridge-group 1 block-unknown-source

no bridge-group 1 source-learning

no bridge-group 1 unicast-flooding

bridge-group 1 spanning-disabled

interface FastEthernet0

no ip address

no ip route-cache

duplex auto

speed auto

bridge-group 1

bridge-group 1 spanning-disabled

interface BVI1

ip address 10.1.10.8 255.255.255.0

no ip route-cache

ip default-gateway 10.1.10.1

ip http server

no ip http secure-server

ip http help-path http://www.cisco.com/warp/public/779/smbiz/prodconfig/help/eag

logging history debugging

logging 10.1.95.250

snmp-server community private RW

control-plane

bridge 1 route ip

line con 0

line vty 0 4

end

Devices "hiding" behind a bridge

Saravanan Lakshmanan — Tue, 12 Jun 2012 00:16:49 GMT

#enable arp caching on wgb

#add static arp entry on respective infrastructure devices

Devices "hiding" behind a bridge

George Stefanick — Tue, 12 Jun 2012 00:41:53 GMT

Paul,

What network are you commecting this WGB to ? A cisco WLC, if so is "passive" client check boxed?

Devices "hiding" behind a bridge

Paul Taylor — Wed, 13 Jun 2012 15:47:43 GMT

If this is an ARP problem, then I would expect the WGB to also have problems.

Yet it can ping the device just fine, and it has the MAC address in its ARP table (as stated in the post).

Adding static ARP entries is not an optimum solution for me (and, technically, contrary to the purpose of ARP), because devices get changed out frequently in my environment, creating a network managment problem.

I'll do it as a last resort, but I'm really hoping for a better solution than that.

Also, please define "respective infrastructure devices".

Devices "hiding" behind a bridge

Paul Taylor — Wed, 13 Jun 2012 15:52:59 GMT

The WGB is in a "gantry crane", 125 feet off of the ground.

The WGB has a Netgear 8 port unmanaged switch connected to it, to which the device in question is attached, as well as a few others.

Also, this problem only affects one of those devices.

We have an inhouse device also connected, that we can use to remotely cycle to power to the WGB or other devices if required.

That unit is not affected by this (so far).

The WGB talks to APs, on 100' light poles, that are controlled by a WLC.

I've poked around on the WGB as well as the WLC, but I cannot find a '"passive" client check box'.

Can you point me towards where to look for it?

Thanks!

Devices "hiding" behind a bridge

George Stefanick — Wed, 13 Jun 2012 17:20:23 GMT

Paul,

On your WLC, under the WLAN (SSID) there is an advance tab. Here is where passive is marked. Also I am checking on a few other items. But this sure sounds like the passive client issue.

Devices "hiding" behind a bridge

George Stefanick — Wed, 13 Jun 2012 17:23:40 GMT

Also ..

Passive Clients Behind a WGB

The controller might not be able to see passive clients behind a WGB. Clients (such as cameras and programmable logic devices) do not initiate a traffic stream unless they are connected. Complete these steps in order avoid this issue:

Add a static MAC filter entry for the passive WGB device and MAC filter entry for the devices that are behind it.
Use this command in order to enable MAC filtering on the WLAN along with aaa override:
config macfilter ipaddress MAC_address IP_address
Add a static entry on the WGB IOS-based device: bridge 1 addressxxxx.xxxx.xxxx forward FastEthernet0
Note: In addition, increase the dot11 activity timer.
Add a static ARP entry on the L3 router:
```
hostname(config)#arp  
      arpa
```

This feature allows the controller to learn the IP address of a passive WGB wired client when the WGB sends an IAPP message to the controller that contains only the MAC address of the WGB wired client. When this message is received from the WGB, the controller checks the local MAC filter list or, if the WGB has roamed, the MAC filter list of the anchor controller for the MAC address of the client. If an entry is found and it contains an IP address for the client, the controller adds the client to the client table of the controller.

Unlike the existing MAC filtering feature for wireless clients, you are not required to enable MAC filtering on the WLAN for WGB wired clients. WGB wired clients that use MAC filtering do not need to obtain an IP address through DHCP to be added to the client table of the controller.

Devices "hiding" behind a bridge

Paul Taylor — Wed, 13 Jun 2012 17:53:24 GMT

I have considered this, but discounted it for the following reasons:

a) I have both outbound TCP and UDP streams that are active when it stops working. So this is not a passive client.

b) I have passive clients on the same WGB that still work when this problem presents itself.

If this is nothing more than a passive client problem, then I can just implement a startup script that start a never ending ping on the client to a well known IP address past the WGB, and the problem should clear up, correct?

But, I have a socket application that runs as a windows service, that connects to a server past the WGB and sends a continuous stream of data.

This is better than a continuous ping, and already makes it an active client, as opposed to a passive one.

I will try these tips, just to see, but my hopes are not high.

Thanks for your input, I'll post the results (though it is intermittent, and may be a few days or more before I see it again).

Regards,

Paul

Devices "hiding" behind a bridge

Paul Taylor — Wed, 13 Jun 2012 18:00:02 GMT

OK, I cannot find anything that matches your notes.

To clarify, by WLC I mean a Wireless LAN Controller.

It is a Cisco 4402.

Across the top I have the following options:

MONITOR WLANs CONTROLLER WIRELESS SECURITY MANAGEMENT COMMANDS HELP

Under WLAN, there is nothing for SSID.

There is an Advanced option to the left, it only gives access to AP Groups.

Are we on the same page?

If so, what am I missing?

Thanks again for your time,

Paul

Devices "hiding" behind a bridge

George Stefanick — Wed, 13 Jun 2012 18:01:17 GMT

Ya, a active ping will keep the app a live for sure. It doesnt sound like a passive issue based on your comment. Does your application use multicast ?

Cisco sends multicast at the highest data rate, if your client is not in the highest PHY rate your client will not get it, meaning your WGB. This will break the application that uses m/c.

Devices "hiding" behind a bridge

Saravanan Lakshmanan — Wed, 13 Jun 2012 19:39:05 GMT

isolate the application & particular wired PC/ASD issue:-

what vlan/subnet the wgb and wired pc is on i.e., management/dyn of wlc.

windows based pc are usually chatty and active, are you using handheld/Application specific device.

check the driver on the affected unit with good working units.

do you see the affected wired client under Monitor that is still associated to the wgb when the issue occurs.

when the issue occurs are you able to ping that pc from wlc.

is WLC ignoring that affected wired client only, what's the status of other wired client from same wgb when issue occurs.

what do you see in wireshark trace taken from affected PC when the issue occurs. confirm syn ack for tcp.

WLC>debug client

Note: 4400 doesn't have passive client option like 5500. For passive client issues with 4400, you need to use MAC filtering on 4400, higher arp timeout/static arp entry on intermediate devices between wgb to WLC & L3.

Devices "hiding" behind a bridge

matt — Thu, 05 Jul 2012 21:00:39 GMT

Well this is really helpful. Something must have cahgned recently as this wasn't a problem unitl we upgraded our WCS code from 6.x to 7.x. I did run into problems on step 2. The CLI kept giving me an error so I added the macfilter via the GUI under the Security tab.

We have medical (PACS) devices that connect via WGB and they use static IPs. They had been working for years until we upgraded. Now I'm adding static ARP entries and the rest of this mess .

If anyone can point to the change that caused this in the release notes, I'd appreciate it. I'm going to go hunt around... If I find it i'll post it here.

Devices "hiding" behind a bridge

Paul Taylor — Fri, 06 Jul 2012 15:20:41 GMT

Saravanan,

thanks for a lot of good tips. There's a lot there to absorb and apply.

First, we are not in a VLANed environment, for a large number of reasons.

A quick description of the environment:

We have very large cranes that work cargo ships.

We also have medium size cranes that straddle rows for containers 7 wide and 5 high (45' high).

Both of these environments exhibit the problem.

We have a variety of equipment, from in-house built PIC based ethernet devices to windows and linux based servers and PCs, and also IP cameras.

The software running on the servers and PCs is home grown, and typically has either a UDP based 1hz output, or a TCP based client/server (these PCs being the server side) application.

Both the UDP and TCP applications, as well as the cameras, are "always on", although the cameras are HTTP based, and so don't stream unless they have a consumer.

The TCP application also only streams when there is a consumer, but an application in the computer room is always on, guaranteeing a consumer 24x7 if the equipment is on.

We NEVER lose connectivity from the UDP devices during these problems.

Also, we have a PIC based power switch that, even when the windows PC gets lost, is still available; we can connect to it via HTTP, and power cycle the WGB, which corrects the problem.

However, that creates a 90-120 second gap in telemetry (the purpose of the entire infrastructre), and causes a loss of data that is detrimental.

Yes, there is a lot we can do, programmatically, to cache this information, but I'd rather just fix the network so that the problem doesn't exist.

Point being, I wouldn't be surprised at this happening to the cameras, because they don't create traffic unless connected to first.

But the PCs are always on, and always transmitting data.

Second point being, physical access to these environments, while they are in operation, is difficult at best (and impossible in the smaller cranes).

So, since I can't access them remotely due to the problem, I can't start a wireshark session to capture the data.

And leaving wireshark always on creates a performance problem of its own.

You can see the catch-22 situation I'm in.

WGB can ping the PC, just nothing past the WGB.

And the PC is in the ARP table of the WGB before and/or after the ping, but that never fixes the problem.

Only adding static bridge forwarding entries, and then removing them, fixes the problem (or a reboot).

Never tried pinging it from the WLC, I'll try that next time.

If I'm ever lucky enough to have this happen when my equipment is not in a production environment, I'll get wireshark going. I already have it loaded, just never have an opportunity.

L3 is a home-built Linux-based router.

The only devices between "wgb to WLC & L3" would be my 3Com switches, which only work at L2 (as far as I know).

I've never changed anything related to ARP/MACs there.

Again, I've never really focused my attention there, since a bridging table change on the WGB fixes the problem.

Thank you for your input, I have more places to do some research now.

Hi Paul,

ddelange — Wed, 13 Apr 2016 13:03:54 GMT

Hi Paul,

i'm currently experiencing a similar issue like you described 4 years ago and i'm curious if you ever found the solution? Looking forward to your reply.

Thanks!

/daniel