Getting Broadcast traffic from one 3745 to another

[dturley]

The topology is simple. Three 3550 switches as the backbone tied together using spanning-tree layer 2 wire speed switching. Very simple stuff there. Introduce 3745 access routers, one attached to each 3550, each loaded with 16port ESW, 1 GigE GBic card, and a 8A/S card.

The problem is we have systems that blow out broadcast traffic that needs to traverse accross all 16-ESWs. We have tried all manor of things but we can not get broadcast traffic to traverse the 1GE port. We can see packets hitting the interface but they are simply getting dropped on the floor.

I can go into more detail if needed but we think we're missing a painfully simple detail. Perhaps something to do with L3 and L2? Perhaps something to do with bridge groups or vlans or helper protocols?

Any wisdom to help us out would be greatly appreciated!

[thisisshanky]

Doensnt the GBIC card go into the NM-16ESW ? If yes put all of the ports of ESW that need to see the broadcast traffic in same vlan and trunk it via 802.1q trunk to the 3550 switch. If all switches have this same vlan defined and also on the ESW ports, they all should see this traffic.

HTH

[dturley]

No, the GBIC goes into the NM-1GE. The NM-16ESW is just a 10/100 Ethernet switch module.

The 3750 isn't a Catalyst so it appears to behave slightly different than a switch. The NM-1GE insists operation at L3 only. We trunked the 16ESW ports together with a default route to the 1GE interface and we *have* been able to get pings, tcp, and unicast udp traffic to flow.

We made up virtual a interface for the 1GE and trunked that to the 16ESW vlan and still have not been able to pass Broadcast traffic.

We can clearly see the traffic is not leaving the 3745 by looking at the packet counters.

We even have taken a flip cable from the 16ESW and plugged directly into the 3550 and that will pass Broadcast just fine. So the backbone is good, the 16ESW in a fundamental ways is good. The snag is traversing the 1GE. Argh!!

[1csscctdc]

I'm not quite sure what you are asking, but let me see if I got this right.

Your configuration is:

|switch|---|router|---|switch|---|router|---|switch|

1.2 1.1 2.1 2.2 2.3 3.1 3.2

Where 1.2 = IP address x.x.1.2 subnet mask 255.255.255.0

OR

Is your configuration:

x.x.1.4 x.x.1.5 x.x.1.6

|switch|-------|switch|-------|switch|

| | |

| | |

x.x.1.1 x.x.1.2 x.x.1.3

|router| |router| |router|

x.x.2.1 x.x.3.1 x.x.4.1

| | |

|clients| |clients| |clients|

Im assuming it is the second config shown here, as that seems to make the most sense from what I gather.. and your clients send broadcasts that need to traverse from network x.x.2.0 to x.x.3.0 etc.

In order for that to work I believe you would need to set IP helper addresses on the client side interface for the other two routers.

Example:

router1#config t

router1(config)#int fa 2/1 <----client side port

router1(config-if)#ip helper x.x.1.2

router1(config-if)#ip helper x.x.1.3

and all routers would need a similar config.

What Im un-sure about is the EtherSwitch Module, I've never used one, so I'm not familiar with the method of setting the IP address to that side of the router. Maybe use a loopback address?

I hope this helps or leads you to a solution, if I misunderstood your setup, I'll try again :)

[1csscctdc]

Sorry the text drawn network didnt come out right, I hope you get the idea...

[dturley]

Thanks, yes your second description is more accurate. I've included a block diagram. Hopefully you can see it.

We tried using ip helper but we think we missed something subtle. The broadcast traffic isn't getting across the 1GE interface. That interface only wants to be L3. The 16ESW is L2. We've tried creating a virtual interface on the 1GE and trunking that to the 16ESW. No good either.

Must be some minute detail we're missing. I suppose the next step is to post our config.

[1csscctdc]

For my own curiosity, are you confusing broadcast with multicast?

And why even have the routers in place? Why not just connect the users directly to the 3550's?

Still trying to come up with a solution.

[lnguye]

Perhaps, your devices are using Ethernet SNAP frames which the 3750 does not recognized. The work around: set the static MAC addresses of the devices using SNAP frames in the 3745.

Good luck.

[dturley]

I do not believe so. We are sending traffic to 255.255.255.255 or x.x.x.255.

We can send unicast UDP. We don't have a utility to test multicast UDP. Not sure if IPerf can do that.

Believe me, I would love to see our software gurus change the software from broadcast to multicast. We could easily route that now and be done.

The 3550s are in core of the backbone network which is located in buildings many hundreds of feet apart from the 3745s. Sure, we could put in fiber transcievers but that is not a very satisfying design.

We went with the 3745s because of the potential to use the Async/Sync serial cards in the same chassis. Hence, the access routers at the end points.

We simply didn't expect to encounter such an annoying snag with the 1GE interface.

I appreciate your and anybodies input. Find a solution, come to the Eastershore of MD and I'll have a brew waiting for you!

[Richard Burts]

Dwayne

I have looked at your block diagram and have a couple of questions. I see addresses 192.168.100.10, .20, and .30. Do I assume that they are the 1GE interface addresses? I also see 10.0.0.1, .2, and .3 which I assume are the management addresses on the 3550s. I am not clear where the 16ESW fit into this. I do not have a clear understanding of what VLANs exist where. Perhaps you could clarify this?

But despite my lack of clear understanding of what you have there I do have guess at the issue you are trying to identify. I believe the main issue is that the 1GE acts as a layer 3 interface. And in Cisco routers a layer 3 interface does not forward broadcasts. You probably already thought about this, since you mention that you have tried to do helper addresses.

I would like to know a little more about what you have done with helper addresses. I am guessing that you have have configured the helper address to forward to the subnet broadcast address. But for this to work correctly it functions as a directed broadcast. And directed broadcast is disabled by default. So on the layer 3 interface which will receive the forwarded frame you need to configure ip directed-broadcast.

HTH

Rick

[lnguye]

rburst is correct that we do not have a firm understanding of the configuration. Do you really want to place the L3 interface in the middle of where you want broadcast to traverse? If for some reason you do, have you tried IP helper-address along with the global command 'ip forward-protocol xxxx' to help push the broadcast packet over a routed interface? If L3 is not needed, have you tried to set the GE as switch port (either as access or trunk)?

[dturley]

Appearently, the GE (NM-1GE) will not allow us to use swith port features. When attempting to do so the response is something like 'feature not supported'. So we are stuck with an L3 interface unless there is something really peculiar we haven't tried. The 3745 isn't a Catalyst and has some new quirks.

Refer to my earlier post about the ip forward-protocol. We turned the helper addressing on yet that also did not traverse the GE interface. That could be an implementation issue on our end, we just don't know yet.

We opened a support ticket, see if Cisco can get us over the hump.

[dturley]

Rick,

Your are correct in your assumptions. That block diagram was perhaps a bit too quick and dirty.

The 10.0.0.x addresses are management IPs on the 3550s. The 192.168.100.x addresses are the 1GE interface addresses. Those are not exactly what we have on the actual equipment but capture the current testing environment layout. The idea is to keep this simple as possible.

We've been looking at a cisco article which right now I can't seem to locate. However, here is a similar article that captures what we think would work:

http://tolikus.hq.aaanet.ru:8080/techdoc/CiscoCookBook/ciscockbk-CHP-23-SECT-14.htm

I will reverify that we had directed broadcast enabled. I believe we did. I know that it is disabled by default.

The question is why did the broadcast helper not work? Is the helper to go on the layer vlan with the 16ESW ports such that the broadcast will be multicast when presented to the 1GE? We tried it both ways. I can reverify that also.

[Richard Burts]

Dwayne

As you probably already know, the helper-address is configured on the interface that receives the broadcast to be forwarded. So if the broadcast source is in a 16ESW then I would expect the helper address to be configured on whatever interface (probably virtual) repersents the layer 3 interface for those layer 2 ports.

The function of helper address is that it takes a broadcast packet and forwards to some destination address. The general assumption is that the destination address will be unicast. The destination address can be a subnet broadcast (directed broadcast) and I assume that this is what you are trying to do. Is this correct? If so then be sure that you have ip directed-broadcast enabled on the interface where the destination subnet is located.

Another potential issue is identification of broadcast packets to be forwarded. Helper address is not intended to forward ALL broadcasts. There is a group of protocols that are enabled by default (DHCP, TFTP, etc). If the broadcast packets that you want to forward are not one of these default protocols then you need to use the ip forward-protocol udp command which would be configured on the interface receiving the initial broadcast (the same interface as the helper-address).

You probably have these already. But I can not find a good description of what is configured where and thought that a review of these principles might be helpful.

It probably would be quite helpful to post configs of at least one 3745 and also its associated 3550. If you do not want to post these on the forum please feel free to EMail them to me. My EMail address is available from my forum profile.

HTH

Rick

[Richard Burts]

As I re-read the thread a few additional questions occur to me.

I assume that these broadcasts that you want to forward are UDP. (If they are TCP we have a whole different problem to solve.) What UDP port number are they?

From your block diagram it appears that all of the 1GE interfaces are in a common subnet. Is this correct?

What about the 16ESWs - are they all in a common VLAN or is there a separate VLAN for each device?

Which translates to an IP addressing and subnetting question: are all 16ESWs is the same subnet or is there a unique subnet for each device?

I am assuming that you are using the 1GE port on the 3745 as the uplink to connect to the 3550. Is that correct?

Are the broadcast sources on the 16ESW, on the 3550, or on both? (You would need an ip helper-address on each layer 3 interface where there was any broadcast source.)

Where are the helper addresses configured? And what is the destination address specified in each helper address?

I am not sure whether the issue is an implementation detail or reflects some problem in the design. Perhaps if we saw config files it could help sort this out.

HTH

Rick