Service provider, ARP and WAN

[loiccruchade]

Hello everyone,

I'm currently working as an internet service provider.

I'm facing an issue with ARP request I think.

Here is the diagram :

Each client router has a default route, to their respective gateway.

Client 1 and 2 have 192.168.0.1

Client 3 have 192.168.1.1

Client 1 and 2 are able to ping Client 3, since they use their default route to reach him.

But Client 1 and 2 are not able to reach each other. After some research, i figured they use the directly connected route. So when Client 1 try to reach Client 2, he create an entry in arp table as INCOMPLETE.

If i configure a static route on those two router (ip route 192.168.0.20 255.255.255.255 192.168.0.1 on Client 1, and ip route 192.168.0.10 255.255.255.255 192.168.0.1 on Client 2), then they are able to reach each other.

I can't figure out how to solve this problem without configuring static routes.

Help guys ! 

[Rafael Carvallo]

Hi,

You show a cloud in he middle called orange services, for what I see it's the one providing the L2 services towards your customers, is it yours? if it isn't then:

Depending on the type of service this cloud is offering your customers can't directly reach each other, so I'd start asking the provider about this.

If the service is an E-TREE (which is what I am thinking), means it's a hub-and-spoke topology, your main site being the hub, all traffic reaches this router and hence it being a router won't engage in regular switching activities, remember ARP works at L2. 

You'd need to ask your provider to give you an E-LAN/VPLS service for this to work in the way I suspect you want it to (using the cloud as a big switch for direct connectivity)

HTH
Please remember to rate useful posts 

[loiccruchade]

Thanks for your help Rafael,

The "Orange services" is our provider.

We rent a 500Mb/s connexion, and collect our client. I don't know the right word in english, but we call that a "collecting door".

The way we work is kind of what you are saying. Our backbone is like a big switch where client are collected, then we route them on the internet (and offer other services as inter-site routing, VOIP, virtual server hosting etc...).

I'll have to check with Orange if we are using a service as you descibed ("E-LAN/VPLS"), but they are very slow to awnser...

If it's not the case, how can i configure my routers to use their gateway even if they have a directly connected route ?

Thanks again for your help

[Rafael Carvallo]

That's a little bit tricky, I don't think you can use a feature called proxy-arp in this very specific scenario. The thing is, your devices know they are on the same subnet so they'll refer to any technology/protocol that allows them to discover the L2 address/path, ARP on IPv4, NDP on IPv6. This is basically how TCP/IP is supposed to work.

If your provider is giving you an E-TREE service and for any reason doesn't want to change it to VPLS/E-LAN, then I'd suggest you to change how you're doing things in case you really need customers to reach each other, try assigning /30 subnets to each customer, cumbersome but I think this would be your only option.

[paul driver]

Hello

apart from the obvious that you have two clients on the same subnet  in normal circumstances they wouldn’t need to be routed to see each other but regards your question about 

---

loiccruchade wrote  how can i configure my routers to use their gateway even if they have a directly connected route 

It it sounds like your clients are arping  for the destination and because I guess proxy arp is also disabled its failing 

Are your default routes by any chance pointing to an next hop interface instead of a next hop IP address?

res

paul

[loiccruchade]

Hi guys, thanks again for the help.

@Rafael Carvallo You were right. I've checked our contract with Orange, and we have VPLS service.

Here is the full diagram :

Before getting to the backbone router, those ARP have to pass two switchs.

So my problem is either located on Orange services, or on those two switchs.

How could a switch fail to transmit ARP requests ?

I'm gonna open a ticket to Orange in the same time since they are very long to awnser.

@paul driver, Thanks for your intervention.

Proxy ARP is enabled, we did not modify this setting.

I already checked our default route on clients routers because i suspected the same thing, but those routes use IP as next hop.

Thanks again for your help guys.

[loiccruchade]

Hi again guys,

A little update about my troubleshooting.

I've used arp debug on switch2.

Switch2 is able to receive ARP request from client, and send response to them.

The client does not receive the ARP response because Orange filter mac adress. You have to configure a virtual mac adress in a specific range.

When I look at my backbone router, a virtual mac adress is configured on the physical interface.

So each sub-interface have this virtual mac adress.

We have many sub-interface. One for each delivered vlan by Orange. So this is why i'm able to ping from my backbone router to a client.

I've read the access specification document again.

It specify that we should use a different virtual mac adress for each vlan.

Actually, we don't, and Orange seems to be fine with it.

But if I would respect this document and configure my backbone routeur to do so, how could I since I can't configure virtual mac adress on sub-interface or vlan interface ?