IWAN Spoke with very uneven transport bandwidth. WWW access issues

[Matthew Needs]

Hi Eveyone,

I have a Spoke site running IWAN which has very uneven upstream transport bandwidth.. So for example,

1x DSL (MPLS) - 400 Kbps UP, 5 Mbps DOWN. (Preferred Private Circuit)

1x 4G (INET) - 10 Mbps UP, 25 Mbps DOWN.

IWAN is behaving as configured, controlling various private flows and moving them between circuits when SLA is exceeded. The spoke is also performing load sharing for uncontrolled traffic by default such as public WWW. And it's noteworthy that the site is upstream bandwidth hungry mainly for WWW traffic.. (Dropbox, O365 etc)

The problem I have is that because of the huge difference in available upstream bandwidth it's a bit of a lottery as to the performance of uncontrolled WWW traffic.. Some users uncontrolled flows experience superior 4g and others experience the slower MPLS (which is often not fast enough) and the flows are stuck there due to the load sharing algorithm..

I'm guessing my only way around this may be to spec a better MPLS circuit or force WWW traffic out of the 4g without load sharing? The only trouble is that changing the load sharing would be global on the MC and therefore effect load sharing on all spoke sites. I also thought of using a private upstream WWW proxy so I can control WWW traffic..

Anyway, I just thought I would put this one out there for the masses to comment on :)

Thanks in advance for any suggestions!

Matt

[vamikhai]

Hello.

As I understood, your design allows PFRv3 to control Internet traffic. If Internet traffic-class is controlled by PFR - it means overall upstream bandwidth should not be exceeded.

And now the question is - what is the "bandwidth" per Tunnel. In your situation it should be "bandwidth 400" (DSL) and "bandwidth 10000" (MPLS) along with respective QoS shaping configuration.

If bandwidth+QoS configuration is correct - please elaborate what do you mean by slowness - it is latency or loss?

If it's about latency on MPLS link - you may need to tune QoS configuration; if it's about loss - you would need to check if PFRv3 does correct decision.

Best regards

[Matthew Needs]

Hello, thanks for the response.. At the moment internet traffic is not controlled but I have been considering options to allow me to do so (such as a private upstream proxy). So the problem is experienced by end users while internet traffic in not controlled.. Some flows choose DSL (400Kbps) and others choose 4G as per standard IWAN load sharing. My problem is that 400Kbps upstream is not always fast enough for some users.. So I would like to confirm - Do I have any other options other than upgrading the MPLS circuit or forcing the internet via the 4g? Assuming I cant control internet traffic of course.

Kind Regards   

[vamikhai]

Hello, Matthew.

Sorry - I didn't understand the situation.

From one hand you say "internet traffic is not controlled" (so no PFR intervention), from the other hand you say "per standard IWAN load sharing".

Could you please clarify - do you see traffic-classes for Internet traffic and what "bandwidth" and QoS is configured on the tunnels?

[Matthew Needs]

Hi Vasilii,

No problem it may be that i'm not explaining or understanding correctly?.. I have the following configured within the default IWAN class on my MC..

load-balance advanced

Yes I am seeing TC's for internet traffic but they are un-controlled and therefore part of the default class.. For example.

Dst-Site-Prefix: 54.194.169.0/28 DSCP: default [0] Traffic class id:19948
Clock Time: 16:06:39 (BST) 11/14/2017
TC Learned: 00:09:53 ago
Present State: UN-CONTROLLED at border X.X.X.X (An attempt to control will be made 00:00:27 later)
Destination Site ID: Internet
Class-Sequence in use: default
Class Name: default
BW Updated: 00:04:36 ago
Route Change History:

My understanding was that while not controlled, the above internet traffic would be distributed across the two transport types in a round robin fashion on a per flow basis? But i guess from your comments that is not the case and would further explain why the issue persists.. I have set the tunnel bandwidths to 400Kbps & 10Mbps. And I have also set per tunnel QoS shaping at the hub BR's to mirror.

At the moment i don't see an easy way to 'control' internet traffic? Other than adding a proxy server so that internet traffic will always be sent to a private IP address..

Thanks for your input

Matt

[vamikhai · Accepted solution]

Hello.

Thank you for the details.

Now I understand, that you have configuration for load-balancing, but Internet traffic class is uncontrolled.

"Un-conftrolled" means PFRv3 does not affect it -> traffic is being forwarded per routing table.

Now we may try to understand why the traffic-class is un-controlled (debug domain master pdp) or change your routing to move best RIB path to Internet circuit.

[Matthew Needs]

Thanks for the confirmation.. That makes total sense. I think the reason why public internet traffic is uncontrolled is because public ip addresses are not part of the site-prefix or enterprise-prefix under the HUB/MC policy. Could we perhaps use a private proxy server so that end users internet access is always to a single private ip address as far as IWAN is concerned? And therefore we could easily control the traffic via IWAN..

[vamikhai]

Hello.

PFRv3 won't control traffic if it's covered by enterprise prefix and not covered by site-prefixes.

It's still a question why internet traffic-class is not controlled and you may need to investigate this (debug master pdp).

Regarding the proxy - it might be a good idea (taking into consideration security concerns), but not mandatory from iWAN perspective.

[Matthew Needs]

Hi Vasilii,

Sorry for the slow reply I have been on leave.. Here is an example debug FYI. I believe this confirms things (IP's removed). Public traffic is following the standard RIB.

*Nov 24 09:29:17.434: CENT:MC:PDP:[0]:Backoff Timer timeout for TC id:9933. Goto pickexit
*Nov 24 09:29:17.434: MC-PDP:[0]:TC[9933, 255.255.255.255{ipv4}, PubIP@{ipv4}, 0x0, 0]: creating candidates for TC id: 9933
*Nov 24 09:29:17.434: MC-PDP:[0]:TC[9933, 255.255.255.255{ipv4}, PubIP@{ipv4}, 0x0, 0]: PDP sorting active/standby for Prefix PubIP@/28
*Nov 24 09:29:17.434: MC-PDP:[0]:TC[9933, 255.255.255.255{ipv4}, PubIP@{ipv4}, 0x0, 0]: Protocol is same and its EIGRP or BGP at this stage
*Nov 24 09:29:17.434: MC-PDP:[0]:TC[9933, 255.255.255.255{ipv4}, PubIP@{ipv4}, 0x0, 0]: Perfect channels not available for this TC
*Nov 24 09:29:17.434: MC-PDP:[0]:TC[9933, 255.255.255.255{ipv4}, PubIP@{ipv4}, 0x0, 0]: best of violated rc: has no policy
*Nov 24 09:29:17.434: MC-PDP:[0]:TC[9933, 255.255.255.255{ipv4}, PubIP@{ipv4}, 0x0, 0]: no exits found. goto uncontrolled

I.e. public traffic is not controlled, I believe due to the following reasons.

lack of parent route for PubIP@/28
lack of PubIP@/28 site-prefix within MC policy

So lack of the above means no controlled traffic-class and therfore no IWAN.. A default route will obviously not match as a parent route as it will not be an exact prefix match.

So in order to control un-proxied internet traffic via a centralised firewall (I.e. User Packet = IP SRC - Private IP@ / IP DST - Public IP@). I'd need a specific IP route to each public IP prefix which could be tricky to impliment and manage. Not to mention the potential issues associated with upstream (Public Cloud ISP) loss/issues effecting private IWAN convergance.

Would you agree with that?

Kind Regards

Matt

[vamikhai]

Hello.

Internet prefixes are not expected to be in site-prefix DB, so I doubt it's the case.

To troubleshoot parent-route please collect:

• "show domain ... master traffic-class dst-site-pfx <affected traffic class> detail";
• "show domain ... border parent" + "show domain .. border channel parent";
• "show ip eigrp top <prent prefix>" or "show ip bgp <parent prefix"from the BR device;
• "show ip route secondary" from the BR device.

PS: feel free to unicast me the data, as it may contain sensitive information.

[Matthew Needs]

Hello,

Thanks for the continued support. I have sent you the required debugs privately.. Looking forward to your thoughts.

Kind Regards

Matt

[Matthew Needs]

For completeness of this post.. After confirmation of my IWAN understanding in this scenario (thanks Vasilii!) I setup a lab of the topology so I could make on the fly changes. I then found that a recursive default route will stop IWAN from controlling public TC's.. Which in turn means no IWAN load sharing as traffic follows the standard RIB.

So if you want IWAN to control public IP TC's for load balancing purposes you'll need to use a standard static default route at the spoke or redistribute 0.0.0.0/0 down from the hub via EIGRP/BGP.

Matt