12-13-2024 01:25 AM
Hi,
Been out of the Cisco config game for quite a few years now so I'm super rusty on this and needing a little help :)...
Im swapping out an edge router for a client who sub-lets some bandwidth/IP addresses on their WAN to tenants within their office building.
The old edge router (2951) was using the old "rate-limit" statements on the dot1q vlan sub interfaces e.g.
interface GigabitEthernet0/2
no ip address
duplex auto
speed auto
interface GigabitEthernet0/2.900
description *Tenant 1 WAN*
encapsulation dot1Q 900
ip address xxx.xxx.xxx.57 255.255.255.252
rate-limit input 100000000 5000000 5000000 conform-action continue exceed-action drop
rate-limit output 100000000 5000000 5000000 conform-action continue exceed-action drop
no cdp enable
!
interface GigabitEthernet0/2.901
description *Tenant 2 WAN*
encapsulation dot1Q 901
ip address xxx.xxx.xxx.49 255.255.255.252
rate-limit input 30000000 5000000 5000000 conform-action continue exceed-action drop
rate-limit output 30000000 5000000 5000000 conform-action continue exceed-action drop
no cdp enable
I need to migrate this config to something more modern as the replacement router (a 4451 running IOS-XE 16.09.06) doesn't take the interface level rate-limit commands so I guess I need to go down the policy/class-map route - which I never originally got into back in the day!
So the setup and goal is as follows....
Router has 3 active interfaces
Gi0/0/0 - Outside WAN (to the ISP NTU) - 1Gbps ISP bandwidth
Gi0/0/1 - Customer Firewall - all remaining bandwidth outside of the below is theirs
Gi0/0/2 - Tenant Firewalls (dot1q VLAN Sub interfaces per tenant) - each of these will have allocated, 30, 50, 100, 200mbps out of the 1Gbps pool, essentially whatever bandwidth they pay for (and as mentioned above, anything left over is for the use of the customer on Gi0/0/1).
The tenant bandwidth allocations needs to be a hard limit, nothing over the bandwidth they pay for can be allowed.
The "Customer", (Gi0/0/1) ideally should be able to burst into available bandwidth if tenants aren't using it - this wasn't in place previously (or at least, no specific config was - Gi0/0/1 had no configuration on it for rate limits or policy before), this isn't essential, but desirable if its possible?
Ive always been one of those engineers that looks at an existing config and thinks how he can adapt it to his needs - but the qos commands just melt this old guys brain!
Thanks in advance for any advice
Paul
12-13-2024 07:23 AM
I am a visual learner: diagram with directions of the rate limited traffic flows available? That is, you are wanting to allocate the 1Gbps g0/0/0 ISP traffic (ingress & egress) to the firewall interfaces/subinterfaces on g0/0/1 and g0/0/2?
12-13-2024 07:38 AM
I'll try and do a diag to help (tbf I'd prefer having one)
To set the scene - My client lease space in their office along with internet connectivity to tenants.
ISP internet feed (1gbps) into g0/0/0 and then we have our clients own FW into g0/0/1 and their tenants FW's are into subinterfaces on g0/0/2
There is no NAT going on - the router is just routing/bridging the WAN IP's to the firewalls
Tenants have been provided with set bandwidth allocations that need to be honoured via QoS (e.g. 30mb, 100mb)
My client then has all remaining bandwidth to themselves (but equally, I'd like to allow my client to burst into the tenant allocations when the tenants arent using it, but as soon as a tenant needs bandwidth, this would take priority over the burst)
Does that make sense?
I'll try and diag it up for you though
12-13-2024 08:27 AM
This is about as rough and ready a diagram I could muster - but all traffic flows are in both directions - no specifics in terms of voice priority etc.... just the basic bandwidth allocations is all Im concerned about - what the tenants do with their allocation is up to them
12-13-2024 08:50 AM - edited 12-13-2024 10:00 AM
interface GigabitEthernet0/2.900
description *Tenant 1 WAN*
encapsulation dot1Q 900
ip address xxx.xxx.xxx.57 255.255.255.252
rate-limit input 100000000 5000000 5000000 conform-action continue exceed-action drop
rate-limit output 100000000 5000000 5000000 conform-action continue exceed-action drop
becomes something like:
policy-map 100Mbps
class class-default
police 100000000 5000000 5000000
interface GigabitEthernet0/2.900
description *Tenant 1 WAN*
encapsulation dot1Q 900
ip address xxx.xxx.xxx.57 255.255.255.252
service-policy input 100Mbps
service-policy output 100Mbps
12-13-2024 12:52 PM
Thank you Joseph!
Thats kinda of what I had in my head, I just wasnt sure if its all I needed (or tbh the structure of what goes where :D)
Ive seen examples where I could add the conform-action and exceed action into the policy map/classes - so I think thats straight forward enough.
Would this alone (obviously repeating for the other sub int's that need different bandwidth thresholds) and not applying any policy to the "Client FW" interface achieve the goal of allowing trhe client to use any and all available bandwidth (including borrowing from the tenants bandwitdh when not in use - but as soon as tenant needs that bandwidth they get it through the policing rule?)
Thanks again for the tip! - Will update with marking solution once I get the commands in place and happy with it
12-13-2024 01:57 PM - edited 12-13-2024 02:11 PM
@paul-adam wrote:
Ive seen examples where I could add the conform-action and exceed action into the policy map/classes - so I think thats straight forward enough.
I believe the policer's conform and exceed defaults match your rate-limit settings.
@paul-adam wrote:
Would this alone (obviously repeating for the other sub int's that need different bandwidth thresholds) and not applying any policy to the "Client FW" interface achieve the goal of allowing trhe client to use any and all available bandwidth (including borrowing from the tenants bandwitdh when not in use - but as soon as tenant needs that bandwidth they get it through the policing rule?)
Via policing alone, or policing across multiple interfaces, don't know how it might be done.
However, on a single interface, using multiple classes it might be done. If the total is not the physical interface's bandwidth, you'll need to use a hierarchal policy where the parent shapes.
Also, with CBWFQ, the client class cannot be totally starved, so, if being used, it will "steal" some bandwidth from the tenants. One way to avoid that, is have minimum amount of bandwidth that the client will always "steal" incorporated into the overall aggregate. This would insure the tenants will obtain their allocated bandwidths.
BTW, for egress, shaping can still limit bandwidth, but it queues excess which might be able to be transmitted with a delay.
Cisco compares the two as:
Traffic policing propagates bursts. When the traffic rate reaches the configured maximum rate, excess traffic is dropped (or remarked). The result is an output rate that appears as a saw-tooth with crests and troughs. In contrast to policing, traffic shaping retains excess packets in a queue and then schedules the excess for later transmission over increments of time. The result of traffic shaping is a smoothed packet output rate.
The next diagram illustrates the key differences between the two traffic options.
12-14-2024 02:47 AM
Hello Paul
See attached for a possible solution..
12-14-2024 04:21 AM
@paul driver "possible solution", as I understand @paul-adam's "ask", I believe, isn't the solution, but he's certainly on the correct track, although the track may be a dead end
I don't have time, at the moment, to further explain, but I expect later today I will have that time.
12-14-2024 11:02 AM - edited 12-14-2024 06:58 PM
@paul driver wrote:
See attached for a possible solution..
Again, much of what Paul suggests is not "wrong", but perhaps more a question of style or taste.
For example, consider:
class-map match-any tenent1_cm
match any
class-map match-any tenent2_cm
match any
policy-map tenent1_pm
class tenent1_cm
police 30000000 conform-action transmit exceed-action drop violate-action drop
policy-map tenent2_pm
class tenent2_cm
police 1000000000 conform-action transmit exceed-action drop violate-action drop
interface GigabitEthernet2.900
bandwidth 30000
encapsulation dot1Q 900 native
service-policy input tenent1_pm
service-policy output tenent1_pm
interface GigabitEthernet2.901
bandwidth 100000
encapsulation dot1Q 901 native
service-policy input tenent2_pm
service-policy output tenent2_pm
Is perfectly functional, but it might be also written as:
policy-map tenent1_pm
class class-default
police 30000000 conform-action transmit exceed-action drop violate-action drop
policy-map tenent2_pm
class class-default
police 1000000000 conform-action transmit exceed-action drop violate-action drop
interface GigabitEthernet2.900
bandwidth 30000
encapsulation dot1Q 900 native
service-policy input tenent1_pm
service-policy output tenent1_pm
interface GigabitEthernet2.901
bandwidth 100000
encapsulation dot1Q 901 native
service-policy input tenent2_pm
service-policy output tenent2_pm
The latter eliminates the need, in this case, for the class maps.
You might also write this as:
policy-map 30Mbps
class class-default
police 30000000 conform-action transmit exceed-action drop violate-action drop
policy-map 100Mbps
class class-default
!original police value was for 1Gbp, but I believe 100 Mbps was intended, as in the bandwidth statement
police 100000000 conform-action transmit exceed-action drop violate-action drop
interface GigabitEthernet2.900
bandwidth 30000
encapsulation dot1Q 900 native
service-policy input 30Mbps
service-policy output 30Mbps
interface GigabitEthernet2.901
bandwidth 100000
encapsulation dot1Q 901 native
service-policy input 100Mbps
service-policy output 100Mbps
Which to use, per tenant or per bandwidth allocation, depends on your expectations. Certainly having a policy for every tenant provides much flexibility, and is fine for a handful. However, as the number of tenants increase, you'll likely want to share the same policy across multiple tenants. In this case, one policy for a specific bandwidth allowance, might be sharable.
In the above, we have police statements that ignored OP's Bc/Be and explicitly coded the conform/exceed/violate actions.
So, I tried the following in CML with the noted results:
policy-map test
class class-default
police 3000000
!actually, when I entered the police command I used 30m for the bit rate
!an option, I recall, not available on earlier IOS versions
interface GigabitEthernet0/0
no ip address
shutdown
duplex auto
speed auto
media-type rj45
service-policy input test
service-policy output test
inserthostname-here#sh policy-map interface g0/0
GigabitEthernet0/0
Service-policy input: test
Class-map: class-default (match-any)
0 packets, 0 bytes
5 minute offered rate 0000 bps, drop rate 0000 bps
Match: any
police:
cir 30000000 bps, bc 937500 bytes
conformed 0 packets, 0 bytes; actions:
transmit
exceeded 0 packets, 0 bytes; actions:
drop
conformed 0000 bps, exceeded 0000 bps
Service-policy output: test
Class-map: class-default (match-any)
0 packets, 0 bytes
5 minute offered rate 0000 bps, drop rate 0000 bps
Match: any
police:
cir 30000000 bps, bc 937500 bytes
conformed 0 packets, 0 bytes; actions:
transmit
exceeded 0 packets, 0 bytes; actions:
drop
conformed 0000 bps, exceeded 0000 bps
!defaults can vary between platforms - but if you wanted the OP's Bc, you could add it
!unless you're really aware of how to use Bc, I would suggest just allowing the default
!notice no Be, or violate actions, both make sense as we drop out of Bc
Next we have the proposed egress policy:
Shaping - egress
---------------------
policy-map shape_child
class tenent1_cm
bandwidth 30000
class tenent2_cm
bandwidth 100000
class class-default
bandwidth 250000
fair-queue
policy-map shape_parent
class class-default
shape average 1000000000
service-policy shape_child
interface GigabitEthernet0/0/0
bandwidth 1000000
service-policy output shape_parent
Firstly, as g0/0 is gig, and assuming we have all gig, shaping isn't necessary (or useful or effective). It would be needed if were working with a logical bandwidth cap.
Secondly, although it seems reasonable to have class statements for each tenant, OP only contrasted all tenants' bandwidth with client's usage, wanting to give priority to tenants' traffic.
The can be pretty closely accomplished by something like:
class-map client
match input-interface g0/1
policy-map toInternet
class client
bandwidth 1
fair-queue !optional
class class-default
bandwidth 999999
fair-queue !optional
interface g0/0
bandwidth 1000000
service-policy output toInternet
The above, though, doesn't totally starve client bandwidth, so if tenants have been allocated the full gig, they are going to be shortchanged (by a very minimal value).
If you truly wanted to preclude taking bandwidth from tenants, on later platforms, you might try:
policy-map toInternet
class client
priority level 2
class class-default
priority level 1
The above, should exactly meet your "ask", except for one thing. What about Internet ingress?
Actually, as even tenant policing is downstream of the Internet connection, one tenant can pull more data down the Internet connection than will be passed through the policer. I.e. they won't receive it, but it uses bandwidth on the Internet link. Hopefully (and usually), not a lot more bandwidth, as most apps will have issues with a high packet lost rate and often slow their transmission rate.
However, the client isn't capped at all. So they could really slam their tenants on downloads. If they want to preclude this, they can also rate limit themselves, on g0/1 (only egress), and try to insure there's some "headroom" above and beyond the aggregate of all the (egress) policers (client and tenants).
This latter issue is the dead end, I alluded to in my prior reply. It's very difficult to manage bandwidth downstream of the bottleneck.
Upstream, in theory, we have g0/1 and g0/2 that can send twice as much data as g0/0 can transmit. But, the g0/0 egress policy can manage such congestion as we desire.
However, downstream, g0/0 is already limited to gig, but we can only indirectly influence the client and/or tenant bandwidth usage coming in. Yea, we can control what the client or tenants actually receive, bandwidth wise, but that's not the same as controlling usage of the Internet bandwidth coming to us.
Years past, there were some 3rd party products (e.g. Packeteer) that could often much better manage ingress bandwidth (especially for TCP), but even they couldn't work as well as upstream QoS egress for bandwidth arbitration (they could also make TCP work even "better"). (These products never really seemed to catch on, possible because the benefits they offered weren't well understood and/or the cost of these products. [Sort of like trying to convince those paying for network equipment why more expensive Cisco devices are "better" than Brand X. Or, laugh, trying to convince many network engineers the benefits of QoS beyond protecting VoIP, etc.])
12-17-2024 12:54 AM
Wow guys @Joseph W. Doherty & @paul driver thanks for the detailed responses - will need a bit to review and confirm my way forward - but this is absolutely amazing, thank you, Im knee deep in another Cisco networking challenge this morning (the joys of introducing mixed vendor solutions into a legacy EIGRP environment! - but this one I have covered :D)
Will let you both know how I go when I get back onto this router config later today.
Thanks again!!
Paul
12-19-2024 02:57 AM
Hey Guys, just to update here - obviously with time of year, change freeze has kicked in so this router wont be going in to production before the festive break - so I have parked working on the config whilst I deal with some other bits that need dealt with - it will be going in at the start of the year though, so I may get back to it to at least get the config ready.
Im definitely going with your advice and suggestions though, but I can only choose one accepted solution I think - thanks again to both of you
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide