Traffic Shaping on Catalyst 9500 to ISP

Isaac Smith · ‎03-17-2022

We have a 10Gig (and 100G in one case) interface on our 9500 connected 10/100 Gig overall bandwidth from an ISP on that circuit. We have multiple remote sites connected over this link at a CIR of 1gbps for each site. We need to make sure that each site that transfers data from the servers behind our 9500 cannot exceed 1Gbp to avoid getting policed by the ISP. Each site is terminated on the 9500 via different dot1q tag and are distinct in subnets.

example:

Site A

WAN interface:

interface GigabitEthernet0/0/2.1100

encapsulation dot1Q 1100

ip address 10.10.0.1/30

Site B

WAN interface:

interface GigabitEthernet0/0/2.1200

encapsulation dot1Q 1200

ip address 10.11.0.1/30

Site C WAN interface:

interface GigabitEthernet0/0/2.1300

encapsulation dot1Q 1300

ip address 10.12.0.1/30

Etc...

Hub - C9500s

interface TwentyFiveGigE1/0/1

bandwidth 10000000

no switchport

mtu 9104

no ip address

//Site A termination

interface TwentyFiveGigE1/0/1.1100

bandwidth 1000000

encapsulation dot1Q 1100

//Site B termination

interface TwentyFiveGigE1/0/1.1200

bandwidth 1000000

encapsulation dot1Q 1100

//Site C termination

interface TwentyFiveGigE1/0/1.1300

bandwidth 1000000

encapsulation dot1Q 1100

etc... What's the best way to shape outbound from the 9500 into the ISP at 1Gbp per dot1q tag or subnet?

I have tried applying a parent/child policy but get various errors as it seems the 9500 can't do queuing on subinterfaces as TAC/ISP have recommended

policy-map SHAPE-1G

class class-default

shape average 990000000

queue-limit 1024 packets

9500(config)#int hu1/0/26.1312

9500(config-subif)#service-policy output SHAPE-1G

Queueing actions are not supported on Subinterface

---------------------------

class-map match-all 1G-CLASS-MAP

match protocol ip

policy-map SHAPE-1G-CHILD

class 1G-CLASS-MAP

policy-map SHAPE-1G

class class-default

shape average 990000000

service-policy SHAPE-1G-CHILD

int hu1/0/26.1312

service-policy output SHAPE-1G

Queueing actions are not supported on Subinterface

---------------------------------------------

We've tried by matching access list and applying to the physical interface but face another issue:

ip access-list extended REMOTE-OFFICE-SUBNETS-ACL

permit tcp 10.1.0.0 0.0.255.255 host 10.10.10.10

permit tcp host 10.10.10.10 10.1.0.0 0.0.255.255

class-map match-any REMOTE-OFFICE-CLASS-MAP

match access-group name REMOTE-OFFICE-SUBNETS-ACL

policy-map SHAPE-1G-CHILD

class REMOTE-OFFICE-CLASS-MAP

policy-map SHAPE-1G

class class-default

shape average 990000000

queue-limit 1024 packets

service-policy SHAPE-1G-CHILD

However I get this error:

9500(config)#int twe2/0/4

9500(config-if)#service-policy output SHAPE-1G

Invalid queuing class-map!!! Queuing actions supported only with dscp/cos/qos-group/precedence/exp based classification!!!

I did apply the shaper to the physical interface and this seemed to work however I still have the question about if that limits the aggregate of traffic on that 10G interface to 1G, effectively limiting the agg speed to 1G. I also played around with the queue-limit on a physical interface. It doesn’t seem to be as effective as shaping but seemed to show some better results then what we have today.

My fear is that if i put a 1Gbp shaper policy on a 10Gig interface that has 10 subinterfaces representing branch offices that each have a 1Gbp CIR that the shaper will limit all 10 sites aggregate transfer to 1Gbp which obviously is not what we want.

mlund · ‎03-17-2022

Hi

I haven't done this on 9500, so don't know if supported. But try this.

ip access-list extended REMOTE-OFFICE-1
permit ip any 10.1.0.0 0.0.255.255
ip access-list extended REMOTE-OFFICE-2
permit ip any 10.2.0.0 0.0.255.255
ip access-list extended REMOTE-OFFICE-3
permit ip any 10.3.0.0 0.0.255.255
!
class-map match-any REMOTE-OFFICE-1
match access-group name REMOTE-OFFICE-1
class-map match-any REMOTE-OFFICE-2
match access-group name REMOTE-OFFICE-2
class-map match-any REMOTE-OFFICE-3
match access-group name REMOTE-OFFICE-3
!
policy-map SHAPE-1G-CHILD
class REMOTE-OFFICE-1
shape average 990000000
class REMOTE-OFFICE-2
shape average 990000000
class REMOTE-OFFICE-3
shape average 990000000
!
policy-map SHAPE-10G
class class-default
shape average 9990000000
service-policy SHAPE-1G-CHILD
!

/Mikael

Joseph W. Doherty · ‎03-17-2022

"What's the best way to shape outbound from the 9500 into the ISP at 1Gbp per dot1q tag or subnet?"

It depends. If the aggregate bandwidth of all your remote sites does not exceed your egress bandwidth capacity, then shaping each "logical" (if supported) would be all you need.

If the aggregate bandwidth of all your remote sites does exceed your egress bandwidth capacity, it still depends. If the aggregate congestion doesn't cause you issues (i.e. some traffic is not more important than others), individual shapers, per remote site, would still be all you need (which appears to be your case, as you don't seem to concern yourself with different traffic types when traffic to a remote site queues).

However if the aggregate bandwidth of all your remote sites does exceed your egress bandwidth capacity, and you want to manage that congestion, then you need to shape the aggregate too.

Seldom have I had to manage subinterface traffic. Further, often switches QoS is inferior to what's available on a router. I.e. don't know whether your 9500 can support something like what @mlund has posted.

I recall (???) since ToS level QoS is at L3, applying a CBWFQ policy on the main interface will effectively apply to all the subinterfaces traffic, so you need to be able to tell apart your subinterfaces at L3. This might be done by addressing or by using the ToS tag. (Remember you have 64 ToS values you may use.)