The best way to upload large ACLs to NX-OS Nexus 7009

[Paul Matthews]

Hi - We are migrating from Catalyst 6509 IOS platforms to Nexus 7009. There's the normal differences in commands

which is well doucumented. We do have some quite large files containing ACLs varying from 10's of lines to several 1000's of

lines. Our normal upload would be done using tftp and then issuing the command 'conf net' on the the 6509. This is no

longer the way to do this on NX-OS. I've tried copy ftp: running-config which works fine for small files but for big ones

it takes a long time, in some cases I've see it takes 20-30 minutes. The initilal tftp uplaod to the 7009 seems OK but the

copy into the running-config is the bit that takes time and initially I thought I'd killed the 7009!! It did finally come back to

the prompt. Am I missing something here? Are the 7009's simply not designed for large ACLs? I did try the configure session (Session Manager)

but I couldn't see a way of uploading a file. I tried creating a new session and then exiting it, copying in a file of the same format and

then commiting it but it didn't seem to acknowledge the file (checksum?). Help!!

Thanks, Paul

Paul Matthews

Swansea University

Wales, UK.

[mgalazka]

If I have a config built for a new device, sometimes I will put my new config in a text file and SCP that file to the switch (slot0, bootflash, etc).  Then I will copy that file into running-config.

i.e.:

copy scp://user@host/path/filename bootflash:

copy bootflash:///filename running-config

This may be a workaround to what you are trying to accomplish?

Hope this helps.

Matt

edited to add: When copying a file into running-config, remember it is merging your changes in to running, as if you were copying/pasting commands into conf t mode.  If you do this with startup-config as destination, it will overwrite your config with the new file. So be mindful of how you do this

[Paul Matthews]

Thanks Matt. It's the actual copy to the running-config that takes forever. I'm beginning to wonder whether the Nexus 7009 is really designed for large ACLs. Our 6509 swallows the large ACL in about 10-15 seconds. The upload is OK in both cases (still quicker on the 6509) but it's the merging into the running configuration that the difference occurs.

I was wondering if any one else has large ACLs on their Nexus platforms?

Thanks,

Paul

[Gregory Snipes]

Honestly, if running ACLs that large are a requirement for you, you may be better off sticking with a traditional catalyst platform. Running multi-thousand line ACLs is not a typical data center requirement. I have only ever seen ACLs that large run on boarder routers and firewalls.

Out of curiosity, why are you running such huge ACLs within your datacenter?

Best Regards,

Greg

[Leo Laohoo]

Paul,

I don't have access to a Nexus 7K, but I do recall the Sup1/2 comes with a USB port.  Have you tried that?

[mgalazka]

Paul,

So I figured I would mess around in the lab for a few minutes. The good news is that I figured out how to copy a file into config session.  The bad news is it's just as slow as you described.

I created a text file like so on my PC, to use session manager to configure an ACL with 16,000 entries:

configure session test

ip access-list test

permit ip any 1.2.3.4/32

...

I copied it to the N7K's bootflash, then copied it into running-config.  The copy from bootflash to running-config took about 1 min 10 sec. Note that this did not apply the ACL into running-config as you would think it might.  It created the config session and did NOT commit.  I then entered config session test and issued 'commit' and it took about 1 min 30 sec to apply.

How big are the ACL's you're working with out of curiosity?

Figuring that wasn't as bad as I thought it might be, I tried an ACL with 65,000 lines - it took almost 10 minutes to copy into run to create the config session... and about 15 minutes to commit.  Frankly I was surprised it even took it.

I'm wondering if it took that long to load in via config session, would it also delay a reload significantly too? Interesting. I might try to find time to mess around with it some more tomorrow.

Matt

[Paul Matthews]

Hi Matt - Your efforts are much appreciated. Your 1 minute times are still better than mine and I would be happy with that, if that's the best! What I did was copy tftp://xx.xx.xx.x/filename running-config. Out of curiosity, I did 'configure session acl'  but I couldn't work out how to get my files contents into the session. Did you do a simple cut and paste or is there a magic way of uploading the config into it ?

Thanks,

Paul

[mgalazka · Accepted solution]

Paul,

You are quite welcome!

Here is the process I used to upload the file "into" config session:

1. On your PC, create a text file with the contents of your ACL, making sure the first line of the file is the configure session . Here is example of file contents:

configure session acl
ip access-list MyACLName
permit ip any 1.2.3.4/32
permit ip any 10.0.2.0/23
...
deny ip any any

2. (Optional) Modify control plane policing policy to allow for fast file transfer, otherwise you will wait longer for the text file to transfer. (I just removed the service policy from the control plane temporarily - as it was a lab device -- not recommended for production of course!).

conf t
control-plane
 no service-policy input copp-system-policy
end

3. Copy file into running-config, either as a one-step from tftp/scp to running-config, or temporarily to bootflash/slot0/volatile/etc, then to run.

copy tftp://server/acl.txt bootflash:
copy bootflash:///acl.txt running-config

4. Confirm config session was created:

show config session

5. Verify that applying the change will not result in TCAM exhaustion:

config session acl
 verify
 end

6. Commit the change

config session acl
 commit
 end

This was the process I used.  Keep in mind on N7K I believe the time to commit ACL changes will be based on the number of linecards, as TCAM is pushed into the linecards.  I have only 2 cards other than the supervisors in this chassis.  But this is what I did to yield the 1 to 2 minute load times for the 16000 line ACL I tested with.  Also keep in mind that the verify step seems to take approximately as long as the commit step.

Hope this helps, and here's my shameless plug to ask you to rate any of the posts you find helpful

Matt

[Paul Matthews]

Thanks Matt - It's definitely quicker your way! That's all I wanted!

Hope I successfully managed to posivitely rate your posts!!

Cheers,

Paul

[Paul Matthews]

Hi - I haven't tried the USB but it's the merge into the running config is the problem. As the the number of ACLs and why are they so large? Around 8,00 lines is the biggest. Yep, quite big and not normally associated I guess with a Nexus if it's used in it's traditional data centre role. We are using two of them to replace two of our Catalyst 6509's which will become end of life. They are running in a fairly basic mode, with many SVI's most of which are vlans used for our managed desktop PCs and have ACLs on them. The main ACL'ed vlan holds our new Active Directory domain and this ACL controls very tightly access to that new domain from all the various subnets across our campus. Our 6509 copes with quite well but moving forward our Cisco supplier listened to our current setup and suggested the Nexus 7009 as a good replacement. On paper the spec is pretty impressive compared  to the 6509 and made good sense! Everything was going well, slowly moving services/vlans across until I looked at doing the larger ACL'ed vlans and that's when I noticed this slowness with regqards to the ACL upload. The rest is fine, it's just getting the ACL into it! Some one out there is going to tell me I shouldn't have bought the 7009 and should have bought a XXX instead! Too late now... I am a little confused, the box is suppoeed capable of handling ACL sizes of 64,000-128,000 lines long so why is it struggling to load 8,000 lines? Imagine how long it would take to load somethig near 64,000!! Thanks for everyones help so far...

Cheers,

Paul

[Gregory Snipes]

What modules are you running in the device? Also do you have the XL licence for the device? ACL performance can vary widely in the Nexus platform depending on your modules and licensing.