This is the first post in hopefully a long line of posts giving short nuggets of tips and tricks for 'content' related products.
I wanted to start the series with the worst possible example: you are using WAAS on you network to accelerate traffic and everything is fine until you get frantic requests to turn it off as it is preventing some important operation to work.
We've all been there: there is little real information, just confusion and people that are urgently asking you how long the problem will still last.
At this moment you (or someone responsible) is going to have to decide on a strategy to follow, the options in general are:
start turning things off and reloading equipment until it works
investigate the root cause of the problem
This is a clear choice: the moment you take corrective actions that you think will fix the problem it becomes very difficult to determine the root cause. The question you have to ask is: "I can guess what the problem is and do something, which might of might not solve the problem. When I do this we will not know what is the cause of the problem afterwards. Or I can first investigate and then fix the problem. What should I do?".
The preferred option from our point of view obviously is that you call TAC with a 'network down' severity issue. We will ask you the same question and together do what is required to get the network back up.
If this is not possible, then please collect as much information as you can:
which clients where impacted
what traffic was impacted
to what servers
when did the problem start
was similar traffic working before
wccp status from routers/switches if appropriate
sysreport or 'show tech' information from the WAAS devices involved
Another point sometimes forgotten is to log all console output on all devices. This might give us the clue to find the root cause...
The requests become more urgent and you decide to try to fix the problem by turning off the interception of traffic by the WAAS. After you have done this WAAS will no longer have an impact on the traffic and whatever problem remains is not due to WAAS (see an exception below). This is better then reloading WAAS as it will preserve some statistics that could give information to find the root cause.
If the impact is limited to one network, server or client you can chose to only ignore traffic from/to this group. With older versions of WAAS you can do this using a bypass static configuration line if you are using WCCP interception. From 4.2 onwards you can use a interception ACL which is easier to use and works both for WCCP and inline interception.
In case that you are not certain that the impact is limited or you are using inline interception with 4.1 you can turn off all interception with ease. For WCCP interception you simply do:
Naturally you will have to repeat this for the other InlineGroups you have active.
After doing either of these steps WAAS is out of the picture and you should check if this fixed the problem. Except for cabling problems in the inline case, because even if the interface acts as a piece of cable now, the traffic still goes across the cables obviously.
In the next installment I plan to talk about the other option: finding a root cause. Feel free to ask questions or ideas for other posts in the comments below...
Hello all, we have ACI 2 spines & 7 leaves, physical servers, VMWare ESX and few FEXes connected.• we try to deploy this EPG on a FEX access port, but our VM server1 see many other VMs (from many other EPGs)we still can access the UI.The EPG is L...
Hello, Everyone, We have just implemented a VMM integration with a vCenter 220.127.116.1100 environment. The vDS was created, along with a single port group corresponding to the pre-provisioned management EPG. We then associated a pre-pro...
after remote leaf discovery and creating upgrade group for the new discovered leafs the above error shown. Description:Fault delegate: A Fabric Node Group (fabricNodeGrp) configuration was not deployed on the fabric node 152 because: Node Not Registe...
Hi all,I use a switch profile to keep the configuration of two Nexus 5548 in sync. That always worked. Now I have the problem that the synchronization only works in one direction. Sync from sw2 is ok, sync from sw1 is not ok. I've already tried a few thin...