Can SVCLC config update between 6509 to ACE Module cause the ACE module to be unresponsive

[RAMAN AZIZIAN]

Hello Everyone,

We ran into a problem few days back during our activity. I wanted to find out if anyone has seen the below incident.

Topic:

Can SVCLC config update between 6509 to ACE Module cause the ACE module to be unresponsive, and require a power shutdown/reset to be performed?

Scenario:

We have a pair of 6509 running VSS, with a single ACE module installed in each chassis. ACE are running in Fault Tolerant mode, and the load-balancers configured as Bridge-Mode.

Our configs already had vlan 100-200,300-600 advertised via (Internal Trunk) between 6509 and ACE module.

I needed to add a new VLAN for our customer (V84 Bridged to V484), and advertise the vlan via SVCLC command.

When the command was issued, the ACE module (active) for unknown (can't tell from the log file) reason failed over to the standby ACE module.

Has anyone seen anything like this before? I just want to make sure it wasn't anything on my end that caused this, from the configuration stand point.

I have attached a sketch depicting my setup.

I woul apprecaite any feedback.

Thanks,

raman

[Gilles Dufour]

Did it crash or simply cause a failover ?

How did you do exactly to modify the SVCLC config ?

I haven't seen this before.

Gilles.

[RAMAN AZIZIAN]

Here's TACs response:

The ACE Module crashed unexpected with a NP Control Store Parity Error which can be due to hardware.
CSCsz65679

Bug Details
ACE Crash due to NP Control Store Parity Error (place holder)
Symptom:

The ACE Module crashed unexpected with a NP Control Store Parity Error which can be due to hardware.

Conditions:

Normal Operations.

Workaround:

None. Monitor the ACE Module and if this reoccurs a RMA should be considered (try to explain this to my customer will not be fun ;-(

In regard to your question about SVCLC:

Currently the SVCLC command on 6509 advertises the following VLAN:  vlan 100-150,200-600

Customer VLAN is 84.

I was going to advertise the new VLAN 84, so it would have been like this:

Before:

svclc vlan-group 1  100-150,200-600

After

svclc vlan-group 1  84,100-150,200-600

When our implementation team applied the command, they were instructed to apply the ACE-admin configs which included the new VLAN 84, and Serverfarm VLAN, 484. After that they were going to apply the configs for the actual ACE that will do all the SLB services. This included the new VLAN's and the BVI info.

The Ops team indicated they could not ssh, telnet, or local session (session slot # proc 0). I requested to check the status of ACE module via

show mod in 6509, and they indicated they noticed unknown was the status.

Anyway, I don't want to make you fall a sleep with my boring explanation. The FT network kept the services running. I guess in the end we were able to truly test the failover/HA. ;-)

As always, thanks for your help.

-Raman