Introduction
Redundancy (or fault tolerance) uses a maximum of two ACE appliances to ensure that your network remains operational even if one of the appliances becomes unresponsive. Redundancy ensures that your network services and applications are always available. Redundancy provides seamless switchover of flows in case an ACE becomes unresponsive or a critical host or interface fails. Redundancy is not supported between an ACE appliance and an ACE module operating as peers. Redundancy must be of the same ACE device type and software release.
Fault Tolerant (FT VLAN)
Redundancy uses a dedicated FT VLAN between redundant ACEs to transmit flow-state information and the redundancy heartbeat. Configure this same VLAN on both peer modules. The two redundant modules constantly communicate over the FT VLAN to determine the operating status of each module. The standby member uses the heartbeat packet to monitor the health of the active member. The active member uses the heartbeat packet to monitor the health of the standby member. For multiple contexts, the FT VLAN resides in the system configuration file. Each FT VLAN on the ACE has one unique MAC address associated with it. The ACE uses these device MAC addresses as the source or destination MACs for sending or receiving redundancy protocol state and configuration replication packets..
Redundancy Protocol and Stateful Failover
Each peer appliance in a redundant group can contain one or more fault-tolerant (FT) groups. Each FT group consists of two members: one active context and one standby context. One virtual MAC address (VMAC) is associated with each FT group. Each FT group acts as an independent redundancy instance. When a switchover occurs, the active member in the FT group becomes the standby member and the original standby member becomes the active member. The ACE uses the heartbeat to probe the peer ACE, rather than probe each context. When an ACE does not receive a heartbeat from the peer ACE, all the contexts in the standby state become active.
Cisco ACE replicates flows on the active FT group member to the standby group member per connection for each context. The replicated flows contain all the flow-state information necessary for the standby member to take over the flow if the active member becomes unresponsive. If the active member becomes unresponsive, the replicated flows on the standby member become active when the standby member assumes mastership of the context. The active flows on the former active member transition to a standby state to fully back up the active flows on the new active member. Note that the ACE does not replicate SSL and other terminated (proxied) connections from the active context to the standby context.
Steps to Replace ACE in a Redundancy Setup
Issue command "ft switchover all" on the standby ACE before making the replacement. This will ensure that all the context are active on single ACE. Then start the procedure as stated below.
(1) Issue “wr mem” on “currently ACTIVE ACE” and backup the configuration.
Create a configuration “checkpoint” on “currently ACTIVE ACE” for EACH context.
(2) Backup (copy) config from each user context, including Admin context, from your currently in production ACE to a FTP server.
(3) Export your current “certs & keys” to a tftp/ ftp/ sftp server from the ACTIVE ACE & then import them on “the new ACE” later.
(4) Power down the ACE module, to be replaced, from the switch CLI in config mode (no power enable module <ACE module #>) and replace it with the replacement module.
(5) Power up the new replacement module from switch CLI (power enable module <ACE module #>).
(6) Once the new module is on line, session into it from the switch.
(7) Configure Admin context with an IP interface VLAN configuration so that you have IP connectivity to the module.
(8) Make sure you upgrade the newly received replacement ACE to exactly the same release of code as that of “currently ACTIVE ACE” .
(9) Configure Admin context with rest of the configuration as per backed up config ( for this ACE) EXCEPT FT configuration.
Note: If you don’t have a config for this module “backed” up. You would need to review Admin context configuration from “ACTIVE ACE” and configure it accordingly. Please make sure you use “peer IP address” information from currently ACTIVE ACE to configure this ACE module.
(10) If you have “ssl-proxy” service configured in any user context, please make sure you IMPORT all your “Certs & Keys” to this new ACE module before configuring your FT configuration. You can import them with option terminal (e.g. crypto import <file name> terminal) otherwise you would have to configure each context with an IP interface to be able to import certs/keys via tftp or ftp or sftp.
The ACE does not synchronize the SSL certificates and key pairs that are present in the active context with the standby context of an FT group. If the ACE performs configuration synchronization and does not find the necessary certificates and keys in the standby context, config sync fails and the standby context enters the STANDBY_COLD state. In order to correct this problem, verify if all certs and keys are installed on both ACE modules.
(11) Configure a FT VLAN interface & FT PEER on “new replacement ACE”.
Configure all FT groups BUT DO NOT “configure them “inservice”.
Make sure you have IP connectivity OVER FT VLAN to “currently ACTIVE ACE”.
Make sure there is a TCP connection setup OVER FT VLAN (show conn should provide you that information).
(12) Make sure “preemption” is NOT enabled for the FT group. If enabled please do remove it and re-add after the module is successfully replaced.
Example:
ft group 1
peer 1
no preempt <===========
peer priority 150
associate-context test
(13) Once you have IP connectivity over FT VLAN to “primary ACE”, now mark the FT GROUP “inservice”.
Example:
ft group 1
peer 1
no preempt
peer priority 150
associate-context test
inservice <===========
(14) At this time the “auto-sync” should “sync” configs between “currently ACTIVE ACE” & “new standby ACE”.
Verify
The following commands will help in verifying the state of FT configuration.
show ft group detail
show ft peer detail
References
ACE 4710 How to replace an damaged unit in FT
ACE replacement. Plz help.
Configuring Redundant ACE Appliances
