A Service Provider experienced intermittent outages, during backup path being down due to fibre maintenance. An SD channel on MFP was switching to the backup path; however the backup path was down due to a fibre cut. They experienced a total of 10 switches resulting in a total of approximately five minutes of outage.
The service was switching back and forth between the main channel and the backup channel and causing outage.
Service backup has been configured for the channel on DCM and backup service is not available or is degraded. Default setting for service backup as below for the I/O card where the output service exists.
DCM configuration was controlled by ROSA VSM
Logs were collected from the DCM and analysed.
DCM logs had several messages as below for the board which had problems:
Oct 17 13:38:54 board1 DCM_IO: ** TRA-INF: Service loss Board=1, Port=0, Ident=59, SID=7 triggered by scrambled at input
Oct 17 13:38:54 board1 DCM_IO: ** TRA-INF: TS loss Board=1, Port=0, Ident=59 triggered by service loss
Oct 17 13:38:55 board1 DCM_IO: ** TRA-INF: Main TS: Board=1, Port=0, Ident=59: Switch to backup source Board=1, Port=2, Ident=31
The service then reverts back to the main:
Oct 17 13:38:56 board1 DCM_IO: ** TRA-INF: Service loss Board=1, Port=0, Ident=59, SID=7 cleared.
Oct 17 13:39:55 board1 DCM_IO: ** TRA-INF: Main TS: Board=1, Port=0, Ident=59: Revert to higher prio source Board=1, Port=0, Ident=59
From the above messages, it is clear that the input service was going in to scrambled mode and this was causing service loss trigger to be activated and the service switching to backup. Once the main service went in to clear, it was reverting back to main service. This back and forth switching was causing the outages.
The cause of the issue was that an unused DPI PID was being periodically scrambled. Since the service backup setting on DCM had “PID list” triggers indicating a switch to backup if scrambling occurred, the switch was occurring even though the scrambled PID was not in use.
Although the backup service was not available, it was still switching to backup because on the Default Service Backup Setting for the card, the option “Don’t Switch to Backup Source in Service Loss” and as it was set to Revertive, it was reverting back to Main when the service loss alarm was cleared on the Main service.
To correct the issue, in VSM, the same service loss template was applied to the backup streams as the main streams. This caused the backup service to alarm in the same way and thus prevented the DCM from switching to backup during the scrambling. Also, it allowed the “scrambling” trigger to remain, in the event that a used PID, e.g. video, became scrambled on the main and not on the backup.
The issue could also be prevented by simply ticking the option “Don’t Switch to Backup Source in Service Loss”. Thus, the service would not switch to backup if the backup service is not available.
Hi, We have a POP with ASR 1002 for BNG and CGNAT.The problem is with the configuration of cgnat, we can't access the cpe from outside of asr by interface of uplink.We tried to accept by using an access list, but without success. Follow the conf...
Hi,Previous we were using Cisco 7606 as BRAS and the issue was Router keeps on rebooted automatically.OLD Details about Cisco 7606Router 7606 #show versionCisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 15.2(2)S...
I have successfully registered my SIP endpoints (which we have developed) with Broadsoft sandbox environment. I can make basic calls without problems. When I try to conduct feature tests like Call Waiting (CW), Call Hold (CH), and Call Transfer (CT)...
I am trying to find if there is any Pros or Cons in deploying MPLS with separate customer AS for each site as opposed to one AS. The end goal is to be able to inject default routes from two DCs and be able to make a subset of sites follow one default rout...