HWIC-3G-HSPA-G Keeps shutting down?

[warren.sullivan]

Hi guys,

We have 16 sites with these cards in 1941 routers as backup links, we have 1 that keeps powering off the modem, it re-establishes when rebooted with #test cell 0/0/0 modem-power-cycle, any ideas? see below for some output, let me know if you need more....

isa-rt02#sh cell 0/0/0 all

Hardware Information

====================

Modem Firmware Version = K2_0_7_19AP C:/WS/FW

Modem Firmware built = 10/26/09

Hardware Version = 1.0

International Mobile Subscriber Identity (IMSI) = 505023305481921

International Mobile Equipment Identity (IMEI) = 012073005052157

Integrated Circuit Card ID (ICCID) = 8961023311644192709

Mobile Subscriber International Subscriber

IDentity Number (MSISDN) =

Factory Serial Number (FSN) = C79259109221005

Modem Status = Online

Current Modem Temperature = 32 deg C, State = Normal

PRI SKU ID = 9993248, SKU Rev. = 1.3

Profile Information

====================

Profile 1 = ACTIVE*

--------

PDP Type = IPv4

PDP address = 10.251.5.90

Access Point Name (APN) = xxxxxxxxx

Authentication = None

Username: , Password:

Profile 16 = INACTIVE

--------

PDP Type = IPv4

Access Point Name (APN) = test

Authentication = None

Username: , Password:

* - Default profile

Data Connection Information

===========================

Data Transmitted = 56923 bytes, Received = 44615 bytes

Profile 1, Packet Session Status = ACTIVE

        IP address = 10.251.5.90

        Negotiated QOS Parameters:

        Precedence = Normal Priority, Delay = Class 1

        Reliability = Unack GTP, LLC, Ack RLC, Protected data

        Peak = 256 kB/sec, Mean = 50000 kB/hr

        Traffic Class = Interactive

        Uplink Max = 2048kbps, Guaranteed = Subscribed

        Downlink Max = 7.2Mbps, Guaranteed = Subscribed

        Max SDU size = 1500 bytes

        SDU error ratio = 1E-4, BER = 1E-5

Profile 2, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 3, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 4, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 5, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 6, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 7, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 8, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 9, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 10, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 11, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 12, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 13, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 14, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 15, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Profile 16, Packet Session Status = INACTIVE

        Inactivity Reason = Normal inactivate state

Network Information

===================

Current Service Status = Normal, Service Error = None

Current Service = Combined

Packet Service = HSDPA (Attached)

Packet Session Status = Active

Current Roaming Status = Home

Network Selection Mode = Automatic

Country = AUS, Network = Optus

Mobile Country Code (MCC) = 505

Mobile Network Code (MNC) = 2

Location Area Code (LAC) = 37250

Routing Area Code (RAC) = 1

Cell ID = 6282

Primary Scrambling Code = 219

PLMN Selection = Automatic

Registered PLMN =  , Abbreviated =

Service Provider = YES OPTUS

Radio Information

=================

Radio power mode = ON

Current Band = WCDMA 2100, Channel Number = 10713

Current RSSI = -63 dBm

Band Selected = Auto

Number of nearby cells = 1

Cell 1

        Primary Scrambling Code = 0xDB

        RSCP = -67 dBm, ECIO = -13 dBm

Modem Security Information

==========================

Card Holder Verification (CHV1) = Disabled

SIM Status = OK

SIM User Operation Required = None

Number of CHV1 Retries remaining = 3

isa-rt02#

isa-rt02#sh controllers cellular 0/0/0

Interface Cellular0/0/0

HSPA/UMTS 900/1900/2100MHz Quad band EDGE/GPRS unit 0,

HWIC cellular modem configuration:

---------------------------

Modem is recognized as valid for this HWIC

manufacture id:  0x00001199     product id: 0x0000683C

Sierra Wireless MC8792V HSPA modem.

Modem Uplink Speed = 2000 kbit.

Modem Downlink Speed = 7200 kbit.

GPS State: GPS disabled

Modem Management Statistics

---------------------------

Modem resets = 29

Last known modem state = 'application' mode

Packets sent = 2283, Packets received = 46800, Packets pending = 0

RSSI  LED = Solid Green

UMTS  LED = OFF

HSPA  LED = ON

GPS NMEA port = Disabled   (Stream OFF)

DM port = Disabled

Nios registers:

-----------------

Nios reset register = 0x12D4B

Nios PC Trace Buffer Pointer register = 0x22B0

PLL1 = Locked          PLL2 = Locked          PLL3 = Locked

HWIC Register Base: 0x10000000

------------------

HWIC Common Registers:

---------------------

id(0x00):  0x00000001     rev(0x01): 0x00000002     status(0x02): 0x00000000

tx crc(0x01): 0x00000000  ctrl(0x06): 0x00008000

HWIC Cell Common Registers:

----------------------------

hwic_cell_rev(0x1000): 0x02092011

intr1_enable(0x1020): 0x00000038  intr2_enable(0x1022): 0x0000005F

intr1_event(0x1020): 0x00000000   intr2_event(0x1022): 0x00000000

HWIC Cell SCC Registers:

----------------------------

dma channel # = 0

ch_mode_cfg: 0x10019035    ch_flag_cfg: 0x00000000  ch_floctrl_cfg: 0x00000000

ch_int_enable: 0x0000EBFF  ch_cmd_stat: 0x00000050  ch_int_stat: 0x001C (shadow)

dma channel # = 1

ch_mode_cfg: 0x10019005    ch_flag_cfg: 0x00001311  ch_floctrl_cfg: 0x00002200

ch_int_enable: 0x0000EBFF  ch_cmd_stat: 0x00000050  ch_int_stat: 0x0001 (shadow)

HWIC Cell DMA Registers:

-------------------------

dma channel # = 0

Rx ring_start_ptr: 0x0EB0BA00     Rx ring_mask_index: 0x01F80038

internal_desc_stat: 0x0800060C    internal_buff_addr: 0x0ECBF154

internal_context: 0x00000013      internal_fifo_data: 0x0ECB26B3

dma channel # = 1

Rx ring_start_ptr: 0x0EB11C00     Rx ring_mask_index: 0x01F800A0

internal_desc_stat: 0x08000400    internal_buff_addr: 0x0ED1DD58

internal_context: 0x00000013      internal_fifo_data: 0x0ED0BB70

dma channel # = 0

Tx ring_start_ptr: 0x0EB0DA00     Tx ring_mask_index: 0x00F800C0

internal_desc_stat: 0x0CC00000    internal_buff_addr: 0x0DDA9034

internal_context: 0x00000501      internal_fifo_data: 0x0B4660CE

dma channel # = 1

Tx ring_start_ptr: 0x0EB0FA00     Tx ring_mask_index: 0x03F80358

internal_desc_stat: 0x0CC00000    internal_buff_addr: 0x0DDA9034

internal_context: 0x00000181      internal_fifo_data: 0x0B4660CE

idb at 0x30DE86F0, driver data structure at 0x2A835F78

0 input aborts on receiving flag sequence

0 throttles, 0 enables

0 overruns, 0 CRC errors

0 resource errors, 0 incomp frame errors

0 input drops

0 transmitter underruns

0 tx_abort 0 tx_reset

Framer Mode: Async-HDLC

Polling: Enabled

dma channel = 0

dma channel initialized.

Receive Ring

rxr head (7)(0x0EB0BA38), rxr tail (7)(0x0EB0BA38)

Transmit Ring

txr head (24)(0x0EB0DAC0), txr tail (24)(0x0EB0DAC0)

dma channel = 1

dma channel initialized.

Receive Ring

rxr head (20)(0x0EB11CA0), rxr tail (20)(0x0EB11CA0)

throttle flag = 0,  throttle val  = (0x00000000)

rx outstanding count  = 12

46996 packet inputs

0 input aborts on receiving flag sequence

0 throttles

0 input errors, 0 overruns

0 input resource errors, 0 input drops

0 input frame errors

2283 packet outputs

0 transmitter underruns

0 transmitter output drops

2 transmitter max tx outstanding packets

Transmit Ring

txr head (107)(0x0EB0FD58), txr tail (107)(0x0EB0FD58)

Firmware buffer pools

Egress  buf pool 0x2048A98 size 256 alloc 250 redzone 0x204D4E8

        0 no buffer

Ingress  buf pool 0x2049050 size 256 alloc 188

         0 no buffer

FIFO Queue Statistics

  ch0: egress enabled   6517   throttles 6517   enables0      ingress overrun

  ch1: egress enabled   2      throttles 2      enables0      ingress overrun

  ch2: egress enabled   0      throttles 0      enables0      ingress overrun

  ch3: egress enabled   0      throttles 0      enables0      ingress overrun

  ch4: egress enabled   0      throttles 0      enables0      ingress overrun

  ch5: egress enabled   0      throttles 0      enables0      ingress overrun

OHCI USB Controller Registers

  IntStatus     0x00000024

  HCCA          0x02077C00

  ControlHeadED 0x020705D0

  BulkHeadED    0x02070610

  DoneHead      0x00000000

  PortStatus    0x00000103

USB Bulk Endpoint Descriptors

IN 1 0x2070690: 00401081 02071F50 02071702 020706D0

  TD 0x2071700: E0040000 020933F0 02072730 020935EF

  TD 0x2072730: E0040000 02097900 020727A0 02097AFF

  TD 0x20727A0: E0040000 02091FA0 02071B60 0209219F

  TD 0x2071B60: E0040000 020921A8 02071F50 020923A7

OUT 1 0x20706D0: 00400881 02071850 02071852 02070710

IN 2 0x2070710: 00401101 02072810 02072810 02070750

OUT 2 0x2070750: 00400901 02072880 02072880 02070790

IN 3 0x2070790: 00401181 020728F0 020728F0 020707D0

OUT 3 0x20707D0: 00400981 02072960 02072960 02070850

IN 5 0x2070850: 00401281 020717E0 020718C2 02070890

  TD 0x20718C0: E0040000 02091D98 02072030 02091F97

  TD 0x2072030: E0040000 020966B8 02072650 020968B7

  TD 0x2072650: E0040000 02095E98 020721F0 02096097

  TD 0x20721F0: E0040000 02095880 02072340 02095A7F

  TD 0x2072340: E0040000 02094C50 02071D90 02094E4F

  TD 0x2071D90: E0040000 02091988 02071620 02091B87

  TD 0x2071620: E0040000 02091168 020726C0 02091367

  TD 0x20726C0: E0040000 020931E8 02071CB0 020933E7

  TD 0x2071CB0: E0040000 02090330 02072490 0209052F

  TD 0x2072490: E0040000 02094228 02072570 02094427

  TD 0x2072570: E0040000 0208FB10 02072500 0208FD0F

  TD 0x2072500: E0040000 020962A8 02071690 020964A7

  TD 0x2071690: E0040000 02092BD0 02072180 02092DCF

  TD 0x2072180: E0040000 020960A0 02072260 0209629F

  TD 0x2072260: E0040000 0208FD18 02071FC0 0208FF17

  TD 0x2071FC0: E0040000 02093E18 02072110 02094017

  TD 0x2072110: E0040000 02098738 020725E0 02098937

  TD 0x20725E0: E0040000 02097D10 02071AF0 02097F0F

  TD 0x2071AF0: E0040000 02093C10 02071770 02093E0F

  TD 0x2071770: E0040000 02097F18 02072420 02098117

  TD 0x2072420: E0040000 0208FF20 02071C40 0209011F

  TD 0x2071C40: E0040000 02095A88 02071930 02095C87

  TD 0x2071930: E0040000 02093A08 020722D0 02093C07

  TD 0x20722D0: E0040000 020964B0 02071EE0 020966AF

  TD 0x2071EE0: E0040000 02098940 020719A0 02098B3F

  TD 0x20719A0: E0040000 02090F60 02071A80 0209115F

  TD 0x2071A80: E0040000 02091B90 020720A0 02091D8F

  TD 0x20720A0: E0040000 020974F0 02071BD0 020976EF

  TD 0x2071BD0: E0040000 020972E8 02071E00 020974E7

  TD 0x2071E00: E0040000 02090948 020723B0 02090B47

  TD 0x20723B0: E0040000 02092FE0 02071E70 020931DF

  TD 0x2071E70: E0040000 020935F8 020717E0 020937F7

OUT 4 0x2070890: 00400A01 02071D20 02071D22 02070910

IN 7 0x2070910: 00401381 02074B80 02072AB0 02070950

  TD 0x2072AB0: E0040000 020968C0 02073DF0 02096ABF

  TD 0x2073DF0: E0040000 02092DD8 02073E60 02092FD7

  TD 0x2073E60: E0040000 02095678 02073ED0 02095877

  TD 0x2073ED0: E0040000 02090538 02073F40 02090737

  TD 0x2073F40: E0040000 020923B0 02073FB0 020925AF

  TD 0x2073FB0: E0040000 02096CD0 02074020 02096ECF

  TD 0x2074020: E0040000 02096AC8 02074090 02096CC7

  TD 0x2074090: E0040000 02090B50 02074100 02090D4F

  TD 0x2074100: E0040000 02091370 02074170 0209156F

  TD 0x2074170: E0040000 02094430 020741E0 0209462F

  TD 0x20741E0: E0040000 020970E0 02074250 020972DF

  TD 0x2074250: E0040000 02090128 020742C0 02090327

  TD 0x20742C0: E0040000 02094638 02074330 02094837

  TD 0x2074330: E0040000 02098328 020743A0 02098527

  TD 0x20743A0: E0040000 020927C0 02074410 020929BF

  TD 0x2074410: E0040000 02094E58 02074480 02095057

  TD 0x2074480: E0040000 02098530 020744F0 0209872F

  TD 0x20744F0: E0040000 02091780 02074560 0209197F

  TD 0x2074560: E0040000 02090740 020745D0 0209093F

  TD 0x20745D0: E0040000 02098120 02074640 0209831F

  TD 0x2074640: E0040000 02093800 020746B0 020939FF

  TD 0x20746B0: E0040000 02095268 02074720 02095467

  TD 0x2074720: E0040000 02095470 02074790 0209566F

  TD 0x2074790: E0040000 02096ED8 02074800 020970D7

  TD 0x2074800: E0040000 02094A48 02074870 02094C47

  TD 0x2074870: E0040000 02091578 020748E0 02091777

  TD 0x20748E0: E0040000 02097B08 02074950 02097D07

  TD 0x2074950: E0040000 02095060 020749C0 0209525F

  TD 0x20749C0: E0040000 020925B8 02074A30 020927B7

  TD 0x2074A30: E0040000 020929C8 02074AA0 02092BC7

  TD 0x2074AA0: E0040000 02094020 02074B10 0209421F

  TD 0x2074B10: E0040000 02095C90 02074B80 02095E8F

OUT 5 0x2070950: 00400A81 02072B20 02072B20 020709D0

IN 9 0x20709D0: 00401481 02072B90 02072B90 02070A10

OUT 6 0x2070A10: 00400B01 02072C00 02072C00 02070A90

IN 11 0x2070A90: 00401581 02072C70 02072C70 02070AD0

OUT 7 0x2070AD0: 00400B81 02072CE0 02072CE0 00000000

Transfer Completion Codes

  Success       : 113717    CRC        : 140       Bit Stuff:0      

  Stall         : 0         No Response: 0         Overrun        

  Underrun      : 0         Other      : 0      

  Buffer Overrun: 0         Buf Underun: 0      

Transfer Errors

  Control Timeout : 0         Canceled Transfers  : 0      

Transfer Failures

  Bulk Transfer   : 84        Interrupt Transfer  : 56     

  Control Transfer: 0         Isochronous Transfer: 0      

Transfer Successes

  Bulk Transfer   : 107609    Interrupt Transfer  : 240    

  Control Transfer: 5868      Isochronous Transfer: 0      

isa-rt02#

[warren.sullivan]

Any ideas guys?

[Tagir Temirgaliyev]

is the ios the same? in all 16 routers?

try to change sim card or 3G card

[warren.sullivan]

Just to follow up so this isn't a dead thread, it was the card jumping between GSM and WCDMA, it was confused and eventually shut itself down, this particular site has very limited coverage so specifying what band to stick on fixed the issue.

isa-rt02#cellular 0/0/0 gsm band wcdma-all-bands

thanks

warren

[mustafizurrahman]

I had a similar issue. I used AT commands to manually force lock to WCDMA. But later changed to auto since when WCDMA signal is poor the site loses comms altogether.

However with another site with similar setup, I did the debugs chat script, debug dial events, debug ppp negotiation,

It appears that the chat script tries to dial but the modem does not respond time to time. Customer was in the impression that 3G link would become dormant while not in use. That was not the case. I still have cisco engineer investigating the root cause. But most likely it would be modem firmware issue.