1921 and EHWIC-4G-LTE-V Failures On Connecting LAN

bill.hill · ‎07-09-2013

I am running into an issue regarding usage of a 1921 (15.3(2)T) and EHWIC-4G-LTE-V (firmware SWI9600M_03.05.10.06).

With the EHWIC installed in the router, I am able to establish a data connection on the Verizon LTE network. RSSI approx -64 consistently.

However, the oddities begin when I connect the LAN (or a laptop, for troubleshooting purposes) to the internal interface (Gig 0/1).

Output when no other devices connected to the router

===============================================

arch-eng-router1#ping 4.2.2.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 4.2.2.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 40/80/224 ms

Profile Information

====================

Profile 1 = ACTIVE*

--------

PDP Type = IPv4

PDP address = 10.172.240.7

Access Point Name (APN) = VZWINTERNET

Authentication = None

Username:

Password:

Primary DNS address = 198.224.169.135

Secondary DNS address = 198.224.170.135

* - Default profile

Output when another device is connected to the router

================================================

[Note: 'debug dialer' and 'debug chat' enabled for output]

*Jul 10 03:34:04.643: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to up

*Jul 10 03:34:05.643: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up

*Jul 10 03:34:09.779: %LINK-5-CHANGED: Interface Cellular0/0/0, changed state to reset

*Jul 10 03:34:09.779: Ce0/0/0 DDR: has total 0 call(s), dial_out 0, dial_in 0

*Jul 10 03:34:09.779: %DIALER-6-UNBIND: Interface Ce0/0/0 unbound from profile Di1

*Jul 10 03:34:09.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:10.207: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:10.211: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:10.779: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to down

*Jul 10 03:34:10.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:11.551: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:11.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:12.551: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:12.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:13.551: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:13.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:14.207: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:14.211: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:14.779: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to down

*Jul 10 03:34:14.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:15.551: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:15.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:15.795: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:15.799: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:16.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:16.795: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:16.799: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:17.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:17.795: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:17.799: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:18.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:19.551: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:19.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:19.795: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:19.799: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:20.771: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:20.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:21.775: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:21.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:22.775: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:22.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:23.783: Di1 DDR: No free dialer - starting fast idle timer

*Jul 10 03:34:23.795: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:23.799: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:24.775: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:24.779: Ce0/0/0 DDR: re-enable timeout

*Jul 10 03:34:24.783: Ce0/0/0 DDR: rotor dialout [best] least recent failure is also most recent failure

*Jul 10 03:34:24.783: Ce0/0/0 DDR: rotor dialout [best] also has most recent failure

*Jul 10 03:34:24.783: Ce0/0/0 DDR: rotor dialout [best]

*Jul 10 03:34:24.783: Di1 DDR: Nailing up the Dialer profile [attempt 16]

*Jul 10 03:34:24.783: Di1 DDR: Dialer dialing - persistent dialer profile

*Jul 10 03:34:24.783: Ce0/0/0 DDR: Dialing cause Persistent Dialer Profile

*Jul 10 03:34:24.783: Ce0/0/0 DDR: Attempting to dial lte

*Jul 10 03:34:24.783: CHAT0/0/0: Attempting async line dialer script

*Jul 10 03:34:24.783: CHAT0/0/0: Dialing using Modem script: lte & System script: none

*Jul 10 03:34:24.783: CHAT0/0/0: process started

*Jul 10 03:34:24.783: CHAT0/0/0: Asserting DTR

*Jul 10 03:34:24.783: CHAT0/0/0: Chat script lte started

*Jul 10 03:34:24.783: CHAT0/0/0: Sending string: AT!CALL1

*Jul 10 03:34:24.783: CHAT0/0/0: Expecting string: OK

*Jul 10 03:34:25.571: CHAT0/0/0: Completed match for expect: OK

*Jul 10 03:34:25.571: CHAT0/0/0: Chat script lte finished, status = Success

*Jul 10 03:34:26.583: Ce0/0/0 DDR: disconnecting call

*Jul 10 03:34:28.775: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:33.999: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:34.999: Di1: No free dialer - starting fast idle timer

*Jul 10 03:34:37.643: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down

*Jul 10 03:34:38.643: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down

*Jul 10 03:34:41.583: Ce0/0/0 DDR: re-enable timeout

*Jul 10 03:34:42.583: Ce0/0/0 DDR: rotor dialout [best] least recent failure is also most recent failure

*Jul 10 03:34:42.583: Ce0/0/0 DDR: rotor dialout [best] also has most recent failure

*Jul 10 03:34:42.583: Ce0/0/0 DDR: rotor dialout [best]

*Jul 10 03:34:42.583: Di1 DDR: Nailing up the Dialer profile [attempt 17]

*Jul 10 03:34:42.583: Di1 DDR: Dialer dialing - persistent dialer profile

*Jul 10 03:34:42.583: Ce0/0/0 DDR: Dialing cause Persistent Dialer Profile

*Jul 10 03:34:42.583: Ce0/0/0 DDR: Attempting to dial lte

*Jul 10 03:34:42.583: CHAT0/0/0: Attempting async line dialer script

*Jul 10 03:34:42.583: CHAT0/0/0: Dialing using Modem script: lte & System script: none

*Jul 10 03:34:42.583: CHAT0/0/0: process started

*Jul 10 03:34:42.583: CHAT0/0/0: Asserting DTR

*Jul 10 03:34:42.583: CHAT0/0/0: Chat script lte started

*Jul 10 03:34:42.583: CHAT0/0/0: Sending string: AT!CALL1

*Jul 10 03:34:42.583: CHAT0/0/0: Expecting string: OK

*Jul 10 03:34:43.671: CHAT0/0/0: Completed match for expect: OK

*Jul 10 03:34:43.671: CHAT0/0/0: Chat script lte finished, status = Success

*Jul 10 03:34:45.671: %LINK-3-UPDOWN: Interface Cellular0/0/0, changed state to up

*Jul 10 03:34:45.671: Ce0/0/0 DDR: Dialer statechange to up

*Jul 10 03:34:45.671: %DIALER-6-BIND: Interface Ce0/0/0 bound to profile Di1

*Jul 10 03:34:45.671: Ce0/0/0 DDR: Dialer call has been placed

*Jul 10 03:34:45.671: Ce0/0/0 DDR: dialer protocol up

*Jul 10 03:34:45.671: Di1 DDR: Persistent Dialer Profile nailed up successfully

*Jul 10 03:34:46.671: %LINEPROTO-5-UPDOWN: Line protocol on Interface Cellular0/0/0, changed state to up

Profile Information

====================

Profile 1 = INACTIVE*

--------

PDP Type = IPv4

Access Point Name (APN) = VZWINTERNET

Authentication = None

Username:

Password:

* - Default profile

arch-eng-router1#ping 4.2.2.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 4.2.2.1, timeout is 2 seconds:

.....

Success rate is 0 percent (0/5)

Ultimately... once the Ethernet cable is attached, the Cellular 0/0/0 interface resets, attempts to connect, connects, resets, etc... Once the Ethernet cable is disconnected, the Cellular 0/0/0 interface successfully reconnects.

I am not really sure how to further troubleshoot this and was hoping someone in the community would have some additional thoughts on how to proceed.

Thanks so much for your time!

John Gentle · ‎06-12-2014

I was wondering if you ever found a solution to this issue? I have it on 15 devices scattered across the US and it is big pain.

bill.hill · ‎06-12-2014

johngentle,

This was quite a while ago... and a temporary solution. If I recall correctly, I believe there was an issue with the NAT configuration (ex: ip nat outside on the Dialer1 interface). This was a neat issue (in hindsight). But, nothing that carried much value going forward. So, I do apologize for not having more information for you.

Thanks!

~Bill

jonathanvanderzee · ‎06-12-2014

I have recently dealt with a similar issue and it was due to NAT not being configured correctly causing the cell card to flop. You really have to NAT all inside traffic regardless of whether is it allowed out or not otherwise Verizon detects it as invalid traffic and drops your cell connection. The blocking of outside traffic to the outside interface should be handled as a "in" ACL on the LAN interface. See some of my config below with some other lines of code I found helpful, tailor to your needs. Let me know how it goes.

ip access-list extended NAT_Traffic
!This is denying any VPN traffic that is outbound on the external interface. If this traffic allows will flop the cell interface.
deny ip any 10.254.254.0 0.0.0.15
!You can modify this to only be the LAN subnets you are actually using but try "any any" just to test.
permit ip any any

ip access-list extended Limited_Internet
!This allows my LAN to communicate with my 10.x.x.x VPN subnets.
permit ip 169.254.0.0 0.0.255.255 10.254.254.0 0.0.0.15

!This allows the one ip address to access the internet and denies all other LAN traffic.
permit ip host 169.254.231.201 any
deny ip 169.254.0.0 0.0.255.255 any

!restrict all traffic except listed below for cell port.
ip access-list extended Secure_Access_In
!I specify the Static IP used but you will have to suit a dynamic IP.
permit tcp any host A.B.C.D eq 22

!Confugure NAT
ip nat inside source list NAT_Traffic interface Cellular0 overload

!Apply ACLs to interfaces
interface Vlan1
!Restricts internet to a limited set of IPs and allows VPN traffic to flow.
ip access-group Limited_Internet in

interface Cellular0
ip access-group Secure_Access_In in

!To allow internet access out from LAN hosts if you have a ACL blocking traffic in on the Cell interface. Always a good idea.

!What to inspect for contect based access control.
ip inspect name INSPECT-IN-OUT tcp
ip inspect name INSPECT-IN-OUT udp
ip inspect name INSPECT-IN-OUT icmp

!where to apply the outgoing CBAC inspection
interface Cellular0
ip inspect INSPECT-IN-OUT out

Hopefully you haven't pulled too much hair out and this helps to fix.

mbroberson1 · ‎12-28-2015

Hi John,

Looks like it's been a while but can you please share your solution. I have a 1921 with the Verizon 4G LTE and am running into a similar issue.

Regards

jonathanvanderzee · ‎12-28-2015

My cli commands and comments contain the fix. Just suit the network addresses to your environment.

The most important lines are:

ip access-list extended NAT_Traffic
!This is denying any VPN traffic that is outbound on the external interface. If this traffic is allowed it will flop the cell interface.
deny ip any 10.254.254.0 0.0.0.15
!You can modify this to only be the LAN subnets you are actually using but try "any any" just to test.
permit ip any any

!Confugure NAT
ip nat inside source list NAT_Traffic interface Cellular0 overload

mbroberson1 · ‎12-28-2015

Hi John,

So did you use a dialer interface anywhere in your config? I'm guessing no since your nat overload statement is referencing the cellular interface.

Regards

jonathanvanderzee · ‎12-28-2015

See the additional lines of config, hope it helps. It has been a while so I would have to dig up documents to understand the reason for these commands.

chat-script ltescript "" "AT!CALL1" TIMEOUT 20 "OK"

interface Cellular0
ip address negotiated
ip access-group Secure_Access_In in
no ip redirects
no ip unreachables
no ip proxy-arp
ip nat outside
ip inspect INSPECT-IN-OUT out
ip virtual-reassembly in
encapsulation slip
load-interval 30
dialer in-band
dialer enable-timeout 60
dialer string ltescript
dialer watch-group 1
async mode interactive
crypto map ezvpnmap

dialer watch-list 1 ip 5.6.7.8 0.0.0.0
dialer watch-list 1 delay route-check initial 60
dialer watch-list 1 delay connect 1

interface Vlan1
ip address 169.254.254.254 255.255.0.0
ip access-group Limited_Internet in
ip nat inside
ip virtual-reassembly in
ip tcp adjust-mss 1400
ip policy route-map clear-df