Strange Verizon 4G Issue Using EHWIC-4G-LTE-VZ

[jonathanw84]

We have a 1941 router at a home office that has a 4G connection for its backup connectivity. This connection has stopped working but appears to be connected to Verizon's network:

ROUTER#show cellular 0/1/0 connection
Profile 1, Packet Session Status = INACTIVE
Profile 2, Packet Session Status = INACTIVE
Profile 3, Packet Session Status = ACTIVE
Cellular0/1/0:
Data Transmitted = 0 bytes, Received = 176 bytes
IP address = 100.65.X.X.
Primary DNS address = 198.224.171.135
Secondary DNS address = 198.224.169.135
Primary DNS IPV6 address = 2001:4888:65:FF00:62E:D:0:0
Secondary DNS IPV6 address = 2001:4888:64:FF00:620:D:0:0

If I try to traceroute to the IP address assigned by Verizon from another computer, I get the following:

tracert -d 100.89.236.124

Tracing route to 100.65.X.X over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  10.69.254.254
  2    <1 ms    <1 ms    <1 ms  10.69.254.250
  3    <1 ms    <1 ms    <1 ms  198.X.X.X
  4     4.4.X.X  reports: Destination net unreachable.

This to me indicates some kind of route advertisement issue in Verizon Wireless' network. Shutting down the interface and bringing it back up will generate a new IP but the result is the same. 

Is anyone else experiencing anything similar? I'm guessing I should just reach out to Verizon at this point. 

Thanks!

[pieterh]

1) there may be an access list present on the router that blocks icmp requests (traceroute) 

2) can you ping to the Verizon gateway from the router ?

3) ask your provider to check the card has not exceeded its data-limit ?

>>> Shutting down the interface and bringing it back up will generate a new IP but the result is the same <<<
this shows your router has a dynamic IP-address
4) does your local configuration allow connection from changing addresses?
    or do you need to modify your local endpoint's configuration to accept this new address every time it changes?

[jonathanw84]

Hello,

Thanks for reply.

1. No ACL in place right now. I removed everything to be sure. Also, the "destination net unreachable" from another computer tells me there is route to these subnets on the public internet.

2. I cannot ping the gateway because it doesn't give you. The standard configuration for this assigns the IP automatically and then you point a default to the interface. 

3. I do plan on calling Verizon but I doubt that this is the case since this plan should have unlimited data. 

4. Yes, IP address on the cellular interface is set to negotiated so a dynamic IP address that can be change is fine. 

I definitely think that something is up with Verizon's service because even from routers that I receive full routes from the ISP via BGP, these IP addresses are not routable:

LAS-IR01-1001X-E-B1F1R03.05-36#show ip route 100.95.X.X
% Subnet not in table

I'll give Verizon a call today. Just wanted to throw this out here in case anybody else has seen something similar as of recently with Verizon's 4G service on an HWIC card.

Thanks! 

[Elliot Dierksen]

First off, you have a default route so that is what is applicable for finding that remote address. Second, that is part of the CGNAT (Carrier Grade NAT) space 100.64.0.0 to 100.127.255.255 (aka 100.64.0.0/10). It isn't a public IP. That looks like there is some kind of remote ACL blocking you, so it is out of your control. I'll add something from my own experience with Verizon which is to make sure you don't let any traffic leave your router without being NAT'd. This is in the North Carolina area, but they would drop the cell call if any traffic left my router destined for RFC1918 addresses.

[jonathanw84]

Thanks Elliot! This makes complete sense and now that I'm thinking about it, I recall having the same issue years ago with Verizon. What I didn't recall is getting an IP from that range from them, but perhaps I did but just didn't remember it.

Do you have any suggestions to prevent this from occurring? We have a primary internet circuit that we NAT a few networks too, and have even specified the interface on the route map used for the NAT to ensure traffic wasn't going over the cellular interface. So I'm unsure as to what would be causing this since traffic should only be traversing the cellular connection if a failover occurs.

Thanks! 

[Elliot Dierksen]

This is what I have used to keep Verizon from dropping calls based on an IP they don't like.

ip access-list extended dialer-out
 permit tcp 100.64.0.0 0.63.255.255 any
 permit udp 100.64.0.0 0.63.255.255 any
 permit icmp 100.64.0.0 0.63.255.255 any
 permit esp 100.64.0.0 0.63.255.255 any
 deny   ip any any log
!
interface Dialer1
 ip access-group dialer-out out