07-12-2011 11:11 AM - edited 03-11-2019 01:57 PM
I have 3 locations that are interconnected with an MPLS type of cloud provided by an ISP , it is transparent to me , currently I have all inter company traffic working but only site 1 is able to reach the internet. I'm running out of ideas and could use some more things to look at or troubleshooting steps.
this is the network diagram
site 3 uses 192.168.3.0/24
site 2 is 2.0/24
site 1 is 1.0/24
(just FYI so diagram makes more sense)
each PC in each site has its gateway set to its local router, so 2.100 (PC) has a gateway of 2.1 (its router in site 2) , 3.100 (PC) has a gateway of 3.1 (its router in site 3) etc..
All sites can reach all other sites on private subnets
for example: 192.168.3.1 can ping 2.1 and 1.1
or 2.1 can ping 3.1 and 1.1 , 100% connectivity seems to exist there.
but... only the 1.0/24 site can get out to the internet!
more examples:
1.100 (PC) can ping 1.1 (Firewall)
2.100 (PC) cannot ping 1.1 (firewall)
2.100 (PC) can ping 1.100 (PC)
1.100 (PC) can ping outside ip on internet
2.100 (PC) cannot ping outside ip on internet
there is only 1 firewall for all 3 sites, all internet traffic should go out through this one firewall, all inter-company traffic does not need to be inspected by the firewall. In theory it is a good setup (in theory, lol)
I need basic ideas of what to try at this point as I'm out of ideas.
My only route is one static route of 0.0.0.0 0.0.0.0 next_hop_IP , clearly this works for my "connected subnet" as internet access is working, why this does not work for my other two subnets is beyond me.
should I somehow specify in the firewall config that traffic from 2.0/24 and 3.0/24 is allowed?
I am trying to configure traceroutes to pass through, I did add inspect icmp to the global config and I can ping from 1.0/24 everywhere, I'm *assuming* this should allow a PC in 2.0/24 or 3.0/24 to also ping and get a reply but that's just an assumption on my part.
I don't know for sure if packets (lets say ping) from 2.100 is actually getting to 1.1 (firewall) , I'm not sure how to test that either at this point. It may just be the firewall dropping the ICMP replies to the other 2 subnets or maybe the packets don't even get there.
any futher help will once again be greatly appreciated! Thank you
07-12-2011 11:15 AM
Hi Martin,
Could you please provide a configuration from your firewall, that would be really helpful, difficult tos ay why its not working, but yes if yolu do not have any nat command for the two networks, they wont be able to access internet.
Thanks,
Varun
07-12-2011 11:25 AM
Here it is, that's good news about the NAT, I didn't think I had to do that so that might be my solution. Please advise with what commands I should add.
: Saved
:
ASA Version 8.2(1)
!
hostname ciscoasa
enable password
passwd
names
!
interface Ethernet0/0
nameif EXTERNAL
security-level 0
ip address 123.123.123.147 255.255.255.240
!
interface Ethernet0/1
nameif PROTECTED
security-level 100
ip address 192.168.1.1 255.255.255.0
!
interface Ethernet0/2
shutdown
no nameif
no security-level
no ip address
!
interface Ethernet0/3
shutdown
no nameif
no security-level
no ip address
!
interface Management0/0
nameif management
security-level 100
ip address 192.168.10.1 255.255.255.0
management-only
!
ftp mode passive
pager lines 24
logging asdm informational
mtu management 1500
mtu PROTECTED 1500
mtu EXTERNAL 1500
no failover
icmp unreachable rate-limit 1 burst-size 1
icmp permit any PROTECTED
icmp permit any EXTERNAL
no asdm history enable
arp timeout 14400
global (EXTERNAL) 101 interface
nat (PROTECTED) 101 0.0.0.0 0.0.0.0
route EXTERNAL 0.0.0.0 0.0.0.0 123.123.123.145 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
dynamic-access-policy-record DfltAccessPolicy
http server enable
http 192.168.10.0 255.255.255.0 management
http 192.168.1.0 255.255.255.0 PROTECTED
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
crypto ipsec security-association lifetime seconds 28800
crypto ipsec security-association lifetime kilobytes 4608000
telnet 0.0.0.0 0.0.0.0 management
telnet 192.168.1.0 255.255.255.0 PROTECTED
telnet timeout 5
ssh timeout 5
console timeout 0
dhcpd address 192.168.1.100-192.168.1.250 PROTECTED
!
threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
webvpn
!
class-map inspection_default
match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
parameters
message-length maximum 512
policy-map global_policy
class inspection_default
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect rtsp
inspect esmtp
inspect sqlnet
inspect skinny
inspect sunrpc
inspect xdmcp
inspect sip
inspect netbios
inspect tftp
inspect icmp
!
service-policy global_policy global
prompt hostname context
Cryptochecksum
: end
no asdm history enable
07-12-2011 11:31 AM
Hi Martin,
There is nothing wrong with nat:
global (EXTERNAL) 101 interface
nat (PROTECTED) 101 0.0.0.0 0.0.0.0
all the suers behind the PROTECTED interface aere allowed internet access. Two things you need to check now, first captures and sercond logs.
Collected the captures on the ASA and take the logs as well when the traffic gets denied by the ASA.
For taking captures:
https://supportforums.cisco.com/docs/DOC-1222
What you need to chek in the captures is, if traffic from the two subnets is reaching the firewall and if yes, does it leave out from the EXTERNAL interface.
Thanks,
Varun
07-12-2011 11:33 AM
thanks for the link, I will do this right now and will post results. Maybe the traffic isn't even reaching the firewall which would at least set me on the right path.
07-12-2011 11:36 AM
Thats right, I suspect that too.
07-12-2011 12:06 PM
well now I am thoroughly confused, lol
I did the capture for ICMP traffic only , which worked...
locally I pinged from 1.123 to a public IP
16: 11:54:07.164847 192.168.1.123 > xxx.228: icmp: echo request
17: 11:54:07.264924 xxx.228 > 192.168.1.123: icmp: echo reply
ok good I thought, then..
from site 3 , 3.1 (telneted into the router) I used routers ping utility to ping 1.1 (firewall)
1: 11:51:40.014662 xxx.150 > 192.168.1.1: icmp: echo request
2: 11:51:42.014220 xxx.150 > 192.168.1.1: icmp: echo request
no replies??? but at least it got there
here's the real confusing part
from site 3, 3.1 pinged outside ip (same as in first test above)
NOTHING in the logs at all
07-12-2011 12:17 PM
Hi Martin,
Try pinging the ip address 4.2.2.2 from a machine in site 3 and take the captures on the ASA, do not get confused with any thing, just check on the routers, whether you have a route pointing towards the ASA PROTECTED interface, thats it, do not check anything, this simple thing would clear out things for us:
access-list cap permit ip host 192.168.3.1 host 4.2.2.2
access-list cap permit ip host 4.2.2.2 host 192.168.3.1
access-list cap permit ip host 123.123.123.147 host 4.2.2.2
access-list cap permit ip host 4.2.2.2 host 123.123.123.147
capture capin access-list cap interface PROTECTED
capture capout access-list cap interface EXTERNAL
after applying these, ping 4.2.2.2
and check "show capture capin" and show capture capout"
does it show anything????
Thanks,
Varun
07-12-2011 01:02 PM
no I get 0 captures, but (there's always a but )
a direct ping to 192.168.1.1 from 3.1 results in 5 icmp: echo requests showing up in the logs, but 0 replies! that part I don't understand
but pinging 4.2.2.2. results in 0 packets captured. My ISP is continuously saying that there's nothing wrong and that the firewall is dropping packets for the 2 subnets in question , 3.0/24 and 2.0/24
07-12-2011 01:31 PM
Hi,
1. As Varun mentioned above, make sure that you have defaulr route statements at remote sites pointing to ASA.
2. Also, as the ASA not running any dynamic routing protocol, you need to add route statements for remote sites subnets on ASA pointing back to routers. route PROTECTED 192.168.2.0 255.255.255.0
hth
MS
07-12-2011 01:35 PM
I will check on both and will add the routes you mentioned, that might make a big difference. I'll report back
07-12-2011 04:45 PM
Marin,
I hope you are doing great,
Also I would like to remind you that in order to verify if the ASA is dropping packets you can do a:
capture asp-drop type asp-drop all
and after that you can do:
show capture asp-drop | include (ip address receiving the ICMP packets)
I hope this will be helpful.
Regards,
Luis Sandi
07-13-2011 05:50 AM
well.. I'm leaning towards this being just a routing issue, but I cannot figure it out for the life of me.
I added the routes back to 2.0/24 and 3.0/24 , the gateway for those was 192.168.1.253 which is the internal port of the router (orange in site 1 router in diagram above). This should work as that router has all the other necessary routes to get elsewhere which I can confirm works, but it accomplished nothing. I also tried using the public IP of the remote sites as the gateway for these routes, same result.
does this make sense?
from a PC (1.100) I can ping the public IP of router in site 2 or 3
from the firewall (1.1) using the ADSM ping tool I cannot get to either of those IPs.
something is very wrong, if I only knew what, lol
07-13-2011 06:29 AM
Hi,
Please try by enabling enable 'same-security-traffic permit intra-interface' on ASA. Also, I don't think it is required but you can enable same-security-traffic permit inter-interface as well. If you still have issues, try to ping inside ip of the ASA from remote site.
if this fails (thats what iam expecting ;-)): then go a step back- ping the site 1 router IP (192.168.1.253) ...go backwards and see where you get the reply.
if you can ping ASA inside IP: enable debug icmp trace and try to ping public IPs. you should see on ASA the logs.
hth
MS
07-13-2011 11:49 AM
I've made some changes and some progress
right now a PC 3.100 can ping firewall's PROTECTED interface (inside) , so 3.100 -> 1.1 works , now I even get replies back for the pings (not just see echo requests on firewall).
but still no internet traffic
when the same PC 3.100 tries going to 4.2.2.2 I see 0 traffic on the fw and of course it does not work.
I want to call cisco and use my smartnet, but I fear they'll just tell me to call the ISP who in turn blames everything on the firewall , oh the joys of IT!
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: