- Cisco Community
- Technology and Support
- Security
- Security Knowledge Base
- ASA-PIX/FWSM: Multicast tips and common problems
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
09-08-2010 07:38 PM - edited 02-21-2020 09:57 PM
- Introduction:
- Documentation:
- Support:
- Configuring multicast:
- Shared tree and Source specific tree:
- Where to add these commands:
- igmp join-group:
- igmp static-group:
- igmp forward interface:
- Terminology:
- What is a first hop router:
- What is a last hop router:
- OIL:
- In coming interface and out going interface for (*,G):
- In coming interface and out going interface for (S,G):
- DR - How to increase the pim priority:
- PIM Chart:
- How to interpret the "sh mroute x.x.x.x"
- sample 1:
- sample 2:
- sample 3:
- How to interpret the "sh mfib x.x.x.x"
- sample 1:
- sample 2:
- Common Problems:
- syslog 106010:
- mfib limit (5000) reached:
- Failed to locate egress interface:
- HSRP:
- TTL (Time to Live Value) too low or set to 1:
- Multlicast over VPN tunnel:
- FP no mcast output intrf
- Debug pim group x.x.x.x shows Assert processing message wins
- Troubleshooting commands:
- On the Firewall:
- On the Router:
- Packet capture sample:
Introduction:
The purpose of this document is to provide assistance to everyone in configuring and troubleshooting multicast through the firewall.
Documentation:
This document is meant to be interpreted with the aid of the official documentation from the configuration guide located here:
ASA:http://www.cisco.com/en/US/docs/security/asa/asa82/configuration/guide/route_multicast.html
FWSM:http://www.cisco.com/en/US/docs/security/fwsm/fwsm32/configuration/guide/ip_f.html#wp1041648
PIX:http://www.cisco.com/en/US/docs/security/pix/pix63/configuration/guide/bafwcfg.html#wp1170913
The Cisco TAC has created another ASA Multicast Troubleshooting and Common Problems guide and posted it to Cisco.com:
http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080befd2a.shtml
Support:
ASA supports igmp forwarding, sparse mode, and bidirectional mode. There is no sparse-dense-mode support. ASA forwards auto-rp packets (unless configured to not to). ASA itself requires static auto-rp config.
Configuring multicast:
Pls. follow this link: http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a00807631d2.shtml
Shared tree and Source specific tree:
I read once that Multicast is like dating service. Sender starts the stream and the DR on the local LAN registers the sender with the dating service (RP). Reciever sends IGMP report and the DR on that local LAN segment sends a PIM join towards the RP- shared tree. Once the dating service connects the two - sender and receiver, then they can talk via the shortest path and do not have to go through the dating service - shared tree anymore.
Where to add these commands:
igmp join-group:
This command when applied under the interface, sends a join for the group address. Think of this as a permanent receiver for the group. This comes in handy when the receivers off of the interface do not send igmp reports in a regular interval. This command makes the firewall to accept and forward the multicast packets. Configuring the firewall to join a multicast group causes upstream routers to maintain multicast routing table information for that group and keep the paths for that group active.
Where to add this command: This command needs to be configured under the interface config mode facing the receivers.
hostname(config-if)# igmp join-group group-address
igmp static-group:
The firewall does not accept the multicast packets but rather forwards them to the specified interface. To configure the security appliance to forward the multicast traffic without being a member of the multicast group, use the igmp static-group command. When this command is added, the ASA sends out an IGMP report out the interface sourcing with its IP address on the interface.
Where to add this command: This command needs to be applied under the interface config facing the receivers.
hostname(config-if)# igmp static-group group-address
igmp forward interface:
Where to add this command: This command when applied under the interface config facing the receivers.
This command will forward all the igmp reports received on the interface towards the interface where the server is located. This command cannot be configured along with PIM.
hostname(config-if)# igmp forward interface outside
Terminology:
What is a first hop router:
A first hop router is the router that is connected to the source which is responsible for registering the source with the RP. If there are two routers connected to the same segment as the source then the one that is the DR (designated router) for that LAN segment will take care of the registering process. Now, if one of the two is the firewall and you want the firewall to take care of the registration process because the RP is on the other side of the firewall then, the firewall should be DR for that LAN segment.
What is a last hop router:
A last hop router is the one that is connected to the receiver. Again if there are two or more routers in the LAN connected to the receiver, the router which is the DR is the one that is resonsible to connect the reciver to the shared tree. If one of the devices is the firewall then, if we want the firewall to process the igmp reports then it has to be the DR with a higher priority.
OIL:
OIL stands for Outgoing Interface List. OIL is always taken from the (*,G) and copied as the OIL for the (S,G).
In coming interface and out going interface for (*,G):
Incoming interface for the (*,G) is always towards the RP. Out going interface is towards the receiver.
In coming interface and out going interface for (S,G):
Incoming interface for (S,G) is always towards the source. Out going interface is towards the receiver.
DR - How to increase the pim priority:
The following example sets the DR (Designated Router) priority for the interface to 5:
hostname(config-if)# pim dr-priority 5
PIM Chart:
(*,G)
(S,G)
| When it is created |
|
|
| OIL info |
|
|
| RPF info |
|
|
| IIF Info |
|
|
| When this entry is deleted? |
|
|
How to interpret the "sh mroute x.x.x.x"
sample 1:
topology:
Receiver--RP---(INT_SCI/8)ASA(INT_MANDAT/0)--(10.2.112.9)source
Group address: 239.252.1.10
ASA#sh mroute 239.252.1.10
Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group,
C - Connected, L - Local, I - Received Source Specific Host Report,
P - Pruned, R - RP-bit set, F - Register flag, T - SPT-bit set,
J - Join SPT
Timers: Uptime/Expires
(10.2.112.9, 239.252.1.10), 00:10:48/00:03:11, flags: SFT
Incoming interface: INT-MANDAT
RPF nbr: 10.2.112.9, Registering
Outgoing interface list:
INT-SCI, Forward, 00:10:48/00:03:29
Tunnel0, Forward, 00:10:48/never
There is no *,G which indicates that the receiver hasn't requested the feed yet. Only the source has started and the source lives
behind the interface named INT-MANDAT. "F" - is the Register Flag. Meaning the firewall hasn't seen a register stop message
from the RP yet. The "Tunnel0" interface is a transient state and goes away once the register stop is received.
RP lives behind the INT-SCI interface.
sample 2:
RPF neighbor all zeros means that the ASA is the first hop router connected local to the sender.
When you see the "T" flat you can jump in joy. This mean the shortest path tree has formed and multicast traffic is being received by the receiver.
topology:
receiver--(inside)ASA(outside)--sender
ASA#sh mroute 230.0.0.10
(*, 230.0.0.10), 00:08:20/never, RP 0.0.0.0, flags: SCL (Connected Local LAN)
Incoming interface: Null -----> receiver is directly connected
RPF nbr: 0.0.0.0
Immediate Outgoing interface list: ------> receiver is off of the inside interface
inside, Forward, 00:08:20/never
(10.135.152.47, 230.0.0.10), 00:08:16/00:03:13, flags: SJT (shortest Path Tree - T)
Incoming interface: outside ------> source is on the outside interface.
RPF nbr: 0.0.0.0
Inherited Outgoing interface list:
inside, Forward, 00:08:20/never ------> receiver is off the inside interface.
sample 3:
topology:
sender(172.25.1.11)--RP(172.20.50.254)--vlan50--nonpci(0)ASA(0)WLAN--vlan30--Receiver(172.20.30.133)
same security level - 0
Group add 225.4.5.7
ASA#sh mroute 225.4.5.7
(*, 225.4.5.7), 01:08:42/never, RP 172.20.50.254, flags: SCLJ
Incoming interface: nonpci ------> (routing table shows int nonpci towards the RP)
RPF nbr: 172.20.50.254
Immediate Outgoing interface list: ------> (receiver is off the interface WLAN)
WLAN, Forward, 01:08:42/never
(172.25.1.11, 225.4.5.7), 00:04:59/00:03:00, flags: SJT
Incoming interface: nonpci ------>(routing table shows int nonpci for the source)
RPF nbr: 172.20.50.254
Inherited Outgoing interface list:
WLAN, Forward, 01:08:42/never------> (receiver is off the interface WLAN)
How to interpret the "sh mfib x.x.x.x"
sample 1:
topology:
(10.204.125.81)R1-------(10.204.125.85/OUT)***ASA***(IN/172.25.250.11)-R2(172.25.250.1)--receiver.
I
--------------
| |
Sender-10.29.95.133 RP-10.29.95.252
Group address: 233.49.81.127
Source or source is on the lower security interface and the receiver is on the higher security interface. It is very important to see the
"F" flag so indicate that the ASA is forwarding multicast traffic.
ASA5520-01# sh mfib
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
AR - Activity Required, K - Keepalive
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
Interface Flags: A - Accept, F - Forward, NS - Negate Signalling
IC - Internal Copy, NP - Not platform switched
SP - Signal Present
Interface Counts: FS Pkt Count/PS Pkt Count
(*,233.49.81.127) Flags: C K
Forwarding: 0/0/0/0, Other: 846486/2/846484
OUT Flags: A NS ----> We are accepting packets on the outside interface
inside Flags: F NS ---> We are forwarding packets to the inside interface.
Pkts: 0/0
(10.29.95.133,233.49.81.127) Flags: K
Forwarding: 1/0/36/0, Other: 0/0/0
OUT Flags: A
inside Flags: F NS
Pkts: 0/1
sample 2:
topology:
receiver--(inside)ASA(XETRA)--router--N/W-RP-Source
source: 193.29.93.62
Group address: 224.0.46.0
ASA5520-01# sh conn
UDP XETRA 193.29.93.62:25100 inside 224.0.46.0:55199, idle 0:00:00, bytes 2649468, flags -
ASA5520-01# sh mfib 224.0.46.0 193.29.93.62
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
AR - Activity Required, K - Keepalive
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
Interface Flags: A - Accept, F - Forward, NS - Negate Signalling
IC - Internal Copy, NP - Not platform switched
SP - Signal Present
Interface Counts: FS Pkt Count/PS Pkt Count
(193.29.93.62,224.0.46.0) Flags: K
Forwarding: 18629/10/1295/105, Other: 4716/0/4716
XETRA Flags: A F ---> XETRA interface is where the source is and we Accept packets from.
Pkts: 0/0
inside Flags: F NS ---> inside interface is where the receiver is where we Forward the packets to.
Pkts: 1598/2
Common Problems:
syslog 106010:
%FWSM-3-106010: Deny inbound udp src OUTSIDE:10.11.21.10/1450 dst DMZ-EMP90:239.226.16.3/30120
With nat-control enabled on the FWSM the above issue can be corrected with the following:
nat (inside) 0 access-list 101
access-list 101 permit ip host 239.226.16.3 host 10.11.21.10
Think of the above as the source when sends out multicast packets the destination is the multicast address 239.226.16.3
and is destined towards the inside. Even though no traffic will be sourced from a multicast address, we just need to provide
translation for the reverse flow from high security to low security.
If this case were an ASA we would see the following in asp drop captures
1. ASA was dropping all the multicast packet for the reason below:
union-asa# sh cap capasp | i 239.0.1.2
38: 12:07:56.782689 10.80.8.38> 239.0.1.2: icmp: echo request Drop-reason: (no-mcast-intrf)
2. nat 0 with acl was configured on this ASA and it didn't allow exemption for our group 239.0.1.2.
Once I added that flow reached the other ASA2 and we saw *,g as well as s,g
mfib limit (5000) reached:
"sh mfib <group-address>" may not show any output. This may due to the fact that the mfib entries max limit may have been reached. This ENH request CSCtj22365 is filed so, that when resolved, the FWSM will send a syslog message indicating that the max mfib entry limit has been reached.
fwsm# sh mfib sum
IPv4 MFIB summary:
5000 total entries [4974 (S,G), 23 (*,G), 3 (*,G/m)]
190440 total MFIB interfaces
Failed to locate egress interface:
Oct 23 2010 21:40:44: %ASA-6-110002: Failed to locate egress interface
for UDP from outside:10.135.152.47/1034 to 230.0.0.10/7060
Look for contradicting lines in the config. In this case where does the sender live on the inside or outside?
static (inside,outside) 10.135.152.47 10.135.152.47 net 255.255.255.255
mroute 10.135.152.47 net 255.255.255.255 outside
In the above case we had to remove the static and "toggle" multicast-routing on the ASA.HSRP:
Let us say that the RP is a loop back address configured on a pair of routers running HSRP. We need to make sure the mroute configured on the firewall points to one of the physical IP addresses and not the HSRP address. When we issue "sh pim int <name>" , the firewall only forms neighbor relationship with the physical IP addresses and not the HSRP address. Also, we cannot configure mroute to both the physical IP addresses. There is no redundancy when it comes to multicast.
Pls. read this link for further explanation: http://www.cisco.com/en/US/tech/tk828/technologies_tech_note09186a0080094aab.shtml
TTL (Time to Live Value) too low or set to 1:
When a router forwards a multicast packet from one interface to another, it decrements the Time To Live (TTL) value in the IP header by one. The firewall does not decrement the TTL value in the IP header but, still it will drop the packet if the TTL is set to 1. It sends the packet only if the resulting TTL value is not zero and is greater than the multicast TTL threshold value of the outgoing interface when configured. If a multicast application on the source allows the setting of a TTL value and results in not matching the mentioned criteria, the packets are dropped by the router. The multicast source needs to set the TTL value as there are number of hops between the source and the receiver.
Multicast TTL threshold:
Multicast TTL value too low or set to 1:
Multlicast over VPN tunnel:
ASA/PIX will not pass multicast traffic over IPsec VPN tunnels. Only unicast traffic over VPN tunnel is supported. The alternative option is to use GRE between two end points on either side of the tunnel and send multicast over GRE and encrypt that traffic and send it over the tunnel.
FP no mcast output intrf
R1(receiver)--(inside)ASA(outside)---R3(rp/sender)
Group 239.1.1.1
"capture capasp typs asp-drop all" may show the following:
ASA1# sh cap capasp | i 239.1.1.1
18: 14:52:02.979731 10.7.123.3.64281 > 239.1.1.1.5000: udp 16 Drop-reason: (no-mcast-intrf) FP no mcast output intrf
Here is the command reference link to sh asp-drop:
Name: no-mcast-intrf
FP no mcast output intrf:
All output interfaces have been removed from the multicast entry.
- OR -
The multicast packet could not be forwarded.
Recommendation:
Verify that there are no longer any receivers for this group.
- OR -
Verify that a flow exists for this packet.
Syslogs:
None
In this case R1 being the receiver was also the DR for that segment so, the ASA did not do anything with the
IGMP reports that it received. In this case making the ASA as the DR resolved the issue.
Debug pim group x.x.x.x shows Assert processing message wins
Topology:
FWSM (10.20.213.1)----vlan 13----FWSM---vlan300--RTR----RP---Sender (172.16.41.205)
|
Receivers (10.20.213.204)----|
|
ASA(10.20.13.2)----|
IGMP reports from the receivers arrive on the FWSM but, the FWSM doesn't send PIM join up to the RP. The egress multicast packets do not show the joins due to CSCsf31515
Debug pim group 239.1.1.247 showed the following:
IPv4 PIM: (172.16.41.205,239.1.1.227)RPT J/P adding Prune on OUTSIDE
IPv4 PIM: (172.16.41.253,239.1.1.227)RPT J/P adding Prune on OUTSIDE
IGMP: Received v2 Report on INSIDE from 10.20.213.201 for 239.1.1.227
IGMP: Updating EXCLUDE group timer for 239.1.1.227
IPv4 PIM: (172.16.41.205,239.1.1.227) Received [15/110] Assert from 10.20.13.2 on INSIDE
IPv4 PIM: (172.16.41.205,239.1.1.227) Assert processing message wins
IPv4 PIM: (172.16.41.205,239.1.1.227) INSIDE Update assert timer (winner 10.20.13.2)
FWSM (backup site)
ASA (production site)
FWSM and ASA though configured to be on the same vlan, were not supposed to see each other. The problem is that the FWSM saw the presence of the ASA but the ASA didn't see the presence of the FWSM. FWSM was DR on VLAN 13 and the ASA was DR (since it did not see the FWSM) as well. Both tried to process multicast packets and ended in a race condition and the Assert messages and winner indicated that.
For more information read this link:
Troubleshooting commands:
On the Firewall:
sh pim neighbor
sh igmp interface
sh igmp traffic
sh mroute x.x.x.x
sh mfib x.x.x.x
debug pim group x.x.x.x
debug igmp group x.x.x.x
On the Router:
sh ip pim interface
sh ip mroute x.x.x.x
sh ip igmp group
debug ip pim
Packet capture sample:
http://packetlife.net/captures/category/multicast/
1. igmp report - IGMP Ver 2 - Multicast packet, destined to the multicast group address.
2. PIM join - PIM Ver 2 - Multicast packet destined to 224.0.0.13
3. Register packet - Is a Unicast packet sent from First hop router to RP address and the multicast packet is encapsualted.
4. Register Stop - Is a Unicast packet sent from the RP IP address to first hop router.
- Mark as Read
- Mark as New
- Bookmark
- Permalink
- Report Inappropriate Content
Excellent stuff Kureli!
- Mark as Read
- Mark as New
- Bookmark
- Permalink
- Report Inappropriate Content
I know this is a document regarding ASA/PIX, but if you have AutoRP configured on your routers, can you use the AutoRP address as the RP address for unsupported devices like ASA/PIX?
If that is supported, does the answer change if you use AnycastRP's peered with MSDP? Should you use a separate (non-anycast) loopback on the anycast routers as the PIM RP?
- Mark as Read
- Mark as New
- Bookmark
- Permalink
- Report Inappropriate Content
Thank you for this wonderful set of explanations and details.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: