Re: Multicast Image Deployment failing

chris.macleod · ‎06-26-2009

Hi,

I am having some problems when deploying images via Multicast using Windows Deployment Server. All imaging sessions are failing after a few minutes. While troubleshooting this issue we tried different images and they fail also - unicast works fine. We have also moved the server to the same switch as the clients and the imaging completes successfully.

Hardware:

Core: Cisco 4507re Sup 6E (All line cards are classic)

Access: Mix of 2950G, 3550, 3560

Switch stacks consist of mix of up to three 2950g and 3550/60 each stack connects to our single 4507re.

Setup:

Clients and Deployment Server are in the same VLAN

All access Switches have default IGMP settings (enabled, v2)

Core 4507 has IP Multicast Routing

Int VLAN x

ip pim sparse-dense mode

Any ideas what I am doing wrong?

Thanks

Chris

Giuseppe Larosa · ‎06-26-2009

Hello Chris,

>> Clients and Deployment Server are in the same VLAN

You should verify that the the 4507 is sending IGMP queries and that the clients answer to them.

The issue could be related to IGMP snooping that attempts to optimize multicast forwarding.

You can verify IGMP snooping settings with

sh ip igmp snooping on switches

Hope to help

Giuseppe

chris.macleod · ‎06-26-2009

Hi Giuseppe,

Thanks for your reply. Here is the output for the show ip igmp snooping command. The Multicast source is on the 2960 and destination is one client on each 2950.

Core_Switch#show ip igmp snooping

Global IGMP Snooping configuration:

-----------------------------------

IGMP snooping : Enabled

IGMPv3 snooping : Enabled

Report suppression : Enabled

TCN solicit query : Disabled

TCN flood query count : 2

Last Member Query Interval : 1000

Vlan 1:

--------

IGMP snooping : Enabled

IGMPv2 immediate leave : Disabled

Explicit host tracking : Enabled

Multicast router learning mode : pim-dvmrp

Last Member Query Interval : 1000

CGMP interoperability mode : IGMP_ONLY

ESK_2950_5#show ip igmp snooping

Global IGMP Snooping configuration:

-----------------------------------

IGMP snooping : Enabled

IGMPv3 snooping (minimal) : Enabled

Report suppression : Enabled

TCN solicit query : Disabled

TCN flood query count : 2

Last member query interval : 1000

Vlan 1:

--------

IGMP snooping : Enabled

Immediate leave : Disabled

Multicast router learning mode : pim-dvmrp

Source only learning age timer : 10

Last member query interval : 1000

CGMP interoperability mode : IGMP_ONLY

ESK_2950_6#show ip igmp snooping

Global IGMP Snooping configuration:

-----------------------------------

IGMP snooping : Enabled

IGMPv3 snooping (minimal) : Enabled

Report suppression : Enabled

TCN solicit query : Disabled

TCN flood query count : 2

Last member query interval : 1000

Vlan 1:

--------

IGMP snooping : Enabled

Immediate leave : Disabled

Multicast router learning mode : pim-dvmrp

Source only learning age timer : 10

Last member query interval : 1000

CGMP interoperability mode : IGMP_ONLY

CALC1_2960_2#show ip ig snooping

Global IGMP Snooping configuration:

-------------------------------------------

IGMP snooping : Enabled

IGMPv3 snooping (minimal) : Enabled

Report suppression : Enabled

TCN solicit query : Disabled

TCN flood query count : 2

Robustness variable : 2

Last member query count : 2

Last member query interval : 1000

Vlan 1:

--------

IGMP snooping : Enabled

IGMPv2 immediate leave : Disabled

Multicast router learning mode : pim-dvmrp

CGMP interoperability mode : IGMP_ONLY

Robustness variable : 2

Last member query count : 2

Last member query interval : 1000

I will enable debugging to verify that queries are being sent/received.

chris.macleod · ‎06-26-2009

Hi,

I have attached some of the debug output for debug ip igmp snooping groups.

Source is connected on 2960

Destination is on seperate stack on the 2950.

Multicast address begins 232.

The 2950 and 2960 look to be sending/receiving queries but I can't see the 4507 sending and receiving any. Are reports the same thing?

Thanks for your help.

Giuseppe Larosa · ‎06-26-2009

Hello Chris,

IGMP snooping activity looks like correct there are queries and replies.

Also core actvity looks like fine the output is different but messages like

003737: Jun 26 14:51:03: IGMPSN: group: Received V2 report for group 239.192.83.80 received on Vlan 1, port Gi1/3

003738: Jun 26 14:51:03: IGMPSN: group: Adding client ip 172.16.62.5, port_id Gi1/3, on vlan 1

003739: Jun 26 14:51:03: IGMPSN: group: Added port Gi1/3 to group 239.192.83.80

are sign of correct behaviour for a L3 device.

Check the trunk ports between the swiches looking for error counters.

What is the bit rate of the streaming?

Are these high definition images in health environment?

Hope to help

Giuseppe

chris.macleod · ‎06-26-2009

Hi,

I have checked both sides of each link from the source to the clients - no errors at all.

The deployment server shows a transfer rate of 11000-12000KBp/s. The rate remains pretty stable for around 3 minutes then drops to 0Kbp/s.

The images are just for a classroom - just under 10GB in size

Thanks again.

Giuseppe Larosa · ‎06-26-2009

Hello Chris,

>> The deployment server shows a transfer rate of 11000-12000KBp/s. The rate remains pretty stable for around 3 minutes then drops to 0Kbp/s.

sorry for the basic question:

But do you mean you see the rate that drops to zero on the switch port connected to the server ?

Hope to help

Giuseppe

chris.macleod · ‎06-26-2009

Sorry I should have said.

I am getting the transfer rate from the Windows Deployment Services console on our Win2008 Server.

Thanks

Chris

Giuseppe Larosa · ‎06-26-2009

Hello Chris,

another thought about igmp snooping:

igmp reports are suppressed by L2 switches.

I wonder if server expects to see igmp reports as a form of feedback from receivers

if so it could stop not hearing the reports

or it expects some different form of feedback.

To check this have an MRTG to graph the usage link on the switch port during an attempt

Hope to help

Giuseppe

chris.macleod · ‎06-26-2009

Hi Giuseppe,

We have actually been using Windows Deployment Services successfully until now. Is it still worth checking? We don't use MRTG but have some solar winds software that we could get some usage graphs from will this be okay?

Thanks for your time.

Chris

Giuseppe Larosa · ‎06-26-2009

Hello Chris,

you could use solar winds as well.

these are the most difficult problems to imvestigate.

You say it worked well up to now.

Verify what have you changed in the network in recent times

Hope to help

Giuseppe

chris.macleod · ‎06-29-2009

Hi,

I have attached a graph that plots rx+tx on the source and two receivers switch ports. The 2960 is the source (1GB Interface) and the two 2950 are the receivers (100Mb).

As for changes I have added some VLAN's recently and added pim config to them as well as a helper address pointing to the deployment server. I have removed this and am now struggling to find anything else that may be causing this problem.

I have checked the switches and they have found a querier and mrouter (4507re).

Thanks

Chris

nate-miller · ‎06-27-2009

3 minutes is the default group membership time.

http://www.cisco.com/application/pdf/paws/68131/cat_multicast_prob.pdf

There's headaches when you've got multicast across multiple switches- disable IGMP snooping (and turn the thing into a flooding scenario!) or enable the igmp querier feature. This may help. I've been having the same problem in the same vlan on the SAME 6500, for what it's worth.

Jake Gillen · ‎06-26-2009

Hi Chris,

You say that the sessions fail after a few minutes. Do the clients ever get any data, or is it like they are hanging and waiting?

If there is more than a single router hop between the clients and the server, I would see if there is a TTL setting and set it to a higher value.

Hope that's not too dumb!

Jake

Jake Gillen · ‎06-26-2009

Nevermind. Same VLAN. Sorry.