This is a cross post from the cisco-voip mailing list. Just trying to broaden my viewer base. That means you aokanlawon You're awesome! But seriously, anyone who can help out, I am grateful for it.
I am troubleshooting a complex call flow with multiple clusters all connected via Non-Gatekeeper Controlled H.323 ICT's, and have narrowed down the failure to a region misconfiguration between a trunk and its transcoder.
My question to the group is about the H.245 MasterSlaveDetermination process, and how CUCM handles this between two CUCM's. Because, as you might see below, it seems to be broken, or at least unpredictable. Or maybe my lack of understanding is blinding me to the obvious answer.
First, let me start with a graphic of the H.225/H.245 messaging between two clusters. What you will see here is that TCS is sent in both directions, and the the initiating side sends an H.245 endSession to the receiving side, before the MasterSlaveDetermination is sent or received by either party.
What the graphic doesn't show, are the details within the messages, but as you may know, the H.225 messaging is littered with customer specific details, so it's a bit of a challenge to simply just post it all to this mailing list. Below are some of the details to help you understand the flow. Please let me know if you need more specific details.
We'll start with the TCS message sent by the .9 host to the .13 host
05:37:28.999 |
H245ASN - TtPid=(5551) -Outgoing #80462 -value MultimediaSystemControlMessage ::= request : terminalCapabilitySet :
{
sequenceNumber 1,
protocolIdentifier { 0 0 8 245 0 10 },
multiplexCapability h2250Capability :
{
maximumAudioDelayJitter 60,
receiveMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo FALSE,
distributedVideo FALSE
}
}
},
transmitMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo FALSE,
distributedVideo FALSE
}
}
},
receiveAndTransmitMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo TRUE,
distributedVideo FALSE
}
}
},
mcCapability
{
centralizedConferenceMC FALSE,
decentralizedConferenceMC FALSE
},
rtcpVideoControlCapability FALSE,
mediaPacketizationCapability
{
h261aVideoPacketization FALSE
},
logicalChannelSwitchingCapability FALSE,
t120DynamicPortCapability FALSE
},
capabilityTable
{
{
capabilityTableEntryNumber 1,
capability nonStandard :
{
nonStandardIdentifier h221NonStandard :
{
t35CountryCode 181,
t35Extension 0,
manufacturerCode 18
},
data '010000001400000000000000C0CEA6B5'H
}
},
{
capabilityTableEntryNumber 2,
capability genericControlCapability :
{
capabilityIdentifier standard : { 0 0 8 323 1 3 1 }
}
},
{
capabilityTableEntryNumber 3,
capability receiveAudioCapability : g729wAnnexB : 6
},
{
capabilityTableEntryNumber 4,
capability receiveAudioCapability : g729AnnexAwAnnexB : 6
},
{
capabilityTableEntryNumber 5,
capability receiveAudioCapability : g729 : 6
},
{
capabilityTableEntryNumber 6,
capability receiveAudioCapability : g729AnnexA : 6
},
{
capabilityTableEntryNumber 7,
capability receiveAndTransmitUserInputCapability : dtmf : NULL
},
{
capabilityTableEntryNumber 8,
capability receiveAndTransmitUserInputCapability : basicString : NULL
},
{
capabilityTableEntryNumber 44,
capability receiveAndTransmitUserInputCapability : hookflash : NULL
}
},
capabilityDescriptors
{
{
capabilityDescriptorNumber 0,
|*^*^*
This is the TCS received on the .9 host from the .13 host
05:37:29.211 |
H245ASN - TtPid=(5551) [0xb0efe090 3040 bytes] -Incoming #103057 -value MultimediaSystemControlMessage ::= request : terminalCapabilitySet :
{
sequenceNumber 1,
protocolIdentifier { 0 0 8 245 0 10 },
multiplexCapability h2250Capability :
{
maximumAudioDelayJitter 60,
receiveMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo FALSE,
distributedVideo FALSE
}
}
},
transmitMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo FALSE,
distributedVideo FALSE
}
}
},
receiveAndTransmitMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo TRUE,
distributedVideo FALSE
}
}
},
mcCapability
{
centralizedConferenceMC FALSE,
decentralizedConferenceMC FALSE
},
rtcpVideoControlCapability FALSE,
mediaPacketizationCapability
{
h261aVideoPacketization FALSE
},
logicalChannelSwitchingCapability FALSE,
t120DynamicPortCapability FALSE
},
capabilityTable
{
{
capabilityTableEntryNumber 1,
capability nonStandard :
{
nonStandardIdentifier h221NonStandard :
{
t35CountryCode 181,
t35Extension 0,
manufacturerCode 18
},
data '010000001400000000000000C0EE86B6'H
}
},
{
capabilityTableEntryNumber 2,
capability genericControlCapability :
{
capabilityIdentifier standard : { 0 0 8 323 1 3 1 }
}
},
{
capabilityTableEntryNumber 3,
capability receiveAudioCapability : g711Ulaw64k : 30
},
{
capabilityTableEntryNumber 4,
capability receiveAudioCapability : g711Alaw64k : 30
},
{
capabilityTableEntryNumber 5,
capability receiveAndTransmitUserInputCapability : dtmf : NULL
},
{
capabilityTableEntryNumber 6,
capability receiveAndTransmitUserInputCapability : basicString : NULL
},
{
capabilityTableEntryNumber 44,
capability receiveAndTransmitUserInputCapability : hookflash : NULL
}
},
capabilityDescriptors
{
{
capabilityDescriptorNumber 0,
simultaneousCapabilities
{
{
3,
4
},
{
5,
6
},
{
44
}
}
}
|*^*^*
What we can see now is a mismatch in supported capabilities: g729 versus g711. Now, it is my understanding that during a dispute of capabilities, the Master gets to dictate the capabilities to use.
The next two messages in the above ladder diagram are both sides ACKing each other's TCS's.
05:37:29.212 |
H245ASN - TtPid=(5551) -Outgoing #80463 -value MultimediaSystemControlMessage ::= response : terminalCapabilitySetAck :
{
sequenceNumber 1
}
|*^*^*
05:37:29.213 |
H245ASN - TtPid=(5551) [0xb0e99b58 1444 bytes] -Incoming #103058 -value MultimediaSystemControlMessage ::= response : terminalCapabilitySetAck :
{
sequenceNumber 1
}
|*^*^*
However, in the CUCM traces, between the Outgoing TCS ACK and the Incoming TCS ACK, we see CUCM Media Resource Manager comparing codecs and regions, and deciding it needs to invoke a transcoder. Even though MasterSlaveDetermination has not yet happen.
05:37:29.213 |DET-MediaManager-(22410)::preCheckCapabilities, region1=ICT-REG, region2=ICT-REG, Pty1 capCount=4 (Cap,ptime)= (15,60) (16,60) (11,60) (12,60), Pty2 capCount=2 (Cap,ptime)= (4,30) (2,30)|4,100,21,5551.2^X.X.X.13^Port 57377
05:37:29.213 |RegionsServer::MatchCapabilities -- kbps=8, capACount=4, capBCount=2|*^*^*
05:37:29.213 |DET-MediaManager-(22410)::preCheckCapabilities, caps mismatch! Xcoder Reqd. kbps(8), filtered A[capCount=4 (Cap,ptime)= (15,60) (16,60) (11,60) (12,60)], B[capCount=0 (Cap,ptime)=] allowMTP=0 numXcoderRequired=1 xcodingSide=2|4,100,21,5551.2^X.X.X.13^Port 57377
05:37:29.213 |DET-MediaManager-(22410)::prepareInitialConnectionList, Party1CapCount=4 Party2CapCount=2 XcoderRequired=1 xcodingSide=2 allowMTP=0|4,100,21,5551.2^X.X.X.13^Port 57377
The timing in the logs, and unfortunately I do not have a PCAP to compare against, is such that the transcoder fails to be invoked due to a region misconfiguration between the ICT and the transcoder. The classic mixup: "We need to transcode between g711 and g729, but the transcoder region to the g711 device is set for 8kbps."
05:37:29.213 |MediaResourceCdpc::getNextMTPDevice MtpDevice=XCODE1 Cepn=57d029f7-023b-2605-459d-af4afab55b6e devCap=0 resoureCount=1|4,100,21,5551.2^X.X.X.13^Port 57377
05:37:29.213 |MediaResourceCdpc(21468)::sendMtpAllocateRequestToDevice MtpResource=XCODE1 Cepn=57d029f7-023b-2605-459d-af4afab55b6e|4,100,21,5551.2^X.X.X.13^Port 57377
05:37:29.228 |MediaResourceCdpc(21468)::resource_rsvp_AllocateMtpResourceErr Device=XCODE1 deviceCapIntersec=0|4,100,21,5551.2^X.X.X.13^Port 57377
05:37:29.228 |MediaResourceCdpc(21468)::adjustDeviceTblXcoder - mCepn=57d029f7-023b-2605-459d-af4afab55b6e, allocateErrBitset=0x44, devCapIntersect=0x0, mandatoryCaps=0x0, resourceCount=1|4,100,21,5551.2^X.X.X.13^Port 57377
05:37:29.228 |MediaResourceCdpc(21468)::adjustDeviceTblXcoder - Codec Mismatch|4,100,21,5551.2^X.X.X.13^Port 57377
So it's at this point in the logs that we see CUCM fail the call due to no media resources available.
05:37:29.233 |!!ERROR!! -MediaManager-(22410)::disconnOnResourceAllocationFailure, ERROR disconnOnResourceAllocationFailure - fails to allocate MTP/XCoder,connCount=2|4,100,21,5551.2^X.X.X.13^Port 57377
It's now at .236 that we see the .9 host send the H.245 endSession to the .13 host
05:37:29.235 |
H245ASN - TtPid=(5550) -Outgoing #80464 -value MultimediaSystemControlMessage ::= command : endSessionCommand : disconnect : NULL
|*^*^*
So that's why the call failed: .9 failed to invoke the media resource, and thus sent an endSession.
What we don't see in the diagram above, is that in the .13 logs, it too was trying to invoke a transcoder based on the TCS it received: g729, because the endpoint: UCCX, is hard coded to g711.
22:37:29.114 |DET-MediaManager-(46351)::preCheckCapabilities, region1=ICT-REG, region2=UCCX-REG, Pty1 capCount=4 (Cap,ptime)= (15,60) (16,60) (11,60) (12,60), Pty2 capCount=2 (Cap,ptime)= (4,30) (2,30)|2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.114 |RegionsServer::MatchCapabilities -- kbps=64, capACount=4, capBCount=2|*^*^*
22:37:29.114 |DET-MediaManager-(46351)::preCheckCapabilities, caps mismatch! Xcoder Reqd. kbps(64), filtered A[capCount=4 (Cap,ptime)= (15,60) (16,60) (11,60) (12,60)], B[capCount=2 (Cap,ptime)= (4,30) (2,30)] allowMTP=0 numXcoderRequired=1
However, it invoked the transcoder successfully, and since the transcoder is 64kbps to UCCX and 8kbps to the ICT, this is why we'll see another TCS sent from the .13 host with g729 as its capabilities.
22:37:29.115 |MediaResourceCdpc::getNextMTPDevice MtpDevice=XCODE2 Cepn=26c772aa-3d19-4289-7dce-1d48d19807d7 devCap=0 resoureCount=1|2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |MediaResourceCdpc(38981)::sendMtpAllocateRequestToDevice MtpResource=XCODE2 Cepn=26c772aa-3d19-4289-7dce-1d48d19807d7|2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |MediaTerminationPointControl(1)::waiting_AllocateMtpResourceReq - (capCount,region),A(4,ICT-REG),B(2,UCCX-REG), reqDevCap=0x0, reqMandatoryCaps=0x0, supDevCap=0x129, passthru=0, resourceCount=1|2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |MediaTerminationPointControl(1)::getResourcesAllocated -- DeviceName=XCODE2 Ci=42721675 ResourceCount=1|2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |MediaTerminationPointControl(1)::getResourcesAllocated -- Logging RegionA=ICT-REG Caps and MTP/XCoder Region=XIR-REG Caps|2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |MediaTerminationPointControl(1)::logCapabilitiesinTrace -- MTP/XCoder Device Caps = 6 4 2 16 11 12 257 259 261 |2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |MediaTerminationPointControl(1)::logCapabilitiesinTrace -- Device Caps = 15 16 11 12 |2,100,21,7815.2^X.X.X.13^Port 57377
22:37:29.115 |RegionsServer::MatchCapabilities -- kbps=8, capACount=4, capBCount=9|*^*^*
Here is the TCS and MasterSalveDetermination from the .13 host that you see in the diagram above, but that's just too late, looks like this:
05:37:29.426 |
H245ASN - TtPid=(5551) [0xb0edd038 3252 bytes] -Incoming #103060 -value MultimediaSystemControlMessage ::= request : terminalCapabilitySet :
{
sequenceNumber 2,
protocolIdentifier { 0 0 8 245 0 10 },
multiplexCapability h2250Capability :
{
maximumAudioDelayJitter 60,
receiveMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo FALSE,
distributedVideo FALSE
}
}
},
transmitMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo FALSE,
distributedVideo FALSE
}
}
},
receiveAndTransmitMultipointCapability
{
multicastCapability FALSE,
multiUniCastConference FALSE,
mediaDistributionCapability
{
{
centralizedControl FALSE,
distributedControl FALSE,
centralizedAudio FALSE,
distributedAudio FALSE,
centralizedVideo TRUE,
distributedVideo FALSE
}
}
},
mcCapability
{
centralizedConferenceMC FALSE,
decentralizedConferenceMC FALSE
},
rtcpVideoControlCapability FALSE,
mediaPacketizationCapability
{
h261aVideoPacketization FALSE
},
logicalChannelSwitchingCapability FALSE,
t120DynamicPortCapability FALSE
},
capabilityTable
{
{
capabilityTableEntryNumber 1,
capability nonStandard :
{
nonStandardIdentifier h221NonStandard :
{
t35CountryCode 181,
t35Extension 0,
manufacturerCode 18
},
data '01000000140000001000E0B7C0EE86B6'H
}
},
{
capabilityTableEntryNumber 2,
capability genericControlCapability :
{
capabilityIdentifier standard : { 0 0 8 323 1 3 1 }
}
},
{
capabilityTableEntryNumber 3,
capability receiveAudioCapability : g729AnnexAwAnnexB : 6
},
{
capabilityTableEntryNumber 4,
capability receiveAudioCapability : g729 : 6
},
{
capabilityTableEntryNumber 5,
capability receiveAudioCapability : g729AnnexA : 6
},
{
capabilityTableEntryNumber 6,
capability receiveAndTransmitUserInputCapability : dtmf : NULL
},
{
capabilityTableEntryNumber 7,
capability receiveAndTransmitUserInputCapability : basicString : NULL
},
{
capabilityTableEntryNumber 44,
capability receiveAndTransmitUserInputCapability : hookflash : NULL
}
},
capabilityDescriptors
{
{
capabilityDescriptorNumber 0,
simultaneousCapabilities
{
{
3,
4,
5
|*^*^*
05:37:29.427 |
H245ASN - TtPid=(5551) [0xb0e99b58 1444 bytes] -Incoming #103061 -value MultimediaSystemControlMessage ::= request : masterSlaveDetermination :
{
terminalType 60,
statusDeterminationNumber 7594376
}
|*^*^*
Which from my experience, would tell me that the .13 host would have won the MasterSlaveDetermination, making the codec of choice be g711 and forcing the .9 host to resolve the mismatch with a transcoder. However, each side tried to fix the codec mismatch, all before MasterSlaveDetermination even happened.
Did each side jump the gun, and begin invoking the transcoders too early? Are the logs out of sequence from what actually happened on the wire? Why would both sides attempt to resolve the codec mismatch?
For comparison, here is the same message flow from the perspective of the .13 host logs.
And in summary, I know what is needed to fix this: fix the region relationship from the transcoder to the g711 device so that it's 64kbps.
I would like to understand what determines which side sends and subsequently wins MasterSlaveDetermination on ICTs? I know it's based on terminalType value and which is higher, and then a tie breaker is the statusDeterminationNumber value. Is this a random election? Is it controllable? I.e., Can I adjust the terminalType such that it forces Master or Slave by setting a very High or Low value respectively?
If you've made it this far in my email, thank you! I realize I have questions littered throughout. It's just the casual tone of my email. If the email doesn't flow very eloquently, I do apologize. It's long and I was going back and forth making edits to different sections.
Over on the mailing list, Jason Aarons pointed me to an ipexpert blog post, which says that if both terminal types are the same, then the election of Master is random. That's good to know, but in my scenario above, the call never got as far as H.245 MasterSlaveDetermination before both sides took action to resolve the codec mismatch.
Blog post: http://blog.ipexpert.com/2010/08/04/gatekeeeperh323-signaling-ras-h225-h245/
Thank you for the link Jason.
Please let me know if you need more details. Thank you for your time.
Anthony Holloway
Please use the star ratings to help drive great content to the top of searches.
