Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Create a new article
20460
Views
45
Helpful
3
Comments

General TelePresence Troubleshooting Template Questions

Matthew Limbrick
Cisco Employee
Cisco Employee

‎08-27-2015 08:53 AM - edited ‎12-05-2022 06:48 AM

Introduction


 
This document describes a list of questions you can ask when troubleshooting different technologies and may not pertain to all scenarios. If you have additional information that can be added, please post them below or send me a direct email (mlimbric@cisco.com) in order to add them to this post.
 
The questions and attached files can be sent to a customer/ partner for use in troubleshooting purposes.
 
In general, I found to get more response from a customer (on an initial email) if you start off asking for all the deployment configurations such as call flow, xstat/ xconfig or log files of endpoints along with same from infrastructure (TMS, VCS, IP addresses, software versions, device models etc). Once you understand the issue and can mentally map out the deployment topology, you will then have a baseline on where to start to troubleshoot. I have also found that asking for everything at once (deployment configs and log captures) is usually overwhelming for the customer/ partner, hence I break it up into 2 parts. Also having these up front in the case allows it to be easier for others to assist such as your mentors, collaborators and / or BU.
 
After the deployment information is gathered, you then want to focus on troubleshooting the issue, such as running DEBUG level logging and taking packet captures if needed. Already having the deployment configurations are helpful when looking at the logs so that you can determine what IP addresses belong to which device.
 
Contributed by Matt Limbrick, Cisco TAC Engineer.
 
 

Requirements

 
There are no specific requirements for this document.
 
 

Components Used

 
This document is not restricted to specific software and hardware versions.
 
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
 
 
 
 
General TelePresence Troubleshooting Questions
 

General:




RCA: (Root Cause Analysis)


- Please keep in mind that most logs are set to INFO level by default and may not contain enough details to provide an accurate Root Cause Analysis (RCA) unless previously changed to DEBUG level logging prior to the issue. In addition, there is a possibility that the logs in regard to your incident may not be available as they are sometimes overwritten with newer entries to save space, so it is important to provide the requested logs ASAP.
 
 
 

Why call details are needed with logs:

I see that you provided some logs to the case for further review however there are no call details associated. In an effort to isolate the issue faster, we would require a general understanding of your particular deployment, I.e. call flow from end to end, what devices exist in the call path, how these devices are configured, time & date of the issue in the logs, where the logs were taken and what exactly was experienced. This would be the key to understanding the logs provided.

- I will need debugs (to see negotiated messaging) and packet captures to see if there is any network related issues, I.e. Jitter, packet loss, no audio or video packets at all

- Please provide a screen shot of any error along with the procedure you performed to replicate the issue

- What is the call flow from end to end and can you provide the call topology (i.e. EP > SIP > VCS > SIP > MCU)?
 
- As this seems to be a network issue, it will be imperative to take packet captures and debug logging at each device the call passes through.

- Please replicate the call with debug logging and/or packet captures enabled with the procedures below:

*** Please provide the call details of the recreate with the following information:***
- Time and date of test?
- Who dialed who including their IP addresses?
- Where are they registered?
- What protocol was used (SIP/ 323 etc)
- What was observed?

- Please list ALL endpoint models and software versions that were in the call at the time of the issue.

- At what time and date did you replicate the issue?

- Please provide IP addresses and configurations of these units including model and software versions

- Please enable debug level logging of these units and replicate the issue, providing us the output afterwards.
 
- As per the TC6.2 release notes pg.40, software release keys are no longer needed. If you are on software lower than TC6.2, I would advise to upgrade to the latest version of TC6.x. Then you can upgrade to TC7.x without a release key

 

https://www.cisco.com/c/dam/en/us/td/docs/telepresence/endpoint/software/tc6/release_notes/tc_software_release_notes_tc6.pdf

 
- As a rule of thumb, you always want to go from the last release of a major update to the 1st x.1 version of the next major release. Ie. VCS x7.0.3 > x7.2.3 > x8.1.1 > x8.2.2. For TMS 13.1.2, upgrade to 13.2.2, install TMSPE 1.0.

- Once I have a general understanding of the deployment, I will know where we can start to test to troubleshoot and resolve this issue
 

Automated log review

 

Once you have all these logs, you can then go to the following link

 

https://cway.cisco.com/csa/

 

Select “Log Analysis” and upload your logs there. Our automatics scripts will review the logs for you and see if there are any obvious issues with the logs to provide you quick feedback. If the tool does not help you resolve the issue, upload the logs to the case and inform me once done so that I can further review.

 
========================

Optimal bandwidth/ video settings for TC Endpoints


========================
After further review of your configs, I can see that you have your default bandwidth settings set to high. Having the “High Definition” option key will enable you to make 720p30 calls. Having the “Premium Resolution” option key enables you to make 720p60, 1080p30 and 1080p60 calls. The SX80 admin guide below pg.148 applies to all TC based endpoints and states the following:

MINIMUM BANDWIDTH FOR RESOLUTION / FRAME RATE (H.264)
• 720p30 from 768 kbps
• 720p60 from 1152 kbps
• 1080p30 from 1472 kpbs
• 1080p60 from 2560 kbps

http://www.cisco.com/c/dam/en/us/td/docs/telepresence/endpoint/sx-series/tc7/administration-guide/sx80-administrator-guide-tc73.pdf

There should never be a need to have calls connect higher than 2.5Mb. Anything higher than this will over drive the codec and possibly cause more quality issue due to the amount of bandwidth required. I recommend you start testing calls at 1152kbps (Codec > Configuration > System Configuration). I would change all bandwidth parameters to be 1152 from current, test then try higher bandwidth settings until you find an optimal connection per your specific network.

*c xConfiguration Conference 1 MaxReceiveCallRate: 1152
*c xConfiguration Conference 1 MaxTotalReceiveCallRate: 1152
*c xConfiguration Conference 1 MaxTotalTransmitCallRate: 1152
*c xConfiguration Conference 1 MaxTransmitCallRate: 1152
*c xConfiguration Conference 1 DefaultCall Rate: 1152

Also to enable 60FPS, you need to go to (Codec > Configuration > System Configuration > Video) and under the specific source, select “Threshold60fps” to start at the resolution required; ideally 1280_720 to start with a default bandwidth of 1152. For the profile setting, we suggest starting at medium and fine tune/ test different options until you find an optimal connection per your specific network. More information can be found on pg. 116 from the link above.
=====================
 
 
 

Acano / CMS:

 
- Please provide me with the IP address, FQDN and software version of each Acano/ CMS server in this deployment and how they are deployed (Which service is on which node, edge etc)
 
- Please list all devices that were in the call when the issue is present. Please include model, software version and IP addresses along with any alias if any.
 

- Please go to (Acano > Logs > Detailed tracing > SIP traffic tracing) and enable logging for a period that you can replicate the issue

 

- Please run a packet capture on CMS and replicate the issue with the following procedures:

 

Follow these steps:

  1. Use the callbridge MMP command to identify which network interface is being used by the Call Bridge:
    acano> callbridge
    Listening interfaces: a
    Preferred interface: none
    .....
  2. Use the pcap <interface> command to capture packets on the interface
    acano> pcap a
    Packet capture running: press Ctrl-C to stop
  3. When you have finished the test, press Ctrl-C
    Transferring packet capture: please wait ....
    Packet capture available in admin-a-20150225-1253.10.pcap
    Captured 2916 packets, 731505 bytes
  4. Download the file with extension .pcap, using SFTP

Note: the pcap file size is limited to 100MB

 

Generating a call diagnostic file on the server

 

API commands:

1) Locate the "Call ID" in question from all the calls in the current space

https://<IP_address>/api/v1/calls?filter=name_of_space e.g. Tracy

 

2) Trigger call diagnostics logging to capture details for the specific "Call ID" in question

https://<IP_address>/api/v1/calls/<call_id>/diagnostics

 

3) Download the call diagnostic file from the web interface on the server

  1. a) logon to the web interface of the CMS
  2. b) go to Status -> General -> Call diagnostic logs
  3. c) locate the call diagnostic created from step2
  4. d) right-click on the [download] link then save link as and save it to your local drive

 

Provide details on the call itself:

1) space name e.g. space bb230138-19a1-4529-b570-49d2c4a970ef (514718 - Tracy H. Keeping )

2) start and end time

3) when the issue occurred

4) name of the system / DID

5) brief description of the issue

 

 

Acano CMS lower than 2.0.x

- Please SCP (or use WinSCP) to access and download the "log" and "live.json" files from the root directory on the Acano/ CMS server with this issue.

 
Acano CMS higher than 2.0.x
- Please SCP (or use WinSCP) to access and download the "logbundle.tar.gz" file from the root directory on the Acano/ CMS server with this issue. Please rename the file adding the node name. If you are unable to download the "logbundle.tar.gz" file from the root directory, download the "log" and "live.json" files from the root directory on the Acano/ CMS server with this issue. Make sure you bundles these together with the same node name to help us identify which file came from which CMS node.
 
Alternatively: (up to 2.4.x)
 
- Please use the CMS log collector to install on a PC that can gather all logs from the CMS in question. The link is below"
 
 

CMS and Skype for Business

Please see the link below on how to collect logs to further troubleshoot:
 
 

=============
General
=============
What is the call flow from end to end (I.e. Skype for business front end > CMS cluster > CUCM)

What are the call details, i.e. what do Skype and CMS participant dial to join the meeting?

=============
CMS
=============
Please provide logs from all CMS involved. This will give us an overal view of the current deployment

- Please SCP (or use WinSCP) to access and download the "logbundle.tar.gz" file from the root directory on the Acano/ CMS server with this issue. Please rename the file adding the node name. If you are unable to download the "logbundle.tar.gz" file from the root directory, download the "log" and "live.json" files from the root directory on the Acano/ CMS server with this issue. Make sure you bundles these together with the same node name to help us identify which file came from which CMS node.

=============
Skype
=============
What is your Lync domain?

What is your Front End pool name?

What are the FQDN/IP addresses of your Front End servers

What is the FQDN/IP of your AVMCU (Audio Video Multipoint Conferencing Unit)

 

Cisco iOS: (Routers/ Switches)

 
 - Can you provide the “show run” from the switch the device is connected to and inform which switch port the device is connected to?
 
- Can you provide the “show mac address-table” from the switch the device is connected to
 
 
 

Conductor:

 
- What SIP registrar is being used with Conductor (I.e. VCS or CUCM) and can you provide the IP, and software versions?

- Please provide me the system configurations of Conductor (XC2.3 or higher) by going to (Conductor > Maintenance > Diagnostics > Diagnostic logging). Select “Start new log” then click “Stop logging” then “Download log”. The zip file you download will have the system configurations in them.
 
- Please provide me screen shots of each Conference template configuration under (Conductor > Conference Configuration > Conference templates) as this information is not in the diagnostic logs.
 
- Please provide me with a Conductor full system snapshot from (Conductor > Maintenance > Diagnostics > System snapshot: Create full snapshot) The file size may be 200Mb - 700Mb. You can upload the file directly to the case using the following tool: https://cway.cisco.com/csc
 

CTMS



For past events:

- At what time and date did the issue occur?
 
- Please provide the CTMS log files from (CTMS > Troubleshoot > Log Files): “Download All”

- Please list ALL endpoint models and software versions that were in the call at the time of the issue.

- Please provide the logs of each endpoint with the attached guide.

- Was this a scheduled, Static or Ad-hoc meeting and if so, what was the meeting ID?

- Can you reproduce the issue?
 
 

If you can replicate the issue:

- Please go to  (CTMS > Troubleshoot > Log Files) and select “DEBUG” logging for All selections (CCS, Conference Manager, Execution Manager, Media Processor, Switching and TelePresence Call-In) and replicate the issue. Once the issue has been replicated, switch the log level back to “INFO” for each option.

- Please provide a port span of both inbound and outbound traffic of CTMS with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html

- At what time and date did you replicate the issue?

- Please provide the CTSM log files from (CTMS > Troubleshoot > Log Files: “Download All”)

- Please provide the Configuration file from (CTMS > Configure > Configuration Files >Export Files)

- Please provide the software version and IP addresses of all your CTMS nodes

- Please list ALL endpoint models and software versions that were in the call at the time of the issue.

- Please provide the logs of each endpoint with the attached guide.

- Was this a scheduled, Static or Ad-hoc meeting and if so, what was the meeting ID?
 
 
 
 

CUCM:



- Please provide the software version and IP addresses of all your CUCM nodes including all PUBs and SUBs if you have any.

- Please enable CUCM traces on all CUCM nodes the call passes through, replicate the issue, extract the logs of the event (RTMT) and provide them to us with the procedures in the link below.
https://supportforums.cisco.com/document/126666/collecting-cucm-traces-cucm-862-tac-sr  
 
- Please provide a port span of both inbound and outbound traffic of the CUCM with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html
 
- Please provide a packet capture on CUCM with the procedures in the link below:
 
 

Phones

 
- Please provide the phone model, software version, DN, Device Name (Description) and IP address.

- Please provide screen shots of the phone configuration from CUCM (Cisco Unified CM Administration > Device > Phone: Find)
 
- Please log into the Web interface of the IP phone and provide a screen shot of the “Device Information” and "Network Setup” pages
 
- Please provide a port span of both inbound and outbound traffic of the IP Phone with the link below as a reference.
https://supportforums.cisco.com/document/44741/collecting-packet-capture-cisco-ip-phone

License issue:



- On ELM, can you go to (ELM > Administration > Diagnostic Logs > Log Settings) and change both "Enterprise License Manager core services” and "Communication with product instances” to “DEBUG”, click “Save” then try to upload the file/ license again? After it fails, go to (ELM > Administration > Diagnostic Logs > Download Logs), choose the time of the event then click “Generate Log File”. This will generate a link to download the logs under "Download the log file to your computer”. Once you have these, please forward them to us for further review.
 
 
 
 
 

Endpoints

 

Immersive:

CTS/ TX

- Please log into the Web page of the CTS codec and navigate to (Log Files > Log files tab), select the “Capture new log files” option, then click on “Capture New Log Files”. Once completed, select the “Download existing log files” option then click on “Download Existing Log Files”. Please provide the downloaded logs.
 
- Please provide me the CTS logs from the endpoint(s) in question using the attached procedures.
https://supportforums.cisco.com/document/103981/capture-cts-logs-and-upload-them-tac-case  
 
 
 
- Please check and reseat the camera connections between the cameras and codecs as per the link below:
https://supportforums.cisco.com/document/106816/isolating-cts-camera-issue 
 


- Please provide a port span of both inbound and outbound traffic of the CTS Endpoint with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html

 
 

IX5000:

 
- Please “Generate Logs” after you replicate and issue then click on “Download Logs” once they have been generated. (IX5000 > Logs > Generate Logs/ Download Logs)

- Please provide a port span of both inbound and outbound traffic of the IX5000 with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html
 
 Cisco Proximity:
 
- Are you using the latest version of the Cisco Proximity App?
 
- Please provide a screen shot of the (Settings > General > About) screen on your iPad/ iPhone.
 
 
- Please launch the mobile Proximity app and while searching for an endpoint, press the “About” link at the bottom of the screen. Please provide a screen shot of the of the “About” page.

T3/ TCU



- Can you provide me with the "last week" logs from the TCU? (https://IPofTCU) > Download Logs > last week with the attached procedures? (TCU Log collecting.docx)
https://supportforums.cisco.com/sites/default/files/attachments/document/tcu_log_collecting.docx
 
- Can you please provide me a copy of the “startup.bat” file from E:\tandberg?

- Can you please provide me a copy of the “application.properties” file from E:\tandberg\configuration?
 
 
 

DX:

 

Android Code

 
Please provide the DX logs from both of the location options below:

1: On-screen menu > Settings > About > Cisco Collaboration Problem Reporting Tool (PRT). Fill in the appropriate information i.e. date/ time of issue and problem description. If the DX unit has an email configured, you can your own email address as the “Customer support email address” to automatically send the PRT file to yourself to confirm you got it, then forward it to me. If not, you can download them from the web interface of the DX unit.

2: Please log into the Web page of the of the unit and download the logs (DX > Console Logs >Download all logs [DownloadAllLogs.tar]). If you are unable to access the web page, it may be disabled in Call Manager. Please ensure the profile of the unit has “Web Access” enabled for the device profile in CUCM.
 
- From a PC connected to the PC port on the DX unit, please open a browser and go to speedtest.net. Click “Begin Test”. When complete, click the “Share This Result” icon (to the right of “Test Again’). Copy the link to the results and email me the link.
 
- Please provide me a packet capture from the DX endpoint using the "Span to PC port" procedures outlined below:
https://supportforums.cisco.com/document/44741/collecting-packet-capture-cisco-ip-phone

- Please download the config.xml file of the endpoint from CUCM with the following procedures:
https://supportforums.cisco.com/sites/default/files/attachments/document/download_xml_config_file.rtf
 

CE code

 
- Please provide me with the CE codec debug logs after replicating the issue. Go to (Codec > Maintenance > System Logs). Under Extended logging, click the drop-down menu on “Start extended logging” and select “Include a Full Packet capture”. Replicate the issue then click on “Stop extended logging” Once completed, download the “Log Archive” and “extendedlogging.pcap”
 
- Please provide me with the CE codec configuration logs. Go to (Codec > Maintenance > System Logs: Download logs archive). Select "Full Call History" from the drop-down menu.
 
 

MXP, TC, CE:

 
- Can you please log into the codec Web interface, download and provide me the diagnostic logs from the codec in question (Diagnostics > Log Files > Download logs archive)
 
- For MXP and TC based endpoints, please provide me with the xconfig/ xstatus of the endpoints (in text format) you are testing with the procedures as outlined in the link below:
https://supportforums.cisco.com/document/12291871/obtaining-system-configurations-xconfig 
http://www.cisco.com/c/en/us/support/docs/unified-communications/telepresence-video-communication-server-vcs/118675-configure-vcs-00.html 
 

MXP:

 
- Please provide me with the MXP codec debug logs after replicating the issue with the attached procedures (pg.29 TelePresence Endpoint Technical Handbook_Rev.11.pdf)
 

- Please provide a port span of both inbound and outbound traffic of the MXP with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html
 
 

TC/ CE based Endpoints TC7.1 or higher:

 

- Please provide me with the TC/ CE / Room OS codec debug logs and packet capture after replicating the issue. Go to (TC Codec > Diagnostics > Log Files) or (CE Codec > Maintenance > System Logs) or (Room OS > Issues and Diagnostics > System Logs). Under Extended logging, click the drop-down menu on “Start extended logging” and select “Include a Full Packet capture”. Replicate the issue then click on “Stop extended logging” Once completed, select the "Full Call History" option under “Log Archive” to download the debug logs and packet capture.

 
MatthewLimbrick_0-1670251690163.png

 

Please also see the following video:
 

Note: The above logging will run only for 3min. If longer time is needed, select only “Start extended logging” and run a port span on each of the endpoints. The packet captures are crucial to see if there is any network issues, i.e. DNS lookup failures or Registration request timeouts etc. By defualt when selecting “Start extended logging”, the logs should run for 10min. If after 10min the issue is not replicated, log back into each codec and re-enable the logging.

 
Call Identification
 
For Room OS software based endpoints, please navigate to (Issues and Diagnostics > Call Logs) and identify which call start time was used for the test.
For TC/ CE software based endpoints, please navigate to (Maintenance > Call Logs) and identify which call start time was used for the test.
 
 
Config logs
 
- Please download the codec config logs. Go to (TC Codec > Diagnostics > Log Files) or (CE Codec > Maintenance > System Logs). Select "Full Call History" from the left drop-down menu
 
 

- Please provide me with the TC codec debug logs after replicating the issue. Go to (Codec > Diagnostics > Log Files). Under Extended logging, click on “Start extended logging”. Replicate the issue then click on “Stop extended logging” Once completed, select the "Full Call History" option under “Log Archive”

 

- Please provide me with the CE codec debug logs after replicating the issue. Go to (Codec > Maintenance > System Logs). Under Extended logging, click the drop-down menu on “Start extended logging” and select “Include a Full Packet capture”. Replicate the issue then click on “Stop extended logging” Once completed, download the “Log Archive” and “extendedlogging.pcap”
 
- Please provide me with a packet capture on the TC endpoint with the attached procedures. Please select “Include a full packet capture”
 
 
 

TC based Endpoints TC7.1 or lower:

 
- For TC codecs lower than TC7.1, please enable debug logging and replicate the issue with the following procedures:
https://supportforums.cisco.com/sites/default/files/attachments/document/tc_codec_debug_procedures.rtf
 
- For TC codecs lower than TC7.1, please enable and login as “root” on the TC endpoint and run a tcpdump to collect a packet capture as per pg.12 in the link below. Be sure to use the commands at the top of the page to create a directory named “traces”
https://supportforums.cisco.com/sites/default/files/attachments/document/cu_ver_-_tcpdump_vcs_ep_and_port_span_.docx
 
 
- Please locate the system in TMS Navigator and click the "Logs" tab. Please enter "100" for the "Number of records to show" and click "Refresh". Please then copy and paste all the details into a text file. This needs to be done for the Trap Log, History and Ticket log for each endpoint that rebooted.
 
 
 
Integrations:
 
- What do you have directly wired to outputs 1 and 2 (I.e. Amp to speakers including device and model) Please include all devices along the audio path all the way to the speakers.
 
- How do you have outputs 1 and 2 wired (I.e. Balanced or unbalanced) and how are they wired on the input side of the next device?
 
- Do you have a wire diagram?
 
 
 
 

Time conversion in logs:

 
Time: Endpoint DEBUGS are marked in “seconds” since last reboot. When reviewing the logs, you can use the following link and use “Convert seconds to day, hours and minutes”
http://www.epochconverter.com/   

Looking at the entry below, we will find that “19133.70” is “5 hours, 18 minutes and 53 seconds.”
19133.70 H323Packet   value RasMessage ::= registrationRequest :
 
 

Polycom:

 

VSX:

 

- For the Polycom VSX, please provide me the “Profile Center settings with the attached guide. (Polycom HDX VSX Logs and Configs.docx)

https://supportforums.cisco.com/sites/default/files/attachments/document/polycom_hdx_vsx_logs_and_configs.docx

 

- Please replicate the issue and provide the “Error logs” (Diagnostics > System Log > Error Log) with the attached guide. (Polycom HDX VSX Logs and Configs.docx)

https://supportforums.cisco.com/sites/default/files/attachments/document/polycom_hdx_vsx_logs_and_configs.docx


- Please provide a port span of both inbound and outbound traffic of the Polycom with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html

 

 

HDX:

 

Ensure DEBUG logging is enabled. Go to Diagnostics > System Log > System Log Settings and select Log Level to DEBUG and ensure H.323 and SIP Trace is checked. Replicate the issue.

Download and zip the following Polycom logs from (Diagnostics > System Log > Download Logs). Please see the attached guide as reference (Polycom HDX VSX Logs and Configs.docx)

- Download support information package (polycom_info_xxxxxxxxxxxxxx.tgz)

- Download system log… (Rename log.txt to system.log.txt)

- Download error log… (Rename log.txt to error.log.txt)


- Please provide a port span of both inbound and outbound traffic of the Polycom with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html

 
 
 

 
 
 

IPVC - 3527 ISDN to IP Gateway

 
- Please enable DEBUG logging under (Gateway > Settings > Advanced > Advanced Commands). Select command: NotifyLevel and Parameter: 6 as noted in the link below. You will then need to Telnet into the IPVC IP address and collect the output to a file using Putty or a similar Telnet client. Be sure to set the command parameter back to "0" when done testing.
 
,

ISDN Gateway

 
- Please provide me a screenshot of the following pages:
(ISDNGW > Status > General)
(ISDNGW > Status > ISDN)
(ISDNGW > Settings > H.323)
(ISDNGW > Settings > SIP)

- Please provide a backup configuration from the ISDNGW, usually found under?  (ISDNGW > Settings > Upgrade > Back up and restore: Save backup file)
 
Note: In general, the only useful trace options on the event log of the ISDN GW are Connection, dspapi, dialplan, ISDN and H.320 (Errors, warning, Information and trace.) Sometimes it is useful to turn BAS up to detailed trace to see exactly how the ISDN capabilities are being exchanged.
 
 
Replicate and log the issue.
=====================
With an endpoint registered to the same VCS as the ISDNGW, make a call through the ISDNGW that would reproduce the issue. Before doing so, collect the following info:

VCS
======
- Please provide the xconfig/ xstatus of each VCS the call traverses through with the procedures in the following link. Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.
https://supportforums.cisco.com/document/12291871/obtaining-system-configurations-xconfig
 
- Can you please turn on debug logging on each VCS in the call flow then replicate the issue with the attached procedures (Control and Expressway if applicable)? This will be done on each VCS  (Control and Expressway if applicable) if in a cluster. (CU ver - VCS Debug logging Procedures.docx)

Endpoint
======

- Please provide configs (depending on type of EP)

ISDNGW
======

- On the “event capture filter” tab (ISDNGW > Logs), change the following items to “Errors, Warnings and Trace” (Make sure to change them back to Errors, warnings and information when done testing)
    - Connection
    - dspapi
    - H.320
    - ISDN
    
- On the “event capture filter” tab (ISDNGW > Logs), change the following items to “Detailed Trace”
    - BAS

- Please enable H.323/SIP logging on the ISDNGW, click "Clear Logs" if available before, then replicate the issue. Ensure to disable  H.323/SIP logging when done testing. Download as XML and add to the case. ( ISDNGW > Logs > H.323/SIP log > "Enable H323/SIP logging)

- Please clear the Event logs, replicate the issue, then “Download as text” the Event logs . (ISDNGW > Logs > Event log)

- Please "Enable CDR permanent storage" if not done already prior to replicating the issue. You can disable afterwards if needed. Replicate the issue and "Download as XML". (ISDNGW > Logs > CDR Log)

Please let me know what the endpoint dialed to get to the ISDN-GW
 
 
=========
Crashes
=========
- Please connect a serial cable to the unit and capture the output when the unit crashes using any terminal program. Default serial port settings are Baud:38400 8, N, 1, N

- Please provide the diagnostic information after the unit crashes or reboots (ISDN-GW > Status > General: "Download diagnostic information")
 
 
 
 

ISDN Link

 
- Please provide me with the xconfig/ xstatus of the endpoint and ISDN Link you are testing with the procedures as outlined in the link below:
https://supportforums.cisco.com/document/12291871/obtaining-system-configurations-xconfig

- What number are you trying to reach when you dial?

- Who is your carrier and do you have BRI or PRI connections?

- Have these lines worked before or are the new circuits?

- Please ssh into the ISDN Link and enable debug logging then replicate the issue with the following commands. Be sure to log the output to a file with your ssh client. I have included logging procedures for ISDN Link 1.0 and above. If you are running software version 1.0, you are highly recommended to upgrade to newer versions so resolve pairing issues and ease in use with dialing strings.

1 - xconfig
2 - xstatus
3 - systemtools calltrace on
4 - log output on
5 - Start the call and recreate the issue. Disconnect the call and turn off logging
6 - log output off
7 - systemtools calltrace off

Once completed, send me the files for further review
 
 
 

Jabber - CUCM



- As the case notes are not very clear, are you trying to deploy Jabber Video for TelePresence (Movi) for VCS or the Cisco Jabber IM, Presence and Video client for Call manager?

- Please install and launch WireShark on the PC hosting the Jabber clients before the user logs in so that we can capture the login/ provisioning details along with bi-directional call signaling and media
 
- From the mobile device, please open a browser and go to speedtest.net. Download and install the appropriate app version for your device. Once installed, open the application and click “Begin Test”. When complete, click the “Share Result” icon in the top middle of the screen. Email the output to yourself and forward it to me. This will provide the internal IP and NAT’d IP Address of the device to compare with the logs, as well as the available bandwidth for the device connection.

- What versions or Jabber are you testing with including the device? (iOS, Android, PC or MAC)

- Do you have a test account (username/ password) we can use to test externally with our in-house tools? This will help test if all you SRV records are setup correctly.

- For Windows, please go to Start and type “%LOCALAPPDATA%\Cisco\Unified Communications\Jabber” (without quotes). You should see a “CSF” directory. Please zip this directory and send it to me. (or request the PRT file)
 
- Please provide a screen shot of the (Settings > General > About) screen on your iPad/ iPhone.
 
PRT Files:
 
Please replicate the issue then provide the PRT file from the Jabber client. For iOS and Android devices, you need to enable DEBUG logging under (Settings > Problem Reporting > Detailed Logging). Enable this feature and sign out. Replicate the issue, create a problem report (PRT) and email it to yourself to ensure you get it. Once you have received it, forward it to me for further review. Below are the locations on each client to create the Problem Report:
 
Windows: Settings (gear icon) > Help > Report a Problem
MAC: Help > Report a Problem
Mobile Devices: Settings > Problem Reporting > Send Problem Report
 
- If you have a CSF client, we will want to install WireShark on the host PC and start a packet capture before you login and stop after the issue has been replicate. Save the output for further review.
 
 
 

Jabber - VCS (Movi)



- What is the call flow, i.e. which client alias calls which client alias and where is each registered to?

- What version of Windows/ MAC OS are these clients installed on, along with the PC models?
 
- Does the camera freeze issue happen over both wireless and wired network connection?

- What kind of camera are you using for the client?
 
- Are you using the Windows generic drivers, or camera specific drivers supplied by the camera manufacturer?

- Are you able to test your camera with Jabber on another PC?

- Have you attempted to update the drivers for your camera and video card?

- Have tried to uninstall Jabber Video software and re-installing?

- At what bandwidth settings were you making the calls for both in and out?

- Have you tested at various bandwidths higher or lower and get different results? If so, which ones?

- Is there a camera management application running, if so, have you disabled it?


Troubleshooting:
 
- (For PC) Please go to Start and type “%LOCALAPPDATA%\Cisco\JabberVideo\” (without quotes). You should see a “Logs” directory. Please zip this directory and send it to me.

- (For Mac) Please open Terminal (Applications > Utilities) and type “open ~/Library/Logs” (without quotes). You should see a “Jabber Video” directory. Please compress this directory and send it to me.

- Can you please provide me the TMS maintenance log files for further review? (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files" or “Download Diagnostic Files”)

- Please provide the configs of each VCS the call traverses through by going to (VCS > Maintenance > Diagnostics > Diagnostic logging) Click "Start new log”, then “Stop logging” followed by “Download log”. This needs to be done on each Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.

- Please enable DEBUG logging of the Jabber client with the attached procedures and replicate the issue. Then provide me the logs for further review. (Jabber Video log level PC.rtf) (Jabber Video log level MAC.rtf)

- Can you please turn on debug logging on each VCS in the call flow with the attached procedures, then replicate the issue with the attached procedures (Control and Expressway if applicable)? This will be done on each VCS  (Control and Expressway if applicable) if in a cluster. (CU ver - VCS Debug logging Procedures.docx)
 
- Please install and run WireShark on the Jabber client PC before logging into the client. Replicate the issue and save the packet capture to further review.
 
 
 
 

Jabber - Guest

 
VCS
 
- Can you please turn on debug logging on each VCS in the call flow then replicate the issue with the attached procedures (Control and Expressway if applicable)? This will be done on each VCS  (Control and Expressway if applicable) if in a cluster. (Default log levels are sufficient) (CU ver - VCS Debug logging Procedures.docx)
 
 
Jabber Client
 
Please replicate the issue and create a problem report (PRT) emailed to yourself to ensure you get it. Once you have received it, forward it to me for further review. For iOS and Android devices, you need to enable DEBUG logging under (Settings > Problem Reporting > Detailed Logging). Enable this feature and sign out. Replicate the issue then provide the PRT file.
 
Windows: Settings (gear icon) > Help > Report a Problem
MAC: Help > Report a Problem
Mobile Devices: Settings > Problem Reporting > Send Problem Report
 
 
OVA Software

You should receive an OVA file after ordering the Jabber Guest software, however if you have not received any email/link to download, you may want to reach out to your sales team to get assistance on this one. Jabber Guest software can only be retrieved from a special link from our manufacturing site and is not downloadable on cisco.com currently. If you did not receive the correct software with your order, your sales team may be able to find the right contacts for you to get further assistance.

As part of the purchase process, you should have received an ISO file. Typically, the OVA is extracted from the ISO. Unfortunately, because of the licensing restrictions on Jabber Guest, we're not given free access to the OVA's.
 
 
 

MCU

 

Deployment Configs:
(These screen shots will be easier to gather configs from as opposed to looking at the backup file)
 
- Please provide me a screenshot of the following pages on your MCU:
(MCU > Status > General)
(MCU > Settings > H.323)
(MCU > Settings > SIP)

(MCU > Settings > ISDN Ports) - if applicable
(MCU > Settings > Upgrade) - port issues
(MCU > Settings > Media ports) - port issues

- Please provide a backup configuration from the MCU, usually found under.  (MCU > Settings > Upgrade > Back up and restore: Save backup file)
 
 
 
Log Captures:

- Please enable H.323/SIP logging on the MCU, click "Clear Logs" if available before, then replicate the issue. Ensure to disable  H.323/SIP logging afterwards. Download as XML and add to the case. ( MCU > Logs > H.323/SIP log > "Enable H323/SIP logging)

- Please provide a port span of both inbound and outbound traffic of the MCU with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html
<br>
Alternatively you can enable packet captures from the Web interface but this method has been known to drop packets. A port span is always the preferred method

- Please clear the Event logs, replicate the issue, then download the Event logs . (MCU > Logs > Event log)
 
- Please configure the MCU syslog to point to a PC running WireShark as per the guide below and enable the following API Event Capture filters:
-
 
- On the “event capture filter” tab (MCU > Logs: Event capture filter), change the following items to “Errors, Warnings and Trace” (Make sure to change them back to "Errors, Warnings and Information" when done testing)
    - SIP
 

- On (MCU > Logs: Syslog tab), apply the IP address of the PC running Wireshark for your port span so that all syslogs are captured there as well

 

- Please "Enable CDR permanent storage" if not done already prior to replicating the issue. You can disable afterwards if needed. Replicate the issue and "Download as XML". (MCU > Logs > CDR Log)

 
 
RCA:
 
- Was H.323/SIP logging on the MCU enabled during the time of incident and if so, can you provide the output? ( MCU > Logs > H.323/SIP log > "Enable H323/SIP logging)
 
- Can you provide the "Event" logs from this MCU? (MCU/ Logs/ Event log: Download as text)

- Please download the “Diagnostic” and “Conference” information at the bottom of the screen (MCU > Status > General)
 
- Do you happen to have the console logs during the crash? (Serial connection with a monitoring device to record output)

- How many systems were in the conference at the time of the issue, types and software versions?

- What systems dropped from the conference?

- How many Web sessions were open at the time the issue was noticed?

- Were any network issues reported by the network team or any packet loss on the MCU at the time of the issue?

- Was the conference scheduled in TMS?

If yes:

- What was the Conference Name/ Title and ID? (TMS/ Booking/ List Conferences)

- Can you provide the TMS logs from the conference? (TMS/ Booking/ List Conferences: select the conference/ Log tab) Copy & Paste all text to a text doc and send back


Packet Captures:

There are 3 methods to capture packet on MCU products as noted below:

1) Local serial console access to the MCU using the nettap command (Never advised)

2) Port Mirroring (Port Span) - Sending a copy of all traffic of the switch port the MCU is plugged into, to another capturing device/ PC. (Recommended) (CU ver - TCPdump VCS EP and Port span .docx)

3) Take a simple Hub and plug it between the MCU and the Switch. Plug a laptop into the same Hub and run a Wireshark packet capture. As a Hub will send the same traffic to every port, this option can be used instead of setting up port
 
 
 

Meeting Place

 
Cisco Meeting Place > Services > Logs > Configure Logging Levels: Change all to DEBUG and click save. Replicate the issue

Cisco Meeting Place > Services > Logs > System Information Capture: Select time/ date of issue and click on Capture Logs.
 
 
 
 

Microsoft Lync/ Skype for Business

 
Lync/ Skype Clients:
- Please enable debug logging on the Microsoft Lync/ Skype for Business client, then replicate the issue as per the procedures in the link below:
 
What is the call flow from end to end, i.e. Lync 2013 client > on-prem Lync 2013 Front-End servers/ Office 365 > Expressway Lync Gateway > VCS Control > MCU > Endpoint (along with their software versions and IP addresses?
 
 

MRA - Collab Edge



 
General
 

Please go the following link and test your deployment config to quickly identify any issues that may be causing a problem. Please select "CollabEdge Validator" at the following link and provide your test credentials to further test your MRA deployment:


https://cway.cisco.com/tools/CollaborationSolutionsAnalyzer/

 

If you are still having issues afterwards,

 

- Please provide the credentials for a test CSF account (username/ password i.e. ciscotac@domain.com) we can use to test externally with our in-house tools. This will greatly help test if all you SRV records and other configurations are setup correctly.
 

- Please provide the configs of each Expressway by going to (Expwy > Maintenance > Diagnostics > Diagnostic logging) Click "Start new log”, then “Stop logging” followed by “Download log”. This needs to be done on each Exp-C and Exp-E in a cluster if applicable.


- Please provide the software version and IP addresses of all your CUCM and IM&P nodes including all PUBs and SUBs if you have any.

- What versions of Jabber are you testing with including the device?

- Can the Jabber client register to CUCM and IM&P services while internal to the network using “Automatic“ settings?
 
- Please provide me the username used when testing and at what time the test was initiated.

- Please provide me with the date/ time the issue is observed and with which user.
 
 
VCS/ Exp

- Please change the following logging options to "DEBUG" on all VCS/ Expressway nodes: (VCS > Maintenance > Diagnostics > Advanced > Support Log Configuration)
    - developer.xcp
    - developer.trafficserver
    - developer.edgeconfigprovisioning
 
 
- Please enable debug logging on both Core and Edge VCS units and replicate the issue (VCS > Maintenance > Diagnostics > Diagnostic logging: Start new log) Ensure “Take a tcpdump while logging” is checked.
(CU ver - TCPdump VCS EP and Port span .docx) (CU ver - VCS Debug logging Procedures.docx)
 
 
Jabber
 
Please replicate the issue and create a problem report (PRT) emailed to yourself to ensure you get it. Once you have received it, forward it to me for further review. For iOS and Android devices, you need to enable DEBUG logging under (Settings > Problem Reporting > Detailed Logging). Enable this feature and sign out. Replicate the issue then provide the PRT file.
 
Windows: Settings (gear icon) > Help > Report a Problem
MAC: Help > Report a Problem
Mobile Devices: Settings > Problem Reporting > Send Problem Report
 
 
 
CUCM
 
- Please enable CUCM traces, replicate the issue, extract the logs of the event and provide them to us with the procedures in the link below.
https://supportforums.cisco.com/document/126666/collecting-cucm-traces-cucm-862-tac-sr
 
- Please provide the software version and IP addresses of all your CUCM and IM&P nodes including all PUBs and SUBs if you have any.
 
IM&P
 
Enable the following debugs:
 
/Cisco Config Manager
/Cisco Connection Manager
/Cisco XCP Router
/Cisco XCP Authentication Service
 
Native packet capture:
utils network capture file <filename>

Use RTMT capture logs > packet capture logs (to download the logs)

 

 
Phones:
 
PRT Collection

To Collect A PRT(Problem Report Tool):
Please make sure the "Web Access" is enabled on the phone
1. Press the "Application" button on the phone.
2. Go to "Phone Information"
3. Press soft key "Report problem"
4. Select "Other" in the "Problem Description"
5. Press the submit soft key on the phone.

Download the PRT from the location indicated in the last two lines you'll need to browse to the Phones Web Interface and select "Console Logs" and scroll to the bottom of the page and you'll see the "Problem Report Tool Logs" and download and send the prt-xxxxxxxxxx.tar.gz to us.

As per the link below, you must be running above VCS x8.7, CUCM 10.6(2) and phone load 11-0-1
 


Note:
This feature is implemented in this version for the purpose of previewing with dependent systems. It is not currently supported and should not be relied upon in your production environment. Full support for this feature is planned for future releases of the VCS software and the interdependent systems below.

The CA list can be found in the Appendix of the latest 8800 Admin Guide (Cisco IP Phone 8811, 8841, 8851, 8851NR, and 8861 Administration Guide for Cisco Unified Communications Manager 10.5)
http://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cuipph/8811_8841_8851_8861/10_5/english/adminguide/P881_BK_C0632068_00_cisco-ip-phone-8811-8841/P881_BK_C0632068_00_cisco-ip-phone-8811-8841_appendix_010001.html 
 
 - Please restart the XCP router services on your IM&P servers. I would do one at a time to ensure there is a backup online at all time. (Cisco IM&P Serviceability > Tools > Control Center – Network Services: IM&P Services > Cisco XCP Router)
 
- Make sure you can login internally after restarting the services on both IM&P servers. Wait about 5min, then go to each (Exp-C > Configuration > Unified Communications > IM and Presence servers). Click on the link under IM&P servers and click “Save” to refresh the connections between Exp-C and the IM&P servers. Do the same for the CUCM links as well (Exp-C > Configuration > Unified Communications > Unified CM Servers)
 
 
 
SSO Logins
 
- What Identity Provide (IdP) server are you using including software version and IP address/ FQDN?

- Is this issue happening for specific users or everyone?
 
- Is the IdP server accessible to the Jabber client from outside your network (i.e. via HTTP proxy or other means)?
 
- What username is being tested in the recreate?
 
Troubleshooting
 
==============
Jabber client
==============
- Install Fiddler and enable "Decrypt HTTPS traffic" under (Fiddler > Tools > Telerik Fiddler Options > HTTPS tab) before attempting to login to the Jabber client. Once issue is replicated, go to (File > Save > All Sessions... > Type: Sessions Archive) Fiddler can be downloaded from http://www.telerik.com/fiddler.
 
- Please replicate the issue and create a problem report (PRT) emailed to yourself to ensure you get it. Once you have received it, forward it to me for further review. For iOS and Android devices, you need to enable DEBUG logging under (Settings > Problem Reporting > Detailed Logging). Enable this feature and sign out. Replicate the issue then provide the PRT file.
 
Windows: Settings (gear icon) > Help > Report a Problem
MAC: Help > Report a Problem
Mobile Devices: Settings > Problem Reporting > Send Problem Report
 
==============
VCS/ Expressway
==============
VCSc/ Expressway-C debugs to enable in addition to normal MRA debug:
- network.http.sso
- network.http.sso.server
 
(on VCSc & VCSe)
    - developer.xcp
    - developer.trafficserver
    - developer.edgeconfigprovisioning
 
==============
CUCM
==============
- Enable SSO debugs on all CUCM nodes independently via CLI only with the following commands:
 
set samltrace level debug
 
To disable debugs:
 
set samltrace level info
 
From RDMT, download the following logs after you replicate the issue:
- Cisco SSO
- Tomcat
- Tomcat Security
 
  

MSE 8000 Codian Supervisor Blade


- Please download the "Diagnostic information" from the MSE (MSE > Status > Download diagnostic information)

- Please provide me the event logs from the MSE (MSE > Logs > Event log tab: Download as text)

- Please download the Alarm event logs from the MSE (MSE > Alarms > Alarms logs tab: Download as text)

- Please provide a backup configuration from the MSE, usually found under. (MSE > Settings > Upgrade > Back up and restore: Save backup file)

- Please provide me a full screen shot of your Port License page (MSE > Port Licenses)

- Please provide a screen shot of (MSE > Hardware > Blades tab)

 

- Please provide a screen shot of (MSE > Hardware > Power supplies)

 

Serial Gateway

 

 

Deployment Configs:
(These screen shots will be easier to gather configs from as opposed to looking at the backup file)
 
- Please provide me a screenshot of the following pages on your MCU:
(Serial-GW > Status > General)
(Serial-GW > Status > Serial)
(Serial-GW > Settings > H.323)
(Serial-GW > Settings > Serial ports)

 

- Please provide a backup configuration from the MCU, usually found under.  (MCU > Settings > Upgrade > Back up and restore: Save backup file)

 
Debug Logging
 
- On the “event capture filter” tab (Serial-GW > Logs), change the following items to “Errors, Warnings and Trace” (Make sure to change them back to Errors, warnings and information when done testing)
    - Connection
    - dspapi
    - H.320
    - SERIAL_CALL_CONTROL
    
- On the “event capture filter” tab (Serial-GW > Logs), change the following items to “Detailed Trace”
    - BAS
 
 

TCS (TelePresence Content Server)

 
What platform generation are you running?
(Gen1 starts 49A1)
(Gen2 starts 49A2)

Which recording is having this issue (recording name)?

Can you provide me with the original recording file before it was transcoded? You can match this filename by looking at the file path in the "Share" tab of the recording as found in the following directory.
(TCS > E:\data\rtpdumps)

Can you provide me with the transcoded recording file?
(TCS > E:\data\media)

Can you compress/ zip the entire TCS server logs directory and send them to me?
(E:\logs)

Can you provide me the Windows and TCS event logs from Window Event viewer? Please choose to "Save log file as" (*.evt). "Zip" them up together before sending as they are usually large in size. Event viewer can be located (TCS > Start > Administrative Tools > Event Viewer)

Are you running internal or external storage?

Are you run running TCS in a cluster and if so, how many?
 
 
 

TMS

 
General:
 
- Can you please provide me the TMS maintenance log files for further review? (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files" or “Download Diagnostic Files”)
 
- Can you RDP into the TMS server and zip the “TMSDEBUG” directory, then provide a copy for further review. (C:\Program Files (x86)\TANDBERG\TMS\data\Logs\TMSDebug)
 

- Can you RDP into the TMS server and provide me the Windows event logs from Window Event viewer (Application, Security, Setup and System)? Please choose to "Save log file as" (*.evt). "Zip" them up together before sending as they are usually large in size. Event viewer can be located (TMS > Start > Administrative Tools > Event Viewer)

 

- Can you please remote desktop into the TMS server and provide me with a screen shot of (Cisco TMS Tools > Security Settings > Advanced Security Settings) page?
 
- Please enable the TMS log levels to DEBUG with the attached procedures and replicate the issue. Once replicated, please provide the TMS maintenance log files for further review and set the log levels back to Warning. (Debug TMS Logs.rtf)
 
(Java versions and environment variables)
- Please RDP into the Windows Server hosting TMS and take a screen shot of following items:
    * Go to Start (run) and type “Sysdm.cpl“ and press enter. Go to Advanced Tab > Environment Variables. In the “System variables” section, search for the “JAVA_HOME” Variable. Please provide the Value of this config.
    * Go to Start (run) and type “appwiz.cpl" and press enter. Provide a screen shot showing the Java Application and details. If "(64-bit)" is next to the name, then the installed version of Java is 64bit. If nothing is next to the name, then it is 32bit.
   
- Can you provide a screenshot of your "Computer Properties" page? (Start, right click on Computer and select Properties)

- Can you please provide me with the SQL error logs as indicated in the following link:
 
 

- Can you please provide me a copy of your TMS SQL databases. You can upload the files directly to the case using the following tool: https://cway.cisco.com/csc  Procedure options can be found here:

http://www.cisco.com/c/en/us/support/docs/conferencing/telepresence-management-suite-tms-version-14/117896-technote-tms-00.html#anc6

 
- Are you provisioning Jabber/ Movi clients, and if so, are you using Legacy Agent or Provisioning Extension mode?
 
- What version of Windows Server are you running your TMS installation on, including SP's installed? Can you provide a screenshot of your "Computer Properties" page that displays this information?
 
- Can you provide me the Windows event logs from Window Event viewer? Please choose to "Save log file as" (*.evt). "Zip" them up together before sending as they are usually large in size. Event viewer can be located (TCS > Start > Administrative Tools > Event Viewer)
 
- Are you using 1 NIC on TMS?

- Is your SQL database on an external server or local on TMS, and what version of SQL server is running?

Can you provide the output from the the procedures at the following link? I want to see if you are running SQL 2005 Express or not.
http://technet.microsoft.com/en-us/library/ms165662%28v=sql.90%29.aspx
 
- What are you having trouble logging into? (The remote desktop TMS Windows environment, the TMS Web application or Jabber/ Movi clients)
 
- Can you please provide me a screen shot of your full TMS General Settings page to include the options keys and serial number and version in the bottom right corner of the screen? (TMS > Administrative  Tools > General Settings)
 
 
TMS-Agent

- Are you able to run TMS Agent Diagnostics on the TMS (5 checks) and on your VCS (20 checks) control unit(s)? Do you have any errors and if so, what are they? (TMS > Administrative Tools > TMS Agent Diagnostics)
 

TMSPE
 
- Can you go to the TMS Provisioning Extension Diagnostics, click “Run Health Check” and provide a screen shot? Is everything green and no Alarms? (TMS > Administrative Tools > Diagnostics > Provisioning Extension Diagnostics) *For newer versions

- Can you go to (TMS > Systems > Navigator), find your VCS control > Provisioning Tab and do all service have a green check mark? (Try to click on Force Refresh to double-check)

- Can you web into the VCS control and ensure the TMSPE services are also green there as well? (VCSc > System > TMS Provisioning Extension Services)
 

Conferences Logs

- Can you please locate the system in TMS Navigator, click the "Logs" tab, and copy/ paste the logs from each category (Trap, History, Call Log, Ticket Log and Audit Log) into a text file and send them back to me? Please mark which logs are which. You may have to choose to show a high number of records (I.e. 1000) so that you can copy more than what is shown by default.

- Can you please provide me a full screen shot of the conference page (TMS/ Booking/ List Conferences: select the conference/Participants Tab) that shows the “Basic Settings” expanded and the “Advanced Settings” expanded.

- Can you provide the TMS logs from the conference? (TMS/ Booking/ List Conferences: select the conference/ Log tab) Copy & Paste all text to a text doc and send back. Alternately, you can go to (TMS > Monitoring > Conference Control Center) and locate the conference. Double-click the conference and go to the “Event Log” tab. You click on “Open As Text”. Please send this file as well.
 
 
Endpoint / Contract Logs

- Please provide a System Overview report (TMS > Systems > System Overview) with the effected systems (or select all) by selecting the systems in the left window. In the right window, select “Select All” then click “View” in the bottom left of the screen. Now click on “Export to Excel” and send me the output Excel file.
 
- Please click on the endpoint in TMS navigator. On the "Settings", click on "Force Refresh",  take a screen shot of the “Summary” tab showing the contract status.
 
 
TMS Maintenance Logs
 
- Can you please provide me the TMS maintenance log files for further review? (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files" or “Download Diagnostic Files”)
 
If you need logs further back than what was obtained from the TMS web interface, you can RDP into the TMS server to pull all logs.
 
C:\Program Files (x86)\TANDBERG\TMS\data\Logs\TMSDebug
(For all TMS debug logs)

C:\Program Files (x86)\TANDBERG\TMS\TMSProvisioningExtension\app\logs
(For all TMSPE logs)
 
Note: Looking at these logs, you can obtain the version of TMS, TMSPE, Java if SQL is internal or external along with different configurations
 
- Log file name i.e. "02_Mar_2015_12_42_45_TMSLogFiles_14.5.0.zip": Shows version of TMS
 
- provisioningextension-wrapper.log: Shows current Java version. Search for last entry using keyword "java version"
 
i.e.: INFO   | jvm 1    | 2015/02/24 09:59:02 | java version:1.7.0_51 (This is Java 7 update 51)
 
 
- tmsprovisioningextension.log: Shows current Java version. Search for last entry using keyword "[na:" (Not as reliable as wrapper.log)
 
i.e.: at java.lang.Thread.run(Unknown Source) [na:1.7.0_51] (This is Java 7 update 51)
 
- tmsprovisioningextensioninstall.log: Shows current version of TMSPE and SQL details. Scroll to bottom of file. Last line will show:
 
MSI (c) (40:28) [22:03:08:226]: Windows Installer installed the product. Product Name: Cisco TelePresence Management Suite Provisioning Extension. Product Version: 1.3.0.8
 
Property(C): DefaultSQLInstance = MSSQLSERVER
Property(C): SqlVersionExpectedMin = 10.0.0.0
Property(C): defSQLINSTANCENAME = SQLEXPRESS
Property(C): defSQLSERVERNAME = local
Property(C): defSQLUSERNAME = **********
Property(C): TMSMinimumVersion = 14.2.0
 
- tmsinfo.xml: Show the following details: and more:
 
  <TMSVersion>14.5.0</TMSVersion>
  <Serial>80A03377</Serial>
  <Name>TMSServerIPv4Address</Name>
  <Value xsi:type="xsd:string">134.133.32.107</Value>
  <Name>HTTPSOnlyMode</Name>
  <Value xsi:type="xsd:boolean">false</Value>
  <Name>UpdateCheckURL</Name>
  <Value xsi:type="xsd:string">https://tms-update.cisco.com/webservices/product/v1.0/Software/SoftwareService_ES</Value>
 
 
 
DEBUG Logging

By default the log files on TMS are not very verbose.  You will need to enable DEBUG logging.  To do this:

1. Log in to the TMS RDP session
2. Open Windows Explorer and navigate to "C:\Program Files
(x86)\TANDBERG\TMS\Services" ("C:\Program Files\TANDBERG\TMS\Services" if
using a 32-bit version of Windows)
3. Open each .exe.config file in Notepad and look for the following tag:
    <root>
      <level value="WARN"/>
4. Change this line in each file where it appears so that it now reads:

    <root>
      <level value="DEBUG"/>
5. Save each respective file

- TMSDatabaseScannerService.exe
- TMSLiveService.exe
- TMSPLCMDirectoryService.exe
- TMSSchedulerService.exe
- TMSServerDiagnosticsService.exe
- TMSSNMPService.exe

6. Reproduce the issue then be sure to place log level back to “WARN”

7. Please provide me the TMS maintenance log files for further review?  (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files" or “Download Diagnostic File
 
 
TMS IIS Logs
 

- Please provide the IIS logs from the TMS server by going to the following 2 locations:

    “C:\inetpub\”. - Zip up the “logs” directory

    “C:\Windows\System32\LogFiles\”. -  Zip the “HTTPERR” directory

- Please provide the “applicationHost.config” file from the TMS server “C:\Windows\System32\inetsrv\config\”
 
 
 

TMSXE

- TMS & Microsoft Exchange
 
=============
TMS:
=============
- Is this a redundant TMS deployment and if so, what are the IP addresses and FQDN of each node and VIP of your NLB

 

- Can you provide specific instances where the issue is occurring? (i.e. TMS conference ID, endpoint names, time/ date)
 
- Check if the room in TMS navigator has "Allow Remote Booking" and that they have not run out of licenses for bookings (TMS Navigator > Endpoint > Settings tab >TMS Scheduling Pane: Allow Remote Bookings)
 
- Can you find the conference in TMS under (TMS/ Booking/ List Conferences) then once found, navigate to the Logs tab and copy/ paste all the logs associated with the conference into a txt file. Please be sure to add the conference ID which can be seen at the top left "View Conference X"
 
- Please provide a screen shot(s) of both your TMS Licenses and Option keys showing the numbers available and the actual option keys themselves (TMS > Administrative Tools > Configuration > General Settings: Licensees and Option Keys)
 
- Can you please provide me the TMS maintenance log files for further review? (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files").
 
- Please provide a full screen shot of the following (TMS > Administrative Tools > TMS Server Maintenance: TMS Services Status)
 
 
=============
TMSXE:
=============
- What version of TMSXE are you running and where is it installed? (Go to Start > Run and type “appwiz.cpl” and hit enter to bring up the “Programs and Features” window. You should be able to see the version to the far right of the “Cisco TMSXE” application. )
 
(This can be found in the "BuildInfo.txt" file under E:\Program Files\ BuildInfo.txt, or the drive TMSXE is installed on)
 
- Is this a redundant deployment of TMSXE and if so, what are the IP addresses and FQDN of each node?

- Please navigate to "C:\ProgramData\Cisco" and then right-click the 'TMSXE' folder and send to zipped folder, then upload the zip file to the case
 
- Please provide me the TMSXE log files on the server where TMSXE is installed. Navigate to the following location, zip the entire “TMSXE” directory and provide for further review:
 
- C:\ProgramData\Cisco\TMSXE\Storage
- C:\ProgramData\Cisco\TMSXE\Config
- C:\ProgramData\Cisco\TMSXE\Logs
 
=============
Exchange:
=============

 

- What version of MS Exchange and SP are you running and on what OS/ SP?

- What version of MS Outlook are the clients using?
 
- Check that the room booked in Outlook is also booked in TMS

- TMS-Service account needs to have full mailbox access to all of the resource mailboxes. For the rooms that have the issue, please provide the output to the following commands from your Exchange admin:
 
Get-CalendarProcessing -Identity "room" | Format-List
Get-mailbox username | fl CalendarRepairDisabled
 
 
==============
Turning on debug logging
==============
The default log level is informational. To change the log level for debugging:
1. Open Notepad as an administrator.
2. Locate the Cisco TMSXE Config folder on your computer, by default located in C:\ProgramData\Cisco\TMSXE\Config. Note that the ProgramData Windows folder is hidden by default.
3. Change the drop-down to look for All Files.
4. Open the file Log4net.config.
5. In the line that says <level value ="INFO" />, replace "INFO" with "DEBUG".
6. Save and close the file.
This setting significantly increases the size of the log. We strongly recommend reverting the log level back to "INFO" after debugging. The steps to revert are the same as above.
 
 
 
 

TMSXN - TMS & Domino Lotus Notes

 
TMS
 
- Can you please provide me the TMS maintenance log files for further review? (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files”)

- Please provide a screen shot(s) of both your TMS Licenses and Option keys showing the numbers available and the actual option keys themselves (TMS > Administrative Tools > Configuration > General Settings: Licensees and Option Keys)
 
 
Domino:

What version of Domino and Fix Pack are you running? (Take a screen shot - On the Domino Console for the server TMSXN is installed on, can you run the following command: show server)

What version of TMSXN did you install on your Domino server? (Take a screen shot - Navigate to the Domino Directory, right-click TMSEventCatcher.dll and select Properties. Click on the Details tab to see the version)

Please provide a screenshot of the original installation file.

Is you Domino server running in a stand alone or clustered configuration? (Take a screen shot - On the Domino Console for the server TMSXN is installed on, can you run the following command: show cluster)

Can you please zip and provide the entire "IBM_TECHNICAL_SUPPORT" directory? (Domino directory/ data)

Please provide me a copy of your "java.policy" file (Domino\jvm\lib\security)

Can you verify that the TMS Admin view has "Automatic update Free/Busy" set to "Yes"? (Open the Video Conference Resources database and click on "TMS Admin" on the left-hand menu
 
 
TMSXN Debug logs:

Can you change the TMSXN Debug mode to "3" and try to double-book a meeting? (Open the Video Conference Resources database and click on "TMS Admin" on the left-hand menu. Change the Debug mode to "3")

Once you have replicated the issue with Debug mode level "3" set, can you zip and provide the entire "IBM_TECHNICAL_SUPPORT" directory? (Domino directory/ data)

Be sure to change the Debug mode back to 0

Please provide me a copy of your "java.policy" (Domino\jvm\lib\security)
 
 
 
 

TPS (TelePresence Server)



- Can you provide a screenshot of the following pages of your TelePresence Server:
(TPS > Status > Status)
(TPS > Configuration > H.323 settings)
(TPS > Configuration > SIP settings)

- Can you "Enable protocols logging" on the TelePresence server and replicate the issue? Ensure all options are checked and to "Clear Log" before the re-create. Please disable after your re-create. (TPS > Logs > Protocols log)
 
- As of TPS 4.1(1.79), you now have the ability to run extended Protocol Logging. Please see the guide below on how to set this up, then replicate the issue and provide the output.
https://supportforums.cisco.com/sites/default/files/attachments/document/how_to_capture_extended_protocol_logging_for_tps_2.docx

- Can you provide me with the Diagnostic and System logs after the recreate? (TPS > Status > Status)     (This has all logs zipped)

- Please provide a port span of both inbound and outbound traffic of the TPS with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html
 
Alternatively you can enable packet captures from the Web interface but this method has been known to drop packets. A port span is always the preferred method
-
 
 

- On the “event capture filter” tab (TPS > Logs: Event capture filter), change the following items to “Errors, Warnings and Trace” (Make sure to change them back to "Errors, Warnings and Information" when done testing)
    - SIP

 

Note: By default, the Event logs only capture a rotating log of the last 2000 entries. More than that will overwrite themselves. To capture more than 2000 events, do the following:

 

- Please setup a PC running Wireshark that is on the same network as the TPS. Go to (TPS > Logs > Syslog) and apply the IP of the PC running Wireshark into the “Host address 1” field, then click “Update syslog settings”. (Leave Facility value at “1 - user level”). Next go to Logs > Event Capture Filter and change “SIP” to “Errors, warnings, information and trace” (EWIT), then click “Update Settings” at the bottom of the screen. Start running the WireShark capture and replicate the issue. In Wireshark, you can use display filter “syslog” to view the live Syslog messages from TPS. Once the issue has been replicated, stop the Wireshark capture and save the file for further review. Be sure to change “SIP” back to “Errors, warnings and information”


- Please change the “API” capture filter to “Errors, warnings, information and trace” for the following modules, then click “Update Settings” under (TPS > Logs > Event capture filter)


- Please provide a port span of both inbound and outbound traffic of the TPS with the attached guide as a reference (CU ver - TCPdump VCS EP and Port span .docx)
 
- What systems dropped from the conference?
 
- How many Web sessions were open at the time the issue was noticed?

- Were any network issues reported by the network team or any packet loss on the MCU at the time of the issue?

- Can you provide a backup configuration from the MCU, usually found under?  (MCU/ Settings/ Upgrade/ Back up and restore: Save backup file)

- Was the conference scheduled in TMS?

If yes:

- What was the Conference Name/ Title and ID? (TMS/ Booking/ List Conferences)

- Can you provide the TMS logs from the conference? (TMS/ Booking/ List Conferences: select the conference/ Log tab) Copy & Paste all text to a text doc and send back
 
 
 
 

VCS



- Please provide the configs of each VCS the call traverses through by going to (VCS > Maintenance > Diagnostics > Diagnostic logging) Click "Start new log”, then “Stop logging” followed by “Download log”. This needs to be done on each Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.

- Please provide the xconfig/ xstatus of each VCS the call traverses through with the procedures in the following link. Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.
https://supportforums.cisco.com/document/12291871/obtaining-system-configurations-xconfig
 
- Please enable debugs of each VCS the call traverses through by going to (VCS > Maintenance > Diagnostics > Diagnostic logging) Click "Start new log”, replicate the issue, then “Stop logging” followed by “Download log”. This needs to be done on each Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster. Ensure “Take a tcpdump while logging” is checked.
 
(Use find / -type f -name "*.pcap" to seach the VCS via root for any .pcap files)
- Please go to (VCS > System > Administration) and change “Session time out (minutes)” to “65535”. If you don’t, the VCS may close your session early and stop logging. Be sure to change back to what it was before after testing
-
- Please login as “root” on both Exp-C and Exp-E and run a tcpdump to collect a packet capture as per pg.4 in the link below. Be sure to use the commands at the top of the page to create a directory named “traces” then execute the yellow highlighted command “For TCPdump without media”.
https://supportforums.cisco.com/sites/default/files/attachments/document/cu_ver_-_tcpdump_vcs_ep_and_port_span_.docx
-
- Can you please turn on debug logging on each VCS in the call flow then replicate the issue with the attached procedures (Control and Expressway if applicable)? This will be done on each VCS  (Control and Expressway if applicable) if in a cluster. (CU ver - VCS Debug logging Procedures.docx)

- As you have deployed VCS x8.2 or higher, please select “Take tcpdump while logging” when running the debugs. If you have VCS x8.2 or below, please perform a TCPdump packet capture with the attached procedures. (CU ver - TCPdump VCS EP and Port span .docx)
 
- Please provide me with a VCS full system snapshot from (VCS > Maintenance > Diagnostics > System snapshot: Create full snapshot) The file size may be 200Mb - 700Mb. You can upload the file directly to the case using the following tool: https://cway.cisco.com/csc

Have you tried other video and audio codecs to see if you have better results (i.e. H263 vs H264)?

Can you provide all the endpoint and infrastructure devices that were in the call (including VCS any MCU at the time the issue was noticed?

Can you please provide the call flow of the systems when the issue is observed?

Do you have a single VCS control, expressway, cluster or a combination of all?

If you are having issues with people outside of your network accessing/ registering into your VCS starter pack, you will need to verify you have all ports opened as necessary. You can verify what ports you need to have opened (according to your specific deployment) by going to (VCS > Maintenance > Tools > Port usage > VCS inbound/ outbound ports). This will show which ports should be opened on your firewall.
 
 
 
 

VMware - UCS/ BE6K

 
- Please provide a VM performance report from the VM in question. With vSphere, connect to the VM host server, expand the list of VMs on the server and click to highlight the specific VM. Go to (File > Report > Performance) and make sure all the “Metric” check boxes are marked. Select a path to download the report then click OK. This will create an Excel file to send us for further review.
 
 

- Please provide me with the full model number and specs of the UCS server (I.e. BE6000H UCS C220 M4 TRC#2)

 

- In vSphere, can you please provide me a screen shot of the top “Host” server “Summary" Tab showing the "Processor Type" and "Image Profile" details in vSphere?

 

- Can you please ensure that you have setup CIMC access as per the link below, then download the “Technical Support Data” as per the attached guide. (CIMC Log collecting BE6K)
http://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-c200-m1-high-density-rack-mount-server/111455-setup-cimc-c-series.html

 

- Please provide me the logs from the specific VM in question with the procedures in the link below:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=653

 

- Please provide the 'tech support' files from both the UCS B-series and UCS C-series. 

https://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-manager/115023-visg-tsfiles-00.html

 



WXeTP (Webex Enabled TelePresence)




General:

At what point in the process does the call setup/ booking fail?

Webex:

What is your Webex site address (i.e. domain.webex com)?

VCS:

- Please provide the configs of each VCS the call traverses through by going to (VCS > Maintenance > Diagnostics > Diagnostic logging) Click "Start new log”, then “Stop logging” followed by “Download log”. This needs to be done on each Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.

TMS:
 
- Can you please provide me the TMS maintenance log files for further review? (TMS > Administrative Tools > TMS Server Maintenance: "Download Log Files" or “Download Diagnostic Files”)


MCU:

- Please provide me a screenshot of the following pages:
(MCU > Status > General)
(MCU > Settings > H.323)
(MCU > Settings > SIP)

- Can you provide a backup configuration from the MCU, usually found under?  (MCU > Settings > Upgrade > Back up and restore: Save backup file)

- Please provide a port span of both inbound and outbound traffic of the MCU with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html

 

XMPP Federation (MRA)



Please provide details of the issue including but not limited to:

- Username of both far end and near clients (i.e. username@domain)

- What messages were sent/ received from each client

- Please provide the credentials for a test CSF account (username/ password i.e. ciscotac@domain.com) we can use to test externally with our in-house tools. This will greatly help test if all you SRV records and other configurations are setup correctly.

- Please provide the configs of each VCS the call traverses through by going to (VCS > Maintenance > Diagnostics > Diagnostic logging) Click "Start new log”, then “Stop logging” followed by “Download log”. This needs to be done on each Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.
 
On your Exp-E, please provide a screen shot of the following pages:
- (Exp-E > Configuration > Unified Communications > Configuration)
- (Exp-E > Configuration > Unified Communications > static routes)
- (Exp-E > Configuration > Unified Communications > Federated domains allow/ deny list) if configured.
 
On your IM&P, please provide me a screen shot of the following pages:
- (IM&P > Presence > Settings) - Ensure "Enable ad-hoc presence subscriptions” is enabled
- (IM&P > Presence > Inter Domain Federation > XMPP Federation > Settings) - Ensure "XMPP Federation Node Status" is off
 
- Please provide the software version and IP addresses of all your CUCM and IM&P nodes including all PUBs and SUBs if you have any.

- What versions of Jabber are you testing with including the device?

- What domains have you tried to Federate with?

- Can you try to federate to my Cisco Jabber account “<CiscoID>@cisco.com”?
 
 - Please restart the XCP router services on your IM&P servers. I would do one at a time to ensure there is a backup online at all time. (Cisco IM&P Serviceability > Tools > Control Center – Network Services: IM&P Services > Cisco XCP Router)
 
- Make sure you can login internally after restarting the services on both IM&P servers. Wait about 5min, then go to each (Exp-C > Configuration > Unified Communications > IM and Presence servers). Click on the link under IM&P servers and click “Save” to refresh the connections between Exp-C and the IM&P servers. Do the same for the CUCM links as well (Exp-C > Configuration > Unified Communications > Unified CM Servers)
 
 
DEBUG Logging
 
VCS:
 
- Please change the following logging options to DEBUG (VCS > Maintenance > Diagnostics > Advanced > Support Log Configuration) each VCS the call traverses through with the procedures in the following link. Control and Expressway (and/or Expressway-Core and Edge) if applicable as well as each VCS if in a cluster.
 
 developer.xcp.federation

- Please enable debug logging on all VCSc/Exp-c & VCSe/Exp-E units as well as each VCS if in a cluster and Ensure “Take a tcpdump while logging” is checked. Try to federate with another domain. Please let me know which client (username, DN and device profile) was used and who they tried to Federate with. 
 
CUPS/ IM&P:
 
1. Log on to the Cisco Unified IM and Presence Serviceability user interface. Choose Trace > Configuration.
2. From the Server drop-down list, chose the IM and Presence Service server, and click Go.
3. From the Service Group list box, choose IM and Presence Services, and click Go.
4. Perform one of the following steps:
a. From the Service drop-down list, select “Cisco XCP Config Manager (Active)” and click Go. Ensure “Apply to All Nodes”, “Trace On” and “Enable All Trace” are all selected with “Debug” level selected from the “Debug Trace Level” dropdown.
b. From the Service drop-down list, select “Cisco XCP Router (Active)” and click Go. Ensure “Apply to All Nodes”, “Trace On” and “Enable All Trace” are all selected with “Debug” level selected from the “Debug Trace Level” dropdown.

XMPP
c. For SIP federation, choose the Cisco XCP SIP Federation Connection Manager service from the
Service drop-down list, and click Go.
d. For XMPP federation, choose the Cisco XCP XMPP Federation Connection Manager service from
the Service drop-down list, and click Go. (if XMPP federation is disabled, the logs will be empty)
 
 
- Please provide a port span of both inbound and outbound traffic of the IM&P server with the link below as a reference.
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst2940/software/release/12-1_19_ea1/configuration/guide/2940scg_1/swspan.html
Labels:
Preview file
1846 KB
Preview file
2413 KB
Preview file
3 KB
Preview file
825 KB
Preview file
1380 KB
Preview file
4 KB
Preview file
3 KB
Preview file
2 KB
Preview file
2 KB
Preview file
303 KB
Preview file
652 KB
Preview file
1 KB
Preview file
1085 KB
Preview file
772 KB
Preview file
245 KB
Preview file
590 KB
Preview file
545 KB
Comments
larry.govedich
Level 1
Level 1
‎03-24-2016 12:25 PM

Good job Matthew, very thorough

Lokesh Kumar Lal
Cisco Employee
Cisco Employee
‎12-27-2016 05:26 PM

Excellent! 

sgalarcett
Community Member
‎02-10-2017 10:48 AM

Excellent information, I will be of great help.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents