Take a look at the logs. Generally, I start with the ipics.log, but there are several important logs in the system.
SSH into the server as root
(Since 2.2 or maybe 4.0) In root's home directory, there is a folder called logs (/root/logs). It contains symlinks to the most common logs we use. The most common are found at the top of the folder, while the less common ones are found in the sub-folders below. This makes it really easy to watch the system in near real-time with the command:
tail -F ~/logs/*
To get a general idea what when wrong, let's look for unique error messages. The following command will distill a 7000 line log file down to a short list of error messages with the option to further filter the results using egrep:
Then use grep to look for interesting keywords you got from the first query to understand the context of the error and if it is worth investigating. The following command will show you all occurrences of "InterestingKeyword" plus 5 lines of context above and below the matching line:
grep -C5 "InterestingKeyword" ipics.log
There is a lot of noise in the logs, so it is nice to compare with another server to see what is "normal".
If you don't see anything related to the symptom you are debugging, expand your search to the other logs:
o~/logs/catalina.out - This is where your thread dumps go, as well as general tomcat (usually non-IPICS) errors show up.
o~/logs/stdout.log and ~/logs/stderr.log - Sometimes you find stuff in here, but since it isn't timestamped, it is difficult to correlate with the other logs.
o~/logs/edr.log - If your problem is related to HA setup or tear-down.
oThen start looking at subsystem logs like nm.log, rcs.log, dms.log, engine.out, and the MIVR logs.
o~/logs/os/latest/ - The install logs
oUse get_all_logs to take a snapshot of all the system logs before they roll-over. If you are on an HA system, remember to run it on the secondary too.
·Be careful not to modify the ipics.log while the server is running. Always copy the log to /tmp before you make any changes or filter-out lines. To prevent accidental changes, open logs with "vi -R ipics.log"
Where to begin?
·There are a number of useful diagnostic tools you can use, such as:
oversions - Dumps the version data for the OS, WAR, and IPICS RPM's
oservice ipics status Overall system status
oservice ipics ha-status HA status
otop-ipics - Good for watching performance of IPICS related processes
o/opt/cisco/ipics/security/checksec - Gives SSH/SSL trust status and diagnoses some common problems.
o/opt/cisco/ippe/bin/systat - Gives overall system status and diagnoses some common problems.
Hi, Can someone help me understand the effect of implementing metric in the address family ipv4 rather than in a specific interface.Please see below config for reference. router isis 123is-type level-2-onlynet xx.xxxx.xxxx.xxxx.xxxx.xxnsf ietflo...
Hello!As a pandemic consequence most users are working remotely and they are connected by VPN remote access.Another consequence are the increasing number of tickets from users claim about quality of their VPN connections.Have anyone had already deploy som...
The purpose of this document is to demonstrate how ISE authenticate / authorize a user that uses a smart card (PIN + Certificate) and password mechanism to login their system. This document describes the components used for this setup, configuration of IS...
Hello,We have just upgraded FTD 2110 firewall to firmware version 6.6.1. Since the AC element count is 800k, FMC shows a warning message "the number of access list elements generated for the access control policy exceeds the limit for this platform", sugg...
So I have come to learn that AMP doesn't have features that I am accustom to. Is there a way, beside creating more policies, to apply an exclusion to a single system? I am needing to create a 5 separate exclusion for my backup software. The machine f...