ISE : ログのリアルタイム監視(tail)について

Yuji Suzuki · ‎2018-02-25

Cisco ISE(Identity Services Engine)では、トラブルシューティングに向けてログを確認する際に、ISE 2.x - Support Bundle の取得方法や、ISE : CLIによる個別ログの取得方法についてで紹介しているようにログを取得・転送するケースが多いかと存じます。

本ドキュメントでは、ファイルのコピーや転送を必要とせず、ほぼリアルタイムにCisco ISEのログを監視する方法をご案内いたします。

まず、SSHにてCisco ISEにログインします。
【注意】
本ドキュメントでご案内する手順は、Console接続でも大部分において可能ではございますが、
SSHに比べて出力速度が遅いことにより、正しくログを確認できない場合がございます。
このためSSHでのご確認を推奨いたします。

まず、terminalでの出力行を無制限にします。
※ここは必須ではございませんが、無制限にした方がログを監視しやすくなります。

ise/admin# terminal length 0

ISE Version 3.2以降では、terminal lengthコマンドはscreen-lengthコマンドに置き換わっておりますので、出力行の制御はscreen-lengthコマンドをご利用ください。

次に監視したいログのファイル名を確認します。
ログファイル名の確認方法については、こちらをご確認ください。

ログファイル名確認ができたら、実際にログの監視を開始します。

ise/admin# sh logging application [ログファイル名] tail

この状態で、問題事象の再現確認等を実施いただくことで、ログをリアルタイムで監視することができます。

以下ではISEのLocal Store Logを監視し、パスワード不正が原因でRadius認証に失敗した例を記載させていただきます。

ise/admin# show logging application localStore/iseLocalStore.log tail
2018-02-25 16:33:03.817 +09:00 0000001663 70011 NOTICE System-Stats: ISE Counters, ConfigVersionId=74, OperationCounters=Counter=16_MnTLogProcessorN:60\,16_CAServiceN:32\,16_CAServiceT:32852\,4_EndpointsProfiled:4\,16_MnTLogProcessorU:6\,16_MnTLogProcessorT:40045\,16_CAServiceU:6\,16_SyslogU:0\,16_SyslogT:423\,16_RMIT:379\,16_RMIU:0\,16_GuestN:7\,16_TCNACDSN:12\,16_TCNACDST:10797\,16_TCNACDSU:2\,16_GuestU:0\,16_GuestT:334\,16_SyslogN:3\,4_EndpointsUpdated:13\,16_MisservicesU:9\,4_HostName_Event_Fetch_FromAD:0\,4_ARPSave:2\,16_MisservicesT:52598\,4_ProbeRadiusEndpointsDetected:2\,4_RemoteSave:1\,16_MisservicesN:195\,4_LocalEndPointReads:8\,16_DBServerN:78\,13_Protocol_Runtime_Context:0\,16_AdminWebappT:53392\,16_AdminWebappU:9\,16_DBServerU:57\,16_DBServerT:361234\,16_JVMN:10\,16_AdminWebappN:65\,16_DBListenerU:1\,16_DBListenerT:1940\,9_PolicySet-Default.Authorization_Policy-Basic_Authenticated_Access:1\,16_JVMT:104257\,16_ISEIndexingEngineT:62474\,16_JVMU:17\,16_ISEIndexingEngineU:10\,16_BYODN:1\,16_BYODT:0\,16_DBListenerN:2\,16_BYODU:0\,16_MessageQueueT:1509\,16_MessageQueueU:0\,16_ISEIndexingEngineN:72\,4_Probe_Requests_Dropped:0\,4_Probe_Requests_Received:0\,4_ArpCache_InsertUpdate_Received:12\,16_MessageQueueN:4\,2_PolicySet-Default.Allowed_Protocol-Dot1X:3\,16_iowait:208\,4_EndpointsDetected:2\,4_ProbeNmapScannedEndpoints:1\,16_MnTSessionDBT:3107\,16_MnTSessionDBU:1\,16_MnTSessionDBN:18\,16_NSFN:48\,16_ProfilerDatabaseN:3\,4_ARPMiss:1\,16_ProfilerDatabaseT:13966\,16_ProfilerDatabaseU:3\,16_NSFU:2\,4_EndpointsSaved:1\,16_NSFT:11070\,16_QuartzN:29\,4_EndpointCache_InsertUpdate_Received:18\,16_QuartzT:35124\,2_PolicySet-Default.Allowed_Protocol-Dot1X.Identity-Default:1\,16_ProfilerN:40\,16_ProfilerT:263\,16_ProfilerU:0\,4_ProfilerCacheHits:8\,16_MnTLogCollectorU:9\,16_RMIN:13\,16_MnTLogCollectorT:54635\,4_EndpointsCached:4\,4_RadiusPacketsReceived:7\,4_ARPRetrieve:2\,4_RemoteUpdate:3\,16_QuartzU:6\,4_NMAP_ScanEvent_Query:8\,16_MnTLogCollectorN:7\,4_ARPHit:1,
2018-02-25 16:33:28.564 +09:00 0000001681 5400 NOTICE Failed-Attempt: Authentication failed, ConfigVersionId=74, Device IP Address=1.150.0.30, Device Port=46638, DestinationIPAddress=1.150.0.12, DestinationPort=1812, RadiusPacketType=AccessRequest, UserName=test, Protocol=Radius, RequestLatency=33, NetworkDeviceName=VLAN1400, User-Name=test, NAS-IP-Address=192.168.0.251, NAS-Port=50103, Service-Type=Framed, Framed-IP-Address=10.2.1.254, Framed-MTU=1500, Callback-ID= qwertyu, Called-Station-ID=00-04-5F-00-0F-D2, Calling-Station-ID=00-12-34-22-1d-e2, NAS-Identifier=localhost, NAS-Port-Type=Ethernet, NAS-Port-Id=GigabitEthernet1/0/3, OriginalUserName=test, NetworkDeviceProfileName=Cisco, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, IsThirdPartyDeviceFlow=false, RadiusFlowType=Wired802_1x, SSID=00-04-5F-00-0F-D2, AcsSessionID=ise-22-vc/308566577/38, AuthenticationIdentityStore=Internal Users, AuthenticationMethod=PAP_ASCII, SelectedAccessService=Default Network Access, FailureReason=22040 Wrong password or invalid shared secret, Step=11001, Step=11017, Step=11117, Step=15049, Step=15008, Step=15048, Step=15004, Step=15041, Step=15006, Step=22072, Step=15013, Step=24210, Step=24212, Step=22040, Step=22057, Step=22061, Step=11003, SelectedAuthenticationIdentityStores=Internal Users, SelectedAuthenticationIdentityStores=All_AD_Join_Points, SelectedAuthenticationIdentityStores=Guest Users, NetworkDeviceGroups=IPSEC#Is IPSEC Device#No, NetworkDeviceGroups=Location#All Locations, NetworkDeviceGroups=Device Type#All Device Types, IdentityPolicyMatchedRule=Default1c888c55-45cb-43f6-925f-da58ccfac0b0, UserType=User, CPMSessionID=0196000cNCGwln/wAZUb6aD8RKLLyM0MSgXfcwHupeBG7ZDpmEQ, EndPointMACAddress=00-12-34-22-1D-E2, ISEPolicySetName=Default, AllowedProtocolMatchedRule=Dot1X, IdentitySelectionMatchedRule=Default, StepData=5= Normalised Radius.RadiusFlowType, StepData=6=Dot1X, StepData=9=All_User_ID_Stores, StepData=10=Internal Users, DTLSSupport=Unknown, Network Device Profile=Cisco, Location=Location#All Locations, Device Type=Device Type#All Device Types, IPSEC=IPSEC#Is IPSEC Device#No, EnableFlag=Enabled, Response={RadiusPacketType=AccessReject; AuthenticationResult=Failed; },
2018-02-25 16:33:39.872 +09:00 0000001699 5400 NOTICE Failed-Attempt: Authentication failed, ConfigVersionId=74, Device IP Address=1.150.0.30, Device Port=54878, DestinationIPAddress=1.150.0.12, DestinationPort=1812, RadiusPacketType=AccessRequest, UserName=test, Protocol=Radius, RequestLatency=29, NetworkDeviceName=VLAN1400, User-Name=test, NAS-IP-Address=192.168.0.251, NAS-Port=50103, Service-Type=Framed, Framed-IP-Address=10.2.1.254, Framed-MTU=1500, Callback-ID= qwertyu, Called-Station-ID=00-04-5F-00-0F-D2, Calling-Station-ID=00-12-34-94-41-2d, NAS-Identifier=localhost, NAS-Port-Type=Ethernet, NAS-Port-Id=GigabitEthernet1/0/3, OriginalUserName=test, NetworkDeviceProfileName=Cisco, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, IsThirdPartyDeviceFlow=false, RadiusFlowType=Wired802_1x, SSID=00-04-5F-00-0F-D2, AcsSessionID=ise-22-vc/308566577/39, AuthenticationIdentityStore=Internal Users, AuthenticationMethod=PAP_ASCII, SelectedAccessService=Default Network Access, FailureReason=22040 Wrong password or invalid shared secret, Step=11001, Step=11017, Step=11117, Step=15049, Step=15008, Step=15048, Step=15004, Step=15041, Step=15006, Step=22072, Step=15013, Step=24210, Step=24212, Step=22040, Step=22057, Step=22061, Step=11003, SelectedAuthenticationIdentityStores=Internal Users, SelectedAuthenticationIdentityStores=All_AD_Join_Points, SelectedAuthenticationIdentityStores=Guest Users, NetworkDeviceGroups=IPSEC#Is IPSEC Device#No, NetworkDeviceGroups=Location#All Locations, NetworkDeviceGroups=Device Type#All Device Types, IdentityPolicyMatchedRule=Default1c888c55-45cb-43f6-925f-da58ccfac0b0, UserType=User, CPMSessionID=0196000cFc8fcg2pRiqN2/GUWjrdBdEP1PoGQDbcp4a9EbGbFuo, EndPointMACAddress=00-12-34-94-41-2D, ISEPolicySetName=Default, AllowedProtocolMatchedRule=Dot1X, IdentitySelectionMatchedRule=Default, StepData=5= Normalised Radius.RadiusFlowType, StepData=6=Dot1X, StepData=9=All_User_ID_Stores, StepData=10=Internal Users, DTLSSupport=Unknown, Network Device Profile=Cisco, Location=Location#All Locations, Device Type=Device Type#All Device Types, IPSEC=IPSEC#Is IPSEC Device#No, EnableFlag=Enabled, Response={RadiusPacketType=AccessReject; AuthenticationResult=Failed; },
2018-02-25 16:33:45.063 +09:00 0000001700 70001 NOTICE System-Stats: ISE Process Health, ConfigVersionId=74, SysStatsAcsProcessHealth= Database Listener=running\, PID: 3200\; Database Server=running\, number of processes: 74\; Application Server=running\, PID: 9374\; Profiler Database=running\, PID: 6727\; ISE Indexing Engine=running\, PID: 11671\; AD Connector=running\, PID: 14084\; M&T Session Database=running\, PID: 6635\; M&T Log Collector=running\, PID: 9510\; M&T Log Processor=running\, PID: 9424\; Certificate Authority Service=running\, PID: 13841\; EST Service=running\, PID: 21900\; SXP Engine Service=disabled\; Docker Daemon=running\, PID: 13175\; Wifi Setup Helper Container=disabled\; pxGrid Infrastructure Service=disabled\; pxGrid Publisher Subscriber Service=disabled\; pxGrid Connection Manager=disabled\; pxGrid Controller=disabled\; PassiveID WMI Service=disabled\; PassiveID Syslog Service=disabled\; PassiveID API Service=disabled\; PassiveID Agent Service=disabled\; PassiveID Endpoint Service=disabled\; PassiveID SPAN Service=disabled\; DHCP Server (dhcpd)=disabled\; DNS Server (named)=disabled,
2018-02-25 16:33:45.063 +09:00 0000001701 70000 NOTICE System-Stats: ISE Utilization, ConfigVersionId=74, SysStatsUtilizationCpu=5.28%, SysStatsUtilizationNetwork=eth0: rcvd = 194750\; sent = 203436 \;rcvd_dropped = 0\; sent_dropped = 0, SysStatsUtilizationNetwork=eth1: rcvd = 15915\; sent = 785 \;rcvd_dropped = 0\; sent_dropped = 0, SysStatsUtilizationMemory=68.65%, SysStatsUtilizationDiskIO=2.07%, SysStatsUtilizationDiskSpace=15% /, SysStatsUtilizationDiskSpace=23% /boot, SysStatsUtilizationDiskSpace=2% /storedconfig, SysStatsUtilizationDiskSpace=33% /opt, SysStatsUtilizationDiskSpace=1% /tmp, SysStatsUtilizationDiskSpace=14% /localdisk, AverageRadiusRequestLatency=116, AverageTacacsRequestLatency=0, DeltaRadiusRequestCount=5, DeltaTacacsRequestCount=0, SysStatsUtilizationLoadAvg=0.38, SysStatsCpuCount=4, SysStatsProcessMemoryMB=8129, ActiveSessionCount=0,
2018-02-25 16:34:06.840 +09:00 0000001719 5400 NOTICE Failed-Attempt: Authentication failed, ConfigVersionId=74, Device IP Address=1.150.0.30, Device Port=41126, DestinationIPAddress=1.150.0.12, DestinationPort=1812, RadiusPacketType=AccessRequest, UserName=test, Protocol=Radius, RequestLatency=29, NetworkDeviceName=VLAN1400, User-Name=test, NAS-IP-Address=192.168.0.251, NAS-Port=50103, Service-Type=Framed, Framed-IP-Address=10.2.1.254, Framed-MTU=1500, Callback-ID= qwertyu, Called-Station-ID=00-04-5F-00-0F-D2, Calling-Station-ID=00-12-34-20-14-98, NAS-Identifier=localhost, NAS-Port-Type=Ethernet, NAS-Port-Id=GigabitEthernet1/0/3, OriginalUserName=test, NetworkDeviceProfileName=Cisco, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, IsThirdPartyDeviceFlow=false, RadiusFlowType=Wired802_1x, SSID=00-04-5F-00-0F-D2, AcsSessionID=ise-22-vc/308566577/40, AuthenticationIdentityStore=Internal Users, AuthenticationMethod=PAP_ASCII, SelectedAccessService=Default Network Access, FailureReason=22040 Wrong password or invalid shared secret, Step=11001, Step=11017, Step=11117, Step=15049, Step=15008, Step=15048, Step=15004, Step=15041, Step=15006, Step=22072, Step=15013, Step=24210, Step=24212, Step=22040, Step=22057, Step=22061, Step=11003, SelectedAuthenticationIdentityStores=Internal Users, SelectedAuthenticationIdentityStores=All_AD_Join_Points, SelectedAuthenticationIdentityStores=Guest Users, NetworkDeviceGroups=IPSEC#Is IPSEC Device#No, NetworkDeviceGroups=Location#All Locations, NetworkDeviceGroups=Device Type#All Device Types, IdentityPolicyMatchedRule=Default1c888c55-45cb-43f6-925f-da58ccfac0b0, UserType=User, CPMSessionID=0196000c1hhI2gJWhoF8WlfVqaTs/3LZKs0VQXJWjHzHNZi0LYM, EndPointMACAddress=00-12-34-20-14-98, ISEPolicySetName=Default, AllowedProtocolMatchedRule=Dot1X, IdentitySelectionMatchedRule=Default, StepData=5= Normalised Radius.RadiusFlowType, StepData=6=Dot1X, StepData=9=All_User_ID_Stores, StepData=10=Internal Users, DTLSSupport=Unknown, Network Device Profile=Cisco, Location=Location#All Locations, Device Type=Device Type#All Device Types, IPSEC=IPSEC#Is IPSEC Device#No, EnableFlag=Enabled, Response={RadiusPacketType=AccessReject; AuthenticationResult=Failed; },
2018-02-25 16:38:03.819 +09:00 0000001720 70011 NOTICE System-Stats: ISE Counters, ConfigVersionId=74, OperationCounters=Counter=4_EndpointsProfiled:6\,4_EndpointsUpdated:21\,4_HostName_Event_Fetch_FromAD:0\,4_ProbeRadiusEndpointsDetected:3\,4_RemoteSave:3\,4_LocalEndPointReads:9\,13_Protocol_Runtime_Context:0\,4_Probe_Requests_Dropped:0\,4_Probe_Requests_Received:0\,4_ArpCache_InsertUpdate_Received:14\,2_PolicySet-Default.Allowed_Protocol-Dot1X:3\,16_iowait:207\,4_EndpointsDetected:3\,4_EndPoint_OraclePersist_Received:55\,4_EndPoint_Profiling_Events:44\,4_EndpointsSaved:3\,4_EndpointCache_InsertUpdate_Received:20\,2_PolicySet-Default.Allowed_Protocol-Dot1X.Identity-Default:3\,4_ProfilerCacheHits:12\,4_EndPoint_OwnerShip_Change:0\,4_EndpointsCached:6\,4_RadiusPacketsReceived:9\,4_RemoteUpdate:3\,4_NMAP_ScanEvent_Query:9,
2018-02-25 16:39:00.618 +09:00 0000001722 70001 NOTICE System-Stats: ISE Process Health, ConfigVersionId=74, SysStatsAcsProcessHealth= Database Listener=running\, PID: 3200\; Database Server=running\, number of processes: 70\; Application Server=running\, PID: 9374\; Profiler Database=running\, PID: 6727\; ISE Indexing Engine=running\, PID: 11671\; AD Connector=running\, PID: 14084\; M&T Session Database=running\, PID: 6635\; M&T Log Collector=running\, PID: 9510\; M&T Log Processor=running\, PID: 9424\; Certificate Authority Service=running\, PID: 13841\; EST Service=running\, PID: 21900\; SXP Engine Service=disabled\; Docker Daemon=running\, PID: 13175\; Wifi Setup Helper Container=disabled\; pxGrid Infrastructure Service=disabled\; pxGrid Publisher Subscriber Service=disabled\; pxGrid Connection Manager=disabled\; pxGrid Controller=disabled\; PassiveID WMI Service=disabled\; PassiveID Syslog Service=disabled\; PassiveID API Service=disabled\; PassiveID Agent Service=disabled\; PassiveID Endpoint Service=disabled\; PassiveID SPAN Service=disabled\; DHCP Server (dhcpd)=disabled\; DNS Server (named)=disabled,
2018-02-25 16:39:00.618 +09:00 0000001721 70000 NOTICE System-Stats: ISE Utilization, ConfigVersionId=74, SysStatsUtilizationCpu=6.00%, SysStatsUtilizationNetwork=eth0: rcvd = 165944\; sent = 93243 \;rcvd_dropped = 0\; sent_dropped = 0, SysStatsUtilizationNetwork=eth1: rcvd = 17982\; sent = 0 \;rcvd_dropped = 0\; sent_dropped = 0, SysStatsUtilizationMemory=68.77%, SysStatsUtilizationDiskIO=2.06%, SysStatsUtilizationDiskSpace=15% /, SysStatsUtilizationDiskSpace=23% /boot, SysStatsUtilizationDiskSpace=2% /storedconfig, SysStatsUtilizationDiskSpace=33% /opt, SysStatsUtilizationDiskSpace=1% /tmp, SysStatsUtilizationDiskSpace=14% /localdisk, AverageRadiusRequestLatency=29, AverageTacacsRequestLatency=0, DeltaRadiusRequestCount=1, DeltaTacacsRequestCount=0, SysStatsUtilizationLoadAvg=0.30, SysStatsCpuCount=4, SysStatsProcessMemoryMB=8129, ActiveSessionCount=0,
2018-02-25 16:43:03.821 +09:00 0000001723 70011 NOTICE System-Stats: ISE Counters, ConfigVersionId=74, OperationCounters=Counter=4_HostName_Event_Fetch_FromAD:0\,13_Protocol_Runtime_Context:0\,4_Probe_Requests_Dropped:0\,4_Probe_Requests_Received:0\,4_ArpCache_InsertUpdate_Received:14\,16_iowait:207\,4_EndpointCache_InsertUpdate_Received:22\,4_NMAP_ScanEvent_Query:9,


2018-02-25 16:43:34.785 +09:00 0000001741 5400 NOTICE Failed-Attempt: Authentication failed, ConfigVersionId=74, Device IP Address=1.150.0.30, Device Port=39148, DestinationIPAddress=1.150.0.12, DestinationPort=1812, RadiusPacketType=AccessRequest, UserName=test, Protocol=Radius, RequestLatency=25, NetworkDeviceName=VLAN1400, User-Name=test, NAS-IP-Address=192.168.0.251, NAS-Port=50103, Service-Type=Framed, Framed-IP-Address=10.2.1.254, Framed-MTU=1500, Callback-ID= qwertyu, Called-Station-ID=00-04-5F-00-0F-D2, Calling-Station-ID=00-12-34-67-93-ba, NAS-Identifier=localhost, NAS-Port-Type=Ethernet, NAS-Port-Id=GigabitEthernet1/0/3, OriginalUserName=test, NetworkDeviceProfileName=Cisco, NetworkDeviceProfileId=b0699505-3150-4215-a80e-6753d45bf56c, IsThirdPartyDeviceFlow=false, RadiusFlowType=Wired802_1x, SSID=00-04-5F-00-0F-D2, AcsSessionID=ise-22-vc/308566577/41, AuthenticationIdentityStore=Internal Users, AuthenticationMethod=PAP_ASCII, SelectedAccessService=Default Network Access, FailureReason=22040 Wrong password or invalid shared secret, Step=11001, Step=11017, Step=11117, Step=15049, Step=15008, Step=15048, Step=15004, Step=15041, Step=15006, Step=22072, Step=15013, Step=24210, Step=24212, Step=22040, Step=22057, Step=22061, Step=11003, SelectedAuthenticationIdentityStores=Internal Users, SelectedAuthenticationIdentityStores=All_AD_Join_Points, SelectedAuthenticationIdentityStores=Guest Users, NetworkDeviceGroups=IPSEC#Is IPSEC Device#No, NetworkDeviceGroups=Location#All Locations, NetworkDeviceGroups=Device Type#All Device Types, IdentityPolicyMatchedRule=Default1c888c55-45cb-43f6-925f-da58ccfac0b0, UserType=User, CPMSessionID=0196000cnh55cviSqjJYA9D1U/SnvdHnu7mtLypYfKexTwkbJL8, EndPointMACAddress=00-12-34-67-93-BA, ISEPolicySetName=Default, AllowedProtocolMatchedRule=Dot1X, IdentitySelectionMatchedRule=Default, StepData=5= Normalised Radius.RadiusFlowType, StepData=6=Dot1X, StepData=9=All_User_ID_Stores, StepData=10=Internal Users, DTLSSupport=Unknown, Network Device Profile=Cisco, Location=Location#All Locations, Device Type=Device Type#All Device Types, IPSEC=IPSEC#Is IPSEC Device#No, EnableFlag=Enabled, Response={RadiusPacketType=AccessReject; AuthenticationResult=Failed; },

その他、ISEを含むAAA(認証・認可・アカウンティング)に関して、設計やトラブルシューティングなどに役立つ情報を以下にまとめてあります。お探しの情報が見つからなかった場合は、是非ご一読ください。

AAA トラブルシューティング