07-01-2021 03:55 AM - edited 07-01-2021 03:59 AM
We have a lab DNA-C that was reimaged with 2.1.2.5 a couple of months ago when we rebuilt the lab. It has been upgraded to 2.1.2.7. It is in a small lab with 4 x Catalyst 9300's, 1 x Catalyst 9800 Wi-Fi controller, 2 x Catalyst 9130AXI APs and a AP1800S Wi-Fi sensor. There are also two Catalyst 3750X's that act as the Fusion routers and provide the connectivity for DNA-C, the WLC, two ESXi hosts, a C1841 with a 16-port Async module for terminal access to everything and an ASA5508-X at the boundary. Running on the ESXi boxes are ISE 2.7, several Windows 2019 Servers (AD, DNS, DHCP & a CA) and 4 x Windows 10 clients with pass-thru adapters for Wi-Fi & Ethernet so we can test dot1x.
It seems that since the upgrade to 2.1.2.7 the influxdb service continually crashes with Out Of Memory errors. If we look at the service from the CLI we see this:
/vault from vault (ro) influxdb: Container ID: docker://02f9d09e0748a947b52fd46601247d1c2fb9e410b16a6a455b1856ecfb667338 Image: maglev-registry.maglev-system.svc.cluster.local:5000/influxdb:1.5.15 Image ID: docker-pullable://maglev-registry.maglev-system.svc.cluster.local:5000/influxdb@sha256:d5fe8539e98c7b807dab68e8f4a83d375ffd8073ddbca6a16bbb65736992e12b Ports: 8083/TCP, 8086/TCP, 8089/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP State: Running Started: Thu, 01 Jul 2021 10:46:03 +0000 Last State: Terminated Reason: OOMKilled Message: 2021-07-01 10:46:02 | ERROR | Invalid Configurations present on InfluxDB Exit Code: 137 Started: Thu, 01 Jul 2021 10:42:58 +0000 Finished: Thu, 01 Jul 2021 10:46:02 +0000 Ready: True Restart Count: 1
I'm tempted to upgrade to 2.2.2.3, however the 'Invalid Configurations' error message is a bit concerning. This error doesn't appear every time, however the service restarts every few minutes. Is this a TAC case?
Solved! Go to Solution.
07-01-2021 07:34 AM
Raised a TAC case for this. Looks like we are hitting CSCvy83860
07-01-2021 05:43 AM
I also have had influxdb issues back in the day on older Gen1 appliances running an older version of code. My experience can be seen here: DNAC Services Down - (InfluxDB Issue) - Cisco Community
I would strongly suggest working with TAC on this one as things could potentially get messy. Good luck & HTH!
07-01-2021 07:34 AM
Raised a TAC case for this. Looks like we are hitting CSCvy83860
07-01-2021 10:34 AM
Good to know. Please share the fix once you are good
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: