cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2741
Views
5
Helpful
3
Replies

influxdb crashing

We have a lab DNA-C that was reimaged with 2.1.2.5 a couple of months ago when we rebuilt the lab.  It has been upgraded to 2.1.2.7.  It is in a small lab with 4 x Catalyst 9300's, 1 x Catalyst 9800 Wi-Fi controller, 2 x Catalyst 9130AXI APs and a AP1800S Wi-Fi sensor.  There are also two Catalyst 3750X's that act as the Fusion routers and provide the connectivity for DNA-C, the WLC, two ESXi hosts, a C1841 with a 16-port Async module for terminal access to everything and an ASA5508-X at the boundary.  Running on the ESXi boxes are ISE 2.7, several Windows 2019 Servers (AD, DNS, DHCP & a CA) and 4 x Windows 10 clients with pass-thru adapters for Wi-Fi & Ethernet so we can test dot1x.

It seems that since the upgrade to 2.1.2.7 the influxdb service continually crashes with Out Of Memory errors.  If we look at the service from the CLI we see this:

     /vault from vault (ro)
  influxdb:
    Container ID:  docker://02f9d09e0748a947b52fd46601247d1c2fb9e410b16a6a455b1856ecfb667338
    Image:         maglev-registry.maglev-system.svc.cluster.local:5000/influxdb:1.5.15
    Image ID:      docker-pullable://maglev-registry.maglev-system.svc.cluster.local:5000/influxdb@sha256:d5fe8539e98c7b807dab68e8f4a83d375ffd8073ddbca6a16bbb65736992e12b
    Ports:         8083/TCP, 8086/TCP, 8089/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    State:         Running
      Started:     Thu, 01 Jul 2021 10:46:03 +0000
    Last State:    Terminated
      Reason:      OOMKilled
      Message:     2021-07-01 10:46:02 | ERROR | Invalid Configurations present on InfluxDB

      Exit Code:    137
      Started:      Thu, 01 Jul 2021 10:42:58 +0000
      Finished:     Thu, 01 Jul 2021 10:46:02 +0000
    Ready:          True
    Restart Count:  1

 

I'm tempted to upgrade to 2.2.2.3, however the 'Invalid Configurations' error message is a bit concerning.  This error doesn't appear every time, however the service restarts every few minutes.  Is this a TAC case?

 

 

1 Accepted Solution

Accepted Solutions

Raised a TAC case for this.  Looks like we are hitting CSCvy83860

Bug Search (cisco.com)

View solution in original post

3 Replies 3

Mike.Cifelli
VIP Alumni
VIP Alumni

I also have had influxdb issues back in the day on older Gen1 appliances running an older version of code.  My experience can be seen here: DNAC Services Down - (InfluxDB Issue) - Cisco Community

I would strongly suggest working with TAC on this one as things could potentially get messy.  Good luck & HTH!

Raised a TAC case for this.  Looks like we are hitting CSCvy83860

Bug Search (cisco.com)

Mike.Cifelli
VIP Alumni
VIP Alumni

Good to know.  Please share the fix once you are good TIA!

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: