Hey, this was great information.
Are you concerned about the timing of logstash and your update script? Perhaps I'm not understanding it correctly, but I'm thinking you're missing between 0 - 299 seconds of data on each run.
Anyhow, comment #2: It looks like they have rabbitmq support, which logstash supports directly. However, I wasn't able to get that to work. I had the same issue as these guys: https://discuss.elastic.co/t/rabbitmq-input-plugin-does-not-adhere-to-pipeline-configuration/159384
I'm not sure if it is a Logstash bug, or a Cisco bug. I'd lean to Logstash, only because the API works when accessed by different methods.
To get the rabbitmq stream to work, I ended up making my own python script that attaches to the stream and forwards it to the real logstash process. I had trouble at first, but a Cisco TAC engineer gave me a test script showing the event stream in text output. I adapted that same script to work for me. Then I used portions of your config to import the data. It would be cool if logstash could just collect it directly, but it keeps reverting to 'localhost' for some reason.
... View more