PaaS Python App hangs until 'ioxclient app console' done, which forces a resume

[GarySmith]

Hi folks

I am running a Python IOx agent that can suddenly hang, mid-execution.  In IOx, the application is still shown as running but no more logs are output (or any other form of interact with other services, e.g. http calls to our server). 

The App has 2 threads and both are hung.

I can, however, then "bring it back to life" by merely executing an ioxclient app console command for the App.  Logging resumes, from some utterly random point in the code (it could have been midway through execution of a function); expected log messages are output with the now current timestamps and all is good once again.  

If I exit out of the console session, however, the App will again freeze up at some point in the future.

It seems the frequency of failure can be increased by increasing App logging levels and redirecting output to console.

Thoughts?

[Steve Zhang · Accepted solution]

The main reason for this is the console/print information occupied all the console cache and make it suspended.
The workaround is to write all the console information in the app log. You may use the app log for debug/troubleshooting instead of the console information.

[GarySmith]

Thanks for that, Steve. It was indeed the problem - a 3rd party module was slyly logging to console when debug logging was enabled.  Overrriding logger settings for that module solved the problem.