Using Python to access SecureX Remote target for HTTPS API Call

[jh.liew]

Hi, I'm not sure if this is feasible so just checking if someone from the community may have an answer to this.

We have some on-premise API endpoints that I would like to query using the orchestration module so we have gone ahead and setup the Secure Remote.

This works when we are using the HTTP service module in Orchestration but it appear to be fairly limited to the size of response from the API. So for this reason we are thinking if using Python module and calling it from within the script would work around the size issue.

However I am having difficulty in making it work although I have set the Orchestration Workflow to use remote target. Any idea if additional code/configuration is required here?

Thanks.

[chrivand]

hi @jh.liew , I am currently doing some research for you if this is possible. Could you light some more context into why the regular HTTP action does not work for you? You mention it is fairly limited in the response size? Can't you do some kind of pagination with a loop? Thanks! 

[jh.liew]

Hi @chrivand I have tried querying the API with various parameters, when a large result is returned which take more than 10 secs or so (based on observation), the HTTP action would just timed out instead. This only happen when the API call is made over SecureX remote towards on-premise hosted API endpoint.

Pagination would be possible I guess but will require some additional work, this is what I'm currently exploring in parallel. Thanks.

P/S: The API is CyberVision's by the way. Using Postman to perform manual query test it is able to return large dataset without issue.

[jh.liew]

Hi @chrivand, I found out what the actual issue now after working with the workflow development in the past few days. It's not the HTTP service timing out but to do with passing an output of that API call exceeding the limit of Python module to accept as input via system arguments.

While we can use pagination, we were looking at processing the data and merge the result - this proof to be too big for Python module to do in a single pass. If we use looping to run Python module multiple times, each of the Python runtime appear to be in its own sandbox and we're back to the same problem - limitation of using system argument to pass data around.

It would be easier if SecureX has some kind of temporary storage area where HTTP service can save into and allow Python module to read directly from.

[osanniko]

Hi @jh.liew ,

Have you tried storing your data in SecureX as a Custom Table and reading from it? https://ciscosecurity.github.io/sxo-05-security-workflows/variables/custom-types