cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1587
Views
3
Helpful
4
Replies

SecureX HTTP Request fails with encoded parameter in body

TomML
Level 1
Level 1

Hi, 

I'm testing a draft workflow that modifies a user's attributes in Duo using the Duo Admin API and based on the "Duo - Admin - Add User to Group" atomic.  Currently it just modifies user's notes:

TomML_0-1683147159716.png

When I use spaces in the notes parameter, it fails the workflow. 

TomML_2-1683148130171.png

TomML_3-1683148179783.png

The workflow passes when there are no spaces (no characters to encode).  Per the Hints in HTTP Request | SecureX orchestration (ciscosecurity.github.io), I think I already created the payload in an "Execute Python Script" block.  But the payload is just text (notes=Some%20notes%20to%20add) not JSON.  How do deal with encoded characters in the request body?  I tried surrounding the [params_encoded$] in quotes but to no avail.

TomML_1-1683147577225.png

However, it works just fine in Python:

 
import requests

method = 'POST'
host = 'api-XXXXXXXX.duosecurity.com'
path = '/admin/v1/users/XXXXXXXXXXXXXX'
params = {'notes': 'Some notes to add'}
skey=<key>
ikey=<ikey>

c = canon(
        method=method,
        host=host,
        path=path,
        params=params
   )

#print(c)

urlPath = c[-2]
pathParm = c[-1]

print(pathParm)

headRequest = signHeader(
        method = method,
        host = host,
        path = path,
        params = params,
        skey=skey,
        ikey=ikey)

headRequest
['Host'] = host
headRequest['Content-Type'] = 'application/x-www-form-urlencoded'

response = requests.request(method=method,
                            url=url,
                            headers=headRequest,
                            data= pathParm)
                       

print(response.status_code)

notes=Some%20notes%20to%20add 
200

Thanks!

4 Replies 4

chrivand
Cisco Employee
Cisco Employee

it sounds to me like you don't need that Pyhton block at all, as you can just format the payload directly in the HTTP request activity. that being said are you sure you are exporting the script variable correctly? why are you not exporting the variable `j` as a normal string? the script variable is not called `params` and you should not export it as secure string either.

Thanks @chrivand .

I am using the "Duo - Admin - Add User to Group" atomic as a guide for my workflow.  The payload for the POST request needs to be formatted as URL-encoded key-value pairs (the same request format used by browsers to submit form data), Duo Admin API | Duo Security

 

EXECUTE PYTHON SCRIPT Generate Request Parameters

Outputs the j variable {"group_id":"XXXX"} to params variable.

TomML_0-1683213520022.png

EXECUTE PYTHON SCRIPT Generate Request Signature

Takes the params variable and outputs it as group_id=XXXX to the params_enc variable.

TomML_1-1683213746991.png

HTTP REQUEST Execute Duo API Call

TomML_2-1683214046625.png

Take the params_enc for the request body.  This API call will fail in my workflow (to modify a user's notes) when the payload contains encoded characters (notes=Some%20notes%20to%20add).  If the payload lacks any encoded characters, it works (notes=NoSpaces).

Fortunately, changing my workflow to alter a user's Duo status back to active works (status=active for the POST request body).  Just curious if there is a workaround for an URL-encoded key-value pairs payload should I want to change a user's notes, name, or any other attribute that could possibly contain spaces or other special characters.  Tried fumbling with a straight Python block to send the POST request.

hi! I did some asking around and please check out these 2 WFs (courtesy of Sudhir Desai):

{
  "workflow": {
    "unique_name": "definition_workflow_025JZ5L701VPZ4lXLAzAIimGPbT6InsMmk5",
    "name": "Duo API POST Parameters Encoder",
    "title": "Duo API POST Parameters Encoder",
    "type": "generic.workflow",
    "base_type": "workflow",
    "variables": [
      {
        "schema_id": "datatype.secure_string",
        "properties": {
          "value": "",
          "scope": "output",
          "name": "post_data_outer",
          "type": "datatype.secure_string",
          "is_required": false,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZ70KBMQAK1VCx7LlNQRdSsOwTKeCzZc",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.string",
        "properties": {
          "value": "",
          "scope": "input",
          "name": "input_post_data",
          "type": "datatype.string",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZASRBINMT7n0bNolCCzQMad2ZSj2UL7",
        "object_type": "variable_workflow"
      }
    ],
    "properties": {
      "atomic": {
        "is_atomic": false
      },
      "delete_workflow_instance": false,
      "description": "This atomic action will create a secure json string for parameters.",
      "display_name": "Duo API POST Parameters Encoder",
      "runtime_user": {
        "target_default": true
      },
      "target": {
        "no_target": true
      }
    },
    "object_type": "definition_workflow",
    "actions": [
      {
        "unique_name": "definition_activity_025JZ9TCJ44V86vCr6muba1LDSjG5PKC1lB",
        "name": "Execute Python Script",
        "title": "Execute Python Script",
        "type": "python3.script",
        "base_type": "activity",
        "properties": {
          "action_timeout": 180,
          "continue_on_failure": false,
          "display_name": "Execute Python Script",
          "script": "\"\"\"\nFunction: _encode_post\nPurpose: This function encodes POST data to help in policy creation\n\"\"\"\npost_data_str = json.dumps($workflow.definition_workflow_025JZ5L701VPZ4lXLAzAIimGPbT6InsMmk5.input.variable_workflow_025JZASRBINMT7n0bNolCCzQMad2ZSj2UL7$)\nencoded = urllib.parse.quote_plus(post_data_str)\n$workflow.definition_workflow_025JZ5L701VPZ4lXLAzAIimGPbT6InsMmk5.output.variable_workflow_025JZ70KBMQAK1VCx7LlNQRdSsOwTKeCzZc$ = {\n  \"payload\": encoded\n}",
          "script_arguments": [
            "",
            "$workflow.definition_workflow_025JZ5L701VPZ4lXLAzAIimGPbT6InsMmk5.input.variable_workflow_025JZASRBINMT7n0bNolCCzQMad2ZSj2UL7$"
          ],
          "script_queries": [
            {
              "script_query": "$workflow.definition_workflow_025JZ5L701VPZ4lXLAzAIimGPbT6InsMmk5.output.variable_workflow_025JZ70KBMQAK1VCx7LlNQRdSsOwTKeCzZc$",
              "script_query_name": "output variable",
              "script_query_type": "secure_string"
            }
          ],
          "skip_execution": false
        },
        "object_type": "definition_activity"
      }
    ],
    "categories": [
      "category_025JZ9M4WAU6Y0SVu2pHIiv6ZiQYb8EChgT"
    ]
  },
  "categories": {
    "category_025JZ9M4WAU6Y0SVu2pHIiv6ZiQYb8EChgT": {
      "unique_name": "category_025JZ9M4WAU6Y0SVu2pHIiv6ZiQYb8EChgT",
      "name": "Duo",
      "title": "Duo",
      "type": "basic.category",
      "base_type": "category",
      "category_type": "custom",
      "object_type": "category"
    }
  }
} 

 

{
  "workflow": {
    "unique_name": "definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd",
    "name": "Duo API Sign Headers",
    "title": "Duo API Sign Headers",
    "type": "generic.workflow",
    "base_type": "workflow",
    "variables": [
      {
        "schema_id": "datatype.secure_string",
        "properties": {
          "value": "",
          "scope": "input",
          "name": "ikey",
          "type": "datatype.secure_string",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZGOFIQ9BE2qmeJ6a4NrhoOceUAsWMF4",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.secure_string",
        "properties": {
          "value": "",
          "scope": "input",
          "name": "skey",
          "type": "datatype.secure_string",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZGKU16SAR2kzFry870YXghVJWlb6YTn",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.string",
        "properties": {
          "value": "",
          "scope": "input",
          "name": "path",
          "type": "datatype.string",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZFWEFTOZH6pLKWGWAQER44PVsm63zkE",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.string",
        "properties": {
          "value": "",
          "scope": "input",
          "name": "host",
          "type": "datatype.string",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZFSOIX5C51uxszxjWHZBXZaF2cxbKhy",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.secure_string",
        "properties": {
          "value": "",
          "scope": "output",
          "name": "signed_header",
          "type": "datatype.secure_string",
          "is_required": false,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZHW3ZI29O3RBugfkj0cBlKQ1NWCMpl7",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.string",
        "properties": {
          "value": "",
          "scope": "input",
          "name": "method",
          "type": "datatype.string",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZFO0KC6SO79p6xMBLws20gZwlUVhsWs",
        "object_type": "variable_workflow"
      },
      {
        "schema_id": "datatype.string",
        "properties": {
          "value": "{}",
          "scope": "input",
          "name": "params",
          "type": "datatype.string",
          "description": "parameters, defaults to {} for a GET method",
          "is_required": true,
          "is_invisible": false
        },
        "unique_name": "variable_workflow_025JZG9W2QC9L3IXHwBgTQ9ajDsBJKZsk9A",
        "object_type": "variable_workflow"
      }
    ],
    "properties": {
      "atomic": {
        "is_atomic": false
      },
      "delete_workflow_instance": false,
      "display_name": "Duo API Sign Headers",
      "runtime_user": {
        "target_default": true
      },
      "target": {
        "no_target": true
      }
    },
    "object_type": "definition_workflow",
    "actions": [
      {
        "unique_name": "definition_activity_025JZF0DDBB2R04d1o3N47Bd3BNcjl6ufwS",
        "name": "Execute Python Script",
        "title": "Execute Python Script",
        "type": "python3.script",
        "base_type": "activity",
        "properties": {
          "action_timeout": 180,
          "continue_on_failure": false,
          "description": "This ",
          "display_name": "Execute Python Script",
          "script": "import base64\nimport email.utils\nimport hmac\nimport hashlib\nimport urllib\n\"\"\"\nfunction: sign\npurpose: this function can be used to construct the \"Authorization\" and \"Date\" headers\ninputs:\n    method: GET, POST [uppercase]\n    host: personal API url [lowercase]\n    path: path to API method [lowercase]\n    params: URL-encoded list of key=value request parameters [alphabetically sorted by key]\n    skey: secret key\n    ikey: integration key\n\noutputs: Return HTTP Basic Authentication (\"Authorization\" and \"Date\") headers.\n\"\"\" \n# create canonical string\nnow = email.utils.formatdate()\ncanon = [now, $workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZFO0KC6SO79p6xMBLws20gZwlUVhsWs$.upper(), $workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZFSOIX5C51uxszxjWHZBXZaF2cxbKhy$.lower(), $workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZFWEFTOZH6pLKWGWAQER44PVsm63zkE$]\nargs = []\nfor key in sorted($workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZG9W2QC9L3IXHwBgTQ9ajDsBJKZsk9A$.keys()):\n    val = $workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZG9W2QC9L3IXHwBgTQ9ajDsBJKZsk9A$[key].encode(\"utf-8\")\n    args.append(\n        '%s=%s' % (urllib.parse.\n                   quote(key, '~'), urllib.parse.quote(val, '~')))\ncanon.append('&'.join(args))\ncanon = '\\n'.join(canon)\n\n# sign canonical string\nsig = hmac.new(bytes($workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZGKU16SAR2kzFry870YXghVJWlb6YTn$, encoding='utf-8'),\n               bytes(canon, encoding='utf-8'),\n               hashlib.sha1)\nauth = '%s:%s' % ($workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZGOFIQ9BE2qmeJ6a4NrhoOceUAsWMF4$, sig.hexdigest())\n\n# return headers\n$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.output.variable_workflow_025JZHW3ZI29O3RBugfkj0cBlKQ1NWCMpl7$ = {'Date': now, 'Authorization': 'Basic %s' % base64.b64encode(bytes(auth, encoding=\"utf-8\")).decode()}",
          "script_arguments": [
            "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZFSOIX5C51uxszxjWHZBXZaF2cxbKhy$",
            "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZGOFIQ9BE2qmeJ6a4NrhoOceUAsWMF4$",
            "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZFO0KC6SO79p6xMBLws20gZwlUVhsWs$",
            "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZG9W2QC9L3IXHwBgTQ9ajDsBJKZsk9A$",
            "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZFWEFTOZH6pLKWGWAQER44PVsm63zkE$",
            "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.input.variable_workflow_025JZGKU16SAR2kzFry870YXghVJWlb6YTn$"
          ],
          "script_queries": [
            {
              "script_query": "$workflow.definition_workflow_025JZE5IC0FFU271F0327WsKRA25FvJaryd.output.variable_workflow_025JZHW3ZI29O3RBugfkj0cBlKQ1NWCMpl7$",
              "script_query_name": "signed header",
              "script_query_type": "secure_string"
            }
          ],
          "skip_execution": false
        },
        "object_type": "definition_activity"
      }
    ],
    "categories": [
      "category_025JZ9M4WAU6Y0SVu2pHIiv6ZiQYb8EChgT"
    ]
  },
  "categories": {
    "category_025JZ9M4WAU6Y0SVu2pHIiv6ZiQYb8EChgT": {
      "unique_name": "category_025JZ9M4WAU6Y0SVu2pHIiv6ZiQYb8EChgT",
      "name": "Duo",
      "title": "Duo",
      "type": "basic.category",
      "base_type": "category",
      "category_type": "custom",
      "object_type": "category"
    }
  }
}

 

Thanks again.  I got it working when using multiple request parameters (notes and email) via Python Script blocks.

EXECUTE PYTHON SCRIPT Generate Request Parameters

TomML_0-1683672793141.png

import sys,json

(notes,email) = sys.argv[1:]

d = {}
d["notes"] = notes
d["email"] = email

j = json.dumps(d)

Execute Python POST API Request

TomML_1-1683672997902.png

 

import base64, email.utils, hmac, hashlib, urllib, requests, json, sys

# Sign Headers function
def signHeader(method, host, path, params, skey, ikey):
"""
Return HTTP Basic Authentication ("Authorization" and "Date") headers.
method, host, path: strings for request
params: dict of request parameters
skey: secret key
ikey: integration key
"""

# create canonical string
now = email.utils.formatdate()
canon = [now, method.upper(), host.lower(), path]
args = []
for key in sorted(params.keys()):
val = params[key].encode("utf-8")
args.append(
'%s=%s' % (urllib.parse.quote(key, '~'), urllib.parse.quote(val, '~')))
canon.append('&'.join(args))
canon = '\n'.join(canon)

# sign canonical string
sig = hmac.new(bytes(skey, encoding='utf-8'),
bytes(canon, encoding='utf-8'),
hashlib.sha1)
auth = '%s:%s' % (ikey, sig.hexdigest())

# return headers
return {'Date': now, 'Authorization': 'Basic %s' % base64.b64encode(bytes(auth, encoding="utf-8")).decode()}

# canonization function
def canon(method, host, path, params):
"""
Returns a list of the five-line request.
method, host, path: strings for request
params: dict of request parameters
"""

# create canonical string
now = email.utils.formatdate()
canon = [now, method.upper(), host.lower(), path]
args = []
for key in sorted(params.keys()):
val = params[key].encode("utf-8")
args.append(
'%s=%s' % (urllib.parse.quote(key, '~'), urllib.parse.quote(val, '~')))
canon.append('&'.join(args))

return canon

#POST Request
(method, host, path, params, skey, ikey) = sys.argv[1:]

method = method
host = host
path = path
params = json.loads(params)
skey = skey
ikey = ikey

c = canon(
method=method,
host=host,
path=path,
params=params
)

urlPath = c[-2]
pathParm = c[-1]

headRequest = signHeader(
method = method,
host = host,
path = path,
params = params,
skey=skey,
ikey=ikey)

headRequest['Host'] = host
headRequest['Content-Type'] = 'application/x-www-form-urlencoded'


url = f"https://{host}{urlPath}"

response = requests.request(method=method,
url=url,
headers=headRequest,
data= pathParm)

print(response.status_code)

TomML_2-1683673078342.png