Part 2- Cisco SecureX hosted Integration Modules and Serverless Relay on AWS

Troja007 · ‎07-21-2020

Related Topics
Prepare Configuration Environment on your Endpoint
Related Topics
Command Line Cheat Sheet

NEWS: Cisco hosted SecureX Integration Modules

Cisco Secure is moving forward with Cisco hosted SecureX Integration Modules, where you do not need Serverless or docker any more. Find details in Part 1 of the documentation.

If you are moving to the Cisco hosted version of the Integration Modules the steps below are not necessary.

Prepare Configuration Environment on your Endpoint

The next section guides you through the configuration of your endpoint environment to do the following steps.

Virtual clean python environment
Downloading Code from Github
Configuring the WebApp locally on your endpoint
Generating the authentication tokens
Uploading the Web App to AWS

The virtual python environment on your endpoint is used to configure a Web app and upload it to AWS Cloud. Enabling the virtual environment is just one single command line statement. You can setup the environment on Windows, Linux or MacOS. The whole environment is based on python/zappa, which is cross-platform.

On Linux/MAC there is often already Python preinstalled on the system, which is mostly an older version. This is fine, there is no need to remove the old version. As you see in the Code Block, Python 2.7.16 is pre-install, Python 3.x is not installed.

admin@MacOS-wkst1 ~ % python --version
Python 2.7.16
admin@MacOS-wkst1 ~ %
admin@MacOS-wkst1 ~ % python3 --version
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.
admin@MacOS-wkst1 ~ %

Step: Endpoint Environment: First we need Python on our system. The easiest way is to download Python 3.7 directly from the phyton.org website. You can also use the command line. There are many examples available in the internet how to install python3. For this guide we used Python 3.7.x.
1. Installing Python on Linux/MacOS:
  1. Download the package an install the package.
  2. After installation check the version.
```
admin@MacOS-wkst1 ~ % python3 --version
Python 3.7.3
admin@MacOS-wkst1 ~ 
```
Step. Endpoint Environment: in my case i also installed the AWS CLI, which makes some steps easier for us. You can download and install the CLI from https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html.
1. Check AWS installation:
  1. Linux/MacOS default path is /usr/local/aws-cli.

Step: Endpoint Environment: Install the Python virtual environment as shown on GitHub: https://github.com/CiscoSecurity/tr-05-serverless-relay#step-1-requirements-installation.
You can choose the directory you want to use for the environment. The environment uses a single directory where all files are stored. This directory is "mounted" as your python virtual environment. Really easy!

In the Linux/MacOS Terminal type the following commands.

Switch to sudo right: sudo su
Note: Always use elevated rights during the virtual python environment configuration and usage.
Go to the root of the disk: cd /
Go to the usr directory: cd /usr/local/
Generate a folder for the virtual environment: mkdir SecureX
Switch into this directory and check this: pwd
The output should be: /usr/local/SecureX
Generate the Python virtual environment: python3 -m venv securex
In this scenario we are naming the virtual environment securex
1. python: starting phyton
2. -m venv: generating a virtual environment
3. securex: the name of the virtual environment

Start the virtual environment: source securex/bin/activate

admin@MacOS-wkst1 SecureX % pwd
/usr/local/SecureX
admin@MacOS-wkst1 SecureX % sudo su
Password:
sh-3.2# source securex/bin/activate
(securex) sh-3.2#

In the next Step we are cloning some files directly from Github to your disk. It also includes a requirements.txt file we are using to install the components we need. To do so, just copy/past the following command into your Terminal window: git clone https://github.com/CiscoSecurity/tr-05-serverless-relay.git

(securex) sh-3.2# pwd
/usr/local/SecureX
(securex) sh-3.2# git clone https://github.com/CiscoSecurity/tr-05-serverless-relay.git
Cloning into 'tr-05-serverless-relay'...
remote: Enumerating objects: 24, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 120 (delta 3), reused 15 (delta 3), pack-reused 96
Receiving objects: 100% (120/120), 36.28 KiB | 807.00 KiB/s, done.
Resolving deltas: 100% (36/36), done.
(securex) sh-3.2#

Switch into the new generated folder: cd /usr/local/SecureX/tr-05-serverless-relay/

Now you can easily install the needed libraries to the disk as outlined on Github: pip install --upgrade --requirement requirements.txt

(securex) sh-3.2# pwd
/usr/local/SecureX/tr-05-serverless-relay
(securex) sh-3.2# pip install --upgrade --requirement requirements.txt
Collecting Authlib==0.14.3
  Downloading Authlib-0.14.3-py2.py3-none-any.whl (215 kB)
     |████████████████████████████████| 215 kB 714 kB/s
Collecting Flask==1.1.2
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
.
## removed approx. 100 lines ##
.
Successfully installed Authlib-0.14.3 Flask-1.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 PyYAML-5.3.1 Werkzeug-1.0.1 argcomplete-1.12.0 boto3-1.14.21 botocore-1.17.21 certifi-2020.6.20 cffi-1.14.0 cfn-flip-1.2.3 chardet-3.0.4 click-7.1.2 cryptography-2.9.2 docutils-0.15.2 durationpy-0.5 future-0.18.2 hjson-3.0.1 idna-2.10 importlib-metadata-1.7.0 itsdangerous-1.1.0 jmespath-0.10.0 kappa-0.6.0 marshmallow-3.6.1 pip-tools-5.2.1 placebo-0.9.0 pycparser-2.20 python-dateutil-2.6.1 python-slugify-4.0.1 requests-2.24.0 s3transfer-0.3.3 six-1.15.0 text-unidecode-1.3 toml-0.10.1 tqdm-4.47.0 troposphere-2.6.2 urllib3-1.25.9 wheel-0.34.2 wsgi-request-logger-0.4.6 zappa-0.51.0 zipp-3.1.0
(securex) sh-3.2#

Checking/Installing the right Phyton Werkzeug tool (Optional).
Note: Python Werkzeug version 0.16.0 is mandatory. Just to be sure the right version is installed in your environment.

Check the version: pip list
One of the last entries shows the Werkzeug Tool and version.

If not showing version 0.16.0, install the right version with the following command: pip install werkzeug==0.16.0

(securex) sh-3.2# pip install werkzeug==0.16.0
Collecting werkzeug==0.16.0
  Downloading Werkzeug-0.16.0-py2.py3-none-any.whl (327 kB)
     |████████████████████████████████| 327 kB 743 kB/s
Installing collected packages: werkzeug
  Attempting uninstall: werkzeug
    Found existing installation: Werkzeug 1.0.1
    Uninstalling Werkzeug-1.0.1:
      Successfully uninstalled Werkzeug-1.0.1
Successfully installed werkzeug-0.16.0
(securex) sh-3.2#

Info: Python Werkzeug 0.16.0 is mandatory!

Check again the version: pip list
It should now show version 0.16.0 for Werkzeug.

Step: Endpoint Environment: To authenticate to the AWS Gateway, we need to place a file on your disk including the AWS access key. As described on Github: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/HOWTO.md. We can use the AWS cli to do this for us.
1. Configure AWS credentials: aws configure. Enter the data as followed (from serverless.csv)
  - your AWS Access Key ID.
  - your AWS Secret Access Key.
  - the nearest AWS location (explained on Github)
  - json as the default output format
  
  Note: for this step we need the serverles.csv file we generated during the AWS user account creation.
```
(securex) sh-3.2# pwd
/usr/local/SecureX
(securex) sh-3.2# aws configure
AWS Access Key ID [None]: AKIAUENCFHXXXXXXXXXX
AWS Secret Access Key [None]: fY3GapfiedAi5VxZyEwK+G+E/+qTjuXXXXXXXXXX
Default region name [None]: us-east-1
Default output format [None]: json
(securex) sh-3.2#
```
2. Update the file, so it points to the right profile on AWS.
  1. Check if the file was successfully created: cat ~/.aws/credentials
```
(securex) sh-3.2# cat ~/.aws/credentials
[default]
aws_access_key_id = AKIAUENCFHXXXXXXXXXX
aws_secret_access_key = fY3GapfiedAi5VxZyEwK+G+E/+qTjuXXXXXXXXXX
(securex) sh-3.2#
```
  2. Now we have to update the file. Replace "default" with "serverless": vi ~/.aws/credentials
    After the update, the file should look like this: cat ~/.aws/credentials
```
(securex) sh-3.2# cat ~/.aws/credentials
[serverless]
aws_access_key_id = AKIAUENCFHXXXXXXXXXX
aws_secret_access_key = fY3GapfiedAi5VxZyEwK+G+E/+qTjuXXXXXXXXXX
(securex) sh-3.2#
```

Step: Endpoint Environment: Let us test the configuration! Before we can test, we need a small change in a configuration file called zappa_settings.json. The settings in this file will be explained in more detail in the next chapter.

switch into the tr-05-serverless-relay directory: cd /usr/local/SecureX/tr-05-serverless-relay/
start to modify the zappa_settings.json file: vi zappa_settings.json

change the line for the s3_bucket value but just adding your Account ID to the end. Best choise is to use your 12-digit AWS Account ID, because it is unique.
Before: "s3_bucket": "zappa-tr-template-relay"
After: "s3_bucket": "zappa-tr-template-relay-123456789012"

(securex) sh-3.2# cat zappa_settings.json
{
    "dev": {
        "app_function": "app.app",
        "aws_region": "us-east-1",
        "exclude": [".*", "*.json", "*.md", "*.txt"],
        "keep_warm": false,
        "log_level": "INFO",
        "manage_roles": false,
        "profile_name": "serverless",
        "project_name": "tr-template-relay",
        "role_name": "tr-serverless-relay-ZappaLambdaExecutionRole",
        "runtime": "python3.7",
        "s3_bucket": "zappa-tr-template-relay-123456789012"
    }
}
(securex) sh-3.2#

Note: ANY S3 Bucket in the whole AWS cloud must be unique.

As a test, upload the app to AWS to generate an empthy Lambda app.
Type this command into your Terminal: zappa deploy dev
- zappa: the local app packaging the app and uploading it to AWS
- deploy: argument for zappa do deploy an app
- dev: this is the "stage" pointing to the definition in the zappa_settings.json file.

Note: You must execute the command from the cd /usr/local/SecureX/tr-05-serverless-relay/ folder! At any time deploying an app for AWS it must be executed from the appropriate local app directory.

If all running fine, you should see a result as shown in the Terminal output.

(securex) sh-3.2# pwd
/usr/local/SecureX/tr-05-serverless-relay
(securex) sh-3.2# zappa deploy dev
Calling deploy for stage dev..
Downloading and installing dependencies..
 - markupsafe==1.1.1: Downloading
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 27.5k/27.5k [00:00<00:00, 570kB/s]
 - cryptography==2.9.2: Using locally cached manylinux wheel
 - cffi==1.14.0: Using locally cached manylinux wheel
Packaging project as zip.
Uploading tr-template-relay-dev-1594901383.zip (10.6MiB)..
.
## Removed Several Lines ##
.
Deploying API Gateway..
Deployment complete!: https://lp0ktyydsa.execute-api.us-east-1.amazonaws.com/dev
(securex) sh-3.2#

Step: Endpoint Environment: Now let´s do a final check. When anything went fine, and the app was uploaded to AWS, there should be a Lambda function available. To check this, do the following.
1. Login to AWS Console as already shown above.
2. Switch to the Lambda configuration page
3. Click the function tr-template-relay-dev.
  
  Note: There is no need to set the SECRET_KEY for this empty app. When adding additional Modules, this is the area to configure the AWS environment variables we need.

Step: Endpoint Environment: Summary: You now have an environment, where you can easily add, remove and update AWS apps used as a SecureX Relay.

Part 2- Cisco SecureX hosted Integration Modules and Serverless Relay on AWS

Related Topics

Prepare Configuration Environment on your Endpoint

Related Topics

Command Line Cheat Sheet