cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

Part 2- Cisco SecureX hosted Integration Modules and Serverless Relay on AWS

1000
Views
10
Helpful
0
Comments

 

Community_Approach_3.png

 

 

NEWS: Cisco hosted SecureX Integration Modules

Cisco Secure is moving forward with Cisco hosted SecureX Integration Modules, where you do not need Serverless or docker any more. Find details in Part 1 of the documentation.

If you are moving to the Cisco hosted version of the Integration Modules the steps below are not necessary.

 

Related Topics

  1. Serverless Relay on AWS for SecureX/CTR 3rd Party Modules - Part 1
  2. Serverless Relay on AWS for SecureX/CTR 3rd Party Modules - Part 2
  3. Serverless Relay on AWS for SecureX/CTR 3rd Party Modules - Part 3

 

Prepare Configuration Environment on your Endpoint

The next section guides you through the configuration of your endpoint environment to do the following steps.

  • Virtual clean python environment
  • Downloading Code from Github
  • Configuring the WebApp locally on your endpoint
  • Generating the authentication tokens
  • Uploading the Web App to AWS

The virtual python environment on your endpoint is used to configure a Web app and upload it to AWS Cloud. Enabling the virtual environment is just one single command line statement. You can setup the environment on Windows, Linux or MacOS. The whole environment is based on python/zappa, which is cross-platform. 


On Linux/MAC there is often already Python preinstalled on the system, which is mostly an older version. This is fine, there is no need to remove the old version. As you see in the Code Block, Python 2.7.16 is pre-install, Python 3.x is not installed.

admin@MacOS-wkst1 ~ % python --version
Python 2.7.16
admin@MacOS-wkst1 ~ %
admin@MacOS-wkst1 ~ % python3 --version
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.
admin@MacOS-wkst1 ~ %

 

 

  1. Step: Endpoint Environment: First we need Python on our system. The easiest way is to download Python 3.7 directly from the phyton.org website. You can also use the command line. There are many examples available in the internet how to install python3. For this guide we used Python 3.7.x.

    1. Installing Python on Linux/MacOS:
      1. Download the package an install the package.
      2. After installation check the version.

        admin@MacOS-wkst1 ~ % python3 --version
        Python 3.7.3
        admin@MacOS-wkst1 ~ 

  2. Step. Endpoint Environment: in my case i also installed the AWS CLI, which makes some steps easier for us. You can download and install the CLI from https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html.
    1. Check AWS installation: 
      1. Linux/MacOS default path is /usr/local/aws-cli.

  3. Step: Endpoint Environment: Install the Python virtual environment as shown on GitHub: https://github.com/CiscoSecurity/tr-05-serverless-relay#step-1-requirements-installation
    You can choose the directory you want to use for the environment. The environment uses a single directory where all files are stored. This directory is "mounted" as your python virtual environment. Really easy!
    1. In the Linux/MacOS Terminal type the following commands.
      1. Switch to sudo right: sudo su
        Note:
        Always use elevated rights during the virtual python environment configuration and usage.
      2. Go to the root of the disk: cd /
      3. Go to the usr directory: cd /usr/local/
      4. Generate a folder for the virtual environment: mkdir SecureX
      5. Switch into this directory and check this: pwd
        The output should be: /usr/local/SecureX
      6. Generate the Python virtual environment: python3 -m venv securex
        In this scenario we are naming the virtual environment securex
        1. python: starting phyton
        2. -m venv: generating a virtual environment
        3. securex: the name of the virtual environment

      7. Start the virtual environment: source securex/bin/activate

        admin@MacOS-wkst1 SecureX % pwd
        /usr/local/SecureX
        admin@MacOS-wkst1 SecureX % sudo su
        Password:
        sh-3.2# source securex/bin/activate
        (securex) sh-3.2#
      8. In the next Step we are cloning some files directly from Github to your disk. It also includes a requirements.txt file we are using to install the components we need. To do so, just copy/past the following command into your Terminal window: git clone https://github.com/CiscoSecurity/tr-05-serverless-relay.git

        (securex) sh-3.2# pwd
        /usr/local/SecureX
        (securex) sh-3.2# git clone https://github.com/CiscoSecurity/tr-05-serverless-relay.git
        Cloning into 'tr-05-serverless-relay'...
        remote: Enumerating objects: 24, done.
        remote: Counting objects: 100% (24/24), done.
        remote: Compressing objects: 100% (19/19), done.
        remote: Total 120 (delta 3), reused 15 (delta 3), pack-reused 96
        Receiving objects: 100% (120/120), 36.28 KiB | 807.00 KiB/s, done.
        Resolving deltas: 100% (36/36), done.
        (securex) sh-3.2#
      9. Switch into the new generated folder: cd /usr/local/SecureX/tr-05-serverless-relay/

      10. Now you can easily install the needed libraries to the disk as outlined on Githubpip install --upgrade --requirement requirements.txt

        (securex) sh-3.2# pwd
        /usr/local/SecureX/tr-05-serverless-relay
        (securex) sh-3.2# pip install --upgrade --requirement requirements.txt
        Collecting Authlib==0.14.3
          Downloading Authlib-0.14.3-py2.py3-none-any.whl (215 kB)
             |████████████████████████████████| 215 kB 714 kB/s
        Collecting Flask==1.1.2
          Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
        .
        ## removed approx. 100 lines ##
        .
        Successfully installed Authlib-0.14.3 Flask-1.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 PyYAML-5.3.1 Werkzeug-1.0.1 argcomplete-1.12.0 boto3-1.14.21 botocore-1.17.21 certifi-2020.6.20 cffi-1.14.0 cfn-flip-1.2.3 chardet-3.0.4 click-7.1.2 cryptography-2.9.2 docutils-0.15.2 durationpy-0.5 future-0.18.2 hjson-3.0.1 idna-2.10 importlib-metadata-1.7.0 itsdangerous-1.1.0 jmespath-0.10.0 kappa-0.6.0 marshmallow-3.6.1 pip-tools-5.2.1 placebo-0.9.0 pycparser-2.20 python-dateutil-2.6.1 python-slugify-4.0.1 requests-2.24.0 s3transfer-0.3.3 six-1.15.0 text-unidecode-1.3 toml-0.10.1 tqdm-4.47.0 troposphere-2.6.2 urllib3-1.25.9 wheel-0.34.2 wsgi-request-logger-0.4.6 zappa-0.51.0 zipp-3.1.0
        (securex) sh-3.2#

         

      11. Checking/Installing the right Phyton Werkzeug tool (Optional).
        Note: Python Werkzeug version 0.16.0 is mandatory. Just to be sure the right version is installed in your environment.
         

        1. Check the version: pip list
          One of the last entries shows the Werkzeug Tool and version.

        2. If not showing version 0.16.0, install the right version with the following command: pip install werkzeug==0.16.0

          (securex) sh-3.2# pip install werkzeug==0.16.0
          Collecting werkzeug==0.16.0
            Downloading Werkzeug-0.16.0-py2.py3-none-any.whl (327 kB)
               |████████████████████████████████| 327 kB 743 kB/s
          Installing collected packages: werkzeug
            Attempting uninstall: werkzeug
              Found existing installation: Werkzeug 1.0.1
              Uninstalling Werkzeug-1.0.1:
                Successfully uninstalled Werkzeug-1.0.1
          Successfully installed werkzeug-0.16.0
          (securex) sh-3.2#

          Info: Python Werkzeug 0.16.0 is mandatory!

           

        3. Check again the version: pip list
          It should now show version 0.16.0 for Werkzeug.

         

  4. Step: Endpoint Environment: To authenticate to the AWS Gateway, we need to place a file on your disk including the AWS access key. As described on Githubhttps://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/HOWTO.md. We can use the AWS cli to do this for us.

    1. Configure AWS credentials: aws configure. Enter the data as followed (from serverless.csv)
       - your AWS Access Key ID.
       - your AWS Secret Access Key.
       - the nearest AWS location (explained on Github)
       - json as the default output format 

       

      Note: for this step we need the serverles.csv file we generated during the AWS user account creation.

       

      (securex) sh-3.2# pwd
      /usr/local/SecureX
      (securex) sh-3.2# aws configure
      AWS Access Key ID [None]: AKIAUENCFHXXXXXXXXXX
      AWS Secret Access Key [None]: fY3GapfiedAi5VxZyEwK+G+E/+qTjuXXXXXXXXXX
      Default region name [None]: us-east-1
      Default output format [None]: json
      (securex) sh-3.2#

       

       

    2. Update the file, so it points to the right profile on AWS.

      1. Check if the file was successfully created: cat ~/.aws/credentials

        (securex) sh-3.2# cat ~/.aws/credentials
        [default]
        aws_access_key_id = AKIAUENCFHXXXXXXXXXX
        aws_secret_access_key = fY3GapfiedAi5VxZyEwK+G+E/+qTjuXXXXXXXXXX
        (securex) sh-3.2#

         

      2. Now we have to update the file. Replace "default" with "serverless": vi ~/.aws/credentials
        After the update, the file should look like this: cat ~/.aws/credentials

        (securex) sh-3.2# cat ~/.aws/credentials
        [serverless]
        aws_access_key_id = AKIAUENCFHXXXXXXXXXX
        aws_secret_access_key = fY3GapfiedAi5VxZyEwK+G+E/+qTjuXXXXXXXXXX
        (securex) sh-3.2#

     

  5. Step: Endpoint Environment: Let us test the configuration! Before we can test, we need a small change in a configuration file called zappa_settings.json. The settings in this file will be explained in more detail in the next chapter.
    1. switch into the tr-05-serverless-relay directory: cd /usr/local/SecureX/tr-05-serverless-relay/
    2. start to modify the zappa_settings.json file: vi zappa_settings.json
    3. change the line for the s3_bucket value but just adding your Account ID to the end. Best choise is to use your 12-digit AWS Account ID, because it is unique.
      Before: "s3_bucket": "zappa-tr-template-relay"
      After: "s3_bucket": "zappa-tr-template-relay-123456789012"

      (securex) sh-3.2# cat zappa_settings.json
      {
          "dev": {
              "app_function": "app.app",
              "aws_region": "us-east-1",
              "exclude": [".*", "*.json", "*.md", "*.txt"],
              "keep_warm": false,
              "log_level": "INFO",
              "manage_roles": false,
              "profile_name": "serverless",
              "project_name": "tr-template-relay",
              "role_name": "tr-serverless-relay-ZappaLambdaExecutionRole",
              "runtime": "python3.7",
              "s3_bucket": "zappa-tr-template-relay-123456789012"
          }
      }
      (securex) sh-3.2#

      Note: ANY S3 Bucket in the whole AWS cloud must be unique.

    4. As a test, upload the app to AWS to generate an empthy Lambda app.
      Type this command into your Terminal: zappa deploy dev
      - zappa: the local app packaging the app and uploading it to AWS
      - deploy: argument for zappa do deploy an app
      - dev: this is the "stage" pointing to the definition in the zappa_settings.json file.

      Note: You must execute the command from the cd /usr/local/SecureX/tr-05-serverless-relay/ folder! At any time deploying an app for AWS it must be executed from the appropriate local app directory.

       
      If all running fine, you should see a result as shown in the Terminal output.

      (securex) sh-3.2# pwd
      /usr/local/SecureX/tr-05-serverless-relay
      (securex) sh-3.2# zappa deploy dev
      Calling deploy for stage dev..
      Downloading and installing dependencies..
       - markupsafe==1.1.1: Downloading
      100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 27.5k/27.5k [00:00<00:00, 570kB/s]
       - cryptography==2.9.2: Using locally cached manylinux wheel
       - cffi==1.14.0: Using locally cached manylinux wheel
      Packaging project as zip.
      Uploading tr-template-relay-dev-1594901383.zip (10.6MiB)..
      .
      ## Removed Several Lines ##
      .
      Deploying API Gateway..
      Deployment complete!: https://lp0ktyydsa.execute-api.us-east-1.amazonaws.com/dev
      (securex) sh-3.2#



  6. Step: Endpoint Environment: Now let´s do a final check. When anything went fine, and the app was uploaded to AWS, there should be a Lambda function available. To check this, do the following.
    1. Login to AWS Console as already shown above.
    2. Switch to the Lambda configuration page
      Lambda_Function_1.png


    3. Click the function tr-template-relay-dev.
      Lambda_Function_2.png


      Note: There is no need to set the SECRET_KEY for this empty app. When adding additional Modules, this is the area to configure the AWS environment variables we need.

  7. Step: Endpoint Environment: Summary: You now have an environment, where you can easily add, remove and update AWS apps used as a SecureX Relay.

    Hint: If you cannot remember the URL for the Web App (used to configure the CTR module) just
    type zappa status dev in the Terminal. Wait a few seconds until the result is shown.
    It includes the API Gateway URL necessary for CTR.

     

 

Related Topics

  1. Serverless Relay on AWS for SecureX/CTR 3rd Party Modules - Part 1
  2. Serverless Relay on AWS for SecureX/CTR 3rd Party Modules - Part 2
  3. Serverless Relay on AWS for SecureX/CTR 3rd Party Modules - Part 3

 

Command Line Cheat Sheet

  • Start the virtual environment: source securex/bin/activate
  • Stop the virtual environment: deactivate