Part 1- Cisco SecureX hosted Integration Modules and Serverless Relay on AWS

Troja007 · ‎07-21-2020

Latest news
Related Topics and other parts of the documentation
Introduction
Infrastructure Overview
Some Definitions of the used components
Configuring and Uploading the WebApp
Configuring the SecureX
Threat Hunt Data Flow
AWS Configuration - Foresight
AWS - Prepare your account and Serverless App - Introduction
AWS - Generate User
AWS - Generate Deployment Policy (ZappaLambdaDeploymentPolicy)
AWS - Generate Execution Policy (ZappaLAmbdaExecutionPolicy)
AWS - Generate Permission Role (tr-serverless-relay-ZappaLambdaExecutionRole)
AWS - Short Summary
Related Topics and other parts of the documentation
Command Line Cheat Sheet
Troubleshooting and FAQs

Latest news

Hello Cisco Community! Cisco Secure is moving forward with Cisco hosted SecureX Integration Modules, where no serverless relay or docker container is needed any more. To switch to the Cisco hosted version of your SecureX Integration Module, just follow these easy steps.

Login to SecureX and navigate to Integration Modules.
Click on Available Integration Modules and take a look at the modules including "Cisco hosted" in the name.
Remove the existing SecureX Integration Module.
Select the Cisto Hosted version of the module and click the New Module button. Enter the API Key from the 3rd party vendor.
Click Save.

Serverless environment

You can still use your existing configured serverless modules. Both configuration options are still available, AWS serverless configuration and Cisco hosted Modules.

If you are moving to the Cisco hosted version of the Integration Modules the steps below are not necessary.

Introduction

Relays for Cisco SecureX 3rd Party Modules enable communication between Cisco SecureX and 3rd Party vendors. The Serverless Template uses AWS as an example, that is very inexpensive and easy to deploy: https://github.com/CiscoSecurity/tr-05-serverless-relay

This Guide is separated into 3 parts
Part 1&2 just need to be done once.

Configuring the AWS environment - Part 1
Configuring your local environment to manage/configure the AWS apps - Part 2
Adding multiple Modules including JWT Token generation - Part 3

Which Modules are possible today?

Modules for Cisco Architecture:https://securex.us.security.cisco.com/help/modules
Third-party Modules:https://www.cisco.com/c/en/us/products/security/threat-response/partners-integrations.html#~methods

Infrastructure Overview

To install, configure and maintain the Serverless Relay, which is an App hosted by Amazon, we need some components. Enclosed a short description what is needed. This is an overview. The page will guide you step-by-step through the necessary steps to make your Serverless Relay up and running.

Serverless Architecture Overview

Some Definitions of the used components

Python on Endpoint: The serverless app can be configured locally on an endpoint using Python. You can use Windows, Linux or MacOS. The example here highlights a virtual python environment, which is enabled by a single command line statement. It generates a clean python environment. It is easy to use, configure it, start it and stop it when not needed any more.
Zappa: The Relay in AWS itself is an application, that can be easily packaged and deployed as an AWS Lambda Function on AWS using Zappa (locally on your endpoint).
GitHub: all Source Code is hosted on GitHub. You do not need any tools to copy code. All code will all downloaded by using simple command line statements.
AWS: You will need an Amazon Account. The Serverless Relay needs some components hosted by AWS, so it can run. An API to upload your configuration, the virtual APP itself, and a S3 bucket to store your files. The Amazon Apps are translating the information between CTR and 3rd Party Integrations, and vice versa.S WebApp is more or less your personal feature extension for SecureX/Threat Response
The AWS WebApp is more or less your personal feature extension for SecureX/Threat Response.
SecureX: The area where you configure the 3rd Party Modules.
3rd Party Modules: Most vendors are providing a free version of the API, which is limited by the amount of queries. So for testing, you can use a free account which is completely free of charge.

Configuring and Uploading the WebApp

After you configured the Configuration of the WebApp locally on your endpoint, there is a simple command to upload or update your WebApp on AWS.

Configuring the SecureX

During the configuration of the Module (AWS WebApp) locally on your endpoint, all needed Authentication Tokens and URLs are generated, which are used to configure your 3rd Party Module in SecureX.

Threat Hunt Data Flow

Why we need the AWS environment? After your module is configured, just start an investigation. The UI shows which Modules have been queried.

During an Investigation, Observables should be checked by 3rd Party Modules. The Threat Intelligence Model, which is a key component for the integrations, checks the data format.
CTR forwards the request to the Serverless App in AWS, providing credentials (authentication token).
The Serverless App translates the CTR request into the right format to query the 3rd Party vendor API, providing credentials (authentication token).
The same for the Response, the AWS Serverless App translates the Response into the right Data Format for CTR.
The response is forwarded from AWS to CTR. The Threat Intelligence Model checks the data format, so it fits to be shown in the Threat Response UI.

AWS Configuration - Foresight

In this section we will Guide you to prepare an AWS environment used to provide the virtual apps for your SecureX Modules. We will do just a few steps.

Generating a Single User
Adding two Policies in IAM
Adding a Role in IAM

AWS - Prepare your account and Serverless App - Introduction

Note, this Step-by-Step Guide shows how to generate an AWS Account for Serverless Applications. Keep in mind, if you have already other components configured in AWS, your Screen may look like a little bit different than the screenshots shown in this guide.

The next Steps will guide you through the implementation of the AWS environment, which acts as a "Service Proxy" between Cisco SecureX/CTR Modules and 3rd Party Modules.

Benefit of this approach are:

Cisco provides Templates for Policies and Roles.
The configuration templates for Policies and Roles are very restrictive, so it is very secure.
The AWS environment is designed to host multiple Serverless Apps, so you can integrate several 3rd Party Modules with this single AWS instance.

AWS Environment

Configuring the AWS environment is necessary one time. You can use the Server Relay for several apps. All the code (Policy Templates as .JSON File) on GitHub is already prepared, so you easily copy/paste the .JSON code, instead of manually generating the policies. This makes it really simple to configure your AWS environment.

Keep in mind, some naming are mandatory!

AWS - Generate User

Note, this Step-by-Step Guide shows how to generate an AWS Account for Serverless Applications. Keep in mind,
if you have already active users configure, your Screen may be look like a little bit different.

Step: If you do not have an Amazon account, generate a new one: https://aws.amazon.com/.
For a new account, just follow the Amazon AWS guideline.
Register to AWS
Step: Login to the AWS Management Console
Step: There are many Services shown here. Look for the Identity and Access Management (IAM) Console as shown in the
Screenshot.
Select IAM Service
Step: On the left context menu under Access Management select users. Afterwards click the Add users button to create a
new user.
Add new AWS user

Step: Creating the User:

AWS NEW User Object Step1: Type the username serverless
Under Select AWS access type select Programmatic access.
Set name and access type
AWS NEW User Object Step2: Click the Next:Permissions button
AWS NEW User Object Step3: Select the setting as outlined in the Screenshot. If there are already Groups available,
do not select any group here.
No permissions in this step
AWS NEW User Object Step4: Do not add any TAG.
no TAG
AWS NEW User Object Step5: Ignore the Message where your User does not have any permission, this is fine.
We will generate an own permission set for the user in the next steps.
Safely ignore permission warning
AWS NEW User Object Step6: Click the Create User Button.

AWS NEW User Object Step7: The Last step shows the Summary and the downloadable CSV File. The File includes
your Access Key ID and the Secret Access Key. We will need this information later to generate all the authentication token.
Download credentials file

Keep the file in a secure way. It can downloaded only once. We need the Secret Key later to generate
the authentication tokens. If you are loosing this key, you may not be able to add additional modules.
OR, you may have to configure most of the configuration from beginning.

Rename the downloaded file from download.csv to serverless.csv.

Note: All documentation on Github uses the "serverless" naming. When testing around the first time, i recommend to use the same naming convention.

Generating local Access Token for AWS API Gateway. You can skip this step. Where Github shows a "generic" documentation,
this guide is step-by-step, so we will do this step later.

Spoiler

Once the user is created and the credentials are downloaded, the best way to store that data is to put it into your AWS credentials file usually located on ~/.aws/credentials (Linux and Mac) or %USERPROFILE%\.aws\credentials (Windows). So make sure to add the user's credentials from the serverless.csv file to the AWS credentials file (manually create an empty one if missing) as a separate profile:

Once the user is created and the credentials are downloaded, the best way to store that data is to put it into your AWS credentials file usually located on ~/.aws/credentials (Linux and Mac) or %USERPROFILE%\.aws\credentials (Windows). So make sure to add the user's credentials from the serverless.csv file to the AWS credentials file (manually create an empty one if missing) as a separate profile:

The settings above will be needed for your Python environment when uploading a Web App to AWS to authenticate to the
AWS API gateway.

Configure Policies and Role

AWS - Generate Deployment Policy (ZappaLambdaDeploymentPolicy)

In this step we are generating a Deployment Policy as outlined https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/HOWTO.md.
This Policy permits an AWS user to install a Web Application in AWS. Cisco already provides the Policy as a .JSON File.
So you can just use Copy/Paste to generate the policy: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/ZappaLambdaDeploymentPolicy.json.

GitHub Documentation.

Step: AWS Deployment Policy: Copy/Paste the content into a Text file: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/ZappaLambdaDeploymentPolicy.json
Before copying the policy to AWS, you have to change the Account ID in the .JSON File (shown in Row 14):
Replace <ACCOUNT_ID> with your real Account ID, so finally the line looks like this: "arn:aws:iam::123456789012x:role/*ZappaLambdaExecutionRole"

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:AttachRolePolicy",
                "iam:GetRole",
                "iam:CreateRole",
                "iam:PassRole",
                "iam:PutRolePolicy"
            ],
            "Resource": [
                "arn:aws:iam::<ACCOUNT_ID>:role/*ZappaLambdaExecutionRole"
            ]
        },
        ######################################################
        ### Change the <ACCOUNT_ID> to your AWS Account ID ###
        ######################################################

Note: Note: Please use the latest code from GitHub..

Step: AWS Deployment Policy: Go to the Identity and Access Management (IAM) console as outlined above.
Step: AWS Deployment Policy: Start to generate a new policy by clicking the Create Policy Button.
Create new policy
Step: AWS Deployment Policy: Switch to the JSON view as shown in the Screenshot. Paste your updated .JSON File into the window. After you copied the .JSON File into the Window click the Review Button.
Change s3_bucket definition
Step: AWS Deployment Policy: Name the Policy ZappaLambdaDeploymentPolicy (so we have the same naming than on
GitHub) and add a Description. Click the Create Policy Button.
Deployment Policy naming
Step: AWS Deployment Policy: When all fine, the Policy is created as shown below.
Policy created
Step: AWS Deployment Policy: Add the new created policy the the user serverless. In the Identity and Access Management
(IAM) console under Access Management click users. Afterwards click the serverless user on the right.
Step: AWS Deployment Policy: Click the "Add permissions" button. Select the ZappaLambdaDeploymentPolicy Policy as
shown in the Screenshot.
Attach the ZappaLambdaDeploymentPolicy to Role
Step: AWS Deployment Policy: Click the "Next Review" button.
Step: AWS Deployment Policy: Click the "Add Permission" button.

AWS - Generate Execution Policy (ZappaLAmbdaExecutionPolicy)

In this step we are generating an execution policy. This policy includes the smallest set of permissions. With this Policy, the user is
allowed to use the SecureX Relay only: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/HOWTO.md

Step: AWS Execution Policy: Start to generate a new Policy as already described in the Step above.

Step: AWS Execution Policy: The Execution Policy is available on GitHub, so you can just copy/past the whole policy in

a .JSON format: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/ZappaLambdaExecutionPolicy.json
The provided code can be directly copied into the AWS configuration UI.

## Always use the latest code from Github ##
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Copy policy as JSON

Step: AWS Execution Policy: Click the "Review Policy" Button.
Step: AWS Execution Policy: Name your policy ZappaLambdaExecutionPolicy. Optional add a Description for your policy.
Name the policy object
Step: AWS Execution Policy: Click the "Create Policy" Button.
Step: AWS Execution Policy: Result
Execution policy result

AWS - Generate Permission Role (tr-serverless-relay-ZappaLambdaExecutionRole)

In this step we are generating a Permission Role for the account. You can find details a the GitHub Documentation why this Role is useful: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/HOWTO.md

Info from Github:

Spoiler

The default IAM policy created by Zappa for executing Lambdas is very permissive. It grants access to all actions for all resources for types CloudWatch, S3, Kinesis, SNS, SQS, DynamoDB, and Route53; lambda:InvokeFunction for all Lambda resources; Put to all X-Ray resources; and all Network Interface operations to all EC2 resources. While this allows most Lambdas to work correctly with no extra permissions, it is generally not an acceptable set of permissions for most continuous integration pipelines or production deployments. Instead, you will probably want to manually manage your IAM policies.

That is why in the Zappa Settings the manage_roles setting is set to false. Also, notice the role_name setting, it makes Zappa look for a custom IAM role named tr-serverless-relay-ZappaLambdaExecutionRole. The role will be automatically attached to your Lambda by Zappa. Moreover, once you have created the role, you will be able to re-use it for any future Lambdas.

NOTE. After having properly configured your serverless user, Zappa must be able to attach such roles (i.e. with the ZappaLambdaExecutionRole suffix) to any of your Lambdas on behalf of serverless. Again, you may check the Deployment Policy one more time to figure out where that permission comes from. Hint: remember the line where you must have already substituted some placeholder with your AWS account ID.

The default IAM policy created by Zappa for executing Lambdas is very permissive. It grants access to all actions for all resources for types CloudWatch, S3, Kinesis, SNS, SQS, DynamoDB, and Route53; lambda:InvokeFunction for all Lambda resources; Put to all X-Ray resources; and all Network Interface operations to all EC2 resources. While this allows most Lambdas to work correctly with no extra permissions, it is generally not an acceptable set of permissions for most continuous integration pipelines or production deployments. Instead, you will probably want to manually manage your IAM policies. That is why in the Zappa Settings the manage_roles setting is set to false. Also, notice the role_name setting, it makes Zappa look for a custom IAM role named tr-serverless-relay-ZappaLambdaExecutionRole. The role will be automatically attached to your Lambda by Zappa. Moreover, once you have created the role, you will be able to re-use it for any future Lambdas. NOTE. After having properly configured your serverless user, Zappa must be able to attach such roles (i.e. with the ZappaLambdaExecutionRole suffix) to any of your Lambdas on behalf of serverless. Again, you may check the Deployment Policy one more time to figure out where that permission comes from. Hint: remember the line where you must have already substituted some placeholder with your AWS account ID.

Note: Using the name tr-serverless-relay-ZappaLambdaExecutionRole is essential. Using a different naming will break functionality.

Step: AWS Execution Role: In the Identity and Access Management (IAM) console under Access Management click roles.
Create new role
Step: AWS Execution Role: Select the AWS service tab and choose the Lambda service. If not already listed under
"Common Use cases", just select Lambda from the List of services.
Select the Lambda Service
Step: AWS Execution Role: Click the "Next:Permissions" Button
Step: AWS Execution Role: Search for the ZappaLambdaExecutionPolicy policy and enable it via the corresponding check-box.
Select the execution policy
Step: AWS Execution Role: Click the "Next: Tags" Button. Do not assign any Tag.
Step: AWS Execution Role: Click the "Next: Review" Button.
Step: AWS Execution Role: Provide the name tr-serverless-relay-ZappaLambdaExecutionRole for the Role.

Note: Using the name tr-serverless-relay-ZappaLambdaExecutionRole is essential!

The name for the role is essential
Step: AWS Execution Role: Click the "Create role" Button.
Step: AWS Execution Role: Role was created successfully
Role created
Step: AWS Execution Role: Open the newly generated Execution Role, select the Trust relationships tab.
Change Trust Relations for role
Step: AWS Execution Role: Click the "Edit trust relationship" Button.

Step: AWS Execution Role: By default, the Service field will be equal to lambda.amazonaws.com.
Add a value to a list: apigateway.amazonaws.com.

## Before Change ##
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

## After Change ##
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "apigateway.amazonaws.com",
          "lambda.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Info: Once again, when configuring new, take a look on GitHub if there was any recent change.

Change Trust Relations for role

Step: AWS Execution Role: Click the "Update Trust Policy" Button.

AWS - Short Summary

So what we have configured so far?

We generated a user, which is used for all operations.
We generated two Policies, a Deployment policy for updating the Server app. And an Execution Policy with restrictive access rights.
We generated a Role for the user, which once again, helps to reduce access rights as much as possible.

Info: Configuring AWS must be done one time. Afterwards multiple Lambda Apps (3rd Party Modules) are hosted as Lambda functions.

Command Line Cheat Sheet

Start the virtual environment: source securex/bin/activate
Stop the virtual environment: deactivate

Troubleshooting and FAQs

Q: I´m getting an AWS related error message during uploading or configuring. What i can do?
A: Take a look at the AWS common error page: https://github.com/CiscoSecurity/tr-05-serverless-relay/blob/develop/aws/CommonErrors.md

Q: I´m getting an error message that zappa is not available.
A: Take a look if your python virtual environment is running.

Q: I´m getting an error message that the local file cannot be packed.
A: Switch to root in your Terminal Windows using e.g. sudo su.

Q: I removed an application from AWS, no i cannot deploy it any more.
A: Take a look if the s3_bucket is still available on AWS. In addition try to use zappa update dev.

DanielMidgley1568 · ‎09-27-2022

This is great. Really helpful! Is there a specific reason to use Zappa over something more widely used like Terraform?

Troja007 · ‎09-27-2022

Hello @DanielMidgley1568 , two things I want to mention here.

It is an example how to deploy the relay module
We already host the relay modules in SecureX. You just need to configure in SecureX

If you want to host your own modules somewhere in the cloud, feel free
Greetings, Thorsten