Myrmex Logo

Getting started

Configuring AWS

Prerequisites

To work with Myrmex and its main plugins, you should to have some notions about:

AWS IAM
AWS Lambda
Amazon API Gateway
OpenAPI (aka Swagger) specification
Node.js

Myrmex aims to experiment quickly, but to really be efficient, it is necessary to know the AWS services that works with Myrmex.

Permissions needed to deploy the project in AWS

Myrmex plugins that manage AWS resources need to have permissions to execute commands that communicate with the AWS API.

To begin to work with Myrmex, the following policies can be used:

IAMFullAccess
AWSLambdaFullAccess
AmazonAPIGatewayAdministrator

The AWS credentials can be set following the documentation from the AWS SDK for Node.js

Installation

Prerequisites: node (version 4 minimum) must be installed.

Installing the Myrmex command line

The myrmex command line has its own npm module. Install it globally.

npm install -g myrmex

We can check that the myrmex command line is correctly installed.

# This command will show the available options and sub-commands
myrmex -h

At this point, there is only one sub-command available: myrmex new. When creating a project, other subcommands will be provided by Myrmex plugins. For every subcommand, every option that is not provided via the command line will provide a prompt to set its value. This way, every Myrmex command can be scripted and is completely interactive at the same time.

Creating a new project

myrmex new is the command that creates new projects. If it is called without argument and/or option, it will provide a prompt to define the configuration of the project.

# Using the command without option and argument,
# the user will be prompted to give information about the project configuration
myrmex new

The prompt will ask to choose a project name and to select the plugins to install among @myrmex/iam, @myrmex/lambda and @myrmex/api-gateway. The plugin selection depend on the project. It is easy to install a plugin later if it is not done at this step (see next section).

myrmex new will perform 3 operations:

create a project folder
install the latest version of Myrmex and the project's plugins
create a myrmex.json configuration file

Once the project is created we can enter its folder. Using myrmex -h we can see that new myrmex sub-commands are available. They are provided by plugins that have been installed in the project.

Installing plugins

myrmex new propose a selection of plugins to install but it is possible to install them or other plugins separately. Myrmex plugins are npm packages. So the following command will install the plugin @myrmex/api-gateway:

npm install @myrmex/api-gateway --save-dev

To be loaded by Myrmex, the plugin needs to be referenced in the myrmex.json file, in the plugins section:

{
  "name": "my-serverless-application",
  "plugins": [
    "@myrmex/api-gateway"
  ]
}

It is possible to disable a plugin by removing it from the myrmex.json file.

Project configuration

Myrmex can be configured in different ways.

The `myrmex.json` file

In the root directory of a Myrmex project, there is a myrmex.json file that contains the configuration of the project.

A myrmex.json file looks like this:

{
  "name": "my-serverless-application",
  "plugins": [
    "@myrmex/iam",
    "@myrmex/api-gateway",
    "@myrmex/lambda"
  ],
  "config": {
    "myConfigKey": "my-config-value",
    "myNestedConfig": {
      "level1": 42,
      "level2": {
        "key1": true,
        "key2": false
      }
    }
  }
}

name: the identifier of the project.
plugins: the list of plugins that Myrmex has to register. A Myrmex plugin can be a node module installed via npm or defined in the project. A plugin installed via npm but specified here will not be registered by Myrmex.
config: the configuration of the project. Each plugin can use the it and inject default values in it.

The command myrmex show-config allows to see the current configuration of a project. This is the response of this command for the previous myrmex.json file.

  "myConfigKey": "my-config-value",
  "myNestedConfig": {
    "level1": 42,
    "level2": {
      "key1": true,
      "key2": false
    }
  },
  "iam": {
    "policiesPath": "iam/policies",
    "rolesPath": "iam/roles"
  },
  "apiGateway": {
    "apisPath": "api-gateway/apis",
    "endpointsPath": "api-gateway/endpoints",
    "modelsPath": "api-gateway/models"
  },
  "lambda": {
    "lambdasPath": "lambda/lambdas",
    "modulesPath": "lambda/modules"
  }
}

This example shows that the three plugins injected the configuration to define the location of the content that they manage. It is possible to override this configuration in the myrmex.json file.

Splitting the configuration into various files

If a project has a many configuration keys, it is possible to split the configuration object into various files in a folder named config.

For the previous example, to define the same configuration that in the previous example, the directory structure will look like this:

project-root
├── myrmex.json
└── config
    └── myNestedConfig.json

The myrmex.json file:

{
  "name": "planet-express",
  "plugins": [
    "@myrmex/iam",
    "@myrmex/api-gateway",
    "@myrmex/lambda"
  ],
  "config": {
    "myConfigKey": "my-config-value"
  }
}

The file config/myNestedConfig.json:

{
  "level1": 42,
  "level2": {
    "key1": true,
    "key2": false
  }
}

The command myrmex show-config will give the same response:

  "myConfigKey": "my-config-value",
  "myNestedConfig": {
    "level1": 42,
    "level2": {
      "key1": true,
      "key2": false
    }
  },
  "iam": {
    "policiesPath": "iam/policies",
    "rolesPath": "iam/roles"
  },
  "apiGateway": {
    "apisPath": "api-gateway/apis",
    "endpointsPath": "api-gateway/endpoints",
    "modelsPath": "api-gateway/models"
  },
  "lambda": {
    "lambdasPath": "lambda/lambdas",
    "modulesPath": "lambda/modules"
  }
}

Using environment variables to override the configuration

It is also possible to define configuration via environment variables. Any environment variable that starts with MYRMEX_ is injected as a configuration key/value. An underscore is used to resolve nested keys.

Because _ is used to resolve nested key, this technique cannot work with configuration keys containing a _.

For example, the configuration of myNestedConfig.level1 can be overrided using the environment variable MYRMEX_myNestedConfig_level1:

export MYRMEX_myNestedConfig_level1=43
myrmex show-config

Result:

{
  "myConfigKey": "my-config-value",
  "myNestedConfig": {
    "level1": "43",
    "level2": {
      "key1": true,
      "key2": false
    }
  },
  "iam": {
    "policiesPath": "iam/policies",
    "rolesPath": "iam/roles"
  },
  "apiGateway": {
    "apisPath": "api-gateway/apis",
    "endpointsPath": "api-gateway/endpoints",
    "modelsPath": "api-gateway/models"
  },
  "lambda": {
    "lambdasPath": "lambda/lambdas",
    "modulesPath": "lambda/modules"
  }
}

Configuration values defined via an environment variable are always of type String

Defining the configuration programmatically

All json configuration files in Myrmex are loaded using the Node.js require() function. This means that they all can be replaced by a javascript file that exposes the desired configuration.

For example, the previous myrmex.json file can be replaced by this myrmex.js file:

module.exports = {
  name: 'planet-express',
  plugins: [
    '@myrmex/iam',
    '@myrmex/api-gateway',
    '@myrmex/lambda'
  ],
  config: {
    myConfigKey: process.env.MY_ENVIRONMENT_VARIABLE || 'my-default-value'
  }
}

IAM plugin

The @myrmex/iam plugin allows to define and deploy AWS policies and roles. It can be used to avoid referencing IAM roles ARNs and simplify the configuration of other plugins.

Prerequisites

To use the @myrmex/iam plugin, it is necessary to have a minimal knowledge about AWS Identity and Access Management.

An AWS user or role that uses the plugin @myrmex/iam must have access to IAM administration. The AWS policy IAMFullAccess gives all necessary permissions.

Installation

Install the npm module in a Myrmex project:

npm install @myrmex/iam --save-dev

Then enable the plugin in the myrmex.json file:

{
  "name": "my-app",
  "plugins": [
    "@myrmex/iam"
  ]
}

Commands

create-policy

Usage:

create-policy [identifier]

Create a new IAM policy in iam/policies/<identifier>.json. The command does not configure the policy but just prepare the file in the right place. By default it is iam/policies/<identifier>.json.

create-role

Usage:

create-role [options] [identifier]

  Options:
    -m, --model <model-identifier>       select a model to quickly create the role configuration
    -p, --policies <policy-identifiers>  select the policies to attach to the role

Create a new IAM role. By default the location of roles is iam/roles/<identifier>.json.

Two predefined role configuration are available:

APIGatewayLambdaInvocation: The AWS policy AWSLambdaRole is attached to it and it can be associated to API Gateway endpoints.
LambdaBasicExecutionRole: The AWS policy AWSLambdaBasicExecutionRole is attached to it and it can be associated to Lambda functions.

It is possible to define custom policies and trust-relationship configuration. Is is also possible to reference policies from the Myrmex project.

Roles can be referenced by other plugins to facilitate the deployment of Myrmex applications: by defining roles in the Myrmex project, deploying some resources like Lambda functions or API definitions can be done on various environments without needing to define specific configuration to reference different roles ARN.

deploy-policies

Usage:

deploy-policies [options] [policy-identifiers]

  Options:
    -e, --environment [environment]  An environment identifier that will be used as a prefix
    -s, --stage [stage]              A stage identifier that will be used as a suffix

Deploy one or more policies in AWS. The --environment option is used as a prefix and the --stage option as a suffix.

deploy-roles

Usage:

deploy-roles [options] [role-identifiers]

  Options:
    -e, --environment [environment]  An environment identifier that will be used as a prefix
    -s, --stage [stage]              A stage identifier that will be used as a suffix

Deploy one or more roles in AWS. The environment option is used as a prefix and the stage option as a suffix.

Definition of IAM policies

The policies managed by Myrmex are nothing more than IAM policy definitions stored as JSON file. This allows to manage the permissions needed by an application alongside the code.

Every time Myrmex has to deploy a policy, it checks if it needs to be created or updated and perform the appropriate action if needed. If a policy already has 5 version (the maximum allowed by AWS), the oldest one will be deleted (unless it as been defined as the default one).

Definition of IAM roles

AWS roles can be configured with JSON files with @myrmex/iam. The structure looks like this:

{
  "description": "Description of the role",
  "managed-policies": [
    "<policy-identifier>"
  ],
  "inline-policies": [
    "<policy-definition>"
  ],
  "trust-relationship": {
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {
        "Service": "<service-identifier>"
      },
      "Action": "sts:AssumeRole"
    }]
  }
}

The managed-policies section contains a list of identifiers of IAM policies already deployed in AWS associated to the role. During the deployment of a role, Myrmex will look for policies matching the identifiers in managed-policies, trying combinations with the environment and the stage options:

<environment>_<identifier>_<stage>
<environment>_<identifier>
<identifier>

The inline-policies section contains a list of AIM policy definitions associated to the role.

The trust-relationship section contains the policy that describe the trusted entities that can assume the role

Integration with `@myrmex/api-gateway`

@myrmex/iam add some functionalities to @myrmex/api-gateway when both are installed in the same project.

Simplify the association of an API endpoint with its invocation role

In the spec.json file that describes an endpoint, the invocation role is configured via the [API Gateway extension to Swagger] (http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html) using the property credentials. It accepts the ARN of an IAM role. Using @myrmex/iam, it is also possible configure this property using the identifier of a role defined in the Myrmex project or the name of a role deployed in AWS.

New behavior for the option `--role <role>` of `myrmex create-endpoint`

This option normally accepts the ARN of a role. With the plugin @myrmex/iam, it also accepts the identifier of an role defined in the Myrmex project or the name of a role deployed in AWS.

If the option is not provided in the command line, a prompt will propose three option to set the value:

Select a role managed by the plugin @myrmex/iam
Select a role in your AWS account
Enter the value manually

Integration with `@myrmex/lambda`

@myrmex/iam add some functionalities to @myrmex/lambda when both are installed in the same project.

Simplify the association of a Lambda with its execution role

In the config.json file that describes the configuration of a Lambda, the invocation role is configured via the using the property params.role. It accepts the ARN of an IAM role. Using @myrmex/iam, it is also possible configure this property using the identifier of a role defined in the Myrmex project or the name of a role deployed in AWS.

New behavior for the option `--role <role>` of `myrmex create-lambda`

This option normally accepts the ARN of a role. With the plugin @myrmex/iam, it also accepts the identifier of an role defined in the Myrmex project or the name of a role deployed in AWS.

If the option is not provided in the command line, a prompt will propose three option to set the value:

Select a role managed by the plugin @myrmex/iam
Select a role in your AWS account
Enter the value manually

Hooks and extensions

Hooks

beforePoliciesLoad()

beforePolicyLoad(definitionFilePath, name)

afterPolicyLoad(policy)

afterPoliciesLoad(policies)

beforeRolesLoad()

beforeRoleLoad(definitionFilePath, name)

afterRoleLoad(role)

afterRolesLoad(roles)

beforeDeployPolicy(policy)

beforeDeployRole(role)

Extensions

myrmex.call('iam:getRoles') returns the roles managed in the current Myrmex project.

myrmex.call('iam:getAWSRole') returns the roles available in AWS.

myrmex.call('iam:retrieveRoleArn', identifier, context) retrieve the ARN of an IAM role using its identifier. Tries combinations with the environment and the stage properties of the context parameter in this order:

<environment>_<identifier>_<stage>
<environment>_<identifier>
<identifier>.

Lambda plugin

The @myrmex/lambda plugin allows to define and deploy Lambdas. It should work with any runtime but has been tested mostly with Node.js and secondly with Python.

Prerequisites

To use the @myrmex/lambda plugin, it is necessary to have a minimal knowledge about AWS Lambda.

An AWS user or role that uses the plugin @myrmex/lambda must have access to Lambda administration. The AWS policy AWSLambdaFullAccess gives all necessary permissions.

Installation

Install the npm module in a Myrmex project:

npm install @myrmex/lambda

Then enable the plugin in the myrmex.json file:

{
  "name": "my-app",
  "plugins": [
    "@myrmex/lambda"
  ]
}

Once the plugin is installed and enabled in the project, the myrmex command line will provide new sub-commands to manage and deploy Lambdas.

Project anatomy

By default, the content managed by the Lambda plugin is located in an lambda directory in the root directory of the project.

Out of the box, for the Node.js runtime, the Lambda plugin allows to separate the definition of Lambdas from the logic of the application by providing a specific place to write node modules but it is not mandatory to use it. @myrmex/lambda helps to define and deploy Lambdas but the developer is responsible of the implementation of the application. Other plugins built on top of the Lambda plugin may be more opinionated.

The directory lambda/lambdas contains the Lambdas definitions. For each of its sub-directory is considered as a Lambda definition. It must contains a config.json file and the code of the Lambda. The name of the subdirectory is used as the Lambda identifier.

The config.json file allows to define the runtime, the timeout, the memory, the role and other properties of the Lambda. The content of the params property is used as the argument of the following methods from the AWS SDK:

Example of config.json file:

{
  "params": {
    "Timeout": 10,
    "MemorySize": "256",
    "Runtime": "nodejs6.10",
    "Handler": "index.handler",
    "Role": "arn:aws:iam::012345678901:role/LambdaBasicExecution"
  }
}

By default, for the Node.js runtime, the directory lambda/modules contains the node modules of the project. For example, some of these modules could be named log, or data-access or authorization etc...

Each module should have a clear responsibility so that each Lambda can embed only the code it needs. This is a recommendation but the developer is free to organize the code as he want. The Lambda plugin does not force you to use a specific project organization.

This is what a project structure could look like:

lambda
├── lambdas                         The Lambdas defined by the application
|   ├── my-nodejs-lambda            The name of this directory is the identifier of a Lambda
|   |   ├── config.json             The configuration of the Lambda (runtime, memory, timeout, execution role...)
|   |   ├── index.js                A node module that exposes the handler
|   |   └── package.json            It is possible to install the dependencies of the Lambda here
|   └── my-python-lambda            Several runtimes can coexist in a project
|       ├── config.json
|       └── lambda_function.py
└── modules                         The node modules of the application - they can be added as a dependency of a Lambda in its
    |                               package.json file
    ├── authorization               The name of this directory is the identifier of a module
    |   ├── package.json            Package file of the module
    |   ├── index.js                Main file of the module
    |   └── test                    If you wish, you can write the code to test the module in this directory
    ├── data-access
    |   ├── package.json
    |   └── index.js
    └── log
        ├── package.json
        └── index.js

Example of config.json file:

{
  "params": {
    "Timeout": 30,
    "MemorySize": 256,
    "Role": "arn:aws:iam::123456789012:role/MyRole",
    "Runtime": "nodejs6.10",
    "Handler": "index.handler"
  }
}

The package.json of a module or a Lambda can declare dependencies with other modules using file paths in the dependencies property:

{
  "name": "data-access",
  "version": "0.0.0",
  "dependencies": {
    "log": "../log"
  }
}

In this case, require('log') will load the module log installed in lambda/modules/data-access/node_modules/log.

It is recommended to use relative paths for portability.

It is recommended to use a recent version of npm to minimize the size of the Lambda packages and facilitate the configuration of nested dependencies. Indeed, npm@2 can behave in an unexpected manner with nested dependencies when using relative file paths.

Configuration

These are Myrmex configuration keys specific to to the @myrmex/lambda plugin.

Default values

Using myrmex show-config after installing the plugin, we can see the default configuration:

{
  "lambda": {
    "lambdasPath": "lambda/lambdas",
    "modulesPath": "lambda/modules"
  }
}

`lambda.lambdasPath`

Path to the folder that contains Lambdas. Default value: lambda/lambdas.

`lambda.modulesPath`

Path to the folder that contains modules for Node.js Lambdas. Default value: lambda/modules.

`lambda.alias`

Set the alias applied when deploying Lambdas.

By setting this configuration, the --alias option of the myrmex deploy-lambdas command does not prompt when not provided via the command line and the configured value is used as the default value.

Setting lambda.alias to an empty string disables the creation/update of an alias and the new version of the Lambda will only be available as LATEST.

Example

Using the myrmex.json file, the plugin configuration can be defined like this:

{
  "name": "A Myrmex project",
  "plugins": [
    "@myrmex/lambda"
  ],
  "config": {
    "lambda": {
      "lambdasPath": "lambdas",
      "modulesPath": "modules",
      "alias": ""
    }
  }
}

Commands

create-lambda

create-lambda [options] [identifier]

  Options:
    -r, --runtime <nodejs|nodejs4.3|nodejs6.10|python2.7|python3.6>  select the runtime
    -t, --timeout <timeout>                                          select the timeout (in seconds)
    -m, --memory <memory>                                            select the memory (in MB)
    -d --dependencies <modules-names>                                select the project modules that must be included in the Lambda (only for nodejs runtimes)
    --role <role>                                                    select the execution role (enter the ARN)

Create a new Lambda. By default the location of Lambdas is lambda/lambdas/<identifier>/.

create-node-module

For the Node.js runtime only.

create-node-module [options] [name]

  Options:
    -d, --dependencies <dependent-modules>  select the node modules that are dependencies of this new one

Prepare a new Node.js module. By default the location of modules is lambda/modules/<name>/.

The creation of nodes modules is just a suggestion to organize the code of a project. The idea is to maintain each component of the application in its own node module to select only relevant components when deploying Lambdas.

Every Lambda can declare its modules dependencies using local paths in its package.json file. Every module can also declare dependencies to other modules that way.

When Myrmex deploys a Lambda, it executes npm install and the dependencies are installed in the node_modules folder.

deploy-lambdas

deploy-lambdas [options] [lambda-identifiers...]

  Options:
    --all                            deploy all lambdas of the project
    -r, --region <region>            select the AWS region
    -e, --environment <environment>  select the environment
    -a, --alias <alias>              select the alias to apply

Deploy one or more Lambdas in AWS. The --environment option is used as a prefix. The --alias option will publish a version in Amazon Lambda and apply an alias. Setting the option to an empty string (--alias "") will skip this.

When deploying Node.js Lambdas, it is recommended to use npm4 to optimize the size of the packages and avoid the resolution of local dependencies with symbolic links (behavior of npm5).

When deploying Lambdas with C/C++ bindings and/or to be sure to create packages with the correct runtime, use the plugin @myrmex/packager

install-lambdas-locally

install-lambdas-locally [lambda-identifiers...]

Deletes the node_modules folder of one or several lambda and runs npm install to re-install it.

test-lambda-locally

test-lambda-locally [options] [lambda-identifier]

  Options:
    -e, --event <event-name>  Event example to use

Executes a Lambda locally. The event option allows to select the example object that will be passed as the first argument. Example objects are defined in json files in lambda/lambdas/<identifier>/events/<event-name>.json. A mock of the conshell object is passed as the second argument.

test-lambda

test-lambda [options] [lambda-identifier]

  Options:
    --event <event-name>             Event example to use
    -r, --region <region>            select the AWS region
    -e, --environment <environment>  select the environment
    -a, --alias <alias>              select the alias to test

Executes a Lambda deployed in AWS. The event option allows to select the example object that will be passed as the first argument. Example objects are defined in json files in lambda/lambdas/<identifier>/events/<event-name>.json. A mock of the conshell object is passed as the second argument.

Setting the option --alias to an empty string (--alias "") will invoke the LATEST version of the Lambda.

Integration with `@myrmex/api-gateway`

@myrmex/lambda add some functionalities to @myrmex/api-gateway when both are installed in the same project.

Associate a Lambda with an API endpoint

In the spec.json file that describes an endpoint, a new extension to Swagger is available to select a Lambda that must be used for the endpoint integration.

{
  "x-myrmex": {
    "apis": [],
    "lambda": "lambda-identifier"
  }
  ... rest of the endpoint specification
}

New option for `myrmex create-endpoint`

When calling myrmex create-endpoint, a new option --lambda <lambda-identifier> is available. This option accepts the identifier of a Lambda managed by @myrmex/lambda.

If the option is not provided in the command line and the option --integration is set to lambda or lambda-proxy, a prompt will propose to select the appropriate Lambda in a list.

The value of --lambda <lambda-identifier> will be set in extension to Swagger described above.

New options for `myrmex deploy-apis`

When calling myrmex create-endpoint, two new options are available:

`--deploy-lambdas <all|partial|none>`

The option --deploy-lambdas <all|partial|none> accepts three possible values:

all will perform the deployment of all Lambdas defined in the Myrmex project before deploying the APIs.
partial will perform the deployment of all Lambdas that are associated to deployed endpoints before deploying the APIs.
none will not deploy any Lambda, but it will retrieve the ARNs of all Lambdas that are associated to deployed endpoints. So these Lambda have to be already deployed with the appropriate alias.

`--alias <alias>`

The option --alias <alias> allows to select the Lambda alias that will be integrated with endpoints.

If the lambda.alias configuration is set and the option is not provided via the command line, no prompt will appear to set the value the configured value is used as the default value.

API Gateway Plugin

Prerequisites

To use the @myrmex/api-gateway plugin, it is necessary to have a minimal knowledge about Amazon API Gateway and the Swagger v2.0 specification.

@myrmex/api-gateway uses the API Gateway Import API feature.

Installation

Install the npm module in a Myrmex project:

npm install @myrmex/api-gateway

Then enable the plugin in the myrmex.json file:

{
  "name": "my-app",
  "plugins": [
    "@myrmex/api-gateway"
  ]
}

Once the plugin is installed and enabled, the myrmex command line will provide new sub-commands to manage and deploy APIs.

Project anatomy

By default, the content managed by @myrmex/api-gateway is located in an api-gateway directory in the root directory of the project.

A project configuration is composed of a list of APIs and a list of endpoints. Each endpoint specification contains the list of APIs where it must be exposed. Therefore, an endpoint GET /my-resource/{id} can be exposed by a mobile-app API and a back-office API, while another endpoint PUT /my-resource can be exposed by the back-office API only. This allows to create different APIs for different purposes while sharing some functionalities.

The directory api-gateway/apis contains the API definitions. Each of its sub-directory has the name of an API identifier and contains a Swagger specification file (without the endpoints definitions).

The directory api-gateway/endpoints contains the endpoints definitions. The structure of its sub-directories represents url paths and HTTP methods. Each HTTP method directory contains the definition of an endpoint. An endpoint definition is composed of its Swagger specification in a spec.json file.

As an example is more easy to understand, that is what a project structure could looks like:

api-gateway
├── apis                              The APIs defined by the application
|   ├── back-office                   The name of this directory is the identifier of an API
|   |   └── spec.json                 The API's Swagger specification (without endpoints definitions)
|   └── mobile-app
|       └── spec.json
└── endpoints                         The endpoints defined by the application
    ├── resource-a                    Creates the path /resource-a
    |   ├── {id}                      Creates the path /resource-a/{id}
    |   |   ├── GET                   Creates the HTTP method GET for the path /resource-a/{id}
    |   |   |   └── spec.json         The Swagger specification of the endpoint GET /resource-a/{id}
    |   |   └── PATCH
    |   |       └── spec.json
    |   └── PUT
    |       └── spec.json
    └── resource-b
        ├── GET
        |   └── spec.json
        └── resource-c
            └── GET
                └── spec.json

This project contains 2 APIs:

back-office
mobile-app

5 endpoints are available:

GET /resource-a/{id}
PATCH /resource-a/{id}
PUT /resource-a
GET /resource-b
GET /resource-b/resource-c

To associate an endpoint to one or more APIs, it is possible to enrich the endpoint specification (spec.json file) with a list of API identifiers, in a Myrmex extension to Swagger.

{
  "x-myrmex": {
    "apis": [
      "back-office",
      "mobile-app"
    ]
  }
  ... rest of the endpoint specification
}

@myrmex/api-gateway uses the API Gateway Import API feature. Therefore, the configuration of AWS-specific authorization and API Gateway-specific API integrations is done with the API Gateway extensions to Swagger.

Commands

create-api

create-api [options] [api-identifier]

  Options:
    -t, --title <title>       The title of the API
    -d, --desc <description>  A description of the API

Create a new API. By default the location of API definitions is api-gateway/apis/<api-dentifier>/. The API definition is a file named spec.json that contains a Swagger object. The command line will to create a simple base configuration. It is possible to write a full Swagger definition here, but the plugin @myrmex/api-gateway is designed to separate API definitions into pieces and generate full definitions when needed.

create-model

create-model [name]

Create a new model. By default the location of models is api-gateway/model/model.json. Models can be referenced as input or output of API endpoints but are not mandatory.

create-endpoint

create-endpoint [options] [resource-path] [http-method]

  Options:
    -a, --apis <api-identifiers>          The identifiers of APIs that expose the endpoint separated by ","
    -s, --summary <endpoint summary>      A short summary of what the operation does
    --auth <authentication-type>          The type of authentication used to call the endpoint (aws_iam|none)
    -i --integration <integration-type>   The type of integration (lambda|http|mock|aws-service)
    -r, --role <role>                     select the role to invoke integration

Create a new endpoint. By default the location of endpoints definitions is api-gateway/endpoints/<resource-path>/<http-method>/. The endpoint definition is a file named spec.json that contains a Swagger path item object. The command line will help to create a simple base configuration. Then, the developer can complete the definition using the Swagger specification and the API Gateway extentions to Swagger.

The definition of an endpoint can be added to one or more APIs using a Myrmex extention to Swagger:

{
  "x-myrmex": {
    "apis": [
      "<api-identifier-a>",
      "<api-identifier-b>"
    ]
  }
  ... rest of the endpoint specification
}

inspect-api

inspect-api [options] [api-identifier]

  Options:
    -c, --colors                  highlight output
    -s, --spec-version <version>  select the type of specification to retrieve: doc|aws|complete

Generate the Swagger definition of an API and print it. There are three different versions of the specification:

The doc version only contains parts of the definitions that belong to the Swagger specification.
The aws version is the definition that is sent to AWS API gateway when performing a deployment.
The complete version contains everything that is defined in the project, including the Myrmex extensions to Swagger.

inspect-endpoint

inspect-endpoint [options] [resource-path] [http-method]

  Options:
    -c, --colors                  highlight output
    -s, --spec-version <version>  select the type of specification to retrieve: doc|aws|complete

Generate the Swagger definition of an endpoint and print it. There are three different versions of the specification:

The doc version only contains parts of the definitions that belong to the Swagger specification.
The aws version is the definition that is sent to AWS API gateway when performing a deployment.
The complete version contains everything that is defined in the project, including the Myrmex extensions to Swagger.

deploy-apis

deploy-apis [options] [api-identifiers...]

  Options:
    -r, --region <region>            select the AWS region
    -e, --environment <environment>  select the environment
    -s, --stage <stage>              select the API stage

Deploy one or more APIs in API Gateway. The --environment option is used as a prefix to the API name in API Gateway. It can be useful to deploy APIs several time in the same AWS account. Values for --environment could be PROD or DEV or JOHN to be able to deploy the API for production, development or a developer environment.

The use of --environment can seem redundant with --stage, but when deploying an API Gateway stage, it is not to possible to access to the testing tools of the AWS console. These tools are available only for the ultimate version of the API. Using --environment can allow several developers to use the testing tools of the AWS console simultaneously. --environment also allows to use --stage for other purposes like API versioning (with values like v1, v2, 2017-01-01 ...).

CORS Plugin

Prerequisites

The @myrmex/cors plugin gives the possibility to easily define CORS configuration for the @myrmex/api-gateway plugin.

Installation

Install the npm module in a Myrmex project:

npm install @myrmex/cors

Then enable the plugin in the myrmex.json file:

{
  "name": "my-app",
  "plugins": [
    "@myrmex/api-gateway",
    "@myrmex/cors"
  ]
}

The plugin does not provide new, commands but it allows to define CORS configuration at various levels in the project.

Principle

The plugin @myrmex/cors aims to simplify the configuration of CORS by consuming simple configuration files and automatically creating OPTIONS http methods for APIs and adding http headers to endpoints.

@myrmex/cors supports the pseudo http method ANY available in API Gateway to define DELETE, GET, HEAD, PATCH, POST and PUT method with a single configuration. @myrmex/cors uses it to apply the configuration correctly but it will not be present in http responses.

Configuration

There are four possible levels of configuration in this order of precedence:

Global configuration
API level configuration
Resource path level configuration
API resource path level configuration

Global configuration

The global configuration applies by default for all endpoints of all APIs.

Using myrmex show-config, it is possible to see the default values:

{
  "apiGateway": {
    "apisPath": "api-gateway/apis",
    "endpointsPath": "api-gateway/endpoints",
    "modelsPath": "api-gateway/models"
  },
  "cors": {
    "Access-Control-Allow-Methods": "DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT,ANY",
    "Access-Control-Allow-Headers": "*",
    "Access-Control-Allow-Origin": "*"
  }
}

By default, all methods, all origins and all headers are allowed. This configuration allows to quickly experiment, but it can be overwritten to restrict the access to the maximum. For example, to allow the access to the APIs only for the domain www.example.com, it is possible to set this configuration in the file myrmex.json at the root of the project:

{
  "name": "my-app",
  "plugins": [
    "@myrmex/api-gateway",
    "@myrmex/cors"
  ],
  "config": {
    "cors": {
      "Access-Control-Allow-Origin": "https://www.example.com"
    }
  }
}

There are several ways to set this kind of configuration in Myrmex.

The command myrmex show-config shows the new configuration:

{
  "apiGateway": {
    "apisPath": "api-gateway/apis",
    "endpointsPath": "api-gateway/endpoints",
    "modelsPath": "api-gateway/models"
  },
  "cors": {
    "Access-Control-Allow-Methods": "DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT,ANY",
    "Access-Control-Allow-Headers": "*",
    "Access-Control-Allow-Origin": "https://www.example.com"
  }
}

API level configuration

To override the global configuration for a specific API, it is possible to use a Myrmex extension to Swagger in the spec.json file of the API. If we want to configure an API that has the identifier back-office to accept request only from https://admin.example.com, we can complete the file /api-gateway/apis/back-office/spec.json like this:

{
  "x-myrmex": {
    "cors": {
      "Access-Control-Allow-Origin": "https://admin.example.com"
    }
  },
  "swagger": "2.0",
  "info": {
    "title": "my-api",
    "description": "An API built with Myrmex"
  },
}

This configuration will apply to all resource paths of the API.

Resource path level configuration

To define a specific configuration for a resource path, it is possible to create to create a cors.json file in the resource path.

To forbid the access to all http methods of the resource path /my/resource/path, we can create a file /api-gateway/endpoints/my/resource/path/cors.json with the following content:

{
  "default": {
    "Access-Control-Allow-Methods": ""
  }
}

API resource path level configuration

The cors.json file at the resource path level accepts API specific configurations. Following the previous example, if we want the API that has the identifier back-office to allow GET and POST request from https://admin.example.com, we can complete the file /api-gateway/endpoints/my/resource/path/cors.json like this:

{
  "default": {
    "Access-Control-Allow-Methods": ""
  },
  "back-office": {
    "Access-Control-Allow-Methods": "GET,POST",
    "Access-Control-Allow-Origin": "https://admin.example.com"
  }
}

Packager plugin

Why ?

When installing Node.js and python dependencies to create a package for Lambda, some modules may not be compatible with the execution environment of Amazon Lambda, because of differences of version of Node.js, python or GCC when modules have C++ bindings. The @myrmex/packager plugin helps to prevent this problem.

How it works

A common solution to this problem is to create the packages on a EC2 instance using an Amazon Linux AMI. To avoid having to use a server to deploy serverless application, @myrmex/packager uses the Docker image myrmex/lambda-packager based on the Amazon Linux image. The image myrmex/lambda-packager comes with all Node.js and python runtimes supported by Amazon Lambda and GCC.

When the plugin @myrmex/lambda deploys a Lambda, @myrmex/packager takes in charge the creation of the package by executing docker run myrmex/lambda-packager with appropriates parameters.

Prerequisites

To use the @myrmex/packager plugin, it is necessary to have Docker installed on the environment that perform the deployment of Lambdas.

Installation

Install the npm module in a Myrmex project:

npm install @myrmex/packager

Then enable the plugin in the myrmex.json file:

{
  "name": "my-app",
  "plugins": [
    "@myrmex/lambda"
    "@myrmex/packager"
  ]
}

Once the plugin is installed and enabled in the project, the @myrmex/lambda plugin will use it to create packages when deploying Node.js and python Lambdas.

Configuration

These are Myrmex configuration keys specific to to the @myrmex/packager plugin.

Default values

Using myrmex show-config after installing the plugin, we can see the default configuration:

{
  "packager": {
    "bucket": null,
    "docker": {
      "useSudo": false,
      "showStdout": false
    }
  }
}

`packager.bucket`

Name of the S3 bucket where the packages will be uploaded. If it does not exists, @myrmex/packager will try to create it. If this configuration key is not provided, @myrmex/packager will directly upload the packages to Lambda without using S3.

`packager.docker.useSudo`

If set to true, @myrmex/packager will use sudo to execute docker run. Alternatively, it is possible to manage Docker as a non-root user using a group.

`packager.docker.showStdout`

It set to true, the output of docker run will be diplayed in the terminal.

Creating plugins

Plugin definition

The Myrmex core is the node module @myrmex/core. It exports an object (referred as the Myrmex instance) that is based on Pebo that is a kind of event emitter.

Myrmex plugins use the Myrmex instance to interact with each other. A Myrmex plugin can be published on npm to be used by anyone, or it can be a project specific plugin.

Myrmex plugins can make use of the following functionalities:

Listen for events fired by other plugins
Fire events that can be listened by other plugins
Attach functions to the Myrmex instance (refered as extensions)

Anatomy of a plugin

A plugin is node module that exports an object that has to respect a specific structure to be used by the Myrmex instance.

Here is a quick overview of an object exported by a Myrmex plugin.

{
  // An unique identifier that should be carefully chosen to not enter in conflict with other plugins
  name: `my-plugin-identifier`,
  version: `0.0.0`,

  // If the plugin is configurable, this place is suggested to define default values
  // This configuration can be overwritten for a project in the file
  // `project-root/config/my-plugin-identifier.json`
  config: {                 // These values will be accessible using
    key1: 'defaultValue1',  // `myrmex.getConfig("my-plugin-identifier.key1")`
    key2: {                 // `myrmex.getConfig("my-plugin-identifier.key2")`
      a: 'defaultValue2A',  // `myrmex.getConfig("my-plugin-identifier.key2.a")`
      b: 'defaultValue2B'   // `myrmex.getConfig("my-plugin-identifier.key2.b")`
    }
  },

  hooks: {
    // These functions are event listeners that will be called
    // when events "eventNameA" and "eventNameB" are fired by Myrmex
    // More details about events below
    eventNameA: function (argA, argB) {
      return Promise.resolve()
    },
    eventNameB: function () {
      return Promise.resolve()
    }
  ],

  extensions: {
    // These functions will be registered in the Myrmex instance
    // so they will be accessible by other plugins
    // More details about extensions below
    extensionNameA: function() {}
    extensionNameB: function() {}
  }
}

The Myrmex instance

Before describing events and extensions, we should have a quick overview about the initialization of the Myrmex instance and the registration of plugins.

Initialization by the Myrmex command line

When the Myrmex command line is called. It checks if it is called from a Myrmex project by searching for a package.json file containing a dependency to @myrmex/core in the current path.

Once a @myrmex/core module have been found, the Myrmex command line loads it and receives an object: the Myrmex instance.

The Myrmex command line then loads the project configuration (including the list of plugins) and applies it to the Myrmex instance. Each plugin is loaded and registered in the Myrmex instance.

Registration of plugins

When the Myrmex instance registers a plugin, it basically performs three operations:

Checks if it there is a list of hooks and create the event listeners.
Checks if it there are extensions and add them to the Myrmex instance.
Create a reference of itself in a myrmex property that is added to the plugin.

So, once a plugin has been registered by the Myrmex instance, it has a new myrmex property that contains the Myrmex instance. It can be used to fire events or call extensions that have been provided by other plugins.

Events mecanism

The event mecanism of Myrmex is designed to allow code injection and works with asynchronous operations.

From the event emitter point of view, when the Myrmex instance fires an event, it returns a promise of an array containing the arguments of the event. These arguments have eventually been transformed by event listeners.

From the event listener point of view, when the listener function is called, it has the possibility to alter the parameters of the event. If the event listener has to perform asynchronous operations to alter theses parameters, it should return a Promise that resolves once the modification is done.

Therefore, an emitter can wait for all listeners to eventually alter the event parameters to continue its task.

Here is a dummy example:

// The Myrmex instance is accessible via the `myrmex` property of a plugin once it has been registered
const myrmex = require('./plugin-index.js').myrmex;

// Create an event listener
myrmex.when('MyEvent', (myArg1, myArg2) => {
  myArg.propB += ' transformed by an event listener';
  myArg.propC = 'property added by an event listener';
  return Promise.resolve();
});

// We define two objects that will be passed as the event arguments
// Passing literals is not recommended because JavaScript will copy them when passing them as arguments
// Myrmex would not be able to retrieve modifications applied to them
const eventArg1 = {
  propA: 'original property',
  propB: 'original property'
}
const eventArg2 = {
  propA: 'original property'
}

myrmex.fire('MyEvent', eventArg1, eventArg2)
.then(arg => {
  console.log(arg[0]);
  //  {
  //    propA: 'original property',
  //    propB: 'original property transformed by an event listener',
  //    propC: 'property added by an event listener'
  //  }
  console.log(arg[1]);
  //  {
  //    propA: 'original property'
  //  }
});

Using this mecanism, Myrmex plugins can create their own events and listen for other plugins events. Several plugins can listen for the same event and use and/or modify its arguments.

For example, before configuring the integration request of API endpoints, the @myrmex/api-gateway plugin fires and event that allows other plugins to be notified about it. Theses plugins have the opportunity to inject data in the event's argument so the plugin @myrmex/api-gateway can generate a compete API specification.

Note that it is not recommended to call myrmex.when() directly when writing a plugin. We prefer to use the hooks property of the object exported by the plugin to organize the code and be sure that the listeners are attached to the events during the registration of the plugin.

Extensions

@todo

Deploying an Express application in Lambda

This tutorial uses express-generator to quickly create a sample application. This sample application is not an API and serves static content but we do not recommend to do that in a real case scenario. You should use Express with API Gateway and Lambda only for APIs.

Starting with an Express application

We will start by creating a basic Express application following the documentation.

# Installation of express-generator
npm install express-generator -g
# Creation of a basic application
express --view=pug express-app
# Installation of dependencies
cd express-app && npm install

We start the application locally to check that everything works correctly.

set DEBUG=express-app:* & npm start

Visit http://127.0.0.1:3000/ to check that the application runs correctly locally.

Adding Myrmex to the project

Be sure that the Myrmex cli is installed.

npm install -g myrmex

Install Myrmex in the Express project

# For the prompt "What is your project name?", leave it empty, so Myrmex will be installed in the same directory that Express
# We will need the plugins @myrmex/iam @myrmex/api-gateway and @myrmex/lambda
myrmex new

Myrmex and its plugins will be installed in the section devDependencies of the file package.json.

Creation of IAM roles

We create two IAM roles: one to allow Lambda to write in cloudwatch and the other to allow API Gateway to invoke Lambda.

myrmex create-role MyExpressAppLambdaExecution --model LambdaBasicExecutionRole
myrmex create-role MyExpressAppLambdaInvocation --model APIGatewayLambdaInvocation

Creation of the Lambda

We create a new Lambda managed by Myrmex.

myrmex create-lambda serverless-express --runtime nodejs6.10 --timeout 30 --memory 256 --role MyExpressAppLambdaExecution

We have to do a small alteration to the package.json of the Express application to tell node that app.js is the file that has to be loaded when require() is used.

// in /package.json
{
  "name": "express-app",
  "version": "0.0.0",
  "main": "app.js",
  ...
}

In the package.json of the Lambda, we define the Express application as a dependency

# in /lambda/lambdas/serverless-express
npm install ../../.. --save

aws-serverless-express is a npm package that wrap an Express application to process requests from API Gateway with Lambda integration. We add it as a dependency of our Lambda.

# in /lambda/lambdas/serverless-express
npm install aws-serverless-express --save

Then we can alter the code of index.js from the Lambda to correctly expose the handler.

// in /lambda/lambdas/serverless-express/index.js
'use strict';
const awsServerlessExpress = require('aws-serverless-express');
const app = require('express-app');
const server = awsServerlessExpress.createServer(app);

exports.handler = (event, context) => awsServerlessExpress.proxy(server, event, context);

Creation of the API

We create an API with two endpoints using ANY method and Lambda proxy integration: one to handle the root url, and another for the rest.

myrmex create-api serverless-express --title "Serverless Express" --desc "An Express application served by API Gateway and Lambda"
myrmex create-endpoint / ANY --apis serverless-express --summary "Proxy to Lambda" --auth none --integration lambda-proxy --lambda serverless-express --role MyExpressAppLambdaInvocation
myrmex create-endpoint /{proxy+} ANY --apis serverless-express --summary "Proxy to Lambda" --auth none --integration lambda-proxy --lambda serverless-express --role MyExpressAppLambdaInvocation

Deployment

myrmex deploy-apis serverless-express --region us-east-1 --environment DEV --stage v0 --deploy-lambdas all --alias ""

The deployment messages will give you the url of the application. It will have the form https://{your-api-id}.execute-api.us-east-1.amazonaws.com/v0. You can test the root url and the url https://{your-api-id}.execute-api.us-east-1.amazonaws.com/v0/users that is also declared in the sample application generated by express-generator. You can also test some bad urls and observe that you receive an error 404.

Creating Lambdas to inspect their execution environment

Creation of three Lambdas sharing the same code

The code generated by this example is available here.

Lets create a new Myrmex project.

myrmex new lambda-profiling @myrmex/iam @myrmex/lambda
cd lambda-profiling

We create a simple execution role for our Lambdas.

myrmex create-role LambdaInspection --model LambdaBasicExecutionRole

We create a node module called inspection. It will inspect its own execution environment.

myrmex create-node-module inspection

The inspection module is created in lambda-profiling/lambda/modules/inspection and only contain a package.json file for now.

Lets create an index.js file that exposes data from the os native module and the process global object in our module.

// lambda-profiling/lambda/modules/inspection/index.js
'use strict';

const os = require('os');

module.exports = () => {
  return {
    host: {
      name: os.hostname(),
      architecture: process.arch,
      memory: os.totalmem(),
      cpus: os.cpus()
    },
    os: {
      platform: process.platform,
      release: os.release(),
      uptime: os.uptime()
    },
    user: {
      uid: process.getuid(),
      gid: process.getgid(),
      groups: process.getgroups
    },
    process: {
      command: process.argv,
      execArgs: process.execArgv,
      pid: process.pid,
      uptime: process.uptime(),
      memoryUsage: process.memoryUsage()
    },
    exec: {
      env: process.env,
      currentDirectory: process.cwd(),
      // mainModule: process.mainModule,
      config: process.config
    }
  };
};

Now we create three Lambdas that will use the inspection module as a dependency. Their are configured with different memory values: 128MB, 512MB and 1536MB.

myrmex create-lambda config-128 --runtime nodejs6.10 --timeout 30 --memory 128 --dependencies inspection --role LambdaInspection
myrmex create-lambda config-512 --runtime nodejs6.10 --timeout 30 --memory 512 --dependencies inspection --role LambdaInspection
myrmex create-lambda config-1536 --runtime nodejs6.10 --timeout 30 --memory 1536 --dependencies inspection --role LambdaInspection

We alter the handler of the three Lambdas to call the inspection module, log its output (it will be visible in cloudwatch) and return it.

// lambda-profiling/lambda/lambdas/config-128/index.js
// lambda-profiling/lambda/lambdas/config-512/index.js
// lambda-profiling/lambda/lambdas/config-1536/index.js
'use strict';

const inspection = require('inspection');

module.exports.handler = function(event, context, cb) {
  const inspect = inspection();
  console.log(JSON.stringify(inspect));
  cb(null, inspect);
};

We can execute the lambdas locally to test them.

# Install dependencies of the Lambdas
myrmex install-lambdas-locally config-128 config-512 config-1536
# Execute Lambdas locally
myrmex test-lambda-locally config-128
myrmex test-lambda-locally config-512
myrmex test-lambda-locally config-1536

Testing the Lambdas in AWS

Lets deploy in AWS!

myrmex deploy-lambdas config-128 config-1536 config-512 --region us-east-1 --environment DEV --alias v0

Now we can execute the Lambdas in AWS using Myrmex, but you can also use the AWS console if you prefer.

myrmex test-lambda config-128 --region us-east-1 --environment DEV --alias v0
myrmex test-lambda config-512 --region us-east-1 --environment DEV --alias v0
myrmex test-lambda config-1536 --region us-east-1 --environment DEV --alias v0

Check the result of the functions to have insights about the execution environment of Lambdas. Observe the variations when calling several times the same function.

TODO: explain results and add a module to profile the use of require() with/without warm up of a Lambda.

Creating APIs for a space delivery company

The need: one application, several APIs exposing various functionalities

The code generated by this example is available here.

Planet Express is a space delivery company that have two types of customers :

Senders like MomCorp and the Slurm factory that use Planet Express services to deliver products
Recipients like the Luna Park or the Mars Vegas Casinos that receive these products

Planet Express, as a futurist company, wants to provide serverless APIs, web and mobile applications for their customers.

Senders need an API to create and track deliveries
Recipients need an API to track and validate the reception of deliveries
Planet Express also needs an API for its Back Office application that manage all deliveries

Following REST principles, a simplified API implementing these functionalities should expose these endpoints :

PUT /delivery to create of a new delivery
GET /delivery/{id} to allow to track a delivery
PATCH /delivery/{id} to modify a delivery
DELETE /delivery/{id} to delete a delivery

We state that a sender cannot modify a delivery once created and a recipient deletes a delivery when it receives it.

Planet Express wants to provide an Swagger specification to document the endpoints for development teams that integrate the services.

To avoid exposing and documenting endpoints not needed by some API consumers, Planet Express wants a specific API for each type of client application : sender, recipient and back-office.

These three APIs belong to the same application and share the same code base. The back-office API will provide access to all functionalities while the consumers of the sender and the recipient APIs will only have access to endpoints that are useful for them.

	`sender`	`recipient`	`back-office`
`POST /delivery`	X		X
`GET /delivery/{id}`	X	X	X
`PATCH /delivery/{id}`			X
`DELETE /delivery/{id}`		X	X

We will not see the implementation of authentication and authorization here. Neither will we see the implementation of the data access. These are aspects of the application that do not rely on Myrmex. They are implemented in node.js modules that could run in any execution environment.

The creation of the application with Myrmex

For each command used in the following sections, it is possible to see its documentation using myrmex <command> -h. Note that the Myrmex cli itself only contains the command myrmex new. Other commands are injected by Myrmex plugins when the cli is executed in the project folder.

Initialization of the project

We create a new Myrmex project named planet-express.

npm install -g @myrmex/cli
myrmex new planet-express @myrmex/iam @myrmex/api-gateway @myrmex/lambda
cd planet-express

Creation of the Lambda containing the application logic

We are free to use as many Lambdas as we want in a Myrmex project. For example, we could associate one specific Lambda for each endpoint. But the Planet Express development team choose to put all the logic of the application in one single Lambda. Parameters provided by API Gateway will allow the lambda to know which portions of code to execute.

The implementation of the Lambda and of the modules is out of the scope of this tutorial. We will create them and leave them empty. This will give an overview of the structure of a Myrmex project and show the integration of Lambdas with API Gateway. Myrmex helps you to organize and deploy serverless projects, but it does not interfere about how you write Node.js code.

Creation of the Lambda's execution role

The Lambda needs to be associated to an IAM role that defines its authorizations in AWS. It could be authorized to access DynamoDB, RDS or S3 depending on the implementation of the application. For now, we use a role that allows the Lambda to write logs in CloudWatch.

myrmex create-role PlanetExpressLambdaExecution --model LambdaBasicExecutionRole

Creation of node modules that can be packaged in the Lambda

Myrmex allows the creation of node modules that can be embedded in the package of the Lambda. The development team decided that they will need two modules:

A logging module that should allow to write logs in various backends (Cloudfront, S3, a database, etc ...)
A module that manage the access to the application data (DynamoDB, Sequelize, mongoose, etc ...)

The data access module uses the logging module. We indicate this dependency so that if a Lambda requires the data access module, it will include the logging module in its package too.

myrmex create-node-module log
myrmex create-node-module data-access --dependencies log

The implementation of the modules will not be detailed here, but they are "common" node.js modules that could be used in any execution environment.

Creation of a Lambda

We create a Lambda, specifying its configuration and its dependencies.

myrmex create-lambda api-generic --runtime nodejs6.10 --timeout 20 --memory 256 --role PlanetExpressLambdaExecution --dependencies data-access,log

Creation of the endpoints and APIs

Now that we have a Lambda able to perform the operations needed by our application, we can define the endpoints that integrate with it and expose them in various APIs.

Creation of the APIs

We create one API for each type of client application.

myrmex create-api back-office --title "Back Office" --desc "Planet Express API for Back Office"
myrmex create-api sender      --title "Sender"      --desc "Planet Express API for sender application"
myrmex create-api recipient   --title "Recipient"   --desc "Planet Express API for recipient application"

Creation of the invocation role

The API endpoints need to be associated to an IAM role that allows to invoke the Lambda.

myrmex create-role PlanetExpressLambdaInvocation -m APIGatewayLambdaInvocation

Creation of the endpoints

We create the four endpoints and associate them to the APIs that have to expose them.

myrmex create-endpoint /delivery/{id} get --apis back-office,recipient,sender --summary "View a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic
myrmex create-endpoint /delivery/{id} patch --apis back-office --summary "Update a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic
myrmex create-endpoint /delivery post --apis back-office,sender --summary "Create a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic
myrmex create-endpoint /delivery/{id} delete --apis back-office,recipient --summary "Delete a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic

Deployment

We deploy the stage v0 of the three APIs on the DEV environment.

myrmex deploy-apis back-office sender recipient --region us-east-1 --stage v0 --environment DEV --deploy-lambdas all --alias ""

Altogether

npm install -g @myrmex/cli
myrmex new planet-express @myrmex/iam @myrmex/api-gateway @myrmex/lambda
cd planet-express

myrmex create-role PlanetExpressLambdaExecution --model LambdaBasicExecutionRole

myrmex create-node-module log
myrmex create-node-module data-access --dependencies log

myrmex create-lambda api-generic --runtime nodejs6.10 --timeout 20 --memory 256 --role PlanetExpressLambdaExecution --dependencies data-access,log

myrmex create-api back-office --title "Back Office" --desc "Planet Express API for Back Office"
myrmex create-api sender      --title "Sender"      --desc "Planet Express API for sender application"
myrmex create-api recipient   --title "Recipient"   --desc "Planet Express API for recipient application"

myrmex create-role PlanetExpressLambdaInvocation -m APIGatewayLambdaInvocation

myrmex create-endpoint /delivery/{id} get --apis back-office,recipient,sender --summary "View a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic
myrmex create-endpoint /delivery/{id} patch --apis back-office --summary "Update a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic
myrmex create-endpoint /delivery post --apis back-office,sender --summary "Create a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic
myrmex create-endpoint /delivery/{id} delete --apis back-office,recipient --summary "Delete a delivery" --integration lambda-proxy --auth none --role PlanetExpressLambdaInvocation --lambda api-generic

myrmex deploy-apis back-office sender recipient --region us-east-1 --stage v0 --environment DEV --deploy-lambdas all --alias ""

Getting started

Getting started

Configuring AWS

Prerequisites

Permissions needed to deploy the project in AWS

Installation

Installing the Myrmex command line

Creating a new project

Installing plugins

Project configuration

The myrmex.json file

Splitting the configuration into various files

Using environment variables to override the configuration

Defining the configuration programmatically

IAM plugin

IAM plugin

Prerequisites

Installation

Commands

create-policy

create-role

deploy-policies

deploy-roles

Definition of IAM policies

Definition of IAM roles

Integration with @myrmex/api-gateway

Simplify the association of an API endpoint with its invocation role

New behavior for the option --role <role> of myrmex create-endpoint

Integration with @myrmex/lambda

Simplify the association of a Lambda with its execution role

New behavior for the option --role <role> of myrmex create-lambda

Hooks and extensions

Hooks

Extensions

Lambda plugin

Lambda plugin

Prerequisites

Installation

Project anatomy

Configuration

Default values

lambda.lambdasPath

lambda.modulesPath

lambda.alias

Example

Commands

create-lambda

create-node-module

deploy-lambdas

install-lambdas-locally

test-lambda-locally

test-lambda

Integration with @myrmex/api-gateway

Associate a Lambda with an API endpoint

New option for myrmex create-endpoint

New options for myrmex deploy-apis

--deploy-lambdas <all|partial|none>

--alias <alias>

API Gateway Plugin

API Gateway Plugin

Prerequisites

Installation

Project anatomy

Commands

create-api

create-model

create-endpoint

inspect-api

inspect-endpoint

deploy-apis

CORS Plugin

CORS Plugin

Prerequisites

Installation

Principle

Configuration

Global configuration

API level configuration

Resource path level configuration

API resource path level configuration

The `myrmex.json` file

Integration with `@myrmex/api-gateway`

New behavior for the option `--role <role>` of `myrmex create-endpoint`

Integration with `@myrmex/lambda`

New behavior for the option `--role <role>` of `myrmex create-lambda`

`lambda.lambdasPath`

`lambda.modulesPath`

`lambda.alias`

Integration with `@myrmex/api-gateway`

New option for `myrmex create-endpoint`

New options for `myrmex deploy-apis`

`--deploy-lambdas <all|partial|none>`

`--alias <alias>`

`packager.bucket`

`packager.docker.useSudo`

`packager.docker.showStdout`