Developing Greengrass Components in Docker: A Step-by-Step Guide

Takahiro Iwasa

Feb 28, 2023

8 min read

Greengrass Greengrass Development Kit IoT

This post describes how to develop AWS IoT Greengrass components in a local environment using the Docker image of Greengrass Core. For more details, refer to the official documentation.

In this guide, we’ll build a Greengrass component that sends messages to AWS IoT Core via MQTT every second. The component will be deployed to a Docker container locally using the Greengrass CLI.

By the end of this guide, your project directory will look like this:

/
|-- components/
|   `-- mqtt_publisher/
|        |-- .gitignore
|        |-- gdk-config.json
|        |-- main.py
|        |-- recipe.yaml
|        `-- requirements.txt
`-- docker/
  |-- greengrass-v2-credentials/
  |   `-- credentials
  |-- .env
  `-- docker-compose.yml

Install the Greengrass Development Kit (GDK):

pip install -U git+https://github.com/aws-greengrass/aws-greengrass-gdk-cli.git@v1.1.0

Initialize your Greengrass component by running gdk component init.

First, create a components directory, and then initialize the component:

mkdir ./components
gdk component init \
  --language python \
  --template HelloWorld \
  --name components/mqtt_publisher

This command generates a basic component structure with the following files and directories:

./
|-- components/
|   |-- mqtt_publisher/
|       |-- src/
|       |   |-- greeter.py
|       |-- tests/
|       |   |-- test_greeter.py
|       |-- .gitignore
|       |-- gdk-config.json
|       |-- main.py
|       |-- README.md
|       |-- recipe.yaml

Update the gdk-config.json file with the component metadata. If the component is not being published to an S3 bucket using gdk component publish, the publish.bucket field does not need to be set.

Here’s an example of the updated gdk-config.json:

{
  "component": {
    "com.example.MqttPublisher": {
      "author": "wasabee.io",
      "version": "0.0.1",
      "build": {
        "build_system": "zip"
      },
      "publish": {
        "bucket": "<PLACEHOLDER_BUCKET>",
        "region": "ap-northeast-1"
      }
    }
  },
  "gdk_version": "1.0.0"
}

For additional details, consult the official documentation on the GDK CLI configuration file.

With this setup, you are ready to start implementing your custom Greengrass component.

Create a main.py script for your component. This script will publish MQTT messages to the /mqtt-publisher topic every second:

import json
import random
from datetime import datetime
from time import sleep

import boto3

client = boto3.client('iot-data')


def main():
    payload = {
        "value": random.randint(1, 10000),
        "datetime": datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
    }
    while True:
        client.publish(
            topic='/mqtt-publisher',
            payload=json.dumps(payload).encode(),
            qos=1,
            contentType='application/json',
        )
        print(f'Message was sent successfully: {payload}')
        sleep(1)


if __name__ == "__main__":
    main()

Dependencies

Create a requirements.txt file to specify the required dependencies for your component:

boto3==1.26.65

These dependencies will be installed during the component installation process as defined in the recipe.yaml.

Define your component’s recipe by creating a recipe.yaml file. The recipe specifies metadata, dependencies, and lifecycle hooks for the component. For more information about the component recipe specification, refer to the official documentation.

Here’s an example:

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "{COMPONENT_NAME}"
ComponentVersion: "{COMPONENT_VERSION}"
ComponentDescription: "This is an mqtt publisher written in Python."
ComponentPublisher: "{COMPONENT_AUTHOR}"
ComponentDependencies:
  aws.greengrass.TokenExchangeService:
    VersionRequirement: '^2.0.0'
Manifests:
  - Platform:
      os: all
    Artifacts:
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/mqtt_publisher.zip"
        Unarchive: ZIP
    Lifecycle:
      Install: "pip3 install --user -r {artifacts:decompressedPath}/mqtt_publisher/requirements.txt"
      Run: "python3 -u {artifacts:decompressedPath}/mqtt_publisher/main.py"

Component Dependencies

The example script relies on the boto3 library to interact with AWS IoT Core. Specify the aws.greengrass.TokenExchangeService component as a dependency in the ComponentDependencies section:

• Purpose: Provides credentials to interact with AWS services.
• Key Feature: The TokenExchangeService runs a local server that provides AWS credentials for your custom component.

For more details, refer to the Greengrass documentation.

AWS IoT Greengrass provides a public component, the token exchange service component, that you can define as a dependency in your custom component to interact with AWS services. The token exchange service provides your component with an environment variable, AWS_CONTAINER_CREDENTIALS_FULL_URI, that defines the URI to a local server that provides AWS credentials.

Lifecycle Hooks

The Lifecycle section specifies commands to execute during component installation and runtime:

1. Install: Installs Python libraries listed in requirements.txt.
2. Run: Executes the main.py script when the component starts.

Placeholders in Recipe

Placeholders in the recipe (e.g., {COMPONENT_NAME}) are replaced with values from gdk-config.json during the build process. These placeholders include:

• {COMPONENT_NAME}
• {COMPONENT_VERSION}
• {COMPONENT_AUTHOR}
• Artifacts URI components like BUCKET_NAME, COMPONENT_NAME, and COMPONENT_VERSION.

Use gdk component build to build the component with the Greengrass Development Kit:

cd components/mqtt_publisher
gdk component build

After building, the artifacts will be located in the greengrass-build directory. There is no need to run gdk component publish for local deployment to a Docker container.

You are now ready to deploy your component to the Greengrass Core running in Docker.

This section explains how to set up Greengrass Core in a Docker container, configure credentials, and deploy components.

From here, work in <PROJECT_ROOT>/docker directory.

Greengrass Core requires AWS security credentials to provision resources automatically. While permanent credentials can be used, temporary credentials via sts get-session-token are highly recommended for enhanced security.

The following AWS resources will be provisioned:

• AWS IoT
  • Greengrass Core Device
  • IoT Thing
  • IoT Thing Group
  • Certificate
  • Policies (two)
  • Token Exchange Role Alias
• AWS IAM
  • Token Exchange Role
  • Token Exchange Role Policy

Generate temporary credentials:

aws sts get-session-token

Save the credentials to a file:

mkdir ./greengrass-v2-credentials
nano ./greengrass-v2-credentials/credentials

Example content for credentials:

[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token = AQoEXAMPLEH4aoAH0gNCAPy...truncated...zrkuWJOgQs8IZZaIv2BXIa2R4Olgk

Create a .env file to configure environment variables for the Greengrass Core installer. Refer to the official documentation for more details.

Example .env file:

GGC_ROOT_PATH=/greengrass/v2
AWS_REGION=ap-northeast-1
PROVISION=true
THING_NAME=MyGreengrassCore
THING_GROUP_NAME=MyGreengrassCoreGroup
TES_ROLE_NAME=GreengrassV2TokenExchangeRole
TES_ROLE_ALIAS_NAME=GreengrassCoreTokenExchangeRoleAlias
COMPONENT_DEFAULT_USER=ggc_user:ggc_group

Create a docker-compose.yml file to run Greengrass Core in Docker. Refer to the documentation for further information.

Example docker-compose.yml:

version: '3.7'

services:
  greengrass:
    init: true
    container_name: aws-iot-greengrass
    image: amazon/aws-iot-greengrass:latest
    volumes:
      - ./greengrass-v2-credentials:/root/.aws/:ro
      - ../components:/root/components
    env_file: .env
    ports:
      - '8883:8883'

Run the container:

docker-compose up -d
docker-compose logs -f greengrass

You should see logs indicating the Nucleus has successfully launched.

aws-iot-greengrass | Launching Nucleus...
aws-iot-greengrass | Launched Nucleus successfully.

Greengrass CLI

Install the Greengrass CLI component (aws.greengrass.Cli) for local deployments. After installation, it can be found in /greengrass/v2/bin.

docker-compose exec greengrass bash
cd /greengrass/v2
ls bin

We recommend that you use this component in only development environments, not production environments. This component provides access to information and operations that you typically won’t need in a production environment. Follow the principle of least privilege by deploying this component to only core devices where you need it.

Token Exchange Service

Deploy the aws.greengrass.TokenExchangeService component to enable your custom component to interact with AWS. This service provides temporary credentials via a local server.

docs.aws.amazon.com

Interact with AWS services - AWS IoT Greengrass

Use the token exchange service to acquire AWS credentials that let custom AWS IoT Greengrass V2 components interact with AWS services.

Greengrass core devices use X.509 certificates to connect to AWS IoT Core using TLS mutual authentication protocols. These certificates let devices interact with AWS IoT without AWS credentials, which typically comprise an access key ID and a secret access key.

Deploying from AWS IoT Greengrass Console

Deploy AWS-provided components, including Greengrass Nucleus, via the AWS IoT Greengrass Console.

Once deployed, you should see success logs in /greengrass/v2/logs/greengrass.log.

[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=SUCCEEDED, ThingName=MyGreengrassCore, JobId=}
[INFO] (pool-2-thread-11) com.aws.greengrass.status.FleetStatusService: fss-status-update-published. Status update published to FSS. {trigger=THING_GROUP_DEPLOYMENT, serviceName=FleetStatusService,
[INFO] (pool-2-thread-11) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist link to last deployment. {link=/greengrass/v2/deployments/previous-success}
[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Received empty jobs in notification . {ThingName=MyGreengrassCore}

Update the GreengrassV2TokenExchangeRole IAM policy to grant permissions for MQTT publishing:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iot:Connect",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Publish",
      "Resource": "arn:aws:iot:*:<AWS_ACCOUNT_ID>:topic//mqtt-publisher*"
    }
  ]
}

Attach the policy:

aws iam put-role-policy \
  --role-name GreengrassV2TokenExchangeRole \
  --policy-name IoTPolicy \
  --policy-document file://policy.json

Deploy your custom component using the Greengrass CLI greengrass-cli deployment create inside the Docker container:

cd /greengrass/v2
bin/greengrass-cli deployment create \
  --recipeDir /root/components/mqtt_publisher/greengrass-build/recipes \
  --artifactDir /root/components/mqtt_publisher/greengrass-build/artifacts \
  --merge "com.example.MqttPublisher=0.0.1"

Check the deployment status using greengrass-cli deployment status:

bin/greengrass-cli deployment status -i <DEPLOYMENT_ID>

You should see the response:

INFO: Connection established with event stream RPC server
<DEPLOYMENT_ID>: SUCCEEDED

Monitor logs to ensure the component is running:

cd /greengrass/v2/logs
tail -f com.example.MqttPublisher.log

Expected log output:

[INFO] (Copier) com.example.MqttPublisher: stdout. Message was sent successfully: {'value': 31, 'datetime': '2023-02-27 12:31:35'}. {scriptName=services.com.example.MqttPublisher.lifecycle.Run, serviceName=com.example.MqttPublisher, currentState=RUNNING}

Use the MQTT test client in the AWS IoT Console to verify that messages are being published to the /mqtt-publisher topic.

1. Navigate to the MQTT test client.
2. Input /# or /mqtt-publisher in the Topic filter field.
3. Click the Subscribe button.

You should see the published messages from your custom component.

Using the AWS IoT Greengrass Docker image makes developing and testing components locally more efficient and flexible. We hope this guide helps you streamline your IoT application development process.

Happy Coding! 🚀