R-Car/Boards/Yocto-Gen3/AWS IoT Greengrass/v5.5.0

Introduction

This page describes how to setup the Yocto environment to use the AWS GreenGrass with R-Car and run it.

Environment

Host PC

OS	Ubuntu 20.04 LTS (64bit)
Memory	8 GB or more
Storage	At least 100 GB free

Boards confirmed to work

Board	SoC	Confirmed
R-Car Starter Kit Premier(H3)	v2.0	OK
CCPF + R-Car Starter Kit Premier(H3)	v2.0	OK
R-Car Starter Kit Pro(M3)	v1.0	OK
R-Car Starter Kit Pro(M3)	v3.0	OK
CCPF + R-Car Starter Kit Pro(M3)	v1.0	OK
CCPF + R-Car Starter Kit Pro(M3)	v3.0	OK

Building the BSP

1. Install the required packages

   $ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
   build-essential chrpath socat cpio python3 python3-pip python3-pexpect \
   xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa \
   libsdl1.2-dev pylint3 xterm docker.io
   

   18px <translate> Note:</translate>If the OS version of the host PC is earlier than "Ubuntu 20.04", the default gcc version is old, so use "gcc-9/g++-9" or later version.

2. Setup Git

   $ git config --global user.email "you@example.com"
   $ git config --global user.name "Your Name"
   

3. Setup script(setup.sh)

   #!/bin/bash
   
   BOARD_LIST=("h3ulcb" "m3ulcb" "h3ulcb-ccpf-sk" "m3ulcb-ccpf-sk")
   TARGET_BOARD=$1
   WORK=`pwd`/${TARGET_BOARD}
   
   # Commit ID
   POKY_COMMIT=6ebb33bdaccaeadff0c85aab27acf35723df00d8
   META_OE_COMMIT=c38d2a74f762a792046f3d3c377827b08aade513
   META_RENESAS_COMMIT=0fe77668f5d9a31a5d10449988c3d8fb8dc475c5
   META_RENESAS_CCPF_COMMIT=b49b57d6e68d5cda70aefbed34e06903484c573b
   META_VIRTUALIZATION_COMMIT=92cd3467502bd27b98a76862ca6525ce425a8479
   META_JAVA_COMMIT=62d6c0653ad69e14c21db2d4482e578400116a1b
   META_AWS_COMMIT=09a9e8845e1c0685d279a5fec12dc2764e67675c
   META_DOCKER_COMMIT=1ca1b5caf6f373dcc49db82dce50f4d8ab9f25cd
   
   Usage () {
       echo "Usage: $0 \${TARGET_BOARD_NAME}"
       echo "BOARD_NAME list: "
       for i in ${BOARD_LIST[@]}; do echo "  - $i"; done
       exit
   }
   
   # Check Param.
   if ! `IFS=$'\n'; echo "${BOARD_LIST[*]}" | grep -qx "${TARGET_BOARD}"`; then
       Usage
   fi
   
   mkdir -p ${WORK}
   cd ${WORK}
   
   # Clone basic Yocto layers in parallel
   git clone git://git.yoctoproject.org/poky &
   git clone git://git.openembedded.org/meta-openembedded &
   git clone git://github.com/renesas-rcar/meta-renesas &
   git clone git://git.yoctoproject.org/meta-virtualization &
   git clone git://git.yoctoproject.org/meta-java &
   git clone git://github.com/aws/meta-aws &
   git clone git://github.com/tkomagata/meta-docker &
   if [ "${TARGET_BOARD}" = "h3ulcb-ccpf-sk" ] || [ "${TARGET_BOARD}" = "m3ulcb-ccpf-sk" ]; then
       git clone git://github.com/renesas-rcar/meta-renesas-ccpf &
   fi
   
   # Wait for all clone operations
   wait
   
   # Switch to proper branches/commits
   cd ${WORK}/poky
   git checkout -b tmp ${POKY_COMMIT}
   cd ${WORK}/meta-openembedded
   git checkout -b tmp ${META_OE_COMMIT}
   cd ${WORK}/meta-renesas
   git checkout -b tmp ${META_RENESAS_COMMIT}
   cd ${WORK}/meta-virtualization
   git checkout -b tmp ${META_VIRTUALIZATION_COMMIT}
   cd ${WORK}/meta-java
   git checkout -b tmp ${META_JAVA_COMMIT}
   cd ${WORK}/meta-aws
   git checkout -b tmp ${META_AWS_COMMIT}
   cd ${WORK}/meta-docker
   git checkout -b tmp ${META_DOCKER_COMMIT}
   if [ "${TARGET_BOARD}" = "h3ulcb-ccpf-sk" ] || [ "${TARGET_BOARD}" = "m3ulcb-ccpf-sk" ]; then
       cd ${WORK}/meta-renesas-ccpf
       git checkout -b tmp ${META_RENESAS_CCPF_COMMIT}
   fi
   
   if [ "${TARGET_BOARD}" = "h3ulcb-ccpf-sk" ] || [ "${TARGET_BOARD}" = "m3ulcb-ccpf-sk" ]; then
       TEMPLATECONF=${WORK}/meta-renesas-ccpf/meta-rcar-gen3/docs/sample/conf/${TARGET_BOARD}/bsp/
   fi
   
   cd ${WORK}
   source poky/oe-init-build-env ${WORK}/build
   
   if [ "${TARGET_BOARD}" = "h3ulcb" ] || [ "${TARGET_BOARD}" = "m3ulcb" ]; then
       cp ${WORK}/meta-renesas/meta-rcar-gen3/docs/sample/conf/${TARGET_BOARD}/poky-gcc/bsp/bblayers.conf ${WORK}/build/conf/
       cp ${WORK}/meta-renesas/meta-rcar-gen3/docs/sample/conf/${TARGET_BOARD}/poky-gcc/bsp/local.conf ${WORK}/build/conf/
   fi
   

4. Using setup script

   $ chmod a+x setup.sh
   $ ./setup.sh <target_board_name>
   

   Since "target_board_name" needs to be changed depending on the board you are using, it should be set as follows.

   
   

5. Append the following to layer configuration(build/conf/bblayers.conf)

     ${TOPDIR}/../meta-openembedded/meta-networking \
     ${TOPDIR}/../meta-openembedded/meta-filesystems \
     ${TOPDIR}/../meta-virtualization \
     ${TOPDIR}/../meta-docker/meta-rcar-gen3 \
     ${TOPDIR}/../meta-java \
     ${TOPDIR}/../meta-aws \
   

6. Append the following to local configuration(build/conf/local.conf)

   # Docker presumes systemd
   DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
   VIRTUAL-RUNTIME_initscripts = "systemd-compat-units"
   
   # add docker to the image
   DISTRO_FEATURES_append = " virtualization docker"
   IMAGE_INSTALL_append = "  docker"
   CORE_IMAGE_EXTRA_INSTALL_append = " kernel-modules"
   
   # Add Greengrass and required packages to the image
   IMAGE_INSTALL_append = " greengrass-bin texinfo sudo openjdk-8"
   EXTRA_IMAGE_FEATURES_append = " ssh-server-openssh"
   
   # Possible provider: cacao-initial-native and jamvm-initial-native
   PREFERRED_PROVIDER_virtual/java-initial-native = "cacao-initial-native"
   
   # Possible provider: cacao-native and jamvm-native
   PREFERRED_PROVIDER_virtual/java-native = "jamvm-native"
   
   # Optional since there is only one provider for now
   PREFERRED_PROVIDER_virtual/javac-native = "ecj-bootstrap-native"
   
   IMAGE_INSTALL_append = " python3-pip"
   

7. Build

   $ cd <target_board_name>
   $ source poky/oe-init-build-env
   $ bitbake core-image-minimal
   

   Building image can take up to a few hours depending on your host system performance.

   After the build has been completed successfully, you should see the output similar to:

   NOTE: Tasks Summary: Attempted 5383 tasks of which 5 didn't need to be rerun and all succeeded.
   

8. build products

   Bitbake has generated all the necessary files in the following directories:

   　<target_board_name>/build/tmp/deploy/images/<target_board_name>/

9. Write image to SD card

   Refer to Loading kernel and rootfs via eMMC/SD card.

   18px <translate> Note:</translate>The above example is for "weston", and should be replaced with "minimal".

Confirmation of Greengrass Core activation

1. Startup R-Car SK

   Insert the uSD card into the R-Car SK and connect the LAN cable.

   Connect the USB debug serial cable to the PC, and turn on the AC power.

   The PC will start the terminal software and make a serial connection.

   To turn on the power, press down the POWER SW (SW8).

   If the CCPF is attached, press the POWER SW (SW4) on the CCPF board.

2. Configure U-Boot to boot from SD card

   Refer to Configure U-Boot to boot from SD card.

3. Confirmation of Greengrass Core activation

   Make sure that Greengrass Core is Active.

   $ systemctl status greengrass.service --no-pager
   

   
   

Configure and check AWS IoT Greengrass

In subsequent chapters, the tags will be included in the title as follows.

• Work on the target board (R-Car SK) is described as <Terminal>.
• Work on the AWS Cloud is described as <Web>.
• Work on the host PC is described as <PC>.

1. <Web> Create an AWS account

   https://console.aws.amazon.com/console/home

   Create an AWS account and log in to the AWS Management Console as the root user.

2. <Web> Creating an IAM user

   Create an IAM user for work with reference to the following.

   https://docs.aws.amazon.com/ja_jp/IAM/latest/UserGuide/id_users_create.html#id_users_create_console

   The following policies should be attached at the time of creation.

   • AWSLambda_FullAccess
   • IAMFullAccess
   • AWSIoTFullAccess
   • AWSGreengrassFullAccess
   • IAMAccessAnalyzerFullAccess

   18px <translate> Note:</translate>Be sure to write down the "Access Key ID," "Secret Access Key," and "Password" that are displayed when users access the system. Note that the secret access key and password can only be confirmed at this time.

   Warning: These three items are very important information for security reasons, and should be managed with great care.

   Sign out the AWS root user and log in with the IAM user you created.

   https://console.aws.amazon.com/console/home

3. <Web> Setting up a region

   Set the desired region from the upper right corner of the screen.

   In this procedure, we will set "Asia Pacific(Tokyo) ap-northeast-1".

4. <Web><Terminal> Creating an IoT Core device

   Run "Confirmation of Greengrass Core activation" beforehand and make sure that the terminal can connect to the Internet by connecting a LAN cable.

   1. <Web> Select "IoT Core" from AWS services.

      
      

   2. <Web> Select "Greengrass" → "Getting started" from the side menu.
   3. <Web> Press "Set up one core device".
   4. <Web><Terminal> Follow the instructions on the screen to perform the following tasks.
      1. <Web> "Step 1: Register a Greengrass core device"

         Enter a core device name of your choice.

         In this procedure, the core device name shall be "R-CarH3SKDevice".

      2. <Web> "Step 2: Add to a thing group to apply a continuous deployment"

         Check "Enter a new group name" and enter a name of your choice.

         In this procedure, the thing group name shall be "R-CarH3SKDeviceGroup".

      3. <Terminal> "Step 3: Install the Greengrass Core software"
         1. "Step 3.1: Install Java on the device"

            No work required as it is included in the image generated by the R-Car SK build.

         2. "Step 3.2: Configure AWS credentials on the device"

            Execute the following command to set the information to access AWS as an environment variable.

            $ export AWS_DEFAULT_REGION=ap-northeast-1
            $ export AWS_ACCESS_KEY_ID=<ACCESS_KEY_ID>
            $ export AWS_SECRET_ACCESS_KEY=<SECRET_ACCESS_KEY>
            

         3. "Step 3.3: Run the installer"

            "Download the installer" requires no work.

            "Run the installer" does not match the path displayed in the web browser, so execute the following command in the terminal.

            $ java -Droot="/greengrass/v2" -Dlog.store=FILE -jar /greengrass/v2/alts/init/distro/lib/Greengrass.jar \
            --aws-region ap-northeast-1 --thing-name R-CarH3SKDevice --thing-group-name R-CarH3SKDeviceGroup \
            --component-default-user ggc_user:ggc_group --provision true --setup-system-service true \
            --deploy-dev-tools true
            

            Replace "R-CarH3SKDevice" and "R-CarH3SKDeviceGroup" with the values set in "Step 1" and "Step 2".

            When the above command is executed, the R-Car SK terminal communicates with AWS IoT and is registered as a Greengrass core device.

            After the command finishes, Greengrass CLI will be deployed to the terminal, and check if the following commands can be executed.

            $ /greengrass/v2/bin/greengrass-cli -V
            

            18px <translate> Note:</translate>You can check the deployment status in the following log.

            $ tail -f /greengrass/v2/logs/greengrass.log
            

5. <Web> Add policies to be sent to IoT Core

   Add a policy for the Lambda function running on the terminal to send messages to the IoT Core in the AWS cloud.

   1. Select "IAM" from AWS services.
   2. Select "Policies" from the side menu.
   3. Select "Create Policy".
   4. Select "IoT" for Service, "Publish" for Actions, and "All Resources" for Resources.

      
      

   5. Click on "Next: Tags" and "Next: Review", then enter any "Name" in the "Review policy", and click on the "Create Policy" button.

      In this procedure, the name shall be "R-CarH3SKDevice-IoT-Publish".

   6. Return to the IAM screen and select "Roles" from the side menu.
   7. Enter "GreengrassV2TokenExchangeRole" in the search and select the role with the same name.
   8. Click "Attach Policies", search for the policy you just created, and click "Attach Policy".
6. <Terminal> Preparing the execution environment for Lambda functions

   Build an environment to run the sample program on a terminal.

   Execute the following commands.

   $ mkdir -p /home/ggc_user
   $ chown ggc_user:ggc_group /home/ggc_user
   $ pip3 install numpy
   

7. <Web><PC> Creating a Lambda Function

   Create a Lambda function to deploy to the terminal.

   1. <PC> Run the following command to create a template for a Lambda function.

      $ mkdir -p ~/lambda && touch ~/lambda/lambda_function.py
      

   2. <PC> Copy and paste the following code into the lambda_function.py that you created.

      import datetime
      import json
      import numpy
      import boto3
      
      class DummyTemperatureSensor(object):
          def __init__(self, loc=25, scale=1, size=1):
              self.loc = loc
              self.scale = scale
              self.size = size
      
          def get_value(self):
              return numpy.random.normal(self.loc, self.scale, self.size)[0]
      
      
      def lambda_handler(event, context):
          print(event)
      
          # AWS IoT connection
          iot = boto3.client('iot-data', endpoint_url='https://xxxxxxxxxxxxxxx-ats.iot.ap-northeast-1.amazonaws.com')
      
          # get temperature
          sensor = DummyTemperatureSensor()
      
          # set publish paramater
          topic='topic/sensor/temperature'
          payload = {
              "timestamp": str( datetime.datetime.now() ), 
              "temperature": sensor.get_value()
          }
          print(payload)
      
          # publish to AWS IoT
          iot.publish(
                  topic=topic,
                  qos=0,
                  payload=json.dumps(payload, ensure_ascii=False)
              )
      

      18px <translate> Note:</translate>The following part of the above code needs to be replaced with its own endpoint.

      　　　　　　https://xxxxxxxxxxxxxxx-ats.iot.ap-northeast-1.amazonaws.com

      The endpoints can be found below.

      1. <Web> Select "IoT Core" from AWS services.
      2. <Web> Select "Settings" from the side menu.
      3. <Web> Refer to "Endpoint" in Device data endpoint.
   3. <PC> Zip the packages.

      Zip the lambda_function.py and boto3 packages you have created.

      $ cd ~/lambda
      $ sudo docker run --rm -v $(pwd):/var/task amazon/aws-sam-cli-build-image-python3.8:latest \
        pip install boto3 -t ./
      $ zip -r lambda_artifact.zip ./
      

      The zip file is copied to a PC that has access to the AWS cloud for uploading to the cloud.

8. <Web> Registering the Lambda function

   Register the created Lambda function to the AWS cloud.

   1. Select "Lambda" from AWS services.
   2. Select "Functions" from the side menu.
   3. Select "Create function".

      
      

   4. Check the "Author from scratch" box.
   5. Give an arbitrary name to the "Function name". In this procedure, "DummyTemperatureSensor" is used.
   6. Select "Python 3.8" as the Runtime.
   7. Select "arm64" as the Architecture.
   8. Other items keep the default values and press "Create Function".
   9. When the screen changes, from the right side of "Code source", select "Upload from" → ".zip file", and upload → save the zip file you created.
   10. Select "Actions" → "Publish new version" in the upper right corner and press Publish.
9. <Web> Deploying the Lambda function

   Deploy the registered Lambda function to the terminal.

   1. Select "IoT Core" from AWS services.
   2. Select "Greengrass" → "Components" from the side menu.
   3. Click on "Create component".

      
      

   4. Check the "Import Lambda function" box.
   5. Select the function that you created in "Lambda function".

      
      

   6. Enter "topic/sensor/get" in the "Topic" field of the Event sources.
   7. Select "AWS IoT Core MQTT" as the "Type" of the Event sources.

      
      

   8. Set the "Memory size" of Container parameters to "64 MB".

      
      

   9. Other items keep the default values and press "Create component".
   10. When the screen changes, press "Deploy" in the upper right corner.
   11. Select the group you created for the deployment destination and click "Next".
   12. Other items keep the default values and press "Next" several times, and press "Deploy" at the last confirmation screen.
10. <WEB> <Terminal> Operation check

    Confirm that the function you created will be deployed on the terminal side.

    Also, check the log on the terminal side to see if messages are being received by AWS IoT → terminal.

    1. <Terminal> Checking the operation of R-Car SK
       1. Confirm that the Lambda function has been deployed.

          Execute the following command to confirm.

          $ /greengrass/v2/bin/greengrass-cli component list
          

          When you execute the command the following log will be output, and you can see that the deployed function (DummyTemperatureSensor) is displayed in the list.

          root@m3ulcb:~# /greengrass/v2/bin/greengrass-cli component list
              :
              :
          
          Component Name: DummyTemperatureSensor
              Version: 1.0.0
              State: RUNNING
              Configuration: {"containerMode":"GreengrassContainer","containerParams":{"devices":{},"memorySize":64000.0,"mountROSysfs":false,"volumes":{}},"inputPayloadEncodingType":"json","lambdaExecutionParameters":{"EnvironmentVariables":{}},"maxIdleTimeInSeconds":60.0,"maxInstancesCount":100.0,"maxQueueSize":1000.0,"pinned":true,"pubsubTopics":{"0":{"topic":"topic/sensor/get","type":"IOT_CORE"}},"statusTimeoutInSeconds":60.0,"timeoutInSeconds":3.0}
          

       2. Check the log.

          Run the following command to check the log.

          $ cd /greengrass/v2/logs/
          $ tail -f <name of created function>.log
          

          Verify that the following log is output.

          serviceName=<name of created function>, currentState=RUNNING
          

    2. <Web><Terminal> Checking the operation of the AWS cloud
       1. <Web> Select "IoT Core" from AWS services.
       2. <Web> Select "Test" → "MQTT Test Client" from the side menu.
       3. <Web> 「Enter "topic/sensor/temperature" in the Topic filter and press "SUBSCRIBE".
       4. <Web> Enter "topic/sensor/get" in the "Topic name" of "Publish to a topic" and click "Publish.

          18px <translate> Note:</translate>The default message payload will be garbled on the terminal side, so change it to any alphanumeric text if necessary.

       5. <Terminal> The message received will be displayed in the log as follows.

          root@m3ulcb:/greengrass/v2/logs# tail -f DummyTemperatureSensor.log
          2021-12-06T04:50:40.524Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:17,{'message': 'AWS IoT ??????????????????????????????'}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING}
          2021-12-06T04:50:41.172Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:31,{'timestamp': '2021-12-06 04:50:41.168002', 'temperature': 27.381898093845034}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING}
          2021-12-06T04:51:14.939Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:17,{'message': 'hello world'}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING}
          2021-12-06T04:51:14.969Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:31,{'timestamp': '2021-12-06 04:51:14.964931', 'temperature': 25.003559249938885}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING
          

       6. <Web> A message is received in the subscription.