GoPiGo - Learn Google Cloud by doing

Ignazio Pollina

Approaching to a new programming language will often mean to run an "Hello World" class, as well as learning different Cloud solutions will mean starting from the basics. What about creating a mini robot instead? Because at Google, they made it differently, with an 8 hours workshop and just a requirement: a computer. The goal of the day was to work in team to build a self-driving robot car by scratch using a Raspberry Pi 3, GoPiGo hardware components furnished by Dexter industries, that plays ball game against other robots, processing inputs via Google Cloud components, making decisions on the car movements and send it back to the car.

To make the process smoother, Google provisioned the code and instructions to the teams.

The GoPiGo Architecture shows how modules interact with the software running on the car via PubSub and IoT Core. A wide angle camera and a laser sensor are being used to collect information from the car. Here some of the main features:

Driving controller that use Google Machine Learning to make driving decisions based on information gathered by the car (camera and sensor)
TensorFlow Object Detection API to process images, to detect the locations of coloured balls
The software on the GoPiGo car runs into the Raspberry Pi and uses Google Cloud IoT Core to communicate to the Cloud Derby modules running on GCP.

Here are the high level steps to install and setup the game:

Initial Project Setup
Deploy and start Object Detection Module on the cloud
Deploy and start Control Module on the cloud
Provision and start GoPiGo Car software on the car
Start the game using the Game Control web App

Project Setup

Google Cloud will host the infrastructure to setup the initial environment to deploy and develop project modules. First thing to do is to create a new project:

Machine Learning

Data preparation is the most important part of the project, as it will drive the car along the field to detect objects that have been annotated. An example here of real time logs:

Google had used and modified a module that instruct an object detector to use the TensorFlow Object Detection API with cats and dogs images. Here the link to the GitHub project: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_pets.md

To get the Machine Learning to work for our robot, here are the steps:

Take images of balls and letters in different light conditions, different distances, and annotate them with bounding boxes and labels. These pictures are then uploaded to a bucket.

Train an object detector using TensorFlow Object Detection API using the transfer learning method
Use the trained model to recognise c0lour and location of balls in images taken by the car's camera

ML Model Training

Transfer learning with TensorFlow (pre-trained neural network) is used for this module based on Pets Tutorial helping us saving huge amount of time of compute tasks. Here a GCP guide too.

Driving Controller

This section will make use of the Driving Controller software module responsible for consuming car's sensor information to make driving decisions and send commands back to the car. An example is to calculate the angle and distance to the object.

The Driving Controller running on the AppEngine, will be set to use full self driving mode using its web app served by a NodeJs, process incoming messages from the car sensors, invoke remote inference runtime using REST API, create control messages and send them back to the car via PubSub. Car software will subscribe the PubSub topic where these control messages are published, interpret commands and issue signals to the car hardware.

GoPiGo Car Assembly

As a team, we decided to work in parallel on both the software and the car assembly to save time. The car's software run on a Raspberry Pi 3 and uses GCP IoT to communicate to the Cloud Derby modules running on GCP. Here some pictures before and after the car assembly:

Final Thoughts

At the end of the day, teams run a game with four different cars, each set to capture a specific coloured ball and bring it back to the base. This has result with lots of fun, shared great time with amazing people, new skills and new ideas, and a 3rd place, gifted by Google with a nice cup and thermos :)

Here some pictures of the day:

Links:

Google Cloud Derby: https://www.cloudderby.io/