CENTERSTAGE TFOD-prop Autonomous Program

This is a tutorial on creating a TensorFlow FIRST Tech Challenge (FTC) autonomous program for the CENTERSTAGE game.

Visit the FTC docs page on Creating Op Modes in blocks if you need help getting started and to create your first driver controlled program.

We will use TensorFlow to detect a Team Prop on the randomized spike mark and also park backstage. We’ll make use of motor encoders to make our robot moves more accurate.

Autonomous Blocks Program Tutorials

The autonomous period include 20 bonus points that can be scored if you are able to use a Team Prop and correctly place the purple and yellow pixels. Assuming you already have a program that recognizes the pixel on the spike mark, now we can change that program to detect the team prop instead and earn bonus points.

Prerequisites/Assumptions

This tutorial assumes:

a robot with two driving wheels, possibly a basic robot from the FIRST robot building resources page. These are sometimes called pushbots;
some way to push pixels around. This robot has a square opening at the front into which a pixel can fit;
A webcam is required for this program, ideally mounted near the top front of your robot. See the FTC Docs page on webcams for more information;
some familiarity with Blocks, possibly a Blocks tutorial;
you’ve created the TFOD-pixel autonomous program. That program detected the pixel on the randomized spike mark and parked backstage.
you’ve created Team Props and created a custom TensorFlow model to detect those props. You also had to prepare videos for up-close TensorFlow detection.

You can probably follow along even if you’re new to Blocks, however this tutorial doesn’t explain how to program in Blocks.

Plan

The plan is to started aligned with the rear edge of tile A4 with rear of the robot flat against the field wall. We then have a complicated multipart plan with many steps and some tests along the way that change where we go next. We’ll develop the program in stages as there are too many unknown details at this stage. This is basically the same plan as TFOD-pixel, but we use team props instead.

The basic plan

move closer to the left spike mark (as looking at the marks from the robot position);
then use TensorFlow to check if there is a team prop there;
If there is a team prop, drive forward and drop off the purple pixel on the left mark and turn and drive to the backstage corner tile A6;
If there is no team prop on the left mark, we turn and check the center spike mark. If the team prop is on the center mark, drive forward and drop off the purple pixel and then turn and park on tile A6;
If the team prop was not found so far we will assume it is on the right side spike mark. Turn toward that mark, drive forward and drop off the purple pixel on the right mark and then turn and park on tile A6.

CENTERSTAGE TensorFlow for Team Props

Obviously you need Team Props to get the bonus points. See the creating team props page for information on creating you own team props. For this program we are using Red and Blue Duplo props.

You should have taken videos of the team props in various sizes, orientations, background and lighting and then trained a custom TensorFlow model. In this case the videos were taken by taking the TFOD-pixel1 program and adding stops after each move where the program pauses to look at the spike mark. Remember TFOD-pixel has to move up close to each spike mark and check each spike mark in turn. So we stop the program as it is looking at each spike mark and then we take a set of videos up close of each team prop.

The above image is a video frame from one of the videos.

The resulting TensorFlow model was downloaded and saved a file with name: redBlueDuploPlain.tflite. Because we’re using Blocks, the file was uploaded to the Robot Controller with that name.

TFOD-prop

Turn your robot on, connect to the Blocks programming environment. Then copy your TFOD-pixel program. My last version of that program was called TFOD-pixel3, so I copied that program to create TFOD-prop3.

Find the initTFOD function and change it as follows:

disable or delete the easyCreateWithDefaults block
Add the indicated MyTfodProcessorBuilder blocks, specify your model file name in the setModelFileName block
create a list with the labels that in in the model file and use in the setModelLabels block
add a myTfodProcessorBuilder.build block which should include a set.MyTfodProcessor block.
optional: adjust the setMinResultConfidence value. It defaults to 0.75, you might need to set it lower to detect your props.

In fact, that’s likely the only REQUIRED change for this program to work. But you could go through the program and replace “pixel” with “prop”. As an example, we created a detectProp function and copied the detectPixel blocks into it with just a few pixel to prop name changes such the propFound variable instead of the pixelFound variable.

Here’s the completed TFOD-prop3 program:

Here’s a video of the program running if the team prop is on the center spike mark.

Next Steps

You could copy the program to make versions that worked on the other three starting positions.

If you had a way to deploy the yellow pixel onto the backdrop, you probably want to use April Tag driving to position in front of the backdrop area that corresponds to the randomized spike mark to get bonus points. From the rear side of the field you probably could use encoder based movement.

Getting Help

It is often possible to use Google (or other search engine) to get help or solve problems. There are lots of resources online. If you’re still stuck you can ask for help here.