MoCap4ROS2 Setup

Learn how to use an OptiTrack system to control a robotic arm.

Arm Follower Demo Using the Hiwonder JetMax Arm

This demo uses the mocap4ros2_optitrack package to configure the Hiwonder JetMax arm's end effector to follow an arbitrary rigid body using OptiTrack cameras and Motive software.

See our OptiTrack Robot Applications document for robotics use case examples.

About this Demo

This demo provides an example workflow to use an OptiTrack system to control a robotic arm. The demo uses a 3DOF robot arm, the Hiwonder JetMax. This robot was selected for its affordability, desktop size, and ROS2 support, with minimal setup required to get up and going. The provided packages from JetMaxRoboticArm work well out of the box.

The package provided by Hiwonder to control the arm with a python package, jetmax_ebap, can also be adapted into a ROS package fairly easily.

While the scope of this example is limited to the specified hardware, similar concepts should work with other robotic arms.

Setup Requirements

Required Hardware

• A computer capable of running Motive and at least 3 OptiTrack cameras to serve as the Motive PC. Please see the Host PC Requirements section of the Installation and License Activation page.

• A second computer running Ubuntu 20.04 or greater (the Ubuntu companion computer). Use a separate PC rather than an instance of WSL2 on the Motive PC. For more information, please see the section Unable to Use WSL2, below.

• A JetMax Hiwonder Robot.

• Two rigid bodies and a calibration square.

• A router or switch with an internet connection, separate from the OptiTrack switch.

• A second Network Interface Card (NIC) for the Motive PC, to connect to the Arm Follower network.

• An ethernet cable to connect the JetMax to the internet-connected router.

Required Software

Visit the OptiTrack Downloads page for the following software packages:

• Motive 3.1 (or above), installed on the Host PC.

• jetmax_optitrack_feedback package, installed on the JetMax's Jetson Nano.

• mocap4ros2_optitrack, installed on the Ubuntu Companion computer (originally from https://github.com/OptiTrack/mocap4ros2_optitrack).

• mocap_msgs, installed on the Ubuntu Companion computer (originally from https://github.com/OptiTrack/mocap_msgs).

• tf_repub, installed on the Ubuntu Companion computer.

Connect Hardware

This system requires two separate networks, using independent switches, to function:

• One switch to network the JetMax, Ubuntu Companion Computer, and Motive PC. This switch can optionally be connected to the internet.

• The OptiTrack camera network, with only the cameras and the Motive PC connected. For more details about configuring the camera network, please see the Ethernet Camera Network Setup chapter.

The diagram below shows how to network the necessary devices together:

Networking configuration.

The Ubuntu companion computer is required to pass data from the Motive PC to the JetMax.

Please see the section Need for companion computer, below, for more information.

Ubuntu Companion Computer Configuration

We tested using Ubuntu 22.04 with ROS2 Iron. Ubuntu 20.04+ with ROS2 Foxy or newer should also work adequately.

Some familiarity with ROS will help in later steps. Tutorials for using ROS2 Iron are available from the docs.ros.org website.

ROS2 versions are dependent on your OS so pay attention to the version you install.

For installation instructions for Ubuntu 22.04, please see https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html

Create Package and Setup Config Files

cd ~
git clone https://github.com/OptiTrack/robot_arm_follower_demo.git
mkdir -p ~/optitrack_companion_ws/src
cd ~/optitrack_companion_ws/src
# copy mocap_msgs, mocap4ros2_optitrack, and tf_repub to the src directory
cp -r ~/robot_arm_follower_demo/packages_on_ubuntu_companion/* ~/optitrack_companion_ws/src
cd ~/optitrack_companion_ws
source /opt/ros/<your-ros-distro>/setup.bash
colcon build

If the build was successful, the optitrack_companion_ws will be populated with build, install, and log directories.

JetMax Computer Configuration

There are two methods for operating the Jetson Nano on the JetMax robot:

1. Using the SD card image that ships with the JetMax, running Ubuntu 18.04.

2. Using an SD card image with a fresh install of Ubuntu 18.04 supported by NVIDIA.

While Ubuntu 18.04 is no longer a supported OS, it is the most recent Ubuntu version that NVIDIA supports for the JetMax. You may be able to run the JetMax using a more recent version of Ubuntu, however that configuration will not be supported by NVIDIA.

If using the SD card image that ships with the JetMax, skip to the Build the Code section.

Install Ubuntu 18.04

1. Reflash the SD following the SD card Getting Started guide from Nvidia

2. Download the jetson expansion board access package on the home directory and install:

   git clone https://github.com/JetMaxRoboticArm/jetmax_ebap.git
   cd jetmax_ebap
   python3 setup.py install

3. Test access to the board using the jetmax_ebap readme. If you receive permissions errors, try using sudo python3 before running commands to see if this allows you to access the GPIO.

4. If permission errors persist, use the chmod or adduser commands to update or add access.

5. Install ROS2 Foxy from https://docs.ros.org/en/foxy/Installation.html.

While ROS2 Foxy is no longer a supported version, it is the version required for Ubuntu 18.04.

For our testing, we flashed the SD card with the official image for the Jetson Nano, which contains Ubuntu 18.04, a version that is end of life and losing support. We installed ROS2 Dashing and, because it is losing support, transform (tf2) messages are not supported. We implemented a workaround to republish the tf2 messages with tf_repub package installed on the Ubuntu companion computer.

To run the entire system on the JetMax, either:

1. Update the OS

2. Develop a solution to use tf2_ros. Perhaps use a C++ executable after building tf2 from source or figure out how to install the tf2_ros python package.

3. Rtun the system without using tf2 messages.

Build the Code

Create the package and setup configuration files.

cd ~
git clone https://github.com/OptiTrack/robot_arm_follower_demo.git
mkdir -p ~/optitrack_robot_ws/src
cd ~/optitrack_robot_ws/src
# copy mocap_msgs, mocap4ros2_optitrack, and tf_repub to the src directory
cp -r ~/robot_arm_follower_demo/packages_on_jetmax/* ~/optitrack_robot_ws/src
cd ~/optitrack_robot_ws
source /opt/ros/<your-ros-distro>/setup.bash
colcon build

OptiTrack Configuration

Please see our Quick Start Guide for detailed instructions to setup an OptiTrack camera system and install the Motive software.

For an easier setup process, keep track of the orientation of the calibration square during the initial configuration. The orientation of the JetMax should match the following image when using a calibration square:

JetMax oriented with the calibration square.

Create Rigid Bodies

For instructions to create rigid bodies, please see the Rigid Body Tracking page.

Create two rigid bodies:

• end_effector - mount this to the end effector of the arm.

• base_link - place this near the robot arm. The exact position or how the rigid body is attached does not matter as long as it's secured and visible to the cameras.

Enable NatNet Streaming

For a detailed overview of streaming, please see the pages in our NatNet chapter.

• Click the streaming button in the lower right corner of the Motive Control Deck to open the Applications Settings panel to the Streaming tab.
• Click to enable NatNet streaming.

• Set the local interface to an IP on the same network as the Ubuntu Companion computer.

Run the Program

With Motive streaming, launch the Ubuntu Companion Computer:

# terminal 1
cd ~/optitrack_companion_ws
source install/setup.bash
ros2 launch mocap_optitrack_driver optitrack2.launch.py

# terminal 2
cd ~/optitrack_companion_ws
source install/setup.bash
ros2 run tf_repub transform_republisher

Launch the JetMax Arm.

We recommend using SSH to connect to the JetMax for this step.

# terminal 1
cd ~/optitrack_robot_ws
source install/setup.bash
ros2 run jetmax_optitrack_feedback jetmax_feedback_node

Ensure the base_link rigid body is near the robot.

# terminal 2
cd ~/optitrack_robot_ws
source install/setup.bash
ros2 topic pub /get_target_pos std_msgs/msg/Bool "data: True" --once

Start moving the base link rigid body to have the robot follow!

What is going on?

In this demo, the JetMax maintains a static distance between its end effector and the target rigid body. This is done by constantly measuring the distance between the end_effector and base_link Rigid Bodies with OptiTrack. The robot sees the change in distance and adjusts its position to compensate.

The program issues a cartesian control command to the robot, then an inverse kinematics engine computes the target joint angles for the robot given a description of the robot's mechanics.

Take a look at the links provided to learn more about locomotion for a robot arm using ROS.

A more comprehensive example of Standard ROS practice can be found here: Universal_Robots_ROS2_Gazebo_Simulation

Issues and Possible Improvements

Need for Companion Computer

The Ubuntu companion computer is needed because the NatNet binary is not yet available for ARM devices. This means that the mocap4ros2_optitrack package had to run on a computer that did not have an ARM architecture.

When the ARM library for NatNet becomes available, mocap4ros2_optitrack will run on an ARM device and a companion computer will no longer be needed.

Unable to use WSL2

The Ubuntu companion computer cannot be a WSL2 instance of the Windows host running Motive because it is difficult to network the WSL2 instance to be on the same LAN as the robot arm computer. By default, the network of the WSL2 instance is a NAT subset defined by the host windows machine. This link may be useful for giving a static IP on the same network as the Windows host, although this is untested: assigning-static-ip-addresses-in-wsl2.md

The mocap4ros2_optitrack plugin could run on a WSL2 standalone PC because Motive and the WSL2 instance can easily exist on the network created by the windows host.

Orientation of the JetMax

The orientation of the JetMax should not need to be fixed. Ideally, in the future, a few markers placed on the base of the robot would allow the calculated target position to account for the orientation of the robot instead of assuming orientation.