MoCap4ROS2 Setup
Learn how to use an OptiTrack system to control a robotic arm.
Last updated
Was this helpful?
v3.2
Learn how to use an OptiTrack system to control a robotic arm.
Last updated
Was this helpful?
This demo uses the mocap4ros2_optitrack package to configure the Hiwonder JetMax arm's end effector to follow an arbitrary rigid body using OptiTrack cameras and Motive software.
See our document for robotics use case examples.
The source code for this tutorial is located at .
This demo provides an example workflow to use an OptiTrack system to control a robotic arm. The demo uses a 3DOF robot arm, the Hiwonder JetMax. This robot was selected for its affordability, desktop size, and ROS2 support, with minimal setup required to get up and going. The provided packages from work well out of the box.
The package provided by Hiwonder to control the arm with a python package, , can also be adapted into a ROS package fairly easily.
While the scope of this example is limited to the specified hardware, similar concepts should work with other robotic arms.
A computer capable of running Motive and at least 3 OptiTrack cameras to serve as the Motive PC. Please see the section of the page.
A second computer running Ubuntu 20.04 or greater (the Ubuntu companion computer). Use a separate PC rather than an instance of WSL2 on the Motive PC. For more information, please see the section , below.
A .
Two and a .
A router or switch with an internet connection, separate from the OptiTrack switch.
A second Network Interface Card (NIC) for the Motive PC, to connect to the Arm Follower network.
An ethernet cable to connect the JetMax to the internet-connected router.
Motive 3.1 (or above), installed on the Host PC.
jetmax_optitrack_feedback package, installed on the JetMax's Jetson Nano.
mocap4ros2_optitrack, installed on the Ubuntu Companion computer (originally from https://github.com/OptiTrack/mocap4ros2_optitrack).
mocap_msgs, installed on the Ubuntu Companion computer (originally from https://github.com/OptiTrack/mocap_msgs).
tf_repub, installed on the Ubuntu Companion computer.
This system requires two separate networks, using independent switches, to function:
One switch to network the JetMax, Ubuntu Companion Computer, and Motive PC. This switch can optionally be connected to the internet.
The diagram below shows how to network the necessary devices together:
We tested using Ubuntu 22.04 with ROS2 Iron. Ubuntu 20.04+ with ROS2 Foxy or newer should also work adequately.
ROS2 versions are dependent on your OS so pay attention to the version you install.
If the build was successful, the optitrack_companion_ws will be populated with build, install, and log directories.
There are two methods for operating the Jetson Nano on the JetMax robot:
Using the SD card image that ships with the JetMax, running Ubuntu 18.04.
While Ubuntu 18.04 is no longer a supported OS, it is the most recent Ubuntu version that NVIDIA supports for the JetMax. You may be able to run the JetMax using a more recent version of Ubuntu, however that configuration will not be supported by NVIDIA.
Reflash the SD following the SD card Getting Started guide from Nvidia
Download the jetson expansion board access package on the home directory and install:
If permission errors persist, use the chmod or adduser commands to update or add access.
While ROS2 Foxy is no longer a supported version, it is the version required for Ubuntu 18.04.
To run the entire system on the JetMax, either:
Update the OS
Develop a solution to use tf2_ros. Perhaps use a C++ executable after building tf2 from source or figure out how to install the tf2_ros python package.
Rtun the system without using tf2 messages.
Create the package and setup configuration files.
For an easier setup process, keep track of the orientation of the calibration square during the initial configuration. The orientation of the JetMax should match the following image when using a calibration square:
Create two rigid bodies:
end_effector - mount this to the end effector of the arm.
base_link - place this near the robot arm. The exact position or how the rigid body is attached does not matter as long as it's secured and visible to the cameras.
Click to enable NatNet streaming.
Set the local interface to an IP on the same network as the Ubuntu Companion computer.
With Motive streaming, launch the Ubuntu Companion Computer:
Launch the JetMax Arm.
We recommend using SSH to connect to the JetMax for this step.
Ensure the base_link rigid body is near the robot.
Start moving the base link rigid body to have the robot follow!
In this demo, the JetMax maintains a static distance between its end effector and the target rigid body. This is done by constantly measuring the distance between the end_effector and base_link Rigid Bodies with OptiTrack. The robot sees the change in distance and adjusts its position to compensate.
Take a look at the links provided to learn more about locomotion for a robot arm using ROS.
The Ubuntu companion computer is needed because the NatNet binary is not yet available for ARM devices. This means that the mocap4ros2_optitrack package had to run on a computer that did not have an ARM architecture.
When the ARM library for NatNet becomes available, mocap4ros2_optitrack will run on an ARM device and a companion computer will no longer be needed.
The orientation of the JetMax should not need to be fixed. Ideally, in the future, a few markers placed on the base of the robot would allow the calculated target position to account for the orientation of the robot instead of assuming orientation.
Visit the page for the following software packages:
The OptiTrack camera network, with only the cameras and the Motive PC connected. For more details about configuring the camera network, please see the chapter.
Please see the section , below, for more information.
Some familiarity with ROS will help in later steps. are available from the docs.ros.org website.
For installation instructions for Ubuntu 22.04, please see
Using an SD card image with a fresh install of supported by NVIDIA.
If using the SD card image that ships with the JetMax, skip to the section.
Test access to the board using the . If you receive permissions errors, try using sudo python3 before running commands to see if this allows you to access the GPIO.
Install ROS2 Foxy from .
For our testing, we flashed the SD card with the official , which contains Ubuntu 18.04, a version that is end of life and losing support. We installed ROS2 Dashing and, because it is losing support, transform (tf2) messages are not supported. We implemented a workaround to republish the tf2 messages with tf_repub package installed on the Ubuntu companion computer.
Please see our for detailed instructions to setup an OptiTrack camera system and install the Motive software.
For instructions to create rigid bodies, please see the page.
For a detailed overview of streaming, please see the pages in our .
Click the streaming button in the lower right corner of the Motive Control Deck to open the panel to the .
The program issues a to the robot, then an engine computes the for the robot given a .
A more comprehensive example of Standard ROS practice can be found here:
The Ubuntu companion computer cannot be a WSL2 instance of the Windows host running Motive because it is difficult to network the WSL2 instance to be on the same LAN as the robot arm computer. By default, the network of the WSL2 instance is a NAT subset defined by the host windows machine. This link may be useful for giving a static IP on the same network as the Windows host, although this is untested:
