Overview

This page provides instructions on how to use the OptiTrack Unreal Engine Live Link plugin. The plugin communicates with Unreal's built-in Live Link system by providing a Live Link source for receiving tracking data streamed from Motive. This plugin can be used for controlling cameras and objects in virtual production applications. When needed, the OptiTrack Unreal Engine Plugin can also be alongside this plugin. For a specific guide to InCamera VFX (i.e. LED Wall Virtual Production) please see this wiki page Unreal Engine: OptiTrack InCamera VFX.

Setup

1. [Motive] Setup rigid body streaming in Motive.

Get Motive streaming with at least one rigid body or Skeleton asset. Make sure the Streaming settings are configured correctly, and the asset is active under the Assets pane.

2. [UE] Install the OptiTrack plugins in Unreal Engine (UE).

You can install the OptiTrack Unreal Engine plugin by putting the plugin files into one of the following directories:

• A global engine plugin can be placed in C:\Program Files\Epic Games\[Engine Version]\Engine\Plugins

• A project-specific plugin can be placed in [Project Directory]\Plugins

3. [UE] Enable the plugins in UE project.

Go to Edit → Plugins and enable two of the required plugins. First one is the OptiTrack - Live Link plugin under Installed group, and the second one is the built-in Live Link plugin under Built-In group.

4. [UE] Open the LiveLink pane

Open the LiveLink pane from Window → Virtual Production → Live Link in the toolbar.

5. [UE] Configure and create a new OptiTrack source

In the LiveLink pane under Source options, go to the OptiTrack Source menu and configure the proper connection settings and click Create. Please make sure to use matching network settings configured from the Streaming pane in Motive.

6. [UE] Check the Connection.

If the streaming settings are correct and the connection to Motive server is successful, then the plugin will list out all of the detected assets. They should have green dots next to them indicating that the corresponding asset has been created and is receiving data. If the dots are yellow, then it means that the client has stopped receiving data. In this case, check if Motive is still tracking or if there is a connection error.

Using the Plugin

Static Meshes or Camera Actors

1. Add the camera object or static mesh object that you wish to move

Add a camera actor from the Place Actors pane or a static mesh from the project into your scene. For the static meshes, make sure their Mobility setting is set to Movable under the Transform properties.

2. Add a LiveLinkController Component

Select an actor you want to animate. In the Details tab select your "actor" (Instance). In the search bar, type in Live Link. Then click on the Live Link Controller from the populated list.

3. Select the target rigid body

Under the Live Link properties in the Details tab click in the Subject Representation box and select the target rigid body.

4. Check

Once the target rigid body is selected, each object with the Live Link Controller component attached and configured will be animated in the scene.

Timecode Setup

When the camera system is synchronized to another master sync device and a timecode signal is feeding into eSync 2, then the received timecode can be used in UE project through the plugin.

1. Set Timecode Provider under project settings

From Edit → Project Settings, search timecode and under Engine - General settings, you should find settings for the timecode. Here, set the the Timecode Provider to LiveLinkTimeCodeProvider.

2. Set OptiTrack source in the Live Link pane as the Timecode Provider

Open the Live Link pane, and select the OptiTrack subject that we created when first setting up the plugin connection. Then, under its properties, check the Timecode Provider box.

3. Check

The timecode from Motive should now be seen in the Take Recorder pane. Take Recorder pane can be found under Window → Cinematics → Take Recorder in the toolbar.

Skeletons

1. Create a new Animation Blueprint

Right click the mesh you would like to use and select "Create > Anim Blueprint"

2. Name and Open the Animation Blueprint

Name the animation blueprint something appropriate, then double click it to open the blueprint.

3. Hook up your Blueprint

Create a "Live Link Pose" component and connect it to the "Output Pose". Assign the "Live Link Subject Name" to the Skeleton that you would like to use.

Change the "Retarget Asset" property in the Details pane of the blueprint editor to "OptiTrackLiveLinkRetarget"

4. Getting the Skeleton to Animate

To animate the Skeleton in real time click the Animation Blueprint from earlier. In the Details pane under the skelteonLive Link Skeleton Animation". After you add that component the mesh should start animating.

To animate the Skeleton in a game, just press the play button. Adding the "Live Link Skeleton Animation" object is not necessary to animate in play mode.

Debugging Note

If the retargeting doesn't match the mesh correctly, then you can create a new OptiTrackLiveLinkRetarget blueprint from scratch and modify the bone mapping names.

MetaHumans

Animating a MetaHuman follows basically the same steps as another Skeleton, but requires hooking into the Skeleton at a very specific location. For more information about MetaHuman setup outside of our scope, please visit Epic Games's website.

1. Find the Skeletal Mesh

Navigate to the Skeletal Mesh for your MetaHuman. This is typically located in a folder such as MetaHumans > [Name] > [Female/Male] > [Height] > [Weight] > Body. Double click the Skeletal Mesh to open the blueprint.

2. Open the AnimGraph Tab

Click the "Blueprint" option on the top bar of new window. In the bottom left corner navigate to My Blueprint > Animation Graphs > AnimGraph and double click.

3. Hook up your Blueprint

You'll see a very complex AnimGraph already setup. Make a new Live Link Pose object like in the Skeleton creation steps. Connect the Input Pose or Control Rig to the input of the Live Link Pose. Connect the output of the Live Link Pose to the Output Pose AnimGraph object.

4. Retarget in the Details Tab

The last step in this window is to set the Retarget Asset to OptiTrackLiveLinkRetarget for the Live Link Pose node. To do this, simply click on the dropdown in the Details tab and click the dropdown under Retarget Asset and select OptiTrackLiveLinkRetarget. After it has been set, click Compile in the top left of this window. You may now close this window and move on to the next steps.

5. Level of Detail (LOD)

MetaHumans will change their Level of Detail (LOD), i.e. how complex the asset is, based on how far the camera is from the actor among other factors. In order for things to animate correctly the Forced LOD must be a minimum of 1 (default of -1). To change this setting, click on the (Instance) you wish to change in the Details tab on the main workspace window. Below the selected (Instance) select the LOD tab. From here you can change the value in the Forced LOD Model field.

6. Animate your MetaHuman

At this point if you drag the base MetaHuman object into the scene then it will animate like other Skeletons.

Standalone Game Mode

For testing the project in standalone game mode, or when developing an nDislay application, the Live Link plugin settings must be saved out and selected as the default preset to be loaded onto the project. If this is not done, the configured settings may not get applied. After configuring the LiveLink plugin settings, save out the preset from the Live Link pane first. Then, open the Project Settings and find Live Link section in the sidebar. Here, you can select the default Live Link preset to load onto the project, as shown in the screenshot below. Once the preset is properly saved and loaded, the corresponding plugin settings will be applied to the standalone game mode.

If all the configuration is correct, the actors will get animated in the newly opened game window when playing the project in the standalone game mode.

MotionBuilder Retargeting

Another path to get data into Unreal Engine is to stream data from Motive -> MotionBuilder (using the OptiTrack MotionBuilder Plugin) -> Unreal Engine (using the Live Link plugin for MotionBuilder). This has the benefit of using the Human IK (HIK) retargeting system in MotionBuilder, which will scale characters of very different sizes/dimensions better than the base Live Link plugin. More information can be found by consulting Unreal Engine's Live Link to MotionBuilder documentation.

Troubleshooting

Plugin Overview

The OptiTrack Unreal Engine Plugins allow you to stream real-time tracking data from Motive into Unreal Engine. This includes tracking data of Rigid Bodies, Skeletons, and HMDs that are tracked within Motive. This article focuses on the organization of those plugins. For basic instructions on setting up a motion capture system, please refer to the Getting Started guide instead.

Included Plugins:

• OptiTrack - Live Link

• OptiTrack - Streaming Client

• OptiTrack - VR Latency Optimization

HMD Compatibility

• A variety of head mounted displays (HMDs) can be integrated using the OptiTrack OpenVR Driver.

• For plugin version 1.23 and above, support for Oculus HMDs has been deprecated.

Motive Data Streaming Setup (Server)

First, you'll want to follow the below instructions to set up the data streaming settings in Motive. Once this is configured, Motive will be broadcasting tracking data onto a designated network interface where client applications can receive them.

Streaming Settings

Open the Data Streaming Pane in Motive's Settings window and set the following settings:

• Enable - Turn on the Enable setting at the top of the NatNet section.

• Local Interface - Choose the desired IP network address from this dropdown to stream data over.

  • Loopback

    • This is the local computer IP address (127.0.0.1 or Localhost).

    • Used for streaming data locally on the PC you are running Motive on that does not interact with the LAN.

    • Good option for testing network issues.

  • 192.168.0.1x (typical, but may be different depending on which interface is used to establish a LAN connection)

    • This IP address is the interface of the LAN either by Wi-Fi or Ethernet.

    • This will be the same address the Client application will use to connect to Motive.

• Transmission Type

  • For streaming over a Wi-Fi network, setting the Transmission Type to Unicast is strongly advised.

• Select desired data types to stream under streaming options:

  • Rigid Bodies - Enabled (required).

  • Skeletons - Optional for Skeleton tracking.

  • Markers (Labeled, Unlabled, Asset) - Disabled for HMDs (advised).

  • Devices - Disabled.

• Skeleton Coordinates

  • Set to Local.

• Bone Naming Convention

  • Set the appropriate bone naming convention for the client application. For example, if the character uses the FBX naming convention, this will need to be set to FBX.

Plugins

Unreal Engine 4

OptiTrack - Live Link ver. 2.27

The Unreal Engine: OptiTrack Live Link Plugin is primarily intended for Virtual Production applications that require use of real-time cameras or Skeleton tracking in Unreal Engine. It interfaces with Unreal Engine's built-in Live Link plugin to allow for for streaming data from Motive in both real-time and runtime. When timecode data is available in Motive, streamed timestamps can be used to align the data with other components within the project.

OptiTrack - Streaming Client ver. 2.27

The Unreal Engine: OptiTrack Streaming Client Plugin is primarily used for Virtual Reality due to it's low latency optimizations. It provides streaming of Rigid Body and Skeleton tracking data from Motive into Unreal Engine, and it also includes helper functions and actors to provide quick integrating of the plugin into the scene.

OptiTrack - VR Latency Optimization ver. 2.27

The OptiTrack - VR Latency Optimization plugin is an optional plugin for SteamVR projects. When enabled, it provides HMD render compensation to minimize latency in VR applications. For users developing for SteamVR using the OptiTrack plugin, whether or not using the OptiTrack OpenVR Driver, it's suggested to have this plugin enabled.

Unreal Engine 5

OptiTrack - Streaming Client ver. 3.0

This version of the plugin is compatible with Unreal Engine 5.

OptiTrack - Streaming Client ver. 3.0

This version of the plugin is compatible with Unreal Engine 5.

Integrating HMDs

OptiTrack motion capture systems can be used to track head mounted displays (HMD) and integrate the tracking data into Unreal Engine for VR applications. For instructions on integrating HMD tracking data into Unreal Engine, please refer to the corresponding page:

• Unreal Engine: HMD Setup

Wireless Multiplayer Setup

When setting up multiplayer games on wireless clients, it is more beneficial for each client to make direct connection to both the tracking-server (Motive) and the game-server, rather than rebroadcasting the streamed tracking data through the game-server. Then, any of the game related actions that interacts with the tracking data can be processed on the game-server, and this server can send out the corresponding updates to the wireless clients. This allows the wireless clients to only receive both the tracking data or updates without having to send back any information; in other words, minimizing the number of data transfers needed. If wireless clients are sending data there will be a minimum of two transfers on the wireless network, and each transfer of data through wireless network is at risk of latency or lost packets.

Overview

The OptiTrack VR driver lets you stream tracking data of the head-mounted display (HMD) and the controllers from Motive into SteamVR. The plugin ships in an installer package (MSI), and it will set up the driver along with a tool for configuring streaming settings. Using this, the tracking data from Motive can be used to override the tracking of the VR system in any applications that are compatible with SteamVR.

Requirements

• Motive version 2.2 or higher

• SteamVR installed on the host computer.

HMD Setup

Place Markers on the HMD

For the camera system to track the HMD, a set of markers must be attached to the HMD. You can either use the active markers (Active HMD clip or Active Tags) or the passive markers. Passive markers are retroreflective markers that reflect infrared light emitted from the IR LEDs on the camera. On the other hand, the active markers are LED markers that emit the IR light and has the intelligence to be uniquely identified.

When using the active markers, you can conveniently put a set of 8 markers onto the HMD by using the HMD Clip, or you can attach the markers from the Tag manually onto the HMD using adhesives and marker posts.

Create HMD Rigid Body in Motive

For using OptiTrack system for VR applications, it is important that the pivot point of HMD Rigid Body gets placed at the appropriate location, which is at the root of the nose in between the eyes. When using the HMD clips, you can utilize the HMD creation tools in the Builder pane to have Motive estimate this spot and place the pivot point accordingly. It utilizes known marker configurations on the clip to precisely positions the pivot point and sets the desired orientation.

Steps

3. Under the Type drop-down menu, select HMD. This will bring up the options for defining an HMD Rigid Body.

4. If the selected marker matches one of the Active clips, it will indicate which type of Active Clip is being used.

5. Under the Orientation drop-down menu, select the desired orientation of the HMD. The orientation used for streaming to Unity is +Z forward and Unreal Engine is +X forward, or you can also specify the expected orientation axis on the client plugin side.

6. Hold the HMD at the center of the tracking volume where all of the active markers are tracked well.

8. Click Create. An HMD Rigid Body will be created from the selected markers and it will initiate the calibration process.

9. During calibration, slowly rotate the HMD to collect data samples in different orientations.

10. Once all necessary samples are collected, the calibrated HMD Rigid Body will be created.

Setting up the OpenVR Driver

Setup Steps

Download and run the installer

Open the configuration program

Once the driver has been successfully installed, launch the configuration utility software (C:\Program Files\OptiTrack\OpenVR Driver\ConfigUtil). Using this tool, you can load and check existing configurations and make changes to the settings as needed. To import current settings, click Load and to save out the changes, click Save.

Please make sure you are running this tool with admin privileges; if not, it might not be able to modify the settings properly. If the configuration software detects a running instance of SteamVR through OpenVR, it will be indicated as Initialized at the very top as shown in the image. Please note that when the settings get modified while SteamVR is running, the SteamVR must be restarted to apply the changes.

Configure connection settings

First, configure the connection settings so that the driver listens to the Motive server where the tracking data is streamed from. The server address must match the address where Motive is streaming the data to, and the local address must match the IP address of the computer on the network where the driver is installed.

Set up the HMD

Save out the configuration

Save the configurations by clicking on Save. This will modify the set of configurations in the steamvr.settings file in the steam installation directory and they will override the HMD tracking with the tracking data from Motive. If you already had an instance of OpenVR or SteamVR running, restart the application to apply the changes.

Confirm the setup

Launch SteamVR. If the driver is successfully set up, you should see a tracker icon added to the right of the HMD icon and the HMD will now be using the motion capture system instead of the base stations. Here, please make sure all of the lighthouse base stations are powered off.

VIVE Controllers

Setting up the controller (optional)

When needed, the Vive controllers can be configured as well. To do so, open the configuration utility tool while SteamVR is running. At the top of the configuration tool, it should indicate OpenVR status as Initialized and the controllers must be showing up in SteamVR. Then, in the controller sections, enable the controllers, specify the override device using the drop-down menu, and input the corresponding streaming ID of the controller Rigid Bodies in Motive. Once everything has been configured, save the changes and restart SteamVR. When the override is configured properly, SteamVR will have an additional tracker icon per each enabled controller.

Data Streaming

Now that the driver is set up, the HMD tracking will be overridden by tracking data from the mocap camera system, and you can integrate HMDs into the game engine through their own VR integration.

Streaming settings in Motive

• Broadcast Frame Data must be set to true.

• Local interface must be set to the desired IP address to stream the tracking data from.

• Streaming of Rigid Bodies must be set to True

• For wireless streaming, use Unicast streaming type.

Notes for Unity users

Please make sure the Unity project is configured for OpenVR development. In Unity, open player settings from Edit → Project Settings → Player and select the OpenVR under the Virtual Reality SDK lists. Once this is set up properly, it will play the scene on the HMD.

Unreal Engine

Make sure Unreal Engine is configured for SteamVR development. Please refer to the Unreal Engine's documentation for more information on developing for SteamVR.

Data Port Note

As of the OpenVR Driver 2.1.0 the auto-detection port default is 1513. In the case where a firewall must configure individual ports to allow or disallow data, this port can be used to allow the OpenVR Driver to connect automatically to Motive.

Streaming Rigid Body/Skeleton data

Aligning world coordinates

Client Origin

When using both the VR driver and the plugins (UE/Unity), it is important that the client origin object is located at the origin without any rotations. In other words, it must have the position set to (0,0,0) and the rotation set to (0,0,0).

Notes for Unreal Engine Users

When using the Unreal Engine plugin, you will need to additionally create a custom pawn for properly aligning the coordinate systems between SteamVR and OptiTrack UE plugin:

2. Create a new pawn. Right-click in Content Browser, and from the context menu, select Blueprint → Blueprint Class → Pawn.

3. Load created Blueprint in the editor and add a camera component.

4. (optional) Double-check that the “Lock to HMD” property is set to true under the camera component properties in the details pane.

5. Select the pawn and set the “Base Eye Height” property to 0 in the details pane.

6. Compile the pawn then add it to the scene.

7. Select the pawn and set the “Auto Possess Player” to “Player 0”.

8. The HMD should now be working for Levels built for VR.

This page provides instructions on how to configure VCS inputs in Unreal Engine. The basic configuration is similar to configuring any other input triggers in Unreal Engine. Please note that only one VCS controller can be connected and configured due to some limitations. Having two controllers connected at the same time is not supported.

Setup Steps

Create VCS Rigid Body in Motive

Create a Rigid Body from your tracking controller’s markers using the Builder pane or by selecting the markers and using the keyboard hotkey CTRL + T. You'll want to orient the controller along the +Z axis during creation to define the 'neutral' or 'zero' orientation.

Configure Data Streaming settings

In Motive, configure the data streaming settings. Use the Data Streaming pane to configure streamed packets. Make sure Rigid Body data is streamed out in order to use VCS.

Create/load an UE Project

Start up a project in Unreal Engine (UE).

Enable Windows RawInput plugin

Go to Edit tab → Plugins to open the plugins panel. Enable the Windows RawInput plugin under the Input Devices group.

Connect VCS plugin through the enabled plugin

In Edit tab → Project Settings, scroll to the bottom on the left side panel until you see Raw Input under the plugins group. Here you will let UE project know which input devices to use.

Find Hardware ID and Product ID of the VCS controllers

To find these IDs, you will need to look at the windows device properties. Go to Windows Control Panel -> Devices and Printers. Then right-click on the VCS controllers to access its properties. In the properties, go to the Hardware tab and click properties for “HID-compliant game controller”.

Once you access the controller properties, go to the details tab. Select Hardware ID in the drop-down menu and the hardware ID (HID) and product ID (PID) will be shown under the highlighted section.

Input the IDs in UE

Under the project settings panel Raw Input plugin properties, input both the vendor ID (Hardware ID) and the product ID (PID) that was found under the controller properties.

Register the Input Buttons

Now the project has the IDs to look for the controllers, next step is to setup and register the input buttons. To do so, you will play the project scene, and trigger on the buttons to register them.

In UE, hit Play and press (~) to access the console. In the console, input command ShowDebug INPUT". This will list out all of the input actions on the left side of the viewport.

Use the keys to register

Use all of the keys on the controller to register the inputs; total three axis and seven buttons. Please note that these keys may not exactly match the keys on your controller.

• Axis 1: Joystick left/right

• Axis 2: Joystick up/down

• Axis 3: Nob rotate

• Button 1: Blue

• Button 2: Black

• Button 3: White

• Button 4: Red

• Button 6: Joystick click

• Button 7: Nob click

Map the Registered Inputs

Now that the buttons have been registered, next step is to map the keys. They will be mapped under Edit → Project Settings → Inputs. Choose either the Axis mapping or the action mapping to map the controls to desired actions.

Use the Registered Inputs

Now that all of the buttons are set up, use them to control the VCS in UE.

This page provides instructions on how to set up the OptiTrack Streaming Client Unreal Engine plugin. This plugin is intended for Virtual Reality customers, but can be used with many other applications.

Streaming Client Setup (Client)

Next step is to configure the client. Follow below instructions to install and configure the OptiTrack Unreal Engine plugin to receive the streamed tracking data.

Enable the Plugin

OptiTrack - Streaming Client Plugin (required)

1. Download the .

2. Extract the contents from the ZIP file.

3. Open the extracted OptiTrack folder, transfer the entire "OptiTrack" folder into the Unreal Engine's plugin directory located in the C:\Program Files\Epic Games\5.#\Engine\Plugins folder (there will be other plugins in that folder already).

4. Open/Create a new Unreal Engine project.

5. Under the Edit menu, click Plugins to open up the panel where all of the available plugins are listed.

6. Browse to OptiTrack section and enable the "OptiTrack - Streaming Client".

7. Click Apply to submit the changes. It will require the Unreal Engine project to be restarted

Set up the Client Origin

Once the OptiTrack - Streaming Client plugin is enabled, the OptiTrack Client Origin actor will be available in Unreal Engine.

OptiTrack Client Origin

OptiTrack Client Origin

The OptiTrack Client Origin class enables the Unreal Engine (client) to communicate with the Rigid Body, Skeleton, and HMD tracking data streamed from Motive.

To add the client origin, simply drag-and-drop the OptiTrack Client Origin from the Place Actors panel into the level. Once the client origin is placed within the level, its position and orientation will reconcile the global origin of Motive in Unreal Engine. In other words, the tracking data will be represented relative to where this Client Origin object is positioned and oriented.

Connecting Unreal Engine to Motive

2. [Unreal] Once the plugin is added and enabled in the project, the OptiTrack Client Origin class will be available from the Place Actors panel.

3. [Unreal] Drag and drop the OptiTrack Client Origin into the scene.

4. [Unreal] Place the OptiTrack Client Origin at the desired location within the scene.

5. [Unreal] Select the instantiated OptiTrackClientOrigin object from the World Outliner panel.

6. [Unreal] In the Details panel, make sure its Auto Connect setting is checked. This configures the client origin to automatically search the network and connect to Motive.

7. Now that the client origin is set, the client origin will attempt to connect to Motive and start receiving the tracking data whenever the scene is played.

Animate Rigid Bodies

OptiTrack Rigid Body Actor

Set Up Steps

1. [Unreal] From the Place Actors panel, search for OptiTrack Rigid Body Actor, then drag-and-drop the actor into the scene.

2. [Unreal] With this Rigid Body actor selected, attach the target actor that you wish to animate using the Details panel. Make sure the target actor's transformation is set to movable.

3. [Unreal] Set the relative locations and rotations to all zeroes on this target actor. This actor should be listed as a child of the Rigid Body actor.

4. [Motive] In Motive, assign a value to Streaming ID property for the target Rigid Body.

5. [Unreal] In the properties of the OptiTrack Rigid Body Actor component, match the Tracking ID with the Streaming ID of the Rigid Body asset in Motive.

6. Make sure both Motive and OptiTrack Client Origin is set up for streaming, hit Play, and the attached actor object will be animated according to the live-streamed Rigid Body tracking data.

RigidBodyComponent Properties

Tracking ID

Hide on Invalid Definition

When this is checked, the corresponding Rigid Body actor will be hidden from the level until the associated Rigid Body data is streamed out from Motive and received by the plugin.

Disable Low Latency Update

Low latency update feature allows Rigid Body position and orientation transform to be updated immediately before rendering minimizing the latency. This is enabled by default. For debugging, you can check this setting to disable this behavior.

Tracking Origin

This sets a specific client origin to use for receiving tracking data. When this is unset, the plugin will default to the first client origin that it finds in the scene.

Respect Parent Transform

When this is set to true, the Rigid Body transform data from Motive will be applied in respect to the parent actor's pivot coordinates. By default, this is set to false, and all of the tracking data will be applied in respect to the pivot axis of the client origin.

Drawing Markers

When needed, you can also draw labeled marker data from Motive into the scene in UE. In most applications, you do not have to draw the markers as Rigid Body data and the Skeleton data will be used instead; however, getting markers generated in the scene may be helpful for debugging purposes. To enable drawing of the markers:

• [UE4] Expand the OptiTrackClientOrigin (Instance) properties, and enable the Draw Markers checkbox.

Animate Skeletons

Tutorial Video

Setup

Follow the below steps to set up Skeleton streaming onto Unreal Engine.

1. Create a Animation Blueprint in the 3D View

Step 2. Right-click the blank space in the Content Browser pane, then select Animation → Animation Blueprint.

Step 3. On the pop-up window, select the OptiTrackAnimInstance at the parent class section at the top and click on the target Skeleton name at the bottom. Then click OK.

Step 4. In the content browser, assign a name to the created animation blueprint.

Step 5. Drag the character blueprint into the scene.

Step 6. Select the character blueprint in the 3D View

• In the Details Pane, select “+ ADD” and create a new an “OptiTrack Skeleton Instance” on the model.

• Set the “Source Skeleton Asset” equal to the Skeleton name in Motive.

2. Setup the Blueprint

**Step 1.**Double-click the animation blueprint in the content browser to open its editor.

**Step 2.**Right-click the animation graph, then create a new "OptiTrack Skeleton".

**Step 3.**Right-click the animation graph, then create a new "Get Streaming Client Origin" and connect its output to the Streaming Client Origin.

**Step 4.**Right-click the animation graph, then create a new "Get Source Skeleton Asset Name" and connect its output to the Source Skeleton Asset Name.

Step 5. Right-click the animation graph, then create a new "Component To Local" and connect the output from "OptiTrack Skeleton" into its input.

**Step 6.**Connect all of the nodes together. The basic animation flow chart should look like the following.

3. Assign Bone Mapping

Step 1. Select the OptiTrack Skeleton plugin in the blueprint graph area.

Step 2. Drop down the Bone Mappings property in the Details Pane.

Step 3. Click “Auto Fill Bone Mapping” to automatically assign the bones in the Skeleton to the OptiTrack Skeleton names.

Note: There is no standard for bone naming conventions, so bone names may vary depending on characters. After doing the auto-fill, review the list and double-check that the auto-assigned names are correct. You may need to manually use the drop-down menu to adjust the assigned mapping for missing, or incorrect, items.

Step 4. Hit "Compile" in the top left to build the blueprint.

4. Setup OptiTrack Streaming

Step 1. Open the 3D View

Step 2. Search OptiTrack Client Origin in the Modes pane.

Step 3. Drag the OptiTrack Client Origin into the 3D scene, then select it to access its properties.

• (Optional) put it at 0,0,0 location.

• Make sure that streaming settings on both Motive and Unreal match.

5. Click _Play_

Notes on bone mapping

The OptiTrack Unreal Engine Skeleton Plugin uses bone mapping, not retargeting. This means that the bone segments in Motive map directly to the character model (bone mapping), instead of being translated into something that is usable by a more abstract biped model (retargeting). Because of this non-anatomical Skeletons will not map correctly without some additional tweaking.

Practically, this means that you will need to do things like turn off the toe mapping for characters with high heels, adjusting the pelvis bone in Motive or in the model for characters with non-anatomical hip bones, and not use bipeds that are too anatomically different than humans, such as a gorilla or swamp monster.

For example, the character sample below has both a non-anatomical pelvis and high heels. It is preferable to use character models that are more anatomically correct, but in this case, you can do a couple things to mitigate these issues:

1. Turn-off toe streaming

In the example below, since this character is wearing heels, any actor providing data for this model will also need to be wearing heels. To get around this you can just turn off the toe mapping in the OptiTrack Unreal Engine Skeleton Plugin.

2. Adjust the bone segments in Motive

The hip segment on the Countess actor is centered in the stomach rather than in the pelvis, the neck bone in Motive is a bit too long for the model, and the shoulders in Motive do not match the width of the character’s shoulders. By adjusting bones' positions and lengths in Motive, you can make the streamed Skeleton to better match the model; however, please note that there are limitations to how much you can do this.)

Bone Scaling

When streaming Skeleton data to animate characters that have different bone lengths compared to the mocap actor, the UE character will need to be scaled accordingly. In this case, the "Scale Bones" feature in the OptiTrack Skeleton node automatically scales the character bones to match the mocap actor. This setting is enabled by default.

Aligning Bones

The OptiTrack Unreal Engine Skeleton Plugin uses bone mapping, not retargeting. This means that the bone segments in Motive map directly to the character model (bone mapping), instead of being translated into something that is usable by a more abstract biped model (retargeting). Because of this, non-anatomical Skeletons will not map correctly without some additional tweaking. Starting from plugin version 1.23, you can tweak the alignment of the bone mapping by adding sockets to the Skeleton blueprint:

Adding Sockets to the Bone Mapping

2. Under the Skeleton tree, right-click on the bone that you wish to add the sockets to.

3. Right click and select_Add Socket_.

4. Go to the Animation blueprint, and change the bone mapping of the bone which you have created sockets for, and map it to the socket that was just created.

5. Play the scene, and adjust the socket location from the Skeleton Editor to adjust alignment of the bone.

Overview

The OptiTrack Unity3D Plugin allows you to stream real-time Rigid Body, skeleton, and HMD tracking data from Motive into Unity. Using the streamed data, objects and characters in the scene can be animated. The plugin contents are distributed in unitypackage format, and you can simply load this file into Unity projects to import its contents. Once imported, included C# scripts can be used for instantiating a client origin and receiving the tracking data. This article focuses on how to set up and use the plugin.

Versions Requirements

• Unity Version: 2017.2 / 2017.1 or above. (2020.3+ recommended)

• Visual Studio 2019 or latest Visual C++ Redistributable

Motive Setup (Server)

Streaming Setup

From Motive, the tracking data can be streamed in real-time either from a live capture (Live Mode) or recorded data (Edit Mode). The streaming settings are configured by modifying the Streaming Settings. NatNet streaming must enabled and the correct IP address must be set.

Streaming in Motive

Open the Streaming Settings in Motive and configure the settings below:

• Enable - Turn on the Enable setting at the top of the NatNet section.

• Local Interface - Choose the desired IP network address from this dropdown to stream data over.

• Transmission Type - Typically you will want to set this to Unicast since it subscribes only to the data you wish to use and normally uses less network bandwidth. This is especially advised if streaming data over WiFi.

• (Optional) If using Multicast, then enable/disable the desired data types. For tracking HMDs, disabling the Marker streaming is advised.

Unity Setup (Client)

Import Plugin Package

While in the Unity project, double-click on the plugin unitypackage file and import the plugin assets into the project. When the package has been successfully imported, the following contents will be available within the project:

Plugin Contents

Setting Up the Client Object

In order to receive tracking data from a server application (e.g. Motive), a client object must be set up. The OptitrackStreamingClient.cs script can be attached to any object to stream data relative to that object. Typically, this script is attached to an empty object or loaded in using the "Client - OptiTrack" prefab object in the Assets/Optitrack/Prefabs folder.

• [Motive] In the Streaming Settings, configure the desired connection settings.

• [Unity] Under the Prefabs folder, import the "Client - OptiTrack" prefab object into the scene, or attach OptitrackStreamingClient.cs script onto an empty object.

• [Unity] In the streaming Client object, configure the connection settings to match the streaming settings in Motive.

• Server Address - IP address of the PC that the server application (Motive) is running on.

• Local Address - Local IP Address of the PC that the client application (Unity) is running on. (Typically, this looks similar to the Server Address except maybe the last digits.)

• Connection Type - Must match Motive. Unicast is recommended.

• [Unity] If you wish to receive tracking data from more than one server instances, you may create multiple objects with the client script attached.

Animating Rigid Body

1. [Unity] On an object that you wish to animate, attach the OpitrackRigidBody.cs script.

2. [Unity] In the Streaming Client entry, link the Client object in which the OptitrackStreamingClient.cs script is attached. By default, it searches for an existing client instance, but this must be specified when there are more than one streaming client objects.

3. [Unity] For the Rigid Body ID entry, input the streaming ID of corresponding Rigid Body asset in Motive. The streaming ID can be found, and changed, under the Rigid Body properties.

4. [Motive] Make sure Motive is tracking and streaming the data.

5. [Unity] Play the scene. The linked object will be animated according to the associated Rigid Body movement in Motive.

Animating Skeleton

By integrating with Unity's animation system, Mecanim, the Unity3D plugin allows Motive to stream full body skeleton data. The skeleton tracking data from Motive is streamed out as hierarchical bone segment orientations, and this data is fed into the Unity's Mecanim system which allows animating characters with different proportions.

Steps

1. [Unity] On Unity characters, attach OptitrackSkeletonAnimator.cs script as one of its components.

2. [Unity] For the Streaming Client entry, link the Client object in which the OptitrackStreamingClient.cs script is attached. By default, it searches for an existing client instance, but this must be specified when there are more than one streaming client objects.

3. [Unity] Enter Skeleton Asset Name which is assigned in Motive

4. [Unity] For the Destination Avatar entry, link to the character's avatar component.

5. [Motive] Make sure Motive is tracking and streaming the data.

6. [Unity] Play the scene. When everything is set up properly, the linked avatar in Unity will be animated according to the streamed skeleton in Motive. The position of the actor will be in its reference position as explained above.

Animating Markers, Etc.

1. [Unity] On the OptiTrack Streaming instance, enable the Draw Markers, "Draw Cameras", or "Draw Force Plates" setting(s).

2. [Motive] Make sure that marker streaming is enabled in Motive if you wish to visualize markers.

3. [Unity] Make sure the streaming setting is set up correctly, and play the scene.

4. [Unity] Each marker, camera, or force plate will be drawn in the scene, as shown in the screenshot below. (Note: Only markers will animate.)\

Integrating HMDs

OptiTrack motion capture systems can be used to track head mounted displays (HMD) and integrate the tracking data into Unity for unique VR applications. For instructions on integrating HMD tracking data into Unreal Engine, please refer to the corresponding page Unity: HMD Setup.

Wireless Multiplayer Setup

When setting up multiplayer games with wireless clients, it is more beneficial for each client to make direct connection to both the tracking-server (Motive) and the game-server, rather than rebroadcasting the streamed tracking data through the game-server. Then, any of the game related actions that interacts with the tracking data can be processed on the game-server, and this server can send out the corresponding updates to the wireless clients. This allows the wireless clients to only receive both the tracking data or updates without having to send back any information; in other words, minimizing the number of data transfers needed. If wireless clients are sending data there will be a minimum of two transfers on the wireless network, and each transfer of data through wireless network is at risk of latency or lost packets.