# Unreal Engine: Live Link Camera Stream Setup

## Overview

Motive’s Live Link plug-in for Unreal Engine includes the ability to stream tracked rigid bodies for Virtual Production and InCamera VFX (ICVFX). 

Motive tracks video cameras using a CinePuck, an active device equipped with an IMU. The CinePuck  mounts directly to the video camera and, when aligned with the camera lens as a Rigid Body, tracks the camera's focal point so it can be replicated virtually in the Unreal Engine environment. 

The Camera Role in Unreal Engine can now be used for the Tracked Camera Rigid Body. This means that you can now connect a lens file with the associated Live Link Camera Role Asset. In Unreal Engine, users can calibrate for lens distortion and nodal offset, and use the camera with a Lens Encoder, if desired. 

This document provides instructions to setup and link a Rigid Body in Motive to a Live Link Camera Role in Unreal Engine. For information on ICVFX production in general, please see the page [Unreal Engine: OptiTrack InCamera VFX](https://docs.optitrack.com/v3.2/virtual-production/unreal-engine-optitrack-incamera-vfx). 

{% hint style="info" %}
We used the standard *InCameraVFX* template in Unreal Engine for our sample project. The template includes all the necessary macros and assets needed for virtual production. 

This template is located under *Film / Video & Live Events* in the Unreal Project Browser.  
{% endhint %}

## Create Camera Asset in Motive

Motive will stream the camera as a Live Link Rigid Body asset. 

#### Connect the CinePuck and BaseStation

* Plug the [BaseStation](https://docs.optitrack.com/v3.2/active-components/active-components-hardware/basestation) into one of the Power over Ethernet (PoE) switches on the OptiTrack camera network. 
* Firmly attach the [CinePuck](https://docs.optitrack.com/v3.2/active-components/active-components-hardware/cinepuck) to the Studio Camera using the SmallRig NATO Rail and Clamps on the cage of the camera.
* The CinePuck can be mounted anywhere on the camera. For best results, put the puck closer to the lens.
* Power on the CinePuck, and let it calibrate the IMU bias. The lights will flash red and orange during calibration and change to green when done.

{% hint style="info" %}
We recommend powering the CinePuck with a USB cable while filming to avoid running out of battery power. A light on the CinePuck will indicate when the power is connected.
{% endhint %}

![CinePuck mounted via SmallRig NATO rig.](https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/c71KYqfTgEYSh3mzAPgu/CinePuck%20mouted%20to%20NATO%20rig.png)

#### Create a Rigid Body

A CinePuck is tracked in Motive as a Rigid Body asset. For detailed instructions on creating a [rigid body asset](https://docs.optitrack.com/v3.2/motive-ui-panes/builder-pane#rigid-body-create), Please see the [Builder pane](https://docs.optitrack.com/v3.2/motive-ui-panes/builder-pane) page. For instructions on pairing the IMU, please see the [IMU Sensor Fusion](https://docs.optitrack.com/v3.2/motive/imu-sensor-fusion) page. 

## Attach the Camera Role in Unreal Engine

* In Unreal Engine, click the <img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/OJuED3ywRNjOQAcaZyHB/UE%20Add%20Source%20Button.png" alt="" data-size="line"> button on the Live Link tab to add a new Live Link source.
* Select *OptiTrack Source*, check *Connect Automatically*, enter the IP address for the Motive PC in the *Server Address* field, the IP address for the Unreal PC in the *Client Address* field, and click *Create*. Enter 127.0.0.1 in both fields if running both on the same PC. Leave the Connection Type as Multicast and click create.

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/9JhSBkDoOAV1wSA1w3me/UE%20-%20LiveLink%20Connect%20to%20Source.png" alt=""><figcaption><p>Add a Live Link source in Unreal Engine.</p></figcaption></figure>

* This will display a list of assets streaming from Motive. Note that the rigid body camera, named *HandCam2*, is currently in the Transform role.
* Click *OptiTrack* in the Subject list to display the OptiTrack Live Link Properties.&#x20;

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/WUN4M6pFx0gmnlxibpJx/Live%20Link%20Select%20OptiTrack.png" alt=""><figcaption><p>Unreal Engine Properties for the OptiTrack Live Link plug-in.</p></figcaption></figure>

* In the *Live Link Roles* section, click the *Live Link Subject* dropdown. This will open the list of Assets streaming from Motive.&#x20;
* Select the camera asset, *HandCam2* in this example.

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/TNd0zdCjDTaGeKgCz4rD/Live%20Link%20Select%20LL%20Subject.png" alt=""><figcaption><p>Select a Live Link Subject in Unreal Engine.</p></figcaption></figure>

* Directly below the Live Link Subject field is a check box to *Stream as Camera (Rigid Bodies Only)*. Check this box to apply the Camera Role to the camera rigid body (HandCam2).&#x20;

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/H8b0r8j60f6tm6cbtzYc/Live%20Link%20Stream%20as%20Camera%20setting.png" alt=""><figcaption><p>Stream as Camera setting in Unreal Engine.</p></figcaption></figure>

## Configure Camera for Streaming

### Add OptiTrack LiveLinkDisplay

Click the <img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/IWbCljse2GVc5hEgrLH7/Unreal%20Engine%20Quick%20Add%20button.png" alt="" data-size="line"> Quick Add button and start typing *OptiTrack* over the menu to activate the search function.  Select the OptiTrack Live Link Display from the list of available options.

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/yqE41QAm4lStv8ptX4Vb/Live%20Link%20Add%20OptiTrackNDisplay.png" alt=""><figcaption><p>Searching from the UE Quick Add button.</p></figcaption></figure>

Move all the NDisplay assets to nest under *OptitrackLiveLinkDisplay i*n the Outliner pane.

### Add the LiveLinkController

* Select the video camera, now nested under *OptitrackLiveLinkDisplay i*n the Outliner pane..&#x20;
* In the Details pane, click the <img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/BlUvz5pPxOmcackiJ6WI/Live%20Link%20Add%20Component%20button.png" alt="" data-size="line"> Add button to add a new component to the camera.

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/upPkJAcC1XoBd4FwDgrj/Live%20Link%20Camera%20Add%20Component.png" alt=""><figcaption><p>Camera Details in Unreal Engine: Add a new component.</p></figcaption></figure>

* Search for and select the *Live Link Controller* component.&#x20;

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/3qQhfIlU9GTTRJz8LnL2/Live%20Link%20-%20Add%20component%20search%20results.png" alt=""><figcaption></figcaption></figure>

* Properties for the new Controller will display in the Camera details pane.&#x20;
* Click the dropdown next to *Subject Representation* and select the camera Rigid Body.&#x20;
* The *Subject* and *Role* fields will both update to display the name of the Rigid Body name.&#x20;

{% hint style="info" %}
The *LiveLinkController* label can only be applied to Rigid Body assets.&#x20;
{% endhint %}

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/FGyZrMjHaaAtevPK3eks/Live%20Link%20Select%20Subject%20Representation.png" alt=""><figcaption><p>Properties for a new Live Link Controller component in Unreal Engine.</p></figcaption></figure>

## Camera Calibration

As noted above, Unreal Engine includes properties that allow you to calibrate a video camera streaming into Unreal Engine. This includes the ability to use a lens encoder with the camera.

#### Lens Encoder

When using a Lens Encoder or a calibrated Lens in Unreal Engine, use the Lens File Picker settings to select the appropriate Lens File.

<figure><img src="https://content.gitbook.com/content/4ne9bLcvxMzpZqIc5yuj/blobs/lDeXN70LZHggBcg9hae0/Live%20Link%20Camera%20Lens%20File%20Picker.png" alt=""><figcaption><p>Live Link Component Camera Role properties in Ureal Engine.</p></figcaption></figure>

{% hint style="info" %}
Please see [Camera Lens Calibration Overview in Unreal Engine](https://docs.unrealengine.com/4.27/en-US/WorkingWithMedia/IntegratingMedia/CameraCalibration/Overview/) for more information on calibrating cameras and working with lens files in Unreal Engine.&#x20;
{% endhint %}