# Labeling

## Labeling Overview

### **Marker Labels**

Marker labels are software tags assigned to identify **trajectories** of reconstructed 3D markers so they can be referenced for tracking individual markers, Rigid Bodies, Skeletons, or Trained Markersets. Labeled trajectories can be exported individually or combined together to compute positions and orientations of the tracked objects.&#x20;

![Marker Labels for a Rigid Body.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/u8SQ94fP5b7vb1EK7io0/Rigid%20Body%20in%20Viewport%20with%20Labels.png)

{% hint style="danger" %}
**Solved Data:** After editing marker data in a recorded *Take*, corresponding [Solved Data](https://docs.optitrack.com/data-recording/data-types#solved-data) must be updated.
{% endhint %}

### **Monitoring Labels**

Labeled or unlabeled trajectories can be identified and resolved from the following places in Motive:

* [**3D Perspective Viewport**](https://docs.optitrack.com/motive-ui-panes/viewport#perspective-view): From the 3D viewport, select *Marker Labels* in the visual aids <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/qXKkNbclTibJAsBhCRnc/Motive%20Visual%20Options%20button.png" alt="" data-size="line"> menu to show marker labels for selected markers.
* [**Labels pane**](https://docs.optitrack.com/motive-ui-panes/labels-pane): The Labels pane lists all the marker labels and corresponding percentage gap for each label. The label will turn magenta in the list if it is missing at the current frame.
* [**Graph View pane**](https://docs.optitrack.com/motive-ui-panes/graph-view-pane): The timeline scrubber highlights in red any frames where the selected label is not assigned to a marker. The *Tracks* view provides a list of labels and their continuity in a captured *Take*.

![The Graph View pane shows unlabeled&#x20;
trajectory gaps on labeled marker.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/OyphwFUAAuXkTJuQ0z1Y/image.png) ![List of labeled markers for the selected Rigid Body and unlabeled markers shown on the Labels pane.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/w9VGetEmeyYlo2LIOhRL/image.png)

## Labeling Methods

There are two approaches to labeling markers in Motive:

* **Auto-label pipeline:** Automatically label sets of Rigid Body, Skeleton, or Trained Markerset markers using calibrated asset definitions. Motive uses the unique marker placement stored in the Asset definition to identify an asset and applies its associated marker labels automatically. This occurs both in real-time and post-processing.&#x20;

{% hint style="info" %}
Auto-labeling applies only to assets enabled with a checkmark <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/LBccogvtipTSKU6lVhDJ/Assets%20Pane%20-%20Enable%20Asset.png" alt="" data-size="line"> in the [Assets pane](https://docs.optitrack.com/motive-ui-panes/assets-pane).
{% endhint %}

* **Manual Label:** Manually label individual markers using the [Labels pane](https://docs.optitrack.com/motive-ui-panes/labels-pane). Use this workflow to give Rigid Bodies and Trained Markersets more meaningful labels.

![Auto-labeled Rigid Body markers.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/HgrsDK8Hbth8EUyE6uQU/Rigid%20Body%20in%20Viewport%20with%20Labels.png) ![Auto-labeled Skeleton markers.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/IICawhb6wgvXwAJlwcqi/Skeleton%20Labels%20displayed.png)

### Auto-label

As noted above, Motive stores information about Rigid Bodies, Skeletons, and Trained Markersets in asset definitions, which are recorded when the assets are created. Motive's auto-labeler uses asset definitions to label a set of reconstructed 3D trajectories that resemble the marker arrangements of active assets.&#x20;

Once all of the markers on active assets are successfully labeled, corresponding Rigid Bodies and Skeletons get tracked in the 3D viewport.

The auto-labeler runs in real-time during Live mode and the marker labels are saved in the recorded *TAKES*. Running the auto-labeler again in post-processing will label the Rigid Body and Skeleton markers again from the 3D data.

### Auto-labeling Steps

#### **From the Data pane**

1. Select the *Take(s)* from the [Data pane](https://docs.optitrack.com/motive-ui-panes/data-pane).
2. Right-click to open the context menu.
3. Click *reconstruct and auto-label* to process the selected *Takes*. This pipeline creates a new set of 3D data and auto-labels the markers that match the corresponding asset definitions.

![Auto-labeling a Take.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/bMlSZ6XQt2neIhvOwFac/Take%20Context%20Menu%20-%20Auto-label%20selected.png)

{% hint style="danger" %}
Be careful when reconstructing a *Take* again either by **Reconstruct** or **Reconstruct and Auto-label.** These processes overwrite the 3D data, discarding any post-processing edits on trajectories and marker labels.&#x20;

Recorded Skeleton marker labels, which were intact during the live capture, may be discarded, and the reconstructed markers may not be auto-labeled correctly again if the Skeletons are never in well-trackable poses during the captured *Take*. This is another reason to always start a capture with a good [calibration pose](https://docs.optitrack.com/skeleton-tracking#calibration-pose) (e.g., a T-pose).
{% endhint %}

### Rename Labels

Label names can be changed through the [Constraints Pane](https://docs.optitrack.com/motive-ui-panes/constraints-pane) or the [Labels Pane](https://docs.optitrack.com/motive-ui-panes/labels-pane).&#x20;

* The [Constraints pane](https://docs.optitrack.com/motive-ui-panes/constraints-pane) displays marker labels for either the selected asset or all assets in the *Take*. Markers that are not part of an asset are not included.&#x20;
* The Labels pane displays marker labels for either the selected asset or all markers in the *Take*.&#x20;

<div><img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/4RcVNWORrpjlYnvwkc7m/Constraints%20Pane%20-%20context%20menu.png" alt="Marker labels in Constraints pane." width="312"> <figure><img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/cEI8aRGQZ1Rayg99m4Ux/Labels%20Pane%20-%20Labels%20context%20menu.png" alt="" width="313"><figcaption><p>Marker labels in Labels pane.</p></figcaption></figure></div>

**To change a marker label:**

* Right-click the label and select *Rename*, or
* Click twice on the label name to open the field for editing.&#x20;

{% hint style="info" %}
We recommend using the single asset view rather than *-All-* when relabeling markers from the Constraints pane.
{% endhint %}

**To switch assets:**

Use the Assets pane or the 3D Viewport to select a different asset or click the <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/mRTJAEjnEfI4sFFVEmjw/Link%20to%203D%20selection%20button.png" alt="" data-size="line"> button in the Constraints pane to unlock the asset selection drop-down.&#x20;

<figure><img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/lg5PUA8XIOcbGEDZzCYZ/Constraints%20Pane%20-%20unlock%20asset%20selection.png" alt="" width="308"><figcaption><p>Asset Selection unlocked in the Constraints Pane.</p></figcaption></figure>

{% hint style="warning" %}

* When *-All-* is selected in the Constraints pane, the marker labels include the asset name as a prefix, e.g., *Bat\_marker1.* Delete the prefix if updating labels from this view.
* The Labels pane does not include the asset name prefix when *-All-* is selected.&#x20;
  {% endhint %}

## Manual Labeling

There are times when it is necessary to **manually label** a section or all of a trajectory, either because the markers of a Rigid Body, Skeleton, or Trained Markerset were misidentified (or unidentified) during capture or because individual markers need to be labeled without using any tracking assets. In these cases, the [Labels pane](https://docs.optitrack.com/motive-ui-panes/labels-pane) in Motive is used to perform manual labeling of individual trajectories.&#x20;

The manual labeling workflow is supported only in post-processing of the capture when a *Take* file (.TAK) has been loaded with 3D data as its playback type. In case of [2D data](https://docs.optitrack.com/reconstruction-and-2d-mode#2d-mode) only capture, the *Take* must be **Reconstructed** first in order to assign, or edit, the marker labels in 3D data.&#x20;

This manual labeling process, along with [3D data editing](https://docs.optitrack.com/motive/data-editing), is typically referred to as *post processing* of mocap data.

## Labels pane

{% embed url="<https://vimeo.com/167944428?embedded=true&owner=15736845&source=video_title>" %}
Labeling Tutorial 2. Manual Labeling in Motive. **This video is based on older version of Motive. There maybe a few differences in Motive 2.0, but the general workflow still remains the same.**
{% endembed %}

The [Labels pane](https://docs.optitrack.com/motive-ui-panes/labels-pane) is used to assign, remove, and edit marker labels in the [3D data](https://docs.optitrack.com/data-recording/data-types#3d-data) and is used along with the [Editing Tools](https://docs.optitrack.com/motive/data-editing) for complete post-processing.

* Shows the labels involved in the Take and their corresponding *percentage of occluded gaps* values. If the trajectory has no gaps (100% complete), no number is shown.&#x20;
* By default, only labeled markers are shown. To see unlabeled markers, click the <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/94iIhCuGrUeuYmrLMnJU/Motive%20Context%20Menu.png" alt="" data-size="line"> button in the upper right corner of the pane and select any layout option other than *Labeled only*.&#x20;
* Labels are color-coded to note the label's status in the current frame of 3D data.  Assigned marker labels are shown in white, while labels without reconstructions and unlabeled reconstructions that are not in the current frame are shown in magenta.&#x20;

Please see the [Labels pane](https://docs.optitrack.com/motive-ui-panes/labels-pane) page for a detailed explanation of each option.

{% hint style="info" %}
The **Tracks View** under the [Graph View pane](https://docs.optitrack.com/motive-ui-panes/graph-view-pane) can be used in conjunction with the Labels pane to quickly locate gaps in a trajectory to see which markers and gaps are associated.&#x20;
{% endhint %}

### Quick Label Mode

The **Quick Label** mode allows you to tag labels with single-clicks in the 3D Viewport and is a handy way to reassign or modify marker labels throughout the capture.&#x20;

![Use the Labels pane to quickly label markers. Click image to enlarge.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/Ya7dBRoDlaI2KXfJJen6/image.png) ![Re-labeling Skeleton markers using the Quick Label Mode.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/7H7r3au1NaNYGxHdYTMQ/image.png)

### Quick Label Mode Steps

1. Select the asset to label, either from the Assets Pane, the 3D Viewport, or from the asset selection drop-down list in the Labels pane.&#x20;

2. This will display all of the asset's markers and their corresponding percentage gap.

3. Click the <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/PJjtGNjUTE7RTPOmei59/Motive%20Context%20Menu.png" alt="" data-size="line"> button and select any option other than *Labeled Only* to see unlabeled markers.&#x20;

4. Select the **Label Range**:
   * **All or Selected:**  Assign labels to a selected marker for all, or selected, frames in a capture.
   * **Spike or Fragment:**  Apply labels to a marker within the frame range bounded by trajectory gaps and spikes (erratic change).&#x20;
   * **Swap Spike or Fragment:**  Apply labels only to spikes created by labeling swaps. &#x20;

5. Inspect the behavior of the selected trajectory then use the **Apply Labels** drop-down list in the Labels pane Settings to apply the selected label to frames forward or frames backward or both.  Click <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/25R6YB4POibHSCjmRGws/Pane%20Settings%20toggled%20OFF.png" alt="" data-size="line"> to display settings, if necessary.&#x20;

6. Click the Mouse Actions button <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/661Xc6jZsQdZfEUMYgxs/Select%20or%20Edit%203D%20Object.png" alt="" data-size="line"> to switch to Quick Label Mode (Or use Hotkey: D). The cursor will change to a finger icon.<br>

   <figure><img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/HElC08lDMS5fcy5RGxU7/Selection%20or%20Quick%20Label.png" alt=""><figcaption><p>Mouse Actions menu.</p></figcaption></figure>

7. &#x20;Select a label from the Labels pane. The label name will display next to the pointed finger until a marker is selected in the 3D Viewport, assigning the label to that marker.\ <br>

   <figure><img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/VG6veUU6RFYTkoww9eHL/Quick%20Label%20Mode%20-%20from%20Viewport.png" alt=""><figcaption><p>Quick Label Mode in the 3D Viewport.</p></figcaption></figure>

8. The **Increment Options** setting determines how the Quick Label mode should behave after a label is assigned.<br>

   <figure><img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/LKYkJJgblAGyhDbtHCP0/Label%20increment%20options.png" alt=""><figcaption></figcaption></figure>

   * &#x20;**Do Not Increment** keeps the same label attached to the cursor.&#x20;
   * **Go To Next Label** automatically advances to the next label in the list, even if it is already assigned to a marker in the current frame. This is the default option.&#x20;
   * **Go To Next Unlabeled Marker** advances to the next label in the list that is not assigned to a marker in the current frame.&#x20;

9. When you are done, toggle back to normal Select Mode using either Hotkey: D or the Mouse Actions menu.

{% hint style="info" %}
When the [3D viewport](https://docs.optitrack.com/motive-ui-panes/viewport#perspective-view) Visual Aids are set to display marker labels and Quick Label mode is toggled on, *all* of the labels for visible markers will appear in the 3D viewport.&#x20;

Uncheck Labels in the viewport Visuals if you do not wish to see them in Quick Label mode.
{% endhint %}

![Hiding marker labels.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/UQydfJdWqmn6jXDBKHG6/image.png)

### Tips for Labeling Skeletons

* The hip bone is the main parent bone, top of the hierarchy, where all other child bones link to. Always label the hip segment first when working with skeletons. Manually assigning hip markers sometimes helps the auto-labeler to label the entire asset.
* Show/Hide **Skeleton visibility** under the Visual Aids <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/KmXNwWy7jlK0oV6Cafet/Viewport%20Visual%20Options%20button.png" alt="" data-size="line"> options in the [perspective view](https://docs.optitrack.com/motive-ui-panes/viewport#perspective-view) to have a better view on the markers when assigning marker labels.
* Toggle **Skeleton selectability** under the Selection Options <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/XdaRxCns28ZmxMZno4CT/Viewport%20Selection%20options%20button.png" alt="" data-size="line"> in the [perspective view](https://docs.optitrack.com/motive-ui-panes/viewport#perspective-view) to use the Skeleton as a visual aid without it getting in the way of marker data.
* Show/Hide **Skeleton sticks and marker colors** under the Visual Aids <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/KmXNwWy7jlK0oV6Cafet/Viewport%20Visual%20Options%20button.png" alt="" data-size="line"> in the [perspective view](https://docs.optitrack.com/motive-ui-panes/viewport#perspective-view) options for intuitive identification of labeled markers as you tag through Skeleton markers.
* Enable the **Quality Visual** setting in the skeleton properties to graphically see:
  * When there are no markers contributing to a bone. The bone will appear red.
  * When a Degree of Freedom limit is reached. The bone will appear blue.

## Labeling Workflows

The labeling workflow is flexible and alternative approaches to the steps in this section can also be used.&#x20;

{% hint style="info" %}
**General Labeling Tips**

* Use the [Graph View pane](https://docs.optitrack.com/motive-ui-panes/graph-view-pane) to monitor occlusion gaps and labeling errors during post-processing.
* [Motive Hotkeys](https://docs.optitrack.com/motive/motive-hotkeys) can increase the speed of the workflow. Use **Z and Shift+Z hotkeys** to quickly find gaps in the selected trajectory.
* Set the Visual Aids for Markers in the perspective view to *Hide for Disabled Assets* then uncheck the <img src="https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/PQxih2oHW3bvyvzCwp3N/image.png" alt="" data-size="line"> box to the left of the asset name in the Assets pane when you are done labeling it to better focus on the remaining unlabeled assets.&#x20;
  {% endhint %}

### Using Combined Reconstruction and Auto-label Pipeline

**Step 1.** In the [Data pane](https://docs.optitrack.com/motive-ui-panes/data-pane), **Reconstruct and auto-label** the take with all of the desired assets enabled.

**Step 2.** In the [Graph View pane](https://docs.optitrack.com/motive-ui-panes/graph-view-pane), examine the trajectories and navigate to the frame where labeling errors are frequent.

**Step 3.** Open the [Labels pane](https://docs.optitrack.com/motive-ui-panes/labels-pane).

**Step 4.** Select an asset that you wish to label.

**Step 5.** From the label columns, click on the marker label that you wish to re-assign.

**Step 6.** Inspect behavior of a selected trajectory and its labeling errors and set the appropriate labeling settings (allowable gap size, maximum spike and applied frame ranges).

**Step 7.** Switch to the QuickLabel mode (Hotkey: D).

**Step 8.** In the [Perspective View](https://docs.optitrack.com/motive-ui-panes/viewport#perspective-view), assign the labels to the corresponding marker reconstructions by clicking on them.

**Step 9.** When all markers have been labeled, switch back to the Select Mode.

### Using Standalone Reconstruction Pipeline and Auto-label Pipeline Separately

**Step 1.** Start with 2D data of a captured *Take* with model assets (Skeletons, Rigid Bodies, or Trained Markersets).

**Step 2.** **Reconstruct and Auto-Label**, or just **Reconstruct**, the *Take* with all of the desired assets enabled under the [Assets pane](https://docs.optitrack.com/motive-ui-panes/assets-pane). If you use reconstruct only, you can skip step 3 and 5 for the first iteration.

**Step 3.** Examine the reconstructed 3D data and inspect the frame range where markers are mislabeled.

**Step 4.** Using the [Labels pane](https://docs.optitrack.com/motive-ui-panes/labels-pane), manually fix/assign marker labels, paying attention to the label settings (direction, max gap, max spike, selected duration).

**Step 5.** Unlabel all trajectories you want to re-auto-label.

**Step 6.** **Auto-Label** the *Take* again. Only the unlabeled markers will get re-labeled, and all existing labels will be kept the same.

**Step 7.** Re-examine the marker labels. If some of the labels are still not assigned correctly from any of the frames, repeat steps 3-6 until complete.

### Labeling Error Fix

The general process for resolving labeling error is:

1. Identify the trajectory with the labeling error.
2. Determine if the error is a swap, an occlusion, or unlabeled.
3. Resolve the error with the correct tool.

* Swap: Use the Swap Fix tool (Edit Tools) or just re-assign each label (Labels pane).
  * When manually labeling markers to fix swaps, set appropriate settings for the labeling direction, max spike, and selected range settings.
* Occlusion: Use the Gap Fill tool (Edit Tools).
* Unlabeled: Manually label an unlabeled trajectory with the correct label (Labels panel).

For more data editing options, read through the [Data Editing](https://docs.optitrack.com/motive/data-editing) page.

![Labeling an unlabeled marker.](https://content.gitbook.com/content/GaZwzcsVav6zPBRZpapU/blobs/P25Dp1BdM1SjXwhvVnNg/Label.gif)
