====== Markerless Motion Capture Overview ======

===== What is Markerless? =====

What most people think of when they hear "markerless motion capture", is the system which uses vision-based deep learning to detect and identify anatomical landmarks (similar to - but not necessarily the same as - the physical markers in marker-based methods), in 2D image space (per camera, per frame). The 3D position of the subject is found through triangulation across multiple calibrated cameras. The output of these systems is therefore generally 4x4 pose matrices that fully represent the pose (3D position and orientation) of the associated [[visual3d:documentation:modeling:segments:segment_overview|segment]]. Examples of vision-based markerless systems include [[visual3d:documentation:third-party:theia|Theia]] and [[visual3d:documentation:third-party:fujitsu|Fujitsu]].

Some people also consider IMU's to be a markerless system. Check out the [[visual3d:documentation:inertial_measurement_units_overview|IMU Overview Page]] to learn more.


{{:visual3d:documentation:image_1_.png?|}}
{{:visual3d:documentation:image_2_.png?|}}


//Markerless motion capture processed using Theia3D//

===== Markerless data in Visual3D =====
==== Automatic Model Building ====

Visual3D has automatic model building for [[visual3d:documentation:third-party:theia|Theia]], [[visual3d:documentation:third-party:fujitsu|Fujitsu]], [[visual3d:documentation:third-party:xsens|XSens]], [[:visual3d_partner_integrations#kinatrax|Kinatrax]], and Hawkeye.

Visual3D builds the model from the body segment position and orientations, in the form of 4x4 transformation matrices. Segment pose can be found in the ROTATION folder under the main file tree.

{{:visual3d:theia_rotation_folder.png?|rotation segments folder structure}}

Traditional [[visual3d:documentation:modeling:segments:segment_overview|segments]] are built using the proximal and distal (often the proximal end of the adjacent body segment) pose matrices, with the origin of the segment coordinate systems at the proximal end of the segment. The segment Z-Axis is defined from distal to proximal, using the right hand rule. All these traditional segments are used in the kinematic chain for [[visual3d:documentation:kinematics_and_kinetics:inverse_dynamics|inverse dynamics]].

[[visual3d:documentation:modeling:segments:virtual_segments|Virtual segments]] enable an alternative definition for the segment. Though, the virtual segments cannot be included in the kinetic chain for inverse dynamics, as the virtual segment will behave differently than the traditional segment. To address this, Theia and XSens models have been implemented in Visual3D with the addition of [[visual3d:documentation:modeling:segments:segment_overview#shadow_segments|shadow segments]]. Shadow segments follow the definition of the virtual segment, but are tracked using a transformation of the original segments pose, so that it can be included in the kinetic chain.






===== Tutorials =====
[[visual3d:documentation:pipeline:pipeline_commands:manage_file_merge|Manage File Merge]] :  combine motion capture files from the same recording session.

[[visual3d:tutorials:knowledge_discovery:assesing_stability_during_gait|Assesing Stability During Gait]]
===== Learn More =====

[[https://en.wikipedia.org/wiki/Deep_learning|Deep learning]]

=====Notes=====
Check the [[sift:documentation:release_notes|release notes]] for specific updates for how the systems models are being built. For example, shadow segments are added automatically to the model for the thorax and feet for Theia and XSens data.