User Tools

Site Tools


visual3d:documentation:pipeline:pipeline_commands:manage_file_merge

Manage_File_Merge

Overview

The Manage File Merge command is designed to merge multiple files that recorded data from the same motion capture trial. This command merges data types from each .c3d file into one trial in the Visual3D workspace. This command was originally called Manage_Theia3D_Merge, but since has been updated to be used for file types other than from Theia3D.

This command can be found in the Pipeline Workshop Dialog under Pipeline Control.

Manage_File_Merge can be used to combine motion capture files from the same recording session. A popular use case for this command would be to merge force plate recordings from a QTM file with a Theia3D motion capture file. The analog signals from the QTM .c3d file would be merge into the Theia3D .c3d file.

This command requires data files to be stored in a particular folder structure to match .c3d files appropriately. Please carefully review the Notes section of this page for more details.

Pipeline Command

Below is the full pipeline command and parameters for Manage_File_Merge

Manage_File_Merge
! /ROOT_FOLDER=
! /HMX_INDEX_MASK=
! /USE_HMX_INDEX=FALSE
! /ACTION=
! /TAGS=ACTION
! /IMPORT_ANALOG_FILES=FALSE
! /MERGE_THEIA3D_FILES=FALSE
! /MERGE_THEIA3D_AND_OTHER_FILES=FALSE
! /MERGE_EVENT_LABELS=FALSE
! /THEIA3D_SESSION_FOLDER=
! /THEIA3D_FILE_MASK=*.c3d
! /THEIA3D_FILE_MASK_FOR_MODEL=
! /THEIA3D_PREFIX=
! /THEIA3D_SUBJECT_TAG=
! /THEIA3D_COLOR=
! /THEIA3D_SEX=UNKNOWN
! /CREATE_MODEL_FOR_EACH_C3D_FILE=FALSE
! /OTHER3D_SESSION_FOLDER=
! /OTHER3D_FILE_MASK=
! /OTHER3D_MODEL_MASK=
! /OTHER3D_MODEL_TEMPLATE_MASK=
! /OTHER3D_PREFIX=
! /OTHER3D_SUBJECT_TAG=
! /OTHER3D_COLOR=
! /NUMBER_OF_SESSION_FILES_ALLOWED=
! /SAVE_DEFAULT_CMZ_FILES=TRUE
! /DEFAULT_CMZ_NAME=<SESSION>
! /OPEN_CMZ_LIBRARY=FALSE
! /VIRTUAL_LAB=
! /INPUT_DATA_FOLDER_PRIMARY=DEFAULT
! /INPUT_DATA_FOLDER_SECONDARY=DEFAULT
! /INPUT_STATIC_DATA_FOLDER=DEFAULT
! /KINETIC_THORAX=RTA
! /ANALOG_FILTER_CUTOFF=12
! /TARGET_FILTER_CUTOFF=12
! /FP_AUTO_BASELINE=DO_NOT_COMPUTE
! /VISUAL3D_PIPELINE_POST_MERGE=
! /VISUAL3D_PIPELINE_PREMERGE=
;

Command Parameters

The following table shows the command parameters and descriptions:

Parameter Description
! /ROOT_FOLDER= The directory root search containing all C3D files that contain the information relevant for analysis.
! /HMX_INDEX_MASK=file mask for Theia .hmx file (if applicable).
! /USE_HMX_INDEX=FALSEIf you would like to use a .hmx file to supersede this information.
! /ACTION=
! /TAGS=ACTIONIf/how file tags should be added onto the files by. Default is to the Action.
! /IMPORT_ANALOG_FILES=FALSEMerge primary files with analog files, such as force plate data (mutually exclusive).
! /MERGE_THEIA3D_FILES=FALSEMerge primary files into other files (mutually exclusive)
! /MERGE_THEIA3D_AND_OTHER_FILES=FALSEMerge primary files with other data, such as marker-based files (mutually exclusive).
! /MERGE_EVENT_LABELS=FALSEMerge event labels between Primary and Secondary .c3d files.
! /THEIA3D_SESSION_FOLDER=Folder mask for primary session files (directory within top level directory to search for primary files).
! /THEIA3D_FILE_MASK=*.c3dFile name mask for primary files. Can include wildcards.
! /THEIA3D_FILE_MASK_FOR_MODEL=File name mask for model file. If blank it will select the first file as the model file. Can include wildcards.
! /THEIA3D_PREFIX=Subject Prefixes for Primary files.
! /THEIA3D_SUBJECT_TAG=Subject tags for Primary files.
! /THEIA3D_COLOR=The color of the Primary file skeleton.
! /THEIA3D_SEX=UNKNOWNSex of the primary file.
! /CREATE_MODEL_FOR_EACH_C3D_FILE=FALSEFor automated model building for Theia3D files, creating a new model for each .c3d.
! /OTHER3D_SESSION_FOLDER=Folder mask for merged files (directory within top level directory to search for merged files).
! /OTHER3D_FILE_MASK=File name mask for merged files. Can include wildcards.
! /OTHER3D_MODEL_MASK=File name mask for static merged files. Can include wildcards.
! /OTHER3D_MODEL_TEMPLATE_MASK=File name mask for model files (if needed). Can include wildcards.
! /OTHER3D_PREFIX=Subject prefixes for merged files.
! /OTHER3D_SUBJECT_TAG=Subject tags for merged files.
! /OTHER3D_COLOR=The color of the secondary data files.
! /NUMBER_OF_SESSION_FILES_ALLOWED=Restricts the number of session files used.
! /SAVE_DEFAULT_CMZ_FILES=TRUEAutomatically save default CMZ file with specified name.
! /DEFAULT_CMZ_NAME=<SESSION>The name that CMZ file should be saved as, related to the hierarchy of the folder structure (either Session or Group+Session).
! /OPEN_CMZ_LIBRARY=FALSEOpen CMZ Library after processing.
! /VIRTUAL_LAB=Set virtual lab (“None” if not desired).
! /INPUT_DATA_FOLDER_PRIMARY=DEFAULT Method by which primary data is stored. DEFAULT is
! /INPUT_DATA_FOLDER_SECONDARY=DEFAULTMethod by which secondary data is stored. DEFAULT is
! /INPUT_STATIC_DATA_FOLDER=DEFAULTMethod by which static secondary data is stored. DEFAULT is
! /KINETIC_THORAX=RTAChoice of Kinetic Thorax for automated model with Theia3D specific files.
! /ANALOG_FILTER_CUTOFF=12The lowpass filter cutoff for the analog files.
! /TARGET_FILTER_CUTOFF=12The lowpass filter cutoff for the target files.
! /FP_AUTO_BASELINE=DO_NOT_COMPUTEMethod by which forceplate data was collected.
! /VISUAL3D_PIPELINE_POST_MERGE=Visual 3D pipeline to run after the merge occurs.
! /VISUAL3D_PIPELINE_PREMERGE=Visual 3D pipeline to run before the merge occurs.

Dialog

The command can be edited in a text editor or in a dialog form. To edit in the dialog pop up form either click on the Edit button in the pipeline workshop or double-click on the pipeline command. The dialog is shown below. There are a variety of inputs that can be used to generate CMZs from Theia3D files:

  • Root path to search: The directory root to search for all of the Files.
  • Open CMZ Library After Processing:
  • Automatically save default CMZ files: How the CMZ file should be saved as.
  • Use Index Theia Files (.hmx): If you would like to use a .hmx file to supersede Theia3D files information.
  • File Tags: If/how file tags should be added onto the files by.

Primary Files:

  • Primary Data Folder: Method by which Theia3D data is stored.
  • Primary Session Folder Mask: Folder mask for Theia3d files (directory within top level directory to search for Theia3D files).
  • Primary File Mask: File name mask for Theia3D files. Can include wildcards.
  • Primary File Mask for Model: File name mask for model file. If blank it will select the first file as the model file. Can include wildcards.
  • Theia3D Subject Tags: Subject tags for Theia3D files.
  • Theia Prefixes: Subject prefixes for Theia3D files.
  • Theia Color: The color of the Theia3D skeleton.
  • Sex: The sex of the Theia3D file for model inertial properties.
  • Choose Kinetic Thorax: Choose which type of thorax to model.
  • Merge Primary Files: Merge primary files into other primary files (mutually exclusive).
  • Import Analog Files: Merge primary files with analog files, such as force plate data (mutually exclusive).
  • Merge Primary Files and Other Files: Merge primary files with other data, such as marker-based files (mutually exclusive).
  • Treat Session Files as Independent:

Secondary Files:

  • Secondary Data Folder: Method by which secondary data is stored.
  • Static Data Folder: Method by which static secondary data is stored.
  • Other Session Folder Mask: Folder mask for merged files (directory within top level directory to search for merged files).
  • Other File Mask: File name mask for merged files. Can include wildcards.
  • Other Static File Mask: File name mask for static merged files. Can include wildcards.
  • Other Model Template Mask: File name mask for model files (if needed). Can include wildcards.
  • Other Subject Tags: Subject tags for merged files.
  • Other Prefixes: Subject prefixes for merged files.
  • Other Color: The color of the secondary data files.
  • Analog Filter Cutoff: The lowpass filter cutoff for the analog files.
  • Target Filter Cutoff: The lowpass filter cutoff for the target files.
  • FP_Auto_Baseline: Method by which forceplate data was collected.
  • Restrict Number of Session Files: Restricts the number of session files used.
  • Visual3D Pipeline Pre-Merge: Visual 3D pipeline to run before the merge occurs.
  • Visual3D Pipeline Post-Merge: Visual 3D pipeline to run after the merge occurs.

Examples

Merge QTM and Theia

Merge Multiple QTM Files

Notes

Folder Structure

There is an assumed basic directory structure for this command's input data that aligns with some useful concepts from the experiments that produced the data. This directory structure organizes the CMZ files and C3D files that contain the information relevant for analysis. This structure has 5 levels, from root folder to deepest leaf they are:

1. StudyAlthough the name of this directory level suggests that the data it contains is only used for one study, this is not necessary. As a top-level concept this is the largest, most general database that is stored in one location. This could be specific to a study, or it could be specific to a project, researcher, lab, or consortium. In these latter cases it is possible to define queries to extract the specific signals that are desired for an individual study.
2. GroupThis level of the directory structure is optional, but it allows the researcher to divide c3d files according to desired parameters, e.g., Women - Men, Old - Young. If this level is omitted from the structure then individual C3D files can be tagged with the necessary parameters to allow for their specific selection.
3. SessionThis directory level can be conceived as a single day (or single effort) of data collection within the database. This is the most important level as this is where the CMZ file and log files get stored. This is also the level of the directory where static models are stored for all subjects involved in the session.
4. TaskThis level of the directory allows the researcher to divide c3d files according to the desired activity, e.g., walking, running, jumping, etc. As with group-level parameters, C3D files can be coded with these different tasks by using tags.
5. TrialAt this level of the directory structure we have individual trials. This means that we have the individual C3D files that are recorded by the motion capture system.

If you are using Theia3D files, and use TMBATCH, the default folder structure will have 6 levels. From root folder to deepest leaf they are:

  1. Study
  2. Group
  3. Session
  4. Task
  5. Trial
  6. pose_filt_0.c3d
visual3d/documentation/pipeline/pipeline_commands/manage_file_merge.txt · Last modified: 2025/05/28 19:44 by wikisysop