visual3d:tutorials:knowledge_discovery:looking_at_large_public_data_sets
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
| visual3d:tutorials:knowledge_discovery:looking_at_large_public_data_sets [2024/06/14 17:32] – created sgranger | visual3d:tutorials:knowledge_discovery:looking_at_large_public_data_sets [2024/07/17 15:46] (current) – created sgranger | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | |**Language: | + | ====== |
| - | + | ||
| - | |===== Contents =====\\ \\ \\ \\ * [[# | + | |
| Biomechanics is coming into a new age, where large datasets are the norm and individually processing the files can be tiresome and time consuming. Captured over long periods of time, datasets can be inconsistent, | Biomechanics is coming into a new age, where large datasets are the norm and individually processing the files can be tiresome and time consuming. Captured over long periods of time, datasets can be inconsistent, | ||
| Line 10: | Line 8: | ||
| - | ===== Large Datasets | + | ==== Large Datasets ==== |
| The dataset used for this tutorial is the raw motion data collected by (Van Criekinge et al [[https:// | The dataset used for this tutorial is the raw motion data collected by (Van Criekinge et al [[https:// | ||
| Line 16: | Line 14: | ||
| This dataset is unique in its size and quality for an open source dataset. Large datasets are essential for scientific observations to be able to accurately draw conclusions, | This dataset is unique in its size and quality for an open source dataset. Large datasets are essential for scientific observations to be able to accurately draw conclusions, | ||
| - | ===== Dataset | + | ==== Dataset ==== |
| The dataset used for able-bodied participants is available [[https:// | The dataset used for able-bodied participants is available [[https:// | ||
| Line 25: | Line 23: | ||
| - | ===== File Naming Conventions | + | ==== File Naming Conventions ==== |
| A common inconsistency in large datasets can be the usage of filenames. Filenames should convey information about a file in a manner that can be easily referenced, eg: calibration files should be named differently from motion files so that they can be separated, and motion files should have a trial number in their file name (if applicable), | A common inconsistency in large datasets can be the usage of filenames. Filenames should convey information about a file in a manner that can be easily referenced, eg: calibration files should be named differently from motion files so that they can be separated, and motion files should have a trial number in their file name (if applicable), | ||
| Line 51: | Line 49: | ||
| - | ===== Building Models | + | ==== Building Models ==== |
| The data set used the Plug in Gait marker set, unfortunately not all that standard Plug in Gait markers were used for example the right and left upper arm markers were missing, so only a model of the lower body can be made, see the tutorial [[https:// | The data set used the Plug in Gait marker set, unfortunately not all that standard Plug in Gait markers were used for example the right and left upper arm markers were missing, so only a model of the lower body can be made, see the tutorial [[https:// | ||
| Line 62: | Line 60: | ||
| - | ===== Inconsistent Subject Prefixes (Stroke Survivors) | + | ==== Inconsistent Subject Prefixes (Stroke Survivors) ==== |
| Most of the markers within the static files provided are prefixed with the subject ID, however the dynamic file markers are not, this will prevent the built model from linking with the dynamic markers. To begin, we will ensure that all static files include a subject-prefix, | Most of the markers within the static files provided are prefixed with the subject ID, however the dynamic file markers are not, this will prevent the built model from linking with the dynamic markers. To begin, we will ensure that all static files include a subject-prefix, | ||
| **Static file marker:**\\ | **Static file marker:**\\ | ||
| - | [[File: | + | {{:stroke_model_prefix.png}}\\ |
| **Dynamic file marker:**\\ | **Dynamic file marker:**\\ | ||
| - | [[File: | + | {{:stroke_dynamic_no_prefix.png}} |
| Stroke Survivors 47 and 58 (TVC47 and TVC58) are both missing subject prefixes. To rectify, run the pipeline file **add_calibration_prefixes.v3s** once for each of these stroke survivors, changing the "/ | Stroke Survivors 47 and 58 (TVC47 and TVC58) are both missing subject prefixes. To rectify, run the pipeline file **add_calibration_prefixes.v3s** once for each of these stroke survivors, changing the "/ | ||
| Line 232: | Line 230: | ||
| '' | '' | ||
| - | ===== Inconsistent Subject Prefixes (Able Bodied) | + | ==== Inconsistent Subject Prefixes (Able Bodied) ==== |
| The markers in the static files of the able bodied participants do not have subject prefixes, however a random selection of the dynamic file markers do have prefixes. In this case it makes sense to strip the subject prefixes from the select files that have them instead of adding them to every other file. This can be done using the " | The markers in the static files of the able bodied participants do not have subject prefixes, however a random selection of the dynamic file markers do have prefixes. In this case it makes sense to strip the subject prefixes from the select files that have them instead of adding them to every other file. This can be done using the " | ||
| Line 262: | Line 260: | ||
| - | ===== Processing C3Ds into CMZs (Stroke Survivors) | + | ==== Processing C3Ds into CMZs (Stroke Survivors) ==== |
| Instead of manually processing each C3D file, we can use another pipeline to automate the process: | Instead of manually processing each C3D file, we can use another pipeline to automate the process: | ||
| Line 456: | Line 454: | ||
| '' | '' | ||
| - | ===== Processing C3Ds into CMZs (Able-Bodied Participants) | + | ==== Processing C3Ds into CMZs (Able-Bodied Participants) ==== |
| Since the naming convention for the able-bodied participants is different from the stroke survivors the " | Since the naming convention for the able-bodied participants is different from the stroke survivors the " | ||
| Line 706: | Line 704: | ||
| '' | '' | ||
| - | ===== Missing Parameters | + | ==== Missing Parameters ==== |
| Upon inspection some of the dynamic files were missing or have incorrectly named parameters needed to calculate the model (height, mass, ankle widths, and knee widths) in the case of the stroke participants this data was provided in a spreadsheet so the values can be entered manually and the models recalculated. | Upon inspection some of the dynamic files were missing or have incorrectly named parameters needed to calculate the model (height, mass, ankle widths, and knee widths) in the case of the stroke participants this data was provided in a spreadsheet so the values can be entered manually and the models recalculated. | ||
| - | [[File: | + | {{:missing.jpg}} |
| Open the CMZ one of Stroke Survivors 24, 34, or 57 (TVC_24.cmz, | Open the CMZ one of Stroke Survivors 24, 34, or 57 (TVC_24.cmz, | ||
| Line 721: | Line 719: | ||
| After completing this, you will have a completely processed dataset! You can follow other [[https:// | After completing this, you will have a completely processed dataset! You can follow other [[https:// | ||
| - | ===== Conclusion | + | ==== Conclusion ==== |
| Through this tutorial we have identified the common pitfalls that may occur when processing a large biomechanics dataset. Using the public dataset from Van Criekinge et al. as an example, we have identified the challenges encountered and presented realistic solutions. Just like the underlying data set, we made all of our files openly available. You are encouraged to work with this dataset as we did and to build upon our work by analyzing the post-processed data. | Through this tutorial we have identified the common pitfalls that may occur when processing a large biomechanics dataset. Using the public dataset from Van Criekinge et al. as an example, we have identified the challenges encountered and presented realistic solutions. Just like the underlying data set, we made all of our files openly available. You are encouraged to work with this dataset as we did and to build upon our work by analyzing the post-processed data. | ||
| - | ===== References | + | ==== References ==== |
| **Paper:** Van Criekinge et al. a full-body motion capture gait dataset of 138 able-bodied adults across the life span and 50 stroke survivors: [[https:// | **Paper:** Van Criekinge et al. a full-body motion capture gait dataset of 138 able-bodied adults across the life span and 50 stroke survivors: [[https:// | ||
| - | |||
| - | Retrieved from "" | ||
visual3d/tutorials/knowledge_discovery/looking_at_large_public_data_sets.1718386330.txt.gz · Last modified: 2024/06/14 17:32 by sgranger