Wiki Documentation Site

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Batch Processing Overview ======

Batch processing enables rapid processing of large datasets and ensures consistent results across projects, labs, and time.

Sift provides two methods of batch processing:
  - the GUI, and manually adding the library and pipelines
  - the command line
      * As a one off
      * Time-based automation
      * Directory scanning for new data automation

===== V3D Engine =====

Sift runs on Visual3D's engine, and can batch run V3D pipelines across multiple CMZs concurrently. 

**When to use**: Use the V3D Engine for processing smaller datasets, single-instance runs, or when first getting started with Sift.

  * Learn more: [[sift:application:run_v3d_engine_dialog|V3D Engine Dialog]]
  * Tutorial: [[sift:tutorials:compute_gps_and_gdi#run_v3d_engine_dialog|Compute GPS and GDI tutorial]]: uses the V3D Engine to run scripts on multiple CMZs at once
 
===== Sift Command Line =====

The Sift command line enables large scale batch processing right from the command line.

**When to use**: Use the Sift Command Line to process large quantities of data, to run continuous automated updates over time, or, for those with experience in development.

  * Learn more: [[sift:command_line|Sift command line]]
  * Tutorial: [[sift:tutorials:openbiomechanics_project:impact_shooting_arm_mechanics_free_throw_accuracy#Using_the_Sift_CMD_Line_for_automation|Using the Sift CMD Line for automation]] section of the impact shooting arm mechanics free throw accuracy tutorial.
==== Time-Based Triggers ====

Set up automatic processes to run at specific times.

**When to use**: Use this method for continuous processing on a daily, weekly, or monthly basis.

  * Learn more:
  * Tutorial: [[sift:tutorials:command_line|Batch Command Line Processing]]

==== Directory Watchers ====

**When to use**: Use this method for automated processing when new data enters a specific directory.

  * Learn more: [[sift:directory_watchers|Directory Watchers]]
  * Tutorial: [[sift:tutorials:using_directory_watchers|Directory Watchers Tutorial]]