This is an old revision of the document!

Batch Command Line Processing Tutorial

In this tutorial, we will guide you through running Sift on the command line. You'll learn how to load a library, execute queries, run PCA (Principal Component Analysis), and use it for outlier detection. After walking through each command step by step, we’ll create a batch script to automate these tasks. We'll also include some logic to further streamline the process. To get started, download the sample data folder. automation_tutorial.zip

Running on Command Line

First, we need to run Sift via the command line. On your keyboard, press Windows Key + R, type “cmd” into the run window, and press Enter to open a command prompt.
Next, navigate to Sift's installation directory.
To launch Sift, call the .exe file and pass the –NoGUI argument
This will open new console where we can interact with Sift
In this console, load a library by entering the command: –LoadLib directory “Path to downloaded library”
Next load and run a predefined query by entering: –LoadQuery file “Path to query definition file”
Select all signals by running –SelectSignals (Ensure all signal groups and work-spaces have been selected)
Run a PCA named “Outlier_PCA” –RunPCA name “Outlier_PCA”
Now, let's perform the first round of outlier detection. Run a Local Outlier Factor (LOF) test with the command: –RunLOF grouping “group” autoExclude This will find any group level outliers and exclude them automatically
To export the results of this LOF test, run: –ExportPCA file “Path to exported txt file” exportFormat “transposed” LOF LOFThrehsold
Next, run another LOF test, this time on workspaces: –RunLOF grouping “workspace” autoExclude
Export the results of this LOF test as well: –ExportPCA file “Path to exported txt file” exportFormat “transposed” LOF LOFThreshold
Finally, save the project to allow for future review in the GUI, if needed: –SaveProject file “Path to saved project”

Now that processing is complete, we have two exports detailing group and workspace-level outliers, along with a saved project file that excludes all detected outliers. This can serve as a baseline for future processing.

Batch Processing

Now that we have a clear understanding of our baseline processing, it's time to automate the workflow!

The Scenario

To effectively automate a process, we need to understand all the details. Let’s create a scenario: \ Imagine we're working with a baseball team, and each day at practice, we gather data that needs to be processed. This data is stored in date-labeled folders within a root directory, and inside each folder are three trials containing relevant C3D files. \

The Plan

Now that we know what we're dealing with, we can plan how to build our batch script. First, we’ll need to check each folder within the root directory. If a folder doesn’t already contain our exports (such as .txt files and .i3d files), we know that folder still needs to be processed. Below is a batch script that accomplishes this. You’ll need to modify the path in targetDir, as well as the path pointing to Sift.exe, to match your computer's setup. Afterward, save the file as a .bat. To do this, create a .txt file and rename it, changing the extension to .bat.

@echo off
setlocal

set "targetDir=C:\Users\shane\OneDrive\Documents\practice"

for /D %%d in ("%targetDir%\*") do (
    set "i3dFileFound="
    for %%f in ("%%d\*.i3d") do (
        if exist "%%f" (
            set "i3dFileFound=true"
        )
    )
    
    if not defined i3dFileFound (
        set "folderWithoutI3d=%%d"
        "C:\Program Files (x86)\Sift\Sift.exe" --NoGui --LoadLib directory "%%d" --LoadQuery file "%%d\query_definition.q3d" --selectSignals --RunPCA name "Outlier_PCA" --RunLOF grouping "group" autoExclude --ExportPCA file "%%d\Group Outliers.txt" exportFormat "transposed" lof lofThreshold --RunLOF grouping "workspace" autoExclude --ExportPCA file "%%d\Workspace Outliers.txt" exportFormat "transposed" lof lofThreshold --SaveProject file "%%d\Outliers.i3d" --Exit
    )
)

endlocal

The Execution

Now that we have a script ready to go, executing our processing will be as easy as double clicking the .bat file, but we can take the automation even further! Leveraging windows Task Scheduler we can set the batch script to execute at a certain time each day

Open the start menu and type Task Scheduler into the search bar
Open Task Scheduler
In the Task Scheduler select create basic task in the top right of the screen
In the popup dialog enter a meaningful name in this case we went with “Baseline Processing” press next
In the next step ensure daily is selected and press next
Next select what time of day you would like the task to run, we picked 12:00AM
In the next step ensure start a program is selected and press next
Now we will browse and select the batch script we created
On the final screen review and press finish
Finally check the existing task to ensure it has been created successfully