Wiki Documentation Site

User Tools

Site Tools

You are here: Main Page » sift » tutorials » Outlier Detection with PCA
sift:tutorials:outlier_detection_with_pca

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
sift:tutorials:outlier_detection_with_pca [2026/02/24 20:06] – wikisysopsift:tutorials:outlier_detection_with_pca [2026/04/13 20:26] (current) – updated pca link wikisysop
Line 17: Line 17:
 [[Sift:Principal_Component_Analysis:Outlier_Detection_for_PCA#Local_Outlier_Factor|Local Outlier Factor (LOF)]] is a outlier detection method that uses the local density around data points to determine if a point is an outlier. In this sense it can find outliers that global detection methods would not, as it identifies outliers in local areas. [[Sift:Principal_Component_Analysis:Outlier_Detection_for_PCA#Local_Outlier_Factor|Local Outlier Factor (LOF)]] is a outlier detection method that uses the local density around data points to determine if a point is an outlier. In this sense it can find outliers that global detection methods would not, as it identifies outliers in local areas.
  
-In Sift, LOF is built upon the [[Sift:Principal_Component_Analysis:Using_Principal_Component_Analysis_in_Biomechanics|PCA module]], to find outliers in the PC workspace scores. As such, we will need to create a PCA analysis. To show the benefits of Local Outlier Factor, we will be using the group "HipAngle_Z", as it has a good shape to demonstrate the effectiveness of LOF (multiple clusters of varying density). Specifically, create a PCA with:+In Sift, LOF is built upon the [[Sift:Principal_Component_Analysis|PCA module]], to find outliers in the PC workspace scores. As such, we will need to create a PCA analysis. To show the benefits of Local Outlier Factor, we will be using the group "HipAngle_Z", as it has a good shape to demonstrate the effectiveness of LOF (multiple clusters of varying density). Specifically, create a PCA with:
   * HipAngle_Z selected (and all workspaces)   * HipAngle_Z selected (and all workspaces)
   * 4 PCs calculated   * 4 PCs calculated
Line 37: Line 37:
 For now, we are looking for outliers from within the entire PCA analysis. We could partition it such that points only calculate their LOF against points in their own group or workspace (which we will show later), but for now we want to look at the whole graph as one, and as such we have selected "Combined Groups". For now, we are looking for outliers from within the entire PCA analysis. We could partition it such that points only calculate their LOF against points in their own group or workspace (which we will show later), but for now we want to look at the whole graph as one, and as such we have selected "Combined Groups".
  
-The number of neighbors is a tuneable parameter within the LOF calculation, and determines how large of a local grouping we will look at. According to the [[https://www.dbs.ifi.lmu.de/Publikationen/Papers/LOF.pdf|original paper]] introducing LOF, a k-value below ~10 can cause issues, so we will at least choose this. They also recommend choosing a k of at least the minimum number of points in a cluster. Looking through our data, a k value >= 20 seems reasonable, so we will use k=20.+The number of neighbors is a tuneable parameter within the LOF calculation, and determines how large of a local grouping we will look at. According to the [[https://dl.acm.org/doi/10.1145/342009.335388|original paper]] introducing LOF, a k-value below ~10 can cause issues, so we will at least choose this. They also recommend choosing a k of at least the minimum number of points in a cluster. Looking through our data, a k value >= 20 seems reasonable, so we will use k=20.
  
 We choose to use 2 PCs, as the workspace scores are presented in 2 dimensions. We choose to use 2 PCs, as the workspace scores are presented in 2 dimensions.
sift/tutorials/outlier_detection_with_pca.1771963587.txt.gz · Last modified: by wikisysop