Data augmentation for inertial sensor based human action recognition using deep learning

Human Activity Recognition (HAR) approaches are predominantly based on supervised deep learning and benefit from large amounts of labeled data - an expensive resource. Data augmentation enriches labeled datasets by adding synthetic data, which is substantially cheaper, and often results in improved model performance, but is very rarely used for sensor data. This work explores data augmentation for inertial-sensor-based HAR by transforming the data through physically interpretable operations. The main studies were conducted on the Opportunity and the Overhead Car Assembly (OCA) datasets. For these experiments, only 20% of the available training data were used, and the experiments were conducted in an 8-fold cross-validation procedure over different subsets of the training data. The results show that simple geometric augmentations can be beneficial in many cases. Timewarping proved to offer the most reliable single augmentation, improving the average F1 score of Opportunity from 0.570 to 0.597 and of OCA Mixed from 0.884 to 0.906. Combining augmentations improved the accuracy in almost all scenarios but to a degree comparable to timewarping. Applying augmentations on all the available training data improved the F1 score compared to the base case with no augmentations, although this effect is more pronounced for datasets with more similar training and test data: for the OCA Mixed variant, the average F1 score improved from 0.917 to 0.933, while for the OCA Leave-One-Out (LOT) variant, the average F1 score did not significantly change. For Opportunity, which similarly to OCA LOT uses a participant-based training-test split, the F1 score improved from 0.684 to 0.697.