Detect and Repair: Robust Self-Supervised Wearable Sensing Under Missing Modalities.

Aboul Hassane Cisse, Shoya Ishimaru

Wearable sensor systems are being increasingly deployed in real-world environments to monitor human activities and cognitive states. However, such systems frequently suffer from degraded or missing sensor modalities due to occlusions, energy constraints, or hardware failures. In this work, we introduce CognifySSL v2.0, a self-supervised learning framework designed to detect and repair missing modalities in real time under simulated real-world missing-modality conditions. The model combines contrastive and masked modeling objectives across multiple physiological and motion signals (e.g., IMU, ECG, EDA) using a fusion architecture with dropout simulation. Evaluation on WESAD demonstrated effective multimodal detection and reconstruction under missing-modality conditions, while experiments on MobiAct validated unimodal robustness and representation learning under sensor dropout. We released our code and interactive visualization dashboard to support reproducibility and future research on robust multimodal fusion.

Read on ELI