Hear to Help
UC Berkeley MIDS Capstone
An edge-deployed AI system that passively monitors household audio to detect elderly falls in real-time,enabling privacy-preserving safety monitoring without cameras or wearables.
The Challenge
93% of adults 65+ want to age in place (CDC)
Every second, an older adult suffers a fall. That adds up to 36 million falls per year and 32,000+ deaths in the U.S. alone. For elderly individuals living independently, undetected falls are one of the leading causes of fatal outcomes.
Expensive, poor reliability, users forget to wear them or refuse due to stigma.
Major privacy concerns, ethical issues, high cost, and limited room coverage.
Our Approach
AudioAid is a Raspberry Pi with an attached microphone that passively listens to household sounds and classifies falls using a Convolutional Neural Network, then instantly sends SMS alerts via Twilio to caregivers and emergency contacts.
Works in any room, any layout. No line-of-sight requirements like cameras , sound travels around corners and through doorways.
Zero user interaction required. No devices to wear, charge, or remember. Always on, always monitoring , completely passive.
Privacy-first design. Audio is processed locally on-device and never stored or transmitted , only classification results trigger alerts.
Technical Deep Dive
With no publicly available fall audio datasets, we built our own from scratch , recording a 150lb test dummy alongside 7 real human volunteers (ages 20–57) across 3 different microphones (22kHz–192kHz sample rates).
Augmentation techniques: frequency masking, time masking, and time stretching to increase dataset diversity and model robustness.
Raw audio waveforms were transformed into visual representations that capture both frequency and temporal information , critical for distinguishing the sharp, broadband impact of a fall from everyday household sounds.
After experimenting with 25+ model configurations, the top-performing architecture uses MFCC input features through a CNN with the following layer structure:
A higher classification threshold of 0.9 (vs standard 0.5) was applied in production to minimize false positives while maintaining zero false negatives , critical for a safety application.
| Precision | Recall | F1 Score | |
|---|---|---|---|
| Fall | 0.97 | 1.00 | 0.99 |
| No Fall | 1.00 | 0.98 | 0.99 |
Mean accuracy: 0.96
Every real fall was correctly detected , the most critical metric for a safety application.
What This Demonstrates
CNN design, training, hyperparameter tuning, and model evaluation across 25+ experiments to optimize for safety-critical performance.
MFCCs, Mel Spectrograms, audio augmentation (time stretch, masking), and feature engineering from raw waveforms.
TensorFlow Lite deployment on Raspberry Pi, real-time inference on constrained hardware with optimized model conversion.
Custom dataset creation from scratch, audio preprocessing pipeline, data augmentation strategies, and data quality validation.
End-to-end ownership from data collection to alerting system , including Twilio integration, UX considerations, and stakeholder communication.
Raspberry Pi configuration, microphone input handling, Sounddevice/Portaudio setup, and hardware-software interface development.
Privacy-first design philosophy, ethical evaluation framework, domain expert consultation, and IRB-style considerations for human subjects.
K-fold validation, threshold tuning for safety-critical thresholds, confusion matrix analysis, and systematic experimentation methodology.
Reflection
With no publicly available fall audio datasets, we designed and executed a custom data collection protocol with volunteers across age groups , building the first dataset of its kind.
No prior experience with audio ML or hardware deployment. Self-taught through research papers, expert consultation, and iterative experimentation across 25+ model configurations.
AudioAid is the first purely sound-based fall detection system , a novel approach in a space dominated by wearable sensors and computer vision.