vixy/headmic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update docs — complete binaural roadmap (10/12 features)

Browse Source 
BINAURAL_ROADMAP: Full status update with implementation details,
three-signal localization table, key discoveries section.

README: Updated features table (ITD, multi-speaker, cocktail party),
new API endpoints (/speakers/tracked, /speakers/focus), file structure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alex
2026-04-12 21:55:09 -05:00
parent 8073e3eb02
commit 5c72491ee9
2 changed files with 126 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
README.md
View File

@@ -49,11 +49,14 @@ Binaural hearing service for Vixy's physical head. Dual mic arrays with spatial
| Wake word detection | Porcupine | CPU | Needs Picovoice key |
| Sound classification | sound_id.py | Coral Edge TPU | 521 classes, ~2ms |
| Speaker identification | speaker_id.py | CPU (Resemblyzer) | Enrollment via API |
| Spatial tracking | spatial.py | USB control | Triangulated gaze + ILD distance |
| Spatial tracking | spatial.py | USB control | 3-signal fusion: DoA + ILD + ITD |
| Distance estimation | spatial.py | audio energy | Proximity zones (intimate/conversational/across_room/far) |
| ITD processing | spatial.py | audio cross-correlation | Sub-ms delay → bearing angle |
| Multi-speaker tracking | multi_speaker.py | XVF3800 beam steering | 2 simultaneous speakers, auto beam lock |
| Cocktail party filtering | multi_speaker.py + audio_stream.py | beam gating + focus | Target speaker isolation |
| Spatial scene mapping | spatial_scene.py | — | Learns where sounds come from, anomaly detection |
| Sound event localization | spatial_scene.py | — | What + where + when log |
| Best-beam selection | audio_stream.py | 2× XVF3800 | Energy-based, 10% hysteresis |
| Best-beam selection | audio_stream.py | 2× XVF3800 | Energy-based or focused attention |
| LED control | xvf3800.py | WS2812 rings | DoA/solid/breath |
| Binaural recording | binaural_recorder.py | 2× XVF3800 | Stereo WAV segments (opt-in) |
@@ -174,8 +177,10 @@ sudo systemctl start headmic
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/doa` | GET | DoA from both arrays + triangulated position + gaze + distance + proximity |
| `/doa` | GET | DoA + triangulated position + ILD + ITD + gaze + distance + proximity |
| `/devices` | GET | XVF3800 connection status, serials, ALSA devices |
| `/speakers/tracked` | GET | Multi-speaker positions, beam mode, lock state, target |
| `/speakers/focus` | POST | Switch cocktail party attention (query: speaker=0\|1) |
| `/scene` | GET | Learned spatial scene (usual direction per category) + last anomaly |
| `/scene/events` | GET | Recent sound events with what + where + when (query: seconds, category) |
| `/scene/heatmap` | GET | Per-category angular distribution for visualization |
@@ -243,9 +248,10 @@ sudo systemctl start headmic
headmic/
├── headmic.py # Main FastAPI service
├── audio_stream.py # Dual arecord streams + best-beam selection
├── spatial.py # Triangulation + ILD distance + smooth gaze + proximity
├── spatial.py # 3-signal fusion (DoA + ILD + ITD) + gaze + proximity
├── spatial_scene.py # Spatial audio scene map + anomaly detection
├── xvf3800.py # USB vendor control (DoA + LEDs)
├── multi_speaker.py # Multi-speaker tracking + beam steering + cocktail party
├── xvf3800.py # USB vendor control (DoA + LEDs + beam steering)
├── sound_id.py # YAMNet sound classification (CPU/Edge TPU)
├── speaker_id.py # Resemblyzer speaker identification
├── binaural_recorder.py # Stereo WAV recording from both ears
@@ -277,4 +283,5 @@ Commands use USB vendor control transfers: `wValue = cmdid`, `wIndex = resid`.
*Built by Vixy on Day 77 (January 17, 2026)*
*Upgraded to dual XVF3800 binaural hearing on Day 161 (April 2026)*
*Full binaural suite (10/12 features) built Day 162*
*"Hey Vivi" — the words that summon me* 💜