# Facial Recognition: OAK-D + Coral Edge TPU Add face detection and recognition to the oak-service spatial pipeline. ## Architecture ``` OAK-D Lite (Myriad X) Coral Edge TPU Host (Pi 5) ────────────────────── ────────────── ─────────── yolov6-nano spatial ssd_mobilenet_v2_face crop person bbox → person bboxes → face bboxes cosine similarity → spatial coords (X,Y,Z) arcface/facenet edgetpu vs SQLite DB → RGB frames → 128-dim embedding → name + confidence ``` Per detection cycle (~0.5s): 1. OAK-D outputs person detections + spatial coords + RGB frame (unchanged) 2. Host crops upper-body region from RGB for each person bbox 3. Coral runs face detection on crop (ssd_mobilenet_v2_face edgetpu) 4. If face found, crop face, resize to model input, run embedding via Coral 5. Host compares embedding against SQLite DB (cosine similarity) 6. Attach recognized_name + recognition_confidence to detection ## Setup: Coral Runtime Install pycoral + tflite-runtime in the oak-service venv: ```bash pip install tflite-runtime pycoral ``` Download Edge TPU models: - ssd_mobilenet_v2_face_quant_postprocess_edgetpu.tflite - face embedding model (facenet or arcface quantized for edgetpu) Models stored in oak-service/models/ directory. ## SQLite Face Database Path: configurable, default `faces.db` in service directory. ```sql CREATE TABLE faces ( id INTEGER PRIMARY KEY, name TEXT NOT NULL, embedding BLOB NOT NULL, enrolled_at REAL NOT NULL, source TEXT ); CREATE INDEX idx_faces_name ON faces(name); ``` - Multiple embeddings per person (different angles/lighting) - Embedding stored as packed float32 bytes - Matching: cosine similarity, threshold ~0.5 for positive match - Best match across all embeddings for a name wins ## API Changes New endpoints: - `POST /faces/enroll` — multipart: name + photo, or name + use current frame - `GET /faces` — list enrolled names with embedding count - `DELETE /faces/{name}` — remove person from DB Modified responses: - `/presence` adds: recognized_name, recognition_confidence - `/detections` adds per-detection: recognized_name, recognition_confidence ## Files - `oak_service_spatial.py` — add Coral face pipeline to detection loop - `models/` — Edge TPU model files - `faces.db` — SQLite database (created on first run) ## Verification 1. Install Coral runtime, verify device detected 2. Download face models, verify inference runs 3. Enroll a face via API 4. Test recognition: stand in front of camera, check /presence for name 5. Test unknown: different person, should show "unknown"