CubeCraft-Creations/remote-rig
11
0
Fork 0
generated from CubeCraft-Creations/Tracehound
Code Issues 9 Pull Requests 1 Actions Packages Projects Releases 1 Wiki Activity
Files
dev-latest
remote-rig/docs/design/camera-auto-discovery.md
T
Hermes 95c225e51b
CI/CD / lint-and-typecheck (pull_request) Successful in 7s
Details
CI/CD / test (pull_request) Successful in 7s
Details
CI/CD / build (pull_request) Failing after 8s
Details
CI/CD / deploy (pull_request) Has been skipped
Details
CUB-229: Design camera auto-discovery and registration flow
2026-05-23 10:06:50 -04:00

20 KiB
Raw Permalink Blame History

Camera Auto-Discovery and Registration Flow — Design Document

Status: Draft | CUB: 229 | Date: 2026-05-23 Depends on: MQTT_CONTRACT.md v1.0.0 | Affects: CUB-189 (POST /cameras), CUB-232 (MQTT subscriber)

1. Overview

When a new ESP32 camera node powers on and connects to the travel router, it must self-register with the RemoteRig hub without any manual configuration. This document defines the auto-discovery protocol, message schemas, database extensions, error handling, and retry behavior.

Design Goals

  1. Zero-touch provisioning — ESP32 node registers itself on first MQTT connect; no dashboard interaction required
  2. Re-registration safe — same node rejoining after a reboot or network blip is recognized and re-associated, not duplicated
  3. Idempotent — replaying an announce due to MQTT retain or offline buffering does not create duplicate cameras
  4. Observable — the dashboard receives real-time SSE events when a camera appears or reconnects
  5. Backward compatible — existing announce format (MQTT_CONTRACT.md) is enhanced, not replaced

2. ESP32 Announce Message (Registration Request)

Topic

remoterig/cameras/+/announce

Direction: ESP32 → Hub | QoS: 2 | Retain: true

Published once on ESP32 first boot (or factory reset). Retained so the hub sees it even if it restarts after the ESP32 came online.

JSON Schema

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "CameraAnnounce",
  "type": "object",
  "required": ["mac_address", "firmware_version", "capabilities"],
  "properties": {
    "mac_address": {
      "type": "string",
      "pattern": "^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$",
      "description": "ESP32 Wi-Fi station MAC address — the stable, globally unique hardware identifier"
    },
    "firmware_version": {
      "type": "string",
      "pattern": "^\\d+\\.\\d+\\.\\d+$",
      "description": "Semver of the ESP32 firmware (e.g. 0.2.0)"
    },
    "capabilities": {
      "type": "array",
      "items": { "type": "string", "enum": ["start_stop", "status", "reboot", "heartbeat"] },
      "minItems": 1,
      "description": "Supported feature flags. Minimal: [\"status\"]. Full: [\"start_stop\", \"status\", \"reboot\", \"heartbeat\"]"
    },
    "friendly_name": {
      "type": "string",
      "maxLength": 64,
      "description": "Default human-readable name (e.g. 'ESP32-AA-BB-CC'). If omitted, hub generates one from the MAC."
    },
    "device_type": {
      "type": "string",
      "enum": ["esp32-gopro", "esp32-generic"],
      "default": "esp32-gopro",
      "description": "Device class for future multi-type support"
    },
    "mqtt_client_id": {
      "type": "string",
      "maxLength": 64,
      "description": "The MQTT client ID the ESP32 connected with (diagnostic)"
    },
    "sdk_version": {
      "type": "string",
      "description": "ESP-IDF or Arduino SDK version (diagnostic)"
    }
  }
}

Example — Minimal

{
  "mac_address": "AA:BB:CC:DD:EE:FF",
  "firmware_version": "0.1.0",
  "capabilities": ["status", "heartbeat"]
}

Example — Full

{
  "mac_address": "AA:BB:CC:DD:EE:FF",
  "firmware_version": "0.2.0",
  "capabilities": ["start_stop", "status", "reboot", "heartbeat"],
  "friendly_name": "GoPro Hero3 #1",
  "device_type": "esp32-gopro",
  "mqtt_client_id": "remoterig-ddeeff",
  "sdk_version": "ESP-IDF v5.1.4"
}

MAC Address as Identity

The ESP32's Wi-Fi station MAC is the only stable, globally unique identifier available on a closed network (no cloud, no serial number burned at factory). It is:

  • Globally unique — OUI-assigned by Espressif
  • Immutable — persists across firmware flashes and reboots
  • Available before MQTT connect — no dependency on hub-assigned ID

The hub maps mac_address → camera_id. The camera_id (e.g. cam-001) is a short, human-friendly alias assigned at registration time.

3. Hub Response Protocol

When the hub processes an announce, it MUST publish a response so the ESP32 knows its registration outcome. The response goes to the command topic for the assigned camera.

Response Topic

remoterig/cameras/<camera_id>/command

Direction: Hub → ESP32 | QoS: 2 | Retain: false

Response Schema

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "RegistrationResponse",
  "type": "object",
  "required": ["command", "request_id"],
  "properties": {
    "command": {
      "type": "string",
      "enum": ["registered", "registration_error"],
      "description": "Outcome of the registration request"
    },
    "request_id": {
      "type": "string",
      "description": "Echo of the announce message's MAC + timestamp hash for correlation"
    },
    "camera_id": {
      "type": "string",
      "pattern": "^cam-\\d{3}$",
      "description": "Assigned camera ID (present on success only)"
    },
    "error_code": {
      "type": "string",
      "enum": ["INVALID_MAC", "CAPABILITY_REQUIRED", "DB_WRITE_FAILED", "RATE_LIMITED"],
      "description": "Machine-readable error code (present on failure only)"
    },
    "error_message": {
      "type": "string",
      "description": "Human-readable error description (present on failure only)"
    },
    "retry_after_sec": {
      "type": "integer",
      "minimum": 5,
      "description": "Suggested retry delay in seconds (present on failure only)"
    },
    "timestamp": {
      "type": "string",
      "format": "date-time",
      "description": "ISO 8601 — hub clock time of the response"
    }
  }
}

Success Response Example

{
  "command": "registered",
  "request_id": "req-AABBCCDDEEFF-1684771200",
  "camera_id": "cam-004",
  "timestamp": "2026-05-23T14:30:00Z"
}

Error Responses

error_code Meaning retry_after_sec ESP32 action
INVALID_MAC MAC address absent or malformed — (fatal) Log error, halt registration
CAPABILITY_REQUIRED No valid capabilities specified — (fatal) Log error, halt registration
DB_WRITE_FAILED Hub database is unavailable (disk full, etc.) 60 Retry after delay
RATE_LIMITED Too many registration attempts in a window 30 Retry after delay

Example error response:

{
  "command": "registration_error",
  "request_id": "req-AABBCCDDEEFF-1684771200",
  "error_code": "DB_WRITE_FAILED",
  "error_message": "Database write failed: disk I/O error",
  "retry_after_sec": 60,
  "timestamp": "2026-05-23T14:30:00Z"
}

ESP32 Retry Logic

ESP32 publishes announce (QoS 2, retain)
  │
  ├── Subscribe to remoterig/cameras/+/command (QoS 2)
  │
  ├── Wait for command = "registered" or "registration_error"
  │
  ├── Timeout after 30s → retry announce (with exponential backoff)
  │         ├── 1st attempt: immediate
  │         ├── 2nd attempt: wait 5s
  │         ├── 3rd attempt: wait 10s
  │         ├── 4th attempt: wait 20s
  │         └── 5th+ attempt: wait 30s, repeat every 30s
  │
  ├── On success (registered): store camera_id in NVS, begin normal status loop
  │
  ├── On fatal error (INVALID_MAC, CAPABILITY_REQUIRED):
  │       Log error, blink LED pattern, do not retry
  │
  └── On transient error (DB_WRITE_FAILED, RATE_LIMITED):
          Wait retry_after_sec (capped at 120s), then re-publish announce

After successful registration: On subsequent boots, the ESP32 reads camera_id from NVS (non-volatile storage). It does NOT re-publish announce unless:

  • camera_id is missing from NVS (factory reset / first boot)
  • The hub publishes command: "reregister" to force re-registration (admin action)

4. Hub Processing Logic

Registration Flow

Hub receives announce on remoterig/cameras/+/announce
  │
  ├── 1. VALIDATE: mac_address present? matches pattern? → if no: publish INVALID_MAC error
  │
  ├── 2. VALIDATE: capabilities non-empty? → if no: publish CAPABILITY_REQUIRED error
  │
  ├── 3. RATE LIMIT: >5 registrations from same IP/MAC in 60s? → RATE_LIMITED error
  │
  ├── 4. LOOKUP: SELECT camera_id FROM cameras WHERE mac_address = ?
  │         │
  │         ├── FOUND → EXISTING CAMERA:
  │         │   ├── Update: friendly_name, firmware_version, capabilities, updated_at
  │         │   ├── Publish registered response with existing camera_id
  │         │   ├── SSE broadcast: "camera_reconnected"
  │         │   └── Clear MQTT stale announce (publish empty retained message)
  │         │
  │         └── NOT FOUND → NEW CAMERA:
  │             ├── Generate camera_id: "cam-NNN" (sequential)
  │             ├── INSERT into cameras
  │             ├── Publish registered response with new camera_id
  │             ├── SSE broadcast: "camera_registered"
  │             └── Clear MQTT stale announce (publish empty retained message)
  │
  └── 5. CLEANUP: Publish zero-byte retained message to announce topic
       (prevents stale announces after camera is registered)

Rate Limiting

To protect against buggy firmware or network loops:

Window Max Attempts Action
60 seconds 5 per MAC Reject with RATE_LIMITED, retry_after_sec: 30
5 minutes 20 per MAC Reject with RATE_LIMITED, retry_after_sec: 60

Rate limit state is in-memory only (not persisted). Restarting the hub resets the counters.

5. Database Schema Changes

Extended cameras Table

-- Migration: 002_add_camera_registration_fields.sql

ALTER TABLE cameras ADD COLUMN firmware_version TEXT;
ALTER TABLE cameras ADD COLUMN capabilities TEXT NOT NULL DEFAULT '["status"]';
ALTER TABLE cameras ADD COLUMN device_type TEXT NOT NULL DEFAULT 'esp32-gopro';
ALTER TABLE cameras ADD COLUMN registration_status TEXT NOT NULL DEFAULT 'pending'
    CHECK(registration_status IN ('pending', 'registered', 'error', 'decommissioned'));
ALTER TABLE cameras ADD COLUMN last_announce_at DATETIME;
ALTER TABLE cameras ADD COLUMN registration_error TEXT;
ALTER TABLE cameras ADD COLUMN mqtt_client_id TEXT;

-- Index for MAC lookups (already exists but confirm)
-- CREATE INDEX IF NOT EXISTS idx_cameras_mac ON cameras(mac_address);

-- Index for registration status filtering
CREATE INDEX IF NOT EXISTS idx_cameras_reg_status ON cameras(registration_status);

-- Index for finding stale registrations (cameras that announced but never sent status)
CREATE INDEX IF NOT EXISTS idx_cameras_last_announce ON cameras(last_announce_at);

Full Table Definition (post-migration)

Column Type Constraints Description
camera_id TEXT PK Hub-assigned short ID, e.g. cam-001
friendly_name TEXT NOT NULL Human-readable name
mac_address TEXT UNIQUE ESP32 Wi-Fi station MAC
firmware_version TEXT — Firmware semver reported by ESP32
capabilities TEXT NOT NULL, DEFAULT '["status"]' JSON array of strings
device_type TEXT NOT NULL, DEFAULT 'esp32-gopro' Device class
registration_status TEXT NOT NULL, DEFAULT 'pending' pending, registered, error, decommissioned
last_announce_at DATETIME — Timestamp of most recent announce
registration_error TEXT — Last registration error message (cleared on success)
mqtt_client_id TEXT — MQTT client ID from the announce
created_at DATETIME NOT NULL, DEFAULT datetime('now') First registration timestamp
updated_at DATETIME NOT NULL, DEFAULT datetime('now') Last update timestamp

Go Model Extension

The existing models.Camera struct gains:

type Camera struct {
    CameraID           string    `json:"camera_id"`
    FriendlyName       string    `json:"friendly_name"`
    MacAddress         string    `json:"mac_address,omitempty"`
    FirmwareVersion    string    `json:"firmware_version,omitempty"`
    Capabilities       []string  `json:"capabilities"`
    DeviceType         string    `json:"device_type"`
    RegistrationStatus string    `json:"registration_status"`
    LastAnnounceAt     *time.Time `json:"last_announce_at,omitempty"`
    RegistrationError  string    `json:"registration_error,omitempty"`
    MqttClientID       string    `json:"mqtt_client_id,omitempty"`
    CreatedAt          time.Time `json:"created_at"`
    UpdatedAt          time.Time `json:"updated_at"`
}

Note on capabilities storage: SQLite does not have a native JSON array type. Store as TEXT (JSON-encoded array). Serialize/deserialize in the Go model layer. Migration default is '["status"]' — the minimum capability for a useful camera.

6. Registration Flow Sequence Diagram

sequenceDiagram
    participant ESP32
    participant Broker as MQTT Broker (Mosquitto)
    participant Hub as Go Hub
    participant DB as SQLite
    participant SSE as SSE Hub
    participant UI as Dashboard UI

    Note over ESP32: Power on / First boot

    ESP32->>ESP32: Read camera_id from NVS
    alt camera_id NOT in NVS (first boot or factory reset)
        ESP32->>Broker: CONNECT (client_id: remoterig-<mac_last6>)
        Broker-->>ESP32: CONNACK

        ESP32->>Broker: SUBSCRIBE remoterig/cameras/+/command (QoS 2)
        Broker-->>ESP32: SUBACK

        ESP32->>Broker: PUBLISH remoterig/cameras/announce (QoS 2, retain)
        Note over ESP32,Broker: {mac_address, firmware_version, capabilities, ...}
        Broker->>Hub: Forward announce

        Hub->>Hub: Validate: MAC present? capabilities non-empty?
        alt Validation fails
            Hub->>Broker: PUBLISH command {command: "registration_error", error_code: "INVALID_MAC"}
            Broker->>ESP32: Forward error
            Note over ESP32: Log error, halt (fatal)
        else Validation passes
            Hub->>Hub: Rate limit check
            alt Rate limited
                Hub->>Broker: PUBLISH command {error_code: "RATE_LIMITED", retry_after_sec: 30}
                Broker->>ESP32: Forward error
                Note over ESP32: Wait retry_after_sec, retry
            else Allowed
                Hub->>DB: SELECT camera_id WHERE mac_address = ?
                alt MAC already registered
                    DB-->>Hub: camera_id = "cam-002"
                    Hub->>DB: UPDATE cameras SET firmware_version, capabilities, friendly_name, ...
                    Hub->>SSE: Broadcast "camera_reconnected"
                else New MAC
                    DB-->>Hub: no rows
                    Hub->>DB: SELECT MAX(camera_id) → "cam-003"
                    Hub->>Hub: Generate "cam-004"
                    Hub->>DB: INSERT INTO cameras (cam-004, ...)
                    Hub->>SSE: Broadcast "camera_registered"
                end

                Hub->>Broker: PUBLISH command {command: "registered", camera_id: "cam-004"}
                Broker->>ESP32: Forward registration response

                Hub->>Broker: PUBLISH announce (zero-byte retain) — clear stale announce

                SSE-->>UI: camera_registered / camera_reconnected event
                UI->>UI: Show new camera card in grid
            end
        end
    else camera_id FOUND in NVS (subsequent boot)
        Note over ESP32: Skip announce, proceed to status loop
        ESP32->>Broker: PUBLISH status (QoS 1, retain)
        Broker->>Hub: Forward status
        Hub->>SSE: Broadcast camera_status
        SSE-->>UI: Update camera card
    end

7. Reconnection vs. Registration

It is critical to distinguish two scenarios:

Scenario A: Reconnection (camera was previously registered)

ESP32 boots → reads camera_id from NVS → publishes status on remoterig/cameras/<id>/status
→ Hub sees status on a known camera_id → updates online flag → SSE broadcast

No announce published. The camera already has its identity.

Scenario B: First Registration (or factory reset)

ESP32 boots → NVS empty → publishes announce → Hub assigns camera_id →
ESP32 stores camera_id in NVS → begins status loop on remoterig/cameras/<id>/status

Scenario C: Hub Restart (ESP32 already running)

Hub restarts → subscribes to remoterig/cameras/+/announce →
MQTT broker delivers retained announce messages →
Hub processes each → re-registration safe (MAC already exists → update only)

This is why announce messages use retain: true. If the hub restarts while ESP32s are running, it re-discovers them from retained announces.

8. Security Considerations

Concern Mitigation
Rogue node spoofing a MAC Closed network (travel router, no internet). MAC filtering at the router level as defense-in-depth (future).
Replay attacks Announce is idempotent — replaying it only updates timestamps, doesn't create duplicates.
Denial of registration Rate limiting (Section 4) prevents flooding.
Unauthorized decommission No decommission MQTT command exists. Decommission is admin-only via HTTP API with API key auth.

9. Open Questions & Decisions

Question Decision Rationale
MAC as identity? ✅ Yes Only globally unique, immutable ID available on a closed network.
camera_id format? cam-NNN (zero-padded sequential) Short, sortable, human-friendly. Collision-free with DB sequence.
Re-registration behavior? Update existing, don't create duplicate Announcing with same MAC = reconnection, not new camera.
Retain on announce? ✅ Yes, cleared after processing Allows hub restart recovery. Cleanup prevents stale data.
Response protocol? Publish to command topic Reuses existing command channel. ESP32 subscribes before publishing announce.
Capabilities stored? ✅ Yes, in capabilities column Enables future feature gating (e.g., "this camera can't start/stop recording").
device_type added? ✅ Yes, default esp32-gopro Allows future camera types (e.g., Raspberry Pi CSI, USB webcam).
Dashboard rename after auto-registration? ✅ Yes (via existing POST /cameras or settings API in future) Already called out in MQTT_CONTRACT.md. No new work in this CUB.
NVS key for camera_id? "cam_id" Simple, unambiguous.

10. Implementation Plan

This design document covers the protocol and schema design. Implementation is tracked in the following sub-issues:

CUB Title Agent Depends On
CUB-229 Design camera auto-discovery and registration flow Dex — (this task)
CUB-229a Migration: add registration fields to cameras table Hex CUB-229
CUB-229b Go model update: Camera struct with new fields Dex CUB-229a
CUB-229c MQTT subscriber: registration response protocol Dex CUB-229b
CUB-229d Rate limiting for announce messages Dex CUB-229b
CUB-229e SSE events: camera_registered / camera_reconnected Dex CUB-229c
CUB-229f ESP32 firmware: NVS storage + announce on first boot Pip CUB-229
CUB-229g ESP32 firmware: command subscription + registration ACK handling Pip CUB-229c
CUB-229h Update MQTT_CONTRACT.md with registration response spec Dex CUB-229
CUB-229i Integration test: camera auto-registration end-to-end Dex/Pip CUB-229e, CUB-229g

11. References

Reference in New Issue View Git Blame Copy Permalink