generated from CubeCraft-Creations/Tracehound
281 lines
12 KiB
Markdown
281 lines
12 KiB
Markdown
# MQTT Message Format Contract — RemoteRig
|
||
|
||
> **Version:** 1.0.0 | **Status:** Draft | **Blocks:** CUB-232 (MQTT subscriber), CUB-174 (ESP32 firmware)
|
||
> **Last updated:** 2026-05-21
|
||
|
||
## Network Architecture
|
||
|
||
```
|
||
┌──────────────────────────────────┐
|
||
│ Travel Router (10.60.1.1) │
|
||
│ DHCP: .100-.200 │
|
||
└──────┬──────────┬──────────┬──────┘
|
||
│ │ │
|
||
┌───────────────┘ │ └───────────────┐
|
||
▼ ▼ ▼
|
||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||
│ ESP32 #1 │ │ ESP32 #2 │ │ Pi Zero 2 W │
|
||
│ 10.60.1.101 │ │ 10.60.1.102 │ │ 10.60.1.56 │
|
||
│ STA→Router │ │ STA→Router │ │ Mosquitto │
|
||
│ MQTT relay │ │ MQTT relay │ │ Go backend │
|
||
└──────┬───────┘ └──────┬───────┘ │ React UI │
|
||
│ UART │ UART └──────────────┘
|
||
▼ ▼
|
||
┌──────────────┐ ┌──────────────┐
|
||
│ ESP8266 #1 │ │ ESP8266 #2 │
|
||
│ STA→GoPro AP │ │ STA→GoPro AP │
|
||
│ HTTP→10.5.5.1│ │ HTTP→10.5.5.1│
|
||
└──────┬───────┘ └──────┬───────┘
|
||
▼ ▼
|
||
┌──────────────┐ ┌──────────────┐
|
||
│ GoPro Hero 3 │ │ GoPro Hero 3 │
|
||
└──────────────┘ └──────────────┘
|
||
```
|
||
|
||
- **Travel router:** Self-contained, no internet. Gateway `10.60.1.1`. DHCP pool: `10.60.1.100-200`
|
||
- **Pi Zero 2 W:** Static IP `10.60.1.56`. Runs Mosquitto (port 1883), Go backend (port 8080), serves React UI
|
||
- **ESP32s:** DHCP from router. Each stays on the travel-router LAN, relays MQTT to/from its paired ESP8266 over UART
|
||
- **User device:** Connects to router, opens `http://10.60.1.56:8080` for dashboard
|
||
|
||
## MQTT Broker
|
||
|
||
- **Host:** `10.60.1.56` (Pi Zero 2 W)
|
||
- **Port:** `1883` (default MQTT, no TLS — closed network)
|
||
- **Auth:** None (closed network, no external access)
|
||
- **Client ID format:** `remoterig-<esp32_mac_last6>` (e.g., `remoterig-a1b2c3`)
|
||
- **QoS:** 1 (at least once) for status/heartbeat. 2 (exactly once) for commands.
|
||
- **Retain:** Status messages use `retain: true` so new subscribers get latest state immediately
|
||
|
||
## Topic Hierarchy
|
||
|
||
```
|
||
remoterig/
|
||
├── cameras/
|
||
│ └── <camera_id>/
|
||
│ ├── status ← ESP32 publishes (retained, QoS 1)
|
||
│ ├── heartbeat ← ESP32 publishes (QoS 1, not retained)
|
||
│ ├── command → Hub publishes (QoS 2)
|
||
│ └── announce ← ESP32 publishes on first boot (QoS 2, retained)
|
||
└── hub/
|
||
└── status ← Hub publishes (retained, QoS 1)
|
||
```
|
||
|
||
### Topic: `remoterig/cameras/<camera_id>/status`
|
||
|
||
**Direction:** ESP32 → Hub
|
||
**QoS:** 1 | **Retain:** true | **Interval:** 30 seconds
|
||
|
||
Published by the ESP32 every 30s using the latest GoPro status received from the paired ESP8266 over UART.
|
||
|
||
```json
|
||
{
|
||
"camera_id": "cam-001",
|
||
"timestamp": "2026-05-21T18:30:00Z",
|
||
"battery_pct": 85,
|
||
"battery_raw": 217,
|
||
"video_remaining_sec": 3420,
|
||
"recording": true,
|
||
"mode": "video",
|
||
"resolution": "1080p",
|
||
"fps": 30,
|
||
"online": true,
|
||
"rssi": -52,
|
||
"uptime_sec": 1247
|
||
}
|
||
```
|
||
|
||
| Field | Type | Required | Description |
|
||
|-------|------|----------|-------------|
|
||
| `camera_id` | string | ✅ | Unique camera identifier (set during registration) |
|
||
| `timestamp` | ISO 8601 | ✅ | ESP32 clock time when status was read |
|
||
| `battery_pct` | int 0-100 | ✅ | Calibrated battery percentage (null if uncalibrated → omit) |
|
||
| `battery_raw` | int 0-255 | — | Raw byte from GoPro status offset 57 |
|
||
| `video_remaining_sec` | int | — | Estimated remaining recording seconds (null if unavailable) |
|
||
| `recording` | bool | ✅ | Whether camera is currently recording |
|
||
| `mode` | string | — | Current mode (e.g., "video", "photo", "burst") |
|
||
| `resolution` | string | — | Current resolution string |
|
||
| `fps` | int | — | Current frames per second |
|
||
| `online` | bool | ✅ | ESP32 can reach the GoPro (false if GoPro AP unreachable) |
|
||
| `rssi` | int | — | Wi-Fi RSSI to GoPro AP (dBm, negative) |
|
||
| `uptime_sec` | int | — | ESP32 uptime in seconds |
|
||
|
||
### Topic: `remoterig/cameras/<camera_id>/heartbeat`
|
||
|
||
**Direction:** ESP32 → Hub
|
||
**QoS:** 1 | **Retain:** false | **Interval:** 60 seconds
|
||
|
||
Lightweight keepalive so the hub can detect dead ESP32s.
|
||
|
||
```json
|
||
{
|
||
"camera_id": "cam-001",
|
||
"timestamp": "2026-05-21T18:31:00Z",
|
||
"uptime_sec": 1307,
|
||
"free_heap": 28672
|
||
}
|
||
```
|
||
|
||
| Field | Type | Description |
|
||
|-------|------|-------------|
|
||
| `camera_id` | string | Camera identifier |
|
||
| `timestamp` | ISO 8601 | Current ESP32 time |
|
||
| `uptime_sec` | int | ESP32 uptime |
|
||
| `free_heap` | int | Free heap in bytes (diagnostic) |
|
||
|
||
**Hub behavior:** If no heartbeat for 120 seconds, mark camera as offline (`online: false` in SSE broadcast).
|
||
|
||
### Topic: `remoterig/cameras/<camera_id>/command`
|
||
|
||
**Direction:** Hub → ESP32
|
||
**QoS:** 2 | **Retain:** false
|
||
|
||
Commands sent from the dashboard to individual cameras.
|
||
|
||
```json
|
||
{
|
||
"command": "start_recording",
|
||
"request_id": "req-abc123",
|
||
"timestamp": "2026-05-21T18:32:00Z"
|
||
}
|
||
```
|
||
|
||
**Supported commands:**
|
||
|
||
| `command` | Description | Response topic |
|
||
|-----------|-------------|----------------|
|
||
| `start_recording` | Start GoPro recording | status (updated on next poll) |
|
||
| `stop_recording` | Stop GoPro recording | status (updated on next poll) |
|
||
| `reboot` | Reboot the ESP32 | — (ESP32 reconnects after boot) |
|
||
|
||
**ESP32 / ESP8266 behavior:**
|
||
- ESP32 receives the MQTT command and forwards it over UART to the paired ESP8266
|
||
- ESP8266 executes the corresponding HTTP command against the GoPro AP
|
||
- Next status publish will reflect the new state
|
||
- If command fails (GoPro unreachable), publish status with `online: false`
|
||
|
||
### Topic: `remoterig/cameras/<camera_id>/announce`
|
||
|
||
**Direction:** ESP32 → Hub
|
||
**QoS:** 2 | **Retain:** true
|
||
|
||
Published once on ESP32 first boot (or factory reset). Used for auto-registration.
|
||
|
||
```json
|
||
{
|
||
"mac_address": "AA:BB:CC:DD:EE:FF",
|
||
"firmware_version": "0.1.0",
|
||
"capabilities": ["start_stop", "status"],
|
||
"friendly_name": "ESP32-AA-BB-CC"
|
||
}
|
||
```
|
||
|
||
| Field | Type | Description |
|
||
|-------|------|-------------|
|
||
| `mac_address` | string | ESP32 Wi-Fi MAC address |
|
||
| `firmware_version` | string | ESP32 firmware semver |
|
||
| `capabilities` | string[] | Supported features |
|
||
| `friendly_name` | string | Default human-readable name |
|
||
|
||
**Hub behavior on first announce:**
|
||
1. Check if MAC already registered → if yes, update `friendly_name` and log
|
||
2. If new MAC → create camera with auto-generated `camera_id = "cam-<NNN>"` (zero-padded sequential)
|
||
3. Respond by publishing: `remoterig/cameras/<camera_id>/command` with `command: "registered"` payload containing the assigned `camera_id`
|
||
4. Broadcast via SSE that a new camera appeared
|
||
|
||
### Topic: `remoterig/hub/status`
|
||
|
||
**Direction:** Hub → All
|
||
**QoS:** 1 | **Retain:** true | **Interval:** 30 seconds
|
||
|
||
Hub health status broadcast.
|
||
|
||
```json
|
||
{
|
||
"version": "0.2.0",
|
||
"uptime_sec": 86400,
|
||
"cameras_online": 3,
|
||
"cameras_total": 4,
|
||
"mqtt_connected": true,
|
||
"db_size_bytes": 1048576
|
||
}
|
||
```
|
||
|
||
## Message Validation Rules
|
||
|
||
### Hub-side (incoming from ESP32)
|
||
|
||
1. **Required fields:** `camera_id` and `timestamp` must be present in all messages
|
||
2. **Timestamp sanity:** Reject if timestamp is > 5 minutes in the future or > 24 hours in the past
|
||
3. **Duplicate detection:** Status messages with same `(camera_id, timestamp)` are ignored (idempotent)
|
||
4. **Schema validation:** Unknown fields are ignored (forward-compatible), missing required fields → log warning + reject
|
||
5. **battery_pct bounds:** If present, must be 0–100. Out of range → clamp to [0,100] with warning
|
||
|
||
### ESP32-side (incoming from hub)
|
||
|
||
1. **Acknowledge commands:** After processing a command, the next status publish reflects the new state
|
||
2. **Unknown commands:** Log and ignore
|
||
3. **Malformed JSON:** Log error, ignore message
|
||
|
||
## Session Lifecycle
|
||
|
||
```
|
||
ESP32 boots
|
||
│
|
||
├── Connects to travel router Wi-Fi
|
||
├── Connects to MQTT broker (10.60.1.56:1883)
|
||
├── Publishes announce (retained) on cameras/<id>/announce
|
||
│
|
||
▼
|
||
┌───────────────────────────────────────────────┐
|
||
│ Main loop (every 30s): │
|
||
│ 1. ESP32 requests/receives status via UART │
|
||
│ 2. ESP8266 polls GoPro HTTP (10.5.5.1) │
|
||
│ 3. ESP8266 returns parsed status over UART │
|
||
│ 4. ESP32 publishes MQTT status (retained) │
|
||
│ 5. Every 60s: ESP32 publishes heartbeat │
|
||
└───────────────────────────────────────────────┘
|
||
│
|
||
├── On MQTT disconnect → reconnect with 1s/2s/4s/8s/16s/30s backoff
|
||
├── On ESP8266/GoPro unreachable → publish status with online: false
|
||
├── On Wi-Fi loss → buffer status locally, replay on reconnect (CUB-230)
|
||
│
|
||
▼
|
||
ESP32 shutdown / watchdog reboot
|
||
```
|
||
|
||
## Data Flow: Start Recording Example
|
||
|
||
```
|
||
1. User clicks "Start" on dashboard
|
||
2. Browser → HTTP POST /api/v1/cameras/cam-001/start → Go backend
|
||
3. Go backend → MQTT publish remoterig/cameras/cam-001/command {command: "start_recording"}
|
||
4. ESP32 receives command and forwards it to ESP8266 over UART
|
||
5. ESP8266 sends HTTP GET to 10.5.5.1/bacpac/SH?t=<password>&p=%01
|
||
6. GoPro starts recording
|
||
7. Next 30s poll: ESP8266 reports status over UART; ESP32 publishes status with recording: true
|
||
8. Go backend receives status, updates SQLite, fans out via SSE
|
||
9. Dashboard updates with pulsing REC indicator
|
||
```
|
||
|
||
## Offline Buffering (future — CUB-230)
|
||
|
||
When ESP32 loses connection to travel router:
|
||
|
||
1. **Buffer:** Store status snapshots in SPIFFS (LittleFS), max 100 entries (~6KB)
|
||
2. **Eviction:** FIFO — oldest dropped when buffer full
|
||
3. **Replay:** On MQTT reconnect, publish buffered messages in chronological order with original timestamps
|
||
4. **Dedup:** Hub ignores duplicates via `(camera_id, recorded_at)` unique constraint in status_logs
|
||
|
||
## Backward Compatibility
|
||
|
||
- **Adding fields:** Safe — unknown fields ignored by both sides
|
||
- **Removing fields:** Mark as optional first, remove in next major version
|
||
- **Changing field types:** New topic path (e.g., `status/v2`) or new field name
|
||
- **New topics:** Add freely — old clients ignore unknown topics
|
||
|
||
## Open Questions
|
||
|
||
1. **NTP/time sync:** How do ESP32s get accurate time without internet? Options: (a) Pi runs NTP server, (b) ESP32 queries Pi's HTTP /api/v1/time endpoint, (c) GPS module. **Recommendation:** Pi runs NTPd, ESP32s use SNTP from `10.60.1.56`.
|
||
2. **Camera naming:** Should `friendly_name` be configurable from dashboard after auto-registration? **Recommendation:** Yes — allow rename via UI, stored in cameras table.
|
||
3. **Firmware OTA:** Should ESP32 firmware updates be possible over this network? **Recommendation:** Yes but out of scope for MVP.
|