2026-05-21 21:08:38 +00:00
# MQTT Message Format Contract — RemoteRig
> **Version:** 1.0.0 | **Status:** Draft | **Blocks:** CUB-232 (MQTT subscriber), CUB-174 (ESP32 firmware)
> **Last updated:** 2026-05-21
## Network Architecture
```
┌──────────────────────────────────┐
2026-06-05 07:47:57 -04:00
│ Travel Router (192.168.8.1) │
2026-05-21 21:08:38 +00:00
│ DHCP: .100-.200 │
└──────┬──────────┬──────────┬──────┘
│ │ │
┌───────────────┘ │ └───────────────┐
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ ESP32 #1 │ │ ESP32 #2 │ │ Pi Zero 2 W │
2026-06-05 07:47:57 -04:00
│ 192.168.8.101 │ │ 192.168.8.102 │ │ 192.168.8.56 │
2026-05-22 17:08:11 -04:00
│ STA→Router │ │ STA→Router │ │ Mosquitto │
│ MQTT relay │ │ MQTT relay │ │ Go backend │
└──────┬───────┘ └──────┬───────┘ │ React UI │
│ UART │ UART └──────────────┘
▼ ▼
┌──────────────┐ ┌──────────────┐
│ ESP8266 #1 │ │ ESP8266 #2 │
│ STA→GoPro AP │ │ STA→GoPro AP │
│ HTTP→10.5.5.1│ │ HTTP→10.5.5.1│
└──────┬───────┘ └──────┬───────┘
▼ ▼
┌──────────────┐ ┌──────────────┐
│ GoPro Hero 3 │ │ GoPro Hero 3 │
└──────────────┘ └──────────────┘
2026-05-21 21:08:38 +00:00
```
2026-06-05 07:47:57 -04:00
- **Travel router:** Self-contained, no internet. Gateway `192.168.8.1` . DHCP pool: `192.168.8.100-200`
- **Pi Zero 2 W:** Static IP `192.168.8.56` . Runs Mosquitto (port 1883), Go backend (port 8080), serves React UI
2026-05-22 17:08:11 -04:00
- **ESP32s:** DHCP from router. Each stays on the travel-router LAN, relays MQTT to/from its paired ESP8266 over UART
2026-06-05 07:47:57 -04:00
- **User device:** Connects to router, opens `http://192.168.8.56:8080` for dashboard
2026-05-21 21:08:38 +00:00
## MQTT Broker
2026-06-05 07:47:57 -04:00
- **Host:** `192.168.8.56` (Pi Zero 2 W)
2026-05-21 21:08:38 +00:00
- **Port:** `1883` (default MQTT, no TLS — closed network)
- **Auth:** None (closed network, no external access)
- **Client ID format:** `remoterig-<esp32_mac_last6>` (e.g., `remoterig-a1b2c3` )
- **QoS:** 1 (at least once) for status/heartbeat. 2 (exactly once) for commands.
- **Retain:** Status messages use `retain: true` so new subscribers get latest state immediately
## Topic Hierarchy
```
remoterig/
├── cameras/
│ └── <camera_id>/
│ ├── status ← ESP32 publishes (retained, QoS 1)
│ ├── heartbeat ← ESP32 publishes (QoS 1, not retained)
│ ├── command → Hub publishes (QoS 2)
│ └── announce ← ESP32 publishes on first boot (QoS 2, retained)
└── hub/
└── status ← Hub publishes (retained, QoS 1)
```
### Topic: `remoterig/cameras/<camera_id>/status`
**Direction: ** ESP32 → Hub
**QoS: ** 1 | **Retain: ** true | **Interval: ** 30 seconds
2026-05-22 17:08:11 -04:00
Published by the ESP32 every 30s using the latest GoPro status received from the paired ESP8266 over UART.
2026-05-21 21:08:38 +00:00
``` json
{
"camera_id" : "cam-001" ,
"timestamp" : "2026-05-21T18:30:00Z" ,
"battery_pct" : 85 ,
"battery_raw" : 217 ,
"video_remaining_sec" : 3420 ,
"recording" : true ,
"mode" : "video" ,
"resolution" : "1080p" ,
"fps" : 30 ,
"online" : true ,
"rssi" : -52 ,
"uptime_sec" : 1247
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `camera_id` | string | ✅ | Unique camera identifier (set during registration) |
| `timestamp` | ISO 8601 | ✅ | ESP32 clock time when status was read |
| `battery_pct` | int 0-100 | ✅ | Calibrated battery percentage (null if uncalibrated → omit) |
| `battery_raw` | int 0-255 | — | Raw byte from GoPro status offset 57 |
| `video_remaining_sec` | int | — | Estimated remaining recording seconds (null if unavailable) |
| `recording` | bool | ✅ | Whether camera is currently recording |
| `mode` | string | — | Current mode (e.g., "video", "photo", "burst") |
| `resolution` | string | — | Current resolution string |
| `fps` | int | — | Current frames per second |
| `online` | bool | ✅ | ESP32 can reach the GoPro (false if GoPro AP unreachable) |
| `rssi` | int | — | Wi-Fi RSSI to GoPro AP (dBm, negative) |
| `uptime_sec` | int | — | ESP32 uptime in seconds |
### Topic: `remoterig/cameras/<camera_id>/heartbeat`
**Direction: ** ESP32 → Hub
**QoS: ** 1 | **Retain: ** false | **Interval: ** 60 seconds
Lightweight keepalive so the hub can detect dead ESP32s.
``` json
{
"camera_id" : "cam-001" ,
"timestamp" : "2026-05-21T18:31:00Z" ,
"uptime_sec" : 1307 ,
"free_heap" : 28672
}
```
| Field | Type | Description |
|-------|------|-------------|
| `camera_id` | string | Camera identifier |
| `timestamp` | ISO 8601 | Current ESP32 time |
| `uptime_sec` | int | ESP32 uptime |
| `free_heap` | int | Free heap in bytes (diagnostic) |
**Hub behavior: ** If no heartbeat for 120 seconds, mark camera as offline (`online: false` in SSE broadcast).
### Topic: `remoterig/cameras/<camera_id>/command`
**Direction: ** Hub → ESP32
**QoS: ** 2 | **Retain: ** false
Commands sent from the dashboard to individual cameras.
``` json
{
"command" : "start_recording" ,
"request_id" : "req-abc123" ,
"timestamp" : "2026-05-21T18:32:00Z"
}
```
**Supported commands: **
| `command` | Description | Response topic |
|-----------|-------------|----------------|
| `start_recording` | Start GoPro recording | status (updated on next poll) |
| `stop_recording` | Stop GoPro recording | status (updated on next poll) |
| `reboot` | Reboot the ESP32 | — (ESP32 reconnects after boot) |
2026-05-22 17:08:11 -04:00
**ESP32 / ESP8266 behavior: **
- ESP32 receives the MQTT command and forwards it over UART to the paired ESP8266
- ESP8266 executes the corresponding HTTP command against the GoPro AP
2026-05-21 21:08:38 +00:00
- Next status publish will reflect the new state
- If command fails (GoPro unreachable), publish status with `online: false`
### Topic: `remoterig/cameras/<camera_id>/announce`
**Direction: ** ESP32 → Hub
**QoS: ** 2 | **Retain: ** true
Published once on ESP32 first boot (or factory reset). Used for auto-registration.
``` json
{
"mac_address" : "AA:BB:CC:DD:EE:FF" ,
"firmware_version" : "0.1.0" ,
"capabilities" : [ "start_stop" , "status" ] ,
"friendly_name" : "ESP32-AA-BB-CC"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `mac_address` | string | ESP32 Wi-Fi MAC address |
| `firmware_version` | string | ESP32 firmware semver |
| `capabilities` | string[] | Supported features |
| `friendly_name` | string | Default human-readable name |
2026-06-05 12:14:00 -04:00
**Camera IDs (self-assigned — "Option B"): ** the node uses a stable
device-derived id (`rig-<last3 MAC bytes>` , e.g. `rig-86d978` ) as its
`camera_id` from first boot, and uses it for all topics
(`announce` /`status` /`heartbeat` /`command` ). There is no hub-assigned
`cam-NNN` and no `registered` reply handshake.
**Hub behavior on announce: **
2026-05-21 21:08:38 +00:00
1. Check if MAC already registered → if yes, update `friendly_name` and log
2026-06-05 12:14:00 -04:00
2. If new MAC → insert the camera using the node's self-assigned `camera_id`
3. Broadcast via SSE that a new camera appeared
> Note: nodes have no real-time clock, so `timestamp` may be absent; the hub
> stamps received-time server-side.
2026-05-21 21:08:38 +00:00
### Topic: `remoterig/hub/status`
**Direction: ** Hub → All
**QoS: ** 1 | **Retain: ** true | **Interval: ** 30 seconds
Hub health status broadcast.
``` json
{
"version" : "0.2.0" ,
"uptime_sec" : 86400 ,
"cameras_online" : 3 ,
"cameras_total" : 4 ,
"mqtt_connected" : true ,
"db_size_bytes" : 1048576
}
```
## Message Validation Rules
### Hub-side (incoming from ESP32)
1. **Required fields: ** `camera_id` and `timestamp` must be present in all messages
2. **Timestamp sanity: ** Reject if timestamp is > 5 minutes in the future or > 24 hours in the past
3. **Duplicate detection: ** Status messages with same `(camera_id, timestamp)` are ignored (idempotent)
4. **Schema validation: ** Unknown fields are ignored (forward-compatible), missing required fields → log warning + reject
5. **battery_pct bounds: ** If present, must be 0–
### ESP32-side (incoming from hub)
1. **Acknowledge commands: ** After processing a command, the next status publish reflects the new state
2. **Unknown commands: ** Log and ignore
3. **Malformed JSON: ** Log error, ignore message
## Session Lifecycle
```
ESP32 boots
│
├── Connects to travel router Wi-Fi
2026-06-05 07:47:57 -04:00
├── Connects to MQTT broker (192.168.8.56:1883)
2026-05-21 21:08:38 +00:00
├── Publishes announce (retained) on cameras/<id>/announce
│
▼
2026-05-22 17:08:11 -04:00
┌───────────────────────────────────────────────┐
│ Main loop (every 30s): │
│ 1. ESP32 requests/receives status via UART │
│ 2. ESP8266 polls GoPro HTTP (10.5.5.1) │
│ 3. ESP8266 returns parsed status over UART │
│ 4. ESP32 publishes MQTT status (retained) │
│ 5. Every 60s: ESP32 publishes heartbeat │
└───────────────────────────────────────────────┘
2026-05-21 21:08:38 +00:00
│
├── On MQTT disconnect → reconnect with 1s/2s/4s/8s/16s/30s backoff
2026-05-22 17:08:11 -04:00
├── On ESP8266/GoPro unreachable → publish status with online: false
2026-05-21 21:08:38 +00:00
├── On Wi-Fi loss → buffer status locally, replay on reconnect (CUB-230)
│
▼
ESP32 shutdown / watchdog reboot
```
## Data Flow: Start Recording Example
```
1. User clicks "Start" on dashboard
2. Browser → HTTP POST /api/v1/cameras/cam-001/start → Go backend
3. Go backend → MQTT publish remoterig/cameras/cam-001/command {command: "start_recording"}
2026-05-22 17:08:11 -04:00
4. ESP32 receives command and forwards it to ESP8266 over UART
5. ESP8266 sends HTTP GET to 10.5.5.1/bacpac/SH?t=<password>&p=%01
6. GoPro starts recording
7. Next 30s poll: ESP8266 reports status over UART; ESP32 publishes status with recording: true
8. Go backend receives status, updates SQLite, fans out via SSE
9. Dashboard updates with pulsing REC indicator
2026-05-21 21:08:38 +00:00
```
## Offline Buffering (future — CUB-230)
When ESP32 loses connection to travel router:
1. **Buffer: ** Store status snapshots in SPIFFS (LittleFS), max 100 entries (~6KB)
2. **Eviction: ** FIFO — oldest dropped when buffer full
3. **Replay: ** On MQTT reconnect, publish buffered messages in chronological order with original timestamps
4. **Dedup: ** Hub ignores duplicates via `(camera_id, recorded_at)` unique constraint in status_logs
## Backward Compatibility
- **Adding fields:** Safe — unknown fields ignored by both sides
- **Removing fields:** Mark as optional first, remove in next major version
- **Changing field types:** New topic path (e.g., `status/v2` ) or new field name
- **New topics:** Add freely — old clients ignore unknown topics
## Open Questions
2026-06-05 07:47:57 -04:00
1. **NTP/time sync: ** How do ESP32s get accurate time without internet? Options: (a) Pi runs NTP server, (b) ESP32 queries Pi's HTTP /api/v1/time endpoint, (c) GPS module. **Recommendation: ** Pi runs NTPd, ESP32s use SNTP from `192.168.8.56` .
2026-05-21 21:08:38 +00:00
2. **Camera naming: ** Should `friendly_name` be configurable from dashboard after auto-registration? **Recommendation: ** Yes — allow rename via UI, stored in cameras table.
3. **Firmware OTA: ** Should ESP32 firmware updates be possible over this network? **Recommendation: ** Yes but out of scope for MVP.
