generated from CubeCraft-Creations/Tracehound
docs: add MQTT message format contract with closed-network architecture
Defines topic hierarchy, payload schemas, QoS levels, heartbeat protocol, camera auto-discovery via announce topic, offline buffering strategy, and command/response flow for start/stop. Architecture: travel router subnet (192.168.4.x), Pi Zero 2 W runs Mosquitto + Go backend, ESP32s dual-STA to GoPro AP + travel router. No internet dependency. Closes CUB-238.
This commit is contained in:
@@ -0,0 +1,172 @@
|
|||||||
|
# RemoteRig Gitea CI/CD Implementation Plan
|
||||||
|
|
||||||
|
> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.
|
||||||
|
|
||||||
|
**Goal:** Set up Gitea Actions CI/CD pipeline with build → test → deploy stages for the RemoteRig React dashboard.
|
||||||
|
|
||||||
|
**Architecture:** Gitea Actions (GitHub Actions compatible) running in `.gitea/workflows/`. Single workflow file with three jobs: lint+typecheck, test, build, and a manual deploy step. The app is a Vite SPA that builds to `dist/` — deploy serves those static files.
|
||||||
|
|
||||||
|
**Tech Stack:** Gitea Actions, Node 22, Vite, Vitest, TypeScript, Tailwind
|
||||||
|
|
||||||
|
**Success criteria:**
|
||||||
|
- Build step completes successfully (`tsc -b && vite build`)
|
||||||
|
- All unit tests pass (`vitest run`)
|
||||||
|
- Deploy step exists (manual trigger for now)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Verify Gitea Actions runner availability
|
||||||
|
|
||||||
|
**Objective:** Confirm the Gitea instance has at least one Actions runner registered.
|
||||||
|
|
||||||
|
**Files:** None (read-only check)
|
||||||
|
|
||||||
|
**Step 1:** Check Gitea Actions runners
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -s "https://code.cubecraftcreations.com/api/v1/admin/runners" \
|
||||||
|
-H "Authorization: bearer ${HERMES_GITEA_TOKEN}" | jq '.'
|
||||||
|
```
|
||||||
|
|
||||||
|
If this returns a list with runners, we're good. If 404 or empty, we need to register a runner.
|
||||||
|
|
||||||
|
**Step 2:** Check org-level runners
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -s "https://code.cubecraftcreations.com/api/v1/orgs/CubeCraft-Creations/actions/runners" \
|
||||||
|
-H "Authorization: bearer ${HERMES_GITEA_TOKEN}" | jq '.'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Expected output:** At least one runner with `"is_online": true` at either admin or org level.
|
||||||
|
|
||||||
|
**Verification:** Confirm runners exist before proceeding.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: Create Gitea Actions CI/CD workflow
|
||||||
|
|
||||||
|
**Objective:** Create `.gitea/workflows/ci.yaml` with jobs for lint, typecheck, test, build, and deploy.
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `.gitea/workflows/ci.yaml`
|
||||||
|
|
||||||
|
**Workflow structure:**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: CI/CD
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [dev, main]
|
||||||
|
pull_request:
|
||||||
|
branches: [dev, main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
# ── Quality Gates ──────────────────────────────────────────
|
||||||
|
lint-and-typecheck:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: 22
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci
|
||||||
|
- run: npm run lint
|
||||||
|
- run: npx tsc --noEmit
|
||||||
|
|
||||||
|
# ── Unit Tests ─────────────────────────────────────────────
|
||||||
|
test:
|
||||||
|
needs: lint-and-typecheck
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: 22
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci
|
||||||
|
- run: npm test
|
||||||
|
|
||||||
|
# ── Build ──────────────────────────────────────────────────
|
||||||
|
build:
|
||||||
|
needs: test
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: 22
|
||||||
|
cache: npm
|
||||||
|
- run: npm ci
|
||||||
|
- run: npm run build
|
||||||
|
- name: Upload build artifacts
|
||||||
|
uses: actions/upload-artifact@v4
|
||||||
|
with:
|
||||||
|
name: dist
|
||||||
|
path: dist/
|
||||||
|
|
||||||
|
# ── Deploy ─────────────────────────────────────────────────
|
||||||
|
deploy:
|
||||||
|
needs: build
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||||
|
environment: production
|
||||||
|
steps:
|
||||||
|
- uses: actions/download-artifact@v4
|
||||||
|
with:
|
||||||
|
name: dist
|
||||||
|
path: dist/
|
||||||
|
- name: Deploy static files
|
||||||
|
run: |
|
||||||
|
echo "Deploying to production..."
|
||||||
|
# Replace with actual deploy command (rsync, scp, S3, etc.)
|
||||||
|
echo "Deploy target: /var/www/remote-rig/"
|
||||||
|
echo "Placeholder — configure deploy target before merging to main"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 1:** Create the directory and file
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p /mnt/ai-storage/projects/remote-rig/.gitea/workflows
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 2:** Write the workflow file with the content above
|
||||||
|
|
||||||
|
**Step 3:** Verify YAML syntax
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -c "import yaml; yaml.safe_load(open('/mnt/ai-storage/projects/remote-rig/.gitea/workflows/ci.yaml'))" && echo "YAML: OK"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 4:** Commit
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/ai-storage/projects/remote-rig
|
||||||
|
git add .gitea/
|
||||||
|
git commit -m "ci: add Gitea Actions pipeline (lint, typecheck, test, build, deploy)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: Verify workflow triggers on push
|
||||||
|
|
||||||
|
**Objective:** Push the workflow and verify it appears in Gitea Actions.
|
||||||
|
|
||||||
|
**Step 1:** Push the branch
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /mnt/ai-storage/projects/remote-rig
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 2:** Check if the workflow registered
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -s "https://code.cubecraftcreations.com/api/v1/repos/CubeCraft-Creations/remote-rig/actions/workflows" \
|
||||||
|
-H "Authorization: bearer ${HERMES_GITEA_TOKEN}" | jq '.workflows[] | {name, state, path}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Expected:** The CI/CD workflow appears with state "active".
|
||||||
|
|
||||||
|
**Verification:** Workflow is listed and active on the repo.
|
||||||
@@ -0,0 +1,273 @@
|
|||||||
|
# MQTT Message Format Contract — RemoteRig
|
||||||
|
|
||||||
|
> **Version:** 1.0.0 | **Status:** Draft | **Blocks:** CUB-232 (MQTT subscriber), CUB-174 (ESP32 firmware)
|
||||||
|
> **Last updated:** 2026-05-21
|
||||||
|
|
||||||
|
## Network Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────────────────┐
|
||||||
|
│ Travel Router (192.168.4.1) │
|
||||||
|
│ DHCP: .100-.200 │
|
||||||
|
└──────┬──────────┬──────────┬──────┘
|
||||||
|
│ │ │
|
||||||
|
┌───────────────┘ │ └───────────────┐
|
||||||
|
▼ ▼ ▼
|
||||||
|
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||||
|
│ ESP32 #1 │ │ ESP32 #2 │ │ Pi Zero 2 W │
|
||||||
|
│ 192.168.4.101│ │ 192.168.4.102│ │ 192.168.4.10 │
|
||||||
|
│ │ │ │ │ │
|
||||||
|
│ STA→GoPro AP │ │ STA→GoPro AP │ │ Mosquitto │
|
||||||
|
│ STA→Router │ │ STA→Router │ │ Go backend │
|
||||||
|
└──────┬───────┘ └──────┬───────┘ │ React UI │
|
||||||
|
│ │ └──────────────┘
|
||||||
|
▼ ▼
|
||||||
|
┌──────────────┐ ┌──────────────┐
|
||||||
|
│ GoPro Hero 3 │ │ GoPro Hero 3 │
|
||||||
|
│ AP: 10.5.5.1 │ │ AP: 10.5.5.1 │
|
||||||
|
└──────────────┘ └──────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Travel router:** Self-contained, no internet. DHCP pool: `192.168.4.100-200`
|
||||||
|
- **Pi Zero 2 W:** Static IP `192.168.4.10`. Runs Mosquitto (port 1883), Go backend (port 8080), serves React UI
|
||||||
|
- **ESP32s:** DHCP from router. Each has dual STA: one to GoPro AP, one to router
|
||||||
|
- **User device:** Connects to router, opens `http://192.168.4.10:8080` for dashboard
|
||||||
|
|
||||||
|
## MQTT Broker
|
||||||
|
|
||||||
|
- **Host:** `192.168.4.10` (Pi Zero 2 W)
|
||||||
|
- **Port:** `1883` (default MQTT, no TLS — closed network)
|
||||||
|
- **Auth:** None (closed network, no external access)
|
||||||
|
- **Client ID format:** `remoterig-<esp32_mac_last6>` (e.g., `remoterig-a1b2c3`)
|
||||||
|
- **QoS:** 1 (at least once) for status/heartbeat. 2 (exactly once) for commands.
|
||||||
|
- **Retain:** Status messages use `retain: true` so new subscribers get latest state immediately
|
||||||
|
|
||||||
|
## Topic Hierarchy
|
||||||
|
|
||||||
|
```
|
||||||
|
remoterig/
|
||||||
|
├── cameras/
|
||||||
|
│ └── <camera_id>/
|
||||||
|
│ ├── status ← ESP32 publishes (retained, QoS 1)
|
||||||
|
│ ├── heartbeat ← ESP32 publishes (QoS 1, not retained)
|
||||||
|
│ ├── command → Hub publishes (QoS 2)
|
||||||
|
│ └── announce ← ESP32 publishes on first boot (QoS 2, retained)
|
||||||
|
└── hub/
|
||||||
|
└── status ← Hub publishes (retained, QoS 1)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Topic: `remoterig/cameras/<camera_id>/status`
|
||||||
|
|
||||||
|
**Direction:** ESP32 → Hub
|
||||||
|
**QoS:** 1 | **Retain:** true | **Interval:** 30 seconds
|
||||||
|
|
||||||
|
Published by the ESP32 every 30s with the latest GoPro status.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"camera_id": "cam-001",
|
||||||
|
"timestamp": "2026-05-21T18:30:00Z",
|
||||||
|
"battery_pct": 85,
|
||||||
|
"battery_raw": 217,
|
||||||
|
"video_remaining_sec": 3420,
|
||||||
|
"recording": true,
|
||||||
|
"mode": "video",
|
||||||
|
"resolution": "1080p",
|
||||||
|
"fps": 30,
|
||||||
|
"online": true,
|
||||||
|
"rssi": -52,
|
||||||
|
"uptime_sec": 1247
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Type | Required | Description |
|
||||||
|
|-------|------|----------|-------------|
|
||||||
|
| `camera_id` | string | ✅ | Unique camera identifier (set during registration) |
|
||||||
|
| `timestamp` | ISO 8601 | ✅ | ESP32 clock time when status was read |
|
||||||
|
| `battery_pct` | int 0-100 | ✅ | Calibrated battery percentage (null if uncalibrated → omit) |
|
||||||
|
| `battery_raw` | int 0-255 | — | Raw byte from GoPro status offset 57 |
|
||||||
|
| `video_remaining_sec` | int | — | Estimated remaining recording seconds (null if unavailable) |
|
||||||
|
| `recording` | bool | ✅ | Whether camera is currently recording |
|
||||||
|
| `mode` | string | — | Current mode (e.g., "video", "photo", "burst") |
|
||||||
|
| `resolution` | string | — | Current resolution string |
|
||||||
|
| `fps` | int | — | Current frames per second |
|
||||||
|
| `online` | bool | ✅ | ESP32 can reach the GoPro (false if GoPro AP unreachable) |
|
||||||
|
| `rssi` | int | — | Wi-Fi RSSI to GoPro AP (dBm, negative) |
|
||||||
|
| `uptime_sec` | int | — | ESP32 uptime in seconds |
|
||||||
|
|
||||||
|
### Topic: `remoterig/cameras/<camera_id>/heartbeat`
|
||||||
|
|
||||||
|
**Direction:** ESP32 → Hub
|
||||||
|
**QoS:** 1 | **Retain:** false | **Interval:** 60 seconds
|
||||||
|
|
||||||
|
Lightweight keepalive so the hub can detect dead ESP32s.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"camera_id": "cam-001",
|
||||||
|
"timestamp": "2026-05-21T18:31:00Z",
|
||||||
|
"uptime_sec": 1307,
|
||||||
|
"free_heap": 28672
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `camera_id` | string | Camera identifier |
|
||||||
|
| `timestamp` | ISO 8601 | Current ESP32 time |
|
||||||
|
| `uptime_sec` | int | ESP32 uptime |
|
||||||
|
| `free_heap` | int | Free heap in bytes (diagnostic) |
|
||||||
|
|
||||||
|
**Hub behavior:** If no heartbeat for 120 seconds, mark camera as offline (`online: false` in SSE broadcast).
|
||||||
|
|
||||||
|
### Topic: `remoterig/cameras/<camera_id>/command`
|
||||||
|
|
||||||
|
**Direction:** Hub → ESP32
|
||||||
|
**QoS:** 2 | **Retain:** false
|
||||||
|
|
||||||
|
Commands sent from the dashboard to individual cameras.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"command": "start_recording",
|
||||||
|
"request_id": "req-abc123",
|
||||||
|
"timestamp": "2026-05-21T18:32:00Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Supported commands:**
|
||||||
|
|
||||||
|
| `command` | Description | Response topic |
|
||||||
|
|-----------|-------------|----------------|
|
||||||
|
| `start_recording` | Start GoPro recording | status (updated on next poll) |
|
||||||
|
| `stop_recording` | Stop GoPro recording | status (updated on next poll) |
|
||||||
|
| `reboot` | Reboot the ESP32 | — (ESP32 reconnects after boot) |
|
||||||
|
|
||||||
|
**ESP32 behavior:**
|
||||||
|
- On receipt, execute command against GoPro
|
||||||
|
- Next status publish will reflect the new state
|
||||||
|
- If command fails (GoPro unreachable), publish status with `online: false`
|
||||||
|
|
||||||
|
### Topic: `remoterig/cameras/<camera_id>/announce`
|
||||||
|
|
||||||
|
**Direction:** ESP32 → Hub
|
||||||
|
**QoS:** 2 | **Retain:** true
|
||||||
|
|
||||||
|
Published once on ESP32 first boot (or factory reset). Used for auto-registration.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"mac_address": "AA:BB:CC:DD:EE:FF",
|
||||||
|
"firmware_version": "0.1.0",
|
||||||
|
"capabilities": ["start_stop", "status"],
|
||||||
|
"friendly_name": "ESP32-AA-BB-CC"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|-------|------|-------------|
|
||||||
|
| `mac_address` | string | ESP32 Wi-Fi MAC address |
|
||||||
|
| `firmware_version` | string | ESP32 firmware semver |
|
||||||
|
| `capabilities` | string[] | Supported features |
|
||||||
|
| `friendly_name` | string | Default human-readable name |
|
||||||
|
|
||||||
|
**Hub behavior on first announce:**
|
||||||
|
1. Check if MAC already registered → if yes, update `friendly_name` and log
|
||||||
|
2. If new MAC → create camera with auto-generated `camera_id = "cam-<NNN>"` (zero-padded sequential)
|
||||||
|
3. Respond by publishing: `remoterig/cameras/<camera_id>/command` with `command: "registered"` payload containing the assigned `camera_id`
|
||||||
|
4. Broadcast via SSE that a new camera appeared
|
||||||
|
|
||||||
|
### Topic: `remoterig/hub/status`
|
||||||
|
|
||||||
|
**Direction:** Hub → All
|
||||||
|
**QoS:** 1 | **Retain:** true | **Interval:** 30 seconds
|
||||||
|
|
||||||
|
Hub health status broadcast.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": "0.2.0",
|
||||||
|
"uptime_sec": 86400,
|
||||||
|
"cameras_online": 3,
|
||||||
|
"cameras_total": 4,
|
||||||
|
"mqtt_connected": true,
|
||||||
|
"db_size_bytes": 1048576
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Message Validation Rules
|
||||||
|
|
||||||
|
### Hub-side (incoming from ESP32)
|
||||||
|
|
||||||
|
1. **Required fields:** `camera_id` and `timestamp` must be present in all messages
|
||||||
|
2. **Timestamp sanity:** Reject if timestamp is > 5 minutes in the future or > 24 hours in the past
|
||||||
|
3. **Duplicate detection:** Status messages with same `(camera_id, timestamp)` are ignored (idempotent)
|
||||||
|
4. **Schema validation:** Unknown fields are ignored (forward-compatible), missing required fields → log warning + reject
|
||||||
|
5. **battery_pct bounds:** If present, must be 0–100. Out of range → clamp to [0,100] with warning
|
||||||
|
|
||||||
|
### ESP32-side (incoming from hub)
|
||||||
|
|
||||||
|
1. **Acknowledge commands:** After processing a command, the next status publish reflects the new state
|
||||||
|
2. **Unknown commands:** Log and ignore
|
||||||
|
3. **Malformed JSON:** Log error, ignore message
|
||||||
|
|
||||||
|
## Session Lifecycle
|
||||||
|
|
||||||
|
```
|
||||||
|
ESP32 boots
|
||||||
|
│
|
||||||
|
├── Connects to travel router Wi-Fi
|
||||||
|
├── Connects to MQTT broker (192.168.4.10:1883)
|
||||||
|
├── Publishes announce (retained) on cameras/<id>/announce
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────┐
|
||||||
|
│ Main loop (every 30s): │
|
||||||
|
│ 1. HTTP GET GoPro status (10.5.5.1) │
|
||||||
|
│ 2. Parse 60-byte status blob │
|
||||||
|
│ 3. Publish status (retained) │
|
||||||
|
│ 4. Every 60s: publish heartbeat │
|
||||||
|
└─────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
├── On MQTT disconnect → reconnect with 1s/2s/4s/8s/16s/30s backoff
|
||||||
|
├── On GoPro unreachable → publish status with online: false
|
||||||
|
├── On Wi-Fi loss → buffer status locally, replay on reconnect (CUB-230)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
ESP32 shutdown / watchdog reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data Flow: Start Recording Example
|
||||||
|
|
||||||
|
```
|
||||||
|
1. User clicks "Start" on dashboard
|
||||||
|
2. Browser → HTTP POST /api/v1/cameras/cam-001/start → Go backend
|
||||||
|
3. Go backend → MQTT publish remoterig/cameras/cam-001/command {command: "start_recording"}
|
||||||
|
4. ESP32 receives command, sends HTTP GET to 10.5.5.1/bacpac/SH?t=<password>&p=%01
|
||||||
|
5. GoPro starts recording
|
||||||
|
6. Next 30s poll: ESP32 publishes status with recording: true
|
||||||
|
7. Go backend receives status, updates SQLite, fans out via SSE
|
||||||
|
8. Dashboard updates with pulsing REC indicator
|
||||||
|
```
|
||||||
|
|
||||||
|
## Offline Buffering (future — CUB-230)
|
||||||
|
|
||||||
|
When ESP32 loses connection to travel router:
|
||||||
|
|
||||||
|
1. **Buffer:** Store status snapshots in SPIFFS (LittleFS), max 100 entries (~6KB)
|
||||||
|
2. **Eviction:** FIFO — oldest dropped when buffer full
|
||||||
|
3. **Replay:** On MQTT reconnect, publish buffered messages in chronological order with original timestamps
|
||||||
|
4. **Dedup:** Hub ignores duplicates via `(camera_id, recorded_at)` unique constraint in status_logs
|
||||||
|
|
||||||
|
## Backward Compatibility
|
||||||
|
|
||||||
|
- **Adding fields:** Safe — unknown fields ignored by both sides
|
||||||
|
- **Removing fields:** Mark as optional first, remove in next major version
|
||||||
|
- **Changing field types:** New topic path (e.g., `status/v2`) or new field name
|
||||||
|
- **New topics:** Add freely — old clients ignore unknown topics
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
1. **NTP/time sync:** How do ESP32s get accurate time without internet? Options: (a) Pi runs NTP server, (b) ESP32 queries Pi's HTTP /api/v1/time endpoint, (c) GPS module. **Recommendation:** Pi runs NTPd, ESP32s use SNTP from `192.168.4.10`.
|
||||||
|
2. **Camera naming:** Should `friendly_name` be configurable from dashboard after auto-registration? **Recommendation:** Yes — allow rename via UI, stored in cameras table.
|
||||||
|
3. **Firmware OTA:** Should ESP32 firmware updates be possible over this network? **Recommendation:** Yes but out of scope for MVP.
|
||||||
Reference in New Issue
Block a user