diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000..dbed0ae --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,228 @@ +# RemoteRig — Project Working Context + +> **Purpose of this file:** a living, high-signal context + decision log for the +> RemoteRig project. It's the primary onboarding doc for humans and for any LLM +> working on the repo. Keep it updated as decisions are made. +> **Last updated:** 2026-06-05. +> +> Deeper references: `docs/CONTEXT.md` (system architecture detail), +> `docs/MQTT_CONTRACT.md` (MQTT topics/payloads), `docs/design/` (design notes), +> `hardware/README.md` (case/wiring/BOM), `README.md` (hub build/deploy). + +--- + +## 1. What this is + +RemoteRig is a multi-camera **GoPro Hero 3 monitoring & control** system for field +recording (large concerts in auditoriums, high-school marching band at stadiums). +Founder: **Joshua / CubeCraft Creations**. + +Scope: **monitor status (battery, recording, link) and start/stop multiple GoPros** +over a closed, self-contained travel-router network. **No video flows through the +hub** — footage records locally to each GoPro's SD card. This keeps the hub a +lightweight control plane. + +## 2. Architecture + +``` +GoPro Hero 3 ──Wi-Fi(10.5.5.1)── ESP-01S ──UART(JSON)── XIAO ESP32-C6 ──Wi-Fi/MQTT── Pi hub ── Dashboard + (per camera) (GoPro bridge) (MQTT bridge + OLED/LED) (Mosquitto+Go+SQLite) +``` + +**Camera node** (one per GoPro), two boards: +- **XIAO ESP32-C6** — main MCU / MQTT bridge. Joins the RemoteRig Wi-Fi, talks MQTT + to the hub, drives the OLED + RGB status LED, reads camera status from the ESP-01S + over UART. Powered from the 5V rail. +- **ESP-01S (ESP8266)** — GoPro Wi-Fi bridge. Joins the GoPro's AP (`10.5.5.1`), + relays status/commands to the XIAO over UART. Powered from its **own 3.3V buck** + (not the XIAO 3V3 pin — Wi-Fi TX spikes ~300 mA). + +**Hub** — Raspberry Pi Zero 2 W (hostname `remote-rig-hub`, user `overseer`): +- **Mosquitto** MQTT broker (`:1883`, anonymous, listens on `0.0.0.0`). +- **Go controller** (`remoterig`, systemd service) — MQTT subscriber → SQLite, + REST API + SSE, serves the embedded React dashboard on `:8080`. +- **SQLite** (`/opt/remoterig/remoterig.db`). +- Decided to **stay on the Zero 2 W** (workload is tiny; a Pi 5 only makes sense if + video preview/ingest is ever added — not planned). + +## 3. Network + +- **RemoteRig travel router**, subnet **`192.168.8.0/24`**, gateway `192.168.8.1`, + has a WAN uplink (internet available — used for the Pi to pull builds). +- **Hub static IP: `192.168.8.56`** (`:1883` MQTT, `:8080` dashboard/API). +- Cameras get DHCP `192.168.8.x`. +- The GoPro AP network (`10.5.5.1`) is separate and only the ESP-01S touches it. +- **History:** the project was originally designed around `10.60.1.0/24`; it was + re-addressed to `192.168.8.0/24` to match the actual travel router (commit + `b0062f1`). Wi-Fi SSID `RemoteRig` (creds in `firmware/data/config.json`). + +## 4. Repository & workflow + +- **Gitea:** `ssh://sc-gitea@code.cubecraftcreations.com:2288/CubeCraft-Creations/remote-rig` + (web/API on `https://code.cubecraftcreations.com`, private repo). +- **Default/integration branch: `dev`** (work lands here; merges from `main`). +- Layout: `firmware/` (PlatformIO), `cmd/`,`internal/`,`pkg/` (Go hub), + `src/` (React/Vite/TS dashboard), `scripts/` (Pi setup/deploy), `.gitea/workflows/` + (CI), `hardware/` (CAD/wiring), `docs/`. + +## 5. Tech stack + +| Area | Choice | +|------|--------| +| Camera firmware | PlatformIO / Arduino | +| C6 env | `seeed_xiao_esp32c6` — **pioarduino** platform fork, **LittleFS**, U8g2 | +| ESP-01S env | `esp8266-camera` — board `esp01_1m`, flash `dout` | +| Hub | **Go 1.25** (single static binary, `//go:embed` frontend) | +| Dashboard | React + Vite + TypeScript + Tailwind (Vitest) | +| Storage | **SQLite** (not Postgres) | +| Broker | **Mosquitto** | +| CI/CD | **Gitea Actions** (pull-based deploy) | + +## 6. Camera node hardware (XIAO ESP32-C6 pin map) + +| Pin | Use | +|-----|-----| +| 5V/VIN | rocker → 5V rail | +| D4/SDA, D5/SCL | 1.3" **SH1106** OLED (I2C @ `0x3C`) | +| D0 / D1 / D2 | RGB STAT LED R/G/B (220Ω each), **common-anode** | +| D6 (TX) / D7 (RX) | UART (`Serial1`) to ESP-01S (crossed) | +| D8 / D10 | **reserved** for ESP-01S UART-OTA control (RST / GPIO0) — not driven yet | +| 5V rail (330Ω) | PWR LED (not an MCU pin) | + +Canonical wiring: Notion "XIAO ESP32-C6 Pin-to-Pin Wiring Diagram" + `hardware/README.md`. + +## 7. Firmware behavior + +**XIAO ESP32-C6 (`firmware/src/esp32-mqtt-bridge.cpp`)** — fw `0.4.0`: +- Loads config from LittleFS `/config.json` (Wi-Fi, broker, camera_id, battery cal). +- **Self-assigned camera_id** = device id `rig-` (e.g. `rig-86d978`) — see + decision #7. Subscribes `remoterig/cameras//command`, announces on + `remoterig/cameras//announce`, publishes `.../status`. +- OLED status panel (CAM id / REC + session timer / BAT / LINK / CAM reachability). +- RGB STAT LED health colors: blue=boot, red=offline, magenta=Wi-Fi-no-hub, + yellow=hub-no-camera, green=healthy. +- Battery calibration: two-point linear (raw offset-57 → %), persisted; `battery_pct` + emitted only when calibrated. +- No-reflash config: `set_camera_config` (MQTT) → forwarded to ESP-01S as `set_config`. + +**ESP-01S (`firmware/src/esp8266-camera-bridge.cpp`)**: +- Joins GoPro AP, polls status, relays JSON over UART; `set_config` persists to LittleFS. +- No status LED (GPIO1 is the UART TX). +- ⚠️ **Known bug:** `fetchStatus()` GETs the **shutter** endpoint + (`/bacpac/SH?...&p=%01`) instead of a real status read — would *start recording* + each poll. Needs the GoPro Hero 3 protocol corrected + validated against a real + camera (also verify password/SSID). **Do not point at a live GoPro until fixed.** + +**Provisioning:** `firmware/data/config.json` is flashed to the C6's LittleFS via +`pio run -e seeed_xiao_esp32c6 -t uploadfs`. Per maintainer decision the real Wi-Fi +password lives in this tracked file (private repo, low-sensitivity closed network). + +## 8. Hub & CI/CD (pull-based deploy) + +**Flow:** `push to dev` → Gitea Actions `build-dev.yaml` builds the React frontend +(into `cmd/server/src/dist`, embedded) + cross-compiles the **arm64** Go binary → +publishes a rolling **`dev-latest`** release (binary + `sha256` + `version.txt`) via +a Node script (`.gitea/scripts/publish-release.mjs`). The Pi's +`remoterig-update.timer` runs `scripts/pi-update.sh` every ~5 min → compares +`version.txt`, downloads + checksum-verifies, **atomically replaces** the binary, +restarts, health-checks (rolls back on failure). + +**Why pull, not push:** the Pi is on a closed travel-router LAN the CI runner can't +reach; the Pi pulls instead. + +- `ci.yaml` — frontend quality gates (lint/typecheck/test/build), single job. +- First-time Pi setup: `sudo bash scripts/setup-pi.sh --config config.yaml` (installs + Mosquitto, the service, the updater timer, static IP). +- Pi files: `/opt/remoterig/{remoterig, config.yaml, update.env, VERSION, deploy.sh, pi-update.sh}`. +- Future ESP-01S firmware OTA: `docs/design/esp01s-uart-ota.md` ("XIAO as flasher"). + +### Gitea Actions runner notes (important) +- Runner `remote-rig-runner`, label **`go-react`** (Dockerized act_runner). Workflows + must use `runs-on: go-react`. +- The `go-react` image has **Node but not Go** → use `setup-go` (its static binary + runs); get Node from the image (**don't** use `setup-node` — its dynamically-linked + Node won't execute here, "cannot execute: required file not found"). +- Gitea doesn't support `actions/upload-artifact@v4`. No `curl`/`jq`/`sudo` on the + runner — the release publish is done in Node. +- The runner's network to github.com/Gitea is flaky (ECONNRESET) → keep few action + clones; `publish-release.mjs` retries. +- Rolling release tag is **`dev-latest`**, NOT `dev` (a `dev` tag collides with the + `dev` branch → ambiguous refs). +- Inspect CI from a dev machine with the **`tea` CLI**: `tea actions runs list|view|logs`, + `tea release list` (note "completed" ≠ success — check Conclusion). + +## 9. Key decisions & gotchas (log) + +1. **MCU:** ESP32-C3 Super Mini → **XIAO ESP32-C6**; C6 needs the **pioarduino** + platform fork. USB-CDC-on-boot for `Serial` over native USB. +2. **Mac build toolchain:** use `~/.platformio/penv/bin/pio` (Python 3.11), **not** + the pyenv 3.9.21 shim (too old for pioarduino). +3. **C6 filesystem = LittleFS** (pioarduino `uploadfs` builds LittleFS, not SPIFFS) — + the firmware reads `/config.json` (data file must be named `config.json`). +4. **Network re-addressed** `10.60.1.0/24` → `192.168.8.0/24`. +5. **Wi-Fi password kept in git** (`firmware/data/config.json`) — maintainer decision + (private repo, low-sensitivity, closed net). +6. **RGB LED is common-anode** (`RGB_COMMON_ANODE 1`); OLED is **SH1106** @ `0x3C`. +7. **Camera registration = "Option B" self-assigned IDs:** node uses `rig-` as a + stable `camera_id` from first boot; the hub registers under that id. No `cam-NNN` + assignment, no `registered`-reply handshake. (`docs/MQTT_CONTRACT.md` updated.) +8. **Hub tolerates clockless status timestamps** — nodes have no RTC; firmware omits + `timestamp`, the hub stamps server-side (it used to reject the status). +9. **ESP-01S updates:** settings change live via `set_config` (no reflash); full + firmware OTA is the future XIAO-as-flasher path (`docs/design/esp01s-uart-ota.md`). +10. **Pull-based deploy** via rolling `dev-latest` release + atomic binary replace + + network retries. +11. **Pi systemd service user = `overseer`** (this Pi has no `pi` user); `setup-pi.sh` + now defaults the service user to `$SUDO_USER`. +12. **Hub embeds the frontend** via `//go:embed all:src/dist`; Vite builds into + `cmd/server/src/dist` (a committed `index.html` placeholder keeps the embed valid). + +## 10. Conventions + +- Production hub/controller in **Go**; Python fine for diagnostics/experiments/migrations. +- **SQLite**, not Postgres. **Timezone: US Eastern.** +- Work on `dev`; commit messages end with a `Co-Authored-By` trailer. +- Canonical design docs live in **Notion** (Remote Rig parent page) and the repo + `docs/`; CAD in Seafile; code/build in Gitea. + +## 11. Current status & open items (2026-06-05) + +**Working / proven on hardware:** +- Hub up on the Pi (Mosquitto + `remoterig` + SQLite), dashboard on `:8080`. +- Full CI/CD loop proven: commit → CI build → `dev-latest` → Pi self-update + (checksum, atomic replace, health-check) → service active. +- C6 (fw `0.4.0`) joins RemoteRig, connects to the broker, announces as `rig-86d978`, + publishes status. ESP-01S UART link alive. + +**In progress / unresolved:** +- **Verifying the hub registered `rig-86d978`** — needs confirmation that the hub is on + build `18db26c2` (Option B) and that the camera is in the DB / dashboard + (`GET /api/v1/cameras`, header `X-API-Key: changeme`). Status currently shows + `online:false` because no GoPro is attached on the bench (expected). +- **GoPro Hero 3 protocol fix** (ESP-01S `fetchStatus` shutter bug) — required before + real battery/recording data; needs a real Hero 3. +- **Rotate the Gitea runner registration token** (was exposed in a setup paste). +- Gitea repo **default-branch HEAD** points at a nonexistent ref — set default branch. +- Optional: clear the stale **retained** MQTT message at + `remoterig/cameras/announce-rig-86d978` (from old firmware). + +## 12. Handy commands + +```bash +# Build/flash C6 firmware (Mac): +~/.platformio/penv/bin/pio run -d firmware -e seeed_xiao_esp32c6 -t upload +~/.platformio/penv/bin/pio run -d firmware -e seeed_xiao_esp32c6 -t uploadfs # provision /config.json +# ESP-01S: needs a 3.3V USB-UART adapter with GPIO0->GND for flash mode. + +# Inspect Gitea CI (Mac, tea CLI logged in as 'overseer'): +tea actions runs list --repo CubeCraft-Creations/remote-rig +tea actions runs view --repo … ; tea actions runs logs --repo … +tea release list --repo CubeCraft-Creations/remote-rig + +# On the Pi: +sudo systemctl start remoterig-update.service # force a pull/deploy +cat /opt/remoterig/VERSION ; systemctl is-active remoterig +journalctl -u remoterig -n 40 --no-pager +mosquitto_sub -h localhost -t 'remoterig/#' -v +curl -s -H "X-API-Key: changeme" http://localhost:8080/api/v1/cameras +```