remote-rig/CONTEXT.md

# RemoteRig — Project Working Context

> **Purpose of this file:** a living, high-signal context + decision log for the
> RemoteRig project. It's the primary onboarding doc for humans and for any LLM
> working on the repo. Keep it updated as decisions are made.
> **Last updated:** 2026-06-05.
>
> Deeper references: `docs/CONTEXT.md` (system architecture detail),
> `docs/MQTT_CONTRACT.md` (MQTT topics/payloads), `docs/design/` (design notes),
> `hardware/README.md` (case/wiring/BOM), `README.md` (hub build/deploy).

---

## 1. What this is

RemoteRig is a multi-camera **GoPro Hero 3 monitoring & control** system for field
recording (large concerts in auditoriums, high-school marching band at stadiums).
Founder: **Joshua / CubeCraft Creations**.

Scope: **monitor status (battery, recording, link) and start/stop multiple GoPros**
over a closed, self-contained travel-router network. **No video flows through the
hub** — footage records locally to each GoPro's SD card. This keeps the hub a
lightweight control plane.

## 2. Architecture

```
GoPro Hero 3  ──Wi-Fi(10.5.5.1)──  ESP-01S  ──UART(JSON)──  XIAO ESP32-C6  ──Wi-Fi/MQTT──  Pi hub  ──  Dashboard
 (per camera)                      (GoPro bridge)            (MQTT bridge + OLED/LED)        (Mosquitto+Go+SQLite)
```

**Camera node** (one per GoPro), two boards:
- **XIAO ESP32-C6** — main MCU / MQTT bridge. Joins the RemoteRig Wi-Fi, talks MQTT
  to the hub, drives the OLED + RGB status LED, reads camera status from the ESP-01S
  over UART. Powered from the 5V rail.
- **ESP-01S (ESP8266)** — GoPro Wi-Fi bridge. Joins the GoPro's AP (`10.5.5.1`),
  relays status/commands to the XIAO over UART. Powered from its **own 3.3V buck**
  (not the XIAO 3V3 pin — Wi-Fi TX spikes ~300 mA).

**Hub** — Raspberry Pi Zero 2 W (hostname `remote-rig-hub`, user `overseer`):
- **Mosquitto** MQTT broker (`:1883`, anonymous, listens on `0.0.0.0`).
- **Go controller** (`remoterig`, systemd service) — MQTT subscriber → SQLite,
  REST API + SSE, serves the embedded React dashboard on `:8080`.
- **SQLite** (`/opt/remoterig/remoterig.db`).
- Decided to **stay on the Zero 2 W** (workload is tiny; a Pi 5 only makes sense if
  video preview/ingest is ever added — not planned).

## 3. Network

- **RemoteRig travel router**, subnet **`192.168.8.0/24`**, gateway `192.168.8.1`,
  has a WAN uplink (internet available — used for the Pi to pull builds).
- **Hub static IP: `192.168.8.56`** (`:1883` MQTT, `:8080` dashboard/API).
- Cameras get DHCP `192.168.8.x`.
- The GoPro AP network (`10.5.5.1`) is separate and only the ESP-01S touches it.
- **History:** the project was originally designed around `10.60.1.0/24`; it was
  re-addressed to `192.168.8.0/24` to match the actual travel router (commit
  `b0062f1`). Wi-Fi SSID `RemoteRig` (creds in `firmware/data/config.json`).

## 4. Repository & workflow

- **Gitea:** `ssh://sc-gitea@code.cubecraftcreations.com:2288/CubeCraft-Creations/remote-rig`
  (web/API on `https://code.cubecraftcreations.com`, private repo).
- **Default/integration branch: `dev`** (work lands here; merges from `main`).
- Layout: `firmware/` (PlatformIO), `cmd/`,`internal/`,`pkg/` (Go hub),
  `src/` (React/Vite/TS dashboard), `scripts/` (Pi setup/deploy), `.gitea/workflows/`
  (CI), `hardware/` (CAD/wiring), `docs/`.

## 5. Tech stack

| Area | Choice |
|------|--------|
| Camera firmware | PlatformIO / Arduino |
| C6 env | `seeed_xiao_esp32c6` — **pioarduino** platform fork, **LittleFS**, U8g2 |
| ESP-01S env | `esp8266-camera` — board `esp01_1m`, flash `dout` |
| Hub | **Go 1.25** (single static binary, `//go:embed` frontend) |
| Dashboard | React + Vite + TypeScript + Tailwind (Vitest) |
| Storage | **SQLite** (not Postgres) |
| Broker | **Mosquitto** |
| CI/CD | **Gitea Actions** (pull-based deploy) |

## 6. Camera node hardware (XIAO ESP32-C6 pin map)

| Pin | Use |
|-----|-----|
| 5V/VIN | rocker → 5V rail |
| D4/SDA, D5/SCL | 1.3" **SH1106** OLED (I2C @ `0x3C`) |
| D0 / D1 / D2 | RGB STAT LED R/G/B (220Ω each), **common-anode** |
| D6 (TX) / D7 (RX) | UART (`Serial1`) to ESP-01S (crossed) |
| D8 / D10 | **reserved** for ESP-01S UART-OTA control (RST / GPIO0) — not driven yet |
| 5V rail (330Ω) | PWR LED (not an MCU pin) |

Canonical wiring: Notion "XIAO ESP32-C6 Pin-to-Pin Wiring Diagram" + `hardware/README.md`.

## 7. Firmware behavior

**XIAO ESP32-C6 (`firmware/src/esp32-mqtt-bridge.cpp`)** — fw `0.4.0`:
- Loads config from LittleFS `/config.json` (Wi-Fi, broker, camera_id, battery cal).
- **Self-assigned camera_id** = device id `rig-<last3 MAC>` (e.g. `rig-86d978`) — see
  decision #7. Subscribes `remoterig/cameras/<id>/command`, announces on
  `remoterig/cameras/<id>/announce`, publishes `.../status`.
- OLED status panel (CAM id / REC + session timer / BAT / LINK / CAM reachability).
- RGB STAT LED health colors: blue=boot, red=offline, magenta=Wi-Fi-no-hub,
  yellow=hub-no-camera, green=healthy.
- Battery calibration: two-point linear (raw offset-57 → %), persisted; `battery_pct`
  emitted only when calibrated.
- No-reflash config: `set_camera_config` (MQTT) → forwarded to ESP-01S as `set_config`.

**ESP-01S (`firmware/src/esp8266-camera-bridge.cpp`)**:
- Joins GoPro AP, polls status, relays JSON over UART; `set_config` persists to LittleFS.
- No status LED (GPIO1 is the UART TX).
- ⚠️ **Known bug:** `fetchStatus()` GETs the **shutter** endpoint
  (`/bacpac/SH?...&p=%01`) instead of a real status read — would *start recording*
  each poll. Needs the GoPro Hero 3 protocol corrected + validated against a real
  camera (also verify password/SSID). **Do not point at a live GoPro until fixed.**

**Provisioning:** `firmware/data/config.json` is flashed to the C6's LittleFS via
`pio run -e seeed_xiao_esp32c6 -t uploadfs`. Per maintainer decision the real Wi-Fi
password lives in this tracked file (private repo, low-sensitivity closed network).

## 8. Hub & CI/CD (pull-based deploy)

**Flow:** `push to dev` → Gitea Actions `build-dev.yaml` builds the React frontend
(into `cmd/server/src/dist`, embedded) + cross-compiles the **arm64** Go binary →
publishes a rolling **`dev-latest`** release (binary + `sha256` + `version.txt`) via
a Node script (`.gitea/scripts/publish-release.mjs`). The Pi's
`remoterig-update.timer` runs `scripts/pi-update.sh` every ~5 min → compares
`version.txt`, downloads + checksum-verifies, **atomically replaces** the binary,
restarts, health-checks (rolls back on failure).

**Why pull, not push:** the Pi is on a closed travel-router LAN the CI runner can't
reach; the Pi pulls instead.

- `ci.yaml` — frontend quality gates (lint/typecheck/test/build), single job.
- First-time Pi setup: `sudo bash scripts/setup-pi.sh --config config.yaml` (installs
  Mosquitto, the service, the updater timer, static IP).
- Pi files: `/opt/remoterig/{remoterig, config.yaml, update.env, VERSION, deploy.sh, pi-update.sh}`.
- Future ESP-01S firmware OTA: `docs/design/esp01s-uart-ota.md` ("XIAO as flasher").

### Gitea Actions runner notes (important)
- Runner `remote-rig-runner`, label **`go-react`** (Dockerized act_runner). Workflows
  must use `runs-on: go-react`.
- The `go-react` image has **Node but not Go** → use `setup-go` (its static binary
  runs); get Node from the image (**don't** use `setup-node` — its dynamically-linked
  Node won't execute here, "cannot execute: required file not found").
- Gitea doesn't support `actions/upload-artifact@v4`. No `curl`/`jq`/`sudo` on the
  runner — the release publish is done in Node.
- The runner's network to github.com/Gitea is flaky (ECONNRESET) → keep few action
  clones; `publish-release.mjs` retries.
- Rolling release tag is **`dev-latest`**, NOT `dev` (a `dev` tag collides with the
  `dev` branch → ambiguous refs).
- Inspect CI from a dev machine with the **`tea` CLI**: `tea actions runs list|view|logs`,
  `tea release list` (note "completed" ≠ success — check Conclusion).

## 9. Key decisions & gotchas (log)

1. **MCU:** ESP32-C3 Super Mini → **XIAO ESP32-C6**; C6 needs the **pioarduino**
   platform fork. USB-CDC-on-boot for `Serial` over native USB.
2. **Mac build toolchain:** use `~/.platformio/penv/bin/pio` (Python 3.11), **not**
   the pyenv 3.9.21 shim (too old for pioarduino).
3. **C6 filesystem = LittleFS** (pioarduino `uploadfs` builds LittleFS, not SPIFFS) —
   the firmware reads `/config.json` (data file must be named `config.json`).
4. **Network re-addressed** `10.60.1.0/24` → `192.168.8.0/24`.
5. **Wi-Fi password kept in git** (`firmware/data/config.json`) — maintainer decision
   (private repo, low-sensitivity, closed net).
6. **RGB LED is common-anode** (`RGB_COMMON_ANODE 1`); OLED is **SH1106** @ `0x3C`.
7. **Camera registration = "Option B" self-assigned IDs:** node uses `rig-<MAC>` as a
   stable `camera_id` from first boot; the hub registers under that id. No `cam-NNN`
   assignment, no `registered`-reply handshake. (`docs/MQTT_CONTRACT.md` updated.)
8. **Hub tolerates clockless status timestamps** — nodes have no RTC; firmware omits
   `timestamp`, the hub stamps server-side (it used to reject the status).
9. **ESP-01S updates:** settings change live via `set_config` (no reflash); full
   firmware OTA is the future XIAO-as-flasher path (`docs/design/esp01s-uart-ota.md`).
10. **Pull-based deploy** via rolling `dev-latest` release + atomic binary replace +
    network retries.
11. **Pi systemd service user = `overseer`** (this Pi has no `pi` user); `setup-pi.sh`
    now defaults the service user to `$SUDO_USER`.
12. **Hub embeds the frontend** via `//go:embed all:src/dist`; Vite builds into
    `cmd/server/src/dist` (a committed `index.html` placeholder keeps the embed valid).
13. **SQLite/modernc datetime:** `modernc.org/sqlite` returns a `COALESCE()`/expression
    as a raw string (no type affinity) → can't scan into `time.Time`. Scan plain
    `DATETIME` columns (returned as `time.Time`) via `sql.NullTime`; `ListCameras`
    `COALESCE`s NULL int/bool status columns. Nodes send no usable timestamp on
    status/heartbeat (numeric `millis()`) — the hub ignores it / stamps server-side.
14. **Legacy id migration:** `handleAnnounce` migrates a MAC registered under a
    different `camera_id` (e.g. a pre-self-id `cam-NNN`) to the node's self-id.
15. **Kiosk API auth:** `api_key: ""` in `config.yaml` = no auth on `/api/v1/*`
    (closed LAN, consistent with anonymous MQTT). A non-empty key requires the SPA
    to send `X-API-Key` too, or the dashboard 401s and shows no cameras.
16. **Ops gotcha:** the pull updater swaps only the **binary**. `config.yaml` is NOT
    auto-deployed — change it on the Pi (`/opt/remoterig/config.yaml` + restart).
17. **GoPro Hero 3 protocol** (validated on a Silver): API host `10.5.5.9`, status
    read `GET /camera/se?t=<pwd>` (binary, starts with 0x00 — read the stream, not
    Arduino String), recording = byte 29, battery = byte 19; record start/stop =
    `/bacpac/SH?t=<pwd>&p=%01/%00`. ESP-01S flashing needs RST tied HIGH (RST→GND
    holds it in reset) and a known-good UART adapter (verify with a TX↔RX loopback).
18. **Control path:** `/cameras/{id}/start|stop` publish `{"command":...}` to
    `remoterig/cameras/<id>/command` via `Subscriber.PublishCommand`; the XIAO forwards
    it over UART to the ESP-01S. (The handlers used to only write a DB row — no command
    was ever sent.)
19. **SSE longevity:** no global `middleware.Timeout` and `WriteTimeout: 0` — a write
    deadline terminates the long-lived `/events/stream` (it was dying at 10s). The SPA
    also **seeds** the list via `GET /api/v1/cameras` on mount (SSE only pushes on change).
20. **Nullable status JSON:** `battery_pct`/`video_remaining_sec` serialized as `null`
    (not `omitempty`) — omitting them became `undefined` in the SPA → "NaN%".
21. **UART is two independent wires:** status (ESP `TX/GPIO1` → XIAO `D7`) and commands
    (XIAO `D6` → ESP `RX/GPIO3`) are separate paths — receiving status does NOT prove
    the command direction works. Verify the command path with the `set_config`
    poll-interval test (status rate should change).

## 10. Conventions

- Production hub/controller in **Go**; Python fine for diagnostics/experiments/migrations.
- **SQLite**, not Postgres. **Timezone: US Eastern.**
- Work on `dev`; commit messages end with a `Co-Authored-By` trailer.
- Canonical design docs live in **Notion** (Remote Rig parent page) and the repo
  `docs/`; CAD in Seafile; code/build in Gitea.

## 11. Current status & open items (2026-06-05)

**Working / proven on hardware:**
- Hub up on the Pi (Mosquitto + `remoterig` + SQLite); **dashboard renders live**
  (kiosk mode `api_key:""`, SSE kept alive, list seeded via REST on mount).
- Full CI/CD loop proven: commit → CI build → `dev-latest` → Pi self-update
  (checksum, atomic replace, health-check) → service active.
- C6 (fw `0.4.0`) self-IDs as `rig-86d978`, registered + listed.
- **GoPro monitoring works (Hero 3 Silver):** ESP-01S joins `goprosilver-1`, reads
  `/camera/se`, and `online:true` + `battery_raw` + `video_remaining_sec` flow
  GoPro → ESP-01S → XIAO → MQTT → hub → SQLite → API/SSE → dashboard.
- Hub publishes start/stop commands to `…/<id>/command` (verified on the bus).

**In progress / unresolved:**
- **Camera CONTROL not working — XIAO→ESP command wire is faulty.** Status (ESP→XIAO,
  `GPIO1→D7`) works, but the command direction (XIAO `D6` → ESP `RX/GPIO3`) does not,
  so `start_recording`/`set_config` never reach the ESP. Confirmed via the `set_config`
  poll-interval test (status rate didn't change). Fix/re-seat that one jumper; then
  Record + live config will work. (See decision #21.)
- **Battery calibration:** `battery_raw` (~56–59) flows; set `set_battery_cal`
  (`raw_min/raw_max`, provisionally 0/100) for `battery_pct` — but this is a *command*,
  so it's blocked by the same XIAO→ESP wire above. `video_remaining` offset (25-26)
  provisional. SPA now shows "N/A" (not "NaN%") when `battery_pct` is null.
- **Pi SD-card health:** a transient `Input/output error` on core binaries cleared on
  reboot — watch for recurrence (failing card); re-image via `setup-pi.sh` if it
  returns (everything is reproducible from git).
- **Rotate the Gitea runner registration token** (was exposed in a setup paste).
- Gitea repo **default-branch HEAD** points at a nonexistent ref — set default branch.
- Optional: clear the stale **retained** MQTT message at
  `remoterig/cameras/announce-rig-86d978` (from old firmware).

## 12. Handy commands

```bash
# Build/flash C6 firmware (Mac):
~/.platformio/penv/bin/pio run -d firmware -e seeed_xiao_esp32c6 -t upload
~/.platformio/penv/bin/pio run -d firmware -e seeed_xiao_esp32c6 -t uploadfs   # provision /config.json
# ESP-01S: needs a 3.3V USB-UART adapter with GPIO0->GND for flash mode.

# Inspect Gitea CI (Mac, tea CLI logged in as 'overseer'):
tea actions runs list --repo CubeCraft-Creations/remote-rig
tea actions runs view <id> --repo … ; tea actions runs logs <id> --repo …
tea release list --repo CubeCraft-Creations/remote-rig

# On the Pi:
sudo systemctl start remoterig-update.service   # force a pull/deploy
cat /opt/remoterig/VERSION ; systemctl is-active remoterig
journalctl -u remoterig -n 40 --no-pager
mosquitto_sub -h localhost -t 'remoterig/#' -v
curl -s -H "X-API-Key: changeme" http://localhost:8080/api/v1/cameras
```