generated from CubeCraft-Creations/Tracehound
docs: add root CONTEXT.md project working-context / decision log
Living context + decision log for humans and LLMs: architecture, network, repo workflow, hardware pin map, firmware behavior, pull-based CI/CD, key decisions/gotchas, current status, and handy commands. Cross-links the deeper docs/ references. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Joshua King
+228
@@ -0,0 +1,228 @@
|
||||
# RemoteRig — Project Working Context
|
||||
|
||||
> **Purpose of this file:** a living, high-signal context + decision log for the
|
||||
> RemoteRig project. It's the primary onboarding doc for humans and for any LLM
|
||||
> working on the repo. Keep it updated as decisions are made.
|
||||
> **Last updated:** 2026-06-05.
|
||||
>
|
||||
> Deeper references: `docs/CONTEXT.md` (system architecture detail),
|
||||
> `docs/MQTT_CONTRACT.md` (MQTT topics/payloads), `docs/design/` (design notes),
|
||||
> `hardware/README.md` (case/wiring/BOM), `README.md` (hub build/deploy).
|
||||
|
||||
---
|
||||
|
||||
## 1. What this is
|
||||
|
||||
RemoteRig is a multi-camera **GoPro Hero 3 monitoring & control** system for field
|
||||
recording (large concerts in auditoriums, high-school marching band at stadiums).
|
||||
Founder: **Joshua / CubeCraft Creations**.
|
||||
|
||||
Scope: **monitor status (battery, recording, link) and start/stop multiple GoPros**
|
||||
over a closed, self-contained travel-router network. **No video flows through the
|
||||
hub** — footage records locally to each GoPro's SD card. This keeps the hub a
|
||||
lightweight control plane.
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
```
|
||||
GoPro Hero 3 ──Wi-Fi(10.5.5.1)── ESP-01S ──UART(JSON)── XIAO ESP32-C6 ──Wi-Fi/MQTT── Pi hub ── Dashboard
|
||||
(per camera) (GoPro bridge) (MQTT bridge + OLED/LED) (Mosquitto+Go+SQLite)
|
||||
```
|
||||
|
||||
**Camera node** (one per GoPro), two boards:
|
||||
- **XIAO ESP32-C6** — main MCU / MQTT bridge. Joins the RemoteRig Wi-Fi, talks MQTT
|
||||
to the hub, drives the OLED + RGB status LED, reads camera status from the ESP-01S
|
||||
over UART. Powered from the 5V rail.
|
||||
- **ESP-01S (ESP8266)** — GoPro Wi-Fi bridge. Joins the GoPro's AP (`10.5.5.1`),
|
||||
relays status/commands to the XIAO over UART. Powered from its **own 3.3V buck**
|
||||
(not the XIAO 3V3 pin — Wi-Fi TX spikes ~300 mA).
|
||||
|
||||
**Hub** — Raspberry Pi Zero 2 W (hostname `remote-rig-hub`, user `overseer`):
|
||||
- **Mosquitto** MQTT broker (`:1883`, anonymous, listens on `0.0.0.0`).
|
||||
- **Go controller** (`remoterig`, systemd service) — MQTT subscriber → SQLite,
|
||||
REST API + SSE, serves the embedded React dashboard on `:8080`.
|
||||
- **SQLite** (`/opt/remoterig/remoterig.db`).
|
||||
- Decided to **stay on the Zero 2 W** (workload is tiny; a Pi 5 only makes sense if
|
||||
video preview/ingest is ever added — not planned).
|
||||
|
||||
## 3. Network
|
||||
|
||||
- **RemoteRig travel router**, subnet **`192.168.8.0/24`**, gateway `192.168.8.1`,
|
||||
has a WAN uplink (internet available — used for the Pi to pull builds).
|
||||
- **Hub static IP: `192.168.8.56`** (`:1883` MQTT, `:8080` dashboard/API).
|
||||
- Cameras get DHCP `192.168.8.x`.
|
||||
- The GoPro AP network (`10.5.5.1`) is separate and only the ESP-01S touches it.
|
||||
- **History:** the project was originally designed around `10.60.1.0/24`; it was
|
||||
re-addressed to `192.168.8.0/24` to match the actual travel router (commit
|
||||
`b0062f1`). Wi-Fi SSID `RemoteRig` (creds in `firmware/data/config.json`).
|
||||
|
||||
## 4. Repository & workflow
|
||||
|
||||
- **Gitea:** `ssh://sc-gitea@code.cubecraftcreations.com:2288/CubeCraft-Creations/remote-rig`
|
||||
(web/API on `https://code.cubecraftcreations.com`, private repo).
|
||||
- **Default/integration branch: `dev`** (work lands here; merges from `main`).
|
||||
- Layout: `firmware/` (PlatformIO), `cmd/`,`internal/`,`pkg/` (Go hub),
|
||||
`src/` (React/Vite/TS dashboard), `scripts/` (Pi setup/deploy), `.gitea/workflows/`
|
||||
(CI), `hardware/` (CAD/wiring), `docs/`.
|
||||
|
||||
## 5. Tech stack
|
||||
|
||||
| Area | Choice |
|
||||
|------|--------|
|
||||
| Camera firmware | PlatformIO / Arduino |
|
||||
| C6 env | `seeed_xiao_esp32c6` — **pioarduino** platform fork, **LittleFS**, U8g2 |
|
||||
| ESP-01S env | `esp8266-camera` — board `esp01_1m`, flash `dout` |
|
||||
| Hub | **Go 1.25** (single static binary, `//go:embed` frontend) |
|
||||
| Dashboard | React + Vite + TypeScript + Tailwind (Vitest) |
|
||||
| Storage | **SQLite** (not Postgres) |
|
||||
| Broker | **Mosquitto** |
|
||||
| CI/CD | **Gitea Actions** (pull-based deploy) |
|
||||
|
||||
## 6. Camera node hardware (XIAO ESP32-C6 pin map)
|
||||
|
||||
| Pin | Use |
|
||||
|-----|-----|
|
||||
| 5V/VIN | rocker → 5V rail |
|
||||
| D4/SDA, D5/SCL | 1.3" **SH1106** OLED (I2C @ `0x3C`) |
|
||||
| D0 / D1 / D2 | RGB STAT LED R/G/B (220Ω each), **common-anode** |
|
||||
| D6 (TX) / D7 (RX) | UART (`Serial1`) to ESP-01S (crossed) |
|
||||
| D8 / D10 | **reserved** for ESP-01S UART-OTA control (RST / GPIO0) — not driven yet |
|
||||
| 5V rail (330Ω) | PWR LED (not an MCU pin) |
|
||||
|
||||
Canonical wiring: Notion "XIAO ESP32-C6 Pin-to-Pin Wiring Diagram" + `hardware/README.md`.
|
||||
|
||||
## 7. Firmware behavior
|
||||
|
||||
**XIAO ESP32-C6 (`firmware/src/esp32-mqtt-bridge.cpp`)** — fw `0.4.0`:
|
||||
- Loads config from LittleFS `/config.json` (Wi-Fi, broker, camera_id, battery cal).
|
||||
- **Self-assigned camera_id** = device id `rig-<last3 MAC>` (e.g. `rig-86d978`) — see
|
||||
decision #7. Subscribes `remoterig/cameras/<id>/command`, announces on
|
||||
`remoterig/cameras/<id>/announce`, publishes `.../status`.
|
||||
- OLED status panel (CAM id / REC + session timer / BAT / LINK / CAM reachability).
|
||||
- RGB STAT LED health colors: blue=boot, red=offline, magenta=Wi-Fi-no-hub,
|
||||
yellow=hub-no-camera, green=healthy.
|
||||
- Battery calibration: two-point linear (raw offset-57 → %), persisted; `battery_pct`
|
||||
emitted only when calibrated.
|
||||
- No-reflash config: `set_camera_config` (MQTT) → forwarded to ESP-01S as `set_config`.
|
||||
|
||||
**ESP-01S (`firmware/src/esp8266-camera-bridge.cpp`)**:
|
||||
- Joins GoPro AP, polls status, relays JSON over UART; `set_config` persists to LittleFS.
|
||||
- No status LED (GPIO1 is the UART TX).
|
||||
- ⚠️ **Known bug:** `fetchStatus()` GETs the **shutter** endpoint
|
||||
(`/bacpac/SH?...&p=%01`) instead of a real status read — would *start recording*
|
||||
each poll. Needs the GoPro Hero 3 protocol corrected + validated against a real
|
||||
camera (also verify password/SSID). **Do not point at a live GoPro until fixed.**
|
||||
|
||||
**Provisioning:** `firmware/data/config.json` is flashed to the C6's LittleFS via
|
||||
`pio run -e seeed_xiao_esp32c6 -t uploadfs`. Per maintainer decision the real Wi-Fi
|
||||
password lives in this tracked file (private repo, low-sensitivity closed network).
|
||||
|
||||
## 8. Hub & CI/CD (pull-based deploy)
|
||||
|
||||
**Flow:** `push to dev` → Gitea Actions `build-dev.yaml` builds the React frontend
|
||||
(into `cmd/server/src/dist`, embedded) + cross-compiles the **arm64** Go binary →
|
||||
publishes a rolling **`dev-latest`** release (binary + `sha256` + `version.txt`) via
|
||||
a Node script (`.gitea/scripts/publish-release.mjs`). The Pi's
|
||||
`remoterig-update.timer` runs `scripts/pi-update.sh` every ~5 min → compares
|
||||
`version.txt`, downloads + checksum-verifies, **atomically replaces** the binary,
|
||||
restarts, health-checks (rolls back on failure).
|
||||
|
||||
**Why pull, not push:** the Pi is on a closed travel-router LAN the CI runner can't
|
||||
reach; the Pi pulls instead.
|
||||
|
||||
- `ci.yaml` — frontend quality gates (lint/typecheck/test/build), single job.
|
||||
- First-time Pi setup: `sudo bash scripts/setup-pi.sh --config config.yaml` (installs
|
||||
Mosquitto, the service, the updater timer, static IP).
|
||||
- Pi files: `/opt/remoterig/{remoterig, config.yaml, update.env, VERSION, deploy.sh, pi-update.sh}`.
|
||||
- Future ESP-01S firmware OTA: `docs/design/esp01s-uart-ota.md` ("XIAO as flasher").
|
||||
|
||||
### Gitea Actions runner notes (important)
|
||||
- Runner `remote-rig-runner`, label **`go-react`** (Dockerized act_runner). Workflows
|
||||
must use `runs-on: go-react`.
|
||||
- The `go-react` image has **Node but not Go** → use `setup-go` (its static binary
|
||||
runs); get Node from the image (**don't** use `setup-node` — its dynamically-linked
|
||||
Node won't execute here, "cannot execute: required file not found").
|
||||
- Gitea doesn't support `actions/upload-artifact@v4`. No `curl`/`jq`/`sudo` on the
|
||||
runner — the release publish is done in Node.
|
||||
- The runner's network to github.com/Gitea is flaky (ECONNRESET) → keep few action
|
||||
clones; `publish-release.mjs` retries.
|
||||
- Rolling release tag is **`dev-latest`**, NOT `dev` (a `dev` tag collides with the
|
||||
`dev` branch → ambiguous refs).
|
||||
- Inspect CI from a dev machine with the **`tea` CLI**: `tea actions runs list|view|logs`,
|
||||
`tea release list` (note "completed" ≠ success — check Conclusion).
|
||||
|
||||
## 9. Key decisions & gotchas (log)
|
||||
|
||||
1. **MCU:** ESP32-C3 Super Mini → **XIAO ESP32-C6**; C6 needs the **pioarduino**
|
||||
platform fork. USB-CDC-on-boot for `Serial` over native USB.
|
||||
2. **Mac build toolchain:** use `~/.platformio/penv/bin/pio` (Python 3.11), **not**
|
||||
the pyenv 3.9.21 shim (too old for pioarduino).
|
||||
3. **C6 filesystem = LittleFS** (pioarduino `uploadfs` builds LittleFS, not SPIFFS) —
|
||||
the firmware reads `/config.json` (data file must be named `config.json`).
|
||||
4. **Network re-addressed** `10.60.1.0/24` → `192.168.8.0/24`.
|
||||
5. **Wi-Fi password kept in git** (`firmware/data/config.json`) — maintainer decision
|
||||
(private repo, low-sensitivity, closed net).
|
||||
6. **RGB LED is common-anode** (`RGB_COMMON_ANODE 1`); OLED is **SH1106** @ `0x3C`.
|
||||
7. **Camera registration = "Option B" self-assigned IDs:** node uses `rig-<MAC>` as a
|
||||
stable `camera_id` from first boot; the hub registers under that id. No `cam-NNN`
|
||||
assignment, no `registered`-reply handshake. (`docs/MQTT_CONTRACT.md` updated.)
|
||||
8. **Hub tolerates clockless status timestamps** — nodes have no RTC; firmware omits
|
||||
`timestamp`, the hub stamps server-side (it used to reject the status).
|
||||
9. **ESP-01S updates:** settings change live via `set_config` (no reflash); full
|
||||
firmware OTA is the future XIAO-as-flasher path (`docs/design/esp01s-uart-ota.md`).
|
||||
10. **Pull-based deploy** via rolling `dev-latest` release + atomic binary replace +
|
||||
network retries.
|
||||
11. **Pi systemd service user = `overseer`** (this Pi has no `pi` user); `setup-pi.sh`
|
||||
now defaults the service user to `$SUDO_USER`.
|
||||
12. **Hub embeds the frontend** via `//go:embed all:src/dist`; Vite builds into
|
||||
`cmd/server/src/dist` (a committed `index.html` placeholder keeps the embed valid).
|
||||
|
||||
## 10. Conventions
|
||||
|
||||
- Production hub/controller in **Go**; Python fine for diagnostics/experiments/migrations.
|
||||
- **SQLite**, not Postgres. **Timezone: US Eastern.**
|
||||
- Work on `dev`; commit messages end with a `Co-Authored-By` trailer.
|
||||
- Canonical design docs live in **Notion** (Remote Rig parent page) and the repo
|
||||
`docs/`; CAD in Seafile; code/build in Gitea.
|
||||
|
||||
## 11. Current status & open items (2026-06-05)
|
||||
|
||||
**Working / proven on hardware:**
|
||||
- Hub up on the Pi (Mosquitto + `remoterig` + SQLite), dashboard on `:8080`.
|
||||
- Full CI/CD loop proven: commit → CI build → `dev-latest` → Pi self-update
|
||||
(checksum, atomic replace, health-check) → service active.
|
||||
- C6 (fw `0.4.0`) joins RemoteRig, connects to the broker, announces as `rig-86d978`,
|
||||
publishes status. ESP-01S UART link alive.
|
||||
|
||||
**In progress / unresolved:**
|
||||
- **Verifying the hub registered `rig-86d978`** — needs confirmation that the hub is on
|
||||
build `18db26c2` (Option B) and that the camera is in the DB / dashboard
|
||||
(`GET /api/v1/cameras`, header `X-API-Key: changeme`). Status currently shows
|
||||
`online:false` because no GoPro is attached on the bench (expected).
|
||||
- **GoPro Hero 3 protocol fix** (ESP-01S `fetchStatus` shutter bug) — required before
|
||||
real battery/recording data; needs a real Hero 3.
|
||||
- **Rotate the Gitea runner registration token** (was exposed in a setup paste).
|
||||
- Gitea repo **default-branch HEAD** points at a nonexistent ref — set default branch.
|
||||
- Optional: clear the stale **retained** MQTT message at
|
||||
`remoterig/cameras/announce-rig-86d978` (from old firmware).
|
||||
|
||||
## 12. Handy commands
|
||||
|
||||
```bash
|
||||
# Build/flash C6 firmware (Mac):
|
||||
~/.platformio/penv/bin/pio run -d firmware -e seeed_xiao_esp32c6 -t upload
|
||||
~/.platformio/penv/bin/pio run -d firmware -e seeed_xiao_esp32c6 -t uploadfs # provision /config.json
|
||||
# ESP-01S: needs a 3.3V USB-UART adapter with GPIO0->GND for flash mode.
|
||||
|
||||
# Inspect Gitea CI (Mac, tea CLI logged in as 'overseer'):
|
||||
tea actions runs list --repo CubeCraft-Creations/remote-rig
|
||||
tea actions runs view <id> --repo … ; tea actions runs logs <id> --repo …
|
||||
tea release list --repo CubeCraft-Creations/remote-rig
|
||||
|
||||
# On the Pi:
|
||||
sudo systemctl start remoterig-update.service # force a pull/deploy
|
||||
cat /opt/remoterig/VERSION ; systemctl is-active remoterig
|
||||
journalctl -u remoterig -n 40 --no-pager
|
||||
mosquitto_sub -h localhost -t 'remoterig/#' -v
|
||||
curl -s -H "X-API-Key: changeme" http://localhost:8080/api/v1/cameras
|
||||
```
|
||||
Reference in New Issue
Block a user