83 lines
6.4 KiB
Markdown
83 lines
6.4 KiB
Markdown
# speed-logger
|
|
|
|
A self-hosted WiFi quality reporter. Clone and run `docker compose up -d` — no other setup required beyond a `.env` file.
|
|
|
|
## Architecture
|
|
|
|
Three Docker containers sharing a named volume (`speed-logger_sqlite-data`) that holds a single SQLite database at `/data/speedtest.db`:
|
|
|
|
- **speedtest** (`measurement/`) — Python + cron inside a Debian slim container. Runs measurements and writes results to SQLite.
|
|
- **ping** — Same image, runs `run_ping.py` as a long-lived loop (not cron) with `network_mode: host` so it can reach the LAN gateway. Pings the gateway plus a rotating pool of external DNS-resolver IPs every `PING_INTERVAL` (5s) and writes to `ping_checks`.
|
|
- **control** — Same image, runs `control.py`: a stdlib `http.server` page on port 80 (host port `CONTROL_PORT`) where household members toggle speedtests on/off from a phone. Writes the flag to the `settings` table; talks to humans only, never to other containers.
|
|
- **grafana** — Stock Grafana image with the `frser-sqlite-datasource` plugin. Reads from the shared SQLite volume. Dashboard and datasource are provisioned automatically from `grafana/provisioning/`. Anonymous read-only access is enabled (`GF_AUTH_ANONYMOUS_*`); because any viewer can POST arbitrary SQL to Grafana's datasource query API, the volume is mounted `:ro` and the datasource sets `pathOptions: mode=ro`. Do not remove either when touching the compose file. This also means the DB must stay in the default rollback journal mode — WAL would require write access to the directory even for readers.
|
|
|
|
No message queues, no ORM, no external services. Everything is plain Python stdlib + subprocess calls.
|
|
|
|
## Key files
|
|
|
|
| File | Role |
|
|
|---|---|
|
|
| `measurement/run_speedtest.py` | Entry point. Shells out to `speedtest` CLI, parses JSON result, writes to DB. |
|
|
| `measurement/run_ping.py` | Long-running loop for the ping container. Detects the default gateway from `/proc/net/route`, pings it plus 3 external IPs per iteration. |
|
|
| `measurement/control.py` | Control page for the household speedtest toggle. Seeds the `settings` row from `SPEEDTEST_ENABLED` on first start; after that the DB row is the source of truth. |
|
|
| `measurement/db.py` | SQLite init and insert. All schema lives here. |
|
|
| `measurement/config.py` | Env vars: `DB_PATH`, `PING_INTERVAL`, `SPEEDTEST_ENABLED` (initial toggle state only; the live state is the `speedtest_enabled` row in `settings`). When disabled, the cron job still fires but exits before running the test, and no failed row is written. |
|
|
| `measurement/crontab` | Cron schedule. Currently `4-59/15` (jittered to avoid running exactly on the quarter-hour, which gave flaky Ookla results). |
|
|
| `measurement/Dockerfile` | Installs Ookla `speedtest` CLI via packagecloud, sets up cron. |
|
|
| `grafana/provisioning/` | Auto-provisioned datasource and dashboard JSON. |
|
|
| `docker-compose.yml` | Wires everything together. Requires `.env` with `GRAFANA_PORT`, `GRAFANA_ADMIN_USER`, `GRAFANA_ADMIN_PASSWORD`. |
|
|
|
|
## Database schema
|
|
|
|
Two tables in SQLite.
|
|
|
|
`speed_tests`:
|
|
|
|
```sql
|
|
id, timestamp (unix float), failed (bool),
|
|
isp, ip, location_code, location_city, location_region,
|
|
latency, jitter,
|
|
down_100kB, down_1MB, down_10MB, down_25MB, down_90th,
|
|
up_100kB, up_1MB, up_10MB, up_90th
|
|
```
|
|
|
|
Currently only `down_90th` and `up_90th` are populated (Ookla provides bandwidth as a single bandwidth figure, mapped to the 90th-percentile columns). `failed=True` rows have no other fields — they represent a run where the speedtest binary itself failed.
|
|
|
|
`ping_checks` (one row per ping, written by the ping container):
|
|
|
|
```sql
|
|
id, timestamp (unix float), target_ip, success (bool), latency_ms, is_gateway (int 0/1)
|
|
```
|
|
|
|
`is_gateway=1` rows are pings to the local router; the gateway-vs-external split is what lets the dashboard distinguish "WiFi/router down" from "ISP down".
|
|
|
|
`settings` (key-value, currently only `speedtest_enabled` = "0"/"1"): shared state between the control page (writes), the speedtest cron job (reads) and Grafana (displays). This is the only cross-container communication channel besides the measurement tables.
|
|
|
|
## Dashboard
|
|
|
|
`grafana/provisioning/dashboards/speed_tests.json` (uid `speed-tests-v2`, title "Network Status"). Layout top to bottom: live UP/DOWN stats (last 60s, independent of the time picker via `strftime('%s','now')`), status-page-style state timelines (Internet/Router plus one row per ping-target provider), ping latency, speed tests, ISP SLA checks, hourly/weekday pattern charts, collapsed raw table.
|
|
|
|
Conventions used throughout:
|
|
|
|
- A time bucket counts as "up" if ≥1 ping in it succeeded; bucket width adapts to the zoom level via `MAX(60, ($__to - $__from) / 1000 / 500)` in SQL so queries stay fast on long ranges.
|
|
- **SLA thresholds are dashboard textbox variables** (`sla_down_max/normal/min`, `sla_up_max/normal/min`) so users can enter their own contract values in the UI without editing files. Defaults are Vodafone GigaZuhause 1000 Kabel (1000/850/600 down, 50/35/15 up, per its Produktinformationsblatt). The SLA panels mirror the German TKG/BNetzA deviation criteria: normal speed in ≥90% of tests, 90% of max reached on ≥2/3 of days, minimum never undercut.
|
|
|
|
## Design principles
|
|
|
|
- **Python stdlib only** inside containers. No pip dependencies, no `uv`, no third-party packages. Use `subprocess`, `socket`, `sqlite3`.
|
|
- **Simple is correct.** Do not add abstractions, config layers, or retry logic beyond what's needed for the specific failure mode being addressed.
|
|
- **Cron for scheduling.** No Python scheduler libraries. The crontab file is the schedule.
|
|
- **One DB, one volume.** Both containers share the same named Docker volume. Do not introduce a second database or a network API between containers.
|
|
- **Comments only when the WHY is non-obvious.** The crontab offset (`4-59/15` instead of `*/15`) is a good example — it exists to prevent false failures from Ookla's rate limiting at exact quarter-hours and is worth a comment.
|
|
|
|
## Development
|
|
|
|
There is no test suite. To iterate:
|
|
|
|
1. Edit files in `measurement/`.
|
|
2. `docker compose build speedtest && docker compose up -d speedtest` to redeploy the measurement container.
|
|
3. Trigger a manual run: `docker exec speedtest python3 /app/run_speedtest.py`
|
|
4. Inspect DB: `docker exec speedtest python3 -c "import sqlite3; ..."`
|
|
|
|
The Grafana dashboard JSON is in `grafana/provisioning/dashboards/speed_tests.json`. Edit it in the Grafana UI (enable "Allow UI updates" is already set in `dashboard.yml`), then export and overwrite the JSON file to persist changes.
|