# speed-logger

A self-hosted WiFi quality reporter. Clone and run `docker compose up -d` — no other setup required beyond a `.env` file.

## Architecture

Three Docker containers sharing a named volume (`speed-logger_sqlite-data`) that holds a single SQLite database at `/data/speedtest.db`:

- **speedtest** (`measurement/`) — Python + cron inside a Debian slim container. Runs measurements and writes results to SQLite.
- **ping** — Same image, runs `run_ping.py` as a long-lived loop (not cron) with `network_mode: host` so it can reach the LAN gateway. Pings the gateway plus a rotating pool of external DNS-resolver IPs every `PING_INTERVAL` (5s) and writes to `ping_checks`.
- **grafana** — Stock Grafana image with the `frser-sqlite-datasource` plugin. Reads from the shared SQLite volume. Dashboard and datasource are provisioned automatically from `grafana/provisioning/`.

No message queues, no ORM, no external services. Everything is plain Python stdlib + subprocess calls.

## Key files

| File | Role |
|---|---|
| `measurement/run_speedtest.py` | Entry point. Shells out to `speedtest` CLI, parses JSON result, writes to DB. |
| `measurement/run_ping.py` | Long-running loop for the ping container. Detects the default gateway from `/proc/net/route`, pings it plus 3 external IPs per iteration. |
| `measurement/db.py` | SQLite init and insert. All schema lives here. |
| `measurement/config.py` | Env vars: `DB_PATH`, `PING_INTERVAL`. |
| `measurement/crontab` | Cron schedule. Currently `4-59/15` (jittered to avoid running exactly on the quarter-hour, which gave flaky Ookla results). |
| `measurement/Dockerfile` | Installs Ookla `speedtest` CLI via packagecloud, sets up cron. |
| `grafana/provisioning/` | Auto-provisioned datasource and dashboard JSON. |
| `docker-compose.yml` | Wires everything together. Requires `.env` with `GRAFANA_PORT`, `GRAFANA_ADMIN_USER`, `GRAFANA_ADMIN_PASSWORD`. |

## Database schema

Two tables in SQLite.

`speed_tests`:

```sql
id, timestamp (unix float), failed (bool),
isp, ip, location_code, location_city, location_region,
latency, jitter,
down_100kB, down_1MB, down_10MB, down_25MB, down_90th,
up_100kB, up_1MB, up_10MB, up_90th
```

Currently only `down_90th` and `up_90th` are populated (Ookla provides bandwidth as a single bandwidth figure, mapped to the 90th-percentile columns). `failed=True` rows have no other fields — they represent a run where the speedtest binary itself failed.

`ping_checks` (one row per ping, written by the ping container):

```sql
id, timestamp (unix float), target_ip, success (bool), latency_ms, is_gateway (int 0/1)
```

`is_gateway=1` rows are pings to the local router; the gateway-vs-external split is what lets the dashboard distinguish "WiFi/router down" from "ISP down".

## Dashboard

`grafana/provisioning/dashboards/speed_tests.json` (uid `speed-tests-v2`, title "Network Status"). Layout top to bottom: live UP/DOWN stats (last 60s, independent of the time picker via `strftime('%s','now')`), status-page-style state timelines (Internet/Router plus one row per ping-target provider), ping latency, speed tests, ISP SLA checks, hourly/weekday pattern charts, collapsed raw table.

Conventions used throughout:

- A time bucket counts as "up" if ≥1 ping in it succeeded; bucket width adapts to the zoom level via `MAX(60, ($__to - $__from) / 1000 / 500)` in SQL so queries stay fast on long ranges.
- **SLA thresholds are dashboard textbox variables** (`sla_down_max/normal/min`, `sla_up_max/normal/min`) so users can enter their own contract values in the UI without editing files. Defaults are Vodafone GigaZuhause 1000 Kabel (1000/850/600 down, 50/35/15 up, per its Produktinformationsblatt). The SLA panels mirror the German TKG/BNetzA deviation criteria: normal speed in ≥90% of tests, 90% of max reached on ≥2/3 of days, minimum never undercut.

## Design principles

- **Python stdlib only** inside containers. No pip dependencies, no `uv`, no third-party packages. Use `subprocess`, `socket`, `sqlite3`.
- **Simple is correct.** Do not add abstractions, config layers, or retry logic beyond what's needed for the specific failure mode being addressed.
- **Cron for scheduling.** No Python scheduler libraries. The crontab file is the schedule.
- **One DB, one volume.** Both containers share the same named Docker volume. Do not introduce a second database or a network API between containers.
- **Comments only when the WHY is non-obvious.** The crontab offset (`4-59/15` instead of `*/15`) is a good example — it exists to prevent false failures from Ookla's rate limiting at exact quarter-hours and is worth a comment.

## Development

There is no test suite. To iterate:

1. Edit files in `measurement/`.
2. `docker compose build speedtest && docker compose up -d speedtest` to redeploy the measurement container.
3. Trigger a manual run: `docker exec speedtest python3 /app/run_speedtest.py`
4. Inspect DB: `docker exec speedtest python3 -c "import sqlite3; ..."`

The Grafana dashboard JSON is in `grafana/provisioning/dashboards/speed_tests.json`. Edit it in the Grafana UI (enable "Allow UI updates" is already set in `dashboard.yml`), then export and overwrite the JSON file to persist changes.