added nextloud tracker
This commit is contained in:
@@ -0,0 +1,3 @@
|
|||||||
|
__pycache__/
|
||||||
|
ctf_status.env
|
||||||
|
*.md.out
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
# ctf_status.py
|
||||||
|
|
||||||
|
Reads the per-author challenge folders straight off the Nextcloud share (WebDAV,
|
||||||
|
no sync, no mount) and tells you who to scream at.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cp ctf_status.env.example ctf_status.env && chmod 600 ctf_status.env
|
||||||
|
$EDITOR ctf_status.env # app password from Nextcloud → Settings → Security
|
||||||
|
```
|
||||||
|
|
||||||
|
## Use
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./ctf_status.py # terminal summary
|
||||||
|
./ctf_status.py -o overview.md # + HedgeDoc markdown to a file
|
||||||
|
./ctf_status.py --stdout | wl-copy # straight to clipboard, paste into HedgeDoc
|
||||||
|
./ctf_status.py --update # overwrite the note in HEDGEDOC_NOTE
|
||||||
|
./ctf_status.py --update <note-url> # ... or an explicit one
|
||||||
|
```
|
||||||
|
|
||||||
|
## Keeping a HedgeDoc note up to date
|
||||||
|
|
||||||
|
`--update` overwrites an **existing** note in place, so the URL you share stays the
|
||||||
|
same. Every run rewrites the whole body, starting with
|
||||||
|
|
||||||
|
```
|
||||||
|
---
|
||||||
|
tags: Logistics, h4tum CTF 2026
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
and a `**Last updated:** <date> <time> <tz>` line under the heading, so a stale note
|
||||||
|
is obvious at a glance. Tags come from `HEDGEDOC_TAGS` (or `--tags`).
|
||||||
|
|
||||||
|
The note's permission has to be **freely** (Menu → Permission), because the run is
|
||||||
|
anonymous. HedgeDoc 1.x has no REST API for editing a note, so the script talks the
|
||||||
|
editor's own engine.io/socket.io + OT protocol over long-polling — it reads the
|
||||||
|
current text, sends one "delete everything, insert this" operation, waits for the
|
||||||
|
server's `ack`, and reads the note back to confirm. Anyone with the note open sees
|
||||||
|
the change live. Still `requests`-only, no new dependencies.
|
||||||
|
|
||||||
|
`--hedgedoc <instance>` (POST a *new* note) is unrelated and only works where
|
||||||
|
anonymous note creation is enabled — on `hedgedoc.cit.tum.de` it is not.
|
||||||
|
|
||||||
|
## Every 30 minutes, on a server
|
||||||
|
|
||||||
|
Copy the repo to the server, create `ctf_status.env` there (it is gitignored, so it
|
||||||
|
does not travel with the code), then:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./install-timer.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
It writes systemd **user** units into `~/.config/systemd/user/`, substituting
|
||||||
|
whatever directory the repo actually sits in, enables the timer, turns on lingering
|
||||||
|
so it keeps firing when you are not logged in, and does one run to prove it works.
|
||||||
|
Re-run it any time; it just overwrites the units.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
systemctl --user list-timers ctf-status.timer # when it next fires
|
||||||
|
journalctl --user -u ctf-status.service -n 50 # what it did
|
||||||
|
systemctl --user start ctf-status.service # run it now
|
||||||
|
systemctl --user disable --now ctf-status.timer # stop it
|
||||||
|
```
|
||||||
|
|
||||||
|
The timer fires at `:00` and `:30` with up to a minute of jitter, and
|
||||||
|
`Persistent=true` catches up a run missed while the machine was down. Plain cron
|
||||||
|
works too, if you would rather:
|
||||||
|
|
||||||
|
```
|
||||||
|
*/30 * * * * /path/to/planner/ctf_status.py --update --quiet
|
||||||
|
```
|
||||||
|
|
||||||
|
`--quiet` keeps a successful run silent, so you only hear about it when the note
|
||||||
|
changed or something broke. Overlapping runs are prevented by a lock file, and the
|
||||||
|
HedgeDoc write is retried (`--retries`, default 3) before the run gives up with a
|
||||||
|
one-line error and a non-zero exit.
|
||||||
|
|
||||||
|
## What it flags
|
||||||
|
|
||||||
|
| verdict | meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| 🔴 SCREAM | nothing submitted, **or** they filled out `template.md` in place |
|
||||||
|
| 🟡 NAG | submitted, but placeholders / empty sections remain |
|
||||||
|
| 🟢 OK | every challenge fully filled out |
|
||||||
|
|
||||||
|
A `template.md` that is still pristine is ignored. One that has real content in it
|
||||||
|
is counted as a challenge *and* flagged — so nobody's work is lost, they just get
|
||||||
|
yelled at.
|
||||||
|
|
||||||
|
Also flagged per challenge: unnamed challenge, `[placeholder]` values left in any
|
||||||
|
of Category/Difficulty/Event/Author/Tester/Status, empty prose sections, and
|
||||||
|
category/difficulty values outside the allowed set.
|
||||||
|
|
||||||
|
## Layout it expects
|
||||||
|
|
||||||
|
```
|
||||||
|
<NC_ROOT>/
|
||||||
|
alice/
|
||||||
|
README.md # ignored
|
||||||
|
template.md # ignored while pristine
|
||||||
|
baby-rsa.md # ← a challenge, any filename
|
||||||
|
bob/
|
||||||
|
```
|
||||||
|
|
||||||
|
Any `.md` that isn't `README.md` is parsed as a challenge, so the file naming
|
||||||
|
convention doesn't have to be enforced by the script. If you want it enforced,
|
||||||
|
add the check in `Challenge.problems`.
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Push the CTF challenge status to HedgeDoc
|
||||||
|
Documentation=file://__DIR__/README.md
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
WorkingDirectory=__DIR__
|
||||||
|
ExecStart=__DIR__/ctf_status.py --update --quiet
|
||||||
|
# Nextcloud walk + HedgeDoc write; never let one run squat on the next timer slot
|
||||||
|
TimeoutStartSec=5min
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Refresh the CTF challenge status every 30 minutes
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=*:0/30
|
||||||
|
# don't hammer Nextcloud and HedgeDoc on the exact half hour
|
||||||
|
RandomizedDelaySec=60
|
||||||
|
AccuracySec=30s
|
||||||
|
# catch up on the last run if the machine was down when it was due
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
# cp to ctf_status.env and chmod 600
|
||||||
|
NC_URL=https://cloud.h4tum.de
|
||||||
|
NC_USER=your-nextcloud-login
|
||||||
|
# Settings -> Security -> "Create new app password" (NOT your real password)
|
||||||
|
NC_PASS=xxxxx-xxxxx-xxxxx-xxxxx-xxxxx
|
||||||
|
# path of the folder holding the per-author folders, relative to your Nextcloud root
|
||||||
|
NC_ROOT=CTF/challenges
|
||||||
|
|
||||||
|
# --- HedgeDoc: the existing note that gets overwritten every run ------------
|
||||||
|
# full URL of the note (its permission must be "freely" so an anonymous
|
||||||
|
# run may write to it), e.g. https://hedgedoc.cit.tum.de/AbCdEf123
|
||||||
|
HEDGEDOC_NOTE=
|
||||||
|
# YAML front-matter tags written at the top of the note
|
||||||
|
HEDGEDOC_TAGS=Logistics, h4tum CTF 2026
|
||||||
Executable
+618
@@ -0,0 +1,618 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""CTF challenge tracker.
|
||||||
|
|
||||||
|
Reads the per-author challenge folders from a Nextcloud share over WebDAV,
|
||||||
|
parses the filled-out challenge templates and reports:
|
||||||
|
* a terminal "who do I have to scream at" list
|
||||||
|
* a HedgeDoc-ready markdown overview
|
||||||
|
|
||||||
|
Config: see ctf_status.env.example -> ctf_status.env (chmod 600)
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import fcntl
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import random
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import quote, unquote, urljoin, urlsplit
|
||||||
|
from xml.etree import ElementTree as ET
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- config ----
|
||||||
|
|
||||||
|
CFG_KEYS = ("NC_URL", "NC_USER", "NC_PASS", "NC_ROOT")
|
||||||
|
OPT_KEYS = ("HEDGEDOC_NOTE", "HEDGEDOC_TAGS", "TITLE")
|
||||||
|
DEFAULT_TAGS = "Logistics, h4tum CTF 2026"
|
||||||
|
DEFAULT_TITLE = "CTF challenge tracking"
|
||||||
|
|
||||||
|
|
||||||
|
def load_env() -> dict:
|
||||||
|
cfg = {}
|
||||||
|
for candidate in (
|
||||||
|
Path(__file__).with_name("ctf_status.env"),
|
||||||
|
Path.home() / ".config" / "ctf-status" / "env",
|
||||||
|
):
|
||||||
|
if candidate.exists():
|
||||||
|
for line in candidate.read_text().splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if not line or line.startswith("#") or "=" not in line:
|
||||||
|
continue
|
||||||
|
k, v = line.split("=", 1)
|
||||||
|
cfg[k.strip()] = v.strip().strip("'\"")
|
||||||
|
break
|
||||||
|
for k in CFG_KEYS + OPT_KEYS:
|
||||||
|
if os.environ.get(k):
|
||||||
|
cfg[k] = os.environ[k]
|
||||||
|
missing = [k for k in CFG_KEYS if not cfg.get(k)]
|
||||||
|
if missing:
|
||||||
|
sys.exit(f"missing config: {', '.join(missing)} (see ctf_status.env.example)")
|
||||||
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- webdav -----
|
||||||
|
|
||||||
|
DAV_NS = "{DAV:}"
|
||||||
|
|
||||||
|
|
||||||
|
class Dav:
|
||||||
|
def __init__(self, base_url: str, user: str, password: str):
|
||||||
|
self.base = base_url.rstrip("/") + f"/remote.php/dav/files/{quote(user)}/"
|
||||||
|
self.s = requests.Session()
|
||||||
|
self.s.auth = (user, password)
|
||||||
|
|
||||||
|
def _url(self, path: str) -> str:
|
||||||
|
return urljoin(self.base, quote(path.strip("/")) + "/")
|
||||||
|
|
||||||
|
def walk(self, root: str) -> list[str]:
|
||||||
|
"""Return all file paths (relative to root) under root."""
|
||||||
|
body = (
|
||||||
|
'<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:prop>'
|
||||||
|
"<d:resourcetype/><d:getcontentlength/></d:prop></d:propfind>"
|
||||||
|
)
|
||||||
|
out: list[str] = []
|
||||||
|
for depth in ("infinity", "1"):
|
||||||
|
try:
|
||||||
|
out = self._propfind(root, body, depth)
|
||||||
|
if out:
|
||||||
|
return out
|
||||||
|
except requests.HTTPError as e:
|
||||||
|
if depth == "infinity" and e.response is not None and e.response.status_code in (403, 405, 507):
|
||||||
|
continue # server forbids Depth: infinity -> recurse manually
|
||||||
|
raise
|
||||||
|
return out
|
||||||
|
|
||||||
|
def _propfind(self, root: str, body: str, depth: str) -> list[str]:
|
||||||
|
r = self.s.request(
|
||||||
|
"PROPFIND", self._url(root),
|
||||||
|
headers={"Depth": depth, "Content-Type": "application/xml"}, data=body,
|
||||||
|
)
|
||||||
|
r.raise_for_status()
|
||||||
|
tree = ET.fromstring(r.content)
|
||||||
|
base_path = unquote(self._url(root).split(self.base.split("/remote.php")[0], 1)[-1])
|
||||||
|
files, dirs = [], []
|
||||||
|
for resp in tree.findall(f"{DAV_NS}response"):
|
||||||
|
href = unquote(resp.findtext(f"{DAV_NS}href") or "")
|
||||||
|
if not href or href.rstrip("/") == base_path.rstrip("/"):
|
||||||
|
continue
|
||||||
|
rel = href.split(base_path, 1)[-1].strip("/") if base_path in href else href
|
||||||
|
is_dir = resp.find(f".//{DAV_NS}collection") is not None
|
||||||
|
(dirs if is_dir else files).append(rel)
|
||||||
|
if depth == "1":
|
||||||
|
for d in dirs:
|
||||||
|
files += [f"{d}/{f}" for f in self._propfind(f"{root}/{d}", body, "1")]
|
||||||
|
return files
|
||||||
|
|
||||||
|
def get(self, path: str) -> str:
|
||||||
|
r = self.s.get(self._url(path).rstrip("/"))
|
||||||
|
r.raise_for_status()
|
||||||
|
return r.content.decode("utf-8", "replace")
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------- parsing ----
|
||||||
|
|
||||||
|
FIELDS = ("Category", "Difficulty", "Event", "Author", "Tester", "Status")
|
||||||
|
HEADING_RE = re.compile(r"^\s{0,3}#{2,4}\s+(.+?)\s*$", re.M)
|
||||||
|
FIELD_RE = re.compile(r"^\s*[-*]\s*\**(?P<key>[A-Za-z ]+?)\**\s*:\s*(?P<val>.*?)\s*$", re.M)
|
||||||
|
SECTION_RE = re.compile(r"^\s{0,3}#{3,5}\s*(?P<name>.+?)\s*$\n(?P<body>.*?)(?=^\s{0,3}#{2,5}\s|\Z)",
|
||||||
|
re.M | re.S)
|
||||||
|
# a value the author never touched: "[crypto, pwn, ...]" or "[Author Name]"
|
||||||
|
PLACEHOLDER_RE = re.compile(r"^\\?\[.*\\?\]$")
|
||||||
|
PROSE_PROMPTS = (
|
||||||
|
"what should the player take home", "please answer in 1-2",
|
||||||
|
"how do you plan on implementing", "needed for infrastructure",
|
||||||
|
"how will the players exploit",
|
||||||
|
)
|
||||||
|
STATUS_ORDER = ["Planning", "PoC Draft", "Testing Ready", "Tested", "Ready"]
|
||||||
|
CATEGORIES = ["crypto", "pwn", "web", "rev", "misc", "osint", "forensic", "hardware"]
|
||||||
|
DIFFICULTIES = ["intro", "easy", "medium", "hard"]
|
||||||
|
|
||||||
|
|
||||||
|
def clean(v: str) -> str:
|
||||||
|
return v.replace("\\[", "[").replace("\\]", "]").strip()
|
||||||
|
|
||||||
|
|
||||||
|
def is_placeholder(v: str) -> bool:
|
||||||
|
v = clean(v)
|
||||||
|
return not v or bool(PLACEHOLDER_RE.match(v))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Challenge:
|
||||||
|
author_dir: str
|
||||||
|
path: str
|
||||||
|
name: str = ""
|
||||||
|
fields: dict = field(default_factory=dict)
|
||||||
|
sections: dict = field(default_factory=dict)
|
||||||
|
in_template: bool = False
|
||||||
|
|
||||||
|
# -- derived -----------------------------------------------------------
|
||||||
|
@property
|
||||||
|
def missing_fields(self) -> list[str]:
|
||||||
|
return [f for f in FIELDS if is_placeholder(self.fields.get(f, ""))]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def missing_sections(self) -> list[str]:
|
||||||
|
return [k for k, v in self.sections.items() if not v]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def named(self) -> bool:
|
||||||
|
return bool(self.name) and not is_placeholder(self.name)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def touched(self) -> bool:
|
||||||
|
"""Did anyone put *any* real content in here?"""
|
||||||
|
return self.named or len(self.missing_fields) < len(FIELDS) or bool(
|
||||||
|
[k for k, v in self.sections.items() if v])
|
||||||
|
|
||||||
|
@property
|
||||||
|
def complete(self) -> bool:
|
||||||
|
return self.named and not self.missing_fields and not self.missing_sections
|
||||||
|
|
||||||
|
def get(self, key: str, dash: str = "—") -> str:
|
||||||
|
v = clean(self.fields.get(key, ""))
|
||||||
|
return dash if is_placeholder(v) else v
|
||||||
|
|
||||||
|
@property
|
||||||
|
def status(self) -> str:
|
||||||
|
return self.get("Status", "—")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def problems(self) -> list[str]:
|
||||||
|
p = []
|
||||||
|
if self.in_template and self.touched:
|
||||||
|
p.append("filled out template.md in place (rename it!)")
|
||||||
|
if not self.named:
|
||||||
|
p.append("no challenge name")
|
||||||
|
if self.missing_fields:
|
||||||
|
p.append("unfilled: " + ", ".join(self.missing_fields))
|
||||||
|
if self.missing_sections:
|
||||||
|
p.append("empty section(s): " + ", ".join(self.missing_sections))
|
||||||
|
cat = self.get("Category", "").lower()
|
||||||
|
if cat and cat != "—" and cat not in CATEGORIES and "?" not in cat:
|
||||||
|
p.append(f"odd category {cat!r}")
|
||||||
|
diff = self.get("Difficulty", "").lower()
|
||||||
|
if diff and diff != "—" and diff not in DIFFICULTIES:
|
||||||
|
p.append(f"odd difficulty {diff!r}")
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def parse_challenge(author: str, path: str, text: str, in_template: bool) -> Challenge:
|
||||||
|
c = Challenge(author_dir=author, path=path, in_template=in_template)
|
||||||
|
m = HEADING_RE.search(text)
|
||||||
|
if m:
|
||||||
|
c.name = clean(m.group(1))
|
||||||
|
for fm in FIELD_RE.finditer(text):
|
||||||
|
key = fm.group("key").strip().title()
|
||||||
|
if key in FIELDS and key not in c.fields:
|
||||||
|
c.fields[key] = fm.group("val")
|
||||||
|
for sm in SECTION_RE.finditer(text):
|
||||||
|
name = clean(sm.group("name"))
|
||||||
|
if name == c.name or name in FIELDS:
|
||||||
|
continue
|
||||||
|
body = sm.group("body")
|
||||||
|
# strip the template's own prompt lines, keep what the author wrote
|
||||||
|
real = [ln.strip() for ln in body.splitlines()
|
||||||
|
if ln.strip() and not ln.strip().startswith(("-", "*", "#"))
|
||||||
|
and not any(p in ln.lower() for p in PROSE_PROMPTS)]
|
||||||
|
c.sections[name] = " ".join(real)
|
||||||
|
return c
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- report ----
|
||||||
|
|
||||||
|
BOLD, DIM, RED, YEL, GRN, CYA, RST = (
|
||||||
|
"\033[1m", "\033[2m", "\033[31m", "\033[33m", "\033[32m", "\033[36m", "\033[0m")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Author:
|
||||||
|
name: str
|
||||||
|
challenges: list = field(default_factory=list)
|
||||||
|
files_seen: list = field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def real(self) -> list:
|
||||||
|
return [c for c in self.challenges if c.touched]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def verdict(self) -> tuple[str, str]:
|
||||||
|
if not self.real:
|
||||||
|
return "SCREAM", "nothing submitted at all"
|
||||||
|
if any(c.in_template for c in self.real):
|
||||||
|
return "SCREAM", "worked inside template.md instead of copying it"
|
||||||
|
bad = [c for c in self.real if not c.complete]
|
||||||
|
if bad:
|
||||||
|
return "NAG", f"{len(bad)}/{len(self.real)} challenge(s) incomplete"
|
||||||
|
return "OK", f"{len(self.real)} challenge(s) done"
|
||||||
|
|
||||||
|
|
||||||
|
def collect(dav: Dav, root: str, no_color: bool) -> dict:
|
||||||
|
paths = [p for p in dav.walk(root) if p.lower().endswith(".md")]
|
||||||
|
authors: dict[str, Author] = {}
|
||||||
|
for p in sorted(paths):
|
||||||
|
parts = p.split("/")
|
||||||
|
if len(parts) < 2:
|
||||||
|
continue # loose file in the root, not an author's
|
||||||
|
adir, fname = parts[0], parts[-1]
|
||||||
|
a = authors.setdefault(adir, Author(adir))
|
||||||
|
a.files_seen.append(fname)
|
||||||
|
if fname.lower() == "readme.md":
|
||||||
|
continue
|
||||||
|
text = dav.get(f"{root}/{p}")
|
||||||
|
a.challenges.append(
|
||||||
|
parse_challenge(adir, p, text, in_template=fname.lower() == "template.md"))
|
||||||
|
# authors with a folder but zero md files still deserve screaming at
|
||||||
|
for p in dav.walk(root):
|
||||||
|
top = p.split("/")[0]
|
||||||
|
if "/" in p:
|
||||||
|
authors.setdefault(top, Author(top))
|
||||||
|
return authors
|
||||||
|
|
||||||
|
|
||||||
|
def print_terminal(authors: dict, c: bool) -> None:
|
||||||
|
def col(x): return x if c else ""
|
||||||
|
order = {"SCREAM": 0, "NAG": 1, "OK": 2}
|
||||||
|
rows = sorted(authors.values(), key=lambda a: (order[a.verdict[0]], a.name.lower()))
|
||||||
|
print(f"\n{col(BOLD)}=== authors ({len(rows)}) ==={col(RST)}")
|
||||||
|
for a in rows:
|
||||||
|
v, why = a.verdict
|
||||||
|
tint = {"SCREAM": RED, "NAG": YEL, "OK": GRN}[v]
|
||||||
|
print(f" {col(tint)}{v:<7}{col(RST)} {a.name:<24} {col(DIM)}{why}{col(RST)}")
|
||||||
|
for ch in a.real:
|
||||||
|
for prob in ch.problems:
|
||||||
|
print(f" {col(DIM)}↳ {Path(ch.path).name}: {prob}{col(RST)}")
|
||||||
|
|
||||||
|
chs = [ch for a in authors.values() for ch in a.real]
|
||||||
|
done = sum(1 for ch in chs if ch.complete)
|
||||||
|
print(f"\n{col(BOLD)}=== {len(chs)} challenges, {done} fully filled out ==={col(RST)}")
|
||||||
|
by_cat = Counter(ch.get("Category", "?").lower() for ch in chs)
|
||||||
|
by_diff = Counter(ch.get("Difficulty", "?").lower() for ch in chs)
|
||||||
|
by_stat = Counter(ch.status for ch in chs)
|
||||||
|
for label, ctr, ref in (("category", by_cat, CATEGORIES),
|
||||||
|
("difficulty", by_diff, DIFFICULTIES),
|
||||||
|
("status", by_stat, STATUS_ORDER)):
|
||||||
|
keys = [k for k in ref if ctr.get(k.lower() if label != "status" else k)]
|
||||||
|
keys += [k for k in ctr if k not in keys and k not in [r.lower() for r in ref]]
|
||||||
|
parts = [f"{k}:{ctr[k]}" for k in keys if ctr.get(k)]
|
||||||
|
print(f" {col(CYA)}{label:<11}{col(RST)} " + " ".join(parts))
|
||||||
|
missing = [k for k in CATEGORIES if not by_cat.get(k)]
|
||||||
|
if missing:
|
||||||
|
print(f" {col(YEL)}no coverage {col(RST)}" + ", ".join(missing))
|
||||||
|
print()
|
||||||
|
|
||||||
|
|
||||||
|
def render_hedgedoc(authors: dict, title: str, tags: str = "") -> str:
|
||||||
|
chs = [ch for a in authors.values() for ch in a.real]
|
||||||
|
stamp = datetime.now().astimezone().strftime("%Y-%m-%d %H:%M:%S %Z")
|
||||||
|
L = []
|
||||||
|
if tags:
|
||||||
|
# HedgeDoc reads this YAML block off the top of the note
|
||||||
|
L += ["---", f"tags: {tags}", "---", ""]
|
||||||
|
L += [f"# {title}", "",
|
||||||
|
f"**Last updated:** {stamp}", "",
|
||||||
|
f"*generated by `ctf_status.py` — {len(chs)} challenges from "
|
||||||
|
f"{len({ch.author_dir for ch in chs})} authors*", "",
|
||||||
|
"## Challenges", "",
|
||||||
|
"| Challenge | Category | Difficulty | Event | Author | Tester | Status |",
|
||||||
|
"| --- | --- | --- | --- | --- | --- | --- |"]
|
||||||
|
for ch in sorted(chs, key=lambda c: (c.get("Category", "zz").lower(),
|
||||||
|
DIFFICULTIES.index(c.get("Difficulty", "").lower())
|
||||||
|
if c.get("Difficulty", "").lower() in DIFFICULTIES else 9,
|
||||||
|
c.name.lower())):
|
||||||
|
name = ch.name if ch.named else f"*(unnamed — {Path(ch.path).name})*"
|
||||||
|
flag = "" if ch.complete else " ⚠️"
|
||||||
|
L.append(f"| {name}{flag} | {ch.get('Category')} | {ch.get('Difficulty')} | "
|
||||||
|
f"{ch.get('Event')} | {ch.get('Author') if not is_placeholder(ch.fields.get('Author','')) else ch.author_dir} | "
|
||||||
|
f"{ch.get('Tester')} | {ch.status} |")
|
||||||
|
|
||||||
|
cols = DIFFICULTIES + ["?"]
|
||||||
|
L += ["", "## Coverage (category × difficulty)", "",
|
||||||
|
"| Category | " + " | ".join(cols) + " | total |",
|
||||||
|
"| --- |" + " --- |" * (len(cols) + 1)]
|
||||||
|
grid = defaultdict(Counter)
|
||||||
|
for ch in chs:
|
||||||
|
grid[ch.get("Category", "?").lower()][ch.get("Difficulty", "?").lower()] += 1
|
||||||
|
for cat in CATEGORIES + [k for k in grid if k not in CATEGORIES]:
|
||||||
|
row = grid.get(cat, Counter())
|
||||||
|
cells = [str(row.get(d, 0) or "·") for d in cols]
|
||||||
|
L.append(f"| {cat} | " + " | ".join(cells) + f" | **{sum(row.values())}** |")
|
||||||
|
tot = Counter()
|
||||||
|
for row in grid.values():
|
||||||
|
tot.update(row)
|
||||||
|
L.append("| **total** | " + " | ".join(f"**{tot.get(d, 0)}**" for d in cols)
|
||||||
|
+ f" | **{sum(tot.values())}** |")
|
||||||
|
|
||||||
|
L += ["", "## Author status", "", "| Author | State | Note |", "| --- | --- | --- |"]
|
||||||
|
icon = {"SCREAM": "🔴", "NAG": "🟡", "OK": "🟢"}
|
||||||
|
for a in sorted(authors.values(), key=lambda a: a.name.lower()):
|
||||||
|
v, why = a.verdict
|
||||||
|
L.append(f"| {a.name} | {icon[v]} | {why} |")
|
||||||
|
L += ["", "## Open issues", ""]
|
||||||
|
any_issue = False
|
||||||
|
for a in sorted(authors.values(), key=lambda a: a.name.lower()):
|
||||||
|
items = [(ch, p) for ch in a.real for p in ch.problems]
|
||||||
|
if not items and a.real:
|
||||||
|
continue
|
||||||
|
any_issue = True
|
||||||
|
L.append(f"- **{a.name}**")
|
||||||
|
if not a.real:
|
||||||
|
L.append(" - nothing submitted at all")
|
||||||
|
for ch, p in items:
|
||||||
|
L.append(f" - `{Path(ch.path).name}`: {p}")
|
||||||
|
if not any_issue:
|
||||||
|
L.append("*none — everyone is done. suspicious.*")
|
||||||
|
return "\n".join(L) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------ hedgedoc (write) ----
|
||||||
|
|
||||||
|
# engine.io v3 packet types
|
||||||
|
EIO_OPEN, EIO_CLOSE, EIO_PING, EIO_PONG, EIO_MSG = "0", "1", "2", "3", "4"
|
||||||
|
# socket.io packet types, as the second character of an EIO_MSG packet
|
||||||
|
SIO_CONNECT, SIO_DISCONNECT, SIO_EVENT, SIO_ERROR = "0", "1", "2", "4"
|
||||||
|
|
||||||
|
|
||||||
|
def u16len(s: str) -> int:
|
||||||
|
"""Length in UTF-16 code units — what JavaScript's `String.length` counts, and
|
||||||
|
therefore what engine.io payload framing and OT operations count. Matters as
|
||||||
|
soon as the markdown holds an emoji (🔴 is one Python char but two units)."""
|
||||||
|
return len(s.encode("utf-16-le")) // 2
|
||||||
|
|
||||||
|
|
||||||
|
def encode_payload(packets: tuple) -> bytes:
|
||||||
|
return "".join(f"{u16len(p)}:{p}" for p in packets).encode("utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def decode_payload(text: str) -> list[str]:
|
||||||
|
buf = text.encode("utf-16-le")
|
||||||
|
out, i = [], 0
|
||||||
|
while i < len(buf):
|
||||||
|
j = i
|
||||||
|
while buf[j:j + 2] != b":\x00":
|
||||||
|
j += 2
|
||||||
|
if j >= len(buf):
|
||||||
|
raise RuntimeError("malformed engine.io payload")
|
||||||
|
n = int(buf[i:j].decode("utf-16-le"))
|
||||||
|
j += 2
|
||||||
|
out.append(buf[j:j + 2 * n].decode("utf-16-le"))
|
||||||
|
i = j + 2 * n
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
class HedgeDocNote:
|
||||||
|
"""Overwrite an existing HedgeDoc 1.x note in place.
|
||||||
|
|
||||||
|
HedgeDoc 1.x has no REST endpoint that edits a note's body, so this speaks the
|
||||||
|
same engine.io/socket.io protocol the web editor uses, over XHR long-polling
|
||||||
|
(no extra dependencies beyond `requests`):
|
||||||
|
|
||||||
|
GET /<note> -> session cookie; the socket rejects us without one
|
||||||
|
GET /socket.io/ -> engine.io handshake, yields a session id
|
||||||
|
poll for 'doc' -> the note's current text and OT revision
|
||||||
|
emit 'operation' -> [-len(old), new]: delete everything, insert ours
|
||||||
|
poll for 'ack' -> the server applied it and told the other editors
|
||||||
|
|
||||||
|
The note's permission must allow us to write ("freely" for an anonymous run).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, url: str, timeout: int = 20):
|
||||||
|
parts = urlsplit(url)
|
||||||
|
path = parts.path.rstrip("/")
|
||||||
|
if not parts.scheme or not parts.netloc or "/" not in path:
|
||||||
|
raise ValueError(f"expected a note URL like https://host/<noteId>, got {url!r}")
|
||||||
|
self.note = path.rsplit("/", 1)[-1]
|
||||||
|
prefix = path[: -len(self.note)].rstrip("/")
|
||||||
|
self.base = f"{parts.scheme}://{parts.netloc}{prefix}"
|
||||||
|
self.timeout = timeout
|
||||||
|
self.sid: str | None = None
|
||||||
|
self.s = requests.Session()
|
||||||
|
self.s.headers["Origin"] = f"{parts.scheme}://{parts.netloc}"
|
||||||
|
|
||||||
|
# -- transport ---------------------------------------------------------
|
||||||
|
def _params(self) -> dict:
|
||||||
|
p = {"EIO": "3", "transport": "polling", "noteId": self.note,
|
||||||
|
"t": format(random.getrandbits(40), "x")}
|
||||||
|
if self.sid:
|
||||||
|
p["sid"] = self.sid
|
||||||
|
return p
|
||||||
|
|
||||||
|
def _open(self) -> None:
|
||||||
|
self.sid = None
|
||||||
|
self.s.get(f"{self.base}/{self.note}", timeout=self.timeout).raise_for_status()
|
||||||
|
r = self.s.get(f"{self.base}/socket.io/", params=self._params(), timeout=self.timeout)
|
||||||
|
r.raise_for_status()
|
||||||
|
pkts = decode_payload(r.text)
|
||||||
|
if not pkts or pkts[0][:1] != EIO_OPEN:
|
||||||
|
raise RuntimeError(f"unexpected engine.io handshake: {pkts[:1]}")
|
||||||
|
self.sid = json.loads(pkts[0][1:])["sid"]
|
||||||
|
|
||||||
|
def _send(self, *packets: str) -> None:
|
||||||
|
r = self.s.post(f"{self.base}/socket.io/", params=self._params(),
|
||||||
|
data=encode_payload(packets),
|
||||||
|
headers={"Content-Type": "text/plain;charset=UTF-8"},
|
||||||
|
timeout=self.timeout)
|
||||||
|
r.raise_for_status()
|
||||||
|
|
||||||
|
def _events(self):
|
||||||
|
"""Poll once, answer keepalives, yield decoded socket.io events."""
|
||||||
|
r = self.s.get(f"{self.base}/socket.io/", params=self._params(), timeout=self.timeout)
|
||||||
|
r.raise_for_status()
|
||||||
|
for p in decode_payload(r.text):
|
||||||
|
if p == EIO_PING:
|
||||||
|
self._send(EIO_PONG)
|
||||||
|
elif p == EIO_CLOSE:
|
||||||
|
raise RuntimeError("hedgedoc closed the connection")
|
||||||
|
elif p[:1] == EIO_MSG:
|
||||||
|
kind, data = p[1:2], p[2:]
|
||||||
|
if kind == SIO_ERROR:
|
||||||
|
raise RuntimeError(f"hedgedoc refused the socket: {data}")
|
||||||
|
if kind == SIO_EVENT:
|
||||||
|
yield json.loads(data)
|
||||||
|
|
||||||
|
def _wait(self, name: str, polls: int = 8):
|
||||||
|
for _ in range(polls):
|
||||||
|
for ev in self._events():
|
||||||
|
if ev and ev[0] == name:
|
||||||
|
return ev[1] if len(ev) > 1 else None
|
||||||
|
raise RuntimeError(f"timed out waiting for {name!r} from hedgedoc")
|
||||||
|
|
||||||
|
def _close(self) -> None:
|
||||||
|
try:
|
||||||
|
self._send(EIO_MSG + SIO_DISCONNECT, EIO_CLOSE)
|
||||||
|
except requests.RequestException:
|
||||||
|
pass # we already got our ack; a tidy goodbye is a nicety
|
||||||
|
|
||||||
|
# -- api ---------------------------------------------------------------
|
||||||
|
def download(self) -> str:
|
||||||
|
r = self.s.get(f"{self.base}/{self.note}/download", timeout=self.timeout)
|
||||||
|
r.raise_for_status()
|
||||||
|
return r.content.decode("utf-8", "replace")
|
||||||
|
|
||||||
|
def replace(self, markdown: str) -> bool:
|
||||||
|
"""Replace the note body. False if it already said exactly this."""
|
||||||
|
self._open()
|
||||||
|
doc = self._wait("doc")
|
||||||
|
old, revision = doc["str"], doc["revision"]
|
||||||
|
if old == markdown:
|
||||||
|
self._close()
|
||||||
|
return False
|
||||||
|
# an OT operation is a list of: n>0 keep n chars, n<0 drop n, str insert
|
||||||
|
ops = ([-u16len(old)] if old else []) + ([markdown] if markdown else [])
|
||||||
|
self._send(EIO_MSG + SIO_EVENT
|
||||||
|
+ json.dumps(["operation", revision, ops, None], ensure_ascii=False))
|
||||||
|
self._wait("ack")
|
||||||
|
time.sleep(2) # give the realtime server a moment to flush the note to its DB
|
||||||
|
self._close()
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def update_hedgedoc(url: str, md: str, retries: int = 3, quiet: bool = False) -> bool:
|
||||||
|
"""Write `md` into the existing note at `url`, then read it back to be sure."""
|
||||||
|
for attempt in range(1, retries + 1):
|
||||||
|
try:
|
||||||
|
note = HedgeDocNote(url)
|
||||||
|
changed = note.replace(md)
|
||||||
|
if note.download() != md:
|
||||||
|
raise RuntimeError("note does not match what we wrote")
|
||||||
|
return changed
|
||||||
|
except (requests.RequestException, RuntimeError, ValueError, KeyError) as e:
|
||||||
|
if attempt == retries:
|
||||||
|
raise
|
||||||
|
if not quiet:
|
||||||
|
print(f"hedgedoc update failed ({e}) — retry {attempt + 1}/{retries}",
|
||||||
|
file=sys.stderr)
|
||||||
|
time.sleep(3 * attempt)
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def push_hedgedoc(base: str, md: str) -> str:
|
||||||
|
r = requests.post(base.rstrip("/") + "/new", data=md.encode(),
|
||||||
|
headers={"Content-Type": "text/markdown"}, allow_redirects=False)
|
||||||
|
r.raise_for_status()
|
||||||
|
loc = r.headers.get("Location", "")
|
||||||
|
return loc if loc.startswith("http") else base.rstrip("/") + loc
|
||||||
|
|
||||||
|
|
||||||
|
def single_run_lock():
|
||||||
|
"""Keep a slow or hung cron run from colliding with the next one."""
|
||||||
|
fh = open(Path(tempfile.gettempdir()) / "ctf_status.lock", "w")
|
||||||
|
try:
|
||||||
|
fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||||
|
except OSError:
|
||||||
|
sys.exit("another ctf_status.py run is still going; skipping this one")
|
||||||
|
return fh # held until the process exits
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__,
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
ap.add_argument("-o", "--out", help="write the HedgeDoc markdown to this file")
|
||||||
|
ap.add_argument("--stdout", action="store_true", help="print the markdown instead of the summary")
|
||||||
|
ap.add_argument("--hedgedoc", metavar="URL",
|
||||||
|
help="POST the markdown to this HedgeDoc instance as a *new* note "
|
||||||
|
"(needs an instance that lets you create notes anonymously)")
|
||||||
|
ap.add_argument("--update", metavar="NOTE_URL", nargs="?", const="",
|
||||||
|
help="overwrite an existing HedgeDoc note in place; defaults to "
|
||||||
|
"HEDGEDOC_NOTE from the config. This is the one to cron.")
|
||||||
|
ap.add_argument("--retries", type=int, default=3,
|
||||||
|
help="attempts at the HedgeDoc write before giving up (default 3)")
|
||||||
|
ap.add_argument("--tags", help="YAML front-matter tags for the note "
|
||||||
|
f"(default {DEFAULT_TAGS!r}, or HEDGEDOC_TAGS)")
|
||||||
|
ap.add_argument("--title", help=f"note heading (default {DEFAULT_TITLE!r}, or TITLE)")
|
||||||
|
ap.add_argument("-q", "--quiet", action="store_true",
|
||||||
|
help="say nothing unless something changed or broke (for cron)")
|
||||||
|
ap.add_argument("--no-color", action="store_true")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
cfg = load_env()
|
||||||
|
title = args.title or cfg.get("TITLE") or DEFAULT_TITLE
|
||||||
|
tags = args.tags if args.tags is not None else cfg.get("HEDGEDOC_TAGS", DEFAULT_TAGS)
|
||||||
|
|
||||||
|
note_url = None
|
||||||
|
if args.update is not None:
|
||||||
|
note_url = args.update or cfg.get("HEDGEDOC_NOTE", "")
|
||||||
|
if not note_url:
|
||||||
|
sys.exit("--update needs a note URL, or HEDGEDOC_NOTE in the config")
|
||||||
|
try:
|
||||||
|
HedgeDocNote(note_url) # fail on a typo now, not after the Nextcloud walk
|
||||||
|
except ValueError as e:
|
||||||
|
sys.exit(str(e))
|
||||||
|
lock = single_run_lock() # noqa: F841 (held for the lifetime of the process)
|
||||||
|
|
||||||
|
dav = Dav(cfg["NC_URL"], cfg["NC_USER"], cfg["NC_PASS"])
|
||||||
|
authors = collect(dav, cfg["NC_ROOT"].strip("/"), args.no_color)
|
||||||
|
if not authors:
|
||||||
|
sys.exit(f"no author folders found under {cfg['NC_ROOT']!r}")
|
||||||
|
|
||||||
|
md = render_hedgedoc(authors, title, tags)
|
||||||
|
if args.out:
|
||||||
|
Path(args.out).write_text(md)
|
||||||
|
if args.stdout:
|
||||||
|
print(md, end="")
|
||||||
|
elif not args.quiet:
|
||||||
|
print_terminal(authors, c=not args.no_color and sys.stdout.isatty())
|
||||||
|
if args.out:
|
||||||
|
print(f"markdown → {args.out}")
|
||||||
|
if args.hedgedoc:
|
||||||
|
print("hedgedoc → " + push_hedgedoc(args.hedgedoc, md))
|
||||||
|
if note_url:
|
||||||
|
try:
|
||||||
|
changed = update_hedgedoc(note_url, md, retries=args.retries, quiet=args.quiet)
|
||||||
|
except (requests.RequestException, RuntimeError, ValueError, KeyError) as e:
|
||||||
|
sys.exit(f"hedgedoc update of {note_url} failed: {e}")
|
||||||
|
if changed or not args.quiet:
|
||||||
|
print(("updated → " if changed else "unchanged → ") + note_url)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Executable
+35
@@ -0,0 +1,35 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
# Install the 30-minute HedgeDoc refresh as systemd *user* units.
|
||||||
|
# Run it as the user that owns ctf_status.env, on the machine that should do the
|
||||||
|
# polling. Re-running it is safe; it just overwrites the units.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd -P)
|
||||||
|
units="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user"
|
||||||
|
|
||||||
|
[ -x "$dir/ctf_status.py" ] || { echo "no executable ctf_status.py in $dir" >&2; exit 1; }
|
||||||
|
[ -f "$dir/ctf_status.env" ] || {
|
||||||
|
echo "no $dir/ctf_status.env — copy ctf_status.env.example to it first" >&2; exit 1; }
|
||||||
|
grep -q '^HEDGEDOC_NOTE=..*' "$dir/ctf_status.env" || {
|
||||||
|
echo "set HEDGEDOC_NOTE=<note url> in $dir/ctf_status.env first" >&2; exit 1; }
|
||||||
|
chmod 600 "$dir/ctf_status.env"
|
||||||
|
|
||||||
|
mkdir -p "$units"
|
||||||
|
for u in ctf-status.service ctf-status.timer; do
|
||||||
|
sed "s|__DIR__|$dir|g" "$dir/$u" > "$units/$u"
|
||||||
|
echo "installed $units/$u"
|
||||||
|
done
|
||||||
|
|
||||||
|
systemctl --user daemon-reload
|
||||||
|
systemctl --user enable --now ctf-status.timer
|
||||||
|
|
||||||
|
# without lingering, user units are torn down as soon as you log out
|
||||||
|
if ! loginctl show-user "$(id -un)" -p Linger --value 2>/dev/null | grep -qx yes; then
|
||||||
|
loginctl enable-linger "$(id -un)" 2>/dev/null \
|
||||||
|
|| echo "could not enable linger; run: sudo loginctl enable-linger $(id -un)" >&2
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "one run now, to prove it works:"
|
||||||
|
systemctl --user start ctf-status.service && echo " ok"
|
||||||
|
systemctl --user list-timers ctf-status.timer --no-pager
|
||||||
Reference in New Issue
Block a user