Automated Sop From Screen Capture — Free, On Your Own Machine, Nothing Uploaded (2026)

Capture & Structure Pipeline

Use ffmpeg with -f x11grab (Linux) or -f gdigrab (Windows) to record at 10 fps, 720p — sufficient for UI text recognition. Name files YYYYMMDD_HHMMSS_taskname.mp4. Feed frames to tesseract via python -m pytesseract with --psm 6 for block detection. Filter OCR output: discard lines < 3 chars, keep only frames where > 15% pixels change (scene detection via scenedetect CLI). Pipe structured JSON: {"timestamp": "00:00:12", "action": "click", "target": "Save button", "context": "Invoice form"}. Use ydotool (Wayland) or AutoHotkey (Windows) to replay and verify each step; log mismatches to review_queue.jsonl.

Local LLM Assembly & Export

Run ollama run llama3.2:3b or phi3:mini (2.7GB) — both fit 8GB RAM. Prompt: "Convert this JSON action log into a numbered SOP with prerequisites, steps, expected outcomes, and rollback notes. Mark ambiguous steps with [REVIEW]." Feed 50-action chunks via stdin. Post-process: replace coordinates with semantic selectors (data-testid, aria-label) using a local DOM snapshot from playwright headless run. Export to Markdown + Mermaid flowchart (graph TD) and PDF via pandoc --pdf-engine=weasyprint. Store in git repo with pre-commit hook running vale for style consistency. Schedule nightly re-run via systemd timer; diff against last version, alert on > 20% step change.