Video To Work Instruction Document — Free, On Your Own Machine, Nothing Uploaded (2026)

Local Toolchain & Model Selection

Install ffmpeg (video splitting), whisper.cpp (ASR), and ollama (LLM) via package manager. Use whisper.cpp with ggml-large-v3-turbo.bin for 30x realtime transcription on Apple Silicon or modern x86 GPUs. For frame extraction, run ffmpeg -i input.mp4 -vf "select=gt(scene\,0.3),scale=1280:-1" -vsync vfr frames/frame_%04d.jpg to capture only visually distinct frames, reducing OCR load by 80%.

Pull llama3.2-vision:11b and qwen2.5:14b-instruct via Ollama. The vision model analyzes frames for UI elements, gestures, and screen states; the text model structures transcripts into numbered steps. Keep models quantized to Q4_K_M (6–9 GB VRAM) for 16 GB unified memory machines. Disable telemetry: OLLAMA_TELEMETRY=false.

Create a project directory with /input, /frames, /transcripts, /analysis, /output. Write a Makefile or justfile to chain commands reproducibly. Version-control the pipeline with Git; commit frames/ and transcripts/ as artifacts for auditability.

Pipeline Execution & Output Hardening

Transcribe: whisper.cpp -m models/ggml-large-v3-turbo.bin -f input/audio.wav -otxt -osrt -ovtt --language en. Feed SRT + frames to vision model via structured prompt: "Extract every discrete action, UI label, keyboard shortcut, and decision point. Output JSON: {step, timestamp