Postgraduate research project · PPKE-ITK
A local, GPU-accelerated vision pipeline that reconstructs the 3D geometry of an ordinary single-camera video and inserts synthetic elements so they sit convincingly in the scene.
Capstone research project · Applied Artificial Intelligence postgraduate specialisation programme · Faculty of Information Technology and Bionics, Pázmány Péter Catholic University (PPKE-ITK)
Abstract
Convincingly adding a synthetic element to existing video is hard.
Capability map
Each domain is a substantial computer-vision problem in its own right.
Temporally consistent per-pixel depth from a single moving camera.
Recovering camera motion, intrinsics and sparse 3D structure.
Video-consistent masks that decide what occludes what.
Diffusion-based image and motion generation, scene-conditioned.
Shadows, matte, colour, grain and blur combined in linear light.
A queue-backed worker system running reproducible, resumable jobs.
Under the hood
A diagnostic representation from an intermediate stage — not the final composite.
Engineering principles
Beyond the models, the project is an exercise in disciplined systems engineering.
Only permissively-licensed weights (Apache-2.0 / MIT / BSD) ever enter the pipeline.
Deterministic manifests record pinned model versions and fixed seeds.
Large models are hot-swapped in and out of VRAM to fit one consumer GPU.
All compositing happens in 32-bit float linear light; gamma only at the edges.
The original footage and audio are preserved; the result is additive.
Runs entirely on local hardware — no frames sent to any third-party API.
Technology
A GPU compute core in Python, wrapped in a typed job-orchestration layer.
This page presents the capabilities and engineering approach of the project.