Media Ingest

Media Ingest is a private, self-hosted tool for inspecting and downloading publicly available video for a broadcast-news workflow. It uses yt-dlp for source extraction and FFmpeg for media inspection, stream merging, and conversion when required.

The project is being developed locally with Docker first. Once the local workflow is reliable and hardened, it will be deployed to an Ubuntu VPS behind Cloudflare Access and Cloudflare Tunnel.

Initial scope

The first useful version will support:

one public, non-live video URL per job
source and format inspection before downloading
an original-quality download mode
a compatible H.264/AAC MP4 mode when the source provides suitable streams
background jobs with progress reporting
source-attribution metadata stored alongside the downloaded media
automatic cleanup of expired files

The initial version will not support browser cookies, authenticated source accounts, playlists, live recording, or multiple simultaneous downloads.

Planned architecture

FastAPI web application
server-rendered HTML with a small amount of JavaScript
yt-dlp embedded through its Python API
FFmpeg and ffprobe
SQLite for durable job state
a separate worker process for downloads
Docker Compose for local and production deployments

Development stages

Create a minimal containerized web application with a health check.
Inspect a URL and display sanitized source metadata and formats.
Add a durable job queue and separate worker.
Download media and report progress.
Add original-quality and compatible-MP4 output modes.
Add cleanup, limits, failure handling, and security tests.
Validate the complete local Docker deployment.
Deploy privately to the VPS through Cloudflare Access and Tunnel.

Each stage should produce a small, working, reviewable commit.

Local development

Local setup instructions will be added with the first application container. Runtime data, downloaded media, local environment files, and secrets must not be committed to Git.

Documentation

The original project discussion is retained in docs/general-idea.pdf as planning context. Decisions that affect the implementation will be documented in this repository rather than relying on that discussion alone.

Responsible use

This tool is intended for material the operator is authorized to retrieve and use. The operator remains responsible for source terms, copyright, licensing, privacy, and appropriate attribution.

2.5 KiB Raw Blame History