68 lines
2.5 KiB
Markdown
68 lines
2.5 KiB
Markdown
|
# Media Ingest
|
||
|
|
|
||
|
|
Media Ingest is a private, self-hosted tool for inspecting and downloading
|
||
|
|
publicly available video for a broadcast-news workflow. It uses yt-dlp for
|
||
|
|
source extraction and FFmpeg for media inspection, stream merging, and
|
||
|
|
conversion when required.
|
||
|
|
|
||
|
|
The project is being developed locally with Docker first. Once the local
|
||
|
|
workflow is reliable and hardened, it will be deployed to an Ubuntu VPS behind
|
||
|
|
Cloudflare Access and Cloudflare Tunnel.
|
||
|
|
|
||
|
|
## Initial scope
|
||
|
|
|
||
|
|
The first useful version will support:
|
||
|
|
|
||
|
|
- one public, non-live video URL per job
|
||
|
|
- source and format inspection before downloading
|
||
|
|
- an original-quality download mode
|
||
|
|
- a compatible H.264/AAC MP4 mode when the source provides suitable streams
|
||
|
|
- background jobs with progress reporting
|
||
|
|
- source-attribution metadata stored alongside the downloaded media
|
||
|
|
- automatic cleanup of expired files
|
||
|
|
|
||
|
|
The initial version will not support browser cookies, authenticated source
|
||
|
|
accounts, playlists, live recording, or multiple simultaneous downloads.
|
||
|
|
|
||
|
|
## Planned architecture
|
||
|
|
|
||
|
|
- FastAPI web application
|
||
|
|
- server-rendered HTML with a small amount of JavaScript
|
||
|
|
- yt-dlp embedded through its Python API
|
||
|
|
- FFmpeg and ffprobe
|
||
|
|
- SQLite for durable job state
|
||
|
|
- a separate worker process for downloads
|
||
|
|
- Docker Compose for local and production deployments
|
||
|
|
|
||
|
|
## Development stages
|
||
|
|
|
||
|
|
1. Create a minimal containerized web application with a health check.
|
||
|
|
2. Inspect a URL and display sanitized source metadata and formats.
|
||
|
|
3. Add a durable job queue and separate worker.
|
||
|
|
4. Download media and report progress.
|
||
|
|
5. Add original-quality and compatible-MP4 output modes.
|
||
|
|
6. Add cleanup, limits, failure handling, and security tests.
|
||
|
|
7. Validate the complete local Docker deployment.
|
||
|
|
8. Deploy privately to the VPS through Cloudflare Access and Tunnel.
|
||
|
|
|
||
|
|
Each stage should produce a small, working, reviewable commit.
|
||
|
|
|
||
|
|
## Local development
|
||
|
|
|
||
|
|
Local setup instructions will be added with the first application container.
|
||
|
|
Runtime data, downloaded media, local environment files, and secrets must not
|
||
|
|
be committed to Git.
|
||
|
|
|
||
|
|
## Documentation
|
||
|
|
|
||
|
|
The original project discussion is retained in
|
||
|
|
[`docs/general-idea.pdf`](docs/general-idea.pdf) as planning context. Decisions
|
||
|
|
that affect the implementation will be documented in this repository rather
|
||
|
|
than relying on that discussion alone.
|
||
|
|
|
||
|
|
## Responsible use
|
||
|
|
|
||
|
|
This tool is intended for material the operator is authorized to retrieve and
|
||
|
|
use. The operator remains responsible for source terms, copyright, licensing,
|
||
|
|
privacy, and appropriate attribution.
|