chore: initialize media ingest project
This commit is contained in:
@@ -0,0 +1,14 @@
|
|||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
.venv
|
||||||
|
__pycache__
|
||||||
|
.pytest_cache
|
||||||
|
.ruff_cache
|
||||||
|
.mypy_cache
|
||||||
|
data
|
||||||
|
downloads
|
||||||
|
docs
|
||||||
|
tests
|
||||||
|
*.pyc
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
root = true
|
||||||
|
|
||||||
|
[*]
|
||||||
|
charset = utf-8
|
||||||
|
end_of_line = lf
|
||||||
|
insert_final_newline = true
|
||||||
|
indent_style = space
|
||||||
|
indent_size = 4
|
||||||
|
trim_trailing_whitespace = true
|
||||||
|
|
||||||
|
[*.{yaml,yml,json,html,css}]
|
||||||
|
indent_size = 2
|
||||||
|
|
||||||
|
[*.md]
|
||||||
|
trim_trailing_whitespace = false
|
||||||
+30
@@ -0,0 +1,30 @@
|
|||||||
|
# Local environment and secrets
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
.pytest_cache/
|
||||||
|
.ruff_cache/
|
||||||
|
.mypy_cache/
|
||||||
|
.venv/
|
||||||
|
|
||||||
|
# Runtime data and downloaded media
|
||||||
|
data/*
|
||||||
|
!data/.gitkeep
|
||||||
|
downloads/
|
||||||
|
|
||||||
|
# Coverage and build output
|
||||||
|
.coverage
|
||||||
|
htmlcov/
|
||||||
|
build/
|
||||||
|
dist/
|
||||||
|
*.egg-info/
|
||||||
|
|
||||||
|
# Editors and operating systems
|
||||||
|
.idea/
|
||||||
|
.vscode/
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
# Media Ingest
|
||||||
|
|
||||||
|
Media Ingest is a private, self-hosted tool for inspecting and downloading
|
||||||
|
publicly available video for a broadcast-news workflow. It uses yt-dlp for
|
||||||
|
source extraction and FFmpeg for media inspection, stream merging, and
|
||||||
|
conversion when required.
|
||||||
|
|
||||||
|
The project is being developed locally with Docker first. Once the local
|
||||||
|
workflow is reliable and hardened, it will be deployed to an Ubuntu VPS behind
|
||||||
|
Cloudflare Access and Cloudflare Tunnel.
|
||||||
|
|
||||||
|
## Initial scope
|
||||||
|
|
||||||
|
The first useful version will support:
|
||||||
|
|
||||||
|
- one public, non-live video URL per job
|
||||||
|
- source and format inspection before downloading
|
||||||
|
- an original-quality download mode
|
||||||
|
- a compatible H.264/AAC MP4 mode when the source provides suitable streams
|
||||||
|
- background jobs with progress reporting
|
||||||
|
- source-attribution metadata stored alongside the downloaded media
|
||||||
|
- automatic cleanup of expired files
|
||||||
|
|
||||||
|
The initial version will not support browser cookies, authenticated source
|
||||||
|
accounts, playlists, live recording, or multiple simultaneous downloads.
|
||||||
|
|
||||||
|
## Planned architecture
|
||||||
|
|
||||||
|
- FastAPI web application
|
||||||
|
- server-rendered HTML with a small amount of JavaScript
|
||||||
|
- yt-dlp embedded through its Python API
|
||||||
|
- FFmpeg and ffprobe
|
||||||
|
- SQLite for durable job state
|
||||||
|
- a separate worker process for downloads
|
||||||
|
- Docker Compose for local and production deployments
|
||||||
|
|
||||||
|
## Development stages
|
||||||
|
|
||||||
|
1. Create a minimal containerized web application with a health check.
|
||||||
|
2. Inspect a URL and display sanitized source metadata and formats.
|
||||||
|
3. Add a durable job queue and separate worker.
|
||||||
|
4. Download media and report progress.
|
||||||
|
5. Add original-quality and compatible-MP4 output modes.
|
||||||
|
6. Add cleanup, limits, failure handling, and security tests.
|
||||||
|
7. Validate the complete local Docker deployment.
|
||||||
|
8. Deploy privately to the VPS through Cloudflare Access and Tunnel.
|
||||||
|
|
||||||
|
Each stage should produce a small, working, reviewable commit.
|
||||||
|
|
||||||
|
## Local development
|
||||||
|
|
||||||
|
Local setup instructions will be added with the first application container.
|
||||||
|
Runtime data, downloaded media, local environment files, and secrets must not
|
||||||
|
be committed to Git.
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
The original project discussion is retained in
|
||||||
|
[`docs/general-idea.pdf`](docs/general-idea.pdf) as planning context. Decisions
|
||||||
|
that affect the implementation will be documented in this repository rather
|
||||||
|
than relying on that discussion alone.
|
||||||
|
|
||||||
|
## Responsible use
|
||||||
|
|
||||||
|
This tool is intended for material the operator is authorized to retrieve and
|
||||||
|
use. The operator remains responsible for source terms, copyright, licensing,
|
||||||
|
privacy, and appropriate attribution.
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
|
||||||
Binary file not shown.
Reference in New Issue
Block a user