diff --git a/README.md b/README.md index fe7c12e..1de74ba 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,10 @@ Telegram bot that monitors cybersecurity RSS feeds and ransomware.live victim da ``` rss_telegram_bot/ ├── threat_intel_bot.py # Main bot — commands, subscriptions, alert dispatch -├── rss_manager.py # Feed fetching, dedup, message formatting -├── content_classifier.py # CVE/actor/malware extraction -├── ransomware_fetcher.py # ransomware.live PRO API — victim feed +├── rss_manager.py # Feed fetching, dedup (incl. cross-source), message formatting +├── content_classifier.py # CVE/actor/malware/MITRE extraction +├── ransomware_fetcher.py # ransomware.live PRO API — victim feed, LV/EE/LT only +├── bot_config.py # Shared category, feed-file, and tuning config ├── check_feeds.py # CLI utility to check feed health ├── feeds/ │ ├── news_feeds.json @@ -22,14 +23,16 @@ rss_telegram_bot/ │ └── review.py # pretty-print and filter results.jsonl ├── .env # not committed — see Setup ├── subscribers.json # auto-managed — chat/topic subscriptions -├── seen_articles.db # SQLite — tracks sent RSS articles (7-day retention) -└── seen_victims.db # SQLite — tracks sent ransomware victims (7-day retention) +├── seen_articles.db # SQLite — tracks sent RSS articles + titles (14-day retention) +└── seen_victims.db # SQLite — tracks sent ransomware victims (14-day retention) ``` ## Setup ```bash -pip install python-telegram-bot feedparser aiohttp beautifulsoup4 +python3 -m venv venv +source venv/bin/activate +pip install -r requirements.txt ``` Create `.env`: @@ -54,7 +57,7 @@ python3 threat_intel_bot.py | `/stats` | Check feed health and subscriber count | | `/help` | Show commands | -Categories: `news`, `malware`, `threat_intel`, `osint`, `research` +Categories: `news`, `malware`, `threat_intel`, `osint`, `research`, `ransomware` **Telegram Topics:** Run `/on_` inside each topic to route categories to separate threads. @@ -75,10 +78,10 @@ Description 🔗 Read Full Article ``` -Ransomware victims (posted to the malware topic): +Ransomware victims (posted to the `ransomware` topic, separate from `malware` so a busy leak day doesn't bury other malware sources): ``` -🦠 company.com claimed by LockBit +💰 company.com claimed by LockBit Sector: Finance | Country: US Data description from leak post @@ -107,7 +110,7 @@ Add or remove feeds by editing the relevant `feeds/*.json` file, then restart th ### ransomware.live PRO API -When `RANSOMWARE_LIVE_API_KEY` is set, the bot polls the PRO API every 5 minutes for newly discovered ransomware victims. New entries are posted to the `malware` topic. Victims are tracked in `seen_victims.db` with 7-day retention. +When `RANSOMWARE_LIVE_API_KEY` is set, the bot polls the PRO API every 5 minutes for newly discovered ransomware victims, filtered to companies in Latvia, Estonia, and Lithuania (`ALLOWED_RANSOMWARE_COUNTRIES` in `bot_config.py`) — the global feed is available directly on ransomware.live. New entries are posted to the `ransomware` topic (subscribe with `/on_ransomware`), kept separate from RSS `malware` articles. Victims are tracked in `seen_victims.db` with 14-day retention. ## Classification @@ -150,10 +153,10 @@ After=network.target [Service] Type=simple -User=root -WorkingDirectory=/root/tele-bots/rss_splited_bot -EnvironmentFile=/root/tele-bots/rss_splited_bot/.env -ExecStart=/root/tele-bots/rss_splited_bot/venv/bin/python3 threat_intel_bot.py +User=ubuntu +WorkingDirectory=/home/ubuntu/rss-tele-bot +EnvironmentFile=/home/ubuntu/rss-tele-bot/.env +ExecStart=/home/ubuntu/rss-tele-bot/venv/bin/python3 threat_intel_bot.py Restart=always RestartSec=10 @@ -167,18 +170,21 @@ sudo systemctl enable --now rss-bot sudo journalctl -u rss-bot -f ``` +Currently deployed and running this way — `enabled` (survives reboot) and `Restart=always` (survives crashes), so it no longer depends on an active SSH session. + ## Deploying Updates From the local project directory: ```bash -rsync -avz ransomware_fetcher.py threat_intel_bot.py rss_manager.py content_classifier.py feeds/threat_intel_feeds.json feeds/news_feeds.json feeds/research_feeds.json root@deployer:/root/tele-bots/rss_splited_bot/ -ssh root@deployer "systemctl restart rss-bot" +rsync -avz --relative ransomware_fetcher.py threat_intel_bot.py rss_manager.py content_classifier.py bot_config.py check_feeds.py requirements.txt ./feeds/threat_intel_feeds.json ./feeds/news_feeds.json ./feeds/research_feeds.json ./feeds/malware_feeds.json ./feeds/osint_feeds.json ubuntu@:/home/ubuntu/rss-tele-bot/ +ssh ubuntu@ "sudo systemctl restart rss-bot" ``` ## Notes -- First run marks all current articles and victims as seen — no flood on startup +- First run (empty seen-DB) marks all current articles and victims as seen — no flood on startup. A restart of an already-initialized bot instead fetches and alerts on anything published while it was stopped, so a redeploy doesn't silently drop real alerts. - Polling interval: 5 minutes - Articles published within the last 48 hours are processed (rolling window, not calendar day) -- `seen_articles.db` and `seen_victims.db` auto-purge entries older than 7 days +- Cross-source duplicates (the same story from a different outlet) are caught by fuzzy title matching, not just exact URL/text matches +- `seen_articles.db` and `seen_victims.db` auto-purge entries older than 14 days