telegram-rss-bot/README.md

# telegram-rss-bot

Telegram bot that monitors cybersecurity RSS feeds and ransomware.live victim data, delivering real-time alerts to Telegram topics. Articles are enriched with CVEs, threat actors, and malware families extracted from content.

## Project Structure

```
rss_telegram_bot/
├── threat_intel_bot.py       # Main bot — commands, subscriptions, alert dispatch
├── rss_manager.py            # Feed fetching, dedup (incl. cross-source), message formatting
├── content_classifier.py     # CVE/actor/malware/MITRE extraction
├── ransomware_fetcher.py     # ransomware.live PRO API — victim feed, LV/EE/LT only
├── bot_config.py             # Shared category, feed-file, and tuning config
├── check_feeds.py            # CLI utility to check feed health
├── feeds/
│   ├── news_feeds.json
│   ├── malware_feeds.json
│   ├── threat_intel_feeds.json   # includes CISA Advisories, Alerts, ICS
│   ├── osint_feeds.json
│   └── research_feeds.json
├── validation/
│   ├── run_validation.py     # continuous validation monitor (saves to results.jsonl)
│   └── review.py             # pretty-print and filter results.jsonl
├── .env                      # not committed — see Setup
├── subscribers.json          # auto-managed — chat/topic subscriptions
├── seen_articles.db          # SQLite — tracks sent RSS articles + titles (14-day retention)
└── seen_victims.db           # SQLite — tracks sent ransomware victims (14-day retention)
```

## Setup

```bash
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

Create `.env`:

```env
BOT_TOKEN=your_token_here
RANSOMWARE_LIVE_API_KEY=your_key_here   # optional — register at api-pro.ransomware.live
```

Run:

```bash
python3 threat_intel_bot.py
```

## Bot Commands

| Command | Action |
|---|---|
| `/on_<category>` | Subscribe this chat/topic to a category |
| `/off_<category>` | Unsubscribe |
| `/stats` | Check feed health and subscriber count |
| `/help` | Show commands |

Categories: `news`, `malware`, `threat_intel`, `osint`, `research`, `ransomware`

**Telegram Topics:** Run `/on_<category>` inside each topic to route categories to separate threads.

## Message Format

RSS articles:

```
{emoji} Title

Description

📡 Source  ·  Published (UTC)
🆔 CVE-2024-1234, CVE-2024-5678      (only if CVEs detected)
👤 APT28, Fancy Bear                  (only if threat actors detected)
🦠 LockBit, Cobalt Strike             (only if malware families detected)

🔗 Read Full Article
```

Ransomware victims (posted to the `ransomware` topic, separate from `malware` so a busy leak day doesn't bury other malware sources):

```
💰 company.com claimed by LockBit

Sector: Finance | Country: US
Data description from leak post

📡 ransomware.live  ·  Published (UTC)
🦠 Lockbit

🔗 View on ransomware.live
```

Enrichment lines only appear when the classifier finds something — clean articles stay minimal.

## Data Sources

### RSS Feeds (64 feeds across 5 categories)

| Category | Sources |
|---|---|
| `news` | Krebs, Bleeping Computer, The Hacker News, Security Week, Dark Reading, The Record, Cyberscoop, NCSC UK, and others |
| `threat_intel` | CISA Advisories, CISA Alerts, CISA ICS, SANS ISC, Unit42, Mandiant, Cisco Talos, Microsoft MSRC, and others |
| `malware` | Malware Traffic Analysis, Kaspersky Securelist, Security Affairs, Sekoia, and others |
| `osint` | DataBreaches.net, Have I Been Pwned, UpGuard, Vulmon Research |
| `research` | Google Project Zero, watchTowr Labs, SpecterOps, Synacktiv, Doyensec, NCC Group, and others |

Add or remove feeds by editing the relevant `feeds/*.json` file, then restart the bot. Run `python3 check_feeds.py` to verify health before deploying.

### ransomware.live PRO API

When `RANSOMWARE_LIVE_API_KEY` is set, the bot polls the PRO API every 5 minutes for newly discovered ransomware victims, filtered to companies in Latvia, Estonia, and Lithuania (`ALLOWED_RANSOMWARE_COUNTRIES` in `bot_config.py`) — the global feed is available directly on ransomware.live. New entries are posted to the `ransomware` topic (subscribe with `/on_ransomware`), kept separate from RSS `malware` articles. Victims are tracked in `seen_victims.db` with 14-day retention.

## Classification

`content_classifier.py` extracts the following from article title + description:

**CVEs** — `CVE-YYYY-NNNNN` regex pattern.

**Threat actors** — APT groups and named adversaries including APT28/29/40/41, Lazarus, Sandworm, Volt Typhoon, Scattered Spider, FIN7, and others. Full list in `THREAT_ACTORS`.

**Malware families** — ransomware, C2 frameworks, loaders, stealers, APT tooling: LockBit, Cobalt Strike, Emotet, QakBot, Sliver, PlugX, and others. Full list in `MALWARE_FAMILIES`. Matching is case-insensitive with word-boundary checking. To add a family, append its display name to the set.

Severity and quality scoring are intentionally absent — the enrichment fields give enough context for the reader to judge importance.

## Validation

A separate monitor for testing without affecting the production bot:

```bash
# runs continuously, same poll cycle as the bot
python3 validation/run_validation.py

# review collected results
python3 validation/review.py                      # all results
python3 validation/review.py --category research  # filter by category
python3 validation/review.py --has cves           # only articles with CVEs
python3 validation/review.py --has malware        # only articles with malware hits
python3 validation/review.py --today              # only today's run
```

Results are saved to `validation/results.jsonl`. Each line is a JSON object with a `type` field (`article` or `victim`) so RSS articles and ransomware victims can be filtered separately.

## Systemd Service (VPS)

`/etc/systemd/system/rss-bot.service`:

```ini
[Unit]
Description=Threat Intel RSS Telegram Bot
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/rss-tele-bot
EnvironmentFile=/home/ubuntu/rss-tele-bot/.env
ExecStart=/home/ubuntu/rss-tele-bot/venv/bin/python3 threat_intel_bot.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now rss-bot
sudo journalctl -u rss-bot -f
```

Currently deployed and running this way — `enabled` (survives reboot) and `Restart=always` (survives crashes), so it no longer depends on an active SSH session.

## Deploying Updates

From the local project directory:

```bash
rsync -avz --relative ransomware_fetcher.py threat_intel_bot.py rss_manager.py content_classifier.py bot_config.py check_feeds.py requirements.txt ./feeds/threat_intel_feeds.json ./feeds/news_feeds.json ./feeds/research_feeds.json ./feeds/malware_feeds.json ./feeds/osint_feeds.json ubuntu@<vps-ip>:/home/ubuntu/rss-tele-bot/
ssh ubuntu@<vps-ip> "sudo systemctl restart rss-bot"
```

## Notes

- First run (empty seen-DB) marks all current articles and victims as seen — no flood on startup. A restart of an already-initialized bot instead fetches and alerts on anything published while it was stopped, so a redeploy doesn't silently drop real alerts.
- Polling interval: 5 minutes
- Articles published within the last 48 hours are processed (rolling window, not calendar day)
- Cross-source duplicates (the same story from a different outlet) are caught by fuzzy title matching, not just exact URL/text matches
- `seen_articles.db` and `seen_victims.db` auto-purge entries older than 14 days