AI News Aggregation Engine
A production news platform that continuously collects articles from RSS feeds, news APIs, Telegram channels, and the open web. It removes duplicates, summarizes each story with AI, and publishes through a Telegram approval workflow, with real-time dashboards over the whole pipeline.
A FastAPI backend with a React and Vite dashboard, PostgreSQL for storage, and Redis as both cache and Celery broker. Background workers handle collection, processing, summarization, and publishing on separate priority queues. Everything is containerized with Docker Compose behind nginx, with Prometheus, Grafana, and an ELK stack for monitoring.
Two-layer de-duplication
Exact SHA-256 hashing and SimHash near-duplicate detection make sure each story is published once, no matter how many sources carry it.
Resilient AI summarization
A multi-provider fallback chain across Groq, DeepSeek, Cohere, OpenAI, Gemini, and Claude keeps summaries flowing when a provider fails, with per-request cost tracking.
Telegram approval workflow
Editors approve, edit, reject, or skip each story from inline Telegram buttons, or switch to fully automatic publishing.
Full observability
Celery workers on a Beat schedule drive the pipeline, with Prometheus, Grafana, and an ELK stack for metrics and logs.