NewsHub - AI-Powered News Aggregation Platform
#redischallenge#devchallenge#database#ai
Varshith V Hegde

Varshith V Hegde @varshithvhegde

About: A simple programmer fond of learning

Location:
Mangalore
Joined:
Jun 30, 2022

NewsHub - AI-Powered News Aggregation Platform

Publish Date: Aug 6
61 10

This is a submission for the Redis AI Challenge: Real-Time AI Innovators.

What I Built

NewsHub is an intelligent news aggregation platform that revolutionizes how users discover and consume news content. Built with Redis 8 as the core real-time data layer, NewsHub combines the power of AI with Redis's advanced capabilities to deliver personalized, contextually relevant news experiences.

This was my first time working with Redis, and the learning curve was steep but incredibly rewarding. I discovered that Redis is far more than just a cache - it's a powerful, multi-model platform capable of handling complex AI workloads. Building NewsHub taught me how Redis can serve as a primary database, vector search engine, and real-time analytics platform all in one.

Key Features

AI-Powered Content Intelligence

  • Smart summarization using Google Gemini AI for concise, meaningful article summaries
  • Advanced sentiment analysis with real-time emotion detection (positive, negative, neutral)
  • Dynamic keyword extraction and AI-driven topic identification
  • Semantic understanding for context-aware content analysis

Advanced Search & Discovery

  • Vector semantic search using Redis-based similarity matching with 768-dimensional embeddings
  • Multi-faceted filtering by topic, sentiment, source, keywords, and date ranges
  • Similar articles engine using cosine similarity algorithms
  • Fuzzy search with typo tolerance and intelligent suggestions

Intelligent Personalization

  • Smart feed management that automatically removes viewed articles from personalized feeds
  • Behavioral analytics tracking reading patterns and engagement metrics
  • Real-time recommendations based on user interaction patterns

Real-Time Analytics & Insights

  • Performance monitoring with cache hit rates, response times, and system health
  • Comprehensive user journey analytics

Performance & Scalability

  • Multi-layer caching with intelligent request, query, and result caching strategies
  • Sub-50ms response times through optimized Redis operations
  • Real-time processing with live news ingestion and AI analysis pipeline

Architecture Design

NewsHub follows a modern microservices architecture with Redis 8 serving as the central intelligence layer. The system is designed for high performance, scalability, and real-time AI processing.

High-Level System Architecture

System Aritecture

Redis 8 Multi-Model Platform Architecture

Redis Architecture

Data Flow & Processing Pipeline

Data Flow Diagram

Demo

Live Application

  • Web App: NewsHub Live Demo
  • API Backend: NewsHub API
  • GitHub Repository:

    GitHub logo Varshithvhegde / newshub

    NewsHub - AI-Powered News Aggregation Platform

    NewsHub - AI-Powered News Aggregation Platform

    NewsHub - AI-Powered News Aggregation Platform

    A modern, full-stack news aggregation and summarization platform built with React, TypeScript, Node.js, and Redis. NewsHub provides users with AI-powered news summaries, personalized feeds, and advanced search capabilities using Google Gemini AI and Redis vector search.

    🚀 Features

    Core Functionality

    • AI-Powered News Summarization: Uses Google Gemini AI for intelligent content analysis
    • Vector Search & Similarity: Redis-based semantic search with embeddings
    • Personalized News Feeds: User preference-based content recommendations
    • Real-time News Fetching: Automated news collection from multiple sources
    • Advanced Search: Filter articles by topic, sentiment, source, and keywords
    • Responsive Design: Optimized for desktop, tablet, and mobile devices

    User Experience

    • Infinite Scroll: Seamless browsing with automatic content loading
    • Article Analytics: View engagement metrics and trending articles
    • Sentiment Analysis: Visual indicators for article sentiment (positive, negative, neutral)
    • Topic Filtering: Browse news by categories and interests
    • Real-time Updates…
    View on GitHub

Youtube Demo


Audio & Accessibility Note:
Please forgive any background noise or my accent in the video. For the best experience, I recommend turning on closed captions. They're available in English and will help make sure you don't miss a thing!

Screenshots

Home Page - Intelligent News Feed

NewsHub Homepage

The main dashboard featuring AI-curated news with sentiment indicators, engagement metrics, and personalized recommendations.

Advanced Search Interface

Advanced Search

Powerful search capabilities with real-time filtering by topic, sentiment

Personalized Article View

Article Details

Detailed article view with AI-generated summaries, sentiment analysis, and similar article recommendations.

Personalized "For You" Feed

Personalized Feed

AI-curated personalized news feed that automatically adapts based on user preferences and reading behavior.

Similar Articles Discovery

Similar Articles

Vector-based semantic similarity engine showing contextually related articles using Redis vector search.

Key Features Demonstrated

  • Intelligent search with vector similarity search achieving sub-100ms response times
  • Real-time personalization with dynamic content curation based on user preferences
  • Smart article removal that automatically removes viewed articles from personalized feeds
  • Sentiment-based filtering with visual sentiment indicators
  • AI-powered trending detection with engagement metrics

How I Used Redis 8

Redis 8 serves as the primary real-time data layer powering NewsHub's intelligent features. Coming from a background where I primarily used traditional databases, learning Redis was eye-opening. I discovered how Redis can handle complex AI workloads while maintaining incredible performance.

Vector Search & Semantic Intelligence

This was the most challenging part of my Redis journey. Understanding how to implement vector search and semantic similarity was completely new to me, but the results were amazing.

// redisService.js - Vector Search Implementation
class RedisService {
  async createVectorIndex() {
    try {
      // Create vector search index for semantic similarity
      await this.client.ft.create('news:vector_index', {
        '$.title': 'TEXT',
        '$.content': 'TEXT',
        '$.embedding': 'VECTOR',
        '$.sentiment': 'TAG',
        '$.topic': 'TAG',
        '$.source': 'TAG',
        '$.publishedAt': 'NUMERIC'
      }, {
        ON: 'JSON',
        PREFIX: 'news:articles:',
        FILTER: '@publishedAt:[0 +inf]',
        SCHEMA: {
          '$.title': { type: 'TEXT', WEIGHT: 2.0 },
          '$.content': { type: 'TEXT', WEIGHT: 1.0 },
          '$.embedding': { 
            type: 'VECTOR', 
            DIM: 768, 
            DISTANCE_METRIC: 'COSINE',
            TYPE: 'FLOAT32'
          },
          '$.sentiment': { type: 'TAG' },
          '$.topic': { type: 'TAG' },
          '$.source': { type: 'TAG' },
          '$.publishedAt': { type: 'NUMERIC' }
        }
      });
      console.log('✅ Vector search index created successfully');
    } catch (error) {
      console.log('Vector index already exists or error:', error.message);
    }
  }

  async findSimilarArticles(articleId, limit = 10) {
    try {
      // Get article embedding
      const article = await this.client.json.get(`news:articles:${articleId}`);
      if (!article || !article.embedding) {
        throw new Error('Article or embedding not found');
      }

      // Vector similarity search
      const query = `*=>[KNN $k @embedding $embedding AS score]`;
      const params = {
        k: limit,
        embedding: article.embedding
      };

      const results = await this.client.ft.search('news:vector_index', query, {
        PARAMS: params,
        RETURN: ['$.title', '$.content', '$.sentiment', '$.topic', 'score'],
        SORTBY: 'score',
        DIALECT: 2
      });

      return results.documents.map(doc => ({
        id: doc.id.replace('news:articles:', ''),
        title: doc.value['$.title'],
        content: doc.value['$.content'],
        sentiment: doc.value['$.sentiment'],
        topic: doc.value['$.topic'],
        similarity: doc.value.score
      }));
    } catch (error) {
      console.error('Error finding similar articles:', error);
      throw error;
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

JSON Storage with Full-Text Search

Learning to use Redis as a primary database was a paradigm shift. The ability to store complex JSON documents while maintaining full-text search capabilities was incredible.

// redisService.js - JSON Storage Implementation
class RedisService {
  async storeArticle(article) {
    try {
      const articleId = `news:articles:${article.id}`;

      // Store article as JSON with full-text search capabilities
      await this.client.json.set(articleId, '$', {
        id: article.id,
        title: article.title,
        content: article.content,
        summary: article.summary,
        sentiment: article.sentiment,
        topic: article.topic,
        source: article.source,
        url: article.url,
        publishedAt: article.publishedAt,
        embedding: article.embedding,
        keywords: article.keywords,
        metadata: {
          wordCount: article.wordCount,
          readingTime: article.readingTime,
          language: article.language
        }
      });

      // Create search index for full-text search
      await this.createSearchIndex();

      console.log(`✅ Article stored: ${articleId}`);
      return article.id;
    } catch (error) {
      console.error('Error storing article:', error);
      throw error;
    }
  }

  async searchArticles(query, filters = {}) {
    try {
      let searchQuery = `(@title:${query} | @content:${query} | @summary:${query})`;

      // Add filters
      if (filters.sentiment) {
        searchQuery += ` @sentiment:{${filters.sentiment}}`;
      }
      if (filters.topic) {
        searchQuery += ` @topic:{${filters.topic}}`;
      }
      if (filters.source) {
        searchQuery += ` @source:{${filters.source}}`;
      }
      if (filters.dateRange) {
        searchQuery += ` @publishedAt:[${filters.dateRange.start} ${filters.dateRange.end}]`;
      }

      const results = await this.client.ft.search('news:search_index', searchQuery, {
        RETURN: ['$.title', '$.content', '$.sentiment', '$.topic', '$.source'],
        SORTBY: '@publishedAt',
        LIMIT: { from: 0, size: 50 }
      });

      return results.documents.map(doc => ({
        id: doc.id.replace('news:articles:', ''),
        title: doc.value['$.title'],
        content: doc.value['$.content'],
        sentiment: doc.value['$.sentiment'],
        topic: doc.value['$.topic'],
        source: doc.value['$.source']
      }));
    } catch (error) {
      console.error('Error searching articles:', error);
      throw error;
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Multi-Layer Caching Strategy

Understanding Redis caching patterns was crucial for performance. I implemented a sophisticated multi-layer approach that dramatically improved response times.

// cacheClearService.js - Multi-layer Caching Implementation
class CacheClearService {
  constructor(redisClient) {
    this.client = redisClient;
    this.cacheLayers = {
      REQUEST: 'cache:requests:',
      QUERY: 'cache:queries:',
      RESULT: 'cache:results:',
      SIMILARITY: 'cache:similarity:',
      USER: 'cache:user:'
    };
  }

  async setCacheWithTTL(key, data, ttl = 3600) {
    try {
      const cacheKey = `cache:results:${key}`;
      await this.client.json.set(cacheKey, '$', {
        data: data,
        timestamp: Date.now(),
        ttl: ttl
      });
      await this.client.expire(cacheKey, ttl);
      console.log(`✅ Cache set: ${cacheKey}`);
    } catch (error) {
      console.error('Error setting cache:', error);
    }
  }

  async getCache(key) {
    try {
      const cacheKey = `cache:results:${key}`;
      const cached = await this.client.json.get(cacheKey);

      if (cached && cached.data) {
        console.log(`✅ Cache hit: ${cacheKey}`);
        return cached.data;
      }

      console.log(`❌ Cache miss: ${cacheKey}`);
      return null;
    } catch (error) {
      console.error('Error getting cache:', error);
      return null;
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Real-Time Personalization Engine

The personalization system was where I learned about Redis's advanced data structures. Implementing automatic article removal from personalized feeds was a great learning experience.

// redisService.js - User Preferences Implementation
class RedisService {
  async storeUserPreferences(userId, preferences) {
    try {
      const userKey = `user:preferences:${userId}`;

      await this.client.hset(userKey, {
        topics: JSON.stringify(preferences.topics),
        sources: JSON.stringify(preferences.sources),
        sentiment: preferences.sentiment,
        language: preferences.language,
        updatedAt: Date.now()
      });

      // Set expiration for user preferences (30 days)
      await this.client.expire(userKey, 30 * 24 * 60 * 60);

      console.log(`✅ User preferences stored: ${userKey}`);
      return true;
    } catch (error) {
      console.error('Error storing user preferences:', error);
      throw error;
    }
  }

  async getPersonalizedNews(userId, limit = 20) {
    try {
      const preferences = await this.getUserPreferences(userId);
      if (!preferences) {
        return await this.getTrendingArticles(limit);
      }

      // Build personalized search query
      let searchQuery = '*';
      const filters = [];

      if (preferences.topics.length > 0) {
        filters.push(`@topic:{${preferences.topics.join('|')}}`);
      }
      if (preferences.sources.length > 0) {
        filters.push(`@source:{${preferences.sources.join('|')}}`);
      }
      if (preferences.sentiment) {
        filters.push(`@sentiment:{${preferences.sentiment}}`);
      }

      if (filters.length > 0) {
        searchQuery = filters.join(' ');
      }

      const results = await this.client.ft.search('news:search_index', searchQuery, {
        RETURN: ['$.title', '$.content', '$.sentiment', '$.topic', '$.source'],
        SORTBY: '@publishedAt',
        LIMIT: { from: 0, size: limit }
      });

      return results.documents.map(doc => ({
        id: doc.id.replace('news:articles:', ''),
        title: doc.value['$.title'],
        content: doc.value['$.content'],
        sentiment: doc.value['$.sentiment'],
        topic: doc.value['$.topic'],
        source: doc.value['$.source']
      }));
    } catch (error) {
      console.error('Error getting personalized news:', error);
      throw error;
    }
  }

  // Auto-remove viewed articles from personalized feed
  async trackArticleView(userId, articleId) {
    try {
      // Add to viewed articles set
      await this.client.sadd(`user:viewed:${userId}`, articleId);

      // Remove from personalized feed
      await this.client.srem(`user:personalized:${userId}`, articleId);

      // Track engagement metrics
      await this.trackEngagement(articleId, 'view');

      console.log(`✅ Article view tracked and removed from personalized feed: ${articleId}`);
    } catch (error) {
      console.error('Error tracking article view:', error);
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Trending Articles & Analytics

Learning to use Redis Sorted Sets for trending algorithms was fascinating. The ability to implement time-decay algorithms directly in Redis was a game-changer.

// redisService.js - Trending Articles Implementation
class RedisService {
  async updateTrendingArticles() {
    try {
      // Get all articles with engagement metrics
      const articles = await this.client.ft.search('news:search_index', '*', {
        RETURN: ['$.title', '$.source', '$.publishedAt'],
        SORTBY: '@publishedAt',
        LIMIT: { from: 0, size: 1000 }
      });

      const trendingScores = {};

      for (const doc of articles.documents) {
        const articleId = doc.id.replace('news:articles:', '');
        const publishedAt = parseInt(doc.value['$.publishedAt']);
        const now = Date.now();

        // Calculate trending score based on recency and engagement
        const timeDecay = Math.exp(-(now - publishedAt) / (24 * 60 * 60 * 1000)); // 24 hours
        const engagementScore = await this.getEngagementScore(articleId);

        trendingScores[articleId] = timeDecay * engagementScore;
      }

      // Store trending articles in sorted set
      const trendingKey = 'trending:articles';
      await this.client.del(trendingKey); // Clear existing

      for (const [articleId, score] of Object.entries(trendingScores)) {
        await this.client.zadd(trendingKey, score, articleId);
      }

      console.log(`✅ Updated trending articles: ${Object.keys(trendingScores).length} articles`);
    } catch (error) {
      console.error('Error updating trending articles:', error);
      throw error;
    }
  }

  async trackEngagement(articleId, action = 'view') {
    try {
      const engagementKey = `metrics:engagement:${articleId}`;
      const actionKey = `${action}:count`;

      await this.client.hincrby(engagementKey, actionKey, 1);
      await this.client.hset(engagementKey, 'last_updated', Date.now());

      // Set expiration for engagement metrics (7 days)
      await this.client.expire(engagementKey, 7 * 24 * 60 * 60);

      console.log(`✅ Tracked engagement: ${action} for article ${articleId}`);
    } catch (error) {
      console.error('Error tracking engagement:', error);
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Performance Results

Through my Redis learning journey, I achieved impressive performance metrics:

Cache Performance

  • Cache hit rate of 85-90% for frequently accessed data
  • Response times under 50ms for cached responses
  • Optimized memory usage with LRU eviction policies

Vector Search Performance

  • Similarity search under 100ms for 1000+ articles
  • 768-dimensional vector storage with COSINE distance
  • Efficient compression with RedisJSON

Scalability Metrics

  • Supports 1000+ concurrent users
  • Efficient JSON storage with full-text search capabilities
  • Sub-second response times for personalized feeds

Learning Experience

This project was my introduction to Redis, and it completely changed my perspective on data storage and caching. I learned that Redis isn't just a cache - it's a complete data platform capable of:

  • Serving as a primary database with JSON documents
  • Powering vector search for AI applications
  • Handling real-time analytics with sorted sets and hashes
  • Managing complex caching strategies across multiple layers
  • Supporting real-time streaming and pub/sub messaging

The learning curve was steep, especially understanding vector search and Redis's query syntax, but the performance and capabilities I achieved made every challenge worthwhile. Building NewsHub taught me how Redis can accelerate AI applications and handle complex, real-time workloads that would typically require multiple specialized systems.

NewsHub demonstrates how Redis 8 can power the next generation of AI applications, serving as a complete real-time intelligence platform. The combination of vector search, JSON storage, multi-layer caching, and real-time analytics creates a seamless, intelligent user experience that adapts in real-time to user behavior and preferences.

Built with passion by Varshith V Hegde

Tech Stack: React, TypeScript, Node.js, Express, Redis 8, Google Gemini AI, Tailwind CSS

Comments 10 total

  • Super Jeo
    Super JeoAug 6, 2025

    It nicely explains all the redis 8 features to full extent

  • Akhil
    AkhilAug 7, 2025

    Good work brother 🙌🏻 nicely explained the use case of redis 8

  • Anik Sikder
    Anik SikderAug 7, 2025

    As someone who's just starting to explore Redis beyond caching, this project is super inspiring. The way you used Redis 8 for vector search, real-time analytics, and AI-powered features like sentiment analysis and summarization is seriously impressive.

    I especially loved the integration of 768-dimensional embeddings and the smart feed personalization it's clear a lot of thought went into both performance and UX.

    Thanks for sharing this! Definitely learned a lot just reading through the architecture and feature set.

    • Varshith V Hegde
      Varshith V HegdeAug 7, 2025

      Thank you so much! 😊
      I’m really glad you found the project inspiring. Honestly, I also only recently started exploring Redis and it’s been such a fun journey. The Redis Stack makes it super easy to experiment with advanced features—vector search come almost out of the box.
      Appreciate your kind words and support—it means a lot!

  • Dishank Gupta
    Dishank GuptaAug 7, 2025

    Great Stuff!

  • Paramanantham Harrison
    Paramanantham HarrisonAug 10, 2025

    Very well done

Add comment