InfinityPlay - Video Streaming Platform

Overview

Netflix-style Video Streaming Platform is a production-level distributed streaming system built with Java and Spring Boot microservices. It brings video upload, content management, FFmpeg-based transcoding, HLS adaptive streaming, AWS S3 storage, Kafka-driven asynchronous processing, Redis caching, and secure presigned URL delivery into a scalable end-to-end video streaming architecture designed to handle large video files and high concurrent traffic.

GitHub: saiifbaba/InfinityPlay

Problem Statement

Traditional video delivery systems struggle to efficiently handle large video files, multiple quality levels, secure storage, and high concurrent streaming traffic. Uploading a large raw video, transcoding it into different resolutions, generating streaming-ready formats, and delivering content securely requires multiple tightly coordinated components. I wanted to build a production-level Netflix-like platform that combines microservices, asynchronous processing, cloud storage, adaptive streaming, and caching into a single scalable video streaming architecture.

Key Features

Content Management

• Manage movies and video metadata through dedicated Content Service
• Store titles, descriptions, genres, thumbnails, and video information in MySQL
• REST APIs for creating, updating, retrieving, and managing video content
• Microservice-based architecture with an API Gateway for centralized access

Video Upload & Storage

• Upload large raw video files through the Video Service
• Store original video files securely in AWS S3
• Use AWS IAM policies and roles for controlled access to video assets
• Publish a video.uploaded Kafka event after successful upload
• Keep video storage decoupled from application services for scalability

FFmpeg Encoding Pipeline

• Encoding Service consumes video.uploaded events asynchronously
• Downloads source videos from AWS S3 for processing
• Uses FFmpeg to transcode videos into multiple qualities:
  • 1080p
  • 720p
  • 480p
  • 360p
• Generates HLS .m3u8 playlists and segmented video chunks
• Uploads encoded HLS assets back to AWS S3
• Publishes a video.encoded Kafka event when processing is complete

Adaptive HLS Streaming

• Converts large videos into small HLS segments for efficient playback
• Generates a master .m3u8 playlist containing multiple quality variants
• Allows the video player to dynamically switch quality based on bandwidth and device conditions
• Supports smooth playback while reducing buffering and unnecessary bandwidth usage
• Streams video segments directly from cloud storage rather than through application servers

Secure Video Delivery

• Streaming Service generates AWS S3 presigned URLs for protected video content
• Prevents exposing permanent public S3 URLs
• Provides time-limited access to playlists and video segments
• Uses Redis to cache frequently requested streaming URLs
• Reduces repeated URL generation and unnecessary AWS requests

Event-Driven Architecture

• Uses Apache Kafka for asynchronous communication between microservices
• video.uploaded triggers the encoding pipeline
• video.encoded notifies downstream services that streaming content is ready
• Kafka partitions allow encoding workloads to scale horizontally
• Services remain loosely coupled and independently deployable

Caching & Performance

• Redis caches frequently accessed data and streaming URL information
• Reduces database and AWS S3-related overhead
• Improves response times for frequently watched content
• Supports scalable access patterns for high-concurrency streaming workloads

Dockerized Deployment

• Each microservice runs inside an isolated Docker container
• Docker Compose manages the complete local infrastructure
• Containers include:
  • API Gateway
  • Content Service
  • Video Service
  • Encoding Service
  • Streaming Service
  • MySQL
  • Redis
  • Kafka
  • Zookeeper
• Enables consistent development, testing, and deployment environments

End-to-End Streaming Workflow

User Uploads Video
        ↓
API Gateway
        ↓
Video Service
        ↓
AWS S3 - Raw Video
        ↓
Kafka: video.uploaded
        ↓
Encoding Service
        ↓
FFmpeg Transcoding
        ↓
1080p / 720p / 480p / 360p
        ↓
HLS Segments + M3U8 Playlists
        ↓
AWS S3 - Encoded Content
        ↓
Kafka: video.encoded
        ↓
Streaming Service
        ↓
Presigned S3 URLs
        ↓
Redis Cache
        ↓
HLS Video Player
        ↓
Adaptive Video Streaming

Tech Stack

• Java 17+ – Core programming language for building scalable backend services
• Spring Boot – Microservice framework for REST APIs, business logic, dependency injection, and service configuration
• Spring Cloud / API Gateway – Centralized routing, request handling, and communication between microservices
• MySQL – Relational database for storing movie metadata, users, content information, and application data
• Redis – High-performance caching layer for frequently accessed data, sessions, and AWS S3 presigned streaming URLs
• Apache Kafka – Event-driven messaging platform for asynchronous communication between upload, encoding, and streaming services
• Zookeeper – Kafka cluster coordination and broker management
• AWS S3 – Scalable cloud object storage for raw videos, encoded HLS segments, playlists, and streaming assets
• AWS IAM – Secure access control and permissions for AWS S3 resources
• FFmpeg – Video processing and transcoding engine for generating multiple resolutions and HLS segments
• HLS (HTTP Live Streaming) – Adaptive streaming protocol using .m3u8 playlists and segmented video files
• Docker – Containerization of microservices and infrastructure components
• Docker Compose – Local orchestration of services, Kafka, Zookeeper, Redis, and MySQL
• HTML / JavaScript – Custom frontend video player for consuming and playing HLS streams

What I Learned

Building this Netflix-like Video Streaming Platform taught me:

• How to design production-oriented microservice architectures with clear service boundaries and independent scalability
• How event-driven architecture works using Kafka and how asynchronous processing prevents long-running video encoding from blocking user requests
• How FFmpeg performs video transcoding and how to generate multiple resolutions for adaptive bitrate streaming
• How HLS works internally using master playlists, variant playlists, and small video segments for smooth adaptive playback
• How AWS S3 can be used as scalable object storage for very large video files without putting the application servers under heavy load
• How presigned URLs provide temporary and controlled access to private S3 objects
• How Redis caching reduces repeated URL generation and improves response times for frequently requested streaming resources
• How Docker and Docker Compose simplify development, infrastructure management, testing, and deployment of distributed services
• How Kafka partitions and consumer scaling can be used to process large numbers of video encoding jobs concurrently
• How to design reliable asynchronous workflows where video upload, encoding, and streaming availability are separate stages

Impact

The platform provides a complete end-to-end video streaming workflow similar to the core backend architecture of platforms like Netflix.

• Scalable Storage – Large raw and encoded video assets are stored in AWS S3 instead of application servers
• Adaptive Streaming – HLS automatically supports multiple video qualities based on available bandwidth and device capabilities
• Asynchronous Encoding – Kafka decouples video uploads from CPU-intensive FFmpeg processing
• Secure Delivery – Private S3 content is delivered through time-limited presigned URLs
• High Performance – Redis caching reduces repeated requests and improves streaming-related response times
• Microservice Architecture – Content, video upload, encoding, and streaming responsibilities are independently deployable
• Production-Oriented – Dockerized infrastructure, Kafka event processing, cloud storage, caching, and secure access patterns provide a foundation for horizontal scaling
• Multi-Quality Support – Automatically produces 1080p, 720p, 480p, and 360p versions from a single uploaded source

Architecture Highlights

• Event-driven video pipeline: Video Service publishes a video.uploaded Kafka event after storing the original file in S3, allowing the Encoding Service to process the video asynchronously
• Independent encoding workers: Encoding Service consumes Kafka events and uses FFmpeg to process multiple videos concurrently, allowing encoding capacity to scale independently
• HLS-based adaptive streaming: FFmpeg converts videos into multiple resolutions and generates HLS playlists and segments, allowing clients to dynamically switch between quality levels
• Cloud-native video storage: AWS S3 stores both original videos and generated HLS assets, keeping large files away from application servers
• Secure content delivery: Streaming Service generates temporary S3 presigned URLs instead of exposing private video objects publicly
• Redis URL caching: Frequently requested presigned URLs and streaming information are cached to reduce repeated computation and external storage requests
• Service isolation: Content, Video, Encoding, and Streaming Services have independent responsibilities and can be scaled separately based on workload
• Kafka-based scalability: Kafka topics and partitions distribute video-processing workloads across multiple encoding consumers
• Containerized infrastructure: Docker and Docker Compose provide reproducible environments for microservices, MySQL, Redis, Kafka, and Zookeeper
• End-to-end processing pipeline: Upload → S3 → Kafka → FFmpeg → HLS → S3 → Kafka → Streaming Service → Redis → HLS Player