Coursify

System Design for Software Engineers

Case Study: Designing a Video Streaming Service

Case Study: Designing a Video Streaming Service

Designing a service like YouTube or Netflix is primarily a challenge of Massive Data Storage and High-Bandwidth Content Delivery. Video files are huge, and users expect them to start playing instantly without buffering, regardless of their device or internet speed.

The Requirements

  1. Functional: Upload videos. Stream videos. Search for videos.
  2. Non-Functional: High availability. Low latency (minimal buffering). Smooth playback across different network conditions (Adaptive Bitrate Streaming).

The Video Pipeline

A video isn't just a single file. To support millions of devices, we must process the raw upload into multiple formats and resolutions.

  • Transcoding: Converting the raw 4K upload into 1080p, 720p, 480p, etc., using different codecs (H.264, VP9).
  • Chunking: Breaking the video into small 2-5 second "chunks". This allows the player to switch resolutions mid-stream if the user's internet slows down.

Adaptive Bitrate Streaming (ABR)

This is the magic that prevents buffering. The video player in your browser or app constantly monitors your download speed. If the speed drops, it automatically requests the next 5-second chunk in a lower resolution (e.g., switching from 1080p to 480p) instead of stopping to buffer.

The Journey of a Video Upload

  1. 1
    Step 1

    The user uploads a 1GB raw .mov file. The Upload Service saves it to a 'Raw Storage' bucket (e.g., S3).

  2. 2
    Step 2

    A message is sent to a queue (Kafka) to trigger the Transcoding Service.

  3. 3
    Step 3

    Multiple workers pick up the task. They split the video into chunks and transcode them into multiple resolutions (1080p, 720p, 480p) and formats (HLS, DASH) in parallel.

  4. 4
    Step 4

    Once finished, the metadata (title, tags, resolution paths) is saved to a NoSQL database, and the processed chunks are moved to 'Production Storage'.

  5. 5
    Step 5

    The video is now ready. When the first user in London requests it, the CDN fetches the chunks from the origin (S3) and caches them at the London Edge location for all future UK users.

Key Components

  • Object Storage (S3/GCS): Used to store the actual video files. It is much cheaper and more scalable than traditional file servers.
  • CDN (Content Delivery Network): Absolutely vital. 99% of video traffic should be served from a CDN edge location, not from your origin server.
  • NoSQL DB (Cassandra/MongoDB): Stores video metadata, user comments, and view counts.

Optimizing for Speed

  1. Edge Caching: Use a CDN to put the video chunks as close to the user as possible.
  2. Prefetching: The video player can guess that you will keep watching and start downloading the next few chunks while you are still watching the current one.
  3. Thumbnail Previews: Generate a "sprite sheet" of thumbnails so users can see a preview when they hover over the seek bar.

Common Mistakes

  • Streaming Directly from App Servers: Your app servers should never touch the video bytes. They only handle metadata. Use the CDN and Object Storage for the actual data.
  • Ignoring Copyright/Safety: Not building an automated 'Content ID' system or moderation pipeline to flag inappropriate content during transcoding.
  • Fixed Bitrate: Providing only one high-quality version of the video. Users on 3G networks will never be able to watch it.

Recap

  • Transcoding and Chunking are the foundation of modern streaming.
  • Adaptive Bitrate Streaming (ABR) provides a smooth user experience.
  • CDNs are the most critical component for global scale.
  • Object Storage is the cost-effective home for petabytes of video data.

Knowledge Check

Question 1 of 3
Q1Single choice

What is the purpose of 'Video Chunking'?

Previous