http://www.collegehumor.com/theinternet
http://www.google.com/about/datacenters/gallery/
http://www.google.com/about/datacenters/gallery/

Timeline

  • September 24, 2009 - Adobe showed us a project called Zeri
  • Sometime in early 2010 - Working Zeri prototype
  • July 11, 2011 - Aaron Colwell shows of v0.1 of Media Source
  • April 24, 2012 - Using Zeri for all capable playbacks
  • September of 2012 - Start shipping an Adaptive Streaming in Flash
  • February of 2013 - 100% of playbacks are Adaptive
  • October 24, 2012 - First Media Source playbacks
  • April 9, 2013 - Media Source Extensions working draft

Chunked Video Playback

The best thing since Sliced Bread

The Past

      // Simple HTTP video streaming
      netStream.play(url)
      
sb

The New World

      // Simple video streaming
      source = new HttpByteSource(url)
      netStream.appendBytes(source.read())
      
sb

But Why?

  • Efficiency
  • Dumber Servers, Smarter Clients
  • Support New Video Containers
  • Adaptive Bitrate Streaming

Efficiency

  • Reduced Oversend by 5%
      // Simple chunked video streaming
      source = new ChunkByteSource(url?range=A-B)
      netStream.appendBytes(source.read())
      
sb

Caching - How does it work?

  • Browser Cache - Disk
  • In-Memory
  • Decoder

Caching - How does it work?

  • Browser Cache
  • In-Memory - Buffer Bar
  • Decoder

Caching - How does it work?

  • Browser Cache
  • In-Memory
  • Decoder - Decoded Frames

Dumber Servers, Smarter Clients

No longer a part of our servers

  • Server side throttling
  • "Pseudo-Streaming" via "start="

Now in our clients

  • Managed read-ahead
  • Client side seek indexing

Dumber Servers, Smarter Clients

No longer a part of our servers

  • Server side throttling
  • "Pseudo-Streaming" via "start="

Now in our clients

  • Managed read-ahead
  • Client side seek indexing

Dumber Servers, Smarter Clients

No longer a part of our servers

  • Server side throttling
  • "Pseudo-Streaming" via "start="

Now in our clients

  • Managed read-ahead
  • Client side seek indexing

Support New Video Containers

taco
This image is a work of the National Institutes of Health, part of the United States Department of Health and Human Services. As a work of the U.S. federal government, the image is in the public domain.
http://commons.wikimedia.org/wiki/File:NCI_Visuals_Food_Taco.jpg

Support New Video Containers

  • H.264
  • VP8
  • AAC
  • Vorbus
  • FLV
  • MP4
  • M2TS
  • Matroska
Meat - Patrick Truby
http://www.flickr.com/photos/patruby/5465944583/

Tortilla - Renee Comet (Photographer)
https://commons.wikimedia.org/wiki/File:Flour_tortilla.jpg

Support New Video Containers

How do we handle knowing the byte offset A for a time T?

  • FLV time to byte
  • MP4 moov atom
  • FMP4 sidx atom
  • DASH/HLS manifests
  • Build a common interface:
    • Container = (time, byte offset)
    • Samples = {codec, payload}

Anatomy of a Video

Adaptive Bitrate Streaming

Adaptive Bitrate Streaming

Adaptive Bitrate Streaming

How to Launch Adaptive Streaming at Scale

write_awesome_code();
knob_value = 0;
while (knob_value < 100) {
  identify_and_fix_problems();
  turn_knob(1);
  notice_new_problem();
  if (really_hurts) {
    turn_knob(-1);
  }
  drink();
  sleep();
}
party();
      

Problems

Problems

Problems

Problems

Baby - Chris Harrison
http://www.flickr.com/photos/cdharrison/280252512/

Servers - RRZEIcons
http://commons.wikimedia.org/wiki/File%3AServer-multiple.svg

Problems

Baby - Chris Harrison
http://www.flickr.com/photos/cdharrison/280252512/

Servers - RRZEIcons
http://commons.wikimedia.org/wiki/File%3AServer-multiple.svg

Problems

HTML5

A New Oven

HTML5 Becomes Competitive

HTML5 Becomes Competitive

GET /crossdomain.xml HTTP/1.1
Host: sn---o097nez.c.youtube.com
<cross-domain-policy>
  <allow-access-from domain="s.ytimg.com" />
  <allow-access-from domain="*.youtube.com" />
</cross-domain-policy>
GET /videoplayback HTTP/1.1
Host: sn---o097nez.c.youtube.com
...
HTTP/1.1 200 OK
...

HTML5 Becomes Competitive

Cross-Origin Resource Sharing

GET /videoplayback HTTP/1.1
Host: sn---o097nez.c.youtube.com
Origin: http://www.youtube.com
...
      
HTTP/1.1 200 OK
Access-Control-Allow-Origin: http://www.youtube.com
Access-Control-Allow-Credentials: True
...
      

Media Source Extensions

Media Source Extensions

V0.1 Draft Interface

constructor(in DOMString type)]
interface MediaSource {
  readonly attribute unsigned long bytesAvailable;

  bool append(in ArrayBuffer data);
  void endOfStream();
  ...
}
      

Media Source Extensions

The appendBytes-style model
doesn't work for the web.
Playing videos is hard.
Axiom 1 of YouTube Engineering

Optimization targets for lossy compression

  • Compression efficiency
  • Resolution scalability
  • Bitrate scalability
  • End-to-end latency
  • Signal fidelity
  • Perceptual fidelity
"Improve compression efficiency by 20%,
with no more than triple the decode complexity."
Adobe Flash targets consistency.
W3C standards target interoperability.

Media Source Extensions

Media Source Extensions

var sb = mediaSource.addSourceBuffer('video/webm; codecs="vp9"');
      

Media Source Extensions

var sb = mediaSource.addSourceBuffer('video/webm; codecs="vp9"');
sb.appendBuffer(mediaBuffer);
      

Media Source Extensions

var sb = mediaSource.addSourceBuffer('video/webm; codecs="vp9"');
sb.appendBuffer(mediaBuffer);
sb.appendBuffer(mediaBuffer2);
      

Media Source Extensions

Performing an Adaptive Upswitch

Media Source Extensions

Performing an Adaptive Upswitch

Always show the best video possible.

Media Source Extensions

Performing an Adaptive Upswitch

Media Source Extensions

Performing an Adaptive Upswitch

Media Source Extensions

Performing an Adaptive Upswitch

Media Source Extensions

Performing an Adaptive Upswitch

Application-Level Streaming

Sliced Bread is Good for You

Speed

Application-Level Streaming

Speed

Experiments

Application-Level Streaming

Experiments

Hundreds of experiments.
Billions of playbacks.

Regional Preference

Application-Level Streaming

Regional Preference

Application-Level Streaming

Regional Preference

Software has bugs.

New Features

Application-Level Streaming

New Features

  • Preloading
  • Fast-start
  • Bufferbloat avoidance
  • Global load balancing
  • Ubiquitous live-streaming

Application-Level Streaming

Thanks For Watching!