How Video Compression Works
Articles Blog

How Video Compression Works

October 17, 2019


Have you ever thought about how video streaming
is possible? Let’s think about how big a typical 1080p
video is: 1920×1080 pixels, 24-bits each, 30 frames per second… That’s almost 1.5 gigabits per second. How can you transmit that much data, over
the air, in real time? The answer is video compression. You might have heard of codecs. A codec is a piece of software that encodes
and decodes data. The encoding part compresses the data, making
it easier to store and transmit. And the decoding part reverses this process,
recreating the original data as closely as possible. Use of codecs is not limited to video, they
can be used to encode and decode many types of signals. But for now, let’s focus on how video codecs
work. In the previous video, we talked about how
still images are compressed. In short, images are compressed by throwing
out the information that is less visible to the human eye and storing redundant data more
efficiently. We can easily extend image compression to
video compression by compressing a video frame by frame. This approach is called spatial or intra-frame
coding. Even doing that alone would significantly
reduce the file size, but we can actually do much more than that. In a typical video, many consecutive frames
tend to be nearly identical. We can make use of this temporal, inter-frame
redundancy to further compress a video. First, let’s think of an extreme case where
nothing moves in a video. Instead of storing each one of these identical
frames we can simply tell our encoder to keep the first frame and repeat it N times. That would save us a lot of space. Now let’s think of a more realistic case where
only some parts in the video don’t change. This time we can do the same thing, but more
locally, by dividing the frames into blocks and repeating only the blocks that don’t change. What if all blocks change between consecutive
frames but some change a lot and some change a little? Instead of checking whether a block has changed
or not we can search for a given block in the next frame within a neighborhood. This process is called block motion estimation. How does this help with compression? Well, instead of saving every frame, we can
save a reference frame and the motion vectors for the blocks. The motion vectors tell us how we should move
the blocks to closely match the next frames. This is called motion compensation. Although motion compensation can reduce the
difference between two consecutive frames greatly, it is usually not enough by itself
to fully recreate the next frame. So, in addition to the motion vectors, we
should also save the frame differences between the actual and motion compensated frames. These differences are known as residual frames. When it’s time to play this video, the decoder
predicts the current frame by taking the previous reference frame, compensating for the motion
using the motion vectors, and adding the residual frame. You might ask couldn’t we just save the original
frames instead of saving the residual frames? We could, but residual frames have much less
information than the full reference frames. Therefore they are highly compressible. Let’s overview the entire process. Traditional video compression algorithms represent
a video as a sequence of reference frames followed by residual frames. There are two types of compression done here:
intra-frame coding and inter-frame. In this video, I focused mostly on the inter-frame
coding part which achieves a high compression efficiency by exploiting the similarities
between consecutive frames. Intra-frame coding, on the other hand, compresses
a frame by throwing out visually redundant information within the frame and storing the
rest more efficiently. Check out my previous video to learn more
about how this is done. The methods I covered here are the very basics
that are used by many codecs including the mainstream H.264 codec, which is also known
as MPEG-4 AVC. Modern video codecs, including H.264, H.265,
and VP9, use sophisticated methods to balance the level of compression and perceptual image
quality without introducing too much computational complexity. Although video compression algorithms we use
today are pretty mature, video compression is still an active area of research. Researchers have already been experimenting
with machine learning models that have the potential to perform better than today’s block-based
hybrid encoding standards. It’s not easy to beat today’s encoding standards
since they had decades to mature and to be tuned in many possible ways. I still think that an end to end trainable
codec will eventually outperform the traditional compression methods, by optimizing perceptual
image quality while minimizing the file size. That’s all for today! I hope you liked it. If you have any comments or questions, let
me know in the comments section below. Subscribe for more videos. As always, thanks for watching, stay tuned,
and see you next time.

Only registered users can comment.

  1. This guy's gonna have the surprise of his life when he sees his video suddenly hit 100k views… Kudos on a nice video though!

  2. Congratulations brother your channel is about to BOOST! Thanks to MKBHD for putting your link in his video. As this channel is really good and educational you will definately gain more subs!

  3. No one: ….

    Literally Nobody:

    MKBHD: ight imma start this man’s whole career!!!

    no really dude that was a great video and that's coming from someone that religiously watches techquicke videos.
    all I can say is keep up the good work man, best wishes… btw subbed

  4. It was such a nice surprise to see that my video was recommended by Marques Brownlee on my wedding day. Thank you Marques and everyone who came here from his channel! This has been a great wedding gift.

  5. Your videos have remarkable quality. I especially love the ones on AI. Was looking for a good channel to learn about AI as well. I'm glad Marquess brought me here… You definitely earned my sub…

  6. SO I WAS SITTING IN A ROOM DIFFERENT FROM THE ONE U ARE IN NOW, and MKBHD got me to watch this vid, WITH THIS OTHER GUY SITTING IN A ROOM DIFFERENT FROM THE ONE MKB was inπŸ˜…πŸ˜…πŸ˜…πŸ˜…πŸ˜…πŸ˜‚πŸ˜‚πŸ˜‚

  7. wow, YouTube rolled back all the views from MKBHD: I saw the counter WELL above 100k, but now it's 15k.
    WTF?

  8. Great video man! You deserve much much more views and like. Keep up your good work. Also congratulations for your wedding πŸ™‚

  9. What a gem of a video. Thanks for making this and thanks to MKBHD for promoting this.
    could you make a video on HEVC encoding and why this was such a recent invention? It seems to save a ton of data without much loss in quality

  10. Thank you MKBHD. This is a serious guy! And by the way most of the techniques described in this video aren't from computer science, they were used back in the 70s and 80s by animation artists (by hand) to produce animation series.

  11. Tbh Marques was right–this is a very clear and concise explanation of video compression. It even introduces technical features like I-frames

  12. "Without introducing too much computational complexity." You haven't tried to compress a 4k HDR iso using the x265 at placebo quality.

  13. Awesome explainer. I knew some of how this worked in theory but had never seen such a straightforward explanation. Cheers!

  14. I actually knew one or two things, about compression but this actually is a lot more. Thanks for this, really didn't expect such good content , mkbhd gang btw

Leave a Reply

Your email address will not be published. Required fields are marked *