Introducing Dispatch

Netflix is pleased to announce the open-source release of our crisis management orchestration framework: Dispatch! Okay, but what is Dispatch? Put simply, Dispatch is: All of the ad-hoc things you’re doing to manage incidents today, done for you, and a bunch of other things you should’ve been doing, but have not had the time! Dispatch helps us effectively…

Quasi Experimentation at Netflix

Experimentation informs much of our decision making at Netflix. We design, analyze, and execute experiments with rigor so that we have confidence that the changes we’re making are the right ones for our members and our business. We have many years of experience running…

Netflix Cloud Security: Detecting Credential Compromise in AWS

Credential compromise is an important concern for anyone operating in the cloud. The problem becomes more obvious over time, as organizations continue to adopt cloud resources as part of their infrastructure without maintaining an accompanying capability to observe and react to these compromises…

Essential Suite — Artwork Producer Assistant

Introduction Netflix continues to invest in content for a global audience with a diverse range of unique tastes and interests. Correspondingly, the member experience must also evolve to connect this global audience to the content that most appeals to each of them. Images that represent…

Netflix OSS and Spring Boot — Coming Full Circle

In 2007, Netflix started on a long road towards fully operating in the cloud. Much of Netflix’s backend and mid-tier applications are built using Java, and as part of this effort Netflix engineering built several cloud infrastructure libraries and systems — Ribbon for load balancing, Eureka for service discovery, and…

Byte Down: Making Netflix’s Data Infrastructure Cost-Effective

Background on data efficiency At Netflix, we invest heavily in our data infrastructure which is composed of dozens of data platforms, hundreds of data producers and consumers, and petabytes of data. At many other organizations, an effective way to manage data infrastructure costs is to…

Introducing SVT-AV1: a scalable open-source AV1 framework

Netflix Headquarters, Winchester Circle. Netflix headquarters circa 2014. It’s a nice building with good architecture! This was the primary home of Netflix for a number of years during the company’s growth, but at some point Netflix had outgrown its home and needed more space. One approach to solve this problem would have been to…

Key Challenges with Quasi Experiments at Netflix

At Netflix, when we can’t run A/B experiments we run quasi experiments! We run quasi experiments with various objectives such as non-member experiments focusing on acquisition, member experiments focusing on member engagement, or video streaming experiments focusing on content delivery. Consolidating on one methodology…

Netflix at Spark+AI Summit 2018

A glimpse at Spark usage for Netflix Recommendations Apache Spark has been an immensely popular big data platform for distributed computing. Netflix has been using Spark extensively for various batch and stream-processed workloads. A substantial list of use cases for Spark computation come from the various applications in the domain of content recommendations and personalization…

ConsoleMe: A Central Control Plane for AWS Permissions and Access

By Curtis Castrapel, Patrick Sanders, and Hee Won Kim At AWS re:Invent 2020, we open sourced two new tools for managing multi-account AWS permissions and access. We’re very excited to bring you ConsoleMe (pronounced: kuhn-soul-mee), and its CLI utility, Weep…

How Our Paths Brought Us to Data and Netflix

by Julie Beckley & Chris Pham This Q&A provides insights into the diverse set of skills, projects, and culture within Data Science and Engineering (DSE) at Netflix through the eyes of two team members: Chris Pham and Julie Beckley. Photo from a team curling offsite — There’s us to the right! [Chris] Julie and I joined the Streaming DSE team at Netflix…

The End of Video Coding?

In the IEEE Signal Processing Magazine issue November 2006 article “Future of Video Coding and Transmission” Prof. Edward Delp started by asking the panelists “Is video coding dead? Some feel that, with the higher coding efficiency of the H.264/MPEG-4 . . . perhaps there is not much more to do…

AVIF for Next-Generation Image Coding

TL; DR We need an alternative to JPEG that a) is widely supported, b) has better compression efficiency and c) has a wider feature set. We believe AV1 Image File Format (AVIF) has the potential. Using the…

Unbundling Data Science Workflows with Metaflow and AWS Step Functions

tl;dr Today, we are releasing a new job scheduler integration with AWS Step Functions. This integration allows the users of Metaflow to schedule their production workflows using a highly available, scalable, maintenance-free service, without…

SVT-AV1: an open-source AV1 encoder and decoder

SVT-AV1 is an open-source AV1 codec implementation hosted on GitHub https://github.com/OpenVisualCloud/SVT-AV1/ under a BSD + patent license. As mentioned in our earlier blog post, Intel and Netflix have been collaborating on the SVT-AV1 encoder and decoder framework since August 2018. The teams have been working closely on SVT-AV1 development, discussing…

Optimized shot-based encodes for 4K: Now streaming!

Netflix has an ever-expanding collection of titles which customers can enjoy in 4K resolution with a suitable device and subscription plan. Netflix creates premium bitstreams for those titles in addition to the catalog-wide 8-bit stream profiles¹. Premium features comprise a…

Open-sourcing Polynote: an IDE-inspired polyglot notebook

We are pleased to announce the open-source launch of Polynote: a new, polyglot notebook with first-class Scala support, Apache Spark integration, multi-language interoperability including Scala, Python, and SQL, as-you-type autocomplete, and more. Polynote provides data scientists and machine learning researchers with a notebook environment…

Extending Vector with eBPF to inspect host and container performance

Improving the tools available to our engineers to help them diagnose, triage, and work through software performance challenges in the cloud is a key goal for the cloud performance engineering team at Netflix. Today we are excited to announce latency…

Keeping Customers Streaming — The Centralized Site Reliability Practice at Netflix

We’re privileged to be in the business of bringing joy to our customers at Netflix. Whether it’s a compelling new series or an innovative product feature, we strive to provide a best-in-class service that people love and can enjoy anytime, anywhere…

Netflix SIRT releases Diffy: A Differencing Engine for Digital Forensics in the Cloud

Can you spot the difference? Hint: it’s not the bow tie. The Netflix Security Intelligence and Response Team (SIRT) announces the release of Diffy under an Apache 2.0 license. Diffy is a triage tool to help digital forensics and incident response (DFIR) teams quickly identify compromised hosts on which to…

Packaging award-winning shows with award-winning technology

Introduction In previous blog posts, our colleagues at Netflix have explained how 4K video streams are optimized, how even legacy video streams are improved and more recently how new audio codecs can provide better aural experiences to our members. In all these cases, prior to being delivered through…

Introducing Netflix Timed Text Authoring Lineage

By: Bhanu Srikanth, Andy Swan, Casey Wilms, Patrick Pearson The Art of Dubbing and Subtitling Dubbing and subtitling are inherently creative processes. At Netflix, we strive to make shows as joyful to watch in every language as in the original language, whether a member watches with original or dubbed audio, closed captions…

Implementing the Netflix Media Database

How Netflix brings safer and faster streaming experience to the living room on crowded networks…

At Netflix, we are obsessed with the best streaming experiences. We want playback to start instantly and to never stop unexpectedly in any network environment. We are also committed to protecting users’ privacy and service security without sacrificing any part of the playback experience. To achieve that…

Engineers Making Movies (AKA Open Source Test Content)

If you’re a die-hard Netflix fan, you may have already stumbled onto the dark and brooding story of a timeless struggle between man and nature, water and fountain, ball and tether known much more plainly as Example Short. In the good old days of…