Hawkins: Diving into the Reasoning Behind our Design System

by Hawkins team member Joshua Godi; with cover art from Martin Bekerman and additional imagery from Wiki Chaves Hawkins may be the name of a fictional town in Indiana, most widely known as the backdrop for one of Netflix’s most popular TV series “Stranger Things,” but the name is so…

Optimizing data warehouse storage

Background At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3, and each day we ingest and create additional Petabytes. At this scale, we can gain a significant amount of performance and cost benefits by optimizing the storage layout (records, objects, partitions)…

Open Sourcing the Netflix Domain Graph Service Framework: GraphQL for Spring Boot

Netflix has developed a Domain Graph Service (DGS) framework and it is now open source. The DGS framework simplifies the implementation of GraphQL, both for standalone and federated GraphQL services. Our framework is battle-hardened by our…

Lessons from Building Observability Tools at Netflix

Netflix at MIT CODE 2020

In November, Netflix was a proud sponsor of the 2020 Conference on Digital Experimentation (CODE), hosted by the MIT Initiative on the Digital Economy. As well as providing sponsorship, Netflix data scientists were active participants, with three contributions. Eskil Forsell and colleagues presented a poster describing Success stories…

Packaging award-winning shows with award-winning technology

Introduction In previous blog posts, our colleagues at Netflix have explained how 4K video streams are optimized, how even legacy video streams are improved and more recently how new audio codecs can provide better aural experiences to our members. In all these cases, prior to being delivered through…

How Netflix uses eBPF flow logs at scale for network insight

Netflix has developed a network observability sidecar called Flow Exporter that uses eBPF tracepoints to capture TCP flows at near real time. At much less than 1% of CPU and memory on the instance, this highly performant sidecar provides…

Open Sourcing Zuul 2

Detecting Performance Anomalies in External Firmware Deployments

Netflix has over 139M members streaming on more than half a billion devices spanning over 1,700 different types of devices from hundreds of brands. This diverse device ecosystem results in a high dimensionality feature space, often with sparse data, and can make identifying device performance issues challenging…

AVIF for Next-Generation Image Coding

TL; DR We need an alternative to JPEG that a) is widely supported, b) has better compression efficiency and c) has a wider feature set. We believe AV1 Image File Format (AVIF) has the potential. Using the…

Empowering the Visual Effects Community with the NetFX Platform

At Netflix, we want to entertain our global membership with series and films from around the world. In line with that, we’re excited to announce NetFX, a cloud-based platform that will make it easier for…

Production Media Management: Transforming Media Workflows by leveraging the Cloud

In this post, we will share a behind-the-scenes look at how Netflix delivers technology and infrastructure to help production…

Android Rx onError Guidelines

“Creating a good API is hard.” — anyone who has created an API used by others As with any API, wrapping your data stream in a Rx observable requires consideration for reasonable error handling and intuitive behavior. The following guidelines are intended to help developers create consistent…

Evolution of Netflix Conductor:

By Anoop Panicker and Kishore Banala Conductor is a workflow orchestration engine developed and open-sourced by Netflix. If you’re new to Conductor, this earlier blogpost and the documentation should help you get started and acclimatized to Conductor. Netflix Conductor: A microservices orchestrator visibility into distributed workflowsmedium.com In the last two years since inception, Conductor has seen wide adoption and…

Netflix Drive

Written by Vikram Krishnamurthy, Kishore Kasi, Abhishek Kapatkar, Tejas Chopra, Prudhviraj Karumanchi, Kelsey Francis, Shailesh Birari In this post, we are introducing Netflix Drive, a Cloud drive for media assets and providing a high level overview of some of its features and interfaces. We intend this to be a first…

Key Challenges with Quasi Experiments at Netflix

At Netflix, when we can’t run A/B experiments we run quasi experiments! We run quasi experiments with various objectives such as non-member experiments focusing on acquisition, member experiments focusing on member engagement, or video streaming experiments focusing on content delivery. Consolidating on one methodology…

Essential Suite — Artwork Producer Assistant

Introduction Netflix continues to invest in content for a global audience with a diverse range of unique tastes and interests. Correspondingly, the member experience must also evolve to connect this global audience to the content that most appeals to each of them. Images that represent…

Introducing SVT-AV1: a scalable open-source AV1 framework

Netflix Headquarters, Winchester Circle. Netflix headquarters circa 2014. It’s a nice building with good architecture! This was the primary home of Netflix for a number of years during the company’s growth, but at some point Netflix had outgrown its home and needed more space. One approach to solve this problem would have been to…

Engineering to Improve Marketing Effectiveness (Part 2) — Scaling Ad Creation and Management

In the first part of this series of blogs, we described our philosophy, motivations, and approach to blending ad technology into our marketing. In addition, we laid out some of the engineering undertakings to solve creative development and localization at scale. In this Part-2…

Beyond Interactive: Notebook Innovation at Netflix

Notebooks have rapidly grown in popularity among data scientists to become the de facto standard for quick prototyping and exploratory analysis. At Netflix, we’re pushing the boundaries even further, reimagining what a notebook can be, who can use it, and…

Netflix SIRT releases Diffy: A Differencing Engine for Digital Forensics in the Cloud

Can you spot the difference? Hint: it’s not the bow tie. The Netflix Security Intelligence and Response Team (SIRT) announces the release of Diffy under an Apache 2.0 license. Diffy is a triage tool to help digital forensics and incident response (DFIR) teams quickly identify compromised hosts on which to…

Introducing Dispatch

Netflix is pleased to announce the open-source release of our crisis management orchestration framework: Dispatch! Okay, but what is Dispatch? Put simply, Dispatch is: All of the ad-hoc things you’re doing to manage incidents today, done for you, and a bunch of other things you should’ve been doing, but have not had the time! Dispatch helps us effectively…

How Netflix uses Druid for Real-time Insights to Ensure a High-Quality Experience

Ensuring a consistently great Netflix experience while continuously pushing innovative technology updates is no easy feat. How can we be confident that updates are not harming our users? And that we’re actually making measurable improvements when we intend to? Using real-time logs from playback devices as a…

Streaming Video Experimentation at Netflix: Visualizing Practical and Statistical Significance

Streaming video experimentation at Netflix seeks to optimize the Quality of Experience (QoE for short) of the videos we stream to our 130 million members around the world. To measure QoE, we look at a wide variety of metrics for each playback session, including play delay; the…

Evolving Regional Evacuation

At Netflix we prioritize innovation and velocity in pursuit of the best experience for our 150+ million global customers. This means that our microservices constantly evolve and change, but what doesn’t change is our responsibility to provide a highly available service that delivers 100+ million hours of daily streaming to…