Netflix at Spark+AI Summit 2018

A glimpse at Spark usage for Netflix Recommendations Apache Spark has been an immensely popular big data platform for distributed computing. Netflix has been using Spark extensively for various batch and stream-processed workloads. A substantial list of use cases for Spark computation come from the various applications in the domain of content recommendations and personalization…

How Netflix microservices tackle dataset pub-sub

Introduction In a microservice architecture such as Netflix’s, propagating datasets from a single source to multiple downstream destinations can be challenging. These datasets can represent anything from service configuration to the results of a batch job, are often needed in-memory to optimize access and must be updated as…

Auto Scaling Production Services on Titus

Over the past three years, Netflix has been investing in container technology. A large part of this investment has been around Titus, Netflix’s container management platform that was open sourced in April of 2018. Titus schedules application containers…

Edge Authentication and Token-Agnostic Identity Propagation

As most developers can attest, dealing with security protocols and identity tokens, as well as user and device authentication, can be challenging. Imagine having multiple protocols, multiple tokens, 200M+ users…

Open Sourcing Zuul 2

Simplifying Media Innovation at Netflix with Archer

Imagine you are developing a quick prototype to sift through all the frames of the movie Bright to find the best shots of Will Smith with an action-packed background. Your goal is to get the computer vision…

Netflix at the Spinnaker Summit 2018

Since releasing Spinnaker as an open source project in late 2015, the Spinnaker user and contributor community has grown by leaps and bounds. Our strong partnership with Google and other companies has led to a plethora of innovations and features that enable rapid, safe, and reliable delivery…

Full Cycle Developers at Netflix

Re-Architecting the Video Gatekeeper

This is the story about how the Content Setup Engineering team used Hollow, a Netflix OSS technology, to re-architect and simplify an essential component in our content pipeline — delivering a large amount of business value in the process. The Context Each movie and show on the Netflix service is carefully curated…

Remote Workstations for the Discerning Artists

Netflix is poised to become the world’s most prolific producer of visual effects and original animated content. To meet that demand, we need to attract the world’s best artistic talent. Artists like to work at places where they can create groundbreaking entertainment instead of worrying about getting…

GraphQL Search Indexing

Almost a year ago we described our learnings from adopting GraphQL on the Netflix Marketing Tech team. We have a lot more to share since then! There are plenty of existing resources describing how to express a search query in GraphQL and paginate…

Byte Down: Making Netflix’s Data Infrastructure Cost-Effective

Background on data efficiency At Netflix, we invest heavily in our data infrastructure which is composed of dozens of data platforms, hundreds of data producers and consumers, and petabytes of data. At many other organizations, an effective way to manage data infrastructure costs is to…

ML Platform Meetup: Infra for Contextual Bandits and Reinforcement Learning

Infrastructure for Contextual Bandits and Reinforcement Learning — theme of the ML Platform meetup hosted at Netflix, Los Gatos on Sep 12, 2019. Contextual and Multi-armed Bandits enable faster and adaptive alternatives to traditional A/B Testing. They enable rapid learning and better decision-making for product rollouts. Broadly speaking, these approaches…

Netflixにおける日本語字幕の導入

Netflixでは、2015年9月の日本における配信サービス開始時から日本語字幕を提供して …

Hawkins: Diving into the Reasoning Behind our Design System

by Hawkins team member Joshua Godi; with cover art from Martin Bekerman and additional imagery from Wiki Chaves Hawkins may be the name of a fictional town in Indiana, most widely known as the backdrop for one of Netflix’s most popular TV series “Stranger Things,” but the name is so…

Open-sourcing Polynote: an IDE-inspired polyglot notebook

We are pleased to announce the open-source launch of Polynote: a new, polyglot notebook with first-class Scala support, Apache Spark integration, multi-language interoperability including Scala, Python, and SQL, as-you-type autocomplete, and more. Polynote provides data scientists and machine learning researchers with a notebook environment…

Packaging award-winning shows with award-winning technology

Introduction In previous blog posts, our colleagues at Netflix have explained how 4K video streams are optimized, how even legacy video streams are improved and more recently how new audio codecs can provide better aural experiences to our members. In all these cases, prior to being delivered through…

Part 2: Scheduling Notebooks at Netflix

At Netflix we’ve put substantial effort into adopting notebooks as an integrated development platform. The idea started as a discussion of what development and collaboration interfaces might look like in the future. It evolved into a strategic bet on notebooks, both as…

Mythbusting the Analytics Journey

by Alex Diamond This Q&A aims to mythbust some common misconceptions about succeeding in analytics at a big tech company. This isn’t your typical recruiting story. I wasn’t actively looking for a new job and Netflix was the only place I applied. I didn’t know anyone who worked there and just submitted my resume through the Jobs page 🤷🏼‍♀️ . I wasn’t even entirely sure what…

Netflix SIRT releases Diffy: A Differencing Engine for Digital Forensics in the Cloud

Can you spot the difference? Hint: it’s not the bow tie. The Netflix Security Intelligence and Response Team (SIRT) announces the release of Diffy under an Apache 2.0 license. Diffy is a triage tool to help digital forensics and incident response (DFIR) teams quickly identify compromised hosts on which to…

MezzFS — Mounting object storage in Netflix’s media processing platform

MezzFS (short for “Mezzanine File System”) is a tool we’ve developed at Netflix that mounts cloud objects as local files via FUSE. It’s used extensively in our media processing platform, which includes services like Archer and runs features like…

Computational Causal Inference at Netflix

Every Netflix data scientist, whether their background is from biology, psychology, physics, economics, math, statistics, or biostatistics, has made meaningful contributions to the way Netflix analyzes causal effects. Scientists from these fields have made many advancements in causal effects research in the past few decades, spanning…

Engineering to Improve Marketing Effectiveness (Part 2) — Scaling Ad Creation and Management

In the first part of this series of blogs, we described our philosophy, motivations, and approach to blending ad technology into our marketing. In addition, we laid out some of the engineering undertakings to solve creative development and localization at scale. In this Part-2…

HDMI — Scaling Netflix Certification

At Netflix, we take the task of preserving the creative vision of our content all the way to a subscriber TV screen very seriously. This significantly increases the scope of our application integration and certification processes for streaming devices like set-top-boxes (STBs) and…

Netflix Studio Engineering Overview

Netflix is revolutionizing the way a modern studio operates. Our mission in Studio Engineering is to build a unified, global, and digital studio that powers the effective production of amazing content. Studio Engineering at Netflix …