Detecting image similarity using Spark, LSH and TensorFlow

As a visual platform, the ability to learn from images to understand our content is important. In order to detect near-duplicate images we use the NearDup system, a Spark- and TensorFlow-based pipeline. At the core of the pipeline is a Spark implementation of batch…

How Pinterest fights misinformation, hate speech, and self-harm content with machine learning

Pinterest is an inspiration network where people come for positivity and new ideas, and there’s nothing inspiring about harmful content. As a visual discovery engine with hundreds of billions of Pins…

Faster Creator Content Distribution at Pinterest

At Pinterest, we are committed to building the best experience for creators to reach and inspire new audiences and for Pinners to easily discover the best ideas for them. We believe more diverse Pin formats and an efficient distribution system that builds a…

Virtual onboarding in the time of COVID-19: The new hire’s new normal

Cindy Xinyi Zhang | Data Scientist, Experimentation and Metrics Science On March 2nd, exactly one week before my full-time position at Pinterest started…

A Pinterest Engineering guide to technical interviews

Technical interviews are the key component to landing an engineering role. Everything from preparation to asking the right questions is important for a successful interview. In this post, Pinterest employees from Recruiting and Engineering share tips and tricks for…

Optimizing Kafka for the cloud

Takeaways Developing locality aware systems and balancing algorithms can help substantially reduce cost Making Kafka Producer and Consumer rack aware helps efficiently route traffic Data-driven engineering decision making is critical to continuously improve systems One of the fundamental principles when operating in the…

La evolución de la búsqueda en Pinterest

This blog article was originally posted in English. To read the English version click here. Cuando Pinterest se lanzó por primera vez, ofrecía una nueva forma de recopilar y organizar ideas de toda la web. Sin embargo, lo que comenzó como un lugar…

Using machine learning to auto-organize boards

While planning one’s life might look different these days, our insights tell us people are saving ideas for the future more than ever. Last month, for instance, Pinterest had a 60% increase in the number…

Supporting React Native at Pinterest

Early last year, a small group of engineers started investigating the possibility of adopting React Native at Pinterest. We’ve been following the ecosystem with interest since it was first open-sourced in 2015. Since then, a robust community has developed around it, and it’s been…

API Profiling at Pinterest

When I walked into Pinterest on the first day of my internship and learned I’d be focusing on profiling the API Gateway service — the core backend service of the Pinterest product — my only thought was “What is profiling?”. Profiling is often shoved aside…

Understanding Pins through keyword extraction

Pinterest is well known for its visual discovery engine, but much of our content is also accompanied by text. As a result, it’s important for us to be able to understand text to ensure we recommend relevant ideas to Pinners. In this blog post…

Goku: Building a scalable and high performant time series database system

Co-authored by Jinghan Xu, Jian Wang & Tian-Ying Chang, Engineers At Pinterest, developers rely on Statsboard to monitor their systems and discover issues. A reliable and efficient monitoring system is very important for development velocity. Historically, we’ve used OpenTSDB to ingest and…

Building a more inclusive way to search

Every day, tens of millions of people search for ideas on Pinterest, whether it’s a recipe to cook for dinner that night or a new hairstyle to try over the weekend. According to a Pinterest study, 70 percent of people use Pinterest to…

Improving HBase backup efficiency at Pinterest

Pinterest has one of the largest HBase production deployments in the industry. HBase is one of the foundational building blocks of our infrastructure and powers many of our critical services including our graph database (Zen), our general-purpose key-value store (UMS), our…

How ads indexing works at Pinterest

Introduction Ads indexing is a data processing pipeline that builds servable ad documents for the ads delivering stack at Pinterest, including ads targeting, retrieval, ranking and auction. The pipeline converts ad campaigns in the raw format to the servable format via a heavy computational…

Secret management in multi-tenant environments

Knox, Pinterest’s open-source secret management service, now supports SPIFFE x509 identity documents as an authentication method. This update enables Knox to manage secrets in multi-tenant environments like Kubernetes. Knox is widely used across Pinterest’s infrastructure to provision secrets into production systems. In an…

Setting Up Snowflake Secondary Roles with Tableau

At Pinterest, we heavily utilize Tableau for our enterprise analytics. In an effort to secure access of sensitive data to the appropriate business units, we moved the enterprise data into a Snowflake data warehouse. Using Snowflake as a data platform, we can create and manage…

Unifying visual embeddings for visual search at Pinterest

Throughout the years, Pinterest has launched a variety of visual search products powered by computer vision to help people discover new ideas to try and products to buy. We started in 2015 with our visual cropping tool, allowing users to search within…

Administrar videos en Android

La función de video se lanzó en la aplicación de Pinterest para Android en 2016 a fin de ofrecer una experiencia de video completamente integrada en la aplicación. Esto incluía la capacidad de admitir varios videos por pantalla…

Knit Con through an employee’s eyes

Originally published on Tobi’s personal blog. It’s not every day I get to spend two days with my coworkers making arepas, practicing Tai chi, learning how to plate dishes, and vibing to six different DJs. Two days immersing myself in the deepest caverns of…

Real-time User Signal Serving for Feature Engineering

More than 320 million unique visitors come to Pinterest every month to discover inspiring content, which also gives advertisers a unique opportunity to get their products in front of people with commercial intent. Curation…

The Dead Code Society

On June 1, 2018, engineer Chris Lloyd created the #dead-code-society Slack channel to be “a sombre place for remembering [---]-only diffs.” Messages are prefixed by a :tombstone: and include links to diffs that contain only file or line deletions. The channel averages a little over one message per day, but they frequently come in bursts as part of refactoring projects and clean-up efforts. As Pinterest has grown as a company, our software unpinning has grown with it. We have hundreds of software engineers producing thousands of changes per week. The net result is an upward trend in total lines of code with a significant portion of our software having been (re)written in just the last few years. The #dead-code-society has helped balance that trend by celebrating the retirement of code from active service. Retired code remains in source control history to be remembered and consulted, but it no longer occupies a prominent place in our workspaces, build systems, or cognitive periphery. Less is more. Carpe diem.

How Pinterest runs Kafka at scale

Pinterest runs one of the largest Kafka deployments in the cloud. We use Apache Kafka extensively as a message bus to transport data and to power real-time streaming services, ultimately helping more than 250 million Pinners around the world discover and do what…

Building a dynamic and responsive Pinterest

In 2015, the majority of content on Pinterest was pregenerated for users prior to login. It was stored statically in HBase and served directly upon entering the service. (More details can be found in the blog post Building a smarter home feed.) …

Refactoring Thrift schemas at Pinterest

The Pinterest tech stack consists of hundreds of microservices written in a variety of languages that use Apache Thrift to communicate with each other. Thrift schemas are used to define our service interfaces, constants and serialized data across all levels of the stack…