Improving data processing efficiency using partial deserialization of Thrift

At Pinterest we’ve worked to greatly improve data processing efficiency. One quote that resonates with our unique approach is from writer Antoine de Saint-Exupéry: “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” Ultimately…

The top Pinterest Engineering blog posts of 2018

The most-read posts of the year Three-day no-meeting schedule for engineers A one year PWA retrospective Supporting React Native How Pinterest runs Kafka at scale How switching our domain structure unlocked international growth PinSage: A new graph convolutional neural network for web-scale recommender systems Looking inside the technology that powers Pinterest Building a more inclusive way to search Building a real-time user action counting system for ads Goku: Building a scalable and high performant time series database system We’ll be back in 2019 with more updates from the team. Happy New Year!

Looking inside the technology that powers Pinterest

Pinterest began as a small startup, and has grown to a company that serves +175B Pins to +250M people. While we have one of the largest datasets online, we have 600+ engineers, and so technology needs to be accessible for each person to play their part in…

Understanding Pins through keyword extraction

Pinterest is well known for its visual discovery engine, but much of our content is also accompanied by text. As a result, it’s important for us to be able to understand text to ensure we recommend relevant ideas to Pinners. In this blog post…

Building Expanded Targeting for Pinterest Ads

With more than 300 million people coming to Pinterest each month, Pinterest offers unique opportunities for advertisers to get their products in front of people with commercial intent. Similar to other platforms, Pinterest offers a variety of advertising targeting products¹. Advertisers can…

Improving HBase backup efficiency at Pinterest

Pinterest has one of the largest HBase production deployments in the industry. HBase is one of the foundational building blocks of our infrastructure and powers many of our critical services including our graph database (Zen), our general-purpose key-value store (UMS), our…

Growth engineering meetup

Synopsis of the talks Paid growth @ Airbnb Ganesh Venkataraman discusses the various channels and techniques…

Refactoring Thrift schemas at Pinterest

The Pinterest tech stack consists of hundreds of microservices written in a variety of languages that use Apache Thrift to communicate with each other. Thrift schemas are used to define our service interfaces, constants and serialized data across all levels of the stack…

Privacy Power-Up: Upgrading Pin Links from HTTP to HTTPS

HTTPS should be everywhere. Your website likely runs over HTTPS, giving your users a secure, encrypted connection, but what about all the outbound links? Can you guarantee that they all use HTTPS? In this post, we describe the steps we took…

How ads indexing works at Pinterest

Introduction Ads indexing is a data processing pipeline that builds servable ad documents for the ads delivering stack at Pinterest, including ads targeting, retrieval, ranking and auction. The pipeline converts ad campaigns in the raw format to the servable format via a heavy computational…

Three-day no-meeting schedule for engineers

About 100 days ago, our product engineering team began experimenting with a three-day no-meeting schedule for individual contributors on our teams. It’s not a big revelation that software development requires long stretches of uninterrupted time to focus. As Pinterest has grown…

Using graph algorithms to optimize Kafka operations, Part 1

Pinterest uses Kafka across the board for data movement. The Logging Platform team alone manages over 3,000 Kafka brokers transporting trillions of messages per day to help bring more than 320 million monthly active Pinners the inspiration to create a life they love. …

Automated cluster management and recovery for Rocksplicator

Rocksplicator at Pinterest Rocksplicator (github) was open sourced by Pinterest one and half years ago. It was designed to solve common problems of building large-scale online stateful data systems on commodity hardware, such as RocksDB replication and request routing. Technical details about how Rocksplicator solves…

Seven best practices for inclusive product design

At Pinterest, our mission is to help you discover and do what you love. We’re building a product for everyone, because the best Pinterest is inclusive and easily usable for people with all levels of abilities. That’s why today we’re sharing our…

Interest Taxonomy: A knowledge graph management system for content understanding at Pinterest

Interest Taxonomy at Pinterest We recently began rolling out the beta version of Pinterest Trends, a new tool that gives a view of the top U.S. search terms on Pinterest within the past 12 months, and a look into when search terms peak for…

PinalyticsDB: A Time Series Database on top of Hbase

PinalyticsDB is Pinterest’s proprietary time series database. At Pinterest, we rely on PinalyticsDB as a backend for storing and visualizing thousands of time series reports such as the sample case below, segmented by country. PinalyticsDB was built several years ago on…

Setting Up Snowflake Secondary Roles with Tableau

At Pinterest, we heavily utilize Tableau for our enterprise analytics. In an effort to secure access of sensitive data to the appropriate business units, we moved the enterprise data into a Snowflake data warehouse. Using Snowflake as a data platform, we can create and manage…

Contextual relevance in ads ranking

Pinterest runs an ads marketplace that balances value for Pinners, partners, and the business. We initially used engagement prediction as our main method to balance the marketplace before eventually adding contextual relevance to increase Pinner value. We have three main surfaces for determining…

Hybrid Search: Building a textual and visual discovery experience at Pinterest

As a visual discovery engine, Pinterest helps you discover ideas even when you don’t know what you’re looking for, or have the words to describe it. Using a mix of technology (including visual search) and community input (thanks to the 175B+ Pins…

Using machine learning to auto-organize boards

While planning one’s life might look different these days, our insights tell us people are saving ideas for the future more than ever. Last month, for instance, Pinterest had a 60% increase in the number…

Building Pin cohesion

At Pinterest, we’re building a visual discovery engine where ideas become actionable with links to more information. Once a Pin is saved, its linked content may be updated or expire over time, and so it’s critical to know…

Cómo redujimos el tamaño de la aplicación de Pinterest para iOS en más de un 30% / 50 MB

This blog was originally published in English. To read the English version, click here. Todos sabemos que el tamaño de las aplicaciones es importante (el tamaño de descarga[1] y el tamaño de instalación local[2]). Además, existe una correlación…

Partitioning Pinlater for Fun(?) and Profit(!)

Contained within the November 2015 announcement of the open-sourcing of Pinlater, our asynchronous job execution system, was a description of some of the improvements that had been made to the system since the original 2014 blog post describing its creation. Notable among…

Supporting React Native at Pinterest

Early last year, a small group of engineers started investigating the possibility of adopting React Native at Pinterest. We’ve been following the ecosystem with interest since it was first open-sourced in 2015. Since then, a robust community has developed around it, and it’s been…

Real-time User Signal Serving for Feature Engineering

More than 320 million unique visitors come to Pinterest every month to discover inspiring content, which also gives advertisers a unique opportunity to get their products in front of people with commercial intent. Curation…