Optimizing Kafka for the cloud

Takeaways Developing locality aware systems and balancing algorithms can help substantially reduce cost Making Kafka Producer and Consumer rack aware helps efficiently route traffic Data-driven engineering decision making is critical to continuously improve systems One of the fundamental principles when operating in the…

Virtual onboarding in the time of COVID-19: The new hire’s new normal

Cindy Xinyi Zhang | Data Scientist, Experimentation and Metrics Science On March 2nd, exactly one week before my full-time position at Pinterest started…

User state-based notification volume optimization

Notifications play an important role in helping Pinterest’s 335M+ monthly active users discover and engage with inspiring content. It’s crucial to send the right number of notifications with high quality content. If we send too few notifications, Pinners might miss out on great opportunities…

Improving HBase backup efficiency at Pinterest

Pinterest has one of the largest HBase production deployments in the industry. HBase is one of the foundational building blocks of our infrastructure and powers many of our critical services including our graph database (Zen), our general-purpose key-value store (UMS), our…

PinalyticsDB: A Time Series Database on top of Hbase

PinalyticsDB is Pinterest’s proprietary time series database. At Pinterest, we rely on PinalyticsDB as a backend for storing and visualizing thousands of time series reports such as the sample case below, segmented by country. PinalyticsDB was built several years ago on…

Ad Metrics Quality and Testing on Mobile

Every day at Pinterest, we help our advertising partners reach Pinners through different ad formats, and we provide them with metrics and analytics on how their ads are performing. Maintaining our partners’ trust and delivering value is critical, so the metrics we expose…

One year in at Pinterest and looking ahead as we grow

A little over a year ago, I joined Pinterest as the SVP of Technology. The world looks different than I thought it would around my first “Pinniversary”, but the opportunities before us are more apparent and abundant than ever before. For me personally, I’m…

Faster Creator Content Distribution at Pinterest

At Pinterest, we are committed to building the best experience for creators to reach and inspire new audiences and for Pinners to easily discover the best ideas for them. We believe more diverse Pin formats and an efficient distribution system that builds a…

Building board archiving and sorting

Connor Montgomery | Software engineer, Product Engineering A Pinterest tradition we’ve had since our early days is the Make-a-thon, our twist on a hackathon. Make-a-thons bring together employees from different disciplines to build something new. In the past, we’ve had projects range from scaling hacks to AR prototypes to…

Keeping Related Pins fresh

If you’ve ever heard someone mention “the Pinterest Rabbit Hole,” they’re likely referring to Related Pins, the ideas you see below a Pin you’ve tapped. Related Pins is a feed of content relevant to the closeup Pin and accounts for 40 percent of engagement…

Simplifying web deploys

Over the past six months, Pinterest has moved to a CI/CD model for our API and web layers, which has truly improved agility by reducing time between merge and production. Prior to…

Pinterest Flink Deployment Framework

Background At Pinterest, stream processing allows us to unlock value from real time data for pinners and partners. The Stream Processing Platform team is working on building a reliable and scalable platform to support many critical streaming applications including real-time experiment analytics…

Automated cluster management and recovery for Rocksplicator

Rocksplicator at Pinterest Rocksplicator (github) was open sourced by Pinterest one and half years ago. It was designed to solve common problems of building large-scale online stateful data systems on commodity hardware, such as RocksDB replication and request routing. Technical details about how Rocksplicator solves…

Geo-blocking media content on Pinterest

Pinterest engineers are tasked with showing Pinners the best ideas related to their interests, which includes taking into account signals like location. As part of these efforts, we built a video and image platform to enable a…

Real-time User Signal Serving for Feature Engineering

More than 320 million unique visitors come to Pinterest every month to discover inspiring content, which also gives advertisers a unique opportunity to get their products in front of people with commercial intent. Curation…

Improving the Quality of Recommended Pins with Lightweight Ranking

The Pinterest corpus is composed of billions of Pins, however, each Pinner only sees a small subset based on their interests when…

Using machine learning to auto-organize boards

While planning one’s life might look different these days, our insights tell us people are saving ideas for the future more than ever. Last month, for instance, Pinterest had a 60% increase in the number…

How we increased engagement using automatic personalized copy testing

Think about the last time you visited a website. Was there a word that helped draw you in? Now think about how that might be different if you spoke a different language, were a different age or grew up in a different part of…

Supporting React Native at Pinterest

Early last year, a small group of engineers started investigating the possibility of adopting React Native at Pinterest. We’ve been following the ecosystem with interest since it was first open-sourced in 2015. Since then, a robust community has developed around it, and it’s been…

Privacy Power-Up: Upgrading Pin Links from HTTP to HTTPS

HTTPS should be everywhere. Your website likely runs over HTTPS, giving your users a secure, encrypted connection, but what about all the outbound links? Can you guarantee that they all use HTTPS? In this post, we describe the steps we took…

Secret management in multi-tenant environments

Knox, Pinterest’s open-source secret management service, now supports SPIFFE x509 identity documents as an authentication method. This update enables Knox to manage secrets in multi-tenant environments like Kubernetes. Knox is widely used across Pinterest’s infrastructure to provision secrets into production systems. In an…

En Pinterest desarrollamos compilaciones para iOS rápidas y confiables

En Pinterest nos enfocamos en ayudar a las personas a descubrir ideas inspiradoras, desde recetas para una cena, hasta productos de estilo y para el hogar que comprar y lugares a los que viajar. Para ello, es…

Using PID controllers to diversify content types on home feed

Every day millions of Pinners visit the home feed to find inspiration on Pinterest. As a member of the home feed ranking team, it’s my job to not only figure out what relevant pins to show Pinners but also to make sure that…

PinText: A Multitask Text Embedding System in Pinterest

Word embeddings have been actively studied and developed by researchers and practitioners in the machine learning community since the neural network language model was proposed. Researchers not only proposed principled algorithms and open-source code, but they also released pre-trained embedding models with public…

How we designed our Continuous Integration System to be more than 50% Faster

Earlier this year, the Engineering Productivity team at Pinterest published a blog called How a one line change decreased our clone times by 99%. In that post, we described how a simple Git configuration sped up clone times in one of our…