Hybrid Search: Building a textual and visual discovery experience at Pinterest

As a visual discovery engine, Pinterest helps you discover ideas even when you don’t know what you’re looking for, or have the words to describe it. Using a mix of technology (including visual search) and community input (thanks to the 175B+ Pins…

Secret management in multi-tenant environments

Knox, Pinterest’s open-source secret management service, now supports SPIFFE x509 identity documents as an authentication method. This update enables Knox to manage secrets in multi-tenant environments like Kubernetes. Knox is widely used across Pinterest’s infrastructure to provision secrets into production systems. In an…

Pinterest Flink Deployment Framework

Background At Pinterest, stream processing allows us to unlock value from real time data for pinners and partners. The Stream Processing Platform team is working on building a reliable and scalable platform to support many critical streaming applications including real-time experiment analytics…

PinText: A Multitask Text Embedding System in Pinterest

Word embeddings have been actively studied and developed by researchers and practitioners in the machine learning community since the neural network language model was proposed. Researchers not only proposed principled algorithms and open-source code, but they also released pre-trained embedding models with public…

A one year PWA retrospective

The idea of building a “Progressive Web App” (PWA) is not new, but its definition has changed with the emergence of key technologies like service workers. Now it’s finally possible to build great experiences in a mobile browser. Being an early adopter can…

Building a dynamic and responsive Pinterest

In 2015, the majority of content on Pinterest was pregenerated for users prior to login. It was stored statically in HBase and served directly upon entering the service. (More details can be found in the blog post Building a smarter home feed.) …

GokuL: Extending time series data storage to serve beyond one day

At Pinterest, developers rely heavily on Statsboard to monitor their systems and get alerted when issues happen. Last year, we introduced our time series database (aka Goku) which stores and serves the most recent one day’s…

User retention at Pinterest: tackling an ambiguous problem

User retention is an important but nebulous idea. It’s hard to acquire users and even harder to help them feel at home through an engaging and relevant experience. So when some of those users start to leave the platform, your first thought is…

Keeping Related Pins fresh

If you’ve ever heard someone mention “the Pinterest Rabbit Hole,” they’re likely referring to Related Pins, the ideas you see below a Pin you’ve tapped. Related Pins is a feed of content relevant to the closeup Pin and accounts for 40 percent of engagement…

Optimizing Kafka for the cloud

Takeaways Developing locality aware systems and balancing algorithms can help substantially reduce cost Making Kafka Producer and Consumer rack aware helps efficiently route traffic Data-driven engineering decision making is critical to continuously improve systems One of the fundamental principles when operating in the…

Ad Metrics Quality and Testing on Mobile

Every day at Pinterest, we help our advertising partners reach Pinners through different ad formats, and we provide them with metrics and analytics on how their ads are performing. Maintaining our partners’ trust and delivering value is critical, so the metrics we expose…

En Pinterest desarrollamos compilaciones para iOS rápidas y confiables

En Pinterest nos enfocamos en ayudar a las personas a descubrir ideas inspiradoras, desde recetas para una cena, hasta productos de estilo y para el hogar que comprar y lugares a los que viajar. Para ello, es…

Unifying visual embeddings for visual search at Pinterest

Throughout the years, Pinterest has launched a variety of visual search products powered by computer vision to help people discover new ideas to try and products to buy. We started in 2015 with our visual cropping tool, allowing users to search within…

Building Pin cohesion

At Pinterest, we’re building a visual discovery engine where ideas become actionable with links to more information. Once a Pin is saved, its linked content may be updated or expire over time, and so it’s critical to know…

How we use AutoML, Multi-task learning and Multi-tower models for Pinterest Ads

People come to Pinterest in an exploration mindset, often engaging with ads the same way they do with organic Pins. Within ads our mission is to help Pinners go from inspiration to action by introducing them to the compelling products and services that…

How we increased engagement using automatic personalized copy testing

Think about the last time you visited a website. Was there a word that helped draw you in? Now think about how that might be different if you spoke a different language, were a different age or grew up in a different part of…

HierTCN: Deep learning models for dynamic recommendations and inferring user interests

As we build a visual discovery engine, it’s crucial to understand user intent and serve relevant content through recommender systems that constantly learn from a cross-section of data to dynamically predict the next idea a Pinner will love. However, we found…

How Pinterest runs Kafka at scale

Pinterest runs one of the largest Kafka deployments in the cloud. We use Apache Kafka extensively as a message bus to transport data and to power real-time streaming services, ultimately helping more than 250 million Pinners around the world discover and do what…

Using machine learning to auto-organize boards

While planning one’s life might look different these days, our insights tell us people are saving ideas for the future more than ever. Last month, for instance, Pinterest had a 60% increase in the number…

Understanding product attributes for shoppers

Every day, people come to Pinterest to shop and find ideas in categories like home and fashion, and it’s our responsibility to show them the most relevant and inspiring products. We measure relevance in a few different dimensions, but most importantly…

Trusting Metrics at Pinterest

Pinterest, like many tech companies, relies heavily on data to help inform decisions and power machine learning. This strategy fundamentally depends on the data being accurate. Over the years we’ve worked to improve the processes behind trusting core decision-making data and ensure metrics remain accurate. How Data Can Go Wrong …

Improving data processing efficiency using partial deserialization of Thrift

At Pinterest we’ve worked to greatly improve data processing efficiency. One quote that resonates with our unique approach is from writer Antoine de Saint-Exupéry: “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” Ultimately…

Managing Videos on Android

Video launched on the Pinterest Android app in 2016 with the goal of bringing a seamless video experience to the app. This included the ability to support multiple videos per screen, while dynamically controlling their playback state by automatically pausing them when they’ve scrolled…

An update on Pixie, Pinterest’s recommendation system

We recently welcomed engineers to our offices for our annual Mad Science Fair, where I shared the latest on Pixie, an advanced graph-based recommendation system. Here’s a recap of that talk, and you can find the video here. As Pinterest grows to more…

Real-time experiment analytics at Pinterest using Apache Flink

At Pinterest, we run thousands of experiments every day, and rely mostly on daily metrics to evaluate performance. The daily pipelines can take 10+ hours to run and sometimes are delayed, which has created some inconvenience in verifying the setup of…