Goku: Building a scalable and high performant time series database system

Co-authored by Jinghan Xu, Jian Wang & Tian-Ying Chang, Engineers At Pinterest, developers rely on Statsboard to monitor their systems and discover issues. A reliable and efficient monitoring system is very important for development velocity. Historically, we’ve used OpenTSDB to ingest and…

Building a dynamic and responsive Pinterest

In 2015, the majority of content on Pinterest was pregenerated for users prior to login. It was stored statically in HBase and served directly upon entering the service. (More details can be found in the blog post Building a smarter home feed.) …

The Fantastic Fixathon

Engineers at Pinterest are always focused on doing what’s best for the product, but working in a codebase that’s clean, well-maintained, and relatively bug-free transforms engineering tasks from being “just work” to enjoyable opportunities to flex our skills. We’re also enriched as engineers, both technically and otherwise, when we take…

Improving HBase backup efficiency at Pinterest

Pinterest has one of the largest HBase production deployments in the industry. HBase is one of the foundational building blocks of our infrastructure and powers many of our critical services including our graph database (Zen), our general-purpose key-value store (UMS), our…

User state-based notification volume optimization

Notifications play an important role in helping Pinterest’s 335M+ monthly active users discover and engage with inspiring content. It’s crucial to send the right number of notifications with high quality content. If we send too few notifications, Pinners might miss out on great opportunities…

How switching our domain structure unlocked international growth

More than half of the 200 million monthly active users on Pinterest use our app outside the U.S. As we continue to make Pinterest even better for our global users, we moved our traffic to country code top-level domains (ccTLDs). An example of this…

PinalyticsDB: A Time Series Database on top of Hbase

PinalyticsDB is Pinterest’s proprietary time series database. At Pinterest, we rely on PinalyticsDB as a backend for storing and visualizing thousands of time series reports such as the sample case below, segmented by country. PinalyticsDB was built several years ago on…

Trusting Metrics at Pinterest

Pinterest, like many tech companies, relies heavily on data to help inform decisions and power machine learning. This strategy fundamentally depends on the data being accurate. Over the years we’ve worked to improve the processes behind trusting core decision-making data and ensure metrics remain accurate. How Data Can Go Wrong …

La evolución de la búsqueda en Pinterest

This blog article was originally posted in English. To read the English version click here. Cuando Pinterest se lanzó por primera vez, ofrecía una nueva forma de recopilar y organizar ideas de toda la web. Sin embargo, lo que comenzó como un lugar…

The next era of Bug Bounty at Pinterest

When a security researcher discovers a bug in a piece of software, the responsible thing to do is inform the company so they can fix it. And so platforms like Pinterest need to provide clear and actionable programs, typically with rewards or recognition, for those with valid reports. For us, that’s come in the form of responsible disclosure policies, which we’ve evolved over the years. We work with Bugcrowd to manage the program and integrate with their existing community of researchers, across a variety of Pinterest properties including pinterest.com subdomains (such as help.pinterest.com), mobile apps, browser extensions and open source projects. Since 2015 we’ve given monetary rewards (or “bounties”) to researchers and have continually raised those rewards. We leverage Bugcrowd’s vulnerability rating taxonomy to fairly assign rewards based on severity, which allows for a reasonable expectation of reward from researchers and helps to focus attention on the most impactful types of bugs. The program has been a big success. Hundreds of researchers have participated in the program. We’ve rewarded more than $35,000 to more than 150 valid non-duplicate submissions, and the highest single reward was $2,500. Today we’re once again announcing increased rewards for all tiers of bugs to show our continued commitment to responsible disclosure and researchers. If you’re a security researcher or developer interested in participating in our program, read the brief and terms on Bugcrowd, and join us.

How we built rearranging Pins

Listening to user feedback is essential to building a product Pinners love and use to discover and save ideas every day. Often, their feature requests present interesting technical challenges for our engineers. For example, today we launched the ability to…

Hybrid Search: Building a textual and visual discovery experience at Pinterest

As a visual discovery engine, Pinterest helps you discover ideas even when you don’t know what you’re looking for, or have the words to describe it. Using a mix of technology (including visual search) and community input (thanks to the 175B+ Pins…

Adventures in big data wonderland: Going down the Pinterest Path

Have you ever started looking for plants on Pinterest, only to end up shopping for pillows instead? Ever find yourself layers of Pins deep into finding a new idea, not sure of how you got there or where you started? When Pinners are in…

Supporting React Native at Pinterest

Early last year, a small group of engineers started investigating the possibility of adopting React Native at Pinterest. We’ve been following the ecosystem with interest since it was first open-sourced in 2015. Since then, a robust community has developed around it, and it’s been…

Building interactive email with AMP

AMP (Accelerated Mobile Pages) is an open source technology created by Google to support fast mobile web pages. Over the years, we’ve partnered with Google and adopted AMP technology in our mobile web pages to deliver a faster, more efficient user experience. Most…

Geo-blocking media content on Pinterest

Pinterest engineers are tasked with showing Pinners the best ideas related to their interests, which includes taking into account signals like location. As part of these efforts, we built a video and image platform to enable a…

How ads indexing works at Pinterest

Introduction Ads indexing is a data processing pipeline that builds servable ad documents for the ads delivering stack at Pinterest, including ads targeting, retrieval, ranking and auction. The pipeline converts ad campaigns in the raw format to the servable format via a heavy computational…

Improving data processing efficiency using partial deserialization of Thrift

At Pinterest we’ve worked to greatly improve data processing efficiency. One quote that resonates with our unique approach is from writer Antoine de Saint-Exupéry: “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” Ultimately…

How we use AutoML, Multi-task learning and Multi-tower models for Pinterest Ads

People come to Pinterest in an exploration mindset, often engaging with ads the same way they do with organic Pins. Within ads our mission is to help Pinners go from inspiration to action by introducing them to the compelling products and services that…

How we reduced Pinterest’s iOS app size by 30+% / 50MB

We all know app size (download size[1] and local install size[2]) matters, and there is a correlation between app size and customer engagement. Many times, people make decisions about using software based on size, and even pay for bandwidth by the megabyte. Not…

Powering Pinterest ads analytics with Apache Druid

The Change When we launched Promoted Pins in 2014, we chose Apache HBase as our database to store and serve all of our reporting metrics. At the beginning of our ads business, this was an appropriate choice because…

The Dead Code Society

On June 1, 2018, engineer Chris Lloyd created the #dead-code-society Slack channel to be “a sombre place for remembering [---]-only diffs.” Messages are prefixed by a :tombstone: and include links to diffs that contain only file or line deletions. The channel averages a little over one message per day, but they frequently come in bursts as part of refactoring projects and clean-up efforts. As Pinterest has grown as a company, our software unpinning has grown with it. We have hundreds of software engineers producing thousands of changes per week. The net result is an upward trend in total lines of code with a significant portion of our software having been (re)written in just the last few years. The #dead-code-society has helped balance that trend by celebrating the retirement of code from active service. Retired code remains in source control history to be remembered and consulted, but it no longer occupies a prominent place in our workspaces, build systems, or cognitive periphery. Less is more. Carpe diem.

What we learned from an iOS app OOMs incident

In early 2020, we started seeing significantly elevated out-of-memory (OOM) crashes in the Pinterest iOS app. That incident resulted in a declined Crash-Free Users Rate (CFUR), from the previous 99% to 96%, which was a steep drop. What was going on? We improved…

A one year PWA retrospective

The idea of building a “Progressive Web App” (PWA) is not new, but its definition has changed with the emergence of key technologies like service workers. Now it’s finally possible to build great experiences in a mobile browser. Being an early adopter can…

Pinterest + ktlint = ❤

It’s been nearly a year since Pinterest’s Android codebase became Kotlin-first, and about two years since we adopted ktlint for Kotlin linting and formatting. Today, we’re sharing that Pinterest has officially taken ownership over the project. Initially, we chose ktlint because of its…