The history of infrastructure at Zendesk (Part 3) — Foundation team forming and evolving

Introduction In Part 2, I wrote about our adventures moving from physical data centers to the Cloud. Let’s move on to how we transformed our organizational structure to take advantage of being in the Cloud and enable self-service. Building the foundation In early 2018, our organizational structure was stereotypically old…

Say Hi to Delivery Boy!

Hot off the heels of releasing Racecar, our easy-to-use framework for running Kafka consumers in Ruby, we’re ready to release yet another Kafka-related library! This time, it’s all about message delivery. Up until now, if you wanted to produce messages to Kafka from a Rails application, you’d need to write…

Validating Cloud Custodian on AWS with Moto

At Zendesk, we adopted Cloud Custodian as a framework to implement compliance across our AWS accounts. Cloud Custodian has a Domain Specific Language that allows anyone with a penchant for reading documentation to create policies that can report or act on resources in AWS, GCP, or Azure. Deleting or re-configuring…

Data store migrations with no downtime

Data store migrations can be complex, especially if the application cannot afford to have maintenance downtime. We will explore some strategies that can help you navigate the complexity. In our case, it was a migration from a NoSQL data store, DynamoDB into a relational data store, MySQL. However, this post…

Racecar v2: A New Foundation

It’s been three years since we introduced Racecar, a simple framework for building Kafka consumer applications that worked seamlessly with or without Ruby on Rails. With Racecar, building a consumer application for processing…

Datadog Log Management from Zero to One

Datadog APM Trace Search from Zero to One is the first part of advanced techniques we use at Zendesk to plan capacity and keep performance characteristics of the multi-tenant system at expected levels. Response times vary between different API endpoints and customers, as the data set grows rapidly, and new…

Blameless Culture

Create a test data generator using Kafka Connect

…

Chaos Engineering comes to Ruby

In a micro-service environment, an unexpected service outage often comes as a surprise and when it happens it’s stressful for engineers, managers, and the clients. If you want to prepare your Ruby application for the next surprise and prevent it early then this post is for you. I will show…

Adding Functionality to Google Pub/Sub: Queue Meta Processing

Data Catalogs, the Luxury of Choice

Datadog App Analytics/Logs from One to N

Leading Machine Learning at Zendesk

I recently participated in an interview series covering careers in emerging technology fields on the French website TOM.travel. I’ve been leading machine learning at Zendesk for the last 6 years and over that time we’ve grown from two people in Melbourne to tens of people on three continents. I discussed…

Recipes for running a successful technical book club at work

This blog post is our learnings from running technical book clubs at Zendesk for the past 2 years. In that time we have run four book clubs: Two books on functional programming languages: Clojure and Scala Reliability is very important to us so we studied the bible on Software Reliability…

The science behind consolidating Answer Bot production Models: Part 2

In the previous post (Part 1), we introduced the issue of consolidating the six production models that are used to provide multilingual support for Answer Bot at Zendesk into a single model. Consolidating these models both simplifies our deployment process for new models, while providing a potentially more scalable solution…

Add Some Smarts To Your Change Data Capture

As the number of your services grows, it becomes error prone and hard…

Xcode Quick Tip – Testing your layout for localization

Recently we encountered buggy UI component positioning when the device language was changed from the default language (English). This is shown below with Dutch. UI component (UISwitch) incorrectly positioned due to ambiguous constraints Due…

Let’s make a message encoder in Ruby

Binary message encoding is a common pattern in modern distributed systems. Tools like Protobuf, Avro and Thrift are increasingly used to decrease payload size and provide typed contracts for message produces and consumers. Many developers will need to utilise these tools at some point, but they can be opaque and…

The history of infrastructure at Zendesk (Part 1) — constant tradeoffs

How probable are collisions with ULID’s monotonic option?

There’s a lot of excitement at Zendesk about ULID. As a universally unique identifier format, ULID has become a popular alternative to UUID in recent years. For most of our history at Zendesk, we’ve used integer IDs internally for database keys, as well as in our public APIs. We even…

Creating the ultimate flexibility via Container components in React

React Components / Zendesk Garden This documentation includes React components and utilities that provide visuals, keyboard-navigation, localization, and…garden.zendesk.com I wrote a Twitter thread on some of the cool stuff we’ve done that could be very useful as a starter for other people. So here’s that thread with some extra goodies, in article form. We’ve utilised a really cool pattern…

Sorbet: Type annotations for Ruby

Part 2 — Zendesk Spotlight: Our Career-Changer Engineers.

Following on from Zendesk Spotlight: Our Career-Changer Engineers I received so many words of wisdom from our career-changer engineers that it was clearly worth sharing it with the world. And judging by your response to Part 1 you think so too! Here is what they had to say. Catherine Software Engineer…

Rotational Security Interns

The internship will…

Zendesk ML Model Building Pipeline on AWS Batch: Monitoring and Load Testing

The series started with Wai Chee Yau’s exploration of AWS Batch and why we ended up choosing it…