An investigation into Kafka Log Compaction

Background At Zendesk, we have been using Kafka in production for several years. Primarily we have been using it to distribute events between services. For example, “a ticket has been changed, and this is what changed”. We are now considering using Kafka to share entities between systems. For example, each time…

Datadog Log Management from Zero to One

Datadog APM Trace Search from Zero to One is the first part of advanced techniques we use at Zendesk to plan capacity and keep performance characteristics of the multi-tenant system at expected levels. Response times vary between different API endpoints and customers, as the data set grows rapidly, and new…

Documentation: it’s more than words

Software development is as much about communication as it is about functionality. At its core, code reflects business processes and business logic. Software never exists solely in a vacuum, it is surrounded by both context and people. Tracking the history of a business and its software becomes an increasingly important…

Collecting Build Metrics from Buddybuild

So after a long hiatus from writing, I’ve decided to try and be a bit more disciplined and write more frequently. This is the first in a series around how we have introduced build time tracking across our CI environments, and how we track it in our development tools. We…

How we run gamedays in Talk

While we can sometimes see our products as simply a bundle of operations with corresponding Service Level Indicators and Objectives, we also need to keep in mind the human factor responsible for maintaining these agreements. Introduction This article idea came after a 2019 EMEA SREcon talk that strongly resonated with me…

Open sourcing IronBank, our Zuora client

Open sourcing node-publisher

The dependency graph of our hypothetical package “A” We’ve just fixed an annoying bug in dependency C and our task is to propagate the fix up until package A. So…

Zendesk ML Model Building Pipeline on AWS Batch: Monitoring and Load Testing

The series started with Wai Chee Yau’s exploration of AWS Batch and why we ended up choosing it…

Adding Functionality to Google Pub/Sub: Queue Meta Processing

Foldable Go

Running a team colocation — Days of the week

The Friday before In preparation for the co-location week, we started off by sending a survey to gather: Everyone’s objective for the week List of questions they’d like answers for by the end of…

iOS How-to — Add Adaptive constraints to support a Universal App

Device screen size (e.g iPhone or iPad) Device Orientation (portrait or landscape) Adaptation (multi-tasking on iPad) In this post, we will learn how to add NSLayoutConstraints…

Data Catalogs, the Luxury of Choice

Zendesk’s journey to Kubernetes

We can use Kubernetes for little things, but there isn’t much point doing it for our original monolithic Ruby on Rails product. Even if we did…

A minimal Python client for talking to TensorFlow Serving

min-tfs-client is a minimal gRPC client for TensorFlow Serving that doesn’t depend on TensorFlow. TL;DR: min-tfs-client is a minimal python client for TensorFlow Serving that allows you to use serverless services (e.g. AWS Lambda) that have deployment size limits (250 Mb uncompressed, at time of writing). It works by removing TensorFlow as a dependency to create tensor protobufs for the prediction request. Feel free…

Chaos Engineering comes to Ruby

In a micro-service environment, an unexpected service outage often comes as a surprise and when it happens it’s stressful for engineers, managers, and the clients. If you want to prepare your Ruby application for the next surprise and prevent it early then this post is for you. I will show…

The history of infrastructure at Zendesk (Part 2) — the messy middle

Making the most of Answer Bot: topic group visualization using t-SNE

Zendesk’s Global Mesh Network- Part 2

The science behind consolidating Answer Bot production Models: Part 2

In the previous post (Part 1), we introduced the issue of consolidating the six production models that are used to provide multilingual support for Answer Bot at Zendesk into a single model. Consolidating these models both simplifies our deployment process for new models, while providing a potentially more scalable solution…

Kafka on Ruby

I have a problem with 1on1 questions…

1on1s can be hard. If you’re a manager you have probably seen multiple articles offering the quick fix of questions to ask your team… A 30 second online search turned up pages of articles with catchy clickbait titles like: One on One Meeting Questions Asked By Great Managers! Top 10…

Datadog APM Trace Search from Zero to One

Trace Search and Analytics by Datadog HQ is a fantastic tool for performance analysis and capacity planning. Trace Search has been in Beta over the past year and was announced for General Availability at Dashcon.io. Please see the keynote, related workshop and also its Github repo. In this blogpost we…

SwiftUI & UIView — A Simple Mistake

I recently made a simple mistake. An implementation which appeared correct, based on its initial usage, turned out to be incorrect when its usage changed. But before we start, although this post describes the mistake I made and shows you how…

App Onboarding with SwiftUI

Onboarding is the process of helping users to understand an app, and hopefully, encouraging them to continue to use it. Onboarding flows usually consist of one or more screens explaining how a user can use the app, and often involve some input from the user, e.g signing up, logging in…