Reverse engineering Sketch’s resizing functionality

My first mission has been diving into the world of rendering our react components back into Sketch, a design tool, to keep the final components in sync. Standing on the shoulders of others, I have…

The history of infrastructure at Zendesk (Part 1) — constant tradeoffs

The science behind consolidating Answer Bot production Models: Part 2

In the previous post (Part 1), we introduced the issue of consolidating the six production models that are used to provide multilingual support for Answer Bot at Zendesk into a single model. Consolidating these models both simplifies our deployment process for new models, while providing a potentially more scalable solution…

Recipes for running a successful technical book club at work

This blog post is our learnings from running technical book clubs at Zendesk for the past 2 years. In that time we have run four book clubs: Two books on functional programming languages: Clojure and Scala Reliability is very important to us so we studied the bible on Software Reliability…

The Dawn of Zendesk’s Machine Learning Model Building Platform with AWS Batch

This article focuses on the new model…

Collecting Build Metrics from Buddybuild

So after a long hiatus from writing, I’ve decided to try and be a bit more disciplined and write more frequently. This is the first in a series around how we have introduced build time tracking across our CI environments, and how we track it in our development tools. We…

Datadog Log Management from Zero to One

Datadog APM Trace Search from Zero to One is the first part of advanced techniques we use at Zendesk to plan capacity and keep performance characteristics of the multi-tenant system at expected levels. Response times vary between different API endpoints and customers, as the data set grows rapidly, and new…

Racecar v2: A New Foundation

It’s been three years since we introduced Racecar, a simple framework for building Kafka consumer applications that worked seamlessly with or without Ruby on Rails. With Racecar, building a consumer application for processing…

The history of infrastructure at Zendesk (Part 3) — Foundation team forming and evolving

Introduction In Part 2, I wrote about our adventures moving from physical data centers to the Cloud. Let’s move on to how we transformed our organizational structure to take advantage of being in the Cloud and enable self-service. Building the foundation In early 2018, our organizational structure was stereotypically old…

Data Catalogs, the Luxury of Choice

Performing Kubernetes Service CIDR Block Migration

History At Zendesk, we started our Kubernetes journey in September 2015, shortly after version 1.0 was publicly available. At the time of v1.0 not much information was available and we didn’t have a lot of experience, so some configuration choices that made sense four years ago have had to be reconsidered…

Running a team colocation — The TL;DR

NO to over-planning, YES to #COLOYOLO Don’t over-plan and take it easy! Make it a working week, don’t try to overload it with meetings. Just remember…

App Onboarding with SwiftUI

Onboarding is the process of helping users to understand an app, and hopefully, encouraging them to continue to use it. Onboarding flows usually consist of one or more screens explaining how a user can use the app, and often involve some input from the user, e.g signing up, logging in…

Create a test data generator using Kafka Connect

…

Evaluating Apache Pulsar

Zendesk currently use Kafka, primarily for publishing of events. We are also considering publishing the state of entities to Kafka, using compacted topics to keep only the latest version of an entity. We do this already, but are considering it for much larger data sets. These compacted topics will be…

Going Where our Team has not Gone Before

How to plan out where you’re going when you don’t know how you’re going to get there. Organising a data science team’s research direction is a challenge in itself and there’s no algorithm with which to optimise it. This quarter our team is trying something new. Generally when our data…

Zendesk’s Global Mesh Network- Part 2

pakkr™ (Part I), One Pipeline to Rule Them All

Co-Author This blog is a collaborative effort with Eric Pak. Prequel This is a story of how we, data scientists and machine learning engineers in Zendesk, came out of the dark age of machine learning tooling, built our in-house tooling pakkr pipeline, and shared it with the world. The Dark Age: Life Pre-pakkr and How We Came Here Nowadays, many companies talk…

Running a team colocation — The beginning of something great

Open sourcing IronBank, our Zuora client

Zendesk’s journey to Kubernetes

We can use Kubernetes for little things, but there isn’t much point doing it for our original monolithic Ruby on Rails product. Even if we did…

A Github Actions Workflow to Generate & Publish your Hugo Website

This short post describes a Github Actions workflow to generate your Hugo-based website and publish it to Github Pages. To learn more about Hugo and GitHub Pages, and how to use them to build your website, see my previous post. What is Github Actions Github Actions is a service from Github which allows you…

Let’s make a message encoder in Ruby

Binary message encoding is a common pattern in modern distributed systems. Tools like Protobuf, Avro and Thrift are increasingly used to decrease payload size and provide typed contracts for message produces and consumers. Many developers will need to utilise these tools at some point, but they can be opaque and…

An investigation into Kafka Log Compaction

Background At Zendesk, we have been using Kafka in production for several years. Primarily we have been using it to distribute events between services. For example, “a ticket has been changed, and this is what changed”. We are now considering using Kafka to share entities between systems. For example, each time…

Datadog APM Trace Search from Zero to One

Trace Search and Analytics by Datadog HQ is a fantastic tool for performance analysis and capacity planning. Trace Search has been in Beta over the past year and was announced for General Availability at Dashcon.io. Please see the keynote, related workshop and also its Github repo. In this blogpost we…