Zendesk’s Global Mesh Network- Part 1

Open sourcing IronBank, our Zuora client

Maintaining Binary Compatibility in Scala

The history of infrastructure at Zendesk (Part 1) — constant tradeoffs

Collecting Build Metrics from Buddybuild

So after a long hiatus from writing, I’ve decided to try and be a bit more disciplined and write more frequently. This is the first in a series around how we have introduced build time tracking across our CI environments, and how we track it in our development tools. We…

Running a team colocation — The TL;DR

NO to over-planning, YES to #COLOYOLO Don’t over-plan and take it easy! Make it a working week, don’t try to overload it with meetings. Just remember…

Data store migrations with no downtime

Data store migrations can be complex, especially if the application cannot afford to have maintenance downtime. We will explore some strategies that can help you navigate the complexity. In our case, it was a migration from a NoSQL data store, DynamoDB into a relational data store, MySQL. However, this post…

Going Where our Team has not Gone Before

How to plan out where you’re going when you don’t know how you’re going to get there. Organising a data science team’s research direction is a challenge in itself and there’s no algorithm with which to optimise it. This quarter our team is trying something new. Generally when our data…

Let’s make a message encoder in Ruby

Binary message encoding is a common pattern in modern distributed systems. Tools like Protobuf, Avro and Thrift are increasingly used to decrease payload size and provide typed contracts for message produces and consumers. Many developers will need to utilise these tools at some point, but they can be opaque and…

The science behind consolidating Answer Bot production Models: Part 2

In the previous post (Part 1), we introduced the issue of consolidating the six production models that are used to provide multilingual support for Answer Bot at Zendesk into a single model. Consolidating these models both simplifies our deployment process for new models, while providing a potentially more scalable solution…

10 tips for growing a globally distributed engineering team

iOS How-to — Add Adaptive constraints to support a Universal App

Device screen size (e.g iPhone or iPad) Device Orientation (portrait or landscape) Adaptation (multi-tasking on iPad) In this post, we will learn how to add NSLayoutConstraints…

Validating Cloud Custodian on AWS with Moto

At Zendesk, we adopted Cloud Custodian as a framework to implement compliance across our AWS accounts. Cloud Custodian has a Domain Specific Language that allows anyone with a penchant for reading documentation to create policies that can report or act on resources in AWS, GCP, or Azure. Deleting or re-configuring…

Machine learning is blossoming at Zendesk

Bill and Ted’s Excellent Adventure! A Short History of Machine Learning at Zendesk We have invested heavily in ML over the past four years and we…

Creating the ultimate flexibility via Container components in React

React Components / Zendesk Garden This documentation includes React components and utilities that provide visuals, keyboard-navigation, localization, and…garden.zendesk.com I wrote a Twitter thread on some of the cool stuff we’ve done that could be very useful as a starter for other people. So here’s that thread with some extra goodies, in article form. We’ve utilised a really cool pattern…

How we improved the observability of a Go project

Background About a year ago, my team took over responsibility for an event driven system written in Go. One of our first steps was to understand what the application does, and how it does it. The app already had stdout logging, rollbar notifications, datadog statsd metrics, and pager alerts. The previous…

Running a team colocation — Days of the week

The Friday before In preparation for the co-location week, we started off by sending a survey to gather: Everyone’s objective for the week List of questions they’d like answers for by the end of…

Implementing mTLS and Securing Apache Kafka at Zendesk

This article was originally posted on the Confluent blog at https://www.confluent.io/blog/how-zendesk-secures-kafka-with-mtls-authentication-system/ Background At Zendesk, Apache Kafka® is one of our foundational services for distributing events among different internal systems. We have pods, which can be thought of as isolated cloud environments where our products run, spread out all across the world…

pakkr™ (Part I), One Pipeline to Rule Them All

Co-Author This blog is a collaborative effort with Eric Pak. Prequel This is a story of how we, data scientists and machine learning engineers in Zendesk, came out of the dark age of machine learning tooling, built our in-house tooling pakkr pipeline, and shared it with the world. The Dark Age: Life Pre-pakkr and How We Came Here Nowadays, many companies talk…

How probable are collisions with ULID’s monotonic option?

There’s a lot of excitement at Zendesk about ULID. As a universally unique identifier format, ULID has become a popular alternative to UUID in recent years. For most of our history at Zendesk, we’ve used integer IDs internally for database keys, as well as in our public APIs. We even…

Blameless Culture

Adding Functionality to Google Pub/Sub: Queue Meta Processing

Chaos Engineering comes to Ruby

In a micro-service environment, an unexpected service outage often comes as a surprise and when it happens it’s stressful for engineers, managers, and the clients. If you want to prepare your Ruby application for the next surprise and prevent it early then this post is for you. I will show…

The history of infrastructure at Zendesk (Part 3) — Foundation team forming and evolving

Introduction In Part 2, I wrote about our adventures moving from physical data centers to the Cloud. Let’s move on to how we transformed our organizational structure to take advantage of being in the Cloud and enable self-service. Building the foundation In early 2018, our organizational structure was stereotypically old…

SwiftUI & UIView — A Simple Mistake

I recently made a simple mistake. An implementation which appeared correct, based on its initial usage, turned out to be incorrect when its usage changed. But before we start, although this post describes the mistake I made and shows you how…