Datadog APM Trace Search from Zero to One

Trace Search and Analytics by Datadog HQ is a fantastic tool for performance analysis and capacity planning. Trace Search has been in Beta over the past year and was announced for General Availability at Dashcon.io. Please see the keynote, related workshop and also its Github repo. In this blogpost we…

Data store migrations with no downtime

Data store migrations can be complex, especially if the application cannot afford to have maintenance downtime. We will explore some strategies that can help you navigate the complexity. In our case, it was a migration from a NoSQL data store, DynamoDB into a relational data store, MySQL. However, this post…

The Joys of Circuit Breaking

Zendesk’s product suite is built with an evolving service oriented architecture, comprising hundreds of micro services and a growing web of interdependencies. Operating and orchestrating such a large scale distributed system can be challenging. This post will explore the importance of circuit breaking and related concepts. A few weeks ago…

Running a team colocation — The beginning of something great

Creating the ultimate flexibility via Container components in React

React Components / Zendesk Garden This documentation includes React components and utilities that provide visuals, keyboard-navigation, localization, and…garden.zendesk.com I wrote a Twitter thread on some of the cool stuff we’ve done that could be very useful as a starter for other people. So here’s that thread with some extra goodies, in article form. We’ve utilised a really cool pattern…

App Onboarding with SwiftUI

Onboarding is the process of helping users to understand an app, and hopefully, encouraging them to continue to use it. Onboarding flows usually consist of one or more screens explaining how a user can use the app, and often involve some input from the user, e.g signing up, logging in…

iOS How-to — Add Adaptive constraints to support a Universal App

Device screen size (e.g iPhone or iPad) Device Orientation (portrait or landscape) Adaptation (multi-tasking on iPad) In this post, we will learn how to add NSLayoutConstraints…

SwiftUI & UIView — A Simple Mistake

I recently made a simple mistake. An implementation which appeared correct, based on its initial usage, turned out to be incorrect when its usage changed. But before we start, although this post describes the mistake I made and shows you how…

Validating Cloud Custodian on AWS with Moto

At Zendesk, we adopted Cloud Custodian as a framework to implement compliance across our AWS accounts. Cloud Custodian has a Domain Specific Language that allows anyone with a penchant for reading documentation to create policies that can report or act on resources in AWS, GCP, or Azure. Deleting or re-configuring…

Foldable Go

Reverse engineering Sketch’s resizing functionality

My first mission has been diving into the world of rendering our react components back into Sketch, a design tool, to keep the final components in sync. Standing on the shoulders of others, I have…

10 tips for growing a globally distributed engineering team

Maintaining Binary Compatibility in Scala

A minimal Python client for talking to TensorFlow Serving

min-tfs-client is a minimal gRPC client for TensorFlow Serving that doesn’t depend on TensorFlow. TL;DR: min-tfs-client is a minimal python client for TensorFlow Serving that allows you to use serverless services (e.g. AWS Lambda) that have deployment size limits (250 Mb uncompressed, at time of writing). It works by removing TensorFlow as a dependency to create tensor protobufs for the prediction request. Feel free…

A Github Actions Workflow to Generate & Publish your Hugo Website

This short post describes a Github Actions workflow to generate your Hugo-based website and publish it to Github Pages. To learn more about Hugo and GitHub Pages, and how to use them to build your website, see my previous post. What is Github Actions Github Actions is a service from Github which allows you…

An investigation into Kafka Log Compaction

Background At Zendesk, we have been using Kafka in production for several years. Primarily we have been using it to distribute events between services. For example, “a ticket has been changed, and this is what changed”. We are now considering using Kafka to share entities between systems. For example, each time…

Making the most of Answer Bot: topic group visualization using t-SNE

The science behind consolidating Answer Bot production Models: Part 1

At Zendesk, we believe that great customer service builds strong customer relationships and that Machine Learning ( ML ) can play a powerful role in creating good customer experiences. Whether it is enhancing search tools or powering entire self-service systems, ML has the…

Rotational Security Interns

The internship will…

Zendesk ML Model Building Pipeline on AWS Batch: Monitoring and Load Testing

The series started with Wai Chee Yau’s exploration of AWS Batch and why we ended up choosing it…

Composable Go

With a lot of languages, it is easy to replace a given function or method call with a mocked version. Dynamically replacing or statically injecting mocked dependencies makes testing code much easier. Personally I’ve found this approach less than ideal in Go, so I’ve been looking for another way to…

The history of infrastructure at Zendesk (Part 1) — constant tradeoffs

Datadog Log Management from Zero to One

Datadog APM Trace Search from Zero to One is the first part of advanced techniques we use at Zendesk to plan capacity and keep performance characteristics of the multi-tenant system at expected levels. Response times vary between different API endpoints and customers, as the data set grows rapidly, and new…

Add Some Smarts To Your Change Data Capture

As the number of your services grows, it becomes error prone and hard…

Building user confidence in machine learning products

I’m a Software Engineer on the Data Products team at Zendesk. Recently I had one of the highlights of my career — working in the team that built Answer Bot. As a front-end specialist, my focus was on crafting the best possible user experience for Zendesk customers…