Datadog APM Trace Search from Zero to One

Trace Search and Analytics by Datadog HQ is a fantastic tool for performance analysis and capacity planning. Trace Search has been in Beta over the past year and was announced for General Availability at Dashcon.io. Please see the keynote, related workshop and also its Github repo. In this blogpost we…

Evaluating Apache Pulsar

Zendesk currently use Kafka, primarily for publishing of events. We are also considering publishing the state of entities to Kafka, using compacted topics to keep only the latest version of an entity. We do this already, but are considering it for much larger data sets. These compacted topics will be…

I have a problem with 1on1 questions…

1on1s can be hard. If you’re a manager you have probably seen multiple articles offering the quick fix of questions to ask your team… A 30 second online search turned up pages of articles with catchy clickbait titles like: One on One Meeting Questions Asked By Great Managers! Top 10…

Validating Cloud Custodian on AWS with Moto

At Zendesk, we adopted Cloud Custodian as a framework to implement compliance across our AWS accounts. Cloud Custodian has a Domain Specific Language that allows anyone with a penchant for reading documentation to create policies that can report or act on resources in AWS, GCP, or Azure. Deleting or re-configuring…

Implementing mTLS and Securing Apache Kafka at Zendesk

This article was originally posted on the Confluent blog at https://www.confluent.io/blog/how-zendesk-secures-kafka-with-mtls-authentication-system/ Background At Zendesk, Apache Kafka® is one of our foundational services for distributing events among different internal systems. We have pods, which can be thought of as isolated cloud environments where our products run, spread out all across the world…

Kafka on Ruby

Leading Machine Learning at Zendesk

I recently participated in an interview series covering careers in emerging technology fields on the French website TOM.travel. I’ve been leading machine learning at Zendesk for the last 6 years and over that time we’ve grown from two people in Melbourne to tens of people on three continents. I discussed…

How probable are collisions with ULID’s monotonic option?

There’s a lot of excitement at Zendesk about ULID. As a universally unique identifier format, ULID has become a popular alternative to UUID in recent years. For most of our history at Zendesk, we’ve used integer IDs internally for database keys, as well as in our public APIs. We even…

SwiftUI & UIView — A Simple Mistake

I recently made a simple mistake. An implementation which appeared correct, based on its initial usage, turned out to be incorrect when its usage changed. But before we start, although this post describes the mistake I made and shows you how…

The science behind consolidating Answer Bot production Models: Part 1

At Zendesk, we believe that great customer service builds strong customer relationships and that Machine Learning ( ML ) can play a powerful role in creating good customer experiences. Whether it is enhancing search tools or powering entire self-service systems, ML has the…

Code Coverage and Xcode

Key to unit testing is code coverage…

Inclusion — getting the basics right around communication

How we improved the observability of a Go project

Background About a year ago, my team took over responsibility for an event driven system written in Go. One of our first steps was to understand what the application does, and how it does it. The app already had stdout logging, rollbar notifications, datadog statsd metrics, and pager alerts. The previous…

Going Where our Team has not Gone Before

How to plan out where you’re going when you don’t know how you’re going to get there. Organising a data science team’s research direction is a challenge in itself and there’s no algorithm with which to optimise it. This quarter our team is trying something new. Generally when our data…

A Beginners Guide to Deploys and Releases

The process of releasing software to users at scale is one that can be particularly confusing for new software engineers, as most don’t have exposure to this part of the software development process until their first internship or job. I found this to be the case during my time at…

Making the most of Answer Bot: topic group visualization using t-SNE

Zendesk’s Global Mesh Network- Part 2

Racecar v2: A New Foundation

It’s been three years since we introduced Racecar, a simple framework for building Kafka consumer applications that worked seamlessly with or without Ruby on Rails. With Racecar, building a consumer application for processing…

Having better One-on-Ones

1:1s are often my…

Add Some Smarts To Your Change Data Capture

As the number of your services grows, it becomes error prone and hard…

Say Hi to Delivery Boy!

Hot off the heels of releasing Racecar, our easy-to-use framework for running Kafka consumers in Ruby, we’re ready to release yet another Kafka-related library! This time, it’s all about message delivery. Up until now, if you wanted to produce messages to Kafka from a Rails application, you’d need to write…

Chaos Engineering comes to Ruby

In a micro-service environment, an unexpected service outage often comes as a surprise and when it happens it’s stressful for engineers, managers, and the clients. If you want to prepare your Ruby application for the next surprise and prevent it early then this post is for you. I will show…

How we use AWS Batch at Zendesk to Build All The Machine Learning Models

In this instalment, I’ll be describing…

iOS How-to — Add Adaptive constraints to support a Universal App

Device screen size (e.g iPhone or iPad) Device Orientation (portrait or landscape) Adaptation (multi-tasking on iPad) In this post, we will learn how to add NSLayoutConstraints…

Running a team colocation — The beginning of something great