Monitor your Segment events through Datadog

In MonkeyLearn we love Segment. We use it to gather all of our analytics on our website, blog and application. It allows us to then forward the data to different analytics apps like Google Analytics and Mixpanel among others.

But, there is a problem: shit happens. Sometimes the addition of a new feature or refactoring will have had an impact on the code snippets within our site that send events to Segment. As you may be thinking, those introduced bugs are manifested as data loss, and that's something that as a product manager / data analyst you want to avoid. Data that isn't tracked is data that can't be reproduced, you'll have a blind spot in the historical behavior of your users.

As a result, we wanted to implement a mechanism where we could be alerted whenever a bug related to a tracking event was detected. That way we could fix the problem asap and minimize data loss.

One way we could solve it is through testing. That is, implement unit/functional tests to verify that the events are being sent whenever a new app or site version is deployed.

Thinking about a second way to solve this, we asked ourselves that if we are sending events to outside BI tools such as Segment, there should be other outside tools that can serve as a monitoring or alert system.

We liked the second approach. With just a click, a monitoring tool would magically start tracking every event that is being received. Then for each event, we can set up a simple rule that tests if the event frequency is below a certain threshold, then set up alerts within our favorite messaging app, for example, via a message in Slack.

In MonkeyLearn we love DataDog. We thought that would be the perfect tool for monitoring and alerting the frequency of our Segment events.

Surprisingly, there's no direct integration between Segment and Datadog, one would think that it should be a one-click integration where you set up your DataDog account within Segment and then magically all the events are sent to your favorite DataDog dashboard. I looked around for a direct integration like this, and even found some possible alternatives to DataDog that offer it, but they didn't have the features that I needed.

At this point, if you are a coder, you begin to think about how you can make an integration yourself. So I started coding. The result was Segment2Datadog.

I basically created a minimal Flask app that exposes an endpoint where you can receive events from Segment and then forward them to DataDog.

You can easily deploy this app with just one click, no coding required, you just need access to your Segment account, DataDog account and have a Heroku account to deploy the app. Just follow the installation steps listed in the readme at the Github repo. You can also deploy it in your favorite infrastructure provider.

The way to send events to Segment is through the Webhooks integration. Just select your Segment source, enable Webhooks as a destination and put the URL of your Segment2Datadog app that you deployed in Heroku. You must pass the source name as a URL parameter like this:

Setting the webhooks on Segment.

Where https://yourapp.herokuapp.com/api/ is the URL of your deployed heroku app. Then you add the /app_name to identify the source of the data, in this case "website".

This is what part of the dashboard we use to monitor the Segment events looks like:

Our Datadog dashboard to monitor the Segment events.

What's nice about DataDog is that you have multiple options on graphs to monitor your events, you can use the tags as filters (the source is one of them). Then each event will have the event name as a tag too. Afterwards, you can set up alerts with monitors, e.g.: send a message to a Slack channel if no "Sign up event" has been registered within 12hs.

This was my quickest solution to my problem, if you managed to solve this easier, would love to know about yours!