How InfluxDB uses DigitalOcean

Who we are

InfluxDB is an open source time series database. It’s designed to handle large amounts of time-based data (e.g. server metrics, user analytics, and sensor data) and provide a robust API and horizontally-scalable architecture to make queries simple and responsive.

The challenge

When we first released InfluxDB as an open source project, it was immediately appealing to developers who didn’t mind building the binary from source or installing one of our pre-built packages to familiarize themselves with the product. In fact, our community has done an amazing job building new libraries, adapters, and visualization tools on top of it.

It didn’t take long, however, before users started asking us about production deployment and hosting options. We quickly realized that we wanted to provide a platform that allowed anyone to create a single node or cluster on the fly. We theorized that running each InfluxDB instance as a separate virtual machine would make it easy to avoid common multi-tenancy issues like resource utilization and security, keeping us from having to add those features to the core database.

Why DigitalOcean?

We chose DigitalOcean primarily because of its amazingly powerful REST API. In order to build the system we imagined, we needed to be able to quickly spin up a virtual machine from an image. Creating a virtual machine through DigitalOcean generally takes less than a minute, so our customers can start using their new server almost immediately, and simultaneously keeps us from having to spend extra money keeping spare servers provisioned.

Additionally, being able to choose from many instance sizes and regions via the API made it incredibly easy for us to address the needs of both international and higher-volume customers.

Finally, having a low-cost server option ($5/mo) made it feasible for us to offer a free trial to new customers.

The architectural details

Our customer-facing application is a simple Ruby on Rails application with a Sidekiq background queue. When a server request is made by a customer, a job is placed on the queue, which initiates the create server call and continues polling the DigitalOcean API for event updates. We have a DigitalOcean image that we keep updated with each new release of InfluxDB, which is already pre-loaded with all of the other components we need for a new server (SSL certificates, open file limits, configuration files, monit, etc). We’re using the DigitalOcean RubyGem for all of our API calls.

Once the droplet is ready, we store all of the details in the database and use the assigned IP address to create a DNS entry through Amazon’s Route 53. We make one final call directly to the InfluxDB instance on the new server to set credentials, and send those in an email to the customer.

For any additional updates that need to be made, we use the SSH key that DigitalOcean has on file, coupled with SSHKit to script out the interactions.

InfluxDB + DigitalOcean

Most importantly, DigitalOcean provided a developer-friendly way to control the create/update/destroy the lifecycle of virtual machines. This has allowed us to make an on-demand platform that lets our customers deploy InfluxDB servers for development, staging, and production environments alike.