The right scaling strategies for peak performance

multi datacenterOne message you never want to see from your service provider, especially if you rely on them to monetize your web properties: “We are currently experiencing platform wide issues. Our engineers are looking at it.”

But datacenter problems can happen to companies large and small, with recent news-making examples coming from cloud computing heavyweights such as Apple, Amazon, and Facebook. One factor behind outages like this is that companies sometimes rely too much on a single datacenter to keep their services available. But in today’s online-all-the-time and socially connected world, no platform is complete if it is not running out of multiple independent datacenters.

Why expand into multiple datacenters?

Ad tech companies such as Altitude and their customers rely on their platforms to be 100% available at all times. Depending on the publisher and the time of day, an outage could cost hundreds of dollars in revenue each minute. Guaranteeing availability may be the biggest and most obvious reason for opening multiple datacenters, but it’s just one of many.

Altitude handles hundreds of thousands of connections per second across its platform, pushing many Gigabits per second (Gbps) out to clients, partners and publishers. It is certainly possible to deliver that kind of network traffic from a single datacenter, but it can be more economical to divide that load across several smaller and simpler installations rather than one grand datacenter.

By breaking the problem into smaller chunks, we have established a reliable and repeatable method for scaling up without having to cross major technical hurdles on our way to 100 Gbps. The end result for our customers is that they never have to worry about our capacity to deliver, and we never slow down.

Another advantage to spreading our platform across the globe is that it brings us closer to our viewers and integration partners. Network traffic is fast, but can feel surprisingly sluggish if you’re trying to pull up a video from halfway around the globe. If we can have our ad servers in the same general neighborhood as our viewers and our partners, we can serve ads with very low latency. Low latency to our partners means they have a better chance of winning a 250 millisecond auction to place an ad. Low latency to our viewers means we can roll that ad before they get tired of waiting and move on.

Challenges with multiple datacenters

Clearly it is important for companies in the ad-tech space to run their platforms from multiple datacenters for reliability, scale, and performance. Why would any company choose not to do so? It turns out that running out of multiple datacenters can be a complicated business. There are many challenges, which boil down to cost and complexity.

Obviously two datacenters are more expensive than one. But for most ad tech companies, two datacenters are not even enough to provide the kind of reliability and performance required to reach a global audience. The costs of building out a global infrastructure can quickly add up in terms of time, money, equipment, services, and people.

Cost may be a major hurdle on the road to multiple datacenters, but complexity is an even bigger one. Just keeping one datacenter in order can be a daunting task with the daily issues of security, patching, testing, and deploying new code. Those issues take on new dimensions of complexity across multiple datacenters. In fact, it is entirely possible to expand to multiple datacenters and end up with more downtime and worse performance than before!

An investment in infrastructure

Despite the challenges, we have made a strategic investment in building out our own platform, on our own hardware, and across multiple datacenters. We have forged strong partnerships with key infrastructure service providers across the globe to keep our costs low. Those partnerships also give us the agility to build up and tear down resources anywhere in the world to adapt to changing traffic patterns. We have also attracted an incredible team of DevOps engineers to tame the complexities of multiple datacenters, ensuring no drama and no downtime.

zconger_350x500By Zachary Conger
VP of IT

Read more from the Tech Blog