Did I put enough in the title to make you think “what could this guy possibly be talking about”? I guess that was the point, and strangely enough, the title does have meaning.
There has been a religious war raging in the data center community for several years now on the subject of server cabinet densities. The question is whether high density server cabinets (typically 15-30kW) should be installed versus distributing your compute over a larger area in low density (typically 4-10kW) cabs. The basic argument is that the overhead of designing for high density, doesn’t justify the benefits. I’m here to say that high density is the best answer, but probably not for the same reasons that most have been arguing.
How Does Sustainability Fit Into This Argument?
Sustainability plays into the density argument because of the facilities. A data center is a complex and expensive facility and building one to cool 25kW server cabinets definitely has added cost vs. the alternative. There’s increased electrical per square foot and of course extra cooling (air conditioning). However, beyond those two things, a data center is still a data center. In fact, even the airflow management (ducting, etc.) when done effectively doesn’t need to change significantly regardless of density, it’s the ability to cycle the right amount of air to and from the correct locations that matters. The real problem with choosing low density vs. high density is it increases the space required for your servers. You also need more network cable (fiber & Ethernet), more cabinets, switches, electrical copper, ladder rack, patch panels, and, well you get the idea.
So my basic tenet is that in the long run it’s not just wasteful and more expensive to plan, build and run low density environments, its bad practice from a sustainability standpoint. Even if you discount all the extra materials I discussed in the previous paragraph, you can’t discount the building of another structure or a larger than necessary one. If you accept the fact that the incremental design and materials required to support 25kW cabinets is only a fraction of the cost and construction material of one facility, how could you possibly justify building a whole new facility to support a 5kW design. Even if you could somehow justify the cost of building low density, you can’t justify the environmental and sustainability concerns associated with the construction and management of extra facilities.
There is roughly 1 ton of carbon created by every ton of cement and total global cement production is estimated to create 7% of all carbon output. In case you were wondering, the cement for a large data center equates to tens of thousands of tons.
So, you’re probably starting to get the picture of my sustainability point, and I haven’t even talked about the extra pavement, roofing materials, steel, cars and trucks, paint, and cleaning materials, etc., etc..
OK, How Does Cloud fit into this Story?
Simply put, IT infrastructure, applications and data benefit from having proximity. Unless you’re distributing workloads to balance geographic risks or latency concerns, the closer you can keep your servers and applications to each other the better. So if you’re building data centers that spread servers across 100 cabinets, when they could have put them in 15-25 cabinets, you’re not only wasting space and money, you’re also causing headaches for your infrastructure and application design teams.
The Switch SuperNAP is designed to 1500 Watts a square foot or roughly 25kW per cabinet at a year round average PUE of 1.24. While you don’t have to build environments that require that kind of density, when you do it provides significant cost savings and potential infrastructure performance improvements.
The original objective of building to 1500 Watts a square foot was to support the real application needs of organizations that are maximizing the use of modern infrastructure. Our customers are running hundreds of racks between 15kW and 28kW and several hundred more are running between 6 and 15kW. Besides the benefits our customers derive from the ability to run modern infrastructure, this density has also allowed us to dramatically reduce our data center construction footprint as compared to the vast majority of other facilities. The result of these low density environments is that we are potentially using up to five times (5X) the amount of space necessary.
To put the previous paragraph into perspective I estimate the total global data center foot print exceeds 700 million square feet and we are adding roughly 10% in new capacity every year. When you consider that up to 18% of a buildings carbon output is created during the manufacture of materials and the construction phase, that’s an immense figure.
So the next time you hear someone talk about their PUE number, ask them two things;
- How is your power generated (Fossil, renewables, Clean, etc.)?
- The energy source is key to determining the actual carbon impact of DC operations, vs. simply measuring the effective use of the power (I.e., PUE)
- What’s your power density per square foot?
When you can make a decision that helps your company’s bottom line and reduces negative impact on the environment, why wouldn’t you?
The links below are a few additional resources used in the writing of this blog: