One of our biggest challenges over the years has been scaling our operations to keep up with the growth in people using Facebook and the new products that help them keep connected with each other. We’re now at a point where we have more than 800 million users, people are uploading more than 250 million photos per day, and we're on the verge of rolling out a product in Timeline that will require enormous computing power to serve.

Back in early 2009, we came to a crossroads for our infrastructure. At that point, we were leasing space in data centers designed for general purpose computing -- meaning we were getting average efficiency at average cost. Facebook was growing rapidly -- the site had 150 million active users at the beginning of 2009, a number that increased to more than 350 million by the end of that year.

We didn’t have hundreds of millions of dollars to throw at the problem. We experimented with optimizing our existing leased facilities, but concluded that we wanted to build a data center of our own.

So we decided to design and build our own servers, software, power supplies, and data centers -- with the goal of building the most efficient computing infrastructure at the lowest possible cost.

This would give us total control over three critical parts of our infrastructure: the data center design, the server design and the software. We could also decide where to build, what to build and how to operate it.

Our goals were simple: Make it as efficient as possible, remove everything extraneous, minimize our impact on the environment and be open about what we were doing.

The launch of our Lulea data center marks the next evolution of our strategy. Not only is it our first data center outside the US -- helping make Facebook even more responsive and reliable for our international users -- but it’s also our first data center to draw its power primarily from renewables, and it features design evolutions like a 70 per cent reduction in our reliance on backup generators.

Progress of the data centre will be updated on the Page if you’d like to keep up to speed with it: https://www.facebook.com/luleadatacenter
You can find out more about the Open Compute Project at: http://opencompute.org/

Lulea Data Center
Lulea background -- Our first decision was where to build. We chose the City of Lulea located in the north of Sweden for several reasons:

• The location provides an excellent climate that allow us to use outside air for cooling for 8-10 months of the year. The result is a significant reduction in energy consumption.
• The site offers a strong power infrastructure. This infrastructure will allow us to implement a reduced generator topology by installing dual redundant substations fed from two independent sources that are directly connected to robust national grid. The redundant substations allow us to reduce the number of generators installed by approximately 70% and minimize the environmental impact due to reduced emission and fuel storage.
• There a lot of synergies between our operations and the City of Lulea and surrounding area. We believe that there are many positive benefits from being located in close proximity to the university.
• We like the accessibility of Lulea.

Data Center Design
The electrical system design is very similar to the Prineville Project and the Open Compute Project (OCP). The design is a highly efficient electrical system which eliminated traditional components in a data center that contributes to energy losses such as the PDUS and centralized double conversion UPS systems.

The new, energy-efficiency technologies Facebook will utilize here include:
• Airside economizer – The HVAC system is also similar to the Prineville Project which uses outside air for datacenter cooling. The Lulea facility will be cooled by simply bringing in colder air from the outside approximately 97% of the year. During low humidity times of the year, the evaporative cooling system will function as humidifiers to meet server humidity requirements.
• Evaporative cooling system – Additional cooling is provided by direct evaporative cooling through pressure atomizing humidifiers. For Prineville, direct evaporative cooling operates 6% a year on average, for Luleå it will be less than 3% due to the cooler climate. This system uses evaporation of water to achieve cooling as opposed to the more traditional chiller systems that requires more energy intensive equipment. For other times of the year, outside air economizer mode will be used.

The PUE for this project is expected to be similar to that of Prineville (1.07) factoring more heating required for the longer winter. This compares to our target of 1.15, an EPA-defined industry state-of-the-art rating, and 1.5 in our leased facilities.

• Re-use of server heat – A portion of the excess heat created by the computer servers will be captured and re-used to heat office space in the facility during the colder months.
• Proprietary Uninterruptible Power Supply (UPS) Technology – All data centers must have an uninterruptible power supply to bridge the gap between a power outage and the ability of backup generators to kick-in. The Lulea Data Center will use a new, patent-pending system for UPS that reduces electricity usage by up to 12 percent.
• Not only is FB pursuing LEED Gold following the US Green Building Council requirements, but FB is also prescribing BREEAM. This is Europe’s standard for best practices in sustainable building design, construction and operation and measures a building's environmental performance.
• Given the robustness of the utility grids, FB will implement a patented generator design. This design will eliminate 85% of the diesel fuel otherwise required by FB’s comparable facilities in the US. This translates to reduced emissions -- both from the facility, and from the vehicles used to bring fuel to the facility.
• Rain water is collected and re-directed to flow into the adjacent creeks to minimize impact to the environment
• Use of low Volatile Organic Compound (VOC) finish materials
• Use of Energy Star Compliant appliances
• Use of low water consumption technologies such as automated faucets and low flush toilets
• Use of insulated metal wall panels to minimize construction materials, time and construction debris from traditional materials
• Use of special equipment and measures designed to minimize audible disturbance and meet the site acoustic requirements
• Preservation of as many trees as possible around the site perimeter
• Special care to the site geography and system so as not to impact the surrounding water shed
• Implementation of a local Quality Manager to provide oversight for compliance with regional requirements

Servers
Assembling the servers is like building a Lego model -- the parts snap together. The servers slot in and out of their racks by flipping a couple of catches, making life easier for data center engineers.

Our servers are taller than the average server, which means we can use bigger fans to cool them -- which benefits efficiency. We also slowed the fans down to save energy. Fans typically account for 10-20 per cent of the energy consumption in a typical server, compared with 2-4 per cent in our server.

The server racks are designed to fit precisely into sea freight and trucking containers, increasing shipping efficiency and reducing cost. We also eliminated the speaker on motherboards, replacing it with a blue LED.

We use our servers as intensively as possible so we don’t build more infrastructure than we need.

Openness
A couple of years ago, Facebook began to apply its hacker culture to the significant infrastructure challenges we face on a daily basis, as a compute-intensive service with an exponentially growing user base. Our engineers ended up building an entirely new infrastructure from scratch — from servers to storage to power supply to the data center itself — that drove a 38 percent reduction in the power required to run a Facebook data center, at a 24 percent lower cost.

But we realized that we didn’t do everything we could have, and we didn’t get it all right. So we then took the unprecedented step of "open-sourcing" all the specifications behind that infrastructure, founding the Open Compute Project in the process. Openness is part of Facebook’s culture, and it was our belief that opening the technology would spur advances that we wouldn’t have discovered if we’d kept this technology secret. Other people are now building on and making improvements to our original designs.

The Open Compute Project has now taken on a life of its own, with individuals and organizations across the industry beginning to build on and make improvements to our original designs. Collectively, we’re demystifying what it takes to deliver large scale distributed computing -- and everyone is benefiting, including companies that manufacture their own servers.

The momentum behind the Open Compute Project continues to build. We recently announced a collaboration with the Open Data Center Alliance to get OCP hardware and designs in front of that organization's 300-plus members, so they can begin to evaluate, adopt, and innovate on them. And we will share further news surrounding the initiative at the second Open Compute Project Summit later today in New York.

ShareThis

Tags: Design & Facilities Management, Hosting & Colocation, Power & Cooling

Read more White Papers»

More DCS TV»

More Audio»

Read more Web Exclusives»

Read more Magazine Articles»

Read more Supplements »

Keep up to date with the latest industry products, services and technologies from the world's leading IT companies.

• Virtualization Success Depends on Network Automation by Infoblox

• The Infoblox VMware vCenter Orchestrator Plug-in by Infoblox

• Automated Storage Tiering on Infortrend’s ESVA Solution by Infortrend

• Why Cloud Backup: Top 10 Reasons by Autonomy

• How to Measure the ROI of Cloud Data Protection by Autonomy

Latest IT jobs from leading companies.

Click here for full listings»