Electrical Explosions Cause Construction Delays at NSA’s $1.2 Billion Spy Compound

Rick Bowmer/AP

AA Font size + Print

Persistent electrical surges apparently sparked explosions and a year-long delay that NSA officials did not disclose. By Aliya Sternstein

While the government was nearing a shutdown and bracing for a default, a $1.2 billion National Security Agency supercomputer homestead in Utah, built to ingest surveillance data, reportedly was wasting hundreds of thousands of dollars on fried equipment.

NSA officials masked the woes out west, at a time when the agency already was facing accusations of lying about domestic spying practices.

As recently as June 19, agency officials told Nextgov the spy center — 15 times the size of MetLife stadium — would be “operational” by Oct. 1. That month, NSA spokeswoman Vanee Vines also said in an email the “exterior is done.  . .  It’s now time for IT guts to be installed,” referring to information technology systems.

But, behind the security perimeter, persistent electrical surges apparently sparked explosions and a year-long delay that NSA officials did not disclose. On Monday night, the Wall Street Journal reported there have been 10 meltdowns during the past 13 months that have prevented NSA from running computers.

The revelations come after ex-NSA contractor Edward Snowden this spring leaked files to the media detailing the agency’s mammoth collections of Internet and phone data, revelations later acknowledged by U.S. officials. The 200-acre Utah computer fortress was designed to warehouse this data.

In June, Harvey Davis, a top agency official, who handles military installations, as well as power, space and cooling for all NSA data centers, said in an interview that the construction – undertaken by the Corps of Engineers – has been an open project. 

The military construction process by design is a very, very transparent process,” he said. “It’s a public discourse. When we give out our request for proposal, that’s through FedBizOpps.gov,” a website that advertises jobs for prospective vendors, so “by its nature, military construction is a very, very open process.”

Project contract papers called for a 65-megawatt center with a chiller plant, fire suppression systems, electrical generators and an uninterruptible power supply backup capacity.

On Tuesday, NSA and Army Corps of Engineers officials declined to answer questions about the failure to disclose that there would be some sort of delay.

Norbert Suter, the Corps chief of construction operations, said in an email that power connectivity remains incomplete. “The final commissioning of the entire electrical system is still ongoing,” he said, declining to address cost overruns. “The Corps is ensuring it functions as required and that it will be completely reliable prior to releasing the project to the customer,” meaning NSA. Suter did not say when the site would turn on. 

He acknowledged there have been a series of glitches. ”During the testing and commissioning of the Utah data center, problems were discovered with certain parts of the electrical system. Issues such as these can arise in any project, and are the reason the Corps tests and reviews every aspect of any project prior to releasing it to the customer,” he said.

More Corps professionals were summoned and the source of the problems was discovered, Suter said, without elaborating further. 

Because of the scope of this project, a specialized team of experts from the Corps’ headquarters was sent to review and assist with addressing these issues. The cause of the electrical issues was identified by the team and is currently being corrected by the contractor,” he said.

The team also uncovered ventilation troubles. In order to function, data centers need an air management system able to cool IT equipment and remove waste heat. Suter said, “An airflow issue was discovered with the generators and the contractor is correcting the problem.”

On Monday night, Vines said, “On the whole, this is why acceptance testing is done - so that issues can be found and resolved before the government takes possession.”

While final assessments continue, Suter said the Corps is ”making preparations to release it to the customer.”

The final tab for the ultimate customer, the taxpayer, is a question mark. Davis this summer said, “I came in under cost,” referring to $100 million in savings gained partly by refusing to let contractors adjust the plan. 

Close [ x ] More from DefenseOne
 
 

Thank you for subscribing to newsletters from DefenseOne.com.
We think these reports might interest you:

  • Software-Defined Networking

    So many demands are being placed on federal information technology networks, which must handle vast amounts of data, accommodate voice and video, and cope with a multitude of highly connected devices while keeping government information secure from cyber threats. This issue brief discusses the state of SDN in the federal government and the path forward.

    Download
  • Military Readiness: Ensuring Readiness with Analytic Insight

    To determine military readiness, decision makers in defense organizations must develop an understanding of complex inter-relationships among readiness variables. For example, how will an anticipated change in a readiness input really impact readiness at the unit level and, equally important, how will it impact readiness outside of the unit? Learn how to form a more sophisticated and accurate understanding of readiness and make decisions in a timely and cost-effective manner.

    Download
  • Cyber Risk Report: Cybercrime Trends from 2016

    In our first half 2016 cyber trends report, SurfWatch Labs threat intelligence analysts noted one key theme – the interconnected nature of cybercrime – and the second half of the year saw organizations continuing to struggle with that reality. The number of potential cyber threats, the pool of already compromised information, and the ease of finding increasingly sophisticated cybercriminal tools continued to snowball throughout the year.

    Download
  • A New Security Architecture for Federal Networks

    Federal government networks are under constant attack, and the number of those attacks is increasing. This issue brief discusses today's threats and a new model for the future.

    Download
  • Information Operations: Retaking the High Ground

    Today's threats are fluent in rapidly evolving areas of the Internet, especially social media. Learn how military organizations can secure an advantage in this developing arena.

    Download

When you download a report, your information may be shared with the underwriters of that document.