Top gun data management keeps Air Force finance in flight
The Air Force uses CRIS to integrate near real-time data from financial managers based in numerous locations and manage day-to-day operations.
It’s no secret that the public sector is facing budget-tightening, paralleling what many private enterprises have been experiencing in recent years. Working with fewer resources is a challenge for IT teams as they work to manage an unprecedented boom in data demands to support analytics-driven objectives. While these challenges can be difficult to work through, constraints can also lead to new breakthroughs. It was exactly these factors that drove the U.S. Air Force upgrade its IT systems to strategically deliver more data while saving time, energy and budget.
The Air Force worked with TekSouth, an Alabama-based systems integration and professional services company that serves both commercial and government customers, including the USAF Commanders’ Resource Integration System. CRIS is the Air Force's authoritative data source for financial management of unclassified appropriated historical data. To manage day-to-day operations, while providing senior leaders at all levels a near real-time snapshot of how operational entities are performing, the Air Force uses CRIS to integrate data from financial managers based in numerous locations.
Mission: Scale a high security, unpredictable global service
The CRIS system provides near real-time reporting for more than 15,000 users running up to 1.2 million queries per month. As part of the upgrade, TekSouth wanted to address increasing pressure for datacenter space and energy conservation, while meeting the USAF’s failover requirements to be always operational in every time zone around the world.
The Air Force financial warehouse posed significant challenges to scalable performance. For example, the warehouse contains over 22 terabytes of data and continues to grow, receiving 250 data feeds daily from 19 systems worldwide. Strict access controls protecting sensitive financial information added even more challenges compared to the workloads of standard data warehouses. In addition, while most data warehouses primarily use canned queries and stored procedures, the vast majority of queries on CRIS are ad-hoc, yet require near real-time responses.
Three times more work in a fraction of the time
Mike Rhodes, TekSouth’s vice president of operations explained the challenge faced in scaling CRIS. “We’ve designed the software CRIS uses to deliver the performance and high uptime the Defense Department needs. But we were limited by the capabilities of traditional hardware.”
With disk I/O being the biggest bottleneck, the architecture is built to scale out and isolate I/O intensive processes from one another. TekSouth then turned to a leading flash memory solution to eliminate the underlying I/O contention that was the system’s biggest constraint.
The results were better than Rhodes hoped. “We tested the system against real-world historical query workloads rather than synthetic benchmarks, which allowed us to see how it would perform in actual deployment. A single server with flash memory doubled the workload capability of a three-server, 21-disk array-based system. At 15 times the standard production workload, we were still operating within the USAF performance requirements,” he said. “The new system supports three times the number of concurrent users and can run three times the number of queries in the same time. Eighty-two percent of ad-hoc queries returned in under 10 seconds, while the average response time for all queries, in total, was less than 23 seconds.”
Efficiency shrinks energy footprint and costs
Being energy conscious is a budget-friendly focus for many companies, but for the Department of Defense, running on less equipment and energy can be an imperative in mobile fields of operation. Before the refresh, the CRIS system consisted of five servers and 27 arrays of disks. After adding flash to the servers, the CRIS system footprint shrank to just three servers that take up about six rack units of space. With flash, the CRIS system delivers double the workload capabilities on far less hardware, which means less to move and power to remain operable in a wide variety of locations.
Global uptime for a force that never sleeps
The Air Force needs CRIS to be available around the clock and around the world. To meet these requirements and ensure data is always ready for analysis in all situations, CRIS was designed to be a distributed, scale-out system with 99.97 percent uptime attributable to its modular architecture that allows redundant subsystems. The flash memory added to its servers makes each piece of hardware much more powerful so it can scale farther. CRIS uses all three servers in a distributed architecture to handle and load balance active queries, without the significant maintenance overhead of the previous, disk-based system. In fact, the solution eliminated maintenance for 27 disk arrays and over 400 disk drives.
No matter what kind of organization IT professionals are supporting, looking at new ways to deliver data to the applications that provide business intelligence more efficiently can lead to breakthroughs that save on resources and unlock new potential. National security may not always be at stake, but examining new approaches like server based flash memory platforms can help uncover opportunity that may make a significant impact on businesses of all types.