The private cloud built for the Central Intelligence Agency by Amazon Web Services has gone live, according to a report from Defense One. Doug Wolfe, CIA’s Chief Information Officer, admitted that the agency realized that there are better specialists out there on some areas, and the time has come to ease the burden of computing-intensive analytics programs used by the agency.
During a keynote at the Strata + Hadoop World conference held in New York City on Oct. 1st, Doug Wolfe talked about the importance of future partnerships with vendors in the big data market, and why the nation’s first line of defense needed to embrace the cloud technology, in order to improve its own mission.
“The big question: what are you really going to do on the cloud? Mission outcome that’s what we really need to be focused on”, said Wolfe. “How do we provide our analysts the most advanced state-of-the-art-analytics and produce intelligence that matters? I think the difference is quite huge, and we’re starting to think that we can really do some innovating things and make a difference in our mission.”
The Strata + Hadoop World conference is an event where data scientists, analysts, and executives keep up to date or develop new data science partnerships. Case studies, tutorials or best practices presentations offer the participants plenty of opportunities to develop new skills and learn all about the emerging data techniques and technologies. Regarded as one of the largest conferences of its kind in the world, Strata + Hadoop World brings together speakers specialized in data issues that are shaping the business, finance, media, fashion, retail, energy, manufacturing, and government worlds. One of the highpoints of the conference was the keynote of Doug Wolfe, CIO at the Central Intelligence Agency. Wolfe offered more details about the Amazon Web Services deal, and why the agency started using its cloud.
CIA’s private Amazon Marketplace
Announced about two years ago, the giant $600 million contract is meant to serve all 17 US intelligence agencies. The contract is also opening doors for partnerships between the CIA and other big data vendors. The objective is to have on-demand computing and analytic services that the intelligence agencies can order (and pay for), from the CIA and NSA, via their private clouds.
“We are really starting to realize the benefits of cloud computing. Amazon has lived up to their side of this bargain, they are rolling in new features,” said Doug Wolfe. “Someone mentioned to me that there’s only two ways to look at this business: you can either keep trying to do it yourself and struggle and try to keep up, or you can turn it over to people who really know what they are doing, and get on a curve that’s consistent with commercial parity, and that’s really what we’ve done with Amazon“, acknowledged the agency’s CIO.
With a private version of Amazon Marketplace, publicly listed vendors in the marketplace can basically see CIA’s operating environment.
“I really think the kinds of tools and products and services and the innovation and energy that you are putting into Hadoop and Spark and all those kinds of engines, are really the kind of thing that we need to collectively take advantage of, for our mission and our business, as well as other businesses across the country,” continued Wolfe. “One of the ways that we intend to do that is by opening up a private copy of Amazon Marketplace, and so vendors who have listed publicly in a marketplace, have now the opportunity to list in a private copy of that marketplace, and that makes a huge difference,” he said.
The Spark in CIA’s eye
A lot of CIA’s attention seems to be focused big data frameworks, specifically on the Spark environment.
“We are also pretty enthusiastic about Spark, and the Spark environment, the Spark streaming, the Spark machine learning, and all the user interfaces that are associated with Spark. There’s a tremendous amount of opportunity there, and I’m excited about the innovation that’s going on in that space as well. Our team is enabling these tools to transition as quickly and rapidly as possible, and putting the power in the hands of analysts and the operators to go make a difference for mission,” said Doug Wolfe.
The main challenge that remains, is connecting the new technology to the workforce. So far, experimentation has been messy admitted Wolfe, but it’s somewhat expected, given that partnering for expertise is not something that the CIA is used to yet. Just the benefits of simplifying the sharing of data between these agencies, should be convincing enough. The Central Intelligence Agency’s move is basically proof of the power of data. Whether it will help with support for missions across the globe, or just improve the processing of individual data points, CIA’s step confirms the importance of data driven decision-making.
