Cornell Leads New NSF-Sponsored Aristotle Cloud Federation Project

These days it is not unusual to hear people bemoan that we have lost the capacity to think big and invest in projects with profound future impacts. This has been true even in the fast-paced communications industry, where things like SDN, NFV and 5G are getting lots of investment. The good news is that Cornell University will lead a five-year, $5 million project sponsored by the National Science Foundation (NSF) to build a federated cloud, known as the Aristotle Cloud Federation.

The federated cloud will be deployed at Cornell University (CU), the University at Buffalo (UB), and the University of California, Santa Barbara (UCSB) and shared by seven science teams with over forty global collaborators. Its objective is to develop data infrastructure building blocks (DIBBs) designed to support scientists and engineers requiring flexible workflows and analysis tools for large-scale data sets. This is a big move. And, as has always been the case regarding the Internet and now the cloud, what happens in academia never stays in academia.

In making the announcement of the project and the grant, the Cornell individuals leading it will include: David Lifka, Director of the Cornell University Center for Advanced Computing (CAC) will lead the project with colleagues Tom Furlani, Director of the UB Center for Computational Research, and Rich Wolski, Professor of Computer Science at UCSB.

As noted, this is a science project. However, it will also address non-scientific areas as well.  Initial users of the cloud federation will be the earth and atmospheric sciences, finance, chemistry, astronomy, civil engineering, genomics, and food science. They were selected based on the diversity of their data analysis requirements and cloud usage modalities.

As Cornell notes regarding the areas of concentration and their intended value:

Their use cases will demonstrate the value of sharing resources and data across institutional boundaries. The overarching goal is optimizing “time to science”—the actual time it takes a researcher to obtain their scientific results. The elasticity provided by sharing resources means researchers don’t have to wait for local resources to become available to get their science started…

Metrics provided by UB’s XDMoD (XD Metrics on Demand) and UCSB’s QBETS (Queue Bounds Estimation Time Series) will enable researchers and administrators to make informed decisions about when to use federated resources outside their institutions.

The project leaders added context to the project and its objectives. “Cloud-based systems are rapidly becoming a key component in the support of research programs in academe and industry. By adding cloud metrics to XDMoD, researchers and senior leaders will be able to obtain detailed operational metrics of cloud systems in order to improve the efficiency of jobs run on the cloud, as well as measure overall cloud performance,” said Furlani. “Efficient use of federated clouds requires the ability to make predictions about where a workload will run best,” added Wolski. “Using XDMoD data and cloud-embedded performance monitors, QBETS will make it possible to predict the effects of federated work-sharing policies on user experience, both in the DIBBs cloud and in the Amazon Web Services (News - Alert) (AWS) Cloud.”

“The goal of the Aristotle Cloud Federation is to develop a federated cloud model that encourages and rewards institutions for sharing large-scale data analysis resources that can be expanded internally with common, incremental building blocks and externally through meaningful collaborations with other institutions, commercial clouds, and NSF cloud resources,” said PI Lifka. The project name—Aristotle—was chosen because Aristotle’s concept “the whole is greater than the sum of its parts” reflects the multi-institutional synergy and collaborations that the federation aspires to create.

The key words here obviously are “federation” and “sharing.”  In addition, for non-marketing people you have to like the Aristotle project name and the “time to science” moniker. 

At the heart of Aristotle will be new allocations and accounting models to allow institutional administrators to track utilization across federated sites and use this data as an exchange mechanism between partner sites.

The deliverable of the effort will be that federation components, documentation, and best practices will be shared with information necessary to:

• Create customized Virtual Machine instances
• Leverage resources at federated sites
• Burst to AWS
• Access, move, and share large-scale data
• Deploy new cloud federations.

The third bullet is worth a shout out for AWS participation.  Jamie Kinney, Senior Manager Scientific Computing, Amazon Web Services, Inc., expressed their enthusiasm saying: “We are excited to work with the Aristotle team to provide cost-effective and scalable infrastructure that helps accelerate the time to science,” said Jamie Kinney, Senior Manager Scientific Computing, Amazon Web Services, Inc. As evidenced in the entire announcement of the grant, those involved in the selected areas are equally as excited about the benefits they foresee.

For its part, Amy Walton, Program Director, Advanced Cyberinfrastructure Division, NSF said:  “This award continues NSF’s multi-year strategy to stimulate exploration of scalable and sustainable data infrastructure models that facilitate collaborative research across disciplines and institutions…“By experimenting with cloud usage metrics, collaborating with a commercial cloud vendor, and exploring pricing/trading allocation mechanisms, the project will provide valuable information about how the innovations work in a range of situations, and how this ‘market approach’ integrates within the larger research ecosystem.”

This news really is significant for next generation communications as cloud federation. Getting it right is very high on the “to do list” of the industry for accelerating cloud adoption.