How Illinois and IBM want to combine the best of both worlds for an "ideal" hybrid cloud

How Illinois and IBM want to combine the best of both worlds for an "ideal" hybrid cloud

Written by Lauren Laws

Necessity is the mother of invention, but the desire to improve leads to innovation. Cloud computing has evolved from its inception over the last 15 years, leading to innovation through agility, scalability, and accessibility. But there’s always another step with innovation, a desire for more and better. That step now is even broader adoption.  

Clouds enable individuals and businesses to not only save but also process and share information. Cloud computing began as an idea to bring all computing infrastructure and software together into hyperscale data centers. Similar to its meteorological counterpart, there is more than one type of cloud, each with their pros and cons.  

Hybrid cloud combines and unifies public cloud, private cloud and on-premises infrastructure to create a single, flexible, cost-optimal IT insfrastructure
Click to enlarge.

Private clouds are dedicated solely to an individual or organization and are tailored specifically to their needs. The data center is controlled by the organization and is within the company’s firewalls. All of this comes at a literal price, though. Private clouds can be expensive and isn’t always a luxury that businesses can afford.  

Public clouds, such as Microsoft Azure and Google Cloud, are provided by third parties and are often relatively inexpensive. Apps are integrated for ease of use, all hardware is controlled elsewhere, and capacity is rented on a needed basis. But the tradeoff is the lack of high-level security that private clouds offer. 

“What I think the industry realized is that the model is good for some things, it’s good for some applications, but it’s not good for everyone. It’s not a one size fits all,” said Giovanni Pacifici, Vice President of Hybrid Cloud at IBM Research.

Cloud computing’s new focus is how to have the best of both worlds. Enter the Goldilocks scenario in the search for the ‘just right’ cloud: a hybrid cloud. 

“The goal for a hybrid cloud is to connect the private and public cloud together in such a way that you can leverage the public cloud economy while maintaining private cloud security and privacy,” said Deming Chen, the Abel Bliss Professor of Engineering in Electrical and Computer Engineering at University of Illinois, and the Hybrid Cloud Thrust Co-Lead of the IBM-Illinois Discovery Accelerator Institute.  

IBM and The Grainger College of Engineering have been working together on the “ideal” hybrid cloud. They plan to instantiate a hybrid cloud testbed system called i2cloud. In October 2021, the $200 million IBM-Illinois Discovery Accelerator Institute was launched as a 10-year collaboration focused on the rapidly growing areas of hybrid cloud and AI, quantum information science and technology, accelerated materials discovery, and sustainability. Illinois faculty members, more than 30 graduate students, and nearly 40 IBM researchers have worked closely together on various aspects of the proposed new cloud system.

Hybrid Cloud

Researchers from the University of Illinois and IBM partner together to develop an open-source hybrid cloud.

The goal is to bring the best of public and private clouds together in terms of ease of use, efficiency and security to make as much of the world’s compute infrastructure as possible easier to use. A particular area of focus is AI-driven automation within the platform to ensure cost and energy efficiency, scalability and availability, and performance for a wide variety of emerging applications. 

Priya Nagpurkar, Director of Cloud Platform Research at IBM's T.J. Watson Research Center, said they want to “find opportunities to rethink and optimize in every layer from the hardware up to the platform of what we envision as a more efficient, more secure and open hybrid cloud ecosystem.”

Hybrid cloud computing faces 5 scientific challenges
Click to enlarge.

The unique feature is that we not only optimize the individual system layers for this hybrid cloud system,” said Chen, “we also co-design and co-optimize a cross layer kind of system [and] make sure the whole hybrid cloud is working in a very coherent way.”   

Researchers said the applications of the system will reach far and wide. For example, an ideal hybrid cloud could be utilized by retail stores to give a better customer experience and relieve a stretched or overloaded workforce. Manufacturers could utilize it for more efficient quality control, or allow for more safety in the manufacturing process.

This is definitely an important technology,” said Chen. “It can enable a lot of new applications such as smart homes and smart agriculture, robotics or digitalized health, and many more.” 

However, there are many steps that must be completed before that point. A key milestone for the system is for it to be automated. Not only would automation allow for the system to run faster and with lower energy consumption, but it would also mean engineers wouldn’t need to constantly oversee it and manually determine the workload balance. 

“Bringing automation to an infrastructure that is standardized and homogeneous is much easier than bringing it to something that is much more heterogeneous,” said Pacifici. “How do I use more intelligent automation to deal with the challenges of managing [hybrid cloud] infrastructure? It’s pervasive, distributed, much more heterogeneous than what you have in a standard public cloud, cookie cutter data center.”  

Pacifici added that an ideal hybrid cloud cannot happen without automation. Simply put, humans aren’t capable of working fast enough for the system. Instead, they need to develop AI with the capability and reliability of engineers and developers to deal with the demand required for what would be a new class of applications.  

A system of this size and scope requires a substantial amount of energy. Ever conscious of the potential size of its carbon footprint, researchers focused on the concept of sustainability and how systems could run on both a lower energy demand and renewable energy. Pacifici said it comes down to having flexibility and making choices on what the correct path would be to get the desired result using less energy. 

“The flexibility is there, but then you need to have the intelligence that makes those decisions based on the different options and trade off that you have,” he said.  

One renewable energy source being explored is solar. IIDAI researchers led by Deming Chen and Klara Nahrstedt (the Grainger Distinguished Chair Professor of Engineering and CSL Director) are examining the feasibility to add solar panels on top of the awning of the CSL studio on UIUC's campus, which would allow the testbed to pull energy from a different source depending on conditions and the task at hand.  

Researchers hope to make several breakthroughs over the next few years. They admit it is an ambitious project, one that not just a single company or organization can tackle alone. Instead, they have a dedicated team from both UIUC and IBM, the innovation of an entire community, and the assistance of an ecosystem on their side.  

“These are tough problems,” said Pacifici. “We all want to make the world better. This is the goal we have.”

This story was published August 30, 2022.