Through visual analytics, Reconstruct saving building contractors time, money

While there are certainly many factors why construction projects aren’t completed on time and under budget, efficiency is a major component. While other industries have made strides this area over the last decade through technology and other means, Mani Golparvar-Fard, associate professor of civil and environmental engineering at the University of Illinois, contends that the construction industry has lagged behind.

“If you look into manufacturing over the last 20 years, more or less the value per worker has doubled, but in construction, the status of productivity has almost remained flat,” Golparvar-Fard said.

There have been a lot of research done to determine the factors contributing to the status of productivity, and five years ago the National Science Foundation put forth an effort to bring visibility to the problems in the construction industry.

Golparvar-Fard has been at the heart of the solution. A civil engineer by trade with a background in computer science, he has used an NSF grant, a partnership with fellow faculty members Derek Hoiem (computer science) and Tim Bretl (aerospace engineering), and most recently a Faculty Entrepreneurial Fellow (FEF) appointment to develop solutions under the auspices of a startup called Reconstruct. The FEF program provides faculty not only financial resources, but also time away from regular teaching duties to focus on bringing an idea to commercialization.

The team has developed a web-based platform, the Flying Superintendent, which uses drones to capture images and video of a construction site and reconstruct them in 3D. Through computer vision analytics, they are able to compare the progress of a site against information models tied to the project.

These ideas have not come in a vacuum. Golparvar-Fard served as a virtual design and construction engineer for Turner Construction, one of the largest firms in industry, where he had a major hand in scheduling and cost estimation. He came to the Illinois campus as part of Turner’s construction of the Ikenberry Commons Dining and Residence Halls and since 2010 has been actively working with more than 50 major companies while serving on the Illinois faculty.

“For the last 10 years, I have been interested in how we can use emerging technologies of computer vision and machine learning to take these sources of data and transform them into actionable analytics for construction companies,” Golparvar-Fard said.

In his years in the construction industry, Golparvar-Fard found that one of the biggest obstacles to productivity is the lack of understanding from all parties on just how much progress has been achieved on the job sites.

“The practices we have to track are really time consuming,” Golparvar-Fard indicated. “Over the past five years we’ve transformed paper based tracking systems to electronic based methods.”

Having autonomous video capabilities minimizes the need for field engineers to walk the job sites to assess the progress. In addition to being time consuming, the process is inefficient, leading to subjective assessments. Oftentimes reports aren’t made until a weekly contractors meeting.

“What that means is it might be a week between the time a problem happens to it surfaces to project management,” Golparvar-Fard said. “You’ve lost that week at a minimum and it’s quite possible that other problems might keep stacking up. In addition, stakeholders walk into meetings and learn about actual problems that have happened without any ability to analyze risk and predicting risk in the future. All of these are leading to poor decision-making.”

In creating a program to analyze the data, Reconstruct wanted to come up with a methodology and approach that is easy to understand by individuals with different levels of expertise.

“We wanted to make sure everyone from an owners side to the contractor’s side and designer side are all on the same page in terms of what needs to be done every day, how much work has been completed, and the risks with respect to the plan in the future,” Golparvar-Fard said. “The fact that we started looking into the visual data is primarily driven by the fact that images and videos are really easy to understand.”

First Golparvar-Fard began capturing video from smart phones, time-lapse cameras, laser-scanning devices, action cameras (like the Go-Pro), and more recently drones.

“With the Flying Superintendent, we started looking into how we could use such visual data to improve planning and scheduling and better coordinate the work that needs to be done between the owner, contractors, sub-contractors, suppliers, engineers and architects and bring better communication tools to ultimately improve transparency on construction sites.”

In addition to coming up with an end-to-end to solution to capture visual data, analyzing it, and reporting it in a way that is meaningful for construction practice, the team is working on predictive analytics to anticipate factors that are likely to cause a future delay.

“This transforms today’s construction monitoring control practices to be more proactive,” Golparvar-Fard said. “Ultimately it improves capabilities of planning and communication on job sites.”

Of course the ultimate proof has come in testing. The pilot project of the Flying Superintendent came at the construction of the $500 million arena for the Sacramento Kings, a project completed by Turner. Through Lincoln Wood, regional manager for virtual design and construction at Turner’s Northern California office, the company financially supported the drone operation. On the back end, Reconstruct successfully generated 3D reality models through web-based platforms, which ultimately benefitted Turner in the areas of safety, planning and project management.

“When we started at the Kings project, we had a simple working prototype,” Golparvar-Fard said. “With the team we had in place, including some great students we had on board, we were able to transform part of this prototype into a solution that could immediately start working. We received a lot of feedback and it really evolved our project from a prototype into a working solution.”

Following word of a successful pilot project, other companies, including many that had partnered with Golparvar-Fard in the past, expressed interest. The most notable was Clark Construction, who solicited the team to work with them on the completion of the McCormick Place Headquarters Hotel in Chicago. The proximity to campus allowed members of the team to become even more involved with the project.

“These experiences are very helpful for us in creating prototypes that are not only scientifically sound, but also engineered enough to actually add value to the bottom line, in other words, on schedule and on budget.”

The team received early backing from the National Center for Supercomputing Applications (NCSA) headquartered on the Illinois campus and later $1 million as part of the NSF effort to support studies on the construction industry. That made it possible to build the prototypes used on the Kings stadium site.

Through additional support from the University of Illinois’ Technology Entrepreneur Center (TEC), the Office of Technology Management (OTM) and this year the opportunity to participate in the FEF program, the Flying Superintendent has quickly moved toward commercialization.

“While the Flying Superintendent is application driven, contributing to the body of knowledge and science, we wanted to take advantage of the U of I entrepreneurial ecosystem to make the startup Reconstruct a success,” explained Golparvar-Fard. “Between us, we had knowledge of the application, knowledge of computer vision, knowledge of robotics, and a group of students who were not only interested in doing basic research, but also excited about taking the solution and applying them to practice.”

Through that ecosystem, Reconstruct was able to create a proper business model and expand the team to include students and faculty from engineering, industrial design, graphics, and the College of Business. Faculty members on the team have also developed a course around the Flying Superintendent.

“The fact that we had the industry’s support allowed us access to projects even before we had a company,” Golparvar-Fard said. “The TEC did a great job in not only technical support, but also financial support in helping us understand the elements of business and commercialization that our team really needed mentorship on.”

As a result of these efforts, Illinois is emerging as leaders in construction monitoring.

“We have contributed to the body of knowledge, we are creating the technology, we are showing that we have engineering power (including graduating students entering the field) to create systems that work and contribute to the bottom line of the projects,” Golparvar-Fard concluded.

In addition, Reconstruct is cognizant to establish projects that show the technology is working, creating a plan for technology development and streamlining the process of turning ideas generated from campus into OTM licensed prototypes to ultimately make a national and global impact. The company is also looking beyond mega projects to those as small as $1 to $2 million.

From a commercialization standpoint, support from TEC enabled the team to pitch the technology through pilot projects to venture capitalist firms with at least four Midwest VC firms backing Reconstruct. -- Serra Capital, IllinoisVENTURES, Hyde Park Angels, and Harbor Street Ventures.

“Now we have a startup company in place, we are thinking about this through a two perspective lens, project leadership and thought leadership” Golparvar-Fard said.

Future goals for Reconstruct are to prove that drones can safely move about autonomously even out of the sight of its operator (which is currently the FAA rule of law) and using ground robots to automate data capture indoors.

“The ecosystem that we have at the College of Engineering and at the University of Illinois, the vision that was set by the Dean for the FEF program, and the leadership at TEC has been instrumental in exciting students, from both an engineering perspective and a business perspective. It ultimately allows us to have a societal impact. The National Academy of Engineering has identified improving productivity in the United States as imperative. As a result, we believe we are establishing our leadership in construction automation in robotics as well as visual analytics.”