Homegrown tech could improve site safety and productivity

In what Monash University claims to be an Australian-first, a team of experts at the University has developed a piece of technology that it said will streamline safety and monitoring procedures on mining and construction sites.

The technology generates accurate, synthetic images of machinery - such as cranes, scissor lifts, bulldozers and dump trucks - which create large datasets for training deep neural networks to analyse the use of machinery on-site and in a variety of conditions. 

It was developed by Dr Mehrdad Arashpour and his team of PhD students and postdoctoral researchers in Monash University’s Department of Civil Engineering.

This new method removes the need for cameras to be placed on construction machinery, Monash University said, which has time, cost and privacy implications.

The technology is currently under Australian patent review.

Digital twin technology "the next frontier of construction"

Construction automation facility gets green light

Doosan "transparent bucket" breaks cover

The Monash University team has designed the technology to process millions of images of any type of machinery in a short amount of time, and provide detailed information on its key physical and operational features.

In other words, before using an excavator at a building site on a wet day, workers can identify all the safety features of this equipment within a digitised version of the working environment, including rain, clouds, as well as reduced light and visibility.

Over time, this technology could be expanded and used on any construction site across the world, depending on the type of machinery in operation, Monash University said.

“Operations involving heavy construction equipment are a critical component of most projects. Heavy construction equipment, vehicles and workers are often required to work closely due to spatial limitations and tight schedules, which often leads to suboptimal performance both in terms of safety and productivity,” said Dr Arashpour.

“Preparing large training datasets for specific tasks is currently a manual process, which is highly time-consuming, labour-intensive, error prone, and subject to privacy concerns.

“Our method randomises various critical features of the scene, such as equipment pose and texture, scene texture and lighting, camera location and field of view, and adds other elements to the scene such as simulated dust and occluding objects.”

Presently, safety compliance training procedures involve placing cameras on site to capture images of the machine in operation. From there, the images are labelled manually, including key points of each piece of equipment.

The generated data set is then used to train the artificial intelligence (AI) models and inferences are made regarding classification of different equipment, pose estimation and activity recognition.

The Monash-developed method uses game engine technology to generate CAD models of heavy machinery which are then accurately and automatically annotated.

Dr Arashpour and his team have so far tested several deep neural networks on the real images of the equipment. The results obtained by training deep neural networks using synthetic images are almost always as accurate as real images, he said.

“As compared to traditional data preparation pipelines, the proposed method does not require any manual annotation, which is a labour-intensive and time-consuming process,” he said.

“Furthermore, synthetically generated datasets are advantageous to manually labelled datasets as they can produce pixel-level annotations for the key points of interest. In contrast, manually annotated datasets are susceptible to human error, and monitoring the quality of annotation is also a cumbersome task.

“Our home-grown technology has the potential to modernise the building and construction industry across the world, and improve safety, quality and productivity.”