The evolution of industrial maintenance

A few months ago, we presented the collaboration with the Rovereto-based company PAMA, a world leader in the manufacture of large machine tools, aimed at adopting innovative technology that would involve the development of computer vision modules for remote troubleshooting and industrial maintenance through an application based on augmented reality, thanks to the application of research from FBK’s Vision Technologies Unit (TeV) led by Stefano Messelodi. In this update we retrace the steps taken in the meantime and outline new goals and open challenges for the near future.

The reasons prompting the initiative

• The growing need to digitize and automate industrial processes has led to the search for new tools and technologies to facilitate systems maintenance and monitoring. The adoption of digital solutions can improve efficiency, reduce downtime and help reduce human error.
• The Covid-19 pandemic has had a significant impact on many areas, including the industrial sector. Travel restrictions and social distancing measures made it harder to visit factories for regular inspections and maintenance. It prompted finding alternative solutions to compensate for the shortage of traveling maintenance staff.
• Delegating supervision and maintenance to customers, providing them with tools and resources to operate machinery o their own, can increase customer satisfaction and enable them to nip many problems in the bud. This approach could reduce the cost and time required for maintenance work by trained staff.
• Implementing innovative solutions such as Augmented Reality for machinery maintenance and supervision can provide customers and facility operators with useful tools to understand and solve systems problems more effectively, and offers competitive advantages to companies, letting them stand out among competitors by positioning themselves as leaders in the adoption of emerging technologies.

From these background premises, the objectives of the industrial collaboration project focused on four aspects:

• The creation of a web interface-based tablet application (web app) for predictive maintenance of industrial machinery, supported by Augmented Reality
• The development of software that will ensure the compatibility of the web app with various devices, such as Android and iOS ones
• The interactive visualization of the operational status of PAMA machinery components
• In case of failure, the web app should allow intuitive navigation of documentation and other useful troubleshooting information. Ultimately, it should allow the opening of a maintenance trouble ticket.

In this process, FBK’s input was developing an Augmented Reality module to be integrated into the PAMA web app. Specifically, the use of Augmented Reality allows interaction with components directly through the tablet camera.

The main challenges for these results to be achieved were the development of markers that could be detected by the tablet camera remotely and that would be resistant to adverse environmental conditions (such as oil and dust accumulation from metal cutting), as well as the ability to ensure reliable operation in dim-light environments.

Web app features and operation

• Login and authentication: operators access the application through a secure web interface, entering their credentials to authenticate. This ensures data security and allows the user experience to be tailored to each operator’s needs and privileges.
• System selection: after authentication, operators can select the specific machinery they wish to work on. The application can provide a list of available machinery, filtered according to the operator’s skills and permissions.
• Display of component information: once the machinery has been selected, the application will display a graphical representation of the key components being monitored. Operators can select individual components to view additional details, such as description, operational status, date of latest maintenance, and other relevant information.
• Augmented Reality Interaction: using the tablet camera, operators can frame machinery components and view additional information in real time through Augmented Reality. This allows them to quickly obtain information about the operating status and make more accurate and intuitive diagnoses.
• Predictive analysis and operation monitoring: the application uses predictive analysis algorithms to assess the operational status of machinery components. Based on the data collected, the application will assign a color to components: green (smooth operation), orange (possible non-serious anomaly), red (possible serious anomaly). Operators can thus quickly identify possible issues and plan appropriate maintenance actions.
• Creation of maintenance tickets and intervention requests: when encountering malfunctions or problems, operators can create a ticket directly from the application, providing details about the problem and actions taken. This facilitates communication between operators and maintenance staff, allowing problems to be solved more efficiently.
• Maintenance history and reporting: the application records actions taken by operators, tickets opened and maintenance performed on equipment. This data can be used to generate in-depth reports and analysis to support maintenance planning, operator training, and production process optimization.
• Updates and support: the application can be updated and upgraded based on operator feedback and emerging needs. The application can provide support and assistance to operators, including guides, tutorials, and contextual suggestions to help them use the application and address problems encountered.

The experience gained opens up possible future developments

• Object recognition: implementation of artificial intelligence algorithms to improve automated recognition of system components without the use of markers. This could further simplify the interaction between the operator and the application, reducing the time required to identify and assess the state of components.
• Improved stability: continuous refinement and optimization of the application and Augmented Reality module to ensure a smooth and stable user experience. This could include dynamic adaptation to internet connection quality, device resource management, and optimization of rendering and computation processes. It might be useful to consider integration with other wearable devices, such as Augmented Reality glasses, to further facilitate operators’ use of the application.
• Advanced predictive analytics: implementation of more sophisticated predictive analytics techniques, based on deep learning, and extension of the database useful for training to further improve the application’s ability to predict anomalies and malfunctions in system components. The goal is to predict failures more accurately and provide operators with more detailed and specific information about potential problems, enabling them to plan targeted and efficient maintenance interventions.