In our last post, we discussed handling complex issues in advanced machines by breaking down the troubleshooting process into component-level troubleshooting and then building more sophisticated hierarchies for handling high-level symptoms. This post will take this concept further and see how we can optimize the machine troubleshooting flow using IoT machine data to improve precision, consistency, and speed dramatically!
Industrial manufacturers are looking for ways to optimize service performance, reduce machine troubleshooting time, and increase equipment uptime. One approach is to use guided troubleshooting software, and another is to focus on the Industrial Internet of Things (IIoT) for providing real-time data from the machine at the component level. Combining the two yields massive operational and strategic value for optimizing machine troubleshooting and reducing equipment downtime.
So, how do you utilize IIoT data to reduce troubleshooting time?
—
A few years ago, field service technicians relied on their ears, nose, eyes, and experience when troubleshooting machines in the field. Then some of these machines started to feature a built-in display where certain state data and even logs could be found. That data would help the technicians get a better insight into how the machine was doing, any error codes that might have been produced (although error codes were still scarce and unreliable), and maybe even a bit of information about what the machine was doing just before the incident.
Later, internet connections became more abundant. The machines could transmit data to the OEM, who could monitor the machine’s performance and pull out historical data when an incident occurred, and a technician had to be dispatched. Some OEMs were collecting massive amounts of state data, but for a long period, all that data wasn’t being used optimally.
Collecting machine state data
We started seeing OEMs retrofitting data collection and transmission devices to older equipment to pull data from products manufactured before the new data capabilities.
Fast forward a few years, and the onboard controller of most new machines now contains extensive data and an abundance of log files, error codes, state data, performance charts, and much more for a great range of components inside the machine. All that data can be utilized to generate a complete picture of how the machine performed before the incident, which will be a strong indicator of where to start troubleshooting the product.
For many years most companies became very good at collecting and storing massive amounts of machine data, but only very few were able to put that data to good use. Of course, the BI departments can pull trends and telemetry from the warehouses and build impressive dashboards for even the most sophisticated angle on almost any piece of interesting data.
With the advent of powerful computers and machine learning, some companies can now utilize all that data for much more active purposes. Intelligent machine learning algorithms can actively predict breakdowns before they occur and help prevent unscheduled downtime.
However, even with the great promise that machine learning has and the endless opportunity to learn from data – it’s all still just that … data. Data cannot stand alone without expert knowledge to be useable in a troubleshooting scenario.
Combining data with know-how
To really make use of all that data, we still need to apply experience to make a qualified interpretation of the data for it to be most useful in a troubleshooting situation.
This is where the troubleshooting software comes into play. The software needs to capture the expertise of the best troubleshooting experts in the workforce and store it in a digital knowledge model that can be distributed easily to the entire workforce and optimized using machine learning. In our experience, knowledge models based on Bayesian networks, using conditional probabilities to drive the suggested repair actions displayed to the technician, are extremely good for this purpose, as they are precise and very scalable.
The real power of data when it comes to troubleshooting is seen when man, machine, and data work together to achieve an amazing troubleshooting experience that makes everybody troubleshoot complex issues in advanced machines on the same high level. By altering the underlying premise for the troubleshooting suggestions using real-time data, the advice given to the technicians by the system becomes context and state aware, and the problems are fixed much faster.
Let’s look at that next…
Changing root cause probabilities with data
One of the biggest advantages of automatically loading and interpreting real-time machine state data using IIoT is the ability to change the underlying probabilities of the root causes of a troubleshooting guide (the knowledge model) even before the engineer sees the first suggested solution action on the screen.
Troubleshooting will certainly have to be handled differently if the machine in question has been operating for 8000 hours vs. 1000 hours. Weather conditions, ambient temperature, seasonal effects, and many other factors all have a role in troubleshooting and by injecting that type of data into the process, the troubleshooting system will be able to suggest solutions based on the actual state of the current machine in its current environment.
Combine that with machine-specific historical data that can reveal how this particular piece of equipment has been operating throughout its lifetime, current error codes, and sensor values, and we get a troubleshooting experience designed and optimized for each individual machine every single time.
But how much effort does it take to build something like this?
Here is an example …
Building an IoT-driven troubleshooting guide in one day
One of the best examples I have when it comes to building an IoT-driven troubleshooting experience is from a company that manufactures equipment for liquid chromatography. These are pretty advanced pieces of equipment, and troubleshooting breakdowns can be quite difficult.
We conducted a one-day workshop in the company headquarters, and the team started the day by bringing the equipment into the conference room, hooking it up to the network, and bringing it online. We then went over, on a high level, what we needed to achieve our goal of demonstrating a fully operational data-driven troubleshooting guide to the managers by the end of the day.
The subject matter experts present quickly agreed that there were two central pieces of data that would be very relevant to the troubleshooting process:
- What type of pump is installed in the equipment?
- The machine can have a different type of pump installed depending on its application.
- What is the result of the pump leak rate test?
- The test determines leak rates in the pump chambers. It’s a component-level test.
Determining which type of pump is installed in the equipment is fairly straightforward, but it is still a manual operation that takes time, so having the system automatically figure this out would be a benefit for the technician.
Likewise, performing the leak test requires that the technician starts the test application on his laptop, connects to the device, sets up the required parameters, conducts the test, and finally, interprets the results of the test. If we could automate both the execution of the test and the evaluation of the data, it would really save the technician a lot of time.
The team told a developer what they needed and the developer promised to be back later in the day with a small piece of code (a data collector) that could connect to the device, obtain product configuration data, execute onboard diagnostic tests and return the data.
We then moved on to building the actual troubleshooting guide by capturing the causes, probabilities, actions, and questions needed to troubleshoot “Baseline issues.”
The team built the guide, the developer built the data collector and by the end of the day, the team was able to start the troubleshooting guide, which in turn automatically got the pump type, performed the automated test, evaluated the test results, and suggested the first corrective action to the technician.
It was quite impressive to experience a team building a complete IIoT data-driven solution in just one day!
That kind of automation can really save a lot of time and resources at scale in service operations.
Let’s move on to look at the impact you can expect from deploying automated data-driven guided troubleshooting.
Impact on your service operations
GE Digital coined the term IIoT in 2012 and helped form the Industrial Internet Consortium to help accelerate the development of the interconnection between connected devices, data, and people.
For Dezide, using machine state data when troubleshooting equipment of any kind wasn’t a new thing in 2012 as we had been doing it for many years under a different name, very simply called “Data Collection.” Every time there is a question in a troubleshooting guide, there is an opportunity to answer that question automatically using interpreted data, thereby saving the user massive amounts of time when fixing problems. As such, we were using data for troubleshooting internet connection issues at telecommunication companies that were capable of measuring physical connections and protocol-level data stream performance at a very early stage. Being able to automatically perform line tests on DSL internet connection cables on both hardware and software levels and have the troubleshooting system interpret the results of the tests was an enormous benefit. It took away difficult manual evaluation of complex data, replaced it with automated uniform data interpretation, and now we were able to present informed solutions to the users in an optimized way.
One customer saw a significant reduction in escalated cases. Still, even more interestingly (and none of us predicted this, and none had any KPIs for it), the technicians in the field quickly started to feel more informed when on-site after the case had been escalated. The equipment state data was used to troubleshoot as much as possible in the service center before dispatching the technician. On-site, the technician could now see the steps performed by the remote team, and they could see what the state of the equipment is based on the data obtained by the Data Collection module via IoT integration.
Likewise, innovative consumer electronics companies like HP Inc. were ahead of time, and already back in 2005, we were building an automated self-service guided troubleshooting experience together with HP Inc. that would run on consumer computers and assist the user when facing printer issues. The solution was installed on 70 million computers all over the world, executing several millions of troubleshooting sessions every month, and each and every one of those sessions would automatically pick up data from the user’s computer and printer to make the troubleshooting experience as smooth as possible. Basic information like operating system, disk usage, software versions, and connection type (USB or network) was utilized but also printer-state data from the device itself like error codes, battery conditions, and much more. Of course, this was the early days of the Internet of Things. No one was talking about the Industrial Internet of Things for the industry yet …
Heavy equipment manufacturers weren’t as fast to enable data collection capabilities, but when the IIoT term caught traction, and more people started realizing the potential power of the concept, the OEMs started to show an interest in automating important parts of the troubleshooting process too.
Common for the examples above is that the results of these projects are very tangible operational results that can be measured in service KPIs like increased first-time fixes, reduced troubleshooting time, fewer spare parts used, reduced average handling time, and ultimately improved customer satisfaction.
Besides the measurable KPIs, a range of more intangible benefits becomes apparent, such as addressing safety issues and enabling uniformity in data interpretation across the entire workforce.
From a company perspective, IIoT can in many industries still be regarded as a differentiation factor that enables these benefits, and from a customer perspective, it can be seen as added benefits that indirectly increase their service experience.
However, as every week goes by, the benefits of utilizing IIoT are becoming an increasingly dominant factor for companies to maintain their competitive abilities in regard to ROI and service delivery. For the customer, IIoT capabilities are no longer just an added benefit; it is becoming an expected integral part of service products.
Let us, therefore, look a bit closer at IIoT from the strategic perspective of service operations.
Impact on your service strategy
The direct operational results achieved by optimizing machine troubleshooting through the use of IoT data are impressive, but if we look at the impact it can have on service from a bigger perspective, things get even more interesting.
For service departments looking at using IoT data to improve their service offering, there are key strategic elements that should be considered. Using a central troubleshooting knowledge base for capturing and sharing know-how is, of course, the foundation for taking the service offerings to the next level. But, if we add automation, we can deliver a truly seamless service experience.
The service departments that I work with have an explicit strategy to deliver best-in-class service to their customers and dealers and actively strategize around automation with IoT data. The opportunities for building a seamless automated troubleshooting knowledge base and the potential for providing a world-class troubleshooting experience to the entire workforce are seen as a growth channel.
In my experience, the service divisions that do best are the ones where everybody troubleshoots at the same high level, and that is only possible if all technicians do things the best way possible and the same way every time, and that is possible with automated guided troubleshooting and IoT data, even in offline environments.
There is so much potential in rolling out automated guided troubleshooting solutions, not only for increasing the effectiveness of the service workforce for your own products but also for foreign and competing products – an opportunity to increase service revenue.
Turning service into a revenue stream is something that we see service organizations take on a more active strategy around as the profit from new product sales starts to diminish. In order to gain a competitive advantage, companies must turn their service technology into products. Products that must be state of the art and very easy to use. That ease of use can be achieved with IoT automation.
Considerations for offline environments
Automatic guided troubleshooting based on IoT data should also be possible for technicians working in offline environments. Even if a technician is working on a machine in a remote area with no internet connection, it should still be possible for the technician to hook up his laptop or tablet/phone to the machine and get real-time state data on the spot.
Most solutions rely on IoT data captured and stored in data warehouses for processing, but if that data is not available when the technician is working offline, we don’t get the full optimized troubleshooting experience.
In the optimal scenario, the troubleshooting solution can automatically get the data it needs when it is online, automatically detect if that is not the case, and then get the data (possibly another set of data) straight from the machine itself in the field.
I have worked with several clients where the impact of the offline ability on the performance of the technician is substantial, as the technician will get all the benefits of the automated troubleshooting solution even when working in offline cases.
Conclusion
As the term IIoT has become increasingly more present in the context of business enterprises, it is also becoming an increasingly important part of the digital transformation of service as IIoT is essential for troubleshooting complex equipment if we want to realize massive improvements in:
- Precision
- The top engineers have determined the optimal evaluation and interpretation of complex data, and that interpretation will be applied in every troubleshooting session. Even in remote offline environments.
- Consistency
- Questions will be answered the same way every time as the difficult data analysis is done automatically, leading to increased precision in troubleshooting.
- Speed
- Technicians do not have to dig out the data manually or learn how to analyze the data. The system will already have analyzed hundreds of parameters before presenting the first solution.
I often speak to people who are working with IIoT, trying to comprehend the possibilities of using machine and context data, and for many companies (both big and small), the IoT is still a big project, a vision, a thing of the future – something they are working on but doesn’t expect many outcomes from yet.
Most projects I see around IIoT are still in the data collection phase. Companies establish the infrastructure for collecting, storing, and processing incredible amounts of data. Several software solutions exist for visualizing huge amounts of data, and there is, of course, a lot to learn, but the real operational value is realized when data is utilized on a daily basis, such as in machine troubleshooting.
Bringing together machines, data, field service technicians, and troubleshooting software results in solutions that deliver improvements in field service troubleshooting efficiency like never before. The reductions in troubleshooting time and increases in first-time fix rates combined with higher precision and quality and less used spare parts deliver increased customer satisfaction and enable the productization of service.
Today we encourage all our customers to leverage the power of machine data as it’s driving very powerful outcomes for the support and service departments.
