I have been working in the field of embedded programming for the last 20 years. When I think about it, I envision it like the air around us: it is everywhere, yet we cannot see it. It makes up the core of software and hardware, yet most users are completely unaware of its importance. And the only time they gain any insight into its role is when it fails to provide the right functionality. Imagine your Fitbit’s connectivity is lost and can’t be restored no matter how hard customer service tries to help. Well, not so scary. But what if a patient monitoring unit or an infusion pump or dialysis system software in the ICU fails? Or the aircraft taking you to your Bali resort will not provide the pilot with accurate data? And I bet you wouldn’t want to run into a malfunctioning self-driving vehicle.
If that seems scary to you, let’s ramp it up a notch: many manufacturers, inspired by the “automate everything” trend, are moving forward from pure mechanics to electronically controlled devices. What they are usually missing is the fact that electronics increase complexity significantly, and software multiplies it. Simply put, they need to do way more testing before releasing their products.
Talking about the testing of software-embedded products, we cannot totally ignore the fact that testing is just an instrument for a broader verification and validation process. We are determined to deliver reliable and safe solutions to the market, and we need to take care of not only the bugs themselves but also the full range of product requirements. And there are tons of them, from hardware requirements, power consumption, and response time to the stability and reliability of software components. So at the end of the day, we need to ensure that 100% of the critical and major requirements are covered and tested. And you’d be surprised by the number of minor defects left to be tested and fixed—or not fixed if the product owner decides it’s not worth it considering the time and effort needed.
Auriga has published numerous posts on embedded testing, hints, pitfalls, and various related topics on its blog. Yes, though technologies are marching ahead quite hastily, automation for embedded testing is lagging behind. And, yes, there are numerous completely unbiased reasons for this situation. We as a company have been there and done that. Development of a toolchain and test suites for hardware that has not been released even as an alpha? Check. Testing of a hardware platform located somewhere in the remote customer’s location with an eight to ten-hour time gap? Done. Testing and bug fixing for the OS HAL/BSP/LSP, its drivers, and the kernel itself, including the entire ecosystem with numerous versions, updates, new features, and peripherals to support? We’ve been doing it since the mid-90s.
And every time, we’re looking into any possibility for test automation. Manual testing is time and resource consuming, and in some cases, it’s almost impossible due to the operation speed of the embedded devices and components. And though monthly rates for manual testers are not skyrocketing, projects are still costly for those taking this road. Test automation is rather expensive and tricky, especially bearing in mind the deserted market of test automation tools for embedded and system-level software. I don’t mean there is nothing we can do with it. On the contrary, if you have a proficient and experienced vendor who knows the business and industry specifics, he will obviously create some test suites from scratch. They will fit your requirements and be tailored to the unique features of your product. Like the robot one of our teams created for our customer to run tests for the complex medical ICU equipment physically imitating the users’ behavior on the production device, so no simulations or modifications were implemented, and the test run duration decreased from three weeks to just a few hours, thus making it a perfect case.
But we have entered the so-called era of the Internet of Things, AI, virtual reality, machine learning—you all know the buzzwords. I see AI as a new frontier for embedded testing—not in terms of writing or executing tests. No, I am not THAT optimistic. But it seems to be potentially a very powerful tool acting as a knowledgeable and comprehensive expert system. Given the input data about the embedded system and hardware components, it will be able to warn testers about possible bottlenecks and inconsistencies and even suggest the most suitable set of tests to ensure full requirement coverage for the product.
Anyway, while we are waiting for AI-tagged initiatives here, life goes on, and the only working solution for automated embedded testing right now is to find the right vendor for your project. I plan to cover the algorithm that will allow you to do this in my next blog piece.
