Like any other field, there is no “silver bullet” when developing software for medical devices; there is not one single solution that is always justified and effective. However, as Auriga’s experience shows, there are two methods that, when combined, make it possible to level out the drawbacks and emphasize the advantages of a program.
At the first stage of the development of a medical device is the hardware-software complex. In this stage, there are practically no alternatives to using a simulator. This is just one of the reasons why full-featured simulators are increasingly being used. Currently, the term “digital twin” most accurately reflects the purposes and methods of application.
In the design and prototyping stage, individual components are developed and tested in the hardware. Then, they are separately assembled into larger units and subsequently united into a complete system. Consequently, this causes a delay for software developers and testing engineers.
Benefits and Drawbacks of Digital Twins in Medical Device Development
There are several basic scenarios that affect the use of digital twins in development:
- While developing a hardware and software platform, the long hardware development period does not allow software development to begin before functional hardware prototypes are developed.
- Digital twins allow software development to begin earlier, but also helps hardware developers. For example, several options for possible solutions can be simulated digitally to choose the best option.
- Various testing options are available when constructing hardware or its components. Testing options can check how the software detects and responds to situations in which the hardware malfunctions. For final checks, the destruction of real components is usually applied, but at the stage of software and control methods development, this is an excess cost.
- It is impossible to equip the workplace of each developer or test engineer with his or her own copy of the device being developed as there are usually many device options being developed concurrently. The costly nature, special condition requirements, or sheer size are factors that disallow certain equipment to be available to each developer or test engineer.
- The study of code coverage for all test suites, from unit tests to automated and manual tests, is necessary to achieve maximum code coverage;
- Test automation: when developing a digital twin, it is important to make it possible to interact with the twin in an automatic mode, which significantly reduces the efforts of the testing team and reduces possible human error.
All this leads to what is called a “left shift” (in the project schedule) in the industry: everything that can be done earlier must be done earlier. Daily regression testing is implemented, and every morning, developers receive a report on the performance of the entire complex.
On the other hand, digital twins have at least two serious drawbacks; they are not critical at the development stage but become significant when the product is about to be released to the field:
- Simulation gap: there will always be a difference between a real physical device and a simulated digital twin based on the mathematical model of the Even if our mathematical model is as accurate as possible or good enough to solve the problem, the real device may behave somewhat differently.
- Simulator validation: according to medical standards, any tool that functions in the automatic mode must be validated, i.e. we must confirm that it is actually performs as intended.
The simulation gap usually becomes noticeable when we already have a hardware prototype and can study its characteristics in detail. In this case, firstly, the simulator models can be modified so that they match the actual behavior as accurately as possible, and secondly, the simulator models can be used to make improvements. We know, for example, that a hardware error rarely appears and only under certain circumstances. Therefore, we modify the model so that this error appears when we need it. This allows for us to accurately test how the software logic responds to this and any other errors (or a combination of them).
As for the validation of the simulator, the more complex and accurate the models are, the more difficult it is to validate the simulation. Often, this significantly increases the total cost of the project. However, the release of the medical device itself—and not its simulator, however ideal it may be—is decisive for the developer. Therefore, at this stage, the simulator usually remains an internal development tool while the developer moves on to another solution.
How Robots Help in Medical Device Testing
A robot has several advantages when testing medical devices:
- No modifications are required for testing in the tested software.
- No hardware modifications in the tested device are necessary only for testing.
Thus, robots can be used to test the same device, which are then produced at the factory and delivered to end users. There are a few more benefits:
- Robots allow for any number of devices to be tested in parallel.
- The time parameters of tests are always recorded (for example, if an error occurs only after a hundred consecutive keystrokes of one or several buttons on the device in second fractions, a human will never be able to repeat such an exact test twice).
- It is possible to vary any time parameters of tests widely.
- All tests are automatically recorded, and it is possible to investigate suspicious or doubtful tests.
- Obviously, robots do not get tired and do not go on vacation.
If the results of the robot’s work are considered, then the medical standards require the validation of the robot itself. Nevertheless, the medical device manufacturer will still have room to maneuver:
- A robot is usually much simpler, and the cost of its validation is much lower than the cost of validating a full-fledged digital twin.
- The formal final validation of the medical device is carried out manually, and the robot is used to detect regressions, stress testing, etc., which significantly reduces the cost of manual testing. Final validation can only be done when all automatic tests are successful, and we are 100% sure that manual testing will not fail and the test engineers will not waste time and effort.
Summary
In conclusion, the combination of the abovementioned methods allows developers to use the advantages and level the disadvantages of all approaches:
- Digital twins: best used at the earliest start of software development, in testing strategies, and debugging and investigating hardware solutions.
- Robots: best used for the testing of the exact same devices that will be delivered to consumers instead of extremely similar devices.
- Manual testing (if necessary): offers no validation costs and combines the most efficient use of labor costs and working time.
