Imagine you have to reduce your software development service provider selection matrix to one or two criteria. What should they be?
(BTW, I personally believe that the people who create RFPs and RFIs for selecting outsourcing vendors should ask themselves what is important for them and why and then concentrate on getting this information instead of asking about corporate social responsibility on several pages. Plus, ask about the price, of course. Hopefully, this little post will at least make you stop and think. Now, back to the topic.)
I can almost hear people saying “Cost” or, if they have more experience, “Total Cost of Outsourcing, aka TCO,” which is a valid answer, but TCO is a “catch all” thing; any good/bad factors can be converted to money and become a part of TCO. The hard part is coming up with a formula for TCO that you can actually use in real-life situations, so it’s hard to estimate TCO reliably before you actually start working with a provider.
Besides, in my opinion, you should first select the providers you are confident can drive the project to success. And then select the cost-optimal solution from the shortlist of two to three companies on the last step. So, is there some factor that is more important than cost-per-hour and easier to research and base your decisions on in real-life scenarios than TCO?
OK, now I can also hear people saying “Previous experience with similar projects.” That’s also well and good. But for your own benefit, I hope that you are not just trying to repeat what others have done but doing some original thinking. And as a result, the top experts in your narrow application domain are collected in your organization, because nobody did before what you are trying to do now in the way you are trying to do it now.
“Fine,” you say, “forget about the narrow domain. Get somebody who has experience with the technologies.” And again, I agree—it’s important. But nowadays, technology is easy; as I wrote before, so many providers will satisfy this criterion. Providing additional training is easy, widespread, and will be needed anyway.
Besides, it’s more cost-efficient (in TCO terms) than over-optimizing your choice based on deeper knowledge in a narrower field. So, although it remains a valid outsourcing provider selection criterion, this doesn’t look like a key one either.
Now, I have criticized every single suggestion I pretended to hear from you. Do I have anything better to suggest? In fact, I do. And now you are free to criticize my proposal.
I believe that the two most important criteria in selecting the right software engineering outsourcing provider should be:
1. Ability to communicate
I mostly mean engineering communications on the project: discussing requirements and implementation ideas, figuring out the architecture of the legacy system, discussing the right compromise between time-to-market and feature set.
I also mean: as opposed to blindly coding per some pre-defined spec that is always incomplete and becomes outdated the moment somebody writes it (and, btw, charging for every single time you change something in that pre-defined set of requirements).
I’ve already written several posts on the subject, so please, take a look here for my arguments in support of that statement.
2. Ability to test efficiently in your technology area
That deserves additional explanation. Testing is the crucial part of your software development lifecycle that defines much more than the quality of the resulting product. It also greatly affects such important things as maintainability (can you change something and still be sure that you don’t ruin anything) and portability (same argument). The quality of testing organization defines if the product you create now will still be alive after a couple years and will be extendable, will be able to be integrated with new gadgets and technologies, and will be supported with relative ease. Or, it turns into a sustaining nightmare that you’d be happy to rewrite from scratch in a new family of products while paying money for a big team of always-unhappy-with-what-you-ask-them-to-do sustaining/field-support/debugging engineers.
It’s not that I’m inventing something new here. The importance of proper testing is well known and widely recognized. From the ageless “The Mythical Man-Month” by Frederick P. Brooks, Jr. (that allocates as much as 50% of a software task to component and system tasks as a rule of thumb) to Test-Driven Development (TDD) methodology, many great minds see testing as one of the key components for success. All I’m saying is that you should apply this wisdom to your provider-selection process.
If you are creating embedded software, for example, check if they know how to manage automated testing for embedded software. Can they measure coverage in system-level software or firmware, if that’s your domain? Check if they are familiar with verification & validation requirements for medical devices, if that’s what you are expecting them to do.And it’s not just important to know how to do that and have experience doing that. It’s important to know how to do that efficiently—which can often be rephrased as how to automate the testing suites so that the tests can be run often and easily, provide meaningful and repeatable results, and remain manageable at the same time (i.e., don’t require changing in multiple places after changes in your software).
Bottom line. If you want just a short summary of what I’ve tried to convey in this post, here it is:
- Focus your provider-selection process and all your RFPs and RFIs on the important stuff. Don’t waste time on what’s non-essential.
- Defer cost comparisons until the second stage of your process: selecting the one from the shortlist. It’s hard to calculate TCO and compare hourly rates before you are sure that the right capabilities are in place.
- Don’t over-optimize provider selection based on in-depth knowledge of specific technologies and a vertical domain. You will overpay and still have to provide ramp-up training in the specifics of your application.
- Concentrate on the following two criteria when selecting vendors who can successfully carry out the task:
- Ability to communicate. Otherwise, you are doomed to repeat all those outsourcing failure stories where the sides didn’t understand each other and budgets and development times grew several times bigger than the original estimates due to bureaucratic change management.
- Ability to test efficiently in your technology area. That often means understanding how to properly organize automated testing—plus, of course, understanding how to comply with all regulations while testing.
