Choosing a software vendor is a potentially daunting task. Software is abstract and theoretical, while at the same time being very expensive. This is a potent mix that can be disastrous if placed in the wrong hands.
Typically organizations that out-source a software project do so because they don't have the necessary internal skills to complete it. Without these skills the company is ill equipped to assess the competency of one software vendor over another.
The following criteria are provided to help IT managers and executives go under the hood and reveal the inner workings of a potential vendor. These questions are not "technical", and should be easily handled by any competent sales person who understands their business. However, we have come across sales representatives that are nothing more than smooth talking back-slappers who will be easily stumped. You may want to ensure a technical/operational person accompanies the sales person for your meeting.
This is the process or plan the vendor follows to complete your project. The purpose of a process is to stop unexpected problems from popping-up and putting the project off schedule, off budget, or both. The process forces a discipline on the development team to answer crucial questions that help mitigate risk. There are two dangers to watch out for:
First, the companies that have no defined process. These are in the minority but certainly are out there. If you ask "What is your development methodology?" and the answer is "I don't know what you mean" cut the meeting short, you are talking to the wrong people.
Second, are the companies that invented their own process. These are in the majority because the market hasn't yet enforced the importance of a standardized methodology. These companies may have fancy acronyms for their process, but watch out: you have no guarantee your project will stay on time or on budget. There is a tremendous difference between the quality and accuracy of a software process used successfully in 10 projects vs 10,000 projects. The "official" processes are tried and true by thousands of companies across thousands of projects around the world. The easiest way to discover if a development process is authentic or fabricated is to ask "What is your process based on?" if they say "Experience" or "We've always done things this way" you know you are dealing with someone who has invented their own wheel.
The industry standard development methodologies are:
Finally, you may be interested to know the actual cost involved in adopting a development methodology. The hard cost is $50 to buy the book, with a soft cost of the discipline to implement it within the organization.
There are many ways of describing a theoretical idea with a diagram. In order for the document creator to correctly convey their ideas to the reader, they must use an understood notation. UML is the most widely adopted and universally accepted software documentation notation.
A universally understood format is protection for you as a client should you need to change vendors. If you have spent $20,000 for the first phase of a project and then must switch vendors, documentation in UML allows you to salvage your investment and help the second vendor hit the ground running. If it is a hodge-podge of Visio flow charts and word documents, kiss your money good bye.
There is no reasonable alternative to UML, if the vendor doesn't use UML pass them by.
It is important to know what the vendor expects you as a client to do. This can shed a lot of insight into the type of resources they do or do not have at their office.
A lesser vendor will expect you to do a lot. You will need to be very specific and write a lot of documentation to tell the vendor what you want built. If the vendor is putting the onus on you to direct them, chances are good that they won't meet your expectations.
A superior vendor will provide consulting/analysis services such that you can throttle the amount of resources you dedicate to the project. This type of vendor allows you to outline the business goals and objectives, and they will translate that into a comprehensive list of requirements, and later the final solution. Since the vendor is relaying back to you their understanding of the total solution, you can be certain the product will meet your expectations.
If there are no requirements based on legacy systems or integration, there are really only two languages to consider for your project: C# and Java. The decision will be an easy one since they are operating system (OS) specific. C# is the clear choice for Microsoft Windows environments and Java for Unix/Linux.
C# and Java are pure Object Oriented Programming (OOP) languages.
OOP has many advantages, just a few of them are:
Code reuse - shortens development time and impact of changes
Extendable - easily modify existing code to do new things
Concise - clean, clear code reduces bugs
Software bugs are a reality. How an error is handled is what differentiates one vendor from another.
Look for a vendor that proactively handles errors. The system should be designed to store error results and notify the programmers automatically. This way the programmers are already solving the problem before any users may have noticed it. A vendor with a history of automated error tracking will provide a solution that works better day one, and will continue to stay operational.
Testing, or Quality Assurance (QA), is a critical component of any successful software project. Ultimately, the quality of testing determines the number of issues you will have day one. Testing is also a very specialized skill. You want a vendor that is familiar with all the levels of testing: Unit Testing, Integration Testing, White Box Testing, Black Box Testing, Code Coverage Testing, Scalability Testing, and Load Testing.
You also want a vendor that uses automated testing. There are two types of
1) Test Driven Development
Here the programmers write the software code in a way so it can be automatically tested at compile time. This allows the programmer to make changes to the software, compile, and discover if his changes have created a problem elsewhere. This is critical to reducing the cost of changes.
2) Automated Testing Tools
Software exists to test software. This is typically used to load test an application by hammering it with virtual users.
Stay away from a vendor that simply assigns some programmers to "hack away" at the application before its release. Programmers typically hate testing and do a rather pathetic job at it, especially when testing their own code.
You don't tell the quality of a master by the size of his crowds.