You've gotten used to evaluating software for use in your organization. You have a specific need to fill, you look around for the list of software that can fill that need, make sure that the feature set matches, that you have the budget, and that the company or vendor is reputable, and can provide the support you need. But how do you evaluate free and open source software (FOSS)?

It takes some different approaches to evaluate FOSS projects. Here are the issues to consider:

Feature set and your requirements: No matter whether the software is FOSS or proprietary, you want to make sure that the software fits your requirements. It might not be a perfect fit, and often, when you are evaluating similar products for the same problem, you need to balance the overlapping feature sets, and issues such as price and support. Remember, just because a FOSS application has no cost to acquire doesn't mean it won't take time and/or resources to implement.

Quality of the development: In most cases, for proprietary applications, you can’t evaluate the code, since you don’t have access to it. With FOSS tools, that option is open to you. Of course, you might not have the knowledge to evaluate the code, but the fact that it is open means that others can. You can know how many developers have worked on the application, what languages it is written in, etc.

A really good tool to help you with this is ohloh.net. It has details about code, the contributors, and other details that can help you evaluate the quality of an application. It also has reviews. You can also find reviews of FOSS projects in many places; a Google search on a particular application is a good place to start.

Another thing to look at is how often an application is updated. Look at the site for the date of the most recent major and minor upgrade. (Minor upgrades are generally upgrades from 1.2.1 to 1.2.2, for instance. Major upgrades would be 1.2 to 1.3.) How often do minor upgrades happen? For active projects, several minor upgrades a year is the norm. No minor upgrade in a year is a red flag, although this is not universal. Some very active projects upgrade more slowly than others. Information from the community will help evaluate that.

Support options: In general, it is always extremely important to look closely at the support options available for any software application. FOSS applications are a little different in this regard. There may actually be a company behind the product where you can get support (generally at a cost). There may also be vendors that will support the application. There are increasing numbers of vendors that support FOSS applications. You can find a list on http://nosi.net.

Often, the richest form of support for an application is the community of users and developers around it. How do you evaluate the support the community can provide? Most projects have forums and/or mailing lists. Visit them and lurk. How many posts are there? (For lists, look at the archives.) How easy does it seem for people to get their questions answered? Many projects also have IRC (Internet Relay Chat) channels. It’s a good thing to visit those as well. How many users are in the channel? Do people get their questions answered?

Flexibility: Unlike proprietary applications, where you are, generally, stuck with what you are given, FOSS applications allow for a lot more flexibility, and the possibility to make an application fit your needs more exactly. But this will always carry a cost.

In general, for FOSS applications, it is actually easier to evaluate the quality of the application and the support options available than it is for proprietary applications. But, at the same time, there are more issues to consider, such as whether to take advantage of the flexibility of an application.

Just remember: the requirements you have for the problem you want to solve should always be in your mind as you evaluate the many aspects of the software under consideration.