First, a question…

When you find a piece of software to fill a specific need on a project, do you:

1. Add it in straight away while proclaiming “Open source is awesome, on to the next feature everybody!”
2. Proceed cautiously, taking a good look at the project’s health before adding it in

If you picked option 1, then congratulations; this blog post is firmly for you.  If you picked option 2 then you’re already ahead of the game, but hopefully this will give you some thoughts about how you assess project health.

A lot of the agencies (and freelancers) that I’ve worked for, or with, have got some way of assessing project health when considering software to add to their project. For larger agencies that might be formal, written-down rules. In smaller agencies its often unwritten “best-practice”. For freelancers, it’s commonly just a mental checklist / gut-feel that gets worked through.

As I’ve moved into working more heavily with new frameworks over the past 6 months, I’ve had to do my fair share of searching for new (to-me) software to plug missing holes. I’ve posted about a lot of the solutions I’ve found in my Stuff I’ve Used series.

There were a couple that I had on my shortlist to cover, but I’ve since decided need to be covered differently. The main reason is that while they solve my problem today, I’m not confident with the health of the projects to view them as long term / maintainable solutions. So – I’ll be talking about them in a blog post soon, but I wanted to write down my thoughts on project health first to give that post some context.

So, how do I assess project health?

I work with different technologies, and the exact measures vary depending on the ecosystem I’m working in. For example a module on drupal.org, a free WordPress plugin, or a repo or package on GitHub have different indicators which you can use. The general principles are the same though. I look at each of the following areas before making my decision.

• Stability
• Active maintenance
• Release management
• Documentation
• Code quality

For each one you can just weigh things up to get to your gut feel. You can use a fancy scoring matrix if that floats your boat, or if you need to set rules / guidelines for a team. You might also have specific “red flags” in each area that would block you from using a project.

Here’s the things I look for.

Stability

What does it mean to be “stable”? In some eco-systems it might be that there is a release that’s marked as “stable”. Drupal.org for example lets module maintainers tag releases as stable / dev etc. Packagist packages generally follow semantic versioning allowing you to infer stability from the version number. However, as a concept it means that there have been a couple of releases, there is basic usage / installation documentation, and there has been some feedback in the form of issues / feature requests.

Some projects don’t even have a release (common with GitHub repos), or have only an initial release. This generally makes me uncomfortable, particularly because it makes it really hard to assess some of the other areas.

There’s nothing to say that new software is bad. However, new software is often subject to change, which can make extra work for you as integration / usage changes – or worse make you stick with an old version to avoid the pain.

I don’t think I’d ever red-flag a project based on “age” – after all newer projects haven’t always acquired “feature-bloat” either. However, it’s certainly easier to feel confident about an established project than a new one.

Active maintenance

This covers a few things, and as ever there are a few different ways to measure it. If the commit history is available, I’d look at how recent the latest commits are, and how sporadic they are. Of course – no recent commits isn’t always a bad thing – stable software doesn’t necessarily need changing regularly. I’d also look at issues raised against the project (if that’s available), to see if issues are responded to, and/or worked on. For GitHub repos, I’d look at outstanding pull-requests to see if they are merged and/or responded to.

If there are issues / pull-requests backing up, and no sign of those being worked on / merged it would definitely count against the project.

Release management

This looks at whether there are releases firstly – I’d hesitate to depend on a project from which you alway just had to go with an arbitrary commit as the version you’re using.

Where there are releases, it’s important to check that they are “usable”. This means a few different things:

• that the version numbering is sensible (semver preferably)
• that each release has clear, plain-english release notes – a list of commits since the last release does not count
• releases are made with a relevant frequency, not too often, not too rarely
• that releases aren’t regularly superceded by bugfix releases to the release

Documentation

A package that you’re going to pick up and use in your project should have some level of documentation.

Important note to developers: If your software has no releases, and no documentation you don’t have an “open-source” project, you’re just hosting your version control in public.

I’m not going to suggest that you only use software that has pages, and pages of documentation. At the bare minimum though I’d expect a project to have a short document that explains:

• what the software should do
• what the software isn’t expected to do
• how it should used
• important dependencies
• basic ‘getting started’ instructions

Code quality

As a developer, I’m fortunate enough that I can generally look at the code and see how well I think it’s written.

I generally check for basic security precautions being taken (proper escaping of SQL to avoid injection, use of tokens to prevent CSRF, and output escaping to avoid XSS). I also check that the software fits the general approaches taken by the eco-system in which it fits. I’ve found that adherence to eco-system best practice is usually a good barometer for whether its been developed by someone who has taken care with their code.

If you’re non technical, or perhaps don’t have experience in the technology you’re pulling in, then if you know someone who is skilled – it’s worth asking them to give it a quick review.

Summary

Hopefully this has given you a feel for some of the things I look at when deciding whether to use a new module / plugin / package. If you’re not doing any evaluation right now – I’d encourage you to start. It saves time in the long run.

Photo credits:

• https://www.flickr.com/photos/dfb_photos/6258167290
• https://en.wikipedia.org/wiki/Land_sailing#/media/File:Sail_wagon_edit1.jpg
• https://www.flickr.com/photos/stone-soup/341495566