Essential tools for running a community-led project

by Ross Gardler on 22 February 2011 , last updated

Introduction

There is no single correct way to build and manage an open community-led project, but there are common tools and practices that we can use to support our work. The key is to use tools that are appropriate to your community and to keep them to a bare minimum. In this document, we examine the tools that are common to most open source projects and suggest ways in which you might use them to build community.

Tools should facilitate, not dictate

Many people think that being open in a project is about spending an inordinate amount of time informing people of current activities. But this need not be the case. If you are careful about selecting your tools and the development process that uses them, the appropriate level of external communication will simply be a by-product of the development process.

To understand how this can be true, we first need to understand that the goal of being open is not to ensure that everyone understands what it going on at any given point in development. Instead, it is about enabling those with sufficient interest to find the information they need. By focusing only on those interested enough to do their homework, we reduce the need for existing members of the community to hand-hold newcomers. This allows them to focus more completely on getting the results they seek. This may appear to be counter to the goal of building community, so let’s pick it apart a little.

In a community-led project, people participate in order to satisfy their own needs - you might call it ‘scratching an itch’. So an individual’s main driver will not be to build a community, but to solve a specific problem they face in their job, studies or hobby. For this reason, projects should be run in such a way that process and community overhead do not get in the way of people pursuing their interests.

That being said, a well-run open source project will have a set of lightweight processes and tools that provide a sensible compromise between keeping people informed and allowing them to get on with ‘scratching their own itch’. We will look at these processes and tools in the next sections.

The basics

Many people new to open source make the mistake of thinking they need to provide the widest possible set of tools to allow interested parties to engage in a way that suits them. However, this is usually a mistake. A tool should be introduced only when there is a clear need for it. Introducing a tool too early will confuse users (‘Which tool do I use to do X?’) and potentially split activity across multiple tools, thus creating an impression of reduced activity.

For an open source project, there are four essential tools and a whole raft of non-essential ones. The core tools that all projects should start with (and limit themselves to) are:

  • website: for communicating the intention and status of the project at any given point in time
  • developer mailing list: for the ongoing exchange of ideas, design and information
  • version control: for code management and communicating code changes for review
  • issue tracker: for planning and communicating work schedule and current activities

The core process for development, using these tools, will typically look something like this:

  • record an issue (bug, new feature, enhancement, etc.) to be addressed in the issue tracker
  • assign the issue to a developer
  • [OPTIONAL] if the work is complex or will introduce significant changes, post intentions to the developer mailing list (often this is an automated mail generated when the issue tracker is updated)
  • get on with the work (engaging with the broader development community through mailing list as necessary)
  • [IF COMMITTER] commit early and often to version control; this will send an automated message to the developer mailing list and allow the code to be reviewed and if necessary changes requested
  • [IF NOT COMMITTER] submit patches or pull requests to the issue tracker frequently; this will send an automated message to the developer mailing list, allowing the patch to be reviewed and, if necessary, changes requested
  • repeat until either the job is complete and all review requirements have been satisfied
  • mark the issue as complete (which sends a notification to the list)
  • update the website when appropriate

An important thing to note about this process is that it takes very little effort on the part of the developer to keep the community informed. The developer is not doing anything that they would not normally be doing in a well-run software project. Most community updates are created automatically by the tools, not by a human. So achieving open communication does not require any additional effort beyond setting up the tools correctly in the first place.

Now that we understand the basic tools and their role in the core development process, we’ll look at each of the tools in more detail. We’ll also consider some of the advantages and disadvantages of other tools your project may adopt as it matures.

Tools for communication

Communication is the key to the success of an open source project. It takes two forms: dissemination and discussion. In the former, the goal is to tell people something; in the latter, it is to have people share ideas and experiences. In this section, we will consider both forms of communication.

When designing a communication strategy, we need to remember that users and contributors may not be in the same place at the same time very often, or even at all. It is therefore vital to have a communication infrastructure that allows people to work on the project when they want and how they want. This infrastructure is the keystone of the whole project. Get this wrong and the participants in your project will not be able to collaborate.

A mailing list is the only required mechanism for two way communication in an open source project, but there are many other mechanisms that projects are tempted to use. In this section, we’ll look at why a mailing list is essential and why everything else is a potential diversion from the core activity of the project.

Mailing lists

Mailing lists are used for discussion and consensus-building, information-sharing and public recognition. They document the project’s development path and provide a written record of instructions and support for those new to the project. Mailing lists are probably the most important tool in the entire community-development armoury.

Mailing lists become embedded in the working practices of most open source software developers. The email client is the first application they open and is the one they return to throughout the day. Powerful features allow mail to be efficiently filtered and managed, and it can be accessed from any device, both online and offline. In short, email is the centre of an open source developer’s world.

Some people argue that the ‘pull’ mechanism of web-based forums - that is, requiring the user to visit an online forum in order to read the latest posts - makes web-based forums superior to email. They claim that not everyone can handle a large volume of email, and most people don’t want unimportant emails arriving in their inbox. Of course, this argument is perfectly valid. However, the key is in the word ‘most’. When setting up a new project, we should consider who we are trying to appeal to. A new project needs new contributors - it does not need, and probably cannot service, new users. Since we want developers, we are likely to be looking at technical people - people who are probably already have their email client at the centre of their working day. For these people, email is the more efficient and effective tool.

This is not to say that online forums have no place in an open source project. In the next section, we will discuss where they can be very useful. However, the point remains that in most cases, a mailing list is more appropriate for a developer community. Furthermore, as we will see in later sections, a mailing list is better able to integrate with the other essential tools a project needs.

We have argued that a mailing list is more appropriate than a forum; however, many of the newer mailing list tools provide both web-based and email client interfaces, so that users can work with whichever mechanism is most appropriate to their needs. If you are in doubt about the utility of a mailing list over a forum for your community, we strongly recommend one of these hybrid tools.

Mail archives

Provided that mailing lists are used appropriately, the mail archives of an openly developed project are one of its most valuable resources. They contain a record of design discussions and decisions, how-to information, use cases, requirements analysis and much more.

All projects should provide searchable archives of their mailing lists and, wherever possible, refer back to them rather than duplicating effort by answering a question for the second time or unnecessarily revisiting a previous decision. As with syndication feeds, the provision of archives should not add to the workload of the project team. Most mailing list providers will also have an archive solution. If this is not the case, there are a number of online facilities that can be used free of charge to archive public mailing lists.

Forums

Forums have the advantage of being easy to use. You do not need to be a power user of email to get the most from a forum. One advantage that forums have over mailing lists is providing a searchable archive in the same place as the discussion interface. However, forums do not have the advanced features, such as offline access, filtering and interoperability, that email provides.

While mailing lists are the best communication device for technical people, it is often argued that users will be less technical and therefore need a different communication mechanism. This is not always true; it will depend on the nature of the project. However, if your users are less technical, a forum for user support may be more appropriate.

Even though a forum may be more appropriate for users, you need to think carefully before creating one (or a separate mailing list for users). Is your project mature enough to attract users in significant numbers? Do you have enough developers to monitor and respond to user queries on the forum?

Creating a forum, or a second mailing list for users, too early will result in forcing users into a separate, ‘darkened room’, where they will be unsure if there is anyone ‘listening’. A second communication channel should only be created when the traffic on your first channel is sufficiently high to justify the split.

Q&A

Recent years have seen widespread adoption of the user-curated Question and Answer site pioneered by StackOverflow. Such sites (usually called StackExchanges, after the most popular software used to host them) are well suited for users of all levels to ask and answer questions. Answers are voted up or down by the community, meaning the resulting “Accepted” answer is displayed directly with the question, while less useful answers will be hidden. Site a site can be a useful support channel to avoid technical discussion channels being filled with duplicated questions. The presentation of questions means that users searching for support will find the answer directly, rather than having to trawl through pages of forum threads.

Blogs

Blogs are perfect for disseminating ideas or news, but are not ideal for active discussion. They typically have a comment facility, but do not usually provide complete facilities for supporting conversation. Like forums, they require the user to visit the site in order to participate. Consequently, not everyone will see follow-up comments. Blogs should not, therefore, be considered as a tool for facilitating discussion. Instead, they can be a tool for announcements or detailed explanations of issues relating to the project. Bogging or podcasting can also help build community relations by allowing participants to show their personalities outside of the “work” environment of development and support channels.

Syndication

Syndication is a means of sharing information in a machine-readable form so that third parties can conveniently include relevant content from your project. Widely used syndication formats such RSS and ATOM feeds are a convenient way for people to pull content from your project and aggregate it in a way that is meaningful to the way they work, or to their community. That is, a syndication feed can appear in many places, such as a user’s mail client, a feed reader, a web-based aggregator or a website. Providing syndicated access to as many of your communications channels as possible increases your chances of reaching new users.

However, syndicated content is not, in itself, a tool for communication. It is merely a tool for providing access to pre-existing content. By selecting the appropriate core tools carefully, you should be able to ensure that syndicated feeds are provided automatically. That is, you should try to choose a tool for your website, communications, issue tracking and version control that automatically provides syndication feeds of appropriate data.

Social networks and micro-blogging

Social networking is often seen as a critical part of community-building. However, the value of these tools in open source projects is not clear. The main problem is that most social networks act as a walled garden, so choosing one network over another forces your potential contributors to use that specific site, which may not be their preferred site.

It may be tempting to syndicate your content into social networking sites. This might be appropriate for some projects, but be careful about splitting your community between official channels and social networking channels.

Website

A website is a mechanism for publishing various forms of information. It is used to provide entry-level information on a project and a gateway to all other parts of the project. A website for an open source project will be just like any other website. Many code hosting platforms such as SourceForge or GitHub will provide hosting for a simple website, but you may wish to host your own. Using a Content Management System makes it easy to keep your website updated and allow other members of the projects to administer the site.

Screencasts

Screencasts are videos that demonstrate software in use. Users can view them without needing to install the software being viewed. They can be very useful in attracting new users and providing part of the documentation for the system. However, they can quickly become stale, as a product’s interface matures and new features are added. The overhead required to maintain them may also make them impractical for extensive use.

For applications that are accessed through a web browser, one technique that can work well is to use automated browser-testing to ensure a bug-free user experience. These automated tests can be run at a slower speed and recorded as a screencast in preparation for each release. Since each release needs to be fully tested and the tests need to be kept up to date with respect to interface and feature updates, this provides a means of semi-automating screencast recording.

Version control system

It is very important to provide a version control system (VCS; also known as revision control system or RCS) for the management of your project resources. This allows you to track who made what changes when, why they were made and, if necessary, to roll back changes that cause problems. Using a VCS is easy; using one correctly is harder. If you’ve never used a VCS before, we recommend that you read our document on version control.

There are 2 styles of VCS - centralised and distributed. Think about which style best suits your project governance and workflows. Your choice of VCS may also affect the code hosting sites available to you, as sites tend to support only a selection of VCSs.

When used correctly, a VCS ensures that developers remain informed, release management is easier, bug-work is recorded, experimentation is facilitated, contributions are managed and write access to resources is controlled. Your VCS is a coordination and management tool, as well as your project’s time machine. It will ensure that everyone is fully informed about all changes to resources and allow anyone to retrieve all resources from any given moment in the life of the project.

Using a VCS effectively requires that all contributors follow the ‘commit early and often’ maxim. For developers following test-drive development, this is easy to do: simply commit when each new test is passed. This sends out a notification that an implementation step has been completed and the issue tracker automatically records the commit against the issue. Other developers can now review the changes and also avoid duplicating effort.

Issue tracking/project management

An issue tracker allows your users to report bugs and request new features. It also allows your project management team to plan work and prioritise community-driven requests.

By using an issue tracker, you can provide a clear indication to your users as what to expect in future releases of your code. This means that if a user has a specific need for a feature or bug-fix that you have no intention of implementing in the short term, they can opt to invest some of their own resources in adding the feature or fixing the bug. Communicating your plans clearly and efficiently to users is the first step towards having them contribute to the development of your project.

Perhaps one of the best uses of an issue tracker is to provide a record of areas that are suitable for newcomers to the project to tackle in order to get a feel for the project. Issues that are simple, or have a mentor available can be flagged as such and community members can draw attention to them when newcomers are looking for somewhere to get started.

Management of intellectual property rights (IPR)

It is vital that you know who owns the copyrights in your project. To do this, you need to be able to trace every contribution back to its original author. In the case of contributions that directly affect your project outputs, such as programme code or documentation, this is most efficiently managed through a version control system (see above) coupled with a clearly defined Contributor Licence Agreement and a submission process using an issue tracker.

Where contributions are not directly in the form of source code or documentation, for example design ideas, it is important that they are captured in a publicly accessible and traceable form. If you conduct your design discussions on a mailing list, the archives of that mailing list are perfect for this.

Where the contribution takes the form of software code, the version control system and issue tracker provide a means for recording the contribution process. A patch is submitted to the issue tracker, where a developer will pick it up and review it. Once the patch is ready to be applied to the code base, the version control system is used and the ‘commit’ is recorded automatically against the issue tracker.

The management of IPR in an open source project is, in many ways, the most important and most onerous task. It does not directly add to the functionality of the software and appears at first glance to be time-consuming and process-driven. However, if you have set up the project resources carefully, the process will be mostly automated and transparent for the majority of developers. Having such a process in place will instill confidence in potential users of your software, and more users means more contributors.

Conclusion

Having the right tools for your project will make it easier to manage the project itself, regardless of whether you attract a community or not. However, just having the tools is only the start, you also need to use them in a consistent and predictable way. If you use the right tools from the outset you will avoid a great deal of pain and create the best possible conditions for building a community around your project when the time is right.

Further reading

Links:

Related information from OSS Watch: