Greg Laden is a blogger, writer and independent scholar who occassionally teaches. He has a PhD from Harvard in Archaeology and Biological Anthropology, as well as a Masters Degree in the same subjects. He is a biological anthropologist, but for many years before going to graduate school to study human evolution, he did archaeology in North America. He thinks of himself as a biologist who focuses on humans (past and present) and who uses archaeology as one of the tools of the trade. Greg blogs regularly on ScienceBlogs at http://www.scienceblogs.com/gregladen/.
Dr. Joseph J. Salvo attended Phillips Andover Academy, received his A.B. degree from Harvard University and his Master and Ph.D. degrees in Molecular Biophysics and Biochemistry from Yale University. Dr. Salvo joined the GE Global Research Center in 1988. His early work focused on the development of genetically modified bacteria and fungus, for the production of novel high performance polymers. In the mid 1990's he turned his group's efforts towards developing large-scale internet-based sensing arrays to manage and oversee
business systems. Most recently, he and his team have developed a number of complex decision engines that deliver customer value through system transparency and knowledge-based computational algorithms. Commercial business implementations of his work are currently active in Europe, and Asia as well as North and South America.
Dr. Peter Tu received his undergraduate degree in Systems Design Engineering from the University of Waterloo, Canada, and his doctorate from Oxford University England.
In 1997, he joined the Visualization and Computer Vision Group at the GE Global Research Center in Niskayuna, NY. He has developed algorithms for the FBI Automatic Fingerprint Identification System. He is the principle investigator for the ReFace program, which has the goal of automatically computing the appearance of a person’s face from skeletal remains. Dr. Tu has also developed a number of algorithms for the precise measurement of specular and high curvature objects. His current focus is the development of intelligent video algorithms for surveillance applications.
The much anticipated Part Three of Jim Hall's exploration of OpenSource ...
Let's review:
In the first part of this article, I discussed the differences between Free Software and Open Source Software. In a Venn Diagram, these are concentric rings. The definition of free software lives within the definition of open source software, which itself sits within the space of all kinds of software, so free software is really a subset of open source software. For the rest of this post, I'll use the term "open source software" generically.
Later, I reviewed Open Source Software in the real world, describing the features that an open source software project must have in order to get itself off the ground.
But once an open source software project has gotten started, how does it continue to thrive? How do you build and maintain a community of user-developers who will be interested in adding new features?
Once again, I'd like to use FreeDOS as a case study, since I consider it a very real example of a successful open source software community.
It starts with a web site
When I created the FreeDOS Project in 1994, the Web didn't really exist. It was not until 1995-1996 that the Mosaic browser became readily available, and several years would pass before most users had access to the Internet from their home. So in the beginning, we used an ftp site (hosted by our friends at SunSITE) to distribute all our software and source code. We conversed with one another, and announced new software releases, via USENET news groups.
Compared to today, that's a very limited way to organize a group of developers. In today's world, any open source software project that wants to thrive must provide - at least - a web site for its users. And more importantly, that web site should be well organized.
The project's web site should provide a continuous stream of news items: where is the project going, new developments, major bugs found and fixed, and mention of the project by outside groups. A great way to encourage developers to contribute is to recognize individuals. To an extent, we all like to see our name in print, even in a transient medium like the Web. Spotlight those who make major contributions - but strike a careful balance. One sure way to kill new interest is to present a culture of "us vs. them".
One common mistake is forgetting that an open source software project is for everyone! Too often, new open source software projects dedicate a page to "members" or "developers". As a result, anyone just joining the project gets the impression that the group is closed, too insular. Remain inclusive; it's not a members-only club!
Your web site must make it easy for new users to find a way to contribute. Not everyone who decides to help will be a developer who can write code. Rather, some users will only be able to provide debugging information on odd hardware configurations. Others may have expertise in user interface design, while others may be skilled technical writers. Find a niche for everyone, and advertise on your web site how new contributors can get involved.
Source code
Being open source software means that you need to provide your source code for inspection. This is also the only way for new user-developers to fix bugs and add new features, if they are so interested. So after setting up a web site, your next task should be to open your source code.
How you share the source code may depend on the size of your project. If your program is relatively small, you may only need provide a zip or tar file containing the source code. For example, when I wrote FreeDOS Cats/Kitten (a library that helps a program support multiple languages) I just shared a zip file with all my source code. Cats/Kitten is very small, about 6KB.
If your project is much larger, you may want to look into a source code repository, such as CVS or Subversion, so that users can download just the components that interest them. Today, this is how most open source software projects provide their source code. Look at any project's web site, and you should be able to quickly find a link to their source code repository.
But for your open source software to be successful, you need to provide the full source code every time you make a release. This expectation is built into the GNU General Public License, but it is also a key to building a successful community. Making the source code available to your users allows for the cooperative development and rapid code improvement that fosters "mind share". Not provide the source code - such as making a "testing-only release" or a "preview version" - means your users will not be able to see how your code works. More importantly, your users will not be able to help you fix bugs. Without a way to contribute, developers tend to lose interest in a project, and find something else to do.
Documentation
Most users often experience a new open source software project through its documentation. Before going through the trouble of downloading new code and testing it out, people want to get acquainted with how the software is supposed to work, and decide if it is right for them. So it is not surprising that many successful open source software projects often put a large effort into making the documentation a first-rate experience.
Writing good documentation is hard, though. Developers who are very technically minded often are not able to write instruction manuals that are easy to understand. So how do projects produce useful and meaningful documentation?
This is such an important step for many projects that some projects split off a separate, related project just to update the documentation. For example, Linux enthusiast Matt Welsh co-founded an effort to provide documentation for fellow Linux users, by writing recipe guides called HOWTOs. Today, Matt's project lives on as the Linux Documentation Project ("LDP") and their HOWTOs receive the same level of praise as any O'Reilly book. In fact, O'Reilly has published several LDP HOWTOs as books.
Recognizing the success of the LDP had in helping users understand Linux, I created a similar documentation effort for FreeDOS. In founding the Linux Documentation Project ("FD-DOC") we followed the spirit of Matt's intent with the LDP. It would be well for any similar documentation effort to follow the same goals:
- The documentation project is primarily a vehicle for enthusiasts and developers to share their knowledge about the system with other users and co-developers. People are motivated to contribute to the documentation effort because they know it helps many people.
- The documentation project, if you have one, should be the de facto place for people to visit to find out about your open source software project.
- As such, it is vital that it is as easy as possible for new authors to contribute to the documentation effort. If you institute complex document standards, create voting organizations, or require participation in mailing lists, you will discourage many potential new contributors from helping you.
- Likewise, the tools to create documentation should be easy to use, widely-available, free, and well-supported. When we started the FD-DOC, we adopted roff (-ms) to typeset our documents. We found it was widely-accepted, and many people already knew how to write with it. Those who were unfamiliar with roff typically had a short learning curve. However, times have changed. I recommend modern open source software projects create a wiki where users can become editors.
It is important to remember that the documentation project should be open for contribution by anyone, and is not a closed, private club. Otherwise, the documentation project will not be interesting to any but the few people who created it in the first place.
Bug tracking
Often a major sticking point for newcomers to open source software is the issue of tracking bugs. To empower your users to be co-developers or user-developers, you must provide them the tools they need to be efficient contributors. A successful open source software project must equip its developers with a bug tracking system. After all, a software project cannot improve itself if it cannot identify what it wrong in the software.
When I think about bug tracking, I am reminded of this quote:
The highest sounds are the hardest to hear. Going forward is a way to retreat. Greater talent shows itself late in life. Even a perfect program still has bugs.-- Geoffrey James, "The Tao of Programming"
There are a number of different bug tracking systems that you can take advantage of. Bugzilla seems to be the most popular these days, but I have also used GNATS and Jitterbug with some success. These bug tracking tools all share a common set of features that you should look for in your open source software project:
- The bug tracking system must be easy to use: entering new bugs, and following existing bugs. If your system is too difficult to use, no one will want to enter new bugs, and problems in your software will remain unfixed.
- The bug tracking system must have a strong query capability. If your users and developers cannot look up existing bugs, you'll find your project will get flooded with lots of duplicate bugs.
- The system must be open to all eyes. This is perhaps the most important - and yet most difficult - feature for traditional software managers to understand. In the cathedral model of software development, bug lists are often hidden from customers. After all, there is no beta released before its time, so why would you advertise features that are broken until you have had time to fix them? However, in the bazaar model of open source software, everyone who uses the software is potentially a new developer for it. In order to foster new fixes and patches to the system, developers must be able to see what work needs to be done, and what features need to be addressed.
Respond to submissions
Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging. Because the source code is available, anyone can contribute patches and new features. Given a bit of encouragement, your users will diagnose problems for you, suggest fixes, and even send you source code that helps to improve your project far more quickly than you could on your own.
However, the only way to keep your users working with you as co-developers is to respond to the patches and suggestions they send you. Your best bet to keeping your community engaged and stimulated is the sight of constant improvement in the system they are using.
In the FreeDOS Project, I wrote a lot of utilities and other programs. My most active development efforts were aimed at our first Install program, and later the Cats/Kitten library. Looking at the Install program, early in the development cycle, a very talented developer named Jeremy Davis found an interest in how the Install program worked, found some bugs, and submitted a few fixes to me. Eager to have help in my coding, I responded immediately by reviewing his patches and checking them into the source code tree, and releasing a new version. Releases of the Install program became more frequent as bugs were quickly identified and patches.
Similarly, when I first wrote the Cats library, it was a small effort to recreate the Unix catgets functionality. But since many people around the world contribute to FreeDOS, my Cats library became popular very quickly. I received patches from different developers, adding features, improving Unix compatibility, and reducing the memory footprint. Releases happened very quickly, sometimes several versions in a single day if we had enough of us collaborating. Eventually, we had introduced so many new features yet in a smaller size, I renamed the library to Cats/Kitten.
None of this would have been possible had I stopped responding to the users who were acting as my co-developers. The key was constant feedback.
The worst thing that can happen to an open source software project is to become unresponsive to its users. A sure way for a project maintainer to kill interest is to reply to developers with "Thanks for the patch, but I was planning to add that anyway in the next release."
It's not pixie dust
Finally, let me remind anyone interested in starting an open source software project that open source software does not imply automatic success. Just because you release your project as "open source software" does not mean you will develop a community to help you, even if you do everything "right". No project, either open source or proprietary, can guarantee that it will survive the test of time.
Jamie Zawinski, a well-known open source software developer, wrote of his time with Netscape as it became "open sourced" as Mozilla:
Open source does work, but it is most definitely not a panacea. If there's a cautionary tale here, it is that you can't take a dying project, sprinkle it with the magic pixie dust of "open source," and have everything magically work out. Software is hard. The issues aren't that simple.
In other words, open source is not a magic bullet. A project must be interesting to other developers in order for them to contribute to it. Realize that your software just may not be that "cool" to many other people.
An example is a project someone once wrote for FreeDOS. The FreeDOS Internet Services Host ("FISH") was written in 1999 by Gregory Ball, as a web server that ran on a PC running FreeDOS. Really, the program was a demonstrate that network applications could be written for FreeDOS using WATTCP, the standard library for TCP/IP in DOS. Gregory deployed FISH on an 80386 SX/25 computer with 4MB of memory, but his web site advertised it would run on a lesser system.
This was a really cool project, and I admired it for its unadulterated "hack value". However, no one really believed that people would run serious web servers on a DOS-based system. Apache was a much more common system that supported more features. So FISH sort of fell by the wayside, and failed to generate enough interest to maintain a community. Eventually, the last FISH server was turned off, and no one noticed. Being "cool" just wasn't enough if people weren't interested in using it.
In the fourth and final installment of these guest posts, Jim will talk about the role of project maintainer during a period of transition.
Technology
Comments
I'd be interested to hear your thoughts on the SourceForge community. It seems to me that moving one's project there, at least as a first move towards Open Sourcing, automatically gets the project the support system of svn/git, bug tracking, discussion groups, etc, so the project authors do not have to spend time figuring out how to do that by themselves.
Posted by: Gray Gaffer | October 23, 2009 2:46 PM
I think SourceForge is a great idea! In fact, the FreeDOS Project is already hosted at SourceForge.
SourceForge provides a lot of the common toolsets that really help a growing open source software project. In my post, I talked about web sites, source code, documentation, bug tracking as being part of the success of a project. SourceForge provides these things to projects, no charge. I find that really amazing, and a great resource to draw on.
One additional benefit of using SF is "outsourcing". I'd much rather use the MediaWiki instance provided by SourceForge to manage the FreeDOS docs, rather than have to install and maintain my own wiki. (We used to do this, in fact, and it was a nightmare.) Similarly, SourceForge gives users a web view into your source code, and provides a workable bug tracking system.
As an example: when we first started the FreeDOS Project, we didn't have much of a bug tracking system. We soon put up a Jitterbug instance, which is an email-based bug tracker. When we changed web hosting companies, we switched to a web-based Jitterbug-compatible system called FDBugTrack. But this was fairly limited, so we eventually moved over to Bugzilla.
Bugzilla was much better, because it had a very rich query/reporting system, and it was considerably easier to use than Jitterbug or BugTrack. The down-side was installing all the patches. I think we upgraded the FreeDOS Bugzilla twice before we stopped for lack of time.
We realized we just couldn't continue upgrading Bugzilla. It took away too much free time that could be better spent writing code for FreeDOS. Our main interest was in creating a free version of DOS, not software management for a web app.
So we moved our open bugs to the SourceForge bug tracker, and created an on-line static archive of our Bugzilla database.
Posted by: Jim Hall | October 23, 2009 4:36 PM
I have an idea for an opensource project, but I don't plan on having any bugs.
Posted by: Greg Laden | October 25, 2009 8:48 PM
Hello_Greg.c?
Posted by: Gray Gaffer | October 26, 2009 2:57 PM
Hello_Greg.c will be bug-free until the day you realize it only supports English text, so users named "Greg" who speak a different native language will not be able to enjoy the program. First bug! The solution is to add internationalization support. Under FreeDOS, this can be done using my Cats/Kitten library.
Posted by: Jim Hall | October 27, 2009 1:03 PM
Amen, brother microwave! When you put in the last thing you forgot, you realize you forgot something else. And getting internationalization to work generically is pretty hard. Take it from experience.
Gregory Pietsch, author of FreeDOS Edlin
Posted by: Gregory Pietsch | November 21, 2009 7:04 PM