TLDP is short for The Linux Documentation Project, an organization of volunteers authoring, reviewing and managing documents about the Linux operating system. Documents basically come in two formats based upon their length. The shorter ones generally are called HOWTOs (or mini-HOWTOs, if they are really short), the longer documents, called guides, deal in-depth with a Linux feature.

The number of topics discussed in these HOWTOs and guides is practically unlimited, ranging from installing the Linux system to managing all kinds of devices, services and environments, to creating your own system from scratch: there's something about it in TLDP, mainly thanks to volunteers who share their experiences. Nevertheless, it must be admitted that there are still some important topics missing from the collection.

All the documentation is freely available in several formats suitable for printing and on-line browsing. The main submission language is English, but several translation efforts, including French, German, Italian, Korean and Chinese, try to make this immense amount of information available to a wider public.

This is only an article, not a book. Some people who have done fantastic work are not mentioned here. I hope these people understand the context of this article and know that I am thinking about them, eventhough they are not all listed by name.

Linux environments tend to change at a rather high speed, so do the docs. Sooner rather than later, submissions about new protocols and applications reach TLDP, outdating older documents. The main problem here is TLDP maintainers usually are rather soft-hearted, so partly out of melancholy, partly out of respect and sometimes partly because of the lack of volunteers for upgrading a document, they tend to archive everything. Some also argue that it is better to have an outdated document than none at all.

Given this information, it might thus be best to stick to the following golden rules when searching the LDP collection:

Most documents contain revision history information in the preface or in an appendix.

As Matt Welsh, one of the co-founders, puts it: "The history of the LDP is a pretty murky memory these days." It started in 1992, before the World Wide Web existed. It's hard to imagine how we did without HTML, but in those days almost everything was FTP and Usenet and dial-in to a BBS was most likely. In the beginning, most of the documentation was in one big file, split into sections, called the Linux FAQ.

Later, Matt got together with Lars Wirzenius and Michael K. Johnson, who had the idea of producing printed Linux documentation. Michael initially started on a kernel hackers guide, Lars did the system administrator guide and Matt wrote the first installation guide. Everything was done in LaTeX, so the only way to read these docs in a reasonably comfortable way was either by printing them out or using a PostScript viewer.

But as Linux capabilities grew, it was no longer possible for one person to maintain everything. Pretty soon, not even several people could manage the job. Thus, the HOWTOs were born, each describing a part of the original big chunk of information. This created an easily extendible system that allows for many authors to contribute to their areas of specialization.

That effort lead to the use of (Linuxdoc) SGML, which enabled the fast generation of all sorts of output formats, including HTML, from one source file or set of files. The first tests were conducted at Sunsite (a famous server machine at the University of North Carolina), which was the first Web site offering information about Linux. Also, when you wanted to download Linux software, Sunsite.unc.edu was the place to go. It still contains some kernel archives--probably by accident, there also are a lot of empty directories these days.

Before the crash (May 2003) I was able to find, via FTP, a document referring to two maintainers of the LDP as it was run by the end of 1994 at UNC. It pointed to Jon Magid and a mysterious Erik with no last name, who was still at Sunsite in 1996.

After extended research in the dungeon server rooms of Google, we can state with almost certainty that the mysterious Erik does have a last name after all. Most likely, we are dealing here with the Erik Troan, who supported possibly half of the Linux users in the 1993-1996 period and later on became the Senior Director of Engineering at Red Hat.

Further research revealed that sometime in 1996, Greg Hankins became the HOWTO coordinator of the LDP project. He was the original author of the Serial HOWTO, which he began maintaining in 1993; he also was one of the main contributors to the SGML-tools development project.

LDP is becoming more popular by the day, and the collection was published on paper several times, in part or entirely. LSL (now CheapBytes) was the publisher of multiple editions. They were called "The Linux Bible", "Dr. Linux", "Linux Getting Started", "Linux the Complete Reference" and "The Linux Encyclopedia".

1998 saw the publication of "Linux Undercover", subtitled "Linux Secrets as Revealed by the Linux Documentation Project". Red Hat was the first to use the new just-in-time production method. Previous printed versions often contained stale HOWTOs, but this one essentially was printed straight from the on-line master documents.

The linuxdoc.org domain was registered and the entire Linux documentation collection moved to it and promptly was mirrored. The relationship with iBiblio (formerly sunsite.UNC.edu) was maintained during the romance with SGI, and the university became a mirror site. The love didn't last, however, and TLDP moved to iBiblio again after the short SGI intermezzo. Paul Jones and his colleagues, responsible for managing TLDP at iBilbio, were very understanding and provided a lot of support, which enabled the centralization of resources in North Carolina.

Prior to the mailinglists, discussion primarily happened in the Usenet newsgroups. In 1996 the [email protected] mailinglist was created. It moved to [email protected] around 1998. The mailing lists were a good thing; I remember that newsfeed in those days was generating enormous amounts of traffic and consumed--for that time--unreasonable amounts of bandwidth. Some ISPs decided to offer only a partial feed or none at all. The ldp-l mailinglist was stopped and replaced by the current lists (discuss, feedback, announce, submit etc.) in 1999.

Lars and his crew started working on an automated system for submission and publication of updates during this period of growth, but the system was never finished.

By 1999, Guylhem Aznar was elected coordinator of the LDP. His job was to unify the LDP: servers were in operation all over the world, there were no real backup mechanisms when people were ill or unavailable. He started by putting together a staff, a team of volunteers that could give structure to TLDP.

The exact configuration of the core team in those days has been preserved. It was composed of a hub, consisting of one main coordinator, plus individual FAQ, Guide and HOWTO coordinators, Greg Ferguson, Joshua Drake and Tim Bynum, respectively. Furthermore, most translation efforts started in 1994 now are running more or less at full speed, and people have been appointed to manage each translation. One project not listed here, although it was among the first, is the German translation effort. As with the recent joining of some Italian translators, it sometimes takes a while for people find one another.

In 1999, the project hosted eight guides, including version 1.0 of the Linux Network Administrator's Guide and beta-1 of the Linux User's Guide. These and other documents still were written mostly in SGML or LaTeX.

The first occurrences of DocBook were seen in 2000; DocBook now is the preferred submission format because it enables easy generation of HTML, PS, PDF and other formats from the source files.

Another novelty that came with the 21st century was the creation of a versioning system. Sergiusz Pawlowicz and Gregory Leblanc were responsible for the setup; Sergiusz still manages our CVS. He also became the listmaster by the end of 2000. Up until then, Debian hosted the TLDP mailing lists.

TLDP project was maturing and growing in every possible way. To this end, David S. Lawyer finalized the LDP Manifesto. David is still the point of contact for all license issues.

A new Web site layout was probably the most visible improvement. The new millennium brought the precursor of the site as it is today.

In some documents you still can see references to the old linuxdoc.org domain. The reason for the domain switch was not pretty, unfortunately, but as it is part of the TLDP story, it should be told.

As is so often the case, goodwill and kindness made selfishness rise to the occasion. Many people are interested in TLDP, but not always for the good of the project.

At the time Guylhem was elected president, there was only one other candidate. Guylhem didn't want that man to feel left out, so he trusted him to be the webmaster. TLDP lost the linuxdoc.org domain because the webmaster managed to claim ownership of it. He also purchased the .com domain and ran a commercial Web site on it on the back of TLDP. Needless to say, this caused a lot of friction.

There is another side to the story, saying that the webmaster (inadvertently or not?) failed to pay the renewal fee for the domain.

Anyway, a new domain had to be found, and tldp.org was short and free. Guylhem took his responsibilities seriously, registered the domain in 2002 and moved the project to the current domain. The team also took this opportunity to broaden the scope of the project, and they combined the move with a restructuring operation that made the project much more efficient.

But all these efforts were not enough. Ever more authors and other volunteers needed guidance, and ever more documents had to be organized. The project hosted a lot of outdated documents by now, which became a bit shameful. Another problem was the random publication of documents. There was so much work and not enough people to do it, so anybody could publish almost anything. Scandal broke loose when a couple of opinionated documents were found, containing tainted and sometimes plainly wrong information that was possibly harmful to the readers.

Thus, 2003 became the year of revamping. A thorough search through the entire collection revealed more old or doubtful documents that were taken off-line for a revision. Documents too old to be useful were moved to the attic. Tabatha Marshall was appointed review coordinator and put together a team of reviewers. Together, they edit new submissions: they check for technical correctness, readability and grammar and spelling errors. Furthermore, they apply the TLDP style so as to give the collection consistency. The Weekly News was revived and offered over RSS feed. Input from the feedback mailing list was followed up once more. The Author Guide was revised to list the new procedures for publishing documents in accordance with the quality control guidelines. A HOWTO generator was created to facilitate submissions by new authors. Beyond these visible accomplishments, hundreds of authors are working together now, everyone of them contributing a small part to this huge project.

People responsible for managing projects often ask us how we do it. This is how. There is no book that tells you how to do it. We are on a road with many bumps and ups and downs, and TLDP seemingly hangs together with hooks and eyes--but it's there and it doesn't go away.

More history

As the Weekly News project also has its ups and downs and is currently in a down, we decided to take the link off the front page and make the old issues available here.