Tools for successful documentation projects

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

At Open Source Summit Japan 2025, Erin McKean talked about the challenges to producing good project documentation, along with some tooling that can help guide the process toward success. It is a problem that many projects struggle with and one that her employer, Google, gained a lot of experience with from its now-concluded Season of Docs initiative. Through that program, more than 200 case studies of documentation projects were gathered that were mined for common problems and solutions, which led to the tools and techniques that McKean described.

She introduced herself as a developer-relations engineer in the Google open-source-programs office; part of her job—"and it's a fun job"—is to "help open-source projects have better docs". She was also an honorary fellow of the "late, lamented" Society for Technical Communication and runs the online, non-profit Wordnik English-language-dictionary web site. Beyond all of that, she runs the Semicolon Appreciation Society; some of us here at LWN should probably join said society.

She worked on the Season of Docs project, which was set up along the lines of the Summer of Code initiative as an opportunity for open-source projects to mentor documentation writers. Season of Docs was started in 2019 by Sarah Maddox and Andrew Chen; fairly quickly, those involved realized that the maintainers of open-source projects "were not able to mentor technical writers because they were software developers". So it switched to making grants to more than 130 open-source projects, which wrote over 200 case studies that can be found on the Season of Docs web site.

Nobody wants to read 200+ case studies all at once, she said, but there is a lot of valuable information in them. So those working on the initiative decided to use the studies as the basis for "some tools for doc maintainers". Given that those tools are the focus of the talk, she expected that those attending cared about open-source documentation and were looking for help to produce better documentation for their project. Beyond that, she believed that attendees were also empowered to make changes to the documentation process of their projects and were willing to accept help in doing so.

The "docs impulse" comes out regularly for those who care about documentation. It often happens due to noticing a problem of some kind: outdated documentation, something that has to be repeated frequently in pull-request or mailing-list comments, something that is obviously missing from the documentation, and so on. Those situations lead to the feeling that "I need docs", but that idea has lots of assumptions built in, such as what the software does, what the users expect, and what project members are capable of with regards to documentation.

Pulling the assumptions out of it leads to a recognition that there is a user or project need that might be solved through documentation. "You don't just want some words, those words have to do something, they have to help someone." Looking deeper, perhaps a process is needed to get there. "Docs don't happen by magic; somebody has to do something in a planned way to make docs." The fact that it has not happened yet indicates that it is real work; "if it were easy, it would be done by now".

It may turn out that documentation is not really what is needed for the identified problem; instead, fixing a bug, rather than documenting a way around it, may make more sense. Or, perhaps, the project's culture is not particularly welcoming, so changing the culture, rather than creating more documentation about conduct, is the right path forward.

Advisor

There are four essential questions that need to be answered for any documentation project, McKean said. "Why are we doing it? And, then, what are we actually doing?" After that, figuring out what process will be used to accomplish it; "How are we going to get it done? And, then, who is going to do it?" Getting a grip on all of that is far more complicated than the simple "I need docs" thought, which is why projects need tools to help.

Based on the case studies, Google has created the Docs Advisor, which is a guide to help answering those questions and more. It can assist projects in picking the right path, learning about their users and what they need, and figuring what documentation already exists and what is missing. Once that is determined, the guide can help "figure out how to do the actual work of writing docs and maintaining them". The Season of Docs folks teamed up with Erin Kissane to produce the guide.

The first step is to determine the level of resources available and the urgency of the work. Projects with limited resources and no urgency should try a planless iteration approach, which applies continuous iterative improvements to the documentation. If the project has limited resources and tight timelines, the mini-overhaul is a good choice; it applies some focused effort to documentation in a particular area or format, such as a new version of the software or a video tutorial, for example. For the project with ample resources and a tight timeline, the heroic overhaul, which tries to address all of the project's documentation woes in one focused effort, may be indicated. The links to the guide give more information about these paths, including the upsides and downsides of each. Those rare projects that have lots of resources and relaxed timelines can do anything they want, McKean said, because they lack any constraints.

Next up is figuring out the project's users and their needs; determining users' range of expertise and the conceptual hurdles they have faced is part of that process. Within the project, identifying the contributors and, "more importantly", the potential contributors is crucial; beyond just helping with writing documentation, they may have documentation needs as well. Taking notes is a vital part of the process, starting with things "off the top of your head" that are known or suspected about the documentation needs of various users.

There are some techniques for filling in any missing information about where the documentation problems lie. For example, friction logging—observing someone doing a specific task using the project and writing down everything that thwarts their efforts—can be used to find areas that need attention. Tracking their reactions and feelings as they go through the exercise will also help show the worst problem areas. Gathering up complaints from bug reports, email or forum discussions, and other sources can help fill in gaps, as can simply asking users specific questions about the project and their usage of it. "Resist the urge to use an LLM [large language model] to summarize this for you; you want exact data." As a last resort, doing a survey is a possibility, but it is difficult to design one that can provide the qualitative feedback needed to direct documentation efforts, she said.

Using that information, it is time to "assess and plan" by prioritizing the needs for documentation and deciding on the overall goals. As part of that, making an inventory of the existing documentation and gauging the level of resources available for writing and editing will help in choosing a structure to work toward for the documentation.

Archetypes

The team worked with Daniel Beck on a set of documentation project archetypes that can help guide different kinds of work. Each of them answers some of the questions that might arise with respect to a given documentation task. She gave "The Migration" as an example, asking attendees if they had ever done a migration of documentation from one platform to another—and whether they would ever want to do so again. "Probably not, right?" she said with a laugh.

There are a number of reasons why a project might want to migrate to a new platform, including that users and contributors cannot find the information they need, people who want to contribute to the documentation cannot do so, or that the content-management system is old and out of date. The archetype can also help show when a specific type of project should not be done. A migration should not be undertaken when only a single vocal contributor wants to switch to their favorite tool, for example. That is simply solving one person's problem, when the goal should be to address problems that many are experiencing. The archetype will also help define what is out of scope, set the end goals, and describe some of the failure risks for that kind of project.

She listed and briefly described the dozen other archetypes. Each of them has a name and accompanying illustration (see "The Manual" at left), a description of the audience it is intended to assist, when to do it (and when not to), the key people, how to figure out what skills will be needed, and so on. For a migration to a static site, for example, a technical writer will need some experience using static sites, but will also need change-management experience. The archetypes refer to each other as possible precursor or add-on tasks; for migration, perhaps "The Prototype" makes sense in order to test whether the new platform provides all of the benefits that are hoped for.

McKean had some caveats about the archetypes, as well. She warned against tunnel vision and immediately over-focusing on a single archetype. "There is a risk of getting overcommitted to a certain kind of docs project, but good outcomes don't happen by force, they happen by lining up your goals, and your resources, and your key abilities with what you want to achieve." Another thing to watch out for is the "Abilene paradox", which is a form of groupthink where the decision that is made is an outcome that no one really wants, which happens because participants mistakenly think that everyone else does want it. "Make sure you get real, enthusiastic buy-in."

Never-ending projects should be avoided, so building in rest stops along the way is important. Pick a point where there will be a tangible intermediate outcome for the project and pause to check in with participants. If only one person wants to continue, though, it makes sense to put the project aside rather than have it rest on the work of a single person.

Another good tool is a pre-mortem, where participants brainstorm all of the ways that the project could fail and work out what might be done to prevent or fix those things. It only makes sense to consider problems that are under the project's control, however; alien invasion, for example, is not something that can (or should be) prepared for.

Onboarding

Bringing technical writers onto a project—assuming they can be found in the first place—can be difficult. "80% of onboarding is tooling", but, as with development, there are countless tools and configurations used for producing documentation. Writers are more technical than might be guessed, she said; they are "often building or duct-taping tools". A good starting point for a new writer on a project is to have them revise the README and project-setup documentation, possibly just as an exercise to get them to the point of being able to make their first change and commit.

The other 20% of onboarding is learning who to ask, "usually about tools". Tech writers thrive in an "informal network", she said; when introducing them to someone, ask that person who the next two people to talk to are. The same questions about the norms will often elicit different responses from various parts of the project, so it is important for the writer to learn about that. "Because, if a technical writer is going to annoy you, and they are absolutely going to annoy you, you want them to annoy you in the right way."

Due to "Cunningham's Law", attributed to wiki creator Ward Cunningham, a writer's first draft of a new document may deliberately be wrong: "because they want you to look at it, they want you to have a reaction, and they want you to point them in the right direction". If that draft were mostly right, "your eyes would skim over it", and they would not get the feedback that they need. Tech writers are students of psychology, she said, "they're going to trick you into helping them".

For some help with onboarding, she recommended the onboarding toolkit that had been developed as a companion to Nicola Yap's presentation at the 2021 Write the Docs conference (YouTube video). "If you're lucky enough to have a tech writer to onboard into your community, definitely look at that."

The project will probably want to use some kind of metric to gauge its progress; her only rule for metrics is that if they change in one direction or another, that needs to cause some change in the project or else it is simply a "vanity metric". If GitHub stars is the metric and having it go up just means that "you pat yourself on the back a little bit harder", it is not a good metric. If the documentation is completely changed and the number of downloads or installs is being used as the metric, what will the project do if those numbers go way down (or up)?

Once a documentation project is underway, the project can consider getting writers together for a sprint. Maddox has a blog post about running such a sprint that McKean recommended. She also suggested the Write the Docs organization—"they're such friendly people, they have great conferences". The organization runs a free Slack network for connecting with others in the community, including looking for contributors on a dedicated channel for open-source projects. There are lots of other documentation-project resources on the web site as well.

Q&A

The first question asked about the lack of semicolons in the presentation and McKean admitted to "hoarding them"—to laughter. The second was, inevitably, about using LLMs instead of documentation writers, and, in particular, how to convince management that human writers are needed. One part of the problem is training, McKean said; if the project is new or the documentation is outdated, where will the LLM get the information it needs for a summary?

Another problem is that the output of LLMs tends to be "well-formed English text", which makes it harder for people to spot errors even when they know the subject matter. "It just flows smoothly into your eyeballs and kind of bypasses your brain sometimes." It is "intended to be statistically average text", so unless management wants statistically average documentation, as opposed to "great docs", it will take extra work create documentation that is engaging for users. LLMs can be useful for smoothing out difficult parts, or assisting writers who are not entirely comfortable using English, but the old "garbage in, garbage out" saying is clearly applicable to LLMs.

For projects (or managers) that want to experiment with LLMs, she suggested using "The Prototype" archetype to test the output. The goals and criteria should be established ahead of time, so that the output can be judged fairly. "Test it on real users, don't just test it with management"; people in management may have a hard time seeing past the money that can be saved.

Speaking of money, her next answer largely consisted of an explanation of why tech writers should be paid for contributions to open-source projects. The question was about finding tech writers for projects and she suggested the Slack channel at the Write for Docs site. She noted that, while tech writing is one of the highest paid writing jobs out there, "it's not the highest paid tech job you can have". Tech writers are often also developers who could make more doing that, but choose writing because they like it more. They do not get the same boost on their resume that a developer gets from contributing to an open-source project; "if you can pay a technical writer, please do". There are tech writing students who are looking for portfolio projects, who can be found on that channel; they may not require payment, but they will likely need much more onboarding and mentoring than a professional tech writer will.

The final question was about projects lacking a tech writer looking to "muddle through" and get the best documentation that they can. McKean noted that most developers may not be technical writers, but they "are almost certainly a technical explainer"; she suggested getting out of the documentation mindset and instead creating an explanation as if it were for an email to a friend. If it needs smoothing out from there, finding someone to read it and make suggestions or using an LLM may help. Instead of starting out with the idea of creating "capital-D docs", simply describe what the project is and does—and why people would want to use it—as project developers have probably already done in email with their friends.

The slides from McKean's talk are available; a video of the talk should appear sometime over the next few weeks on the Linux Foundation YouTube channel. [Update: That video has now been posted.]

[ I would like to thank the Linux Foundation, LWN's travel sponsor, for assistance with traveling to Tokyo for Open Source Summit Japan. ]

Index entries for this article
Conference	Open Source Summit Japan/2025

---