The role of technical writers in the age of RAG and agentic LLM solutions
Good news: They’re gonna need us more than ever, fellow writers!
RAG solutions
Retrieval-augmented generation (RAG) is a technique for using a large language model (LLM) to generate reliable, accurate output by grounding that output in content from a knowledge base.
For example, to use RAG to answer a question about a software product, you would perform the following steps:
- Search the online product documentation for an article that contains information to answer the question
- Extract the HTML contents of that article and convert it to plain text
- Submit the following prompt to an LLM:
Article:
------
<article-text-here>
------
Answer the following question using only information from the article.
If there is no good answer in the article, say "I don't know".
Question: <user-question-here>
Answer:Writers, here’s where they need us!
For many domains, LLMs cannot generate useful output without the article-in-the-prompt pattern shown above*. Where will those articles come from? Writers.
*In a nutshell: If the content that was used to pre-train an LLM doesn’t contain a sufficient amount of text related to a domain, then the LLM cannot generate good output for that domain. This could happen if the domain is esoteric or is so recent that no pre-training content could have mentioned it yet. The way to solve this problem is to include information about that domain in the prompt, like the <article-text> above.
Example
Imagine you have invented a new hand-held writing implement, called the carbonWrite 9000. Because you just invented it, there’s no way information about your new product could be in the data that was used to pre-train any LLM. As a result, if you asked an LLM about the carbonWrite 9000, you would not get an accurate answer:
However, if you include a snippet from the carbonWrite 9000 documentation in your prompt, you will get accurate answers:
In this example, the product documentation is what enables the LLM to answer questions about the carbonWrite 9000.
A RAG solution won’t work without pulling up-to-date, accurate, domain-specific content into prompts.
Agentic solutions
Currently, the idea of agentic solutions is still evolving. But here are terms being used to describe these LLM solutions, from a simple, very prescribed approach to a complex one that allows for the most variability in how goals are accomplished:
- Assistants — Solutions that classify user input and then return hard-coded or LLM-generated answers, or run pre-defined code blocks to perform actions.
- Workflows — Solutions that use pre-defined criteria to trigger a chain of components that execute specific tasks one after another. Each component might use LLMs to identify runtime parameters from natural language input or from the output of an earlier component and to generate output for the next component in the chain.
- Agents — Solutions that understand a user’s goal, plan how to accomplish that goal, choose which tools could be used to perform the tasks, and then act to accomplish the goal, all while being transparent and communicating with the user. LLMs are used in multiple ways: understanding a user’s goal might involve classifying intent and extracting entities to understand specifics; planning might involve generating a list of steps; choosing tools might involve predicting which tool has the highest match to the goal and the plan; acting might involve generating function names, commands, or API calls; and communicating with the user might involve generating activity logs or status updates.
- Autonomous agents — Fully autonomous agents are agents that are proactive, anticipating users’ needs instead of only responding to user input. And they are adaptive, learning from outcomes and from interaction with their environment. LLMs can be used to analyze a user’s environment and status to identify proactive actions that would help the user. And LLMs can be used in “reflection” activities, where an LLM is prompted to analyze the solution’s own results to find better ways to solve problems in the future. The generated analysis is fed back into prompts and programming logic.
Distinctions between these types of solutions can be fuzzy. For example, an assistant might trigger a workflow, or one component in a workflow might invoke an agent, or an agent might trigger a workflow or hand a conversation over to an assistant.
Writers, here’s where they need us!
Like with RAG solutions, assistants, workflows, and agents need to pull in relevant content to ground different kinds of prompts. Where will that content come from? Writers.
Example
Imagine the carbonWrite 9000 has an optional light attachment for writing in the dark. A user is enjoying writing in the dark very much, but is annoyed by how often the light’s battery needs to be recharged. So they say:* “I wish the battery on this thing lasted longer!”
*The carbonWrite 9000 has a voice interface, by the way.
If there was an autonomous agent to support carbonWrite 9000 users, you can imagine its programming logic might do something like the following:
- Classify the user input to determine if the user is asking a question or making a request/giving an instruction.
- Search the carbonWrite 9000 documentation for relevant information.
- If the user input was classified as a request or an instruction, determine if there is a tool to accomplish the user’s goal by prompting an LLM grounded in the relevant information.
- If there is a tool to call, then determine which framework or infrastructure is needed to execute the call by prompting an LLM, with the previous prompt and generated output included as context.
- Generate the call syntax and parameters by prompting an LLM, with the previous prompts and generated output included as context.
The following image shows prompts and generated output for steps 3–5:
This scenario is simple, but you can see how the logic would flow along and how LLMs could be used at each step of the way. And you can see how important that article from the product documentation is! Without that article, the agent would not have been able to do any of this:
Agentic solutions won’t work without pulling up-to-date, accurate, domain-specific content into different kinds of prompts at run time.
Technical writers will need to adapt
For RAG solutions to be successful, the knowledge base should be optimized for that use:
- Search engine optimization is more important than ever, because if a RAG solution cannot find the relevant content, the solution won’t work.
- The structure and writing style of the articles themselves must be optimized for LLMs to extract needed information.
One way to approach creating knowledge base content for RAG solutions is to use question-driven content development:
For agentic solutions, content teams will need to get creative about making relevant information available in different formats. (This would be similar to the way that many writers already work in YAML or JSON files today, to collaborate with designers on GUI content or with developers on error messages.)
For example, the following post demonstrates using a metadata JSON structure as the knowledge base for an LLM-powered natural language interface that has basic agentic properties:
We have adapted before
Technical writers are experts at adapting. Here are examples of ways writers have adapted how we write and publish content in the last two decades:
- From books to websites — When companies started publishing product documentation online, writers adopted topic-based writing.
- Search engines — When product documentation first moved online, there was often an online index, just like the index of a book. Then search engines came along, so writers learned how to optimize their content for search.
- User communities — Enthusiastic software users have shared tips in user groups since the beginning of the internet. Many technical writers branch out from their product documentation work to contribute to online user communities, by publishing articles and answering technical questions. From a content strategy perspective, it makes sense to meet users where they are, to make sure information being shared in online communities is technically accurate, and to respond to trends in users’ comments and questions.
- Mobile devices — When product documentation first moved online, many people were reading that documentation on desktop computers (or printing PDFs and reading those! 😆) These days, many people are now reading product documentation on tablets with touch screens or on the tiny screens of mobile phones. Readers search, navigate, and consume content differently depending on the type of device they are using. See: stats and reading comprehension study
- Video — When product documentation first moved online, few people had the bandwidth to watch how-to videos. But now, many people actually prefer to get their information by watching videos. As such, today’s content strategy must also include video content. See: stats and academic paper
In other words, updating how we write and publish content so RAG and agentic solutions can be successful isn’t the first time and won’t be the last time writers adapt to our ever-changing industry.
Conclusion
Some people think that the advent of LLMs means we don’t need writers.
But it turns out that those fancy-pants RAG and agentic solutions everyone is so excited about cannot work without pulling in up-to-date, accurate, domain-specific content that writers create.
So, actually, writers are needed more than ever!
Sure, we’ll have to adapt a bit. But it’s ironic that our old-fashioned writing skills are powering these cutting-edge LLM solutions.
