Our journey began with a bold challenge I gave our Information Experience team:
“Build a flying car; don’t spend effort trying to make horses run faster.”
For us, the "horse" was the traditional search functionality to find product documentation along a human-created navigation structure. The "flying car" was something far more ambitious: a truly conversational experience where users could simply ask, "Does product X do Y?" and get a direct answer.
We knew our users were spending quite a bit of time hunting for answers to their product questions. Complex questions could take up to five minutes to find pertinent information. That was five minutes too many. We needed a new way for people to find the product information they needed without digging through pages of documentation.
We focused on three core problems:
- Speed: How fast can a user get a valuable answer to their product question?
- Trust: The assistant must never invent unsupported guidance.
- Security: It had to respect our existing permissions and system architecture.
The early results gave us confidence: after introducing the Documentation Assistant internally, the time for those complex questions dropped to under a minute. An overwhelming 90% of surveyed users said finding information they needed felt “very easy.”
So how did we build this “Flying car”?
Under the hood, the Guidewire Documentation Assistant is a Retrieval-Augmented Generation (RAG) system. It's a specialist, not a generalist. It’s an LLM that only knows Guidewire product documentation.
- Grounded in Docs: It queries a semantic database of our documentation, finds the relevant chunks, and uses only those to construct an answer.
- A "Responsible No": If a query is outside of Guidewire product information, it will explicitly say “I don’t know,” redirecting the user back to what answers it can help the user find. This was vital for building trust.
- Access-Controlled: The Documentation Assistant relies on our existing security backend, ensuring users only see answers based on documentation they are already authorized to access.
The Journey to Conversation
We treated the “information finding” experience as our top priority throughout the project, which was critical to our content strategy and architecture. Key to teaching the assistant were the “golden Q&A sets.” We extracted common user intents from our search logs, created ideal answers, and used them as our automated accuracy test set.
We deliberately started small, running stress tests and user research with a beta tester group to observe how people framed their questions and when they fell back to traditional search. The feedback loop was constant, helping us refine everything from the retrieval parameters to the Documentation Assistant experience.
This work also forced a massive content cleanup. We standardized URLs, reduced redundant content, and integrated better with our taxonomy to handle Guidewire-specific terminology. This less-visible content and infrastructure work is the foundation that can correctly interpret and find the right information every time.
Today, the Documentation Assistant is a specialized RAG solution with strong early metrics. We’re backed by a systematic testing framework that lets us measure and improve accuracy over time. We look at questions without answers, we evaluate what was found and what was not found, we improve answers.
The “flying car” is now reliably off the ground. In the first 60 days since external launch, we helped over 5,000 customers, partners, and employees who rely on Guidewire documentation for their projects to find the 45,000 answers they needed, effortlessly.