Everyone wants to talk about AI models. Which one is best? Should you choose an open model or a frontier model? Is it worth waiting to see what comes next?
They’re reasonable questions, especially when the technology is changing so quickly. But after helping build Guidewire’s approach to agentic AI over the past couple of years, I’ve come to believe that choosing a model is only one part of the challenge.
Putting agentic AI into production requires much more. You need the right context, clear boundaries, meaningful evaluation, and the experience to know what to do when something goes wrong.
Give AI the Context It Needs
To make a general-purpose model useful inside an insurance company, you have to give it the information and guidance that fit the work you’re asking it to do.
This includes relevant insurance data, workflows, business rules, and regulatory requirements. An agent supporting a claims professional will need different information and tools from one helping an underwriter or developer.
One of the harder lessons we’ve learned is that more information doesn’t always produce a better result. A claim or policy can contain thousands of fields and relationships, and giving an agent access to all of them can actually make its behavior less predictable. Agents tend to perform better when they receive the specific information they need.
The same principle applies to authority. An agent should have enough access to complete its assigned job without being able to take unrelated actions outside that scope.
Getting an Agent Ready for Production
A successful demo can tell you whether an idea has potential, but production introduces a different set of questions.
Before an agent goes live, teams need to understand what it’s allowed to do, how far the impact could extend if something goes wrong, and when a person should remain involved. They also need evaluations that verify the agent follows business rules and responds appropriately to unexpected situations.
Testing should include adversarial scenarios, too. Teams need to know whether an agent stays within its role when pushed beyond its boundaries and whether it escalates when it lacks enough information to make a sound decision.
Evaluation continues after deployment. Teams should watch for changes in behavior and maintain a clear record of the actions an agent took and the information it used.
Business analysts and end users also need to be involved in defining and evaluating what good performance looks like.
Start Building Experience Early
That is why I would be cautious about waiting for agentic AI to mature before getting started.
When organizations moved to the cloud, there was already a relatively established playbook. Agentic AI is different. Companies are still learning how to evaluate and govern these systems.
At Guidewire, we started with simple use cases and demos to understand where the technology worked and where it struggled. Moving toward production taught us lessons around evaluations, guardrails, authority, and operations.
Those lessons took time.
I don’t think insurers should move fast and break things. Agentic AI demands a thoughtful approach. But organizations should start early enough to build experience with the technology and learn how it fits their own business.
The models will keep changing, but learning how to use them responsibly takes time, and each lesson helps inform the next. Organizations that wait will still have to go through the same learning process when they begin, and by then, they may be further behind than they expected.
How Guidewire Is Helping Insurers Move Forward
At Guidewire, we describe our AI strategy in four words. AI everywhere for everyone.
We’re building AI into the systems insurers already use while giving customers flexibility in how they put it to work. They can use agents built by Guidewire, build their own with the Guidewire Agentic Framework, and choose the model that best fits each task.
That flexibility gives insurers room to adapt as the technology evolves. But the ability to put AI to work safely and effectively is something organizations build over time. The sooner they begin, the better prepared they’ll be for what comes next.
This blog is inspired by a conversation between Brian Desmond, Chief Marketing Officer at Guidewire, and Ben Chang, VP of Engineering – Application Platform. Watch the full Guidewire Conversation.
Listen to additional Guidewire Conversations here.