When I joined Altitude in 2025 as Head of Engineering, the company had a working rules-engine product and a thesis that LLMs would let it serve more conditions, more workflows, and more enterprise customers without exploding the maintenance cost of the rules library. The pivot from a rules-based core to an LLM- and agent-based core became the year's defining technical bet.
The pivot took longer than a refactor and shorter than a rebuild. It also looked different in months four through six than I expected in month one. What follows is the playbook I would hand to another CTO facing the same fork — for digital health, for revenue cycle, for any clinical product whose first generation was rules-based and whose next generation should not be.
The shape of the work, in order
Step 1. Stabilize the existing rules engine
Do not refactor what you are about to replace. Freeze it where you can. Triage incoming work into two buckets: "do in the old system because we cannot wait" and "wait for the new system." The discipline of writing the second bucket is what creates the budget for the pivot. Without it, the team is doing two systems' worth of work and the new one starves.
The hard part of this step is political, not technical. Customers want new features. Sales wants new conditions covered. The engineering team has the muscle memory of building in the old system. The CTO's job is to draw the line and hold it for two quarters.
Step 2. Design the new substrate end to end
Skill files, retrieval, governance, evaluation, logging, deployment. On paper, before code. The deliverable is a 10–20 page document that an engineer outside the project could pick up and roughly understand.
The piece teams most often skip in this step is governance — the part of the system that handles human review, drift detection, and incident response. Skipping governance in the design phase means it gets bolted on later, and the bolt-on is always less reliable than the integrated version.
For a healthcare product, this is also the step where you write the BAA and HIPAA story for the substrate. See HIPAA-compliant AI automation for what the auditor will want to see when this substrate hits production.
Step 3. Migrate one scenario
The simplest, least-clinical, lowest-stakes one. Build the substrate around it. Discover the gaps the design document missed. Rewrite the design document.
The instinct here is to pick the most impressive scenario for a demo. Resist it. The first scenario's job is to harden the substrate, not to wow the board. The board can wait two months for the better demo.
Step 4. Migrate three more scenarios in parallel
Same substrate. Each scenario will shake out a different gap — one will reveal an evaluation weakness, one will reveal a governance gap, one will reveal a performance issue. Fix them in the substrate, not in the scenario.
This is the step where the substrate either earns its keep or you discover that you built the wrong substrate. If three scenarios in parallel feel manageable, the substrate is good. If they feel like three separate builds, the substrate is not yet right. Either way, this step gives you the answer.
Step 5. Reach the pivot threshold
Half the volume on the new substrate. Golden cases passing. Governance running. The rules engine is now a fallback, not the system of record. The pivot threshold is what the board sees and what changes the company's external story.
This is also the step where the team starts to feel the relief. The two-system tax has been heavy. Crossing the threshold means the new system is real and the old one's days are numbered.
Step 6. Sunset the rules engine scenario by scenario
On a schedule the business can plan around. Not all at once. Not on a developer's hunch. A documented schedule, communicated to customers and to internal teams.
The temptation to delete the rules engine the day the pivot threshold is reached is strong. Do not. The rules engine is the rollback path for any new-substrate issue. Keep it warm for a quarter after the last scenario migrates off. Then sunset it deliberately.
The org changes that make the pivot work
The technical sequence is the easy half. The harder half is the organizational change. Three moves matter most:
Move 1. Restructure ownership around the substrate, not the scenarios
In a rules-engine product, ownership tends to be by scenario or by condition. A team owns diabetes, a team owns cardiac, etc. In an LLM-substrate product, ownership shifts. There is a substrate team that owns the platform — retrieval, evaluation, governance, observability — and there are scenario teams that own the prompts, the skill files, and the clinical content.
Making this restructure happen at the same time as the technical pivot is the failure mode. Sequence it. Get the substrate working with one team owning everything end to end, then split the org once the substrate is stable.
Move 2. Add governance roles before they are needed
The first time you wish you had a clinical reviewer in the design room is also the latest you can comfortably add one without re-doing work. Hire (or contract) the governance roles in the substrate design phase, not in the production deployment phase. See SOC 2 and AI governance for who needs to exist on the org chart.
Move 3. Communicate the pivot externally on a quarterly cadence
Customers, board, prospective hires. The pivot is a year-long story. If the only update they get is "we're working on it," they fill in the silence with skepticism. A short quarterly note — what we shipped, what we learned, what is next — keeps the trust intact.
The budget shape
The pivot is roughly equivalent to a 1.5x engineering load for six months, then a 0.7x load for the next six (because the substrate work compounds while the rules-engine maintenance falls off). Plan the budget for the 1.5x peak. If you cannot get budget for the peak, the pivot will stall in step 4 because the substrate is not getting enough load to harden.
The right way to communicate this to the board is honest. "We are paying a tax for two systems for two quarters in order to retire one of them and unlock the new product surface." Most boards will accept that framing if the alternative — staying on the rules engine — is articulated as costing more in the long run.
The signal that the pivot has worked
You stop arguing internally about whether to migrate the next scenario. The new substrate is the obvious place for new work. The rules engine is a thing the team maintains because it is still in production, not because it is where they want to be. New engineers join and ramp onto the substrate, not the rules engine.
When that signal is real, the pivot has crossed the line from "in progress" to "done." The remaining work is sunset and cleanup. The hard part is behind you.
If your team is staring at a similar pivot, book a call. I will tell you what is two months of work and what is a year of work, on the call. The full version of this playbook — including the milestone cadence, the budget templates, and the way to communicate the pivot to the board — is in the Healthcare AI Automation Playbook. For the technical detail on what replaces the rules engine, see replacing clinical rules engines with LLM reasoning.