Chapter 10: Short-term Implementation The First 90 Days

Ali Süleyman Topuz

“The distance between a plan and a production system is not measured in lines of code. It is measured in the quality of decisions made under pressure, with incomplete information, on a deadline.”

The Transition from Mapping to Building

The journey mapping work of Chapter 9 produced something concrete: a friction inventory, a set of AI enablement points with clear design briefs, and a picture of which journeys to improve first and why. That work is done in the planning register it is documented, prioritized, and agreed upon. The first 90 days are about executing against it.

The transition from planning to execution surfaces a set of challenges that planning work tends to conceal. A design brief that was specific enough to agree on in a meeting may turn out to be underspecified when a developer sits down to implement it. A data source that was “available” during the demand analysis may turn out to require three weeks of preparation before it is usable in production. An organizational agreement to allocate two engineers to the first AI project may dissolve when those engineers are pulled back to address a critical bug in the core product.

These challenges are not signs that the planning was inadequate. They are the normal texture of early AI implementation, and the teams that navigate the first 90 days successfully are the ones that expect them rather than being surprised by them. The first 90 days are not a delivery sprint they are a learning sprint with delivery milestones. That reframing is not a lower standard. It is a more accurate description of what the first 90 days of AI implementation actually are, and it produces better outcomes than treating the first 90 days as a conventional software delivery cycle.

Structuring the First 90 Days

The most reliable structure for the first 90 days divides the period into three 30-day phases, each with a distinct primary objective and a set of specific outputs that define whether the phase was successful.

Days 1–30: Foundation and First Build. The objective of the first 30 days is to stand up the technical foundation needed for the first AI system and produce a working internal prototype. At the end of this phase, the team should have the core infrastructure in place API access, logging, the retrieval layer if the first use case requires it and a prototype that demonstrates the AI behavior on real inputs from the target use case. The prototype does not need to be integrated into the product. It does not need to handle edge cases. It needs to show that the AI can produce the right kind of output for the right kind of input, enough to ground a concrete conversation about what the production system should do and where the current prototype falls short.

Days 31–60: Evaluation and Internal Validation. The objective of the second 30 days is to evaluate the prototype rigorously and validate it with internal users who represent the experience that customers will have. At the end of this phase, the team should have a documented evaluation quality metrics measured on a representative test set, failure modes identified and catalogued and feedback from internal users who have interacted with the system in conditions close to production use. This is also the phase in which product integration work begins: the AI capability is connected to the actual product interface, even if behind a feature flag.

Days 61–90: Controlled Rollout and First Learning Loop. The objective of the third 30 days is to move from internal validation to a controlled rollout with a small set of real customers a beta cohort selected for their willingness to provide feedback and their relative tolerance for early-stage experiences. At the end of this phase, the team should have real-world performance data: customer usage rates, quality feedback, edge cases that internal testing did not anticipate, and at least one round of iteration driven by what the rollout revealed. The system should not yet be available to all customers, but it should be operating in production, with real users, under real conditions.

The three-phase structure is a framework, not a rigid schedule. Teams working on simpler use cases may move through phases faster. Teams working on use cases with more complex data dependencies may spend more time in the first phase before the prototype is possible. The framework’s value is in the discipline it imposes: each phase has a clear test for success, and failing that test is information information about what needs to be addressed before moving to the next phase, not a reason to skip to it anyway.

Setting Up the Implementation Team

The first AI implementation is an organizational experiment as much as a technical one. The team that builds the first system will develop instincts, practices, and architectural opinions that shape every subsequent system. Getting the team configuration right in the first 90 days matters more than it will for any subsequent project, because the patterns set here propagate forward.

A functional first AI implementation team for a B2B SaaS company at the Nexus scale requires four distinct roles. Not four people these roles can overlap, and in smaller teams they often do but four distinct contributions that must all be present.

The technical lead owns the architecture and implementation. This person needs to understand both the AI system design (from Chapters 7 and 8) and the product’s existing infrastructure well enough to integrate the new system cleanly. They make the implementation decisions that are too consequential to be made incrementally: which model to use, how to design the prompt, what the retrieval strategy is, how the system degrades when confidence is low. These decisions should be documented as they are made, because they form the beginning of the team’s architectural memory.

The product lead owns the user experience and the success criteria. This person translates the design brief from the journey map into specific, observable behaviors that the team can build toward: what does the system say in the easy case, the hard case, and the failure case? What is the triggering condition? What does the user control? The product lead also owns the evaluation criteria they define what “good enough for internal testing” looks like and what “good enough for a customer beta” looks like, which are different thresholds and should be defined before the builds begin rather than negotiated after.

The domain expert provides the ground-truth understanding of what the AI system needs to know. For a customer-facing AI use case, this is typically someone from customer success a person who has run hundreds of the interactions the AI will be trained to assist with and who can evaluate AI outputs with the judgment of experience. For an internal AI use case, this is the operational expert in the workflow the AI is supporting. Domain expert involvement in the first 90 days is not optional. The team that builds an AI system without deep domain expertise builds toward a specification that looks right on paper and fails in practice.

The evaluation lead owns the measurement layer. This role is frequently collapsed into one of the other three, which is acceptable if the person taking it on has the discipline to approach evaluation as a first-class concern rather than an afterthought. The evaluation lead builds the test set, runs the metrics, and critically maintains the record of how quality changes as the system is iterated. Without this role, the team has no reliable signal about whether their changes are improving the system or degrading it.

Building the First System

The first AI system the team builds should be built to be iterated, not built to be finished. This principle, obvious in retrospect, is violated by most first AI implementations because the instincts that make traditional software development work building toward a specification, integrating all components before testing, deferring refactoring until the feature is complete produce poor outcomes in AI development.

AI development has a different cadence. The most valuable work happens not when the system is finished but when the first real inputs are run through it and the team sees, for the first time, what the system actually does versus what they expected it to do. That first encounter with real behavior is the foundation of everything that follows: it reveals the gaps in the specification, the data quality problems that abstract analysis missed, the user experience failure modes that were invisible until the system was running. Reaching that encounter as quickly as possible before the team has committed to an architecture based on what they expected is the most important discipline in the first 30 days.

The practical implication is to start with the smallest possible vertical slice of the use case. Not the most representative case, not the edge cases, not the full range of inputs the system will eventually handle the simplest case. One category of query, one type of user state, one well-understood scenario. Get the system to handle that case correctly. Then expand.

This approach conflicts with the temptation to build comprehensively from the start, which is driven by the reasonable fear that the simple case will require the same infrastructure as the complex case and that starting simple will mean throwing away work. In AI systems, that fear is usually unfounded. The infrastructure the API integration, the logging layer, the evaluation harness is reusable regardless of how simple or complex the first scenario is. The things that must be rebuilt when the team’s understanding deepens are the prompts, the retrieval configurations, and the edge case handling: exactly the things that should not be locked in before seeing real behavior.

Prompt Architecture

The prompt is the primary technical decision in the first 30 days, and it deserves more deliberate attention than most teams give it. A well-structured prompt is not simply an instruction it is a specification of the system’s behavior, including what the system should do when the input is ambiguous, what it should not do under any circumstances, and how it should communicate the boundaries of its knowledge.

Prompt architecture for a B2B SaaS AI use case has five components. The system instruction defines the AI’s role, scope, and constraints: what it is, what it knows, and what it does not do. The context assembly section specifies what organization-specific information is provided to the AI at inference time: customer account data, retrieved documentation, session state. The input framing section defines how the user’s input is presented to the model and what metadata about the input context is included. The output specification defines the format, length, and style of the response the model should produce. The fallback instruction defines what the model should do when it does not have enough information to answer the query a critical component that many first implementations omit, and that produces some of the most visible failures in early production systems.

Each of these components should be tested independently before the full prompt is assembled. The system instruction should be tested to verify that the model stays within scope. The context assembly should be tested to verify that the model uses the provided context rather than relying on prior training knowledge. The fallback instruction should be tested with inputs that are deliberately outside the scope of the system to verify that the model acknowledges its limitations rather than fabricating an answer.

Evaluation from Day One

The evaluation harness the infrastructure for measuring whether the system is performing well should be built at the same time as the system, not after it. This is a discipline that most teams resist because it feels like overhead when the primary goal is to get a working prototype. It is not overhead. It is the mechanism by which the team learns whether their implementation decisions are moving in the right direction.

The minimum evaluation infrastructure for a first AI system consists of three components: a test set of representative inputs with documented expected outputs, a scoring function that measures the quality of actual outputs against expected outputs, and a logging layer that captures every inference input, output, and user interaction in a queryable form.

The test set is the most important of the three. It should be built before the system is built, by the domain expert and the product lead working together. The test set answers the question: what does good look like, concretely, for the range of inputs this system will handle? Building the test set before the system is built forces a specificity of thinking about expected behavior that is more valuable than any amount of post-hoc evaluation.

Managing Early Stakeholder Expectations

The first 90 days of AI implementation are a period of high organizational visibility and high uncertainty. Stakeholders who approved the AI investment have expectations about what they will see and when, and those expectations are frequently calibrated to demos and vendor presentations rather than to the actual experience of building AI systems. Managing the gap between those expectations and the reality of early implementation is a communication task that is as important as the technical work.

The core principle of stakeholder communication in the first 90 days is to be specific about what is being measured and why. “The system is not ready yet” is a communication failure. “The system is handling the configuration guidance case correctly in 78% of test scenarios, and the primary failure mode incorrect guidance when the account type is non-standard is the focus of this sprint’s prompt engineering work” is a communication success. The second version gives the stakeholder something to evaluate, something to ask questions about, and something to update their expectations against. It also demonstrates a level of rigor that builds confidence even when the numbers are not yet where they need to be.

The first 90 days should include at least three formal stakeholder updates: at the end of each 30-day phase. Each update should cover three things: what was built or validated in the phase, what the evaluation showed about quality and readiness, and what the next phase will focus on and what success in that phase looks like. This cadence is not about demonstrating progress it is about maintaining the organizational trust that sustains investment through the difficult middle period of any implementation.

The most dangerous communication pattern in early AI implementation is the one that protects stakeholders from bad news until the bad news is unavoidable. A stakeholder who learns in week ten that the data quality problem identified in week three has been blocking progress for seven weeks has no opportunity to help resolve it and every reason to lose confidence in the team’s ability to manage the project. A stakeholder who learns in week three that a data quality problem has been identified, that the team has a resolution plan, and that the timeline impact will be two weeks has the opportunity to accelerate the resolution and maintains confidence in the team’s judgment.

Controlled Rollout: From Internal to Customer-Facing

The transition from internal validation to a controlled customer-facing rollout is one of the most consequential moments in the first 90 days. Done well, it produces the first real signal about whether the AI system improves the customer experience in the way the journey map predicted. Done poorly, it damages customer relationships at the worst possible moment when the product team is still learning and the system is not yet at its best.

Beta cohort selection is the first and most important decision in the rollout phase. The beta cohort should be small ten to twenty customers for most use cases and selected on three criteria: demonstrated willingness to provide feedback (customers who have participated in product research in the past are good candidates), appropriate usage patterns (customers who actually use the journey the AI is targeting, frequently enough that they will encounter the AI system regularly during the beta period), and reasonable risk tolerance (customers who are not in a critical operational phase where any instability would be unacceptable).

Avoid selecting the beta cohort from your largest or most strategically important customers, regardless of how willing they say they are to participate. The risk asymmetry is unfavorable: if the system performs well, the strategic customer’s experience is marginally better than it would be after a general launch. If the system has an early failure mode, the strategic customer’s trust is damaged in a way that takes significant time and effort to repair. The beta cohort should include customers where the upside of a positive experience is real but the downside of a problem is recoverable.

Beta program design should be explicit about what the customer is participating in. Customers in the beta cohort should know they are testing an early version of a new AI feature, that the team is actively monitoring quality and iterating, and that their feedback is the primary input to those iterations. This transparency serves two purposes. It sets appropriate expectations the customer who knows they are in an early beta is less alarmed by a system output that is suboptimal than a customer who believes they are using a finished feature. And it creates the feedback relationship that makes the beta period valuable: customers who understand their role as collaborators provide far more useful feedback than customers who experience the feature as they would any other product update.

Rollout instrumentation should be in place before the first customer encounters the AI feature. The minimum instrumentation for a controlled rollout consists of session-level tracking (which customers have encountered the AI, how many times, in what contexts), quality indicators (user acceptance rates for AI suggestions, explicit feedback ratings if the interface includes them, downstream behavior signals like whether the user completed the journey step successfully), and anomaly alerts (triggers that flag unexpected outputs very long responses, responses that fail format checks, responses that reference information the system should not have access to). Without this instrumentation, the controlled rollout produces very little learning, which defeats its purpose.

When Early Problems Arise

Every first AI implementation encounters problems in the first 90 days. Some are technical the retrieval quality is lower than expected, the model hallucinates in a specific input pattern, the latency is higher than acceptable. Some are experiential users do not understand what the AI is doing, the timing of AI assistance is wrong, the tone of AI-generated content conflicts with the product’s voice. Some are operational the logging volume is higher than the infrastructure was sized for, the evaluation process takes longer than estimated, a data pipeline dependency blocks the next phase.

The response to early problems should not be to hide them, to minimize them, or to work around them without addressing the root cause. The response should be to name them precisely, diagnose them systematically, and resolve them in the order of their impact on the beta customer experience and the delivery timeline.

Technical quality problems in early AI systems almost always have one of three root causes: prompt design problems (the model is not receiving the right instructions or context), data quality problems (the retrieval layer is surfacing irrelevant or incorrect content), or scope creep problems (the system is being asked to handle inputs that it was not designed for). The diagnosis should explicitly test each of these before concluding that the underlying model is the problem. Model capability is rarely the limiting factor for B2B SaaS use cases at the complexity level appropriate for a first implementation; prompt design and data quality are the far more common culprits.

Experiential problems are typically diagnosed through user sessions rather than metrics. If users are consistently ignoring AI suggestions, the timing or framing of the intervention is likely wrong. If users are frequently overriding AI suggestions in the same direction always lengthening a response the AI shortened, always choosing a different option than the one the AI recommended first the model’s calibration is likely misaligned with user preferences. The solution to experiential problems is iteration on the UX and the prompt, not escalation to a more capable model.

Operational problems are the category most likely to be underestimated in the first 90 days. AI systems generate significantly more operational overhead than traditional software features: evaluation cycles, prompt iteration cycles, feedback review cycles, and quality monitoring create ongoing work that the team must budget for explicitly. If the first 90 days are staffed at the level of a typical software sprint with no overhead for AI-specific operational work, the team will discover they are under-resourced at the worst possible moment when the beta rollout is live and real customers are encountering real problems.

The Sprint Rhythm of AI Development

AI development does not fit cleanly into a standard two-week software sprint. The work cycles at a different granularity prompt changes and retrieval configuration adjustments can be tested and iterated in hours, while meaningful quality evaluation requires enough inference volume to produce statistically reliable signal, which may take days. The result is a mismatch: either the team is running sprints that are too short to produce reliable evaluation signal, or they are running sprints that are too long and delaying the learning that should be driving their decisions.

The sprint structure that works best for AI development in the first 90 days is not a fixed time-box but a gated progression: each sprint ends when its success criterion is met, not when its calendar slot expires. The gating criterion for the first sprint the prototype sprint is a working internal prototype that handles the primary scenario correctly on at least five real-world inputs. The gating criterion for the second sprint the evaluation sprint is a documented quality score on a twenty-item test set, with the primary failure modes identified and diagnosed. The gating criterion for the third sprint the integration sprint is a product-integrated feature behind a feature flag, reviewable by internal users in the actual product context. The gating criterion for the fourth sprint the beta sprint is a live beta cohort with instrumentation active and the first week of real-world interaction logged.

This gating structure decouples AI development from the two-week clock that most engineering teams are accustomed to. Some sprints will complete in a week; others may take three. The goal is not to normalize the duration but to normalize the quality criterion. A team that ships at the end of two weeks regardless of whether the gating criterion is met is shipping arbitrarily they are optimizing for cadence over quality, and AI systems penalize that tradeoff more severely than traditional software does, because the quality failures are often invisible until they appear in production.

The practical implication for sprint planning is to size the work based on the gating criterion, not the calendar. The prompt engineering and retrieval configuration work for the prototype sprint might take one engineer four days. The evaluation infrastructure and test set construction for the evaluation sprint might take two engineers six days. The product integration work for the integration sprint might take three engineers a week. Plan each sprint to completion of its criterion, then move to the next. This produces a rhythm that respects both the pace of AI iteration and the quality standards that production deployment requires.

Alongside the development rhythm, the first 90 days should establish the documentation habit that will sustain the team’s ability to improve the system over time. The most important document to maintain is the decision log: a running record of every significant prompt change, retrieval configuration adjustment, or UX decision, annotated with the reasoning behind the change and the evaluation result that followed. A decision log is not a design document it is a record of what was tried, why, and what happened. Teams that maintain it discover, months into a project, that it is the most valuable artifact they produced in the first 90 days. Teams that skip it spend significant time re-deriving conclusions they reached and then forgot.

The First Learning Loop

The most important output of the first 90 days is not the AI system itself it is the learning loop that the team establishes around the system. A learning loop is the regular process by which the team reviews system performance, identifies improvement opportunities, implements changes, and measures whether the changes improved things. A system without a learning loop degrades over time as the world changes and the system does not. A system with an effective learning loop improves over time, asymptotically approaching the quality ceiling that the underlying approach allows.

In the first 90 days, the learning loop should run at high frequency weekly, for a team actively iterating. The weekly loop consists of four steps. Review: pull the week’s evaluation metrics, session logs, and beta customer feedback and identify the top three quality or experience issues. Diagnose: for each issue, identify the root cause with enough specificity to support a solution. “The system gives wrong answers on equipment configuration questions” is not a diagnosis; “the system gives wrong answers on questions about three-phase electrical panel configurations because the documentation for that equipment type is not in the retrieval index” is a diagnosis. Iterate: implement the highest-priority fix, whether it is a prompt change, a data update, or a UX adjustment. Implement one change at a time, so that the effect of each change can be measured independently. Measure: run the test set again after the change and compare the scores. If the scores improved, the change is confirmed. If the scores did not improve or declined, revert and diagnose differently.

The weekly learning loop is also the foundation of the team’s institutional knowledge about the system. The diagnosis and iteration decisions why we changed the prompt this way, what we expected, what we saw should be documented in a log that becomes the system’s development history. This history is invaluable when a new team member joins, when a regression occurs that requires tracing back to its cause, or when a decision made in the first 90 days turns out to need revisiting six months later.

Key Takeaways

• The first 90 days are a learning sprint with delivery milestones, not a delivery sprint. The most important output is the learning loop established around the system, not the system itself.
• Structure the first 90 days into three 30-day phases: foundation and first build, evaluation and internal validation, and controlled rollout with first iteration. Each phase has a specific success criterion that should be defined before the phase begins.
• A functional implementation team requires four distinct contributions: technical lead, product lead, domain expert, and evaluation lead. Domain expert involvement is not optional the team that builds without it builds toward a specification that looks right and fails in practice.
• Build the first system around the simplest vertical slice of the use case, not the most representative or comprehensive one. The infrastructure is reusable; reaching real behavior as quickly as possible is more valuable than building comprehensive coverage from the start.
• Evaluation infrastructure a test set, a scoring function, and a logging layer should be built at the same time as the system, not after it. The test set should be built before the system, by the domain expert and product lead, so that expected behavior is specified before actual behavior is observed.
• Beta cohort selection should prioritize feedback quality, appropriate usage patterns, and risk tolerance over strategic importance. Do not beta-test with your most important customers.
• Stakeholder communication should be specific about what is being measured and what the current numbers show. Protecting stakeholders from early bad news is a communication failure that produces worse outcomes than early transparency.