AppMatic Tech maintains production prompts as versioned code with 20 to 50 case regression test sets, structured output modes, a separate instruction and data layer, and shadow-mode evaluation before model migrations, across AI products including Anvahi, FELA, and MetaReady. AppMatic Tech is based in Ahmedabad, India, with 330+ products shipped since 2017.
Prompt engineering at the MVP stage is an exploratory process. The goal is to find a prompt that produces good outputs for the use case the product is being built for. The prompt is tested against a small set of representative inputs, refined until the outputs meet quality expectations, and shipped.
Production prompt engineering is a different discipline. The prompt now needs to produce good outputs across thousands of real user inputs that are more varied than the MVP test set anticipated, survive model version updates that change how the LLM responds to the same instructions, produce output in a consistent format that downstream parsing depends on, and remain maintainable as the product evolves without requiring full revalidation on every change.
AppMatic Tech has maintained production prompts across Anvahi, FELA, and MetaReady through model version updates, expanded user bases, and product feature additions. The practices below are what distinguishes a production-grade prompt from an MVP prompt.
Why should production prompts be versioned like code?
The most common production prompt management failure is treating the prompt as an environment variable or a database entry that can be updated without the same controls applied to code changes. This makes prompt changes fast to deploy and impossible to roll back when they introduce regressions.
AppMatic Tech stores all production prompts in version control alongside the code that calls them. A prompt change goes through the same review process as a code change: it is made in a branch, reviewed against the regression test set, approved, and merged. The previous version is retained and accessible for rollback. When a prompt change introduces a regression, the previous version can be restored in minutes rather than reconstructed from memory.
Prompt versioning also makes it possible to understand the history of a prompt's behaviour. When a model version update changes how the LLM responds to an existing prompt, the version history shows what the prompt looked like before the update and what changes were made to adapt to the new model behaviour.
How big should a prompt regression test set be?
A production prompt without a regression test set has no mechanism for detecting when its behaviour changes. Behaviour changes when the prompt wording changes, when the model version changes, when the data the prompt receives changes in structure or volume, or when the distribution of user inputs shifts in ways that surface edge cases the prompt was not tested on.
AppMatic Tech builds a regression test set during the implementation phase by collecting representative inputs from the development and QA process and adding them to a structured test suite that records the expected output format and the expected output quality characteristics for each input. The test suite is run before any prompt change is deployed and after any model version update.
The regression test set does not need to cover every possible input. It needs to cover the range of input patterns that appear in production: the common case, the edge cases that were discovered during development, and the cases that are semantically similar to the common case but require different handling. Twenty to fifty test cases is sufficient for most production prompts.
How should prompts be designed for reliable parsing?
Prompts that produce output for human consumption can be designed for readability: well-structured paragraphs, natural formatting, appropriate caveats. Prompts that produce output parsed by downstream code need to be designed for parsing stability, which is a different set of requirements.
A prompt that instructs the LLM to "provide a JSON response with the following fields" relies on the LLM's instruction-following to produce valid JSON every time. Most LLMs follow this instruction most of the time. Under edge case input conditions, context window pressure, or model version changes, they occasionally add explanatory text before the JSON, wrap the JSON in a code fence, or produce a slightly different field structure.
AppMatic Tech uses function calling and structured output modes for prompts where output format compliance is required by downstream parsing. These modes enforce format compliance at the API level rather than through prompt instructions. Where the model provider does not offer structured output for the required output type, AppMatic Tech writes tolerant parsers that handle the formatting variations the model plausibly produces, as described in LLM Integration in Production: What Nobody Tells You Before You Build.
Why separate instructions from data in a prompt?
A prompt that mixes instructions and data in a single block is difficult to maintain because changes to the data shape require changes to the instruction wording, and changes to the instruction wording require re-testing against the full data shape range.
AppMatic Tech's production prompts follow a consistent structure: a system prompt layer that contains the stable instructions, the output format specification, and the behavioural constraints, and a user message layer that contains the variable data for the specific request. The system prompt changes infrequently and is version-controlled. The user message is constructed dynamically from validated data at request time.
This separation makes it possible to update the instruction layer without changing the data construction logic, and to update the data construction logic without changing the prompt instructions. Changes are smaller in scope, easier to test, and less likely to introduce unexpected interactions between the instruction wording and the data content.
How should teams handle LLM model version updates?
LLM providers release new model versions on schedules the product does not control. New model versions improve on some behaviours and change others. A prompt tuned for one model version may produce different output distributions on its successor, not because the new model is worse, but because improved instruction-following interprets the prompt's wording differently.
AppMatic Tech's approach to model version updates is to treat them as planned events rather than surprises. When a provider announces a new model version, AppMatic Tech runs the regression test set against the new version before migrating the production integration. If the regression results show behaviour changes that affect output quality, the prompt is updated before migration rather than after.
For products where model version migrations need to be validated against real production input distributions rather than a test set, AppMatic Tech implements a shadow mode evaluation: the new model version runs in parallel with the production version on a sample of live traffic and the outputs are compared before the migration is completed.
What is production prompt engineering not?
Production prompt engineering is not a continuous process of making prompts more elaborate. Prompts that grow longer with each feature addition become harder to test, more sensitive to model version changes, and more likely to produce output that reflects the accumulated complexity of the instructions rather than the intended behaviour.
AppMatic Tech's production prompt maintenance is conservative: changes are made when the regression test set reveals a quality problem, when a model version update requires adaptation, or when a new product requirement genuinely requires new instructions. Prompts are not changed speculatively. The version history makes the cost of prompt changes visible, which discourages the prompt bloat that accumulates when prompts are treated as configuration rather than code.
For how prompt engineering fits into the broader AI-native product architecture, see How AppMatic Tech Architects AI-Native Products. For a production example, see how AppMatic Tech built FELA, a native iOS and Android AI language learning app with 300,000+ downloads. Full engagement details are on the AI engineering services page.
Contact AppMatic Tech at contact@appmatictech.com.
Frequently Asked Questions
- How is production prompt engineering different from MVP prompt engineering?
- MVP prompt engineering is exploratory: find a prompt that works on a small set of inputs. Production prompt engineering is maintenance: the prompt must hold across thousands of varied real inputs, survive model version updates, keep a stable output format for downstream parsing, and change safely as the product evolves. AppMatic Tech treats the two as different disciplines.
- Where should production prompts be stored?
- In version control, next to the code that calls them. AppMatic Tech changes prompts on a branch, reviews them against the regression test set, and merges them like code, keeping every previous version so a regression can be rolled back in minutes. Prompts stored as environment variables or database rows are fast to change and hard to roll back.
- How many test cases does a prompt regression set need?
- Twenty to fifty cases is enough for most production prompts, provided they cover the common case, the edge cases found during development, and inputs that look similar to the common case but need different handling. AppMatic Tech runs the set before every prompt change and after every model version update.
- What is shadow mode evaluation for an LLM migration?
- Shadow mode runs a new model version in parallel with the production version on a sample of live traffic, and compares the outputs before the migration completes. AppMatic Tech uses it when a model change must be validated against the real distribution of user inputs rather than a fixed test set.
- Should production prompts get longer as features are added?
- No. Prompts that grow with every feature become harder to test and more sensitive to model updates. AppMatic Tech changes production prompts only when the regression set shows a quality problem, a model update requires adaptation, or a new requirement genuinely needs new instructions.