Training your own AI model: why you almost never need to — and which of the four stages is enough instead

When you type "train your own AI model" into a search engine, you almost never actually want to train a model. You want two things: the AI should know your company's knowledge, and it should follow your rules. That is a procurement question, not a training question. And for this procurement question there are four stages, which differ in effort and cost by orders of magnitude.

I write this guide from the implementer's perspective: at happycoding we build AI features for mid-sized companies, among them RAG systems on pgvector and PostgreSQL, operated on EU infrastructure. The most common request that reaches us goes roughly: "We want an AI with our knowledge." The second most common: "What does our own model cost?" The same decision tree answers both questions, and that is exactly what you get here.

Up front, the radically honest core message: as a mid-sized company, you almost never need your own model. The model trained from scratch is the most expensive and slowest of the four stages, and in the vast majority of cases it does not even solve your problem.

The good news sits in the same sentence: the stage that actually solves your problem is faster and cheaper than you probably think. The difference between the right stage and the wrong one decides whether your project delivers in weeks or fails over months.

This article is the tree, not the leaves: for each stage you get the classification, the decision criteria, and the cost range with a source. The depth of the individual stages, such as the detailed comparison of RAG versus fine-tuning or the operational practice of self-hosting, I cover in separate companion articles. This is about the one question that comes before all others: which stage do you need in the first place?

This guide is written for managing directors and heads of IT who are responsible for an AI investment without building models themselves. You need no technical background: I define every technical term before I use it. And if you are in a hurry: the section "The decision tree" further down sums everything up in four questions. The sections before it deliver the reasoning and the numbers you can use to defend the answers internally.

For orientation, I will first put the four stages side by side. Each stage answers a different guiding question, and each stage plays in a different cost league.

Stage 1: structured prompting. You make better use of an existing model: with clean context, examples, and enforced output formats. Cost range: pure API usage, often two-digit to low three-digit euro amounts per month. Guiding question: is the model enough if I hand it everything it needs?

Stage 2: RAG over your data. The AI looks things up in your documents before every answer. Your knowledge stays in your database, the model stays unchanged. Cost range: a software project within the usual scope of custom development, no training budget. Guiding question: does the AI fail for lack of knowledge?

Stage 3: fine-tuning. You adjust an open model with your own examples, today practically always via LoRA on open-weight models. Cost range: the GPU costs of a training run are 10 to 20 USD; the real effort sits in data preparation and evaluation. Guiding question: does the AI fail on behavior, format, or style, even though the knowledge is there?

Stage 4: training your own model. You build a model from scratch. Cost range: three-digit million amounts for compute alone, plus teams that do not exist at mid-sized companies. Guiding question: honestly, none that ever arises for a mid-sized company.

Across these four stages lies a dimension that is often confused with them: where it all runs. Whether a model runs with a US provider, with an EU provider, or on your own server is not a fifth stage. It is a decision you can make at any of stages 1 to 3. More on that later, because exactly this cross-cutting dimension is missing from most guides on the topic.

Beyond cost, the stages differ in a second quantity that often matters more to decision-makers: the time to the first usable result. Stage 1 delivers in days, often in an afternoon. Stage 2 is a project of weeks to a few months, depending on the state of your documents.

Stage 3 depends almost entirely on how quickly you can assemble clean example data. Stage 4 is measured in years and teams. Working through the stages from the bottom up buys you not just the cheaper entry, but also the earlier reality check.

A note on how to read the numbers in this article: every cost figure carries its source and year, and volatile prices additionally carry the retrieval date. AI prices change fast, and a guide that gives you numbers without provenance is not a guide.

Let us start with the stage that needs no project: the structured use of existing models. It is chronically underestimated because it sounds unspectacular. Yet it handles an astonishing share of the tasks for which companies initially ask about "their own AI".

Three techniques make the difference between a toy and a tool:

Providing context. Modern models process very long inputs. If your request brings the relevant material along, say the product list, the style guidelines, or the draft contract, the model needs no "training": it simply reads the information at the moment of the request.

Few-shot examples. You show the model two to five examples of input and desired output, directly in the prompt. For many classification and rewriting tasks, that is enough: the model picks up patterns from examples more reliably than from abstract instructions.

Structured outputs. You force the model into a fixed output format, such as a JSON schema. That turns a chatbot into a system component whose answers your software can process directly.

What does this cost? For context, the list prices of the OpenAI API, retrieved August 18, 2026: GPT-5 costs 1.25 USD per million input tokens and 10.00 USD per million output tokens. The smaller gpt-5-mini sits at 0.25 and 2.00 USD, the even smaller gpt-5-nano at 0.05 and 0.40 USD. Between the largest and the smallest model class lies a factor of 25 on the input price: at this stage, model choice is your biggest cost lever, not prompt length.

An example makes the stage tangible: a company wants to sort incoming emails automatically into the categories order, complaint, invoice question, and other. The prompt contains the four categories with one definition each, five example emails with the correct assignment for each, and the instruction to return nothing but a JSON object with category and reasoning.

The smallest model class is often enough for this: at the nano prices above, classifying a thousand emails costs fractions of a euro. No training, no database, no project proposal: a carefully built prompt and an API call.

When is stage 1 enough? As long as the task needs no private knowledge beyond the context you supply. And as long as the model's behavior can be steered with instructions and examples. Typical cases from our practice: pre-sorting emails by request type, generating quote texts from bullet points, converting free text into database fields.

You recognize the limit by two signals. First: you are copying the same hundred pages of knowledge into every prompt, because otherwise the model does not know your content. Second: you are polishing the prompt for the twentieth time, because the form or tone of the answers will not fall into place. The first signal points to stage 2, the second to stage 3.

Now to the stage where most projects that start with "train our own model" end up: RAG, spelled out Retrieval-Augmented Generation. The principle in one sentence: before the model answers, your system retrieves the relevant passages from your documents and puts them in front of the model as context. The AI looks things up in your knowledge instead of memorizing it.

With that, RAG clears away the most common misunderstanding in this debate: no, with RAG the model does not permanently learn from your data. It is not trained on a single one of your documents. That is exactly the advantage, not the flaw.

Your knowledge sits in a database you control: you can update documents, delete them, and protect them with access rights, and the next answer reflects the new state. A model with knowledge trained into it can do none of this: it does not forget on command, it knows no access rights, and it goes stale with every passing day.

Here is how we implement it: your documents are split into sections and stored as embeddings, that is, as numeric vectors that capture meaning. As the vector database we use pgvector, an extension of PostgreSQL: your knowledge then sits in the same database as the rest of your application data, with the same backups and the same access rules.

We typically run all of this on Supabase or directly on EU servers. The point of these details is not pride in technology: it is that RAG is an ordinary software project. Database, pipeline, search, display: all of it craft, none of it a research lab.

When is RAG the right stage? When the AI fails for lack of knowledge: it does not know your products, your contracts, your internal processes. If that knowledge changes continuously, RAG is in fact the only sensible choice, because a document change is a database update and not a new training run.

There are two decision criteria you should check honestly before you start. First, the state of your data: RAG can only find what is findable. A knowledge base of outdated folder copies, scanned faxes, and three competing versions of the same price list will deliver contradictory answers even with the best search.

The clean-up work before the project is therefore part of the project. Second, traceability: a good RAG system shows the source passages for every answer. For a mid-sized company this is more than convenience, because an answer with a source citation is one your employees can verify; one without is one they must simply believe.

The line between RAG and fine-tuning, that is, the question of "look up knowledge or train in behavior", has so many facets that it deserves its own article: you find the detailed comparison in RAG vs. fine-tuning here on the blog. For the decision tree, the short form is enough:

On to the stage that attracts the most myths: fine-tuning, the further training of an existing model with your own examples. Up front, the classification: fine-tuning is the answer far less often than the term suggests. It does not answer the question "how does the AI get my knowledge?" but the question "how does the AI get my behavior?". Knowledge belongs in RAG. Fine-tuning wins in three concrete cases:

Format and style fidelity. The model must hit a very specific output form with absolute reliability: the structure of your expert reports, the tone of your correspondence, an in-house markup language. Instructions in the prompt often get you to 95 percent here; fine-tuning closes the gap.

Classification with a narrow output space. Thousands of inputs must be sorted into a fixed scheme, say support tickets into thirty categories. A small, fine-tuned model frequently beats a large general-purpose model here: in accuracy, speed, and price.

Latency and cost pressure at high volume. When a small open-weight model takes over the job of a large API model after fine-tuning, unit costs and response times drop. This only pays off from a substantial request volume onward, but then it pays off permanently.

Technically, fine-tuning today almost always means: LoRA or related PEFT methods on an open-weight model such as Llama, Mistral, Qwen, or Gemma. LoRA does not retrain the whole model; it lays small additional matrices over the existing weights: the compute effort drops by orders of magnitude as a result.

How many examples you need depends on the case: by common practical experience, a few hundred curated example pairs are often enough for style and format adjustments, while narrower specialist tasks tend to need one thousand to five thousand. Quality matters more than quantity: a contradictory dataset will reliably train the contradictions right in.

Just as important as the winning cases are the losing ones. Fine-tuning loses when you want to use it to store knowledge: facts that change belong in RAG, because a state trained in once goes stale and cannot be selectively corrected.

It also loses when your example data is thin or inconsistent: then the training run amplifies the noise instead of the pattern. And it always loses when a better prompt would have produced the same result: this check is uncomfortable, because it makes the smaller project the right answer, but that is exactly what question 1 in the decision tree is for.

Now the calculation that surprises many. A rented H100 GPU with 80 GB costs 1.99 to 2.89 USD per hour depending on the cloud tier; an A100 with 80 GB sits at 1.19 to 1.59 USD (Runpod price list, retrieved August 18, 2026). A LoRA run on a model in the 7-to-8-billion-parameter class typically needs a few hours on a single H100. Do the math yourself: four to six hours times roughly two to three USD comes to about 10 to 20 USD in pure GPU costs per training iteration.

So in fine-tuning, compute is the cheapest line item of the whole calculation. What is expensive are the person-days before and after: collecting, cleaning, and curating example data, and then measuring properly whether the fine-tuned model is actually better than the base model with a good prompt. Whoever skips this evaluation is not doing fine-tuning; they are doing hope.

At this point, an aside that matters more for your planning than any technical question: if you knew fine-tuning as an "I upload my data to OpenAI" service, that model is on its way out. OpenAI is winding down its own self-serve fine-tuning platform. The timeline is in the official deprecation overview (developers.openai.com, retrieved August 18, 2026) and reads like this:

Since May 7, 2026: organizations that have not previously used fine-tuning no longer get access.

Since July 2, 2026: organizations that have not run inference on one of their fine-tuned models in the preceding 60 days can no longer create new training jobs.

From January 6, 2027: even active existing customers can no longer start new fine-tuning jobs. Existing fine-tuned models keep running, but only until the respective base model itself is retired.

OpenAI names no successor product. What follows from this? Whoever builds a fine-tuning strategy on a hosted platform today is building on a platform whose largest provider is in the middle of a retreat. That is not a side note; it is an architecture argument: fine-tuning today means open-weight models whose weights you own and whose trained state nobody can switch off on you. This is exactly why I describe stage 3 above as LoRA on open models and not as a button in an API dashboard.

For you as a decision-maker, this is at its core good news: the open variant is not only the future-proof one; with rented GPU hours it is also transparently calculable. You trade the convenience of a platform for ownership of your most important project outcome: the trained model state together with the curated training data.

This yields a test question for every offer that promises you fine-tuning: who owns the weights afterwards, and on which platform is the trained state created? If the answer is "on a provider's hosted platform", ask what happens to your investment when that provider discontinues the product. The OpenAI timeline above is the proof that this question is not theoretical: it has a date.

Now to the stage this article is named after: training a model from scratch, in the jargon pre-training. Here I will simply let the numbers speak, and sourced ones at that.

The Stanford AI Index Report 2024 puts figures on the compute costs of large training runs, based on estimates by the research institute Epoch AI. Three data points draw the curve: training the original Transformer of 2017, the architecture behind today's model generation, cost around 930 USD.

Training GPT-4 cost around 78.4 million USD. Training Gemini Ultra cost around 191.4 million USD. From three digits to nine digits in seven years: that is the price trajectory at the frontier, for compute alone, without salaries, without data, without the many runs that fail.

A fourth data point settles the matter for mid-sized companies once and for all. The AI Index Report 2025 estimates, again based on Epoch AI, the training costs of Llama 3.1 405B at around 170 million USD. The remarkable part: Llama 3.1 is an open-weight model. Meta invested 170 million USD in a training run and hands you the finished weights free of charge. You can download the result of a nine-digit investment, run it on European servers, and adapt it to your case via LoRA.

Add to this what the compute estimates do not even include. A pre-training venture needs a research team you would have to recruit against the salaries of the AI labs. It needs training data at a scale and quality no mid-sized company possesses. And it needs the willingness to write off failed runs, because nobody gets it right on the first attempt. The published numbers each price one successful run: the total bill of a program sits above that.

With that, the guiding question of stage 4 answers itself. What business case could justify a mid-sized company investing nine digits in its own training run, whose result would lag behind freely available models? I have never encountered one in practice. There are niches for specialized pre-training, say in pharmaceutical research or in states with sovereignty programs. For a mid-sized company whose goal is "AI with our knowledge and our rules", stage 4 is not an option; it is a category error.

Now to the topic that has only been hinted at across the four stages: where does all of this run? Many guides mix this question into the stage question, yet it is independent of it. Whether you prompt, run RAG, or fine-tune: you can run each of these stages against a US API, against an EU provider, or with a self-hosted open-weight model on your own infrastructure. The stage determines what happens to the model. The cross-cutting dimension determines who sees your data while it happens.

For many mid-sized companies, this dimension is the real trigger of the search for "their own model": it is not about training at all, but about data sovereignty. Patient data, engineering data, client records: there are datasets a company will not or must not entrust to a US service.

The answer to that, however, is not stage 4 but the cross-cutting dimension: run an open model on European infrastructure and drive any of stages 1 to 3 on top of it. What to watch for in terms of data protection, I have described in the practical guide to GDPR-compliant AI.

The cross-cutting dimension has more than two settings. Between "US API" and "your own server" lie gradations: European model providers with API access, EU compute zones of large cloud providers with contractual commitments, managed GPU instances at European hosters. Which gradation you need is determined by your data and your legal counsel, not by the technology. What matters for the decision is only this: you must answer this question separately from the stage question, because every combination can be built.

What does self-hosting cost? A concrete anchor: with the GEX131, Hetzner offers a dedicated GPU server with an RTX PRO 6000 Blackwell Max-Q and 96 GB of graphics memory for 1,197.30 euros net per month plus a 599 euro setup fee, following the June 15, 2026 price adjustment (Hetzner price list, retrieved August 18, 2026). A card like that runs quantized open-weight models of medium size at production-usable speed.

Whether this beats the API is not a matter of belief but a formula: self-hosting pays off when your monthly API costs, that is, your token volume times the price per million tokens, permanently exceed server rent plus operating effort. With strongly fluctuating load, the API usually wins; with constantly high load or hard data sovereignty requirements, your own hosting does.

Calculate with your numbers, not with mine: both sides of the price change constantly, which is why I deliberately give you a formula here instead of a break-even figure.

Everything beyond that, from model choice to operations, belongs in its own article: you find it in the guide to self-hosting LLMs here on the blog. For the decision tree, this is enough: EU self-hosting is not a fifth stage but a slider you can set at every stage.

One aspect is missing from practically every guide on model training, and it can get more expensive than any GPU bill: the question of when you legally become a model provider yourself. The European AI Act has its own set of duties for providers of general-purpose AI models, GPAI for short: technical documentation, transparency about training data, copyright safeguards. Whoever merely uses a finished model does not carry these duties. But whoever substantially modifies a model can grow into the provider role.

Where the line sits is currently only sketched indicatively. In its GPAI guidelines of July 18, 2025, the European Commission names a compute criterion as orientation: whoever spends more than one third of the original training compute on modifying a model is, by that reading, to count as the provider of the modified model.

The exact labeling of this figure matters: it is an indicative figure from the European Commission's non-binding GPAI guidelines of July 18, 2025, not legally binding. I deliberately only touch on the question here and refrain from any legal interpretation of my own: you find the classification of the AI Act for software buyers in my article on the EU AI Act for software buyers, and binding answers for your individual case come from a law firm.

Why do I mention the question at all, if it will hardly ever apply to you? For two reasons. First, it belongs in every serious consultation on stage 3, because "can this trigger provider duties for me?" is a legitimate concern you should not first hear from a competitor. Second, it shows a pattern that confirms the entire decision tree: the higher the stage, the more responsibility migrates over to you, technically as well as legally.

For the decision tree, one reassuring observation remains: a LoRA fine-tune consumes a tiny fraction of the original training compute and thus sits far below this indicative threshold. The legal question is therefore no reason to forgo stage 3. It is a reason to document it deliberately: whoever records what they changed on the model and with what effort can later answer the provider-role question with facts instead of a shrug.

Before we walk through the questions of the decision tree, let me clear up three misconceptions we regularly encounter in first conversations. Each of them sends projects to a more expensive stage than they need, or to a cheaper one than they can bear.

"But the AI learns from our data as we use it." At stages 1 and 2: no. An API call and a RAG query do not change the model; your knowledge is read, not learned.

This is easily confused with the question of whether a provider may use inputs to improve its models: that is a contractual question you settle in the data processing terms, not a property of the technology. Whoever jumps straight to their own model out of fear of "learning along" is solving a contract problem with a millions budget.

"A trained model is the most secure form of data sovereignty." If anything, the opposite is true. Information trained into a model knows no access rights: a fine-tuned model can blurt out everything it has ever learned to any user, the salary list to the intern included. A RAG system, by contrast, checks before every answer which documents the requesting user may see. Data sovereignty comes from where the system runs and from access control, not from training.

"Fine-tuning teaches the model our company knowledge." The classic. Fine-tuning shapes behavior: form, tone, classification sharpness. As a knowledge store it is unreliable and unmaintainable, because facts can be neither selectively updated nor selectively deleted. Knowledge belongs in the database of stage 2, behavior in the example data of stage 3. Whoever mixes the two gets a model that confidently quotes outdated prices: in the right tone, with the wrong content.

Now I will bring the stages together into the actual tool: four guiding questions that lead you through the stages in order of cost, from the bottom up. Go through them in sequence, and stop at the first stage that solves your problem.

A word on application before the questions come: ask them per use case, not per company. "We are introducing AI" is not an answerable question; "our support team should get answers with source citations from the manuals" is. If you have three use cases, you run the tree three times, and it is completely normal to land on different stages along the way.

Question 1: does an existing model solve the task if you hand it everything it needs? Try it seriously: with clean context, few-shot examples, and structured output. If yes, you are done, and you are done at stage 1. This question gets skipped surprisingly often, because an API call does not look like enough of a project.

Question 2: does the model fail for lack of knowledge? It does not know your products, contracts, or processes, and this knowledge will not permanently fit into every prompt? Then RAG is your stage: the knowledge goes into a searchable database, the model stays untouched. That is the default case for mid-sized companies.

Question 3: does the model fail on behavior, format, or style, even though the knowledge is available? The answers are factually correct but off in form, tone, or classification sharpness, and prompt examples will not close the gap? Then, and only then, fine-tuning is up: as LoRA on an open-weight model, with clean evaluation against the base model.

Question 4: must the processing happen under your data sovereignty, without exception? You ask this question not after the others but alongside them: it does not decide the stage, it decides where things run. If the answer is yes, you run the chosen stage with an open model on EU infrastructure.

And stage 4, training from scratch? It does not appear in the tree as a question, because no realistic answer corresponds to it in the mid-sized world: the numbers from the section before last practically rule it out, and the freely available open-weight models take away its last remaining reason.

The tree gives you one stage per problem, but real systems often combine several stages, because they solve several problems. Three patterns come up in practice again and again, and all three stay well below stage 4.

RAG plus structured prompting. The standard pattern: the search of stage 2 delivers the source passages, the techniques of stage 1 shape them into the answer, say a short assessment with a source list in a fixed JSON format. Almost every knowledge system we build is exactly this combination. It also shows why stage 1 is never wasted work: your prompts live on inside the RAG system.

RAG plus a fine-tuned small model. When a knowledge system reaches very high request volume, a small model fine-tuned via LoRA can take over the answer shaping from the large API model: the knowledge keeps coming from the database, the behavior from the fine-tuning, and unit costs drop. This pattern is the most common serious fine-tuning case we see, and it emerges as an upgrade of a running RAG system, not as a first project.

Any stage plus EU operation. The cross-cutting dimension lays itself over both patterns: the same RAG system runs either against an API or against a self-hosted open-weight model on European servers. Built cleanly, that is a configuration switch, not a rebuild. This is why it pays to ask the data sovereignty question early: not because it breaks the architecture, but so the architecture provides for the switch from the start.

The patterns show the principle of the whole article in miniature: you build upward from cheap, replaceable building blocks instead of starting with the most expensive one.

Before you take the decision tree into budget planning, I want to straighten out a distortion that shapes almost all cost discussions: the public debate talks about GPU prices as if they were the core of the bill. For stages 1 to 3 they are not. I worked out the 10 to 20 USD GPU costs of a LoRA run above, and a RAG system likewise produces monthly compute costs closer to a server budget than to a project budget.

Run the proportions for comparison: a single consultant day costs more than fifty LoRA training runs at 20 USD each. Whoever negotiates an AI project while haggling over GPU prices is negotiating the smallest line item of the bill. The right question to ask any provider is instead: how many person-days go into data preparation and evaluation, and who delivers them?

The real cost block is person-days, in three places. Data preparation: reviewing documents, sorting out duplicates and outdated versions, clarifying access rights, and for fine-tuning additionally curating example pairs. In our project practice this is regularly the largest single item, and it cannot be automated away, because it demands domain knowledge only your people have.

Evaluation: a fixed set of test questions with target answers, against which every change is measured. Without this yardstick you cannot know whether a change was an improvement or a step backward dressed up as one.

Operations and upkeep: knowledge bases age, models get discontinued, prompts need adjusting. AI features are software, and software lives.

This honest calculation has a pleasant consequence for your negotiating position: because compute is cheap, the stages are permeable upward. Whoever starts with RAG and later discovers a genuine fine-tuning case has lost nothing: the curated data and the evaluation questions from the RAG project are exactly the groundwork the fine-tuning needs. The decision tree is therefore not a one-way street but a sequence: you start at the cheapest stage that could solve your problem, and you climb only with measurements in hand.

To close the classification, a look into practice: what does this actually look like at our end when a company approaches us wanting "its own model"? A typical case: a mid-sized company with twenty years of quotes, manuals, and service reports that wants to make this knowledge usable for sales and support.

The request says "our own model"; the diagnosis after the decision tree is almost always stage 2: a RAG system over the document base, with clean access rights and a source citation for every answer. Sometimes a small fine-tuning component is added later, say for an in-house report format. A training run from scratch has not come out of a single project yet.

What matters to me is the order: first the use case, then the stage, then the technology. An AI project that starts with the technology choice optimizes the wrong variable. How we embed AI features in custom software, from requirements to operations, is described on our custom software development page. If your case leans more toward recurring processes, say document intake, data transfer, or reporting, you find the right entry point under process automation.

And if your actual question is how much decision-making latitude you want to hand over to an AI: that is not a model procurement question but a governance question, which I cover in the guide to agentic AI.

One more note for the budget: in Germany, regional funding programs subsidize digitalization projects of this kind, depending on the federal state. A RAG project at stage 2 is exactly the order of magnitude where such programs have a noticeable effect: the right stage makes your project calculable not just technically, but also in funding terms.

If you are standing in front of this decision right now, I suggest a simple path. First: formulate the one use case that causes the most pain, in two sentences. Second: go through the four questions of the decision tree and note at which stage you get stuck and whether the data sovereignty question is mandatory for you. Third: check which data for that stage already exists, because that is where the real effort arises.

With these three notes you are better prepared than most project inquiries that reach us: you know what you want, which stage it takes, and where the work sits.

If you want a second opinion along the way, talk to us: we walk through the decision tree together with your concrete case, name the right stage, and will honestly tell you that stage 1 is enough when stage 1 is enough. That is not sales rhetoric but self-interest: projects on the right stage get finished, and finished projects are the only reference that counts. Book a free initial consultation: 30 minutes, no sales pressure, with a clear recommendation at the end.