Adapting and serving models
Adaptation and serving are different problems.
What you will learn
- Explain: instructions, knowledge and adaptation
- Apply the idea in an example: Adapting and serving models
- Recognize limitations and verify the exercise outcome
Adaptation and serving are different problems. One changes behavior; the other makes a model available in a system.
This is a conceptual or external lab. It does not assume the application exposes every control described.
How it works, step by step
Instructions, knowledge and adaptation
A prompt changes request instructions and examples. RAG changes available evidence without modifying the model. Fine-tuning adjusts parameters for repeated behaviors or tasks and needs quality data and evaluation. LoRA/PEFT trains a smaller parameter set for efficient adaptation. Distillation transfers behavior from one model to another; quantization reduces numeric precision for efficient inference, with tradeoffs. None automatically repairs wrong sources. For frequently changing rules, updating documents is usually easier to verify than retraining.
Choose by task and capabilities
Check language, context, tool calling, structured output and supported input modalities. Compare models on identical requests using the same rubric and a realistic budget. Temperature changes sampling, not truth. Reasoning effort allocates effort for compatible models and can increase time and cost; accepted values vary. Local models offer infrastructure control but require resources and maintenance. Open weights do not guarantee an unrestricted license or complete training code. Catalogs and prices change; read the current model card.
Cost belongs to the whole path
Count model calls, input and output tokens, embeddings, reranking, vision and tools. Reasoning can consume billable tokens without an equally long visible response. Indexing costs and conversation costs differ. Use current prices rather than a permanent number from a course. A simple estimate is tokens/1,000,000 × price plus additional operations; application credits may have their own conversion. Compare cost per successful task, not just per call. Parallel execution can reduce latency while increasing total consumption. A budget needs a verified stopping condition.
The visual map
Follow the solid arrows for the main flow. Dashed blue arrows supply data or context; dashed pink arrows show feedback or returning results. Colors and shapes distinguish models, stores, decisions and outputs. On smaller screens, scroll horizontally to follow the entire diagram.
A complete example
LoRA can adapt style using clean data. Quantization may reduce serving memory but quality needs checking. Distillation requires evaluating the smaller model on the same tasks.
Try it yourself
- Build a checklist for data, license, memory, latency and quality.
- Record the input, source and expected outcome before running the experiment. Use only the fictional data in the example.
- Follow the diagram stages. At every step record what information is received and produced; do not confuse intermediate output with the final outcome.
- Repeat after removing necessary information or making the input ambiguous. Check whether the system clarifies, stops or invents an answer.
- Compare with the explained solution. Keep the configuration, date, result and an explanation for differences. Change one thing and retest.
An explained solution
Compare baseline, adaptation and serving optimization separately using unseen test data. A successful exercise lets you show the connection between input, stages and outcome. When information is missing, a cautious answer is more useful than invented details. Compare more than style: check conditions, sources and operations too.
When it helps and what can go wrong
Local serving also requires security, updates and hardware capacity. Choose this approach when it improves a measured need. Keep a simple baseline and compare outcomes using identical inputs. One successful example does not establish reliability in every situation.
Check your understanding
Is fine-tuning the first choice for current stock?
No. Use an authorized live source for rapidly changing information.
Does zero temperature guarantee correctness?
No. It may reduce variation, but a stable answer can still be wrong.
What outcome should this exercise produce?
Compare baseline, adaptation and serving optimization separately using unseen test data.
Words to remember
- Fine-tuning: Adapting model parameters using task-specific examples.
- Reasoning effort: A model-dependent setting controlling reasoning effort.
- Latență / Latency: Time until a useful result.
Sources and your next step
To prepare: X03 — Generative AI for images, audio and video