AI Engineering for Backend Developers: Modules
The program teaches a backend engineer to build AI features with the same engineering discipline as any service: LLM API integration, schema-validated tool calling, RAG, evaluation and regression tests, prompt injection defense, privacy, and cost and latency budgets. The focus is reliable backend integration, not model research.
01
LLM API integration
Call a model API as reliably as any other external dependency.
- Request/response structure and tokens
- Timeouts, retries, rate limits and circuit breakers
- Streaming responses and backpressure
- Abstraction to allow changing providers
- Handling non-deterministic responses
Lab: An LLM client adapter in a Spring Boot service with retries, timeouts and an error mapping.
02
Tool/function calling and schema validation
Let a model invoke backend operations while keeping control.
- Tool description and JSON Schema
- Validating and rejecting model-supplied arguments
- Authorisation: the user's rights, not the model's
- Side-effecting operations and a confirmation step
- Multi-step tool loops and limits
Lab: A simple agent with order status and refund tools, with schema validation and authorisation checks.
03
Retrieval-augmented generation
Ground answers in the organisation's own data and cite sources.
- Chunking strategies and the size trade-off
- Embeddings and vector search
- Hybrid search and re-ranking
- Grounding and source citation
- Knowing when not to answer: not enough context
Lab: RAG over a document set: indexing with PostgreSQL pgvector, retrieval and an answer that cites sources.
04
Evaluation and regression tests
Measure AI feature quality with repeatable checks, not impressions.
- Golden datasets and scenario selection
- Measuring retrieval and generation quality separately
- Rule-based checks and the limits of model-as-judge
- Regression checks in CI
- Effect of prompt and model changes
Lab: A 30-example evaluation set for the RAG lab and a regression test running in CI.
05
Guardrails and prompt injection defense
Stop untrusted text from changing the system's behaviour.
- Direct and indirect prompt injection
- Validating input and output
- Least privilege for tools
- Output sanitisation and rendering risk
- Checking against the OWASP LLM risk list
Lab: Add a malicious document to the RAG lab, reproduce the attack and test the defenses.
06
Privacy and data handling
Stay in control of what is sent to a model and what is stored.
- Masking personal data
- Retention periods and log policy
- Provider terms and data residency questions
- Audit trail and traceability
- User consent and where legal advice belongs
Lab: A filter that masks personal fields in input and an audit log design.
07
Cost and latency budgets
Define the limits that make an AI feature sustainable in production.
- Calculating token cost
- Caching: exact and semantic
- Routing to a smaller model and fallback
- Asynchronous processing and queues
- Per-user limits
Lab: A cost and latency model: a table of assumptions and a limit mechanism.
08
Observability of AI features
See what answered, why, and at what cost.
- Traces: request, retrieval, model call, tool
- Metrics: latency, errors, tokens, refusal rate
- Balancing prompt and response logging with privacy
- Quality signals and user feedback
- Alerting and the SLO concept
Lab: Observe a RAG service with OpenTelemetry traces and a Grafana dashboard.