AI Engineering Buildcamp: From RAG to Agents

This course takes you from core concepts to production-grade AI systems through hands-on, project-focused modules.

1. LLMs & RAG

• Learn Large Language Models and Retrieval-Augmented Generation. Build conversational agents using the OpenAI SDK and create data-processing pipelines.
• Outcome: A RAG pipeline with real data.

2. Agentic Flows + MCP

• Add agentic behavior with function calling, using libraries like PydanticAI, and Agents SDK. Expose tools via MCP.
• Outcome: A capable, tool-using agents.

3. Testing & Evaluation

• Improve through testing and offline evaluation. Use LLMs as judges to compare approaches. Learn tools like Evidently and LangWatch.
• Outcome: A thoroughly tested and evaluated assistant.

4. Monitoring & Guardrails

• Use Grafana, Pydantic Logfire and OpenTelemetry for observability and safety.
• Outcome: Real-time monitoring.

5. Use Cases

• Create two agents: a website generator and a code reviewer. Learn about other use cases.

6. Capstone

• Build an end-to-end AI application with your data.
• Outcome: A portfolio-ready project.

Hackathon: Collaborate on real-world problems.

What you’ll learn

Create your own production-ready AI application in 6 weeks

Build a Fully Functional AI Assistant from Scratch

• A conversational AI assistant that answers questions from GitHub repositories, YouTube transcripts, or internal documentation.
• Use Retrieval-Augmented Generation and the OpenAI API.

Add Agentic Behavior to Your AI Systems

• Build systems that can reason, make decisions, and take actions with function calling.
• Use tools like PydanticAI and OpenAI’s Agent SDK.
• Extend the capabilities of your agent with MCP.

Use Testing and Evaluation to Improve Prompts and Results

• Test the application with unit tests and judges.
• Learn how to evaluate your application with ranking metrics, simulate user queries, and use LLMs to judge outputs.
• Select the best prompt, model and chunking strategy using the data-driven approach.

Monitor Your Application

• Set up real-world monitoring using Grafana, Pydantic Logfire, Evidently, and LangWatch.
• Track costs and token usage in real-time.
• Add guardrails to prevent the application misuse.

Build 8+ Projects

• FAQ Assistant, YouTube Video Q&A system, Wikipedia search and summary system, and a documentation agent.
• AI Coding agent, Deep Research agent and a Code Evaluator agent.
• Two more projects: capstone and hackathon at the end.

• Design and build your own end-to-end AI application from scratch.
• This could be anything from a resume reviewer to a podcast summarizer – fully tested, evaluated and monitored.

Who this course is for

• Data Scientists and ML Engineers proficient at coding who want to integrate AI into their projects
• Software Engineers curious about LLMs who want to build AI applications
• AI Enthusiasts who are stuck at the tutorial phase and want to create something end-to-end

Prerequisites

Coding
We will program a lot

Python, Git, Docker, command line
We will rely on these tools when building the assistant

OpenAI key or alternative
We will use OpenAI for building the AI agent

Homepage