What you need to do
The practical activities and techniques you need to master – what to practice, what to get good at, and what to keep doing.
Models
Understanding the landscape of machine learning approaches – not the math, but the mental models that let you choose the right tool. Regression models predict continuous values (price, time). Decision trees and random forests classify and predict based on feature splits – interpretable and effective for structured data. Large Language Models (LLMs) are statistical prediction machines that generate text by predicting the next token based on training patterns. They don't 'know' things – they generate plausible continuations. This explains both their strengths (fluent text, pattern matching, code completion) and limitations (hallucinations, unreliable math, no information after training cutoff). Temperature controls randomness: 0–0.2 for factual tasks, 0.8–1.0 for creative generation. Understanding model types helps you avoid the trap of using LLMs for everything when a simpler model would be faster, cheaper, and more reliable.
Prompt engineering
Writing effective prompts that consistently produce the output you need. Core techniques: be specific and explicit (describe the exact output format), provide context (who is the audience, what's the goal), use few-shot examples (show 2–3 examples of desired input/output pairs), chain-of-thought ('think step by step' for reasoning tasks), and system prompts (set the persona, constraints, and behavior rules). Structure complex prompts with clear sections: role, context, task, format, constraints. Break complex tasks into a sequence of LLM calls where each step's output feeds the next – a single prompt trying to do everything produces mediocre results.
Working with embeddings and RAG
Grounding AI responses in your own data instead of relying on the model's training. Embeddings convert text into numerical vectors that capture meaning – similar meanings produce similar vectors. Vector databases (pgvector, Pinecone, Weaviate) store these embeddings for fast similarity search. RAG (Retrieval-Augmented Generation) combines retrieval with generation: ingest documents, chunk them into passages, generate embeddings, store in a vector database. At query time, find similar passages and include them in the prompt as context. Start with 500–1000 token chunks with 100-token overlap. Use RAG when you need AI to answer questions about your specific data.
MCP
The Model Context Protocol – the open standard for connecting AI to external tools and data sources. Before MCP, every integration required a custom connector. MCP provides a universal protocol: AI applications connect to MCP servers that expose tools, resources, and prompts. Anthropic released MCP in November 2024; by 2025, OpenAI, Google DeepMind, and thousands of community-built servers had adopted it. In December 2025, Anthropic donated MCP to the Linux Foundation. MCP enables AI assistants to access databases, APIs, file systems, and development tools through a standardized interface – eliminating custom integration code for each data source.
Evals
Measuring and improving AI system quality for your specific use case – not relying on generic benchmarks. Evals are to AI systems what tests are to software: they define expected behavior and catch regressions. Three approaches: automated metrics (BLEU, ROUGE for text similarity – fast but crude), LLM-as-judge (use one model to evaluate another's output – flexible but requires careful prompt design), and human evaluation (gold standard but expensive). Build an eval suite specific to your use case: create a dataset of input-output pairs that represent your quality bar, run your system against them, and measure pass rates. Run evals on every prompt change or model upgrade.
Fine-tuning
Understanding when to fine-tune and when prompting is enough. Fine-tuning trains a model on your specific data to improve performance on your tasks. Worth considering when: prompt engineering has hit a ceiling, you need consistent behavior that prompting can't achieve, or you have enough domain-specific data. Not worth the investment when: you have fewer than 1,000 high-quality examples, prompting gets you 80%+ of the way, or the task changes frequently. The progression: prompting → few-shot examples → RAG → fine-tuning. Each step adds complexity and cost, so only escalate when the previous step is genuinely insufficient.
Frameworks to know
The mental models and structured approaches that guide how you think about this skill – not rules to follow, but lenses to see through.
RACE prompt framework
A structured approach to writing effective AI prompts with four components: Role (define who the AI should act as – 'You are a senior product manager'), Action (specify what you want done – 'Write acceptance criteria'), Context (provide background information – the feature description, user persona, constraints), and Expectation (define the output format and quality criteria – 'Use Given-When-Then format, cover happy path and 3 edge cases'). RACE ensures prompts are complete and reproducible. Most prompt failures come from missing one of these four components – especially Context and Expectation. Apply RACE to any AI interaction: coding assistants, content generation, analysis tasks, or brainstorming sessions.
Good vs. bad
Common mistakes paired with what experienced practitioners do instead – and why the product builder approach changes the game.
AI literacy is the meta-skill that's reshaping every other skill in the product builder toolkit. It's not a separate discipline – it's a capability multiplier. A product builder who understands prompt engineering writes PRDs faster. One who understands RAG builds better search features. One who understands evals ships AI features with confidence. The question isn't whether to learn AI – it's how quickly you can integrate it into everything you already do.
The biggest misconception about AI in product development is that you need deep ML expertise to build AI-powered features. You don't. What you need is practical literacy: understanding what models can and can't do, how to write effective prompts, how to evaluate quality, and how to design around failure modes. This is more like learning to use a database than learning to build one. The API does the hard part – your job is to use it well.
MCP is the most underappreciated development in AI infrastructure. Before MCP, every AI integration was custom plumbing – connect to Slack's API this way, GitHub's API that way, your database a third way. MCP standardizes all of it into a single protocol. It's USB-C for AI: one standard connector that works with everything. The product builders who understand MCP will build AI features 10× faster than those who don't, because they won't be reinventing integration patterns for every data source.
The progression I recommend for product builders learning AI: start by using Claude or ChatGPT daily for your own work (writing, research, code). Then learn prompt engineering – the techniques that turn mediocre outputs into excellent ones. Then build a simple RAG pipeline to ground AI in your product's data. Then add evals so you can ship with confidence. Each step takes a few weeks and compounds on the previous one. Within three months, you'll be building AI features that would have seemed impossible before.
From the blog
Articles covering this skill – practical guides, case studies, and deep dives.
Go deeper
Books to deepen your understanding beyond what fits on this page.
The most practical guide to understanding how LLMs work internally – not to build your own from scratch, but to develop the mental model that makes you a better AI product builder.
Comprehensive guide to building AI applications in production – covers prompt engineering, RAG, fine-tuning, evals, deployment, and the full lifecycle of AI-powered products.
Practical guide to prompt engineering techniques for developers – covers zero-shot, few-shot, chain-of-thought, and advanced patterns for building AI-powered applications.
How to design and deploy ML systems in production – covers data engineering, feature engineering, model deployment, and the infrastructure decisions that determine whether AI features succeed or fail.
