Chapter 10 — Shipping: Building the Portfolio That Gets the Job

You've built real systems. You know how transformers predict tokens, how to engineer prompts precisely, how to give models tools and knowledge, how to orchestrate multi-agent workflows, how to measure quality rigorously, how to defend against adversarial inputs, and how to run all of it at production scale. That's a substantial, rare skill set. The engineers who have it and can demonstrate it are being hired aggressively, at senior or staff levels, by companies building AI-native products. But demonstrating it is not automatic. Your skills are invisible until they're expressed as shipped, observable work. This chapter is about converting everything you've built into the evidence that makes your capabilities legible to employers and users alike.

Part 1: What a Hiring-Ready AI Portfolio Actually Looks Like

Most engineers with strong AI skills have one of two portfolio problems: either they have no projects at all (they learned in isolation, nothing is public), or they have a graveyard of half-finished Jupyter notebooks that demonstrate learning but not engineering. Neither converts.

The right signal for a senior AI engineer portfolio is: complete, deployed, measured systems with real trade-off reasoning documented. Not "I implemented RAG." "I implemented RAG over a 2M-document legal corpus, measured faithfulness at 0.91 after implementing hybrid search + re-ranking, and documented why I chose Qdrant over pgvector for this specific use case."

The distinction is tradeoffs + numbers. Any engineer can implement RAG by following a tutorial. A senior engineer makes architectural choices, understands why they chose what they chose, and has quantified whether it worked. That's the evidence hiring managers are looking for.

The Four Portfolio Artifacts

Every project you want to represent in a portfolio needs exactly four things:

1. A clean, navigable repository. Not clean code in the "perfectly refactored" sense. Clean in the "can a stranger understand this in 5 minutes" sense. Good README. Logical directory structure. No dead code, no commented-out experiments, no test2_final_v3.py.

2. A README that tells the story. See Part 2.

3. A 90-second demo. A screen recording (Loom works) that shows the system working on a real input, including at least one interesting edge case or failure mode you've thought about. Narrate what you're showing and why the design decision it demonstrates is interesting. No music, no titles, no editing — just clear narration over a working demo.

4. A trace screenshot. One screenshot from your observability stack (LangSmith, Braintrust, your own traces) showing a real agent run or RAG retrieval with the intermediate steps visible. This demonstrates you actually instrumented the thing — not a screenshot of the UI, a screenshot of the internals.

If a project doesn't have all four, it's not portfolio-ready. It's a work in progress.

Part 2: The README Structure That Converts

The README is your technical writing sample and your system design document in one. It's the first thing a hiring manager, a recruiter, and a technical interviewer looks at. The version that gets you an interview is not the version with the most features listed.

Here is the structure that works:

Section 1: The Problem (2-3 sentences)

What problem does this solve, for whom, and why is it non-trivial? Not "this is a RAG system" but "legal researchers at mid-size law firms spend 4-6 hours per case searching precedents across 10 different sources. This system retrieves and synthesizes relevant case law in under 30 seconds."

Section 2: Architecture (diagram + 2-3 paragraphs)

Show the system architecture as an ASCII diagram or a simple image. Then explain: what are the main components, how do they connect, and — critically — what are the key design decisions?

The design decisions paragraph is where senior engineers are separated from junior ones. "I used Qdrant because it supports payload filtering, which I needed to scope retrieval to documents the user has access to" is better than "I used a vector database." One shows thinking; the other shows familiarity.

System Architecture:

User Query
    │
    ▼
Query Rewriter (claude-haiku) ──→ Hybrid Search (Qdrant + Elasticsearch)
                                           │
                                   Cross-encoder Reranker
                                           │
                                  Context Constructor
                                           │
                                 Final Synthesis (claude-opus)
                                           │
                                    Streaming Response

Section 3: Trade-offs Made

Two or three trade-offs you considered and how you resolved them. This is the highest-signal section.

Examples:

• "I chose parent-child chunking over fixed-size chunking because my corpus is technical documentation where context spans multiple paragraphs. Fixed-size chunking produced a 12% drop in faithfulness score on my eval set."
• "I deliberately didn't implement multi-agent coordination for this use case — the task structure is predictable enough that a deterministic pipeline with three model calls outperforms an agent loop in both cost and reliability."
• "I used pgvector instead of a dedicated vector database because operational simplicity mattered more than query latency at my scale (<100k docs). I would switch to Qdrant if the corpus grew past 500k documents."

Section 4: Eval Numbers

The numbers that prove it works. This is the differentiator that almost no portfolio projects include.

Format: Metric | Value | Method | What would change it

Metric                  | Value | Method              | Notes
------------------------|-------|---------------------|----------------------------
Context Relevance (C|Q) | 0.91  | LLM-as-judge        | Drop below 0.85 → fix chunking
Answer Faithfulness(A|C)| 0.94  | LLM-as-judge        | -
Answer Relevance (A|Q)  | 0.88  | LLM-as-judge        | -
End-to-end correctness  | 0.82  | Ground-truth dataset| 200 hand-labeled QA pairs
P95 Latency             | 4.2s  | Production traces   | -
Cost per query          | $0.003| Traced              | Target <$0.005 at scale

Hiring managers look at this section and understand immediately: this engineer didn't just build it, they measured whether it works.

Section 5: Local Setup

Exactly enough to run it locally. Three commands maximum, or it's too complex. Document the environment variables needed. Link to the eval dataset if it's public.

Part 3: Building a Real UI — Product-Grade vs. Internal Tool

A working API is not a product. Users interact with interfaces. Whether your project gets used (and therefore demonstrates real-world impact) depends on whether there's something to click on.

Two tiers:

Internal tool (Streamlit): For projects that will be used by you or a small team. Streamlit lets you build a functional UI in ~50 lines of Python — no JavaScript, no deployment complexity. Good enough to demo, fast to build, easy to iterate on. Not appropriate for end-user products.

# Streamlit RAG app — complete in ~50 lines
import streamlit as st
from rag_pipeline import RAGPipeline

st.title("Legal Precedent Search")
rag = RAGPipeline()

query = st.text_input("Enter your legal research question")

if query:
    with st.spinner("Searching precedents..."):
        result = rag.query(query)
    
    st.subheader("Answer")
    st.write(result.answer)
    
    with st.expander("Sources used"):
        for chunk in result.sources:
            st.write(f"**{chunk.document_title}** (relevance: {chunk.score:.2f})")
            st.write(chunk.text)
    
    with st.expander("Eval scores"):
        st.metric("Faithfulness", f"{result.eval_scores.faithfulness:.2f}")
        st.metric("Relevance", f"{result.eval_scores.answer_relevance:.2f}")

Product-grade (Vercel AI SDK + Next.js): For projects you want to present as production SaaS features. The Vercel AI SDK provides hooks for streaming, tool use state, and chat UI that handle the hard parts of AI-powered frontend development.

// Vercel AI SDK streaming chat component
'use client';
import { useChat } from 'ai/react';

export function RAGChat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: '/api/chat',
    onError: (error) => console.error(error),
  });

  return (
    <div className="flex flex-col h-screen">
      <div className="flex-1 overflow-y-auto p-4 space-y-4">
        {messages.map((message) => (
          <div key={message.id} className={message.role === 'user' ? 'text-right' : 'text-left'}>
            <div className={`inline-block p-3 rounded-lg ${
              message.role === 'user' ? 'bg-blue-500 text-white' : 'bg-gray-100'
            }`}>
              {message.content}
            </div>
          </div>
        ))}
        {isLoading && <div className="text-gray-400">Thinking...</div>}
      </div>
      
      <form onSubmit={handleSubmit} className="p-4 border-t">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask a question..."
          className="w-full p-2 border rounded"
        />
      </form>
    </div>
  );
}

The Vercel AI SDK handles streaming automatically — the useChat hook manages the SSE connection, accumulates streamed tokens, and re-renders the UI with each new token. You get a ChatGPT-like experience in 30 lines of React.

The deployment decision: An undeployed project is a local project. Deploying doesn't have to be complex — Vercel deploys Next.js apps in two minutes from GitHub. Railway or Render deploy Python backends in a few minutes. The value of deployment is that you can share a URL, not just a repository. A URL that demos the system live converts far better in a portfolio than a video of the system running locally.

Part 4: Deployment — Making It Accessible

Deployment for a portfolio project has one requirement: the URL must work when a technical interviewer visits it during the interview.

This rules out: your laptop, any server that you'd have to manually start, and any deployment that requires the interviewer to create accounts.

Simple deployment options:

Vercel (frontend) + Railway (backend): Vercel auto-deploys from GitHub on push, handles edge caching, serves SSR. Railway deploys any Python/Node application in a container, provides a Postgres/Redis addon, and gives you an HTTP endpoint in minutes. Together they handle 95% of AI application deployment cases.

Hugging Face Spaces: For Gradio or Streamlit applications, Hugging Face Spaces provides free hosting with a public URL. Perfect for internal tool-style demos.

Modal: For GPU-intensive or batch processing workloads. Serverless Python deployment that auto-scales to zero (no cost when idle) and can run GPU workloads when needed.

What you don't need for a portfolio project: Kubernetes, custom VPCs, load balancers, auto-scaling groups. These are production infrastructure concerns for real traffic. Portfolio projects need to demonstrate AI engineering, not infrastructure engineering.

The only infrastructure requirement: your deployment should handle at least 10 concurrent users without falling over. This is achievable with any of the options above without configuration work.

Part 5: Writing About Your Trade-offs — Making Tacit Decisions Legible

The most differentiated skill in an AI engineering portfolio isn't the implementation — it's the ability to explain why. Every senior AI engineer makes dozens of architectural decisions per project. Most of these decisions live inside the engineer's head as pattern matching and intuition. Converting that to written explanation is a separate skill, and it's one that compounds over time.

The format that works: decision + alternative considered + reason for choice + signal that would change the decision.

Bad: "I used hybrid search."

Good: "I used hybrid search (vector + BM25) rather than pure vector search because this corpus has significant exact-match query patterns — users often search for specific regulation numbers (e.g., 'GDPR Article 17') where BM25 substantially outperforms semantic similarity. My evaluation showed hybrid search improved context precision from 0.71 to 0.84 on my regulatory query subset. If the corpus shifted to purely conceptual queries without exact-match patterns, pure vector search would be simpler to operate for comparable quality."

This paragraph demonstrates: you know what hybrid search is and why it exists (Chapter 4), you evaluated it rather than just implementing it (Chapter 7), you understand when it would stop being the right choice, and you can communicate all of this clearly. That's four dimensions of competence expressed in four sentences.

The engineering blog post is the highest-leverage vehicle for this type of writing. A post titled "How I improved RAG faithfulness from 0.74 to 0.94 by switching to parent-child chunking" or "Why I stopped using agents for this use case" is more visible, more shareable, and demonstrates more reasoning depth than a repository README alone. Engineers who write about their work are perceived as more senior than engineers who don't, even when the underlying technical work is identical.

Part 6: Capstone 3 — Production SaaS Feature

The three capstones form a progression: Capstone 1 (Chapter 4/7) is a standalone retrieval system with rigorous evaluation. Capstone 2 (Chapters 5-6/8-9) is a multi-agent workflow with security hardening and production infrastructure. Capstone 3 combines both into a production SaaS feature that a real user could use.

The scope: pick a real problem you or someone you know has. Build a feature that solves it and could reasonably be part of a SaaS product. Requirements:

Real UI. Vercel AI SDK + Next.js. Streaming. At least one agent interaction visible to the user (not just a response box — show intermediate steps, tool calls, or reasoning).

Real RAG. At minimum hybrid search + re-ranking. Connected to a real document corpus (your own documents, public data, or a data source relevant to the problem). Not a toy 10-document collection.

Eval numbers. At minimum faithfulness and answer relevance, measured against a 50+ case ground-truth dataset, displayed somewhere accessible (in the README, in an eval dashboard, anywhere a reviewer can find them).

Cost attribution. Tracked. Reported in the README as "cost per query: $X at current scale."

Deployment. Live URL. Works for an interviewer who visits it right now.

The specific topic is less important than the execution. Ideas that work well:

• A "second brain" tool that answers questions about your own past writing, notes, and documents
• A codebase assistant for a specific framework or technology you work with daily
• A research assistant for a domain you know deeply (legal, medical, financial — pick one)
• An internal tool for a workflow you or your team currently does manually
• A monitoring assistant that analyzes logs and surfaces anomalies with explanations

One constraint: it must be a tool you will actually use. Projects built for the portfolio that serve no real purpose are detectable in interviews. Projects built to solve a real problem produce genuine opinions about trade-offs.

Part 7: The Job Search Layer — Translating Portfolio to Interview

Your portfolio has three jobs: to get you the initial call, to give you things to talk about in technical screens, and to give you credibility before the system design round.

Getting the Initial Call

Applications without warm introductions have a low hit rate in AI roles at the moment — there's high demand and the hiring funnel has a lot of noise. The highest-leverage path is being visible in the community before you apply: writing, open source contributions, or public projects that have gotten attention.

If you can't rely on inbound interest, the next best approach is a direct email to a hiring manager or technical lead with a link to your best project and a one-sentence statement of what problem it solved and what your eval numbers showed. "I built a RAG system over [corpus] that achieved faithfulness 0.94 — I'd love to hear what technical problems [company] is working on." This is shorter and more effective than a cover letter.

Technical Screens

Most AI engineering technical screens are one of: system design, code review, or "tell me about a project you've worked on." Your portfolio projects are the source material for all three.

For "tell me about a project": use the story structure from Part 2 — problem → architecture → trade-offs made → eval numbers → what you'd do differently. Practice this out loud. The ability to tell a 5-minute coherent technical story about a project you built is a separate skill from building the project.

For system design: you've now designed five major systems (single agent, RAG, multi-agent, secure agent, production-hardened system). You can discuss trade-offs with specificity because you've made them. When asked "how would you design a RAG system for X?" you have real opinions, not textbook answers.

What to Say in the Evals Round

Many top AI teams now have an evals-specific screen: "how would you measure whether this system is working?" This is Chapter 7, applied to whatever system they describe. The structure: error analysis first, define application-specific metrics, implement LLM-as-judge with calibration, offline regression harness, online monitoring with the data flywheel. If you can describe this end-to-end with the specific metrics you'd use and the tools you'd consider, you've passed this screen.

Part 8: What Top-1% Looks Like in 2026

The field has moved fast. Top 1% in 2024 meant "can build a RAG system." Top 1% in 2025 meant "can build and eval a multi-agent system." In 2026, the bar has risen again.

Top 1% in 2026 means:

Shipped systems, not just built systems. Deployed, used, measured. The ability to go from "I want to build X" to "X is live with real users and I have eval numbers" end-to-end, without help.

Evals as first-class engineering. Not an afterthought. Not "I ran some tests." An eval framework built in parallel with the system, with calibrated judges, a growing ground-truth dataset, and a data flywheel. The eval numbers are visible in the README.

Reasoning depth, not tool fluency. In an interview, you can explain why prompt caching works at the KV cache level, why ReAct improves agent decision quality, why hybrid search beats pure vector search for exact-match queries, and why constrained decoding guarantees structural validity without guaranteeing semantic correctness. You derive these from first principles, not from documentation. This is the gap that separates the engineers who got here from reading this book versus the ones who read about tools without understanding the mechanisms.

Production instincts. "Does this work?" is only part of the question. "Does this work at 100 requests per minute? What does it cost? What breaks at 10x scale? How do I know when it's degrading?" These questions surface immediately and naturally when you think about a system. They're not an afterthought.

Taste. Knowing when not to use an agent. Knowing when RAG is wrong for the problem. Knowing when a simple prompt chain is better than a multi-agent system. The ability to say "here's the simpler approach and why it's better" is a senior signal that distinguishes engineers from enthusiasts.

The engineers who land the best roles in AI in 2026 are not necessarily the ones who know the most frameworks. They're the ones who have built real things, measured whether they work, written about the reasoning behind their choices, and can explain the physical mechanisms behind the techniques they use. That's the person this book set out to produce.

Vocabulary Map

Practice Problems

Problem 1 — Failure Analysis

A hiring manager for an AI engineer role looks at two portfolios. Portfolio A has six projects listed on GitHub, each with a README describing what was built. Portfolio B has two projects, each with: a 200-word problem statement, an architecture diagram, three specific trade-offs with justifications, eval numbers in a table, a deployed URL, and a 90-second Loom demo. Which portfolio converts better and why? Identify the five specific signals Portfolio B provides that Portfolio A lacks, and for each signal explain what it tells a technical interviewer about the engineer's capabilities.

Problem 2 — Design from Scratch

You want to build Capstone 3: a "meeting assistant" that joins recorded team meetings (you provide transcripts), extracts action items and decisions, and can answer questions about what was discussed. Design the complete system: architecture with all components, the specific UI approach, the eval strategy (what metrics, how measured, ground truth dataset construction), the trade-offs you'd document in the README, and the 90-second demo script (describe what you'd show and what you'd say).

Problem 3 — Synthesis

You're in a technical interview. The interviewer says: "Tell me about the most technically interesting challenge you solved in your RAG system." You have implemented everything in Chapter 4 — all six evals, hybrid search, re-ranking, query rewriting. Write the 5-minute verbal response you would give. It should cover: what the challenge was (physical explanation of why it's hard), what you tried first and why it didn't work, what you ultimately built and the key design decisions, and what the eval numbers showed.

Problem 4 — Go Deeper

Most AI engineering blog posts describe how to implement something ("here's how to build a RAG system"). The highest-signal blog posts describe why a common approach is wrong and what to do instead. Identify three techniques from this book that are commonly implemented incorrectly in production systems you've seen in the wild (tutorials, open source projects, blog posts). For each: what the common approach is, why it fails, what the correct approach is, and what evidence would let someone verify your claim. This is a "go deeper" problem because it requires synthesis across the book.

Problem 5 — Connect to Every Chapter

This book was built around one chain: A model is a frozen next-token predictor → you control it only through the prompt → but a prompt can't act on the world, so you add tools → but the model lacks your private knowledge, so you add RAG → but one call can't do multi-step work, so you add agents → but one agent overflows, so you add multi-agent + memory → but it's all nondeterministic, so you add evals → but it's an attack surface, so you add security → but it must meet cost/latency budgets, so you add production architecture → and you ship it as evidence. For each link in this chain, identify: (a) the specific failure that motivated the next step, (b) the physical mechanism by which the new technique fixes it, and (c) what the system would do wrong if you skipped that step and went directly to the next one.

What You Built

You set out to become a top-1% applied AI engineer. Not model training, not research — the building side. You wanted to understand the mechanisms, not memorize the APIs. You wanted to ship, not demo.

Here's what you have now. You know why models are frozen, stateless, and stochastic, and you know which of those properties each technique in this book exists to address. You know how to steer a model's output distribution with precision and how to build a regression harness that catches degradation before users do. You know what happens on the wire during a tool call and why the model never actually executes anything. You know how to build a retrieval system that finds the right documents and how to measure whether it's actually finding them. You know how to build an agent loop from scratch, when to use one and when not to, and how to keep it from going off the rails. You know how to coordinate multiple agents with shared memory, how to handle every failure mode that can bring a multi-agent system down, and how to measure quality rigorously enough to know when your improvements are real. You know the attack surface your agent creates and how to defend it with depth. You know how to turn a working demo into a profitable, scalable production system. And you know how to make all of that visible to the people who should know it.

The field is moving fast. New models will be released before you finish reading this sentence. New techniques will be invented. Frameworks will be deprecated and replaced. But the first principles — why models work, why the failure modes exist, why each layer of the stack was added to fix a specific failure in the layer before it — those are stable. Build on the first principles, and you'll be able to navigate whatever comes next.

Now ship something.