Data Engineering and Agentic AI Evaluation

A textbook on how raw data becomes a table people trust, and how you grade an agent when its output is a whole trajectory, not a single number. Every chapter runs real experiments on a laptop (Postgres, Spark, Redpanda, Delta Lake, Iceberg, DuckDB, Docker task grading) and reports what it measured. The prose is Hinglish (Hindi in Roman script) with English headings and code.

Read it: Data-Engineering-and-Agentic-AI-Evaluation.pdf

Contents

Part I: Data Engineering

• 1. What Data Engineering Really Is
• 2. How Data Is Stored
• 3. Inside a Database
• 4. SQL an Engineer Must Own
• 5. Getting Data In: Batch and CDC
• 6. Distributed Processing
• 7. Warehouse, Lake, Lakehouse
• 8. Data Modelling
• 9. Orchestration
• 10. Streaming
• 11. Data Quality and Contracts
• 12. Governance, Privacy and Cost
• 13. Data System Design

Part II: Agentic AI Evaluation

• 14. What an Agent Is, and Why It's Hard to Grade
• 15. Evaluation Foundations
• 16. Task-Level Evaluation
• 17. Trajectory Evaluation
• 18. LLM as a Judge
• 19. Human Evaluation
• 20. Building an Eval Harness
• 21. Safety and Robustness Evals
• 22. Evaluation in Production
• 23. Data Engineering for Evals
• 24. Capstone and Case Studies

Part III: Reference

• 25. Tool Atlas by Scale
• 26. Interview Problem Bank
• 27. Hands-On Lab Setup

Projects

projects/ has two runnable projects: capstone-eval (a mini SWE-style eval with hidden tests in Docker) and trajectory-pipeline (ingest, quality gate, dedup and pass@k with error bars).

Build it yourself

Each chapter is an HTML fragment in src/, and toc.py holds the table of contents.

make        # chapters, a single-page book.html and the PDF (needs Google Chrome)
make serve  # read it at http://localhost:8000

Written by Abhishek Verma.