Agentic AI Tech Stack Explained

An agentic AI stack is a layered system, not a logo board. Infrastructure runs models and agents. Foundation models produce or inspect language. Orchestration frameworks coordinate steps, tools, and state. Vector databases and embedding models support semantic retrieval. Ingestion tools turn webpages, PDFs, images, and documents into usable chunks. Memory/context tools decide what survives across turns. Evaluation and monitoring decide whether the whole system is still trustworthy.

The running example is an agent that answers Does customer C-104 qualify for a renewal discount? It must ingest contracts, retrieve the right clause, call CRM tools, remember prior approvals, draft an answer, and prove the answer with citations. No single layer owns that whole promise.

Start from the request, not the logo. For C-104, the agent receives user intent, loads policy instructions, retrieves contract chunks, calls CRM, chooses a model, drafts an answer, validates citations, and logs the trace. A good design makes each hop explicit because each hop can fail independently.

This is where the stack becomes operational. LiteLLM or an AI gateway might route model calls. LangGraph, AutoGen, Dify, or a smaller workflow harness might own the state machine. LlamaIndex or Haystack might retrieve documents. Qdrant, Milvus, Pinecone, Weaviate, Redis, ChromaDB, Elastic, Vespa, or pgvector might store vectors. Ragas, Phoenix, MLflow, W&B, Deepchecks, or a custom eval set might measure quality. The exact product mix is secondary to the control boundaries.

The image separates ingestion tools, embedding models, and vector databases. That is correct, but they must be designed together. Scrapy, Beautiful Soup, Diffbot, Firecrawl, Docling, LlamaParse, Amazon Textract, and Apache Tika extract content. Voyage AI, Nomic, fastText, Hugging Face, OpenAI, spaCy, VectorFlow, or Cohere turn text into vectors. The vector store indexes those vectors with metadata.

The contract must say what document version was ingested, how it was chunked, which embedding model and dimension produced the vectors, and which tenants can retrieve it. If any of those fields drift, the retrieval system can return confident but wrong evidence. For deeper retrieval mechanics, use RAG Evaluation and Vector Search Filters.

Orchestration frameworks are not just prompt wrappers. They decide control flow: retrieve first or call a tool, loop or stop, request human approval or execute. Memory and context tools decide what prior facts, user preferences, and task state re-enter the prompt. That makes these layers a trust boundary, not decoration.

For C-104, the agent may remember that sales leadership approved a custom discount last quarter. That memory is useful only if it has a source, timestamp, owner, and expiry. Letta, Mem0, Zep, Cognee, ChromaDB, LangChain memory modules, and LlamaIndex memory patterns should be judged by write policy and retrieval policy, not by whether they sound persistent.

The top and second layers of the image are easy to leave until the end: deployment infrastructure plus evaluation and monitoring. In production they should be designed first. Infrastructure controls latency, cost, data residency, provider keys, scaling, and failover. Evaluation controls whether the system is good enough to release and whether it remains good enough after prompts, models, documents, or tools change.

Do not treat model leaderboards as product evals. Your support-renewal agent needs a representative set of renewal questions, contract versions, ambiguous cases, missing-source cases, and tool-denial cases. Track answer faithfulness, citation accuracy, tool-call correctness, p95 latency, cost per request, escalation rate, and reviewer overrides.

Trigger. The ingestion layer parses a stale renewal-contract PDF, embeds it with an older model, and stores it without contractVersion. Symptom. The agent confidently tells customer C-104 they do not qualify for a discount, even though the current contract says they do. Root mechanism. The orchestration layer worked, the model call worked, and the vector query returned neighbors, but the data contract was broken before generation began.

Recovery: remove stale chunks, rebuild the index with a pinned embedding model, require contractVersion filters, and rerun the affected queries. Prevention: add an ingestion manifest, retrieval evals by contract version, and release gates for parser or embedding changes. The stack was not missing a logo; it was missing a boundary.