microsoft-extensions-ai

# Microsoft.Extensions.AI

## Trigger On

- building or reviewing `.NET` code that uses `Microsoft.Extensions.AI`, `Microsoft.Extensions.AI.Abstractions`, `IChatClient`, `IEmbeddingGenerator`, `ChatOptions`, or `AIFunction`
- adding `IImageGenerator`, local-model chat via Ollama, AI app templates, or the `.NET AI` quickstarts for assistants and MCP
- choosing between low-level AI abstractions, provider SDKs, vector-search composition, evaluation libraries, and a fuller agent framework
- adding streaming chat, structured output, embeddings, tool calling, telemetry, caching, or DI-based AI middleware
- wiring `Microsoft.Extensions.VectorData`, `Microsoft.Extensions.DataIngestion`, MCP tooling, or evaluation packages around a provider-agnostic AI app

## Workflow

1. Classify the request first: plain model access, tool calling, embeddings/vector search, evaluation, image generation, local-model prototyping, MCP bootstrap, or true agent orchestration.
2. Default to `Microsoft.Extensions.AI` for application and service code that needs provider-agnostic chat, embeddings, middleware, structured output, and testability.
3. Reference `Microsoft.Extensions.AI.Abstractions` directly only when authoring provider libraries or lower-level reusable integration packages.
4. Model `IChatClient` and `IEmbeddingGenerator` composition explicitly in DI. Keep options, caching, telemetry, logging, and tool invocation inspectable in the pipeline.
5. Treat chat state deliberately. For stateless providers, resend history. For stateful providers, propagate `ConversationId` rather than assuming all providers behave the same way.
6. Use `Microsoft.Extensions.VectorData` and `Microsoft.Extensions.DataIngestion` as adjacent building blocks for RAG instead of hand-rolling store abstractions prematurely. Treat the embedding model, vector dimensions, and collection schema as one owned contract: changing any of them means reindexing rather than reusing old vector data. In `dotnet/extensions` v10.5.0, update any named-argument usage of `VectorStoreVectorAttribute` from `Dimensions:` to `dimensions:`; this is source-breaking only.
7. Treat the `.NET AI` quickstarts as bootstrap paths, not finished architecture. They now cover minimal assistants, MCP client/server flows, local models, app templates, and image generation. Start there for a vertical slice, then harden the DI, telemetry, and evaluation story here.
8. Escalate to `microsoft-agent-framework` when the requirement becomes agent threads, multi-agent orchestration, higher-order workflows, durable execution, or remote agent hosting.
9. Validate with real providers, realistic prompts, and evaluation gates so the abstraction layer actually buys portability and reliability.

## Architecture

```mermaid
flowchart LR
  A["Task"] --> B{"Need agent threads, multi-agent orchestration, or remote agent hosting?"}
  B -->|Yes| C["Use Microsoft Agent Framework on top of `Microsoft.Extensions.AI.Abstractions`"]
  B -->|No| D{"Need provider-agnostic chat, embeddings, tools, typed output, or evaluation?"}
  D -->|Yes| E["Use `Microsoft.Extensions.AI`"]
  E --> F["Compose `IChatClient` / `IEmbeddingGenerator` in DI"]
  F --> G["Add caching, telemetry, tools, vector data, and evaluation deliberately"]
  D -->|No| H["Use plain provider SDKs or deterministic .NET code"]
```

## Core Knowledge

- `Microsoft.Extensions.AI.Abstractions` contains the core exchange contracts such as `IChatClient`, `IEmbeddingGenerator<TInput, TEmbedding>`, message/content types, and tool abstractions.
- `Microsoft.Extensions.AI` adds the higher-level application surface: middleware builders, automatic function invocation, caching, logging, and OpenTelemetry integration.
- Most apps and services should reference `Microsoft.Extensions.AI`; provider and connector libraries usually reference only the abstractions package.
- `IChatClient` centers on `GetResponseAsync` and `GetStreamingResponseAsync`. The returned `ChatResponse` or `ChatResponseUpdate` objects carry messages, tool-related content, metadata, and optional conversation identifiers.
- Local-model quickstarts still route through the same `IChatClient` abstraction. Ollama-backed clients are useful for low-cost prototyping, offline dev loops, and portability testing, but you still own chat history replay, latency, and model-quality tradeoffs.
- `ChatOptions` is the normal control plane for model ID, temperature, tools, `AdditionalProperties`, and provider-specific raw options.
- Tool calling is modeled with `AIFunction`, `AIFunctionFactory`, and `FunctionInvokingChatClient`. Ambient data can flow through closures, `AdditionalProperties`, `AIFunctionArguments.Context`, or DI.
- Tool calling can target local .NET methods, external APIs, or MCP-backed tools. The model requests calls; your app still owns execution, validation, and side-effect boundaries.
- Tool definitions consume request tokens. Keep tool descriptions short and register only the tools relevant for the current conversation or workflow.
- `FunctionInvokingChatClient` can handle the tool-invocation loop and parallel tool-call responses automatically when the provider/model supports that shape.
- `IEmbeddingGenerator` is the standard abstraction for semantic search, vector indexing, similarity, and cache-key generation. Pair it with `Microsoft.Extensions.VectorData.Abstractions` for vector store operations, and keep the embedding model, collection dimensions, and chunking/versioning story aligned so reindexing stays explicit.
- `IImageGenerator` is the experimental MEAI image surface. Treat `MEAI001` as an intentional opt-in, keep image generation separate from chat concerns, and compose logging/caching/hosting middleware around it the same way you would for `IChatClient`.
- `Microsoft.Extensions.DataIngestion` gives you the document-side RAG pipeline: `IngestionDocument`, document readers like MarkItDown/Markdig, document processors such as `ImageAlternativeTextEnricher`, chunkers, chunk proc