feat: add extraction schema, sidebar nav, few-shot prompting, and prompt settings

Overhaul extraction pipeline with new TradeItem model, conversation flow, and dedicated extraction endpoint. Add sidebar navigation with NavMenu component and landing page. Introduce few-shot prompting service and tests. Add prompt settings and email upload specs. Update OpenSpec tooling with improved export-spec and extract-feature commands. Archive completed changes and export full specs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 23:39:23 +01:00
parent 7a5c22593a
commit 5b027eb0db
83 changed files with 4242 additions and 296 deletions
--- a/openspec/exports/chat-agent-full-openspec.md
+++ b/openspec/exports/chat-agent-full-openspec.md
@@ -0,0 +1,131 @@
+# OpenSpec Artifacts for: AI Chat Agent with Streaming & Rich Text
+
+On the target machine with OpenSpec + Claude Code:
+1. Create `openspec/config.yaml` (see below)
+2. Save `proposal.md` to `openspec/changes/chat-agent-full/proposal.md`
+3. Save `design.md` to `openspec/changes/chat-agent-full/design.md`
+4. Save `tasks.md` to `openspec/changes/chat-agent-full/tasks.md`
+5. Run `/opsx:apply chat-agent-full`
+6. Use the portable spec (`chat-agent-full-spec.md`) as reference if the AI needs detail
+
+---
+
+## config.yaml
+
+Adapt to your project name and constraints, save as `openspec/config.yaml`:
+
+```yaml
+schema: spec-driven
+
+context: |
+  <Your Project Name> — <one-line description>.
+  Tech stack: .NET 9, C# 13, Blazor WASM, ASP.NET Core Web API.
+  Key libraries: MudBlazor 9.2.0, Semantic Kernel 1.74.0, Markdig 1.1.1.
+  <Any target-specific constraints>.
+```
+
+---
+
+## proposal.md
+
+**Integration Rule**: This feature is additive only. DO NOT modify existing files, components, services, or patterns. If the target already has an equivalent service (HTTP wrapper, markdown renderer, etc.), use the existing one. If a task conflicts with existing code, stop and notify the user before proceeding. Existing applicationX code takes precedence in all cases.
+
+Build a complete AI chat interface on an existing MudBlazor app:
+- Chat page with streaming AI responses via SSE
+- Backend using Semantic Kernel with tool calling (extraction plugin)
+- Multi-turn conversation support
+- Rich text rendering (Markdig + HTML sanitization)
+- Full xUnit test coverage
+
+Assumes MudBlazor is already installed and configured.
+
+---
+
+## design.md
+
+### Integration: additive only
+- This feature is a GUEST in the target application — add new files, never modify existing ones
+- If the target already has a service that overlaps (e.g., its own HttpClient wrapper, markdown renderer, error handling), use theirs — do not create a duplicate
+- Conform to the target's code style, naming conventions, and DI patterns
+- If a task would require modifying existing code, **stop and notify the user** — the user decides whether to skip, adapt, or redesign
+- Only touch existing files to add a nav link or a DI registration line — never restructure
+
+### Streaming: SSE over SignalR
+- SSE is simpler — one-directional text stream over HTTP, no WebSocket negotiation
+- Blazor WASM supports streaming via `SetBrowserResponseStreamingEnabled(true)` + `ResponseHeadersRead`
+- Client parses line-by-line: `data: {"text":"..."}\n\n` for deltas, `data: [DONE]\n\n` for end, `data: {"error":"..."}` for errors
+- **CRITICAL SSE loop pattern** — use `while ((line = await reader.ReadLineAsync()) != null)`. Do NOT use `reader.EndOfStream` — it does a synchronous peek that Blazor WASM's fetch-backed stream rejects at runtime
+
+### AI orchestration: Semantic Kernel
+- SK provides chat completion connectors, plugin system, and auto function calling
+- OpenAI connector works with any OpenAI-compatible endpoint (e.g. local proxy) — base URL **must** include `/v1`
+- ExtractionPlugin imported per-request via `ImportPluginFromObject` (not at Kernel build, avoids plugin state leaking across requests)
+- `FunctionChoiceBehavior.Auto()` lets the LLM autonomously call tools and retry
+
+### Client architecture
+- **Typed HttpClient** (`ChatApiClient`) — centralizes API paths, easy to mock for tests, IHttpClientFactory manages handler lifetime
+- **MarkdownService** (singleton) — Markdig pipeline is immutable/thread-safe; two-pass sanitization (regex strip script/style, then tag allowlist) rather than a sanitization library to keep dependencies minimal
+- **Rendered HTML cache** — `Dictionary<ChatMessage, string>` prevents re-running Markdig on all completed messages during every `StateHasChanged` while streaming
+
+### Chat UI layout — "Sales Assistant" page
+- Page renders **inside MudMainContent** — do NOT add a separate AppBar or layout
+- Route: `/sales-assistant`, add MudNavLink in existing sidebar NavMenu
+- Flexbox column: message-list (flex 1, scrollable) + input-area (pinned bottom)
+- MudPaper bubbles: user right-aligned (primary bg), assistant left-aligned (surface bg)
+- `::deep` CSS selectors needed for MudBlazor child components and MarkupString-injected HTML
+- Auto-scroll via JS interop (`scrollTop = scrollHeight`) — no built-in Blazor scroll API
+- **Container height**: `calc(100vh - 64px)` — CRC app uses regular MudAppBar (~64px), not Dense. MudDrawer width is handled by MudLayout automatically — no horizontal calc needed
+
+### Multi-turn
+- Full conversation history sent with each request (all messages with non-empty content)
+- Empty assistant placeholder excluded to avoid confusing the LLM
+
+### Test strategy
+- API: `WebApplicationFactory<Program>` with mocked `IChatCompletionService` — tests SSE contract without hitting real LLM
+- Client: mock `HttpMessageHandler` with canned SSE response streams
+- Requires `public partial class Program { }` in API for test factory access
+
+---
+
+## tasks.md
+
+### Phase 1: Shared Models
+- [ ] Create `ChatMessage.cs` in Shared/Models — Role (string), Content (string), Timestamp (DateTime)
+- [ ] Create `ChatRequest.cs` — List<ChatMessage> Messages
+- [ ] Create `HealthResponse.cs` — Status (string), Timestamp (DateTime)
+- [ ] Create `ExtractedFields.cs` — Client?, Project?, Hours?, Rate?, Currency?, Date? (required); Description?, PoNumber? (optional)
+- [ ] Create `ValidationResult.cs` — IsValid (bool), Errors (List<string>)
+
+### Phase 2: API Backend
+- [ ] Add NuGet: Microsoft.SemanticKernel 1.74.0, Connectors.OpenAI 1.74.0
+- [ ] Create `ExtractionPlugin.cs` in Plugins/ — [KernelFunction] validates extracted fields JSON
+- [ ] Configure Program.cs: AddControllers, AddOpenAIChatCompletion (base URL must include /v1), AddKernel, ExtractionPlugin singleton, CORS for client origin
+- [ ] Create `HealthController.cs` — GET /api/health returns HealthResponse
+- [ ] Create `ChatController.cs` — POST /api/chat, streams via SK GetStreamingChatMessageContentsAsync, SSE format: data: {"text":"..."}\n\n and data: [DONE]\n\n
+- [ ] Add `public partial class Program { }` for test accessibility
+- [ ] Add appsettings.json: ResponsesApi:BaseUrl, ResponsesApi:Model
+
+### Phase 3: Client Services
+- [ ] Add NuGet: Microsoft.Extensions.Http 9.0.4, Markdig 1.1.1
+- [ ] Create `ChatApiClient.cs` — typed HttpClient with GetHealthAsync and SendChatStreamingAsync (IAsyncEnumerable<string>). Use SetBrowserResponseStreamingEnabled(true) + ResponseHeadersRead. Parse SSE with ReadLineAsync null check (not EndOfStream).
+- [ ] Create `MarkdownService.cs` — Markdig with UseAdvancedExtensions, two-pass HTML sanitization (script/style strip, tag allowlist)
+- [ ] Register in Program.cs: AddMudServices, AddSingleton<MarkdownService>, AddHttpClient<ChatApiClient>
+- [ ] Add wwwroot/appsettings.json with ApiBaseUrl_Http and ApiBaseUrl_Https
+
+### Phase 4: Chat UI
+- [ ] Create `SalesAssistant.razor` at route "/sales-assistant" — message list with MudPaper bubbles, MudTextField with send icon, Enter key handler
+- [ ] Add MudNavLink to existing sidebar NavMenu: "Sales Assistant", icon SmartToy, href /sales-assistant
+- [ ] Streaming: append tokens to assistant message, StateHasChanged per token, auto-scroll via JS interop
+- [ ] Assistant messages: render as (MarkupString) from MarkdownService inside .markdown-body div, with rendered HTML cache
+- [ ] User messages: plain MudText
+- [ ] Thinking indicator: MudProgressCircular when assistant content empty during streaming
+- [ ] New Chat button: clears messages, visible when conversation exists, disabled during streaming
+- [ ] Multi-turn: send all non-empty messages with each request
+- [ ] Create `SalesAssistant.razor.css` — flex layout with `calc(100vh - 64px)` height (64px = regular AppBar), message bubbles (user right/primary, assistant left/surface), markdown styles (code blocks, tables, blockquotes, headings, links)
+
+### Phase 5: Tests
+- [ ] Create xUnit test project for API with WebApplicationFactory, mock IChatCompletionService
+- [ ] Test HealthController (200 + valid response), ChatController (SSE streaming, error handling), ExtractionPlugin (validation logic)
+- [ ] Create xUnit test project for Client with mock HttpMessageHandler
+- [ ] Test ChatApiClient (delta parsing, error events), MarkdownService (rendering + sanitization)
+- [ ] Verify: `dotnet test`
--- a/openspec/exports/chat-agent-full-spec.md
+++ b/openspec/exports/chat-agent-full-spec.md
@@ -0,0 +1,206 @@
+# Feature: AI Chat Agent with Streaming & Rich Text
+## Target: Existing MudBlazor app (.NET 9 / Blazor WASM + ASP.NET Core API)
+## Includes: basic-chat-interface, wire-responses-api, migrate-to-semantic-kernel, multi-turn-conversations, add-test-coverage, enable-rich-text-display
+## Skipped: migrate-claude-md-to-openspec (project scaffolding only)
+
+## Integration Rule
+This feature is **additive only**. Existing applicationX code takes precedence over this spec in all cases.
+- **DO NOT** modify existing files, components, layouts, services, or patterns in the target
+- **DO NOT** replace existing patterns (e.g., if the target uses a different HttpClient pattern, use theirs)
+- **DO** add new files, new nav links, new routes, new DI registrations
+- **DO** conform to the target's existing code style, naming, and project structure
+- If the target already has an equivalent service (markdown renderer, HTTP wrapper, etc.), **use theirs**
+- If a task conflicts with existing target code, **stop and notify the user** — do not skip silently; the user decides whether to skip, adapt, or redesign
+
+## Assumes
+- .NET 9 solution with Blazor WASM client + ASP.NET Core API projects + Shared class library
+- MudBlazor 9.2.0 installed and configured (AddMudServices, providers in MainLayout)
+- MudLayout with MudAppBar and MudMainContent in MainLayout.razor
+
+## Target Layout (CRC app)
+```
+|------ ~220px ------|-------------- fluid ---------------|
+|                    | ≡  CRC  0.0.0    APR-CRC-PROD-... | ← MudAppBar (regular, ~64px)
+| Home               |------------------------------------|
+| Pricer             |                                    |
+| Market Data        |  MudMainContent (@Body)            |
+| XVA Statistics     |  ← Chat page renders here          |
+| Sales              |                                    |
+| Sales Assistant ★  |                                    |
+|                    |                                    |
+| MudDrawer (~220px) |                                    |
+|---------------------|-----------------------------------|
+```
+- MudAppBar: **regular height (~64px)**, not Dense — hamburger toggle, title + version, env badge
+- MudDrawer: left, **~220px**, toggled by hamburger, contains MudNavMenu
+- MudMainContent: fluid width, pages render via @Body
+- "Sales Assistant" added as new MudNavLink in existing MudNavMenu
+- Chat page renders **inside MudMainContent** — must not set its own AppBar or layout
+- Available viewport: `height: calc(100vh - 64px)`, width: fluid minus drawer when open
+
+## Packages
+
+**API project:**
+- `Microsoft.SemanticKernel` 1.74.0
+- `Microsoft.SemanticKernel.Connectors.OpenAI` 1.74.0
+
+**Client project:**
+- `Microsoft.Extensions.Http` 9.0.4
+- `Markdig` 1.1.1
+
+**Test projects (xUnit):**
+- `xunit`, `xunit.runner.visualstudio`, `Microsoft.NET.Test.Sdk`
+- API tests: `Moq`, `Microsoft.AspNetCore.Mvc.Testing`
+- Client tests: `Moq`
+
+## Architecture
+
+Three-project solution: WASM client sends chat requests to ASP.NET Core API, which processes them through Semantic Kernel (pointed at an OpenAI-compatible endpoint) and streams token deltas back as SSE. Client renders assistant markdown as sanitized HTML. An extraction plugin demonstrates SK tool calling.
+
+## Shared Models
+### `ChatMessage` — Shared/Models/
+- `string Role` ("user" | "assistant"), `string Content`, `DateTime Timestamp`
+
+### `ChatRequest` — Shared/Models/
+- `List<ChatMessage> Messages`
+
+### `HealthResponse` — Shared/Models/
+- `string Status`, `DateTime Timestamp`
+
+### `ExtractedFields` — Shared/Models/
+- Required: `string? Client`, `string? Project`, `decimal? Hours`, `decimal? Rate`, `string? Currency`, `string? Date`
+- Optional: `string? Description`, `string? PoNumber`
+
+### `ValidationResult` — Shared/Models/
+- `bool IsValid`, `List<string> Errors`
+
+## Components
+
+### API: `Program.cs`
+- Register `AddControllers()`, `AddOpenAIChatCompletion()` with configurable endpoint, `AddKernel()`, ExtractionPlugin singleton
+- CORS policy allowing Blazor client origin, any header/method
+- Config keys: `ResponsesApi:BaseUrl` (default `http://localhost:8317/v1`), `ResponsesApi:Model`, `ResponsesApi:ApiKey`
+- IMPORTANT: Base URL must include `/v1` — the OpenAI SDK appends `chat/completions` directly
+- Add `public partial class Program { }` for test accessibility
+
+### API: `ChatController` — Controllers/
+- POST `/api/chat` accepts `ChatRequest`, returns `text/event-stream`
+- Get `IChatCompletionService` from Kernel, convert messages to `ChatHistory`
+- Import ExtractionPlugin from DI: `_kernel.ImportPluginFromObject(plugin, "Extraction")`
+- Use `OpenAIPromptExecutionSettings` with `FunctionChoiceBehavior.Auto()`
+- Stream via `GetStreamingChatMessageContentsAsync()`, emit non-empty chunks as SSE
+- SSE format: `data: {"text":"<delta>"}\n\n`, completion: `data: [DONE]\n\n`, error: `data: {"error":"<msg>"}\n\n`
+- Catch `HttpRequestException` → error SSE event, `TaskCanceledException` → silent (client disconnect)
+
+### API: `HealthController` — Controllers/
+- GET `/api/health` → returns `HealthResponse` with status "Healthy" and UTC timestamp
+
+### API: `ExtractionPlugin` — Plugins/
+- `[KernelFunction("validate_extracted_fields")]` method accepting JSON string
+- Deserialize to `ExtractedFields`, validate required fields non-null, Hours/Rate > 0
+- Return `ValidationResult` as JSON string
+
+### Client: `Program.cs`
+- Register `AddMudServices()`, `MarkdownService` (singleton)
+- Register `AddHttpClient<ChatApiClient>` with API base URL from config
+- Config: `ApiBaseUrl_Https`, `ApiBaseUrl_Http` in wwwroot/appsettings.json
+- Auto-detect HTTPS from `HostEnvironment.BaseAddress`
+
+### Client: `ChatApiClient` — Services/
+- Typed HttpClient wrapper
+- `GetHealthAsync()` → GET api/health, returns `HealthResponse?`
+- `SendChatStreamingAsync(ChatRequest)` → `IAsyncEnumerable<string>`
+  - Build `HttpRequestMessage` manually
+  - Call `httpRequest.SetBrowserResponseStreamingEnabled(true)` (Blazor WASM extension)
+  - Send with `HttpCompletionOption.ResponseHeadersRead`
+  - Parse SSE line-by-line: extract `"text"` field, throw on `"error"` field, stop on `[DONE]`
+  - SSE loop: see "Critical Patterns" section below — do NOT use `reader.EndOfStream`
+
+### Client: `MarkdownService` — Services/
+- Singleton wrapping Markdig with `UseAdvancedExtensions()` pipeline
+- `ConvertToHtml(string markdown)` → sanitized HTML string
+- Two-pass sanitization:
+  1. Strip `<script>` and `<style>` blocks with content (regex)
+  2. Filter tags against allowlist: p, h1-h6, strong, em, code, pre, ul, ol, li, a, table, thead, tbody, tr, th, td, br, blockquote
+- Only `href` attribute allowed (on `<a>` only)
+
+### Client: `SalesAssistant.razor` — Pages/, route `/sales-assistant`
+- Page title: "Sales Assistant"
+- Add `MudNavLink` in existing sidebar NavMenu: icon `Icons.Material.Filled.SmartToy`, href `/sales-assistant`
+- Message list with MudPaper bubbles: user (right-aligned, primary bg), assistant (left-aligned, surface bg)
+- MudTextField with send icon adornment, Enter key handler, disabled during streaming
+- Empty state with centered title text
+- "New Chat" button (MudButton) above input, visible when messages exist, disabled during streaming
+- Streaming: append tokens to assistant message, call `StateHasChanged()` + auto-scroll per token
+- Assistant messages: render via `(MarkupString)GetRenderedHtml(message)` inside `.markdown-body` div
+- Rendered HTML cache (`Dictionary<ChatMessage, string>`) — cache on stream completion, render fresh during streaming
+- Auto-scroll via JS interop: `document.querySelector('.message-list').scrollTop = .scrollHeight`
+- Thinking indicator: `MudProgressCircular` when assistant content is empty and streaming
+- Multi-turn: send all messages with non-empty content (excludes empty assistant placeholder)
+- User messages: plain `MudText`, no markdown
+
+### Client: `SalesAssistant.razor.css` — scoped styles
+- `.chat-container`: flex column, `height: calc(100vh - 64px)`, max-width 800px, centered within MudMainContent
+- `.message-list`: flex 1, overflow-y auto, flex column, gap 0.75rem
+- Message alignment: `.message-user` flex-end, `.message-assistant` flex-start
+- `::deep .bubble-user`: primary color bg, white text, rounded except bottom-right
+- `::deep .bubble-assistant`: surface bg, border, rounded except bottom-left
+- `::deep .markdown-body` styles: code blocks (gray bg, monospace), tables (bordered), blockquotes (left border primary), headings (scaled down), links (primary color, underline)
+
+## Wiring (dependency order)
+
+1. **Shared project**: Create all models (ChatMessage, ChatRequest, HealthResponse, ExtractedFields, ValidationResult)
+2. **API Program.cs**: AddControllers → AddOpenAIChatCompletion → AddKernel → AddSingleton<ExtractionPlugin> → AddCors → Build → UseCors → UseAuthorization → MapControllers
+3. **Client Program.cs**: AddMudServices → AddSingleton<MarkdownService> → AddHttpClient<ChatApiClient> → Build → RunAsync
+4. **API appsettings.json**: `{"ResponsesApi": {"BaseUrl": "http://localhost:8317/v1", "Model": "claude-sonnet-4-6"}}`
+5. **Client wwwroot/appsettings.json**: `{"ApiBaseUrl_Http": "http://localhost:7000", "ApiBaseUrl_Https": "https://localhost:7100"}`
+
+## Critical Patterns (copy these, do not improvise)
+
+### SSE read loop in Blazor WASM
+```csharp
+// DO NOT use reader.EndOfStream — it does a synchronous peek,
+// which Blazor WASM's fetch-backed stream rejects at runtime.
+string? line;
+while ((line = await reader.ReadLineAsync()) != null)
+{
+    if (!line.StartsWith("data: ")) continue;
+    var data = line.Substring(6);
+    if (data == "[DONE]") yield break;
+    // parse {"text":"..."} or {"error":"..."}
+}
+```
+
+### Chat container height
+```css
+/* 64px = MudAppBar regular height in the CRC app.
+   The chat page renders inside MudMainContent which already
+   sits below the AppBar, so use 100% of the parent height
+   rather than calculating from viewport. If MudMainContent
+   does not have height set, fall back to calc(100vh - 64px). */
+.chat-container {
+    height: calc(100vh - 64px);
+    /* MudDrawer width is handled by MudLayout — the chat container
+       is fluid within MudMainContent, no horizontal calc needed. */
+}
+```
+
+## Behavior (non-obvious)
+
+- Base URL **must** include `/v1` — SK's OpenAI connector appends `chat/completions` directly
+- WASM streaming requires **both** `SetBrowserResponseStreamingEnabled(true)` and `ResponseHeadersRead`
+- Only emit SSE chunks where `!string.IsNullOrEmpty(chunk.Content)` — tool call chunks have no text
+- Cache rendered HTML for completed messages to avoid re-running Markdig on every `StateHasChanged`
+- `public partial class Program { }` at bottom of API Program.cs for `WebApplicationFactory<Program>` in tests
+- ExtractionPlugin is imported per-request via `ImportPluginFromObject` (not at Kernel build time)
+- Chat sends all messages with non-empty content — filters out the empty assistant placeholder
+- Chat container height assumes MudAppBar Dense (48px) — if your AppBar height differs, adjust the calc() or use a CSS variable
+- On mobile viewports, `100vh` includes the browser address bar — use `100dvh` if targeting mobile
+
+## Test Coverage
+
+- **HealthControllerTests**: GET /api/health returns 200 with valid HealthResponse
+- **ChatControllerTests**: Mock `IChatCompletionService`, verify SSE text deltas + [DONE]; verify error handling; use `WebApplicationFactory<Program>`
+- **ExtractionPluginTests**: Valid fields → IsValid; missing required → errors; invalid JSON → error; zero hours → error
+- **ChatApiClientTests**: Mock `HttpMessageHandler` with canned SSE streams; verify delta parsing order; verify error event throws
+- **MarkdownServiceTests**: Verify rendering (bold, italic, code, lists, tables, links, headings); verify sanitization (script/style stripping, event handler removal, tag allowlist)