feat: add extraction schema, sidebar nav, few-shot prompting, and prompt settings
Overhaul extraction pipeline with new TradeItem model, conversation flow, and dedicated extraction endpoint. Add sidebar navigation with NavMenu component and landing page. Introduce few-shot prompting service and tests. Add prompt settings and email upload specs. Update OpenSpec tooling with improved export-spec and extract-feature commands. Archive completed changes and export full specs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
local
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-05
|
||||
@@ -0,0 +1,42 @@
|
||||
## Context
|
||||
|
||||
The app currently uses a minimal MudLayout with just MudAppBar (Dense) + MudMainContent, and a single page at `/`. To support multiple pages, we need standard MudBlazor navigation: a collapsible MudDrawer with a NavMenu component.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Add collapsible MudDrawer with hamburger toggle in the AppBar
|
||||
- Create a NavMenu component with a "Sales Assistant" link
|
||||
- Move chat page to `/sales-assistant` route
|
||||
- Maintain the Blazor tutorial style with inline comments
|
||||
|
||||
**Non-Goals:**
|
||||
- Adding multiple pages beyond the existing chat (just the navigation structure)
|
||||
- Changing the AppBar from Dense to regular
|
||||
- Adding a default landing page (redirect `/` → `/sales-assistant` instead)
|
||||
|
||||
## Decisions
|
||||
|
||||
### MudDrawer configuration
|
||||
- **Variant**: `DrawerVariant.Mini` — collapses to icon-width rather than fully hiding, so the user always sees the nav rail
|
||||
- **Alternative considered**: `DrawerVariant.Responsive` — auto-hides on small screens. Rejected because Mini gives a more consistent desktop experience and the app is desktop-first.
|
||||
- **ClipMode**: `DrawerClipMode.Always` — drawer sits below the AppBar, not beside it
|
||||
|
||||
### NavMenu as separate component
|
||||
- Extract `NavMenu.razor` into `Layout/` alongside MainLayout rather than inlining nav links
|
||||
- This is standard Blazor project structure and keeps MainLayout focused on shell layout
|
||||
- The NavMenu will use `MudNavMenu` with `MudNavLink` items
|
||||
|
||||
### Route change: `/` → `/sales-assistant`
|
||||
- The chat page moves to `/sales-assistant` to match navigation naming
|
||||
- Add a redirect component at `/` that navigates to `/sales-assistant` on init
|
||||
- This avoids a blank landing page while keeping the URL structure clean
|
||||
|
||||
### AppBar hamburger toggle
|
||||
- Add `MudIconButton` with `Icons.Material.Filled.Menu` as the first element in the AppBar
|
||||
- Toggle `_drawerOpen` bool that binds to `MudDrawer.Open`
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **Chat container height**: Currently uses `calc(100vh - 48px)` assuming Dense AppBar (48px). MudDrawer with ClipMode.Always doesn't affect vertical calc, so this should remain correct. Verify after implementation.
|
||||
- **Breaking bookmarks**: Anyone bookmarking `/` will need to update to `/sales-assistant`. Mitigated by the redirect at `/`.
|
||||
@@ -0,0 +1,26 @@
|
||||
## Why
|
||||
|
||||
The app currently has no navigation — just a single page at `/`. Adding a sidebar drawer with navigation enables the app to grow to multiple pages while providing standard MudBlazor layout structure (AppBar + Drawer + MainContent).
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a MudDrawer to MainLayout with a NavMenu component containing a "Sales Assistant" link
|
||||
- Add a hamburger toggle button in the AppBar to open/close the drawer
|
||||
- Move the existing chat page from `/` to `/sales-assistant`
|
||||
- Add a landing page or redirect at `/` so the app has a default route
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `sidebar-navigation`: Collapsible sidebar drawer with navigation menu, hamburger toggle, and route structure
|
||||
|
||||
### Modified Capabilities
|
||||
- `chat-ui`: Route changes from `/` to `/sales-assistant`
|
||||
|
||||
## Impact
|
||||
|
||||
- **MainLayout.razor**: Add MudDrawer, hamburger icon, drawer toggle state
|
||||
- **New NavMenu component**: Shared/NavMenu.razor with MudNavGroup/MudNavLink items
|
||||
- **Chat.razor**: Route changes from `@page "/"` to `@page "/sales-assistant"`
|
||||
- **Chat.razor.css**: Height calc may need adjustment if AppBar Dense changes
|
||||
- No new packages — MudDrawer/MudNavMenu are part of MudBlazor
|
||||
@@ -0,0 +1,15 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Chat page is default route
|
||||
|
||||
The chat page SHALL be routed at `/sales-assistant`. The root URL (`/`) SHALL redirect to `/sales-assistant`.
|
||||
|
||||
#### Scenario: App opens to chat via redirect
|
||||
|
||||
- **WHEN** the user navigates to the root URL `/`
|
||||
- **THEN** the browser redirects to `/sales-assistant` and the chat page is displayed
|
||||
|
||||
#### Scenario: Direct navigation to sales-assistant
|
||||
|
||||
- **WHEN** the user navigates to `/sales-assistant`
|
||||
- **THEN** the chat page is displayed
|
||||
@@ -0,0 +1,38 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Collapsible sidebar drawer
|
||||
|
||||
The application SHALL have a MudDrawer in MainLayout that contains a navigation menu. The drawer SHALL be toggleable via a hamburger icon button in the AppBar.
|
||||
|
||||
#### Scenario: Drawer visible on load
|
||||
|
||||
- **WHEN** the application loads
|
||||
- **THEN** the sidebar drawer is displayed in its default open state with navigation links visible
|
||||
|
||||
#### Scenario: Drawer toggles on hamburger click
|
||||
|
||||
- **WHEN** the user clicks the hamburger icon in the AppBar
|
||||
- **THEN** the drawer toggles between open and collapsed states
|
||||
|
||||
### Requirement: Navigation menu with Sales Assistant link
|
||||
|
||||
The sidebar drawer SHALL contain a MudNavMenu with a "Sales Assistant" navigation link that routes to `/sales-assistant`.
|
||||
|
||||
#### Scenario: Sales Assistant link present
|
||||
|
||||
- **WHEN** the drawer is open
|
||||
- **THEN** a "Sales Assistant" link with a SmartToy icon is visible in the navigation menu
|
||||
|
||||
#### Scenario: Clicking Sales Assistant navigates to chat
|
||||
|
||||
- **WHEN** the user clicks the "Sales Assistant" link
|
||||
- **THEN** the browser navigates to `/sales-assistant` and the chat page renders in MudMainContent
|
||||
|
||||
### Requirement: NavMenu is a separate component
|
||||
|
||||
The navigation menu SHALL be implemented as a separate `NavMenu.razor` component in the Layout folder, referenced from MainLayout.
|
||||
|
||||
#### Scenario: NavMenu renders inside drawer
|
||||
|
||||
- **WHEN** MainLayout renders
|
||||
- **THEN** the NavMenu component renders inside the MudDrawer with its navigation links
|
||||
@@ -0,0 +1,29 @@
|
||||
## 1. MainLayout: Add Drawer and Hamburger Toggle
|
||||
|
||||
- [x] 1.1 Add a `_drawerOpen` bool field (default `true`) and a `ToggleDrawer` method to MainLayout.razor
|
||||
- [x] 1.2 Add a `MudIconButton` with `Icons.Material.Filled.Menu` as the first element in the MudAppBar, wired to `ToggleDrawer`
|
||||
- [x] 1.3 Add a `MudDrawer` with `Open="@_drawerOpen"`, `ClipMode="DrawerClipMode.Always"`, `Elevation="2"` inside the MudLayout, before MudMainContent
|
||||
- [x] 1.4 Reference `<NavMenu />` inside the MudDrawer
|
||||
|
||||
## 2. NavMenu Component
|
||||
|
||||
- [x] 2.1 Create `Layout/NavMenu.razor` with a `MudNavMenu` containing a single `MudNavLink` — text "Sales Assistant", icon `Icons.Material.Filled.SmartToy`, href `/sales-assistant`
|
||||
- [x] 2.2 Add inline tutorial comments explaining MudNavMenu, MudNavLink, and how href-based navigation works in Blazor
|
||||
|
||||
## 3. Chat Page Route Change
|
||||
|
||||
- [x] 3.1 Change `Chat.razor` route from `@page "/"` to `@page "/sales-assistant"`
|
||||
- [x] 3.2 Update the page title from "Chat Agent" to "Sales Assistant"
|
||||
|
||||
## 4. Root Redirect
|
||||
|
||||
- [x] 4.1 Create a `Pages/Index.razor` component at `@page "/"` that redirects to `/sales-assistant` on initialization using `NavigationManager.NavigateTo` (Home.razor already exists as health check page at /health)
|
||||
|
||||
## 5. Chat Container Height Adjustment
|
||||
|
||||
- [x] 5.1 Verify `Chat.razor.css` height calc still works with the drawer layout — Dense AppBar is 48px, drawer does not affect vertical space. Adjust if needed.
|
||||
|
||||
## 6. Verification
|
||||
|
||||
- [x] 6.1 Build the client project (`dotnet build`) and confirm no compilation errors
|
||||
- [x] 6.2 Run existing tests (`dotnet test`) and confirm they pass
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-06
|
||||
@@ -0,0 +1,63 @@
|
||||
## Context
|
||||
|
||||
The extraction endpoint (`POST /api/chat/extract`) and few-shot prompting infrastructure exist on the backend. The client needs a way to trigger extraction by uploading email files and then handle the multi-turn extraction conversation (disambiguation, result presentation).
|
||||
|
||||
The current Chat.razor has a text input with send button and Enter key handling. The extraction flow adds a new input modality (file drop) and conversation mode tracking.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Enable email file upload via drag-and-drop and file picker
|
||||
- Route uploaded emails to the extraction endpoint
|
||||
- Support disambiguation follow-up within the same conversation
|
||||
- Present extraction results clearly in the chat stream
|
||||
|
||||
**Non-Goals:**
|
||||
- .msg file parsing (MVP accepts .html only — users save emails as HTML first)
|
||||
- Structured result UI (tables, editable fields) — the agent streams formatted text/markdown
|
||||
- Offline/batch extraction of multiple emails
|
||||
- Email preview or parsing on the client before sending to the API
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Drag-and-drop on the message area, not a separate upload zone
|
||||
|
||||
**Decision:** The entire chat message area (`.message-list`) acts as the drop target, with visual feedback when a file is dragged over.
|
||||
|
||||
**Why:** Drag-and-drop onto the conversation area is the natural gesture — it mirrors how you'd "give" a document to someone in a chat. A separate upload widget adds visual clutter.
|
||||
|
||||
**Alternative considered:** Dedicated upload zone above the input area. Rejected — takes permanent screen space for an occasional action.
|
||||
|
||||
### 2. HTML files only for MVP
|
||||
|
||||
**Decision:** Accept `.html` files only. Do not support `.msg` parsing in this change.
|
||||
|
||||
**Why:** .msg is a proprietary Microsoft format requiring either a server-side library (like MsgReader) or a JS parser. HTML is what Outlook "Save As" produces and is trivially read via the File API. Supporting .msg can be a follow-up change.
|
||||
|
||||
### 3. Conversation mode tracking with `_isExtractionMode`
|
||||
|
||||
**Decision:** Add a boolean `_isExtractionMode` flag to Chat.razor. When an email is uploaded, set it to `true`. All subsequent `SendMessage()` calls route to the extraction endpoint (passing the original email HTML + growing message list). "New Chat" resets to `false`.
|
||||
|
||||
**Why:** After initial extraction, the user needs to reply to disambiguation questions. Those replies must go to the extraction endpoint with full context, not the general chat endpoint. The mode flag is the simplest routing mechanism.
|
||||
|
||||
**Alternative considered:** Separate extraction page/component. Rejected — breaks the conversational flow and duplicates the chat UI.
|
||||
|
||||
### 4. File reading via Blazor JS interop
|
||||
|
||||
**Decision:** Use `InputFile` component or JavaScript interop with the File API to read the dropped file's text content. Send the HTML string to the extraction endpoint.
|
||||
|
||||
**Why:** Blazor WASM has `InputFile` for file picker but drag-and-drop requires JS interop for the `drop` event. We need both: `InputFile` for the button, JS interop for drag-and-drop.
|
||||
|
||||
### 5. Agent streams results as markdown, no special result UI
|
||||
|
||||
**Decision:** The extraction agent's response (including the final JSON table) streams as markdown rendered by the existing rich text display. No special result component.
|
||||
|
||||
**Why:** The rich text rendering already handles tables, code blocks, and formatted output. The agent naturally presents results as a markdown table. A structured result component adds complexity without clear UX benefit at this stage.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[Large email files]** → HTML emails with embedded images can be large. Mitigation: the API receives the HTML string only — embedded images are base64 in the HTML and the LLM will ignore them. If size becomes an issue, strip image tags client-side before sending.
|
||||
|
||||
**[Mode confusion]** → User may not realize they're in extraction mode. Mitigation: show a visual indicator (e.g., chip or banner) when `_isExtractionMode` is true, and include "New Chat" to reset.
|
||||
|
||||
**[Drop event handling in Blazor]** → Blazor's built-in event handling for drag-and-drop is limited. Mitigation: use a small JS interop function for the drop handler that reads the file and calls back into .NET.
|
||||
@@ -0,0 +1,29 @@
|
||||
## Why
|
||||
|
||||
The extraction agent and few-shot prompting infrastructure exist on the backend, but the chat UI has no way to send emails to the extraction endpoint. Users need to drag-and-drop or upload email files (.html) to trigger extraction. The client must route email uploads to `POST /api/chat/extract` and handle the conversational extraction flow, including disambiguation questions from the agent and result presentation.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **Add drag-and-drop zone** to `Chat.razor` that accepts email files (.html)
|
||||
- **Add file picker button** as an alternative upload method
|
||||
- **Route uploaded emails** to the extraction endpoint via `ChatApiClient.SendExtractionStreamingAsync()`
|
||||
- **Handle extraction conversation flow** — initial extraction streams in, user can reply to disambiguation questions, follow-ups continue via the extraction endpoint
|
||||
- **Present extraction results** — the agent's streamed response includes formatted output; optionally add a "Copy JSON" action
|
||||
- **Track conversation mode** — after an email upload, subsequent messages route to the extraction endpoint until "New Chat" resets to general mode
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `email-upload`: Defines the drag-and-drop upload zone, file handling, visual feedback, and supported formats
|
||||
- `extraction-conversation-flow`: Defines the client-side conversation mode tracking, routing between general chat and extraction, and result presentation
|
||||
|
||||
### Modified Capabilities
|
||||
- `chat-ui`: Add the upload zone to the chat input area and track conversation mode (general vs extraction)
|
||||
|
||||
## Impact
|
||||
|
||||
- **UI changes**: New drop zone and upload button in Chat.razor, visual feedback during drag-over
|
||||
- **Chat.razor.css**: Styling for drop zone states (idle, drag-over, uploading)
|
||||
- **ChatApiClient**: Already has `SendExtractionStreamingAsync` from the previous change — this change wires it to the UI
|
||||
- **Conversation state**: New `_isExtractionMode` flag in Chat.razor to route messages correctly
|
||||
- **Depends on**: `update-extraction-schema` and `few-shot-prompt-infrastructure` (extraction endpoint must exist)
|
||||
@@ -0,0 +1,30 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Message input
|
||||
|
||||
The chat page SHALL provide a text input area at the bottom of the page where the user can type and submit messages. The input area SHALL also include a file upload button for triggering email extraction.
|
||||
|
||||
#### Scenario: Submit via button
|
||||
|
||||
- **WHEN** the user types text and clicks the send button
|
||||
- **THEN** the message is added to the conversation and the input is cleared
|
||||
|
||||
#### Scenario: Submit via Enter key
|
||||
|
||||
- **WHEN** the user types text and presses Enter
|
||||
- **THEN** the message is submitted (same as clicking send)
|
||||
|
||||
#### Scenario: Empty input blocked
|
||||
|
||||
- **WHEN** the user attempts to send an empty or whitespace-only message
|
||||
- **THEN** nothing is sent and no message is added
|
||||
|
||||
#### Scenario: Input disabled during streaming
|
||||
|
||||
- **WHEN** the assistant is currently streaming a response
|
||||
- **THEN** the input field, send button, and upload button are disabled until streaming completes
|
||||
|
||||
#### Scenario: Upload button opens file picker
|
||||
|
||||
- **WHEN** the user clicks the upload button in the input area
|
||||
- **THEN** a file picker dialog opens filtered to .html files
|
||||
@@ -0,0 +1,38 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Drag-and-drop email upload
|
||||
|
||||
The chat message area SHALL accept files dragged from the desktop or file explorer. When a supported file is dropped, the client SHALL read the file content and send it to the extraction endpoint.
|
||||
|
||||
#### Scenario: Drag HTML file onto chat
|
||||
|
||||
- **WHEN** the user drags an .html file over the message area
|
||||
- **THEN** a visual drop indicator appears (e.g., highlighted border, overlay text "Drop email here")
|
||||
|
||||
#### Scenario: Drop HTML file triggers extraction
|
||||
|
||||
- **WHEN** the user drops an .html file onto the message area
|
||||
- **THEN** the client reads the HTML content, sends it to `POST /api/chat/extract`, and streams the extraction response in the chat
|
||||
|
||||
#### Scenario: Unsupported file type rejected
|
||||
|
||||
- **WHEN** the user drops a non-.html file (e.g., .pdf, .docx)
|
||||
- **THEN** the client shows a brief error message indicating only .html files are supported
|
||||
|
||||
### Requirement: File picker upload button
|
||||
|
||||
The chat input area SHALL include an upload button (e.g., attachment icon) that opens a file picker dialog for selecting .html email files.
|
||||
|
||||
#### Scenario: Upload via file picker
|
||||
|
||||
- **WHEN** the user clicks the upload button and selects an .html file
|
||||
- **THEN** the client reads the HTML content and sends it to the extraction endpoint, same as drag-and-drop
|
||||
|
||||
### Requirement: Upload disabled during streaming
|
||||
|
||||
The upload zone and file picker SHALL be disabled while a response is streaming.
|
||||
|
||||
#### Scenario: Drop during streaming
|
||||
|
||||
- **WHEN** the user attempts to drop a file while the assistant is streaming
|
||||
- **THEN** the drop is ignored and no extraction request is sent
|
||||
@@ -0,0 +1,47 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Extraction mode tracking
|
||||
|
||||
The chat page SHALL track whether the current conversation is in extraction mode. Extraction mode is entered when an email is uploaded and exited when the user starts a new chat.
|
||||
|
||||
#### Scenario: Enter extraction mode on upload
|
||||
|
||||
- **WHEN** the user uploads an email file
|
||||
- **THEN** the conversation enters extraction mode and subsequent messages are routed to the extraction endpoint
|
||||
|
||||
#### Scenario: Exit extraction mode on New Chat
|
||||
|
||||
- **WHEN** the user clicks "New Chat" while in extraction mode
|
||||
- **THEN** the conversation exits extraction mode and returns to general chat routing
|
||||
|
||||
### Requirement: Extraction mode visual indicator
|
||||
|
||||
The chat page SHALL display a visual indicator when in extraction mode so the user knows their messages are part of an extraction conversation.
|
||||
|
||||
#### Scenario: Indicator shown in extraction mode
|
||||
|
||||
- **WHEN** the conversation is in extraction mode
|
||||
- **THEN** a visual indicator (e.g., chip, banner, or subtitle) is visible showing the extraction context
|
||||
|
||||
#### Scenario: Indicator hidden in general mode
|
||||
|
||||
- **WHEN** the conversation is in general chat mode
|
||||
- **THEN** no extraction indicator is shown
|
||||
|
||||
### Requirement: Follow-up messages route to extraction endpoint
|
||||
|
||||
In extraction mode, text messages typed by the user SHALL be sent to the extraction endpoint with the original email HTML and full conversation history, not to the general chat endpoint.
|
||||
|
||||
#### Scenario: User replies to disambiguation question
|
||||
|
||||
- **WHEN** the agent asks "Which legal entity?" and the user types "1"
|
||||
- **THEN** the client sends an ExtractionRequest with the original email HTML plus all messages (assistant question + user reply) to `POST /api/chat/extract`
|
||||
|
||||
### Requirement: Email upload message in chat
|
||||
|
||||
When an email is uploaded, the chat SHALL display a user message indicating the upload (e.g., showing the filename) before the extraction response streams in.
|
||||
|
||||
#### Scenario: Upload message displayed
|
||||
|
||||
- **WHEN** the user drops "trade_request.html"
|
||||
- **THEN** a user message appears in the chat like "[Uploaded: trade_request.html]" followed by the streaming extraction response
|
||||
41
openspec/changes/archive/2026-04-06-email-upload-ux/tasks.md
Normal file
41
openspec/changes/archive/2026-04-06-email-upload-ux/tasks.md
Normal file
@@ -0,0 +1,41 @@
|
||||
## 1. Drag-and-Drop Infrastructure
|
||||
|
||||
- [x] 1.1 Add JS interop function for drag-and-drop file reading — a small JS function that listens for `dragover`/`drop` events on a given element, reads the dropped file as text, and invokes a .NET callback with the filename and content
|
||||
- [x] 1.2 Add the JS interop script to `wwwroot/index.html` or a separate `.js` file referenced there
|
||||
- [x] 1.3 Wire the drag-and-drop JS interop to the `.message-list` element in Chat.razor — register on `OnAfterRenderAsync`, dispose on component disposal
|
||||
|
||||
## 2. File Upload Button
|
||||
|
||||
- [x] 2.1 Add `MudIconButton` with attachment icon next to the send button in the input area
|
||||
- [x] 2.2 Add hidden `InputFile` component accepting `.html` files, triggered by the icon button click
|
||||
- [x] 2.3 Handle `InputFile.OnChange` — read the selected file content as string, trigger extraction
|
||||
|
||||
## 3. Drop Zone Visual Feedback
|
||||
|
||||
- [x] 3.1 Add `_isDragOver` boolean state to Chat.razor, toggled by dragenter/dragleave events from JS interop
|
||||
- [x] 3.2 Add CSS class `.drag-over` to `.message-list` when `_isDragOver` is true — highlighted border, subtle overlay with "Drop email here" text
|
||||
- [x] 3.3 Add `.drag-over` styles to Chat.razor.css
|
||||
|
||||
## 4. Extraction Mode and Routing
|
||||
|
||||
- [x] 4.1 Add `_isExtractionMode` boolean and `_emailHtml` string fields to Chat.razor
|
||||
- [x] 4.2 When an email file is read (via drop or file picker): set `_isExtractionMode = true`, store email HTML in `_emailHtml`, add a user message showing "[Uploaded: filename.html]"
|
||||
- [x] 4.3 Create `SendExtractionMessage()` method — builds `ExtractionRequest` with `_emailHtml` and conversation messages, calls `ChatApiClient.SendExtractionStreamingAsync()`, streams response into assistant message (same pattern as `SendMessage()`)
|
||||
- [x] 4.4 Modify `SendMessage()` — if `_isExtractionMode`, build `ExtractionRequest` with `_emailHtml` + all messages and call the extraction endpoint instead of the chat endpoint
|
||||
- [x] 4.5 On file drop/upload: call `SendExtractionMessage()` for the initial extraction
|
||||
- [x] 4.6 Modify `NewChat()` to reset `_isExtractionMode = false` and `_emailHtml = ""`
|
||||
|
||||
## 5. Extraction Mode Indicator
|
||||
|
||||
- [x] 5.1 Add a `MudChip` or small banner below the tab header showing "Extraction Mode" when `_isExtractionMode` is true
|
||||
- [x] 5.2 Style the indicator in Chat.razor.css
|
||||
|
||||
## 6. Guard Rails
|
||||
|
||||
- [x] 6.1 Reject non-.html files in both drop handler and InputFile handler — show a snackbar or inline message
|
||||
- [x] 6.2 Disable drop zone and file picker during streaming (`_isStreaming` flag)
|
||||
|
||||
## 7. Build and Verify
|
||||
|
||||
- [x] 7.1 Build the solution (`dotnet build`) and confirm no compilation errors
|
||||
- [x] 7.2 Run all tests (`dotnet test`) and confirm they pass
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-05
|
||||
@@ -0,0 +1,51 @@
|
||||
## Context
|
||||
|
||||
The chat page currently has a single-panel layout: message list + input. The system prompt is absent (no system message in ChatHistory), and model parameters like temperature use Semantic Kernel defaults. For prompt engineering and debugging, these need to be editable in the UI without restarting the server.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Tabbed UI: Chat, System Prompt, Model Settings — all on the same page
|
||||
- System prompt and model settings sent with each chat request
|
||||
- Backend applies them to SK's ChatHistory and OpenAIPromptExecutionSettings
|
||||
- Settings persist in the browser session (survive tab switches, not page reloads)
|
||||
|
||||
**Non-Goals:**
|
||||
- Persisting settings to disk or server (future — save/load prompt profiles)
|
||||
- Phase 2 prompt templates and few-shot examples (scoped in proposal, not implemented here)
|
||||
- Changing the SSE streaming contract
|
||||
|
||||
## Decisions
|
||||
|
||||
### Tabbed layout using MudTabs
|
||||
- Use `MudTabs` with `MudTabPanel` for each section: Chat, System Prompt, Model Settings
|
||||
- **Alternative considered**: MudDrawer panels or separate pages. Rejected because tabs keep everything on one page — switching between prompt and chat should be instant with no navigation.
|
||||
- The Chat tab contains the existing message list and input (unchanged)
|
||||
- System Prompt tab: a `MudTextField` with `Lines="10"` for multi-line editing
|
||||
- Model Settings tab: `MudNumericField` or `MudSlider` for Temperature (0.0–2.0), TopP (0.0–1.0), MaxTokens (1–4096)
|
||||
|
||||
### Settings sent per-request, not stored server-side
|
||||
- `ChatRequest` gains optional `SystemPrompt` (string?) and `Settings` (ModelSettings?) properties
|
||||
- Backend treats them as nullable — if absent, defaults apply (no system prompt, SK default temperature)
|
||||
- This keeps the API stateless and avoids server-side session management
|
||||
- **Alternative considered**: Server-side settings endpoint. Rejected — adds complexity for a single-user debugging tool.
|
||||
|
||||
### ModelSettings as a shared DTO
|
||||
- New `ModelSettings.cs` in Shared/Models with `Temperature` (double?), `TopP` (double?), `MaxTokens` (int?)
|
||||
- All fields nullable — only set values override defaults
|
||||
- Maps directly to `OpenAIPromptExecutionSettings` properties on the backend
|
||||
|
||||
### System prompt applied as ChatHistory system message
|
||||
- `chatHistory.AddSystemMessage(request.SystemPrompt)` as the first entry before user/assistant messages
|
||||
- SK and OpenAI APIs treat the system message as behavioral instructions for the model
|
||||
|
||||
### Tab state persists in component fields
|
||||
- `_systemPrompt` and `_modelSettings` are component-level fields, not per-tab
|
||||
- Switching tabs doesn't reset values (MudTabs preserves panel content by default)
|
||||
- Values are lost on page refresh — acceptable for a debugging tool
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **[Tab switch loses scroll position]** → MudTabs renders all panels but hides inactive ones, so scroll position in the chat tab is preserved
|
||||
- **[Large system prompts inflate request size]** → Acceptable for single-user debugging; no size limit enforced
|
||||
- **[Temperature/TopP interaction]** → Standard OpenAI behavior: setting both is allowed but not recommended. Show a note in the UI, don't enforce.
|
||||
@@ -0,0 +1,35 @@
|
||||
## Why
|
||||
|
||||
When testing and debugging the AI chat agent, the system prompt and model parameters (temperature, top-p, max tokens) are hardcoded or absent. Exposing these in the UI lets the developer iterate on prompt engineering without restarting the server, and makes the app useful as a prompt testing workbench.
|
||||
|
||||
## What Changes
|
||||
|
||||
### Phase 1: Expose system prompt and model parameters
|
||||
- Add tabbed UI to the chat page: **Chat** tab (existing conversation), **System Prompt** tab (editable text area), **Model Settings** tab (temperature, top-p, max tokens sliders/inputs)
|
||||
- Extend the API contract: `ChatRequest` gains optional `SystemPrompt` and `ModelSettings` fields
|
||||
- Backend applies the system prompt as the first message in ChatHistory and passes model settings to execution settings
|
||||
|
||||
### Phase 2: Prompt templates with few-shot examples (future)
|
||||
- System prompt becomes a template with placeholder variables
|
||||
- UI for adding few-shot input/output example pairs
|
||||
- Template engine generates the final system prompt from template + examples
|
||||
- *Phase 2 is scoped but NOT implemented in this change*
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `prompt-settings-ui`: Tabbed interface for system prompt editing and model parameter controls
|
||||
- `prompt-settings-api`: API contract extensions for system prompt and model parameters
|
||||
|
||||
### Modified Capabilities
|
||||
- `chat-ui`: Chat page changes from single-panel to tabbed layout
|
||||
- `chat-streaming`: API accepts optional system prompt and model settings in the request
|
||||
|
||||
## Impact
|
||||
|
||||
- **ChatRequest.cs** (Shared): Add `SystemPrompt` and `ModelSettings` properties
|
||||
- **New ModelSettings.cs** (Shared): Temperature, TopP, MaxTokens model
|
||||
- **ChatController.cs** (API): Apply system prompt to ChatHistory, pass model settings to execution settings
|
||||
- **Chat.razor** (Client): Wrap in MudTabs, add System Prompt and Model Settings tab panels
|
||||
- **ChatApiClient.cs** (Client): Pass new fields in requests
|
||||
- No new packages — MudTabs, MudTextField, MudSlider are all part of MudBlazor
|
||||
@@ -0,0 +1,20 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Chat endpoint proxies to Responses API
|
||||
|
||||
The API backend SHALL expose `POST /api/chat` that accepts a `ChatRequest` containing messages, an optional system prompt, and optional model settings. The request is processed using a Semantic Kernel chat completion service. When a system prompt is provided, it SHALL be added as the first system message in the ChatHistory. When model settings are provided, non-null values SHALL be applied to the execution settings.
|
||||
|
||||
#### Scenario: Successful chat request with system prompt
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with messages and a system prompt
|
||||
- **THEN** the API creates a ChatHistory with the system prompt as the first message, followed by the conversation messages, and processes them through Semantic Kernel
|
||||
|
||||
#### Scenario: Successful chat request with model settings
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with messages and model settings (e.g., Temperature=0.3)
|
||||
- **THEN** the API applies the settings to OpenAIPromptExecutionSettings before calling the Semantic Kernel
|
||||
|
||||
#### Scenario: Successful chat request without optional fields
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with only messages (no system prompt, no settings)
|
||||
- **THEN** the API processes the request with default behavior (no system message, default execution settings)
|
||||
@@ -0,0 +1,15 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Chat page is default route
|
||||
|
||||
The chat page SHALL be routed at `/sales-assistant` (or `/` with redirect). The page content SHALL be wrapped in a MudTabs container with the conversation UI in the first tab panel.
|
||||
|
||||
#### Scenario: Page loads with chat tab active
|
||||
|
||||
- **WHEN** the user navigates to the chat page
|
||||
- **THEN** the Chat tab is active showing the message list and input area
|
||||
|
||||
#### Scenario: Chat functionality unchanged
|
||||
|
||||
- **WHEN** the user sends a message from the Chat tab
|
||||
- **THEN** the assistant response streams in exactly as before, with the same SSE contract and rendering behavior
|
||||
@@ -0,0 +1,43 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: ModelSettings shared model
|
||||
|
||||
The Shared project SHALL define a `ModelSettings` class with nullable properties: `Temperature` (double?), `TopP` (double?), `MaxTokens` (int?). Null values indicate "use server default".
|
||||
|
||||
#### Scenario: All fields null
|
||||
|
||||
- **WHEN** a ModelSettings instance has all null fields
|
||||
- **THEN** the backend uses Semantic Kernel default values for all parameters
|
||||
|
||||
#### Scenario: Partial override
|
||||
|
||||
- **WHEN** a ModelSettings instance has Temperature set but TopP and MaxTokens null
|
||||
- **THEN** only Temperature is overridden; other parameters use defaults
|
||||
|
||||
### Requirement: System prompt in chat request
|
||||
|
||||
The `ChatRequest` SHALL accept an optional `SystemPrompt` (string?) property. When present and non-empty, the backend SHALL insert it as the first system message in the ChatHistory before user/assistant messages.
|
||||
|
||||
#### Scenario: System prompt provided
|
||||
|
||||
- **WHEN** a ChatRequest includes a non-empty SystemPrompt
|
||||
- **THEN** the ChatHistory starts with a system message containing that text, followed by the conversation messages
|
||||
|
||||
#### Scenario: System prompt absent
|
||||
|
||||
- **WHEN** a ChatRequest has a null or empty SystemPrompt
|
||||
- **THEN** the ChatHistory contains only user and assistant messages (no system message)
|
||||
|
||||
### Requirement: Model settings in chat request
|
||||
|
||||
The `ChatRequest` SHALL accept an optional `Settings` (ModelSettings?) property. When present, the backend SHALL apply non-null values to `OpenAIPromptExecutionSettings` before calling the Semantic Kernel.
|
||||
|
||||
#### Scenario: Temperature override
|
||||
|
||||
- **WHEN** a ChatRequest includes Settings with Temperature = 0.5
|
||||
- **THEN** the OpenAIPromptExecutionSettings.Temperature is set to 0.5
|
||||
|
||||
#### Scenario: No settings provided
|
||||
|
||||
- **WHEN** a ChatRequest has null Settings
|
||||
- **THEN** the backend uses default OpenAIPromptExecutionSettings (only FunctionChoiceBehavior.Auto is set)
|
||||
@@ -0,0 +1,53 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: System prompt editor tab
|
||||
|
||||
The chat page SHALL include a "System Prompt" tab with a multi-line text area where the user can enter a system prompt. The system prompt value SHALL persist across tab switches within the same session.
|
||||
|
||||
#### Scenario: User enters a system prompt
|
||||
|
||||
- **WHEN** the user navigates to the System Prompt tab and types text
|
||||
- **THEN** the text is stored in the component state and included in the next chat request
|
||||
|
||||
#### Scenario: System prompt survives tab switch
|
||||
|
||||
- **WHEN** the user enters a system prompt, switches to the Chat tab, then switches back
|
||||
- **THEN** the system prompt text is unchanged
|
||||
|
||||
### Requirement: Model settings tab
|
||||
|
||||
The chat page SHALL include a "Model Settings" tab with controls for Temperature, TopP, and MaxTokens. Each control SHALL display its current value and allow adjustment within valid ranges.
|
||||
|
||||
#### Scenario: Temperature control
|
||||
|
||||
- **WHEN** the user adjusts the Temperature control
|
||||
- **THEN** the value is constrained to 0.0–2.0 and included in the next chat request's settings
|
||||
|
||||
#### Scenario: TopP control
|
||||
|
||||
- **WHEN** the user adjusts the TopP control
|
||||
- **THEN** the value is constrained to 0.0–1.0 and included in the next chat request's settings
|
||||
|
||||
#### Scenario: MaxTokens control
|
||||
|
||||
- **WHEN** the user sets the MaxTokens value
|
||||
- **THEN** the value is constrained to 1–4096 and included in the next chat request's settings
|
||||
|
||||
#### Scenario: Default values
|
||||
|
||||
- **WHEN** the user has not changed any model settings
|
||||
- **THEN** the controls show default values (Temperature: 1.0, TopP: 1.0, MaxTokens: empty/unset) and no overrides are sent to the API
|
||||
|
||||
### Requirement: Tabbed page layout
|
||||
|
||||
The chat page SHALL use MudTabs with three tab panels: "Chat" (the existing conversation UI), "System Prompt" (the prompt editor), and "Model Settings" (the parameter controls).
|
||||
|
||||
#### Scenario: Chat tab is default
|
||||
|
||||
- **WHEN** the page loads
|
||||
- **THEN** the Chat tab is active and the conversation UI is displayed
|
||||
|
||||
#### Scenario: Tab switching
|
||||
|
||||
- **WHEN** the user clicks a different tab
|
||||
- **THEN** the corresponding panel is displayed and the previous panel is hidden but retains its state
|
||||
@@ -0,0 +1,40 @@
|
||||
## 1. Shared Models
|
||||
|
||||
- [x] 1.1 Create `ModelSettings.cs` in Shared/Models — `double? Temperature`, `double? TopP`, `int? MaxTokens`
|
||||
- [x] 1.2 Add `string? SystemPrompt` and `ModelSettings? Settings` properties to `ChatRequest.cs`
|
||||
|
||||
## 2. API Backend
|
||||
|
||||
- [x] 2.1 Update `ChatController.Post()` — if `request.SystemPrompt` is non-empty, call `chatHistory.AddSystemMessage(request.SystemPrompt)` before adding user/assistant messages
|
||||
- [x] 2.2 Update `ChatController.Post()` — if `request.Settings` is non-null, apply non-null Temperature, TopP, MaxTokens to `OpenAIPromptExecutionSettings`
|
||||
|
||||
## 3. Client UI — Tabbed Layout
|
||||
|
||||
- [x] 3.1 Wrap the existing Chat.razor content (chat-container div) inside `<MudTabs>` with three `<MudTabPanel>` elements: "Chat", "System Prompt", "Model Settings"
|
||||
- [x] 3.2 Add component fields: `_systemPrompt` (string), `_temperature` (double?), `_topP` (double?), `_maxTokens` (int?)
|
||||
|
||||
## 4. System Prompt Tab
|
||||
|
||||
- [x] 4.1 Add a `MudTextField` with `Lines="10"`, `Variant="Variant.Outlined"`, bound to `_systemPrompt`, with placeholder text explaining what a system prompt does
|
||||
- [x] 4.2 Include the `_systemPrompt` value in the `ChatRequest` built by `SendMessage()`
|
||||
|
||||
## 5. Model Settings Tab
|
||||
|
||||
- [x] 5.1 Add `MudNumericField<double?>` for Temperature (min 0.0, max 2.0, step 0.1) with label and helper text
|
||||
- [x] 5.2 Add `MudNumericField<double?>` for TopP (min 0.0, max 1.0, step 0.1) with label and helper text
|
||||
- [x] 5.3 Add `MudNumericField<int?>` for MaxTokens (min 1, max 4096) with label and helper text
|
||||
- [x] 5.4 Include the model settings in the `ChatRequest` built by `SendMessage()`
|
||||
|
||||
## 6. Client Service
|
||||
|
||||
- [x] 6.1 Verify `ChatApiClient.SendChatStreamingAsync()` serializes the new `ChatRequest` fields correctly (SystemPrompt, Settings) — no changes expected since it already serializes the full object
|
||||
|
||||
## 7. Styling
|
||||
|
||||
- [x] 7.1 Adjust `Chat.razor.css` — the chat-container height calc needs to account for the MudTabs header height (~48px). The tabs header sits inside the content area below the AppBar.
|
||||
|
||||
## 8. Verification
|
||||
|
||||
- [x] 8.1 Build the solution (`dotnet build`) and confirm no compilation errors
|
||||
- [x] 8.2 Run existing tests (`dotnet test`) and confirm they pass
|
||||
- [x] 8.3 Update any tests that construct `ChatRequest` if the new nullable fields cause issues — no updates needed, nullable fields don't break existing tests
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-06
|
||||
@@ -0,0 +1,69 @@
|
||||
## Context
|
||||
|
||||
The extraction agent (Semantic Kernel with auto-invoke tools) needs few-shot examples to reliably map sales emails to TradeItem JSON. The `update-extraction-schema` change provides the real schema and tools. This change adds the prompting infrastructure and a dedicated endpoint.
|
||||
|
||||
Up to 100 input/output example pairs are available. Research shows 3-5 well-chosen examples are optimal for few-shot prompting — more can degrade performance by consuming context and introducing noise.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Load curated few-shot examples from disk and assemble a reusable ChatHistory prefix
|
||||
- Provide a fixed instruction template for extraction (not user-editable)
|
||||
- Create a dedicated extraction endpoint with the correct prompt and tools
|
||||
- Keep the general chat endpoint unchanged
|
||||
|
||||
**Non-Goals:**
|
||||
- Dynamic example selection (RAG-like similarity matching) — future enhancement
|
||||
- Email upload UI (separate change: `email-upload-ux`)
|
||||
- Building or curating the actual example content (user provides these)
|
||||
- Evaluation pipeline for the ~95 non-few-shot examples (future work)
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Examples as conversation turns (not system prompt string)
|
||||
|
||||
**Decision:** Inject few-shot examples as alternating User/Assistant messages in the ChatHistory, after the system message and before the real email.
|
||||
|
||||
**Why:** Chat models treat conversation turns as prior context — the model "sees" the examples as things it already did correctly. This is more effective than embedding examples in the system prompt string, where they're treated as instructions rather than demonstrated behavior.
|
||||
|
||||
### 2. Examples loaded once at startup, cached as ChatHistory prefix
|
||||
|
||||
**Decision:** `FewShotService` reads example files at startup, builds a `ChatHistory` prefix (system message + example turns), and caches it as a singleton. Each extraction request clones this prefix and appends the real email.
|
||||
|
||||
**Why:** Example files don't change at runtime. Loading once avoids repeated disk I/O. Cloning the cached prefix is cheap (ChatHistory is a list of message objects).
|
||||
|
||||
**Alternative considered:** Load examples per-request. Rejected — unnecessary I/O for static data.
|
||||
|
||||
### 3. Instruction template as an embedded text file
|
||||
|
||||
**Decision:** Store the extraction instruction template as a text file at `examples/extraction/instruction-template.txt`, loaded by FewShotService alongside the examples.
|
||||
|
||||
**Why:** Keeps the prompt text editable without recompilation. Co-located with the examples it references. Not in appsettings.json because it's multi-line prose, not configuration.
|
||||
|
||||
### 4. Separate extraction endpoint, not a mode flag on /api/chat
|
||||
|
||||
**Decision:** `POST /api/chat/extract` as a new controller action, separate from `POST /api/chat`.
|
||||
|
||||
**Why:** The extraction path uses a completely different ChatHistory (few-shot prefix, not user system prompt), different tools (extraction plugins only), and a different request DTO. A mode flag on the existing endpoint would add branching complexity. Separate endpoints make each path clear.
|
||||
|
||||
**Alternative considered:** Mode flag on ChatRequest (e.g. `"mode": "extract"`). Rejected — the request shapes diverge enough to warrant separate DTOs and endpoints.
|
||||
|
||||
### 5. ExtractionRequest includes conversation messages for follow-up
|
||||
|
||||
**Decision:** `ExtractionRequest` contains `EmailHtml` (string, the email to extract) plus `Messages` (list, optional follow-up conversation for disambiguation).
|
||||
|
||||
**Why:** After initial extraction, the agent may ask disambiguation questions. Follow-up user replies need to be sent back with the full conversation context so the agent can continue. The first request has only `EmailHtml`; subsequent requests include the growing `Messages` list.
|
||||
|
||||
### 6. Example folder uses numbered subdirectories
|
||||
|
||||
**Decision:** `examples/extraction/few-shot/01/input.html + output.json`, `02/`, etc.
|
||||
|
||||
**Why:** Numbered prefixes control ordering in the ChatHistory. Each subdirectory is one example, keeping input and output together. Easy to add/remove/reorder examples by renaming directories.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[Example quality determines extraction quality]** → Poorly chosen few-shot examples will mislead the model. Mitigation: document selection criteria (diversity of swap structures, currencies, breakclause values). The user curates the 3-5 examples.
|
||||
|
||||
**[Instruction template drift]** → If the schema changes, the instruction template must be updated manually. Mitigation: the template references the TradeItem field names explicitly, making it obvious when they're out of sync.
|
||||
|
||||
**[ChatHistory size with few-shot examples]** → Each example adds ~2-5KB of tokens. With 5 examples that's ~10-25KB, well within model context limits. Not a risk at current scale but would be if dynamic selection adds more examples later.
|
||||
@@ -0,0 +1,31 @@
|
||||
## Why
|
||||
|
||||
The extraction agent needs few-shot examples to reliably produce correct structured output from sales emails. Without examples, the agent relies entirely on the instruction template and tool descriptions, which cannot fully convey the implicit mapping conventions (date parsing from "OB" prefix, flattening swap legs, currency symbol mapping, breakclause defaults). A curated set of 3-5 input/output examples injected as conversation turns in the ChatHistory dramatically improves extraction accuracy. The remaining ~95 available examples serve as an evaluation set for offline quality testing.
|
||||
|
||||
Additionally, the extraction workflow needs a dedicated API endpoint separate from general chat, since it uses a different system prompt, different tools, and the few-shot ChatHistory prefix.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **Create examples folder structure** at `examples/extraction/few-shot/` with numbered subdirectories, each containing `input.html` (email) and `output.json` (expected ExtractionResult)
|
||||
- **Create extraction instruction template** — a fixed system prompt defining the extraction task, schema, and mapping rules (separate from the user-editable system prompt)
|
||||
- **Create a FewShotService** that loads examples from disk at startup and pre-assembles a ChatHistory prefix (system message + alternating user/assistant turns)
|
||||
- **Add `POST /api/chat/extract` endpoint** that uses the few-shot ChatHistory, appends the real email, and streams the extraction response via SSE
|
||||
- **Create `ExtractionRequest` DTO** for the extraction endpoint (email content + optional follow-up messages for disambiguation)
|
||||
- **Update client `ChatApiClient`** with a method for the extraction endpoint
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `few-shot-prompting`: Defines the example folder structure, loading mechanism, ChatHistory assembly, and instruction template for few-shot extraction prompting
|
||||
- `extraction-endpoint`: Defines the dedicated extraction API endpoint, its request/response contract, and how it differs from the general chat endpoint
|
||||
|
||||
### Modified Capabilities
|
||||
- `chat-streaming`: Add the extraction endpoint alongside the existing chat endpoint, sharing the same SSE streaming contract
|
||||
|
||||
## Impact
|
||||
|
||||
- **New files**: examples folder, FewShotService, instruction template, ExtractionRequest DTO, extraction controller action
|
||||
- **Configuration**: example folder path in appsettings.json
|
||||
- **API surface**: new `POST /api/chat/extract` endpoint
|
||||
- **Client**: new method on ChatApiClient (no UI changes — that's the email-upload-ux change)
|
||||
- **Depends on**: `update-extraction-schema` (needs TradeItem schema for examples and validation tools)
|
||||
@@ -0,0 +1,25 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Chat endpoint proxies to Responses API
|
||||
|
||||
The API backend SHALL expose `POST /api/chat` that accepts a `ChatRequest` containing messages, an optional system prompt, and optional model settings. The request is processed using a Semantic Kernel chat completion service. When a system prompt is provided, it SHALL be added as the first system message in the ChatHistory. When model settings are provided, non-null values SHALL be applied to the execution settings. A separate `POST /api/chat/extract` endpoint SHALL handle extraction-specific requests with few-shot prompting.
|
||||
|
||||
#### Scenario: Successful chat request with system prompt
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with messages and a system prompt
|
||||
- **THEN** the API creates a ChatHistory with the system prompt as the first message, followed by the conversation messages, and processes them through Semantic Kernel
|
||||
|
||||
#### Scenario: Successful chat request with model settings
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with messages and model settings (e.g., Temperature=0.3)
|
||||
- **THEN** the API applies the settings to OpenAIPromptExecutionSettings before calling the Semantic Kernel
|
||||
|
||||
#### Scenario: Successful chat request without optional fields
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with only messages (no system prompt, no settings)
|
||||
- **THEN** the API processes the request with default behavior (no system message, default execution settings)
|
||||
|
||||
#### Scenario: Extraction request routed to dedicated endpoint
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat/extract` with email HTML
|
||||
- **THEN** the API uses the few-shot ChatHistory prefix and extraction tools instead of the general chat configuration
|
||||
@@ -0,0 +1,43 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Extraction API endpoint
|
||||
|
||||
The API SHALL expose `POST /api/chat/extract` that accepts an `ExtractionRequest` containing the email HTML content and optional follow-up conversation messages. The endpoint SHALL use the few-shot ChatHistory prefix (not the user-editable system prompt) and load extraction-specific SK plugins.
|
||||
|
||||
#### Scenario: Initial extraction request
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat/extract` with email HTML and no follow-up messages
|
||||
- **THEN** the API assembles the few-shot ChatHistory, appends the email as the final user message, and streams the extraction response via SSE
|
||||
|
||||
#### Scenario: Follow-up disambiguation request
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat/extract` with email HTML and follow-up messages (e.g., user selecting a counterparty)
|
||||
- **THEN** the API assembles the few-shot ChatHistory, appends the email, appends all follow-up messages, and streams the continuation response via SSE
|
||||
|
||||
#### Scenario: SSE streaming contract
|
||||
|
||||
- **WHEN** the extraction endpoint streams a response
|
||||
- **THEN** it uses the same SSE format as `/api/chat`: `data: {"text":"..."}\n\n` for deltas and `data: [DONE]\n\n` for completion
|
||||
|
||||
### Requirement: ExtractionRequest DTO
|
||||
|
||||
The system SHALL define an `ExtractionRequest` class with `EmailHtml` (string, required) and `Messages` (List<ChatMessage>, optional) for follow-up conversation context.
|
||||
|
||||
#### Scenario: First request has email only
|
||||
|
||||
- **WHEN** the user uploads an email for the first time
|
||||
- **THEN** the ExtractionRequest contains `EmailHtml` with the email content and an empty `Messages` list
|
||||
|
||||
#### Scenario: Follow-up request includes conversation
|
||||
|
||||
- **WHEN** the user replies to a disambiguation question
|
||||
- **THEN** the ExtractionRequest contains the original `EmailHtml` plus `Messages` with the full assistant/user exchange since the extraction started
|
||||
|
||||
### Requirement: Extraction endpoint uses extraction tools only
|
||||
|
||||
The extraction endpoint SHALL import only the extraction-specific SK plugins (counterparty lookup, trade validation, currency validation, schema validation). General chat tools (if any) SHALL NOT be loaded for extraction requests.
|
||||
|
||||
#### Scenario: Tool isolation
|
||||
|
||||
- **WHEN** the extraction endpoint processes a request
|
||||
- **THEN** only extraction-related KernelFunctions are available to the LLM
|
||||
@@ -0,0 +1,52 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Few-shot example folder structure
|
||||
|
||||
The system SHALL store few-shot examples at `examples/extraction/few-shot/` with numbered subdirectories (e.g., `01/`, `02/`). Each subdirectory SHALL contain `input.html` (the example email) and `output.json` (the expected ExtractionResult JSON).
|
||||
|
||||
#### Scenario: Example folder layout
|
||||
|
||||
- **WHEN** the application starts
|
||||
- **THEN** it reads example pairs from `examples/extraction/few-shot/` in numeric directory order
|
||||
|
||||
#### Scenario: Adding a new example
|
||||
|
||||
- **WHEN** a new subdirectory (e.g., `04/`) is added with `input.html` and `output.json`
|
||||
- **THEN** the new example is included in the few-shot ChatHistory prefix after the next application restart
|
||||
|
||||
### Requirement: Extraction instruction template
|
||||
|
||||
The system SHALL load a fixed instruction template from `examples/extraction/instruction-template.txt` that defines the extraction task, the TradeItem schema, and the mapping rules (date parsing, leg flattening, currency mapping, breakclause defaults). This template is NOT the user-editable system prompt.
|
||||
|
||||
#### Scenario: Template loaded at startup
|
||||
|
||||
- **WHEN** the application starts
|
||||
- **THEN** the instruction template is loaded from disk and used as the system message in the extraction ChatHistory
|
||||
|
||||
#### Scenario: Template content
|
||||
|
||||
- **WHEN** the instruction template is loaded
|
||||
- **THEN** it contains the TradeItem field definitions, expected JSON output format, and explicit mapping rules
|
||||
|
||||
### Requirement: ChatHistory assembly with few-shot examples
|
||||
|
||||
The system SHALL provide a `FewShotService` that assembles a reusable ChatHistory prefix at startup: the instruction template as a system message, followed by alternating User (input.html) and Assistant (output.json) messages for each example. Each extraction request SHALL clone this prefix and append the real email as the final user message.
|
||||
|
||||
#### Scenario: ChatHistory prefix structure
|
||||
|
||||
- **WHEN** the service assembles the prefix with 3 examples
|
||||
- **THEN** the ChatHistory contains: 1 system message + 3 user messages + 3 assistant messages (7 messages total)
|
||||
|
||||
#### Scenario: Prefix cached and cloned per request
|
||||
|
||||
- **WHEN** an extraction request arrives
|
||||
- **THEN** the service clones the cached prefix (not re-reading from disk) and appends the email content as a new user message
|
||||
|
||||
### Requirement: Evaluation example folder
|
||||
|
||||
The system SHALL support an `examples/extraction/evaluation/` folder for bulk examples used in offline testing. This folder is NOT loaded at startup and NOT used in the few-shot prompt.
|
||||
|
||||
#### Scenario: Evaluation folder ignored at runtime
|
||||
|
||||
- **WHEN** the application starts
|
||||
- **THEN** it does not load examples from `examples/extraction/evaluation/`
|
||||
@@ -0,0 +1,36 @@
|
||||
## 1. Example Folder Structure
|
||||
|
||||
- [x] 1.1 Create directory structure: `examples/extraction/few-shot/` and `examples/extraction/evaluation/`
|
||||
- [x] 1.2 Add placeholder example `01/` with `input.html` (sample email HTML) and `output.json` (sample ExtractionResult JSON matching TradeItem schema). Use realistic but anonymized data.
|
||||
- [x] 1.3 Add placeholder example `02/` with a different email pattern (e.g., single swap, different currency)
|
||||
- [x] 1.4 Add placeholder example `03/` with an edge case (e.g., breakclause = "Y", or unusual counterparty name)
|
||||
|
||||
## 2. Instruction Template
|
||||
|
||||
- [x] 2.1 Create `examples/extraction/instruction-template.txt` with the fixed extraction system prompt: task description, TradeItem schema definition (all 7 fields with types), mapping rules (date parsing, leg flattening, currency symbol → ISO code, breakclause default), and expected JSON output format
|
||||
|
||||
## 3. FewShotService
|
||||
|
||||
- [x] 3.1 Create `FewShotService.cs` in the API project — constructor loads instruction template and all few-shot examples from disk, assembles a ChatHistory prefix (system message + alternating user/assistant turns)
|
||||
- [x] 3.2 Add `CloneWithEmail(string emailHtml)` method that clones the cached ChatHistory prefix and appends the email as a user message
|
||||
- [x] 3.3 Add `CloneWithEmailAndMessages(string emailHtml, List<ChatMessage> messages)` method for follow-up disambiguation requests — clones prefix, appends email, appends follow-up messages
|
||||
- [x] 3.4 Register `FewShotService` as singleton in `Program.cs`
|
||||
- [x] 3.5 Add `Examples:FewShotPath` configuration in `appsettings.json` pointing to the examples folder
|
||||
|
||||
## 4. ExtractionRequest DTO
|
||||
|
||||
- [x] 4.1 Create `ExtractionRequest.cs` in Shared/Models with `EmailHtml` (string, required) and `Messages` (List<ChatMessage>, optional)
|
||||
|
||||
## 5. Extraction Endpoint
|
||||
|
||||
- [x] 5.1 Add `Extract` action to `ChatController` (or new `ExtractionController`) — `POST /api/chat/extract` accepting `ExtractionRequest`
|
||||
- [x] 5.2 In the Extract action: get ChatHistory from `FewShotService.CloneWithEmail()` or `CloneWithEmailAndMessages()` based on whether Messages are present
|
||||
- [x] 5.3 Import extraction-specific plugins only (not general chat plugins)
|
||||
- [x] 5.4 Stream response via SSE using the same format as the existing chat endpoint
|
||||
- [x] 5.5 Update `ChatApiClient` on the client side — add `SendExtractionStreamingAsync(ExtractionRequest request)` method mirroring the existing streaming pattern
|
||||
|
||||
## 6. Build and Verify
|
||||
|
||||
- [x] 6.1 Build the solution (`dotnet build`) and confirm no compilation errors
|
||||
- [x] 6.2 Run all tests (`dotnet test`) and confirm they pass
|
||||
- [x] 6.3 Add unit test for `FewShotService` — verify it loads examples and assembles correct ChatHistory structure (message count, roles, ordering)
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-06
|
||||
@@ -0,0 +1,73 @@
|
||||
## Context
|
||||
|
||||
The extraction pipeline currently uses a placeholder schema (`ExtractedFields` with Client, Project, Hours, Rate, Currency, Date) and a single in-process validation method. The real domain is CVA trade extraction from sales emails: HTML emails containing swap/leg tables need to be parsed into an array of `TradeItem` objects. Validation and enrichment require calling 3-5 existing external APIs (counterparty lookup, trade validation, etc.), some of which return multiple candidates requiring user disambiguation.
|
||||
|
||||
The `ExtractionPlugin` is already registered with Semantic Kernel and auto-invoked via `FunctionChoiceBehavior.Auto()`. The tool calling loop and SSE streaming infrastructure are in place.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Replace placeholder models with the real TradeItem schema
|
||||
- Replace single validation method with multiple external API tool wrappers
|
||||
- Support disambiguation workflow where tool results return candidate lists
|
||||
- Keep the external API integration configurable and testable
|
||||
|
||||
**Non-Goals:**
|
||||
- Few-shot prompting infrastructure (separate change: `few-shot-prompt-infrastructure`)
|
||||
- Email upload/drag-drop UX (separate change: `email-upload-ux`)
|
||||
- Building the external APIs themselves (they already exist)
|
||||
- Changing the SSE streaming contract or client-side rendering
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Schema structure: wrapper with items array
|
||||
|
||||
**Decision:** Use `ExtractionResult { List<TradeItem> Items }` rather than returning a flat `TradeItem`.
|
||||
|
||||
**Why:** A single email typically contains multiple swaps, each with multiple legs. Each leg becomes a separate `TradeItem`. The wrapper allows returning all items from one extraction pass.
|
||||
|
||||
**Alternative considered:** Returning `List<TradeItem>` directly. Rejected because a wrapper object is more extensible (could add metadata like extraction confidence, email subject, etc. later) and produces cleaner JSON (`{"items": [...]}` vs bare array).
|
||||
|
||||
### 2. TradeItem uses snake_case JSON property names
|
||||
|
||||
**Decision:** Use `[JsonPropertyName("valuedate")]` etc. to produce snake_case JSON output.
|
||||
|
||||
**Why:** The existing external APIs and downstream consumers expect snake_case. The C# properties will use PascalCase per convention, with JSON attributes for serialization.
|
||||
|
||||
### 3. One plugin class with multiple methods vs multiple plugin classes
|
||||
|
||||
**Decision:** Single `ExtractionPlugin` class with 3-5 `[KernelFunction]` methods.
|
||||
|
||||
**Why:** All tools serve the same extraction workflow. SK discovers functions by class, so a single class keeps registration simple (`ImportPluginFromObject`). If tools grow beyond 6-7, split into focused plugin classes.
|
||||
|
||||
**Alternative considered:** Separate plugin class per external API. Rejected as premature — adds registration complexity for no benefit at 3-5 methods.
|
||||
|
||||
### 4. External API calls via typed HttpClients
|
||||
|
||||
**Decision:** Register typed `HttpClient` instances for each external API via `AddHttpClient<T>()` in `Program.cs`. Each plugin method receives its client via constructor injection.
|
||||
|
||||
**Why:** Typed HttpClients are the standard ASP.NET Core pattern. They support per-client base URL configuration, DI, and are easily mockable in tests.
|
||||
|
||||
### 5. Disambiguation via tool return values, not special UI
|
||||
|
||||
**Decision:** When a tool like `lookup_counterparty` finds multiple matches, it returns the candidate list as JSON. The LLM agent sees the candidates and asks the user to choose via natural language in the streamed response.
|
||||
|
||||
**Why:** This uses the existing streaming conversation infrastructure with zero client changes. The agent is already conversational — it simply says "I found 3 matches, which one?" and the user replies.
|
||||
|
||||
**Alternative considered:** Structured disambiguation UI (radio buttons, dropdowns). Rejected for this change — adds client complexity and a new response type. Can be added later as a UX enhancement.
|
||||
|
||||
### 6. ExtractionPlugin conditionally loaded per request
|
||||
|
||||
**Decision:** Continue importing extraction plugins per-request in the controller (`_kernel.ImportPluginFromObject`), not at startup.
|
||||
|
||||
**Why:** General chat requests don't need extraction tools. Per-request import keeps the tool list clean for non-extraction conversations, reducing token usage and avoiding confusing the LLM with irrelevant tools.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[External API availability]** → Plugin methods should handle HTTP errors gracefully and return structured error responses that the LLM can reason about (e.g., "Counterparty API unavailable, proceeding without legal entity lookup"). The agent can then inform the user.
|
||||
|
||||
**[Breaking change to ExtractedFields]** → Any code referencing the old schema breaks. Mitigation: this is a small codebase with known consumers. Update all references in the same change. Tests will catch missed spots.
|
||||
|
||||
**[LLM disambiguation quality]** → The agent must correctly interpret candidate lists and present clear choices. Mitigation: tool descriptions must be explicit about what the return values mean. Few-shot examples (next change) will reinforce the pattern.
|
||||
|
||||
**[External API response format coupling]** → Plugin methods parse external API responses. If those APIs change, plugins break. Mitigation: typed response DTOs with defensive deserialization. External APIs are stable and owned by the same team.
|
||||
@@ -0,0 +1,28 @@
|
||||
## Why
|
||||
|
||||
The current extraction schema (`ExtractedFields`) uses placeholder fields (Client, Project, Hours, Rate, Currency, Date) that don't match the real domain. The actual use case is CVA (Credit Valuation Adjustment) trade extraction from sales emails — parsing HTML emails into structured trade items with fields like counterparty, trade_id, pv, and legal_entity. The single `ExtractionPlugin.ValidateExtractedFields()` method also needs to be replaced with multiple tools that wrap existing external APIs for counterparty lookup, trade validation, and other checks.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **Replace `ExtractedFields.cs`** with real domain models: `ExtractionResult` (wrapper) and `TradeItem` (per-trade fields: valuedate, counterparty, legal_entity, trade_id, display_ccy, pv, breakclause)
|
||||
- **Replace `ExtractionPlugin.cs`** single validation method with 3-5 SK plugin methods, each wrapping an existing external API (counterparty lookup, trade validation, currency validation, schema validation)
|
||||
- **Update `ValidationResult.cs`** to support richer results — candidate lists for disambiguation, not just pass/fail
|
||||
- **Add typed HttpClients** for the external validation/lookup APIs, configured via `appsettings.json`
|
||||
- **Update existing tests** that reference the old `ExtractedFields` and `ExtractionPlugin`
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `extraction-schema`: Defines the real TradeItem schema, ExtractionResult wrapper, and the mapping rules from email content to structured output (date format, flattening swap legs, breakclause defaults)
|
||||
- `extraction-tools`: Defines the external API tool plugins — counterparty lookup (with disambiguation), trade validation, currency validation, and final schema validation
|
||||
|
||||
### Modified Capabilities
|
||||
- `agent-extraction`: Update requirements to reference the real schema (TradeItem) instead of generic "predefined fields", and add disambiguation workflow where tool results require user selection (e.g., counterparty/legal_entity tuples)
|
||||
|
||||
## Impact
|
||||
|
||||
- **Shared models**: `ExtractedFields.cs` replaced — **BREAKING** for any code referencing old fields
|
||||
- **API plugins**: `ExtractionPlugin.cs` rewritten with new method signatures — **BREAKING** for existing tool calling behavior
|
||||
- **External dependencies**: New HTTP calls to existing external APIs (counterparty, trade, currency)
|
||||
- **Configuration**: New `appsettings.json` entries for external API base URLs
|
||||
- **Tests**: Existing extraction-related tests need rewriting against new schema and tools
|
||||
@@ -0,0 +1,53 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Predefined extraction schema
|
||||
|
||||
The system SHALL define the extraction schema as a `TradeItem` class with fields: valuedate, counterparty, legal_entity, trade_id, display_ccy, pv, breakclause. Extraction output SHALL be wrapped in an `ExtractionResult` containing a `List<TradeItem>`. All extraction output MUST conform to this schema.
|
||||
|
||||
#### Scenario: Output conforms to schema
|
||||
|
||||
- **WHEN** the agent produces extracted fields from an email
|
||||
- **THEN** every item in the output is a valid TradeItem with all required fields matching expected types
|
||||
|
||||
#### Scenario: Multiple items from one email
|
||||
|
||||
- **WHEN** the agent extracts data from an email containing multiple trade legs
|
||||
- **THEN** the output ExtractionResult contains one TradeItem per trade leg
|
||||
|
||||
### Requirement: Autonomous validation via tool calling
|
||||
|
||||
The agent SHALL validate extracted fields by calling external API tools exposed as Semantic Kernel functions. Validation tools include counterparty lookup, trade validation, currency validation, and schema validation. Each tool returns structured results that the agent reasons about.
|
||||
|
||||
#### Scenario: Validation passes
|
||||
|
||||
- **WHEN** the agent calls the schema validation tool with a complete and correct ExtractionResult
|
||||
- **THEN** the tool returns a success result and the agent returns the final output to the user
|
||||
|
||||
#### Scenario: Validation fails with fixable errors
|
||||
|
||||
- **WHEN** a validation tool returns errors for missing or malformed fields
|
||||
- **THEN** the agent re-reads the source text and attempts to fix the extraction without user intervention
|
||||
|
||||
#### Scenario: Counterparty disambiguation required
|
||||
|
||||
- **WHEN** the counterparty lookup tool returns multiple candidate (counterparty, legal_entity) tuples
|
||||
- **THEN** the agent presents the candidates to the user as a numbered list in the chat and waits for the user to select one before completing the extraction
|
||||
|
||||
### Requirement: Human-in-the-loop clarification
|
||||
|
||||
When the agent escalates to the user, the user SHALL be able to provide the missing information in natural language, and the agent SHALL incorporate the clarification and re-attempt extraction. Disambiguation of counterparty/legal_entity tuples is a specific case of human-in-the-loop clarification.
|
||||
|
||||
#### Scenario: User provides clarification
|
||||
|
||||
- **WHEN** the agent asks for clarification about missing fields and the user responds
|
||||
- **THEN** the agent incorporates the user's response into the conversation context and produces an updated extraction
|
||||
|
||||
#### Scenario: User selects counterparty from candidates
|
||||
|
||||
- **WHEN** the agent presents a numbered list of counterparty/legal_entity candidates and the user replies with a selection
|
||||
- **THEN** the agent populates the `legal_entity` field on all relevant TradeItems and proceeds with validation
|
||||
|
||||
#### Scenario: Clarification via normal chat
|
||||
|
||||
- **WHEN** the agent escalates for clarification
|
||||
- **THEN** the clarification request appears as a regular assistant message in the chat UI, and the user responds via the normal chat input
|
||||
@@ -0,0 +1,63 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TradeItem schema
|
||||
|
||||
The system SHALL define a `TradeItem` class with the following fields representing a single trade leg extracted from a sales email:
|
||||
- `valuedate` (string, dd/MM/yyyy format)
|
||||
- `counterparty` (string, full legal name as it appears in the email)
|
||||
- `legal_entity` (string, nullable — populated after counterparty disambiguation via lookup tool)
|
||||
- `trade_id` (long, Murex trade identifier)
|
||||
- `display_ccy` (string, ISO currency code e.g. "GBP", "USD")
|
||||
- `pv` (double, present value)
|
||||
- `breakclause` (string, "Y" or "N")
|
||||
|
||||
JSON serialization SHALL use snake_case property names via `[JsonPropertyName]` attributes.
|
||||
|
||||
#### Scenario: All fields populated
|
||||
|
||||
- **WHEN** the extraction agent produces a TradeItem with all fields
|
||||
- **THEN** the JSON output contains all seven fields with snake_case keys and correct types
|
||||
|
||||
#### Scenario: Legal entity null before disambiguation
|
||||
|
||||
- **WHEN** the extraction agent produces a TradeItem before counterparty lookup
|
||||
- **THEN** the `legal_entity` field is null and all other fields are populated
|
||||
|
||||
### Requirement: ExtractionResult wrapper
|
||||
|
||||
The system SHALL define an `ExtractionResult` class containing a `List<TradeItem> Items` property. All extraction output from a single email SHALL be wrapped in this object.
|
||||
|
||||
#### Scenario: Single email with multiple trade legs
|
||||
|
||||
- **WHEN** an email contains two swaps with two legs each (4 trades total)
|
||||
- **THEN** the ExtractionResult contains an `items` array with 4 TradeItem objects
|
||||
|
||||
#### Scenario: JSON output structure
|
||||
|
||||
- **WHEN** the ExtractionResult is serialized to JSON
|
||||
- **THEN** the output has the shape `{"items": [{"valuedate": "...", ...}, ...]}`
|
||||
|
||||
### Requirement: Extraction mapping rules
|
||||
|
||||
The extraction agent SHALL follow these mapping rules when converting email content to TradeItems:
|
||||
- Each swap leg (identified by a unique Murex trade ID) becomes a separate TradeItem
|
||||
- The `valuedate` SHALL be parsed from date references in the email (e.g., "OB 27/11/2025") and formatted as dd/MM/yyyy
|
||||
- The `counterparty` SHALL be the full legal entity name as stated in the email prose
|
||||
- The `display_ccy` SHALL be derived from the currency symbol or code in the email (e.g., "£" or "PV (£)" → "GBP")
|
||||
- The `breakclause` SHALL default to "N" if not explicitly mentioned in the email
|
||||
- The `pv` SHALL be the numeric present value without formatting (no commas, no currency symbols)
|
||||
|
||||
#### Scenario: Flatten multi-leg swap into individual items
|
||||
|
||||
- **WHEN** the email contains a swap with Coupon Leg (Murex 79353083) and APD leg (Murex 79353084)
|
||||
- **THEN** the output contains two separate TradeItems, one per Murex ID
|
||||
|
||||
#### Scenario: Currency symbol to ISO code mapping
|
||||
|
||||
- **WHEN** the email shows PV values in "PV (£)" column
|
||||
- **THEN** the `display_ccy` field is set to "GBP"
|
||||
|
||||
#### Scenario: Default breakclause
|
||||
|
||||
- **WHEN** the email does not mention break clauses
|
||||
- **THEN** all TradeItems have `breakclause` set to "N"
|
||||
@@ -0,0 +1,80 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Counterparty lookup tool
|
||||
|
||||
The extraction plugin SHALL expose a `lookup_counterparty` Semantic Kernel function that accepts a counterparty name string and calls the external counterparty API. The tool SHALL return a list of candidate (counterparty, legal_entity) tuples.
|
||||
|
||||
#### Scenario: Single match found
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches exactly one record
|
||||
- **THEN** the tool returns a single candidate with the counterparty name and legal entity ID
|
||||
|
||||
#### Scenario: Multiple matches found (disambiguation needed)
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches multiple records
|
||||
- **THEN** the tool returns all matching candidates so the agent can present them to the user for selection
|
||||
|
||||
#### Scenario: No match found
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches no records
|
||||
- **THEN** the tool returns an empty list and an informative message so the agent can ask the user for clarification
|
||||
|
||||
### Requirement: Trade validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_trade` Semantic Kernel function that accepts a trade ID and calls the external trade validation API to verify the trade exists.
|
||||
|
||||
#### Scenario: Valid trade ID
|
||||
|
||||
- **WHEN** the tool is called with a known trade ID
|
||||
- **THEN** the tool returns a success result confirming the trade exists
|
||||
|
||||
#### Scenario: Invalid trade ID
|
||||
|
||||
- **WHEN** the tool is called with an unknown trade ID
|
||||
- **THEN** the tool returns an error result so the agent can flag it to the user
|
||||
|
||||
### Requirement: Currency validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_currency` Semantic Kernel function that accepts a currency code and calls the external currency validation API to verify it is a valid ISO currency code.
|
||||
|
||||
#### Scenario: Valid currency code
|
||||
|
||||
- **WHEN** the tool is called with "GBP"
|
||||
- **THEN** the tool returns a success result
|
||||
|
||||
#### Scenario: Invalid currency code
|
||||
|
||||
- **WHEN** the tool is called with an unrecognized code
|
||||
- **THEN** the tool returns an error with suggestions for valid codes
|
||||
|
||||
### Requirement: Schema validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_schema` Semantic Kernel function that accepts the full ExtractionResult JSON and validates that all required fields are present and correctly typed for every TradeItem.
|
||||
|
||||
#### Scenario: Valid extraction result
|
||||
|
||||
- **WHEN** the tool is called with a complete and correctly typed ExtractionResult JSON
|
||||
- **THEN** the tool returns a success result with no errors
|
||||
|
||||
#### Scenario: Missing required fields
|
||||
|
||||
- **WHEN** the tool is called with a TradeItem missing the `trade_id` field
|
||||
- **THEN** the tool returns a failure result listing the missing fields and which item they belong to
|
||||
|
||||
### Requirement: External API configuration
|
||||
|
||||
All external API base URLs SHALL be configurable via `appsettings.json` under an `ExternalApis` section. Each tool's HttpClient SHALL read its base URL from configuration at startup.
|
||||
|
||||
#### Scenario: Configuration at startup
|
||||
|
||||
- **WHEN** the API starts
|
||||
- **THEN** it reads external API base URLs from the `ExternalApis` configuration section and configures typed HttpClients accordingly
|
||||
|
||||
### Requirement: External API error handling
|
||||
|
||||
Each tool SHALL handle HTTP errors from external APIs gracefully, returning a structured error message that the LLM agent can reason about rather than throwing exceptions.
|
||||
|
||||
#### Scenario: External API unavailable
|
||||
|
||||
- **WHEN** a tool calls an external API that is unreachable
|
||||
- **THEN** the tool returns an error result with a descriptive message (e.g., "Counterparty API unavailable") so the agent can inform the user
|
||||
@@ -0,0 +1,38 @@
|
||||
## 1. Replace Shared Models
|
||||
|
||||
- [x] 1.1 Create `TradeItem.cs` in Shared/Models with fields: `Valuedate` (string), `Counterparty` (string), `LegalEntity` (string?), `TradeId` (long), `DisplayCcy` (string), `Pv` (double), `Breakclause` (string). Add `[JsonPropertyName]` attributes for snake_case serialization.
|
||||
- [x] 1.2 Create `ExtractionResult.cs` in Shared/Models with `List<TradeItem> Items` property
|
||||
- [x] 1.3 Delete the old `ExtractedFields.cs`
|
||||
- [x] 1.4 Update `ValidationResult.cs` to support candidate lists — add `List<CandidateMatch>? Candidates` property for disambiguation results (each with `Name` and `LegalEntity` fields)
|
||||
|
||||
## 2. Configure External API HttpClients
|
||||
|
||||
- [x] 2.1 Add `ExternalApis` section to `appsettings.json` with base URLs for counterparty, trade, and currency APIs
|
||||
- [x] 2.2 Create typed HttpClient service `CounterpartyApiClient` with `LookupAsync(string name)` method returning candidate tuples
|
||||
- [x] 2.3 Create typed HttpClient service `TradeApiClient` with `ValidateAsync(long tradeId)` method
|
||||
- [x] 2.4 Create typed HttpClient service `CurrencyApiClient` with `ValidateAsync(string currencyCode)` method
|
||||
- [x] 2.5 Register all typed HttpClients in `Program.cs` via `AddHttpClient<T>()` with base URLs from configuration
|
||||
|
||||
## 3. Rewrite ExtractionPlugin
|
||||
|
||||
- [x] 3.1 Replace `ExtractionPlugin.ValidateExtractedFields()` with `LookupCounterparty(string name)` — calls `CounterpartyApiClient`, returns candidate list as JSON. Include `[KernelFunction]` and `[Description]` attributes explaining the tool returns candidates for disambiguation.
|
||||
- [x] 3.2 Add `ValidateTrade(long tradeId)` method — calls `TradeApiClient`, returns valid/invalid result as JSON
|
||||
- [x] 3.3 Add `ValidateCurrency(string currencyCode)` method — calls `CurrencyApiClient`, returns valid/invalid result as JSON
|
||||
- [x] 3.4 Add `ValidateSchema(string extractionResultJson)` method — validates the full ExtractionResult JSON against TradeItem schema locally (required fields present, correct types, breakclause is Y or N)
|
||||
- [x] 3.5 Inject typed HttpClients into ExtractionPlugin via constructor. Register ExtractionPlugin in DI with its dependencies.
|
||||
- [x] 3.6 Add error handling in each tool method — catch `HttpRequestException`, return structured error JSON instead of throwing
|
||||
|
||||
## 4. Update Controller
|
||||
|
||||
- [x] 4.1 Update `ChatController.Post()` — no changes needed, DI resolution via GetRequiredService works with scoped registration to use the new `ExtractionPlugin` constructor (pass DI-resolved instance with HttpClients)
|
||||
|
||||
## 5. Update Tests
|
||||
|
||||
- [x] 5.1 Remove or rewrite tests referencing old `ExtractedFields` schema
|
||||
- [x] 5.2 Add unit tests for `ValidateSchema` method against TradeItem (valid, missing fields, wrong types)
|
||||
- [x] 5.3 Add unit tests for ExtractionPlugin tool methods with mocked HttpClients (success, error, multi-candidate responses)
|
||||
|
||||
## 6. Build and Verify
|
||||
|
||||
- [x] 6.1 Build the solution (`dotnet build`) and confirm no compilation errors
|
||||
- [x] 6.2 Run all tests (`dotnet test`) and confirm they pass
|
||||
131
openspec/exports/chat-agent-full-openspec.md
Normal file
131
openspec/exports/chat-agent-full-openspec.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# OpenSpec Artifacts for: AI Chat Agent with Streaming & Rich Text
|
||||
|
||||
On the target machine with OpenSpec + Claude Code:
|
||||
1. Create `openspec/config.yaml` (see below)
|
||||
2. Save `proposal.md` to `openspec/changes/chat-agent-full/proposal.md`
|
||||
3. Save `design.md` to `openspec/changes/chat-agent-full/design.md`
|
||||
4. Save `tasks.md` to `openspec/changes/chat-agent-full/tasks.md`
|
||||
5. Run `/opsx:apply chat-agent-full`
|
||||
6. Use the portable spec (`chat-agent-full-spec.md`) as reference if the AI needs detail
|
||||
|
||||
---
|
||||
|
||||
## config.yaml
|
||||
|
||||
Adapt to your project name and constraints, save as `openspec/config.yaml`:
|
||||
|
||||
```yaml
|
||||
schema: spec-driven
|
||||
|
||||
context: |
|
||||
<Your Project Name> — <one-line description>.
|
||||
Tech stack: .NET 9, C# 13, Blazor WASM, ASP.NET Core Web API.
|
||||
Key libraries: MudBlazor 9.2.0, Semantic Kernel 1.74.0, Markdig 1.1.1.
|
||||
<Any target-specific constraints>.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## proposal.md
|
||||
|
||||
**Integration Rule**: This feature is additive only. DO NOT modify existing files, components, services, or patterns. If the target already has an equivalent service (HTTP wrapper, markdown renderer, etc.), use the existing one. If a task conflicts with existing code, stop and notify the user before proceeding. Existing applicationX code takes precedence in all cases.
|
||||
|
||||
Build a complete AI chat interface on an existing MudBlazor app:
|
||||
- Chat page with streaming AI responses via SSE
|
||||
- Backend using Semantic Kernel with tool calling (extraction plugin)
|
||||
- Multi-turn conversation support
|
||||
- Rich text rendering (Markdig + HTML sanitization)
|
||||
- Full xUnit test coverage
|
||||
|
||||
Assumes MudBlazor is already installed and configured.
|
||||
|
||||
---
|
||||
|
||||
## design.md
|
||||
|
||||
### Integration: additive only
|
||||
- This feature is a GUEST in the target application — add new files, never modify existing ones
|
||||
- If the target already has a service that overlaps (e.g., its own HttpClient wrapper, markdown renderer, error handling), use theirs — do not create a duplicate
|
||||
- Conform to the target's code style, naming conventions, and DI patterns
|
||||
- If a task would require modifying existing code, **stop and notify the user** — the user decides whether to skip, adapt, or redesign
|
||||
- Only touch existing files to add a nav link or a DI registration line — never restructure
|
||||
|
||||
### Streaming: SSE over SignalR
|
||||
- SSE is simpler — one-directional text stream over HTTP, no WebSocket negotiation
|
||||
- Blazor WASM supports streaming via `SetBrowserResponseStreamingEnabled(true)` + `ResponseHeadersRead`
|
||||
- Client parses line-by-line: `data: {"text":"..."}\n\n` for deltas, `data: [DONE]\n\n` for end, `data: {"error":"..."}` for errors
|
||||
- **CRITICAL SSE loop pattern** — use `while ((line = await reader.ReadLineAsync()) != null)`. Do NOT use `reader.EndOfStream` — it does a synchronous peek that Blazor WASM's fetch-backed stream rejects at runtime
|
||||
|
||||
### AI orchestration: Semantic Kernel
|
||||
- SK provides chat completion connectors, plugin system, and auto function calling
|
||||
- OpenAI connector works with any OpenAI-compatible endpoint (e.g. local proxy) — base URL **must** include `/v1`
|
||||
- ExtractionPlugin imported per-request via `ImportPluginFromObject` (not at Kernel build, avoids plugin state leaking across requests)
|
||||
- `FunctionChoiceBehavior.Auto()` lets the LLM autonomously call tools and retry
|
||||
|
||||
### Client architecture
|
||||
- **Typed HttpClient** (`ChatApiClient`) — centralizes API paths, easy to mock for tests, IHttpClientFactory manages handler lifetime
|
||||
- **MarkdownService** (singleton) — Markdig pipeline is immutable/thread-safe; two-pass sanitization (regex strip script/style, then tag allowlist) rather than a sanitization library to keep dependencies minimal
|
||||
- **Rendered HTML cache** — `Dictionary<ChatMessage, string>` prevents re-running Markdig on all completed messages during every `StateHasChanged` while streaming
|
||||
|
||||
### Chat UI layout — "Sales Assistant" page
|
||||
- Page renders **inside MudMainContent** — do NOT add a separate AppBar or layout
|
||||
- Route: `/sales-assistant`, add MudNavLink in existing sidebar NavMenu
|
||||
- Flexbox column: message-list (flex 1, scrollable) + input-area (pinned bottom)
|
||||
- MudPaper bubbles: user right-aligned (primary bg), assistant left-aligned (surface bg)
|
||||
- `::deep` CSS selectors needed for MudBlazor child components and MarkupString-injected HTML
|
||||
- Auto-scroll via JS interop (`scrollTop = scrollHeight`) — no built-in Blazor scroll API
|
||||
- **Container height**: `calc(100vh - 64px)` — CRC app uses regular MudAppBar (~64px), not Dense. MudDrawer width is handled by MudLayout automatically — no horizontal calc needed
|
||||
|
||||
### Multi-turn
|
||||
- Full conversation history sent with each request (all messages with non-empty content)
|
||||
- Empty assistant placeholder excluded to avoid confusing the LLM
|
||||
|
||||
### Test strategy
|
||||
- API: `WebApplicationFactory<Program>` with mocked `IChatCompletionService` — tests SSE contract without hitting real LLM
|
||||
- Client: mock `HttpMessageHandler` with canned SSE response streams
|
||||
- Requires `public partial class Program { }` in API for test factory access
|
||||
|
||||
---
|
||||
|
||||
## tasks.md
|
||||
|
||||
### Phase 1: Shared Models
|
||||
- [ ] Create `ChatMessage.cs` in Shared/Models — Role (string), Content (string), Timestamp (DateTime)
|
||||
- [ ] Create `ChatRequest.cs` — List<ChatMessage> Messages
|
||||
- [ ] Create `HealthResponse.cs` — Status (string), Timestamp (DateTime)
|
||||
- [ ] Create `ExtractedFields.cs` — Client?, Project?, Hours?, Rate?, Currency?, Date? (required); Description?, PoNumber? (optional)
|
||||
- [ ] Create `ValidationResult.cs` — IsValid (bool), Errors (List<string>)
|
||||
|
||||
### Phase 2: API Backend
|
||||
- [ ] Add NuGet: Microsoft.SemanticKernel 1.74.0, Connectors.OpenAI 1.74.0
|
||||
- [ ] Create `ExtractionPlugin.cs` in Plugins/ — [KernelFunction] validates extracted fields JSON
|
||||
- [ ] Configure Program.cs: AddControllers, AddOpenAIChatCompletion (base URL must include /v1), AddKernel, ExtractionPlugin singleton, CORS for client origin
|
||||
- [ ] Create `HealthController.cs` — GET /api/health returns HealthResponse
|
||||
- [ ] Create `ChatController.cs` — POST /api/chat, streams via SK GetStreamingChatMessageContentsAsync, SSE format: data: {"text":"..."}\n\n and data: [DONE]\n\n
|
||||
- [ ] Add `public partial class Program { }` for test accessibility
|
||||
- [ ] Add appsettings.json: ResponsesApi:BaseUrl, ResponsesApi:Model
|
||||
|
||||
### Phase 3: Client Services
|
||||
- [ ] Add NuGet: Microsoft.Extensions.Http 9.0.4, Markdig 1.1.1
|
||||
- [ ] Create `ChatApiClient.cs` — typed HttpClient with GetHealthAsync and SendChatStreamingAsync (IAsyncEnumerable<string>). Use SetBrowserResponseStreamingEnabled(true) + ResponseHeadersRead. Parse SSE with ReadLineAsync null check (not EndOfStream).
|
||||
- [ ] Create `MarkdownService.cs` — Markdig with UseAdvancedExtensions, two-pass HTML sanitization (script/style strip, tag allowlist)
|
||||
- [ ] Register in Program.cs: AddMudServices, AddSingleton<MarkdownService>, AddHttpClient<ChatApiClient>
|
||||
- [ ] Add wwwroot/appsettings.json with ApiBaseUrl_Http and ApiBaseUrl_Https
|
||||
|
||||
### Phase 4: Chat UI
|
||||
- [ ] Create `SalesAssistant.razor` at route "/sales-assistant" — message list with MudPaper bubbles, MudTextField with send icon, Enter key handler
|
||||
- [ ] Add MudNavLink to existing sidebar NavMenu: "Sales Assistant", icon SmartToy, href /sales-assistant
|
||||
- [ ] Streaming: append tokens to assistant message, StateHasChanged per token, auto-scroll via JS interop
|
||||
- [ ] Assistant messages: render as (MarkupString) from MarkdownService inside .markdown-body div, with rendered HTML cache
|
||||
- [ ] User messages: plain MudText
|
||||
- [ ] Thinking indicator: MudProgressCircular when assistant content empty during streaming
|
||||
- [ ] New Chat button: clears messages, visible when conversation exists, disabled during streaming
|
||||
- [ ] Multi-turn: send all non-empty messages with each request
|
||||
- [ ] Create `SalesAssistant.razor.css` — flex layout with `calc(100vh - 64px)` height (64px = regular AppBar), message bubbles (user right/primary, assistant left/surface), markdown styles (code blocks, tables, blockquotes, headings, links)
|
||||
|
||||
### Phase 5: Tests
|
||||
- [ ] Create xUnit test project for API with WebApplicationFactory, mock IChatCompletionService
|
||||
- [ ] Test HealthController (200 + valid response), ChatController (SSE streaming, error handling), ExtractionPlugin (validation logic)
|
||||
- [ ] Create xUnit test project for Client with mock HttpMessageHandler
|
||||
- [ ] Test ChatApiClient (delta parsing, error events), MarkdownService (rendering + sanitization)
|
||||
- [ ] Verify: `dotnet test`
|
||||
206
openspec/exports/chat-agent-full-spec.md
Normal file
206
openspec/exports/chat-agent-full-spec.md
Normal file
@@ -0,0 +1,206 @@
|
||||
# Feature: AI Chat Agent with Streaming & Rich Text
|
||||
## Target: Existing MudBlazor app (.NET 9 / Blazor WASM + ASP.NET Core API)
|
||||
## Includes: basic-chat-interface, wire-responses-api, migrate-to-semantic-kernel, multi-turn-conversations, add-test-coverage, enable-rich-text-display
|
||||
## Skipped: migrate-claude-md-to-openspec (project scaffolding only)
|
||||
|
||||
## Integration Rule
|
||||
This feature is **additive only**. Existing applicationX code takes precedence over this spec in all cases.
|
||||
- **DO NOT** modify existing files, components, layouts, services, or patterns in the target
|
||||
- **DO NOT** replace existing patterns (e.g., if the target uses a different HttpClient pattern, use theirs)
|
||||
- **DO** add new files, new nav links, new routes, new DI registrations
|
||||
- **DO** conform to the target's existing code style, naming, and project structure
|
||||
- If the target already has an equivalent service (markdown renderer, HTTP wrapper, etc.), **use theirs**
|
||||
- If a task conflicts with existing target code, **stop and notify the user** — do not skip silently; the user decides whether to skip, adapt, or redesign
|
||||
|
||||
## Assumes
|
||||
- .NET 9 solution with Blazor WASM client + ASP.NET Core API projects + Shared class library
|
||||
- MudBlazor 9.2.0 installed and configured (AddMudServices, providers in MainLayout)
|
||||
- MudLayout with MudAppBar and MudMainContent in MainLayout.razor
|
||||
|
||||
## Target Layout (CRC app)
|
||||
```
|
||||
|------ ~220px ------|-------------- fluid ---------------|
|
||||
| | ≡ CRC 0.0.0 APR-CRC-PROD-... | ← MudAppBar (regular, ~64px)
|
||||
| Home |------------------------------------|
|
||||
| Pricer | |
|
||||
| Market Data | MudMainContent (@Body) |
|
||||
| XVA Statistics | ← Chat page renders here |
|
||||
| Sales | |
|
||||
| Sales Assistant ★ | |
|
||||
| | |
|
||||
| MudDrawer (~220px) | |
|
||||
|---------------------|-----------------------------------|
|
||||
```
|
||||
- MudAppBar: **regular height (~64px)**, not Dense — hamburger toggle, title + version, env badge
|
||||
- MudDrawer: left, **~220px**, toggled by hamburger, contains MudNavMenu
|
||||
- MudMainContent: fluid width, pages render via @Body
|
||||
- "Sales Assistant" added as new MudNavLink in existing MudNavMenu
|
||||
- Chat page renders **inside MudMainContent** — must not set its own AppBar or layout
|
||||
- Available viewport: `height: calc(100vh - 64px)`, width: fluid minus drawer when open
|
||||
|
||||
## Packages
|
||||
|
||||
**API project:**
|
||||
- `Microsoft.SemanticKernel` 1.74.0
|
||||
- `Microsoft.SemanticKernel.Connectors.OpenAI` 1.74.0
|
||||
|
||||
**Client project:**
|
||||
- `Microsoft.Extensions.Http` 9.0.4
|
||||
- `Markdig` 1.1.1
|
||||
|
||||
**Test projects (xUnit):**
|
||||
- `xunit`, `xunit.runner.visualstudio`, `Microsoft.NET.Test.Sdk`
|
||||
- API tests: `Moq`, `Microsoft.AspNetCore.Mvc.Testing`
|
||||
- Client tests: `Moq`
|
||||
|
||||
## Architecture
|
||||
|
||||
Three-project solution: WASM client sends chat requests to ASP.NET Core API, which processes them through Semantic Kernel (pointed at an OpenAI-compatible endpoint) and streams token deltas back as SSE. Client renders assistant markdown as sanitized HTML. An extraction plugin demonstrates SK tool calling.
|
||||
|
||||
## Shared Models
|
||||
### `ChatMessage` — Shared/Models/
|
||||
- `string Role` ("user" | "assistant"), `string Content`, `DateTime Timestamp`
|
||||
|
||||
### `ChatRequest` — Shared/Models/
|
||||
- `List<ChatMessage> Messages`
|
||||
|
||||
### `HealthResponse` — Shared/Models/
|
||||
- `string Status`, `DateTime Timestamp`
|
||||
|
||||
### `ExtractedFields` — Shared/Models/
|
||||
- Required: `string? Client`, `string? Project`, `decimal? Hours`, `decimal? Rate`, `string? Currency`, `string? Date`
|
||||
- Optional: `string? Description`, `string? PoNumber`
|
||||
|
||||
### `ValidationResult` — Shared/Models/
|
||||
- `bool IsValid`, `List<string> Errors`
|
||||
|
||||
## Components
|
||||
|
||||
### API: `Program.cs`
|
||||
- Register `AddControllers()`, `AddOpenAIChatCompletion()` with configurable endpoint, `AddKernel()`, ExtractionPlugin singleton
|
||||
- CORS policy allowing Blazor client origin, any header/method
|
||||
- Config keys: `ResponsesApi:BaseUrl` (default `http://localhost:8317/v1`), `ResponsesApi:Model`, `ResponsesApi:ApiKey`
|
||||
- IMPORTANT: Base URL must include `/v1` — the OpenAI SDK appends `chat/completions` directly
|
||||
- Add `public partial class Program { }` for test accessibility
|
||||
|
||||
### API: `ChatController` — Controllers/
|
||||
- POST `/api/chat` accepts `ChatRequest`, returns `text/event-stream`
|
||||
- Get `IChatCompletionService` from Kernel, convert messages to `ChatHistory`
|
||||
- Import ExtractionPlugin from DI: `_kernel.ImportPluginFromObject(plugin, "Extraction")`
|
||||
- Use `OpenAIPromptExecutionSettings` with `FunctionChoiceBehavior.Auto()`
|
||||
- Stream via `GetStreamingChatMessageContentsAsync()`, emit non-empty chunks as SSE
|
||||
- SSE format: `data: {"text":"<delta>"}\n\n`, completion: `data: [DONE]\n\n`, error: `data: {"error":"<msg>"}\n\n`
|
||||
- Catch `HttpRequestException` → error SSE event, `TaskCanceledException` → silent (client disconnect)
|
||||
|
||||
### API: `HealthController` — Controllers/
|
||||
- GET `/api/health` → returns `HealthResponse` with status "Healthy" and UTC timestamp
|
||||
|
||||
### API: `ExtractionPlugin` — Plugins/
|
||||
- `[KernelFunction("validate_extracted_fields")]` method accepting JSON string
|
||||
- Deserialize to `ExtractedFields`, validate required fields non-null, Hours/Rate > 0
|
||||
- Return `ValidationResult` as JSON string
|
||||
|
||||
### Client: `Program.cs`
|
||||
- Register `AddMudServices()`, `MarkdownService` (singleton)
|
||||
- Register `AddHttpClient<ChatApiClient>` with API base URL from config
|
||||
- Config: `ApiBaseUrl_Https`, `ApiBaseUrl_Http` in wwwroot/appsettings.json
|
||||
- Auto-detect HTTPS from `HostEnvironment.BaseAddress`
|
||||
|
||||
### Client: `ChatApiClient` — Services/
|
||||
- Typed HttpClient wrapper
|
||||
- `GetHealthAsync()` → GET api/health, returns `HealthResponse?`
|
||||
- `SendChatStreamingAsync(ChatRequest)` → `IAsyncEnumerable<string>`
|
||||
- Build `HttpRequestMessage` manually
|
||||
- Call `httpRequest.SetBrowserResponseStreamingEnabled(true)` (Blazor WASM extension)
|
||||
- Send with `HttpCompletionOption.ResponseHeadersRead`
|
||||
- Parse SSE line-by-line: extract `"text"` field, throw on `"error"` field, stop on `[DONE]`
|
||||
- SSE loop: see "Critical Patterns" section below — do NOT use `reader.EndOfStream`
|
||||
|
||||
### Client: `MarkdownService` — Services/
|
||||
- Singleton wrapping Markdig with `UseAdvancedExtensions()` pipeline
|
||||
- `ConvertToHtml(string markdown)` → sanitized HTML string
|
||||
- Two-pass sanitization:
|
||||
1. Strip `<script>` and `<style>` blocks with content (regex)
|
||||
2. Filter tags against allowlist: p, h1-h6, strong, em, code, pre, ul, ol, li, a, table, thead, tbody, tr, th, td, br, blockquote
|
||||
- Only `href` attribute allowed (on `<a>` only)
|
||||
|
||||
### Client: `SalesAssistant.razor` — Pages/, route `/sales-assistant`
|
||||
- Page title: "Sales Assistant"
|
||||
- Add `MudNavLink` in existing sidebar NavMenu: icon `Icons.Material.Filled.SmartToy`, href `/sales-assistant`
|
||||
- Message list with MudPaper bubbles: user (right-aligned, primary bg), assistant (left-aligned, surface bg)
|
||||
- MudTextField with send icon adornment, Enter key handler, disabled during streaming
|
||||
- Empty state with centered title text
|
||||
- "New Chat" button (MudButton) above input, visible when messages exist, disabled during streaming
|
||||
- Streaming: append tokens to assistant message, call `StateHasChanged()` + auto-scroll per token
|
||||
- Assistant messages: render via `(MarkupString)GetRenderedHtml(message)` inside `.markdown-body` div
|
||||
- Rendered HTML cache (`Dictionary<ChatMessage, string>`) — cache on stream completion, render fresh during streaming
|
||||
- Auto-scroll via JS interop: `document.querySelector('.message-list').scrollTop = .scrollHeight`
|
||||
- Thinking indicator: `MudProgressCircular` when assistant content is empty and streaming
|
||||
- Multi-turn: send all messages with non-empty content (excludes empty assistant placeholder)
|
||||
- User messages: plain `MudText`, no markdown
|
||||
|
||||
### Client: `SalesAssistant.razor.css` — scoped styles
|
||||
- `.chat-container`: flex column, `height: calc(100vh - 64px)`, max-width 800px, centered within MudMainContent
|
||||
- `.message-list`: flex 1, overflow-y auto, flex column, gap 0.75rem
|
||||
- Message alignment: `.message-user` flex-end, `.message-assistant` flex-start
|
||||
- `::deep .bubble-user`: primary color bg, white text, rounded except bottom-right
|
||||
- `::deep .bubble-assistant`: surface bg, border, rounded except bottom-left
|
||||
- `::deep .markdown-body` styles: code blocks (gray bg, monospace), tables (bordered), blockquotes (left border primary), headings (scaled down), links (primary color, underline)
|
||||
|
||||
## Wiring (dependency order)
|
||||
|
||||
1. **Shared project**: Create all models (ChatMessage, ChatRequest, HealthResponse, ExtractedFields, ValidationResult)
|
||||
2. **API Program.cs**: AddControllers → AddOpenAIChatCompletion → AddKernel → AddSingleton<ExtractionPlugin> → AddCors → Build → UseCors → UseAuthorization → MapControllers
|
||||
3. **Client Program.cs**: AddMudServices → AddSingleton<MarkdownService> → AddHttpClient<ChatApiClient> → Build → RunAsync
|
||||
4. **API appsettings.json**: `{"ResponsesApi": {"BaseUrl": "http://localhost:8317/v1", "Model": "claude-sonnet-4-6"}}`
|
||||
5. **Client wwwroot/appsettings.json**: `{"ApiBaseUrl_Http": "http://localhost:7000", "ApiBaseUrl_Https": "https://localhost:7100"}`
|
||||
|
||||
## Critical Patterns (copy these, do not improvise)
|
||||
|
||||
### SSE read loop in Blazor WASM
|
||||
```csharp
|
||||
// DO NOT use reader.EndOfStream — it does a synchronous peek,
|
||||
// which Blazor WASM's fetch-backed stream rejects at runtime.
|
||||
string? line;
|
||||
while ((line = await reader.ReadLineAsync()) != null)
|
||||
{
|
||||
if (!line.StartsWith("data: ")) continue;
|
||||
var data = line.Substring(6);
|
||||
if (data == "[DONE]") yield break;
|
||||
// parse {"text":"..."} or {"error":"..."}
|
||||
}
|
||||
```
|
||||
|
||||
### Chat container height
|
||||
```css
|
||||
/* 64px = MudAppBar regular height in the CRC app.
|
||||
The chat page renders inside MudMainContent which already
|
||||
sits below the AppBar, so use 100% of the parent height
|
||||
rather than calculating from viewport. If MudMainContent
|
||||
does not have height set, fall back to calc(100vh - 64px). */
|
||||
.chat-container {
|
||||
height: calc(100vh - 64px);
|
||||
/* MudDrawer width is handled by MudLayout — the chat container
|
||||
is fluid within MudMainContent, no horizontal calc needed. */
|
||||
}
|
||||
```
|
||||
|
||||
## Behavior (non-obvious)
|
||||
|
||||
- Base URL **must** include `/v1` — SK's OpenAI connector appends `chat/completions` directly
|
||||
- WASM streaming requires **both** `SetBrowserResponseStreamingEnabled(true)` and `ResponseHeadersRead`
|
||||
- Only emit SSE chunks where `!string.IsNullOrEmpty(chunk.Content)` — tool call chunks have no text
|
||||
- Cache rendered HTML for completed messages to avoid re-running Markdig on every `StateHasChanged`
|
||||
- `public partial class Program { }` at bottom of API Program.cs for `WebApplicationFactory<Program>` in tests
|
||||
- ExtractionPlugin is imported per-request via `ImportPluginFromObject` (not at Kernel build time)
|
||||
- Chat sends all messages with non-empty content — filters out the empty assistant placeholder
|
||||
- Chat container height assumes MudAppBar Dense (48px) — if your AppBar height differs, adjust the calc() or use a CSS variable
|
||||
- On mobile viewports, `100vh` includes the browser address bar — use `100dvh` if targeting mobile
|
||||
|
||||
## Test Coverage
|
||||
|
||||
- **HealthControllerTests**: GET /api/health returns 200 with valid HealthResponse
|
||||
- **ChatControllerTests**: Mock `IChatCompletionService`, verify SSE text deltas + [DONE]; verify error handling; use `WebApplicationFactory<Program>`
|
||||
- **ExtractionPluginTests**: Valid fields → IsValid; missing required → errors; invalid JSON → error; zero hours → error
|
||||
- **ChatApiClientTests**: Mock `HttpMessageHandler` with canned SSE streams; verify delta parsing order; verify error event throws
|
||||
- **MarkdownServiceTests**: Verify rendering (bold, italic, code, lists, tables, links, headings); verify sanitization (script/style stripping, event handler removal, tag allowlist)
|
||||
@@ -20,27 +20,37 @@ The agent SHALL extract a predefined set of key-value pairs from user-provided n
|
||||
|
||||
### Requirement: Predefined extraction schema
|
||||
|
||||
The system SHALL define a fixed set of known field names and types as a strongly-typed C# class. All extraction output MUST conform to this schema.
|
||||
The system SHALL define the extraction schema as a `TradeItem` class with fields: valuedate, counterparty, legal_entity, trade_id, display_ccy, pv, breakclause. Extraction output SHALL be wrapped in an `ExtractionResult` containing a `List<TradeItem>`. All extraction output MUST conform to this schema.
|
||||
|
||||
#### Scenario: Output conforms to schema
|
||||
|
||||
- **WHEN** the agent produces extracted fields
|
||||
- **THEN** every key in the output matches a field defined in the schema and values match expected types
|
||||
- **WHEN** the agent produces extracted fields from an email
|
||||
- **THEN** every item in the output is a valid TradeItem with all required fields matching expected types
|
||||
|
||||
#### Scenario: Multiple items from one email
|
||||
|
||||
- **WHEN** the agent extracts data from an email containing multiple trade legs
|
||||
- **THEN** the output ExtractionResult contains one TradeItem per trade leg
|
||||
|
||||
### Requirement: Autonomous validation via tool calling
|
||||
|
||||
The agent SHALL validate extracted fields by calling a validation tool function. The validation tool checks that all required fields are present and correctly typed.
|
||||
The agent SHALL validate extracted fields by calling external API tools exposed as Semantic Kernel functions. Validation tools include counterparty lookup, trade validation, currency validation, and schema validation. Each tool returns structured results that the agent reasons about.
|
||||
|
||||
#### Scenario: Validation passes
|
||||
|
||||
- **WHEN** the agent calls the validation tool with a complete and correct extraction
|
||||
- **WHEN** the agent calls the schema validation tool with a complete and correct ExtractionResult
|
||||
- **THEN** the tool returns a success result and the agent returns the final output to the user
|
||||
|
||||
#### Scenario: Validation fails with fixable errors
|
||||
|
||||
- **WHEN** the validation tool returns errors for missing or malformed fields
|
||||
- **WHEN** a validation tool returns errors for missing or malformed fields
|
||||
- **THEN** the agent re-reads the source text and attempts to fix the extraction without user intervention
|
||||
|
||||
#### Scenario: Counterparty disambiguation required
|
||||
|
||||
- **WHEN** the counterparty lookup tool returns multiple candidate (counterparty, legal_entity) tuples
|
||||
- **THEN** the agent presents the candidates to the user as a numbered list in the chat and waits for the user to select one before completing the extraction
|
||||
|
||||
### Requirement: Autonomous retry with iteration cap
|
||||
|
||||
The agent SHALL retry extraction autonomously up to 3 times when validation fails. After exhausting retries, the agent MUST escalate to the user.
|
||||
@@ -57,13 +67,18 @@ The agent SHALL retry extraction autonomously up to 3 times when validation fail
|
||||
|
||||
### Requirement: Human-in-the-loop clarification
|
||||
|
||||
When the agent escalates to the user, the user SHALL be able to provide the missing information in natural language, and the agent SHALL incorporate the clarification and re-attempt extraction.
|
||||
When the agent escalates to the user, the user SHALL be able to provide the missing information in natural language, and the agent SHALL incorporate the clarification and re-attempt extraction. Disambiguation of counterparty/legal_entity tuples is a specific case of human-in-the-loop clarification.
|
||||
|
||||
#### Scenario: User provides clarification
|
||||
|
||||
- **WHEN** the agent asks for clarification about missing fields and the user responds
|
||||
- **THEN** the agent incorporates the user's response into the conversation context and produces an updated extraction
|
||||
|
||||
#### Scenario: User selects counterparty from candidates
|
||||
|
||||
- **WHEN** the agent presents a numbered list of counterparty/legal_entity candidates and the user replies with a selection
|
||||
- **THEN** the agent populates the `legal_entity` field on all relevant TradeItems and proceeds with validation
|
||||
|
||||
#### Scenario: Clarification via normal chat
|
||||
|
||||
- **WHEN** the agent escalates for clarification
|
||||
|
||||
@@ -6,12 +6,27 @@ Define the streaming AI response pipeline — backend chat endpoint using Semant
|
||||
|
||||
### Requirement: Chat endpoint proxies to Responses API
|
||||
|
||||
The API backend SHALL expose `POST /api/chat` that accepts a list of messages and processes them using a Semantic Kernel chat completion service. The kernel is configured with an OpenAI connector pointed at the existing CLIProxyAPI proxy.
|
||||
The API backend SHALL expose `POST /api/chat` that accepts a `ChatRequest` containing messages, an optional system prompt, and optional model settings. The request is processed using a Semantic Kernel chat completion service. When a system prompt is provided, it SHALL be added as the first system message in the ChatHistory. When model settings are provided, non-null values SHALL be applied to the execution settings. A separate `POST /api/chat/extract` endpoint SHALL handle extraction-specific requests with few-shot prompting.
|
||||
|
||||
#### Scenario: Successful chat request
|
||||
#### Scenario: Successful chat request with system prompt
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with a message list
|
||||
- **THEN** the API processes the messages through the Semantic Kernel and returns the response
|
||||
- **WHEN** the client sends a POST to `/api/chat` with messages and a system prompt
|
||||
- **THEN** the API creates a ChatHistory with the system prompt as the first message, followed by the conversation messages, and processes them through Semantic Kernel
|
||||
|
||||
#### Scenario: Successful chat request with model settings
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with messages and model settings (e.g., Temperature=0.3)
|
||||
- **THEN** the API applies the settings to OpenAIPromptExecutionSettings before calling the Semantic Kernel
|
||||
|
||||
#### Scenario: Successful chat request without optional fields
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat` with only messages (no system prompt, no settings)
|
||||
- **THEN** the API processes the request with default behavior (no system message, default execution settings)
|
||||
|
||||
#### Scenario: Extraction request routed to dedicated endpoint
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat/extract` with email HTML
|
||||
- **THEN** the API uses the few-shot ChatHistory prefix and extraction tools instead of the general chat configuration
|
||||
|
||||
### Requirement: Streaming response delivery
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ The chat page SHALL display messages in a vertically scrolling list, with each m
|
||||
|
||||
### Requirement: Message input
|
||||
|
||||
The chat page SHALL provide a text input area at the bottom of the page where the user can type and submit messages.
|
||||
The chat page SHALL provide a text input area at the bottom of the page where the user can type and submit messages. The input area SHALL also include a file upload button for triggering email extraction.
|
||||
|
||||
#### Scenario: Submit via button
|
||||
|
||||
@@ -45,7 +45,12 @@ The chat page SHALL provide a text input area at the bottom of the page where th
|
||||
#### Scenario: Input disabled during streaming
|
||||
|
||||
- **WHEN** the assistant is currently streaming a response
|
||||
- **THEN** the input field and send button are disabled until streaming completes
|
||||
- **THEN** the input field, send button, and upload button are disabled until streaming completes
|
||||
|
||||
#### Scenario: Upload button opens file picker
|
||||
|
||||
- **WHEN** the user clicks the upload button in the input area
|
||||
- **THEN** a file picker dialog opens filtered to .html files
|
||||
|
||||
### Requirement: Thinking indicator
|
||||
|
||||
@@ -100,9 +105,24 @@ The message list SHALL automatically scroll to the newest message when a new mes
|
||||
|
||||
### Requirement: Chat page is default route
|
||||
|
||||
The chat page SHALL be the default route (`/`) of the application.
|
||||
The chat page SHALL be routed at `/sales-assistant` (or `/` with redirect). The page content SHALL be wrapped in a MudTabs container with the conversation UI in the first tab panel.
|
||||
|
||||
#### Scenario: App opens to chat
|
||||
#### Scenario: App opens to chat via redirect
|
||||
|
||||
- **WHEN** the user navigates to the root URL
|
||||
- **WHEN** the user navigates to the root URL `/`
|
||||
- **THEN** the browser redirects to `/sales-assistant` and the chat page is displayed
|
||||
|
||||
#### Scenario: Direct navigation to sales-assistant
|
||||
|
||||
- **WHEN** the user navigates to `/sales-assistant`
|
||||
- **THEN** the chat page is displayed
|
||||
|
||||
#### Scenario: Page loads with chat tab active
|
||||
|
||||
- **WHEN** the user navigates to the chat page
|
||||
- **THEN** the Chat tab is active showing the message list and input area
|
||||
|
||||
#### Scenario: Chat functionality unchanged
|
||||
|
||||
- **WHEN** the user sends a message from the Chat tab
|
||||
- **THEN** the assistant response streams in exactly as before, with the same SSE contract and rendering behavior
|
||||
|
||||
42
openspec/specs/email-upload/spec.md
Normal file
42
openspec/specs/email-upload/spec.md
Normal file
@@ -0,0 +1,42 @@
|
||||
## Purpose
|
||||
|
||||
Define the email upload UX — drag-and-drop, file picker, and upload behavior constraints during streaming.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Drag-and-drop email upload
|
||||
|
||||
The chat message area SHALL accept files dragged from the desktop or file explorer. When a supported file is dropped, the client SHALL read the file content and send it to the extraction endpoint.
|
||||
|
||||
#### Scenario: Drag HTML file onto chat
|
||||
|
||||
- **WHEN** the user drags an .html file over the message area
|
||||
- **THEN** a visual drop indicator appears (e.g., highlighted border, overlay text "Drop email here")
|
||||
|
||||
#### Scenario: Drop HTML file triggers extraction
|
||||
|
||||
- **WHEN** the user drops an .html file onto the message area
|
||||
- **THEN** the client reads the HTML content, sends it to `POST /api/chat/extract`, and streams the extraction response in the chat
|
||||
|
||||
#### Scenario: Unsupported file type rejected
|
||||
|
||||
- **WHEN** the user drops a non-.html file (e.g., .pdf, .docx)
|
||||
- **THEN** the client shows a brief error message indicating only .html files are supported
|
||||
|
||||
### Requirement: File picker upload button
|
||||
|
||||
The chat input area SHALL include an upload button (e.g., attachment icon) that opens a file picker dialog for selecting .html email files.
|
||||
|
||||
#### Scenario: Upload via file picker
|
||||
|
||||
- **WHEN** the user clicks the upload button and selects an .html file
|
||||
- **THEN** the client reads the HTML content and sends it to the extraction endpoint, same as drag-and-drop
|
||||
|
||||
### Requirement: Upload disabled during streaming
|
||||
|
||||
The upload zone and file picker SHALL be disabled while a response is streaming.
|
||||
|
||||
#### Scenario: Drop during streaming
|
||||
|
||||
- **WHEN** the user attempts to drop a file while the assistant is streaming
|
||||
- **THEN** the drop is ignored and no extraction request is sent
|
||||
51
openspec/specs/extraction-conversation-flow/spec.md
Normal file
51
openspec/specs/extraction-conversation-flow/spec.md
Normal file
@@ -0,0 +1,51 @@
|
||||
## Purpose
|
||||
|
||||
Define the extraction conversation flow — mode tracking, visual indicators, follow-up message routing, and upload message display.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Extraction mode tracking
|
||||
|
||||
The chat page SHALL track whether the current conversation is in extraction mode. Extraction mode is entered when an email is uploaded and exited when the user starts a new chat.
|
||||
|
||||
#### Scenario: Enter extraction mode on upload
|
||||
|
||||
- **WHEN** the user uploads an email file
|
||||
- **THEN** the conversation enters extraction mode and subsequent messages are routed to the extraction endpoint
|
||||
|
||||
#### Scenario: Exit extraction mode on New Chat
|
||||
|
||||
- **WHEN** the user clicks "New Chat" while in extraction mode
|
||||
- **THEN** the conversation exits extraction mode and returns to general chat routing
|
||||
|
||||
### Requirement: Extraction mode visual indicator
|
||||
|
||||
The chat page SHALL display a visual indicator when in extraction mode so the user knows their messages are part of an extraction conversation.
|
||||
|
||||
#### Scenario: Indicator shown in extraction mode
|
||||
|
||||
- **WHEN** the conversation is in extraction mode
|
||||
- **THEN** a visual indicator (e.g., chip, banner, or subtitle) is visible showing the extraction context
|
||||
|
||||
#### Scenario: Indicator hidden in general mode
|
||||
|
||||
- **WHEN** the conversation is in general chat mode
|
||||
- **THEN** no extraction indicator is shown
|
||||
|
||||
### Requirement: Follow-up messages route to extraction endpoint
|
||||
|
||||
In extraction mode, text messages typed by the user SHALL be sent to the extraction endpoint with the original email HTML and full conversation history, not to the general chat endpoint.
|
||||
|
||||
#### Scenario: User replies to disambiguation question
|
||||
|
||||
- **WHEN** the agent asks "Which legal entity?" and the user types "1"
|
||||
- **THEN** the client sends an ExtractionRequest with the original email HTML plus all messages (assistant question + user reply) to `POST /api/chat/extract`
|
||||
|
||||
### Requirement: Email upload message in chat
|
||||
|
||||
When an email is uploaded, the chat SHALL display a user message indicating the upload (e.g., showing the filename) before the extraction response streams in.
|
||||
|
||||
#### Scenario: Upload message displayed
|
||||
|
||||
- **WHEN** the user drops "trade_request.html"
|
||||
- **THEN** a user message appears in the chat like "[Uploaded: trade_request.html]" followed by the streaming extraction response
|
||||
47
openspec/specs/extraction-endpoint/spec.md
Normal file
47
openspec/specs/extraction-endpoint/spec.md
Normal file
@@ -0,0 +1,47 @@
|
||||
## Purpose
|
||||
|
||||
Define the extraction-specific API endpoint — request/response contract, few-shot ChatHistory integration, and tool isolation from the general chat endpoint.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Extraction API endpoint
|
||||
|
||||
The API SHALL expose `POST /api/chat/extract` that accepts an `ExtractionRequest` containing the email HTML content and optional follow-up conversation messages. The endpoint SHALL use the few-shot ChatHistory prefix (not the user-editable system prompt) and load extraction-specific SK plugins.
|
||||
|
||||
#### Scenario: Initial extraction request
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat/extract` with email HTML and no follow-up messages
|
||||
- **THEN** the API assembles the few-shot ChatHistory, appends the email as the final user message, and streams the extraction response via SSE
|
||||
|
||||
#### Scenario: Follow-up disambiguation request
|
||||
|
||||
- **WHEN** the client sends a POST to `/api/chat/extract` with email HTML and follow-up messages (e.g., user selecting a counterparty)
|
||||
- **THEN** the API assembles the few-shot ChatHistory, appends the email, appends all follow-up messages, and streams the continuation response via SSE
|
||||
|
||||
#### Scenario: SSE streaming contract
|
||||
|
||||
- **WHEN** the extraction endpoint streams a response
|
||||
- **THEN** it uses the same SSE format as `/api/chat`: `data: {"text":"..."}\n\n` for deltas and `data: [DONE]\n\n` for completion
|
||||
|
||||
### Requirement: ExtractionRequest DTO
|
||||
|
||||
The system SHALL define an `ExtractionRequest` class with `EmailHtml` (string, required) and `Messages` (List<ChatMessage>, optional) for follow-up conversation context.
|
||||
|
||||
#### Scenario: First request has email only
|
||||
|
||||
- **WHEN** the user uploads an email for the first time
|
||||
- **THEN** the ExtractionRequest contains `EmailHtml` with the email content and an empty `Messages` list
|
||||
|
||||
#### Scenario: Follow-up request includes conversation
|
||||
|
||||
- **WHEN** the user replies to a disambiguation question
|
||||
- **THEN** the ExtractionRequest contains the original `EmailHtml` plus `Messages` with the full assistant/user exchange since the extraction started
|
||||
|
||||
### Requirement: Extraction endpoint uses extraction tools only
|
||||
|
||||
The extraction endpoint SHALL import only the extraction-specific SK plugins (counterparty lookup, trade validation, currency validation, schema validation). General chat tools (if any) SHALL NOT be loaded for extraction requests.
|
||||
|
||||
#### Scenario: Tool isolation
|
||||
|
||||
- **WHEN** the extraction endpoint processes a request
|
||||
- **THEN** only extraction-related KernelFunctions are available to the LLM
|
||||
67
openspec/specs/extraction-schema/spec.md
Normal file
67
openspec/specs/extraction-schema/spec.md
Normal file
@@ -0,0 +1,67 @@
|
||||
## Purpose
|
||||
|
||||
Define the TradeItem extraction schema, ExtractionResult wrapper, and mapping rules for converting sales email content into structured trade data.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: TradeItem schema
|
||||
|
||||
The system SHALL define a `TradeItem` class with the following fields representing a single trade leg extracted from a sales email:
|
||||
- `valuedate` (string, dd/MM/yyyy format)
|
||||
- `counterparty` (string, full legal name as it appears in the email)
|
||||
- `legal_entity` (string, nullable — populated after counterparty disambiguation via lookup tool)
|
||||
- `trade_id` (long, Murex trade identifier)
|
||||
- `display_ccy` (string, ISO currency code e.g. "GBP", "USD")
|
||||
- `pv` (double, present value)
|
||||
- `breakclause` (string, "Y" or "N")
|
||||
|
||||
JSON serialization SHALL use snake_case property names via `[JsonPropertyName]` attributes.
|
||||
|
||||
#### Scenario: All fields populated
|
||||
|
||||
- **WHEN** the extraction agent produces a TradeItem with all fields
|
||||
- **THEN** the JSON output contains all seven fields with snake_case keys and correct types
|
||||
|
||||
#### Scenario: Legal entity null before disambiguation
|
||||
|
||||
- **WHEN** the extraction agent produces a TradeItem before counterparty lookup
|
||||
- **THEN** the `legal_entity` field is null and all other fields are populated
|
||||
|
||||
### Requirement: ExtractionResult wrapper
|
||||
|
||||
The system SHALL define an `ExtractionResult` class containing a `List<TradeItem> Items` property. All extraction output from a single email SHALL be wrapped in this object.
|
||||
|
||||
#### Scenario: Single email with multiple trade legs
|
||||
|
||||
- **WHEN** an email contains two swaps with two legs each (4 trades total)
|
||||
- **THEN** the ExtractionResult contains an `items` array with 4 TradeItem objects
|
||||
|
||||
#### Scenario: JSON output structure
|
||||
|
||||
- **WHEN** the ExtractionResult is serialized to JSON
|
||||
- **THEN** the output has the shape `{"items": [{"valuedate": "...", ...}, ...]}`
|
||||
|
||||
### Requirement: Extraction mapping rules
|
||||
|
||||
The extraction agent SHALL follow these mapping rules when converting email content to TradeItems:
|
||||
- Each swap leg (identified by a unique Murex trade ID) becomes a separate TradeItem
|
||||
- The `valuedate` SHALL be parsed from date references in the email (e.g., "OB 27/11/2025") and formatted as dd/MM/yyyy
|
||||
- The `counterparty` SHALL be the full legal entity name as stated in the email prose
|
||||
- The `display_ccy` SHALL be derived from the currency symbol or code in the email (e.g., "£" or "PV (£)" → "GBP")
|
||||
- The `breakclause` SHALL default to "N" if not explicitly mentioned in the email
|
||||
- The `pv` SHALL be the numeric present value without formatting (no commas, no currency symbols)
|
||||
|
||||
#### Scenario: Flatten multi-leg swap into individual items
|
||||
|
||||
- **WHEN** the email contains a swap with Coupon Leg (Murex 79353083) and APD leg (Murex 79353084)
|
||||
- **THEN** the output contains two separate TradeItems, one per Murex ID
|
||||
|
||||
#### Scenario: Currency symbol to ISO code mapping
|
||||
|
||||
- **WHEN** the email shows PV values in "PV (£)" column
|
||||
- **THEN** the `display_ccy` field is set to "GBP"
|
||||
|
||||
#### Scenario: Default breakclause
|
||||
|
||||
- **WHEN** the email does not mention break clauses
|
||||
- **THEN** all TradeItems have `breakclause` set to "N"
|
||||
84
openspec/specs/extraction-tools/spec.md
Normal file
84
openspec/specs/extraction-tools/spec.md
Normal file
@@ -0,0 +1,84 @@
|
||||
## Purpose
|
||||
|
||||
Define the Semantic Kernel tool functions for extraction validation — counterparty lookup, trade validation, currency validation, schema validation, external API configuration, and error handling.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Counterparty lookup tool
|
||||
|
||||
The extraction plugin SHALL expose a `lookup_counterparty` Semantic Kernel function that accepts a counterparty name string and calls the external counterparty API. The tool SHALL return a list of candidate (counterparty, legal_entity) tuples.
|
||||
|
||||
#### Scenario: Single match found
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches exactly one record
|
||||
- **THEN** the tool returns a single candidate with the counterparty name and legal entity ID
|
||||
|
||||
#### Scenario: Multiple matches found (disambiguation needed)
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches multiple records
|
||||
- **THEN** the tool returns all matching candidates so the agent can present them to the user for selection
|
||||
|
||||
#### Scenario: No match found
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches no records
|
||||
- **THEN** the tool returns an empty list and an informative message so the agent can ask the user for clarification
|
||||
|
||||
### Requirement: Trade validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_trade` Semantic Kernel function that accepts a trade ID and calls the external trade validation API to verify the trade exists.
|
||||
|
||||
#### Scenario: Valid trade ID
|
||||
|
||||
- **WHEN** the tool is called with a known trade ID
|
||||
- **THEN** the tool returns a success result confirming the trade exists
|
||||
|
||||
#### Scenario: Invalid trade ID
|
||||
|
||||
- **WHEN** the tool is called with an unknown trade ID
|
||||
- **THEN** the tool returns an error result so the agent can flag it to the user
|
||||
|
||||
### Requirement: Currency validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_currency` Semantic Kernel function that accepts a currency code and calls the external currency validation API to verify it is a valid ISO currency code.
|
||||
|
||||
#### Scenario: Valid currency code
|
||||
|
||||
- **WHEN** the tool is called with "GBP"
|
||||
- **THEN** the tool returns a success result
|
||||
|
||||
#### Scenario: Invalid currency code
|
||||
|
||||
- **WHEN** the tool is called with an unrecognized code
|
||||
- **THEN** the tool returns an error with suggestions for valid codes
|
||||
|
||||
### Requirement: Schema validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_schema` Semantic Kernel function that accepts the full ExtractionResult JSON and validates that all required fields are present and correctly typed for every TradeItem.
|
||||
|
||||
#### Scenario: Valid extraction result
|
||||
|
||||
- **WHEN** the tool is called with a complete and correctly typed ExtractionResult JSON
|
||||
- **THEN** the tool returns a success result with no errors
|
||||
|
||||
#### Scenario: Missing required fields
|
||||
|
||||
- **WHEN** the tool is called with a TradeItem missing the `trade_id` field
|
||||
- **THEN** the tool returns a failure result listing the missing fields and which item they belong to
|
||||
|
||||
### Requirement: External API configuration
|
||||
|
||||
All external API base URLs SHALL be configurable via `appsettings.json` under an `ExternalApis` section. Each tool's HttpClient SHALL read its base URL from configuration at startup.
|
||||
|
||||
#### Scenario: Configuration at startup
|
||||
|
||||
- **WHEN** the API starts
|
||||
- **THEN** it reads external API base URLs from the `ExternalApis` configuration section and configures typed HttpClients accordingly
|
||||
|
||||
### Requirement: External API error handling
|
||||
|
||||
Each tool SHALL handle HTTP errors from external APIs gracefully, returning a structured error message that the LLM agent can reason about rather than throwing exceptions.
|
||||
|
||||
#### Scenario: External API unavailable
|
||||
|
||||
- **WHEN** a tool calls an external API that is unreachable
|
||||
- **THEN** the tool returns an error result with a descriptive message (e.g., "Counterparty API unavailable") so the agent can inform the user
|
||||
56
openspec/specs/few-shot-prompting/spec.md
Normal file
56
openspec/specs/few-shot-prompting/spec.md
Normal file
@@ -0,0 +1,56 @@
|
||||
## Purpose
|
||||
|
||||
Define the few-shot prompting infrastructure for extraction — example folder structure, instruction template, ChatHistory assembly, and evaluation folder.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Few-shot example folder structure
|
||||
|
||||
The system SHALL store few-shot examples at `examples/extraction/few-shot/` with numbered subdirectories (e.g., `01/`, `02/`). Each subdirectory SHALL contain `input.html` (the example email) and `output.json` (the expected ExtractionResult JSON).
|
||||
|
||||
#### Scenario: Example folder layout
|
||||
|
||||
- **WHEN** the application starts
|
||||
- **THEN** it reads example pairs from `examples/extraction/few-shot/` in numeric directory order
|
||||
|
||||
#### Scenario: Adding a new example
|
||||
|
||||
- **WHEN** a new subdirectory (e.g., `04/`) is added with `input.html` and `output.json`
|
||||
- **THEN** the new example is included in the few-shot ChatHistory prefix after the next application restart
|
||||
|
||||
### Requirement: Extraction instruction template
|
||||
|
||||
The system SHALL load a fixed instruction template from `examples/extraction/instruction-template.txt` that defines the extraction task, the TradeItem schema, and the mapping rules (date parsing, leg flattening, currency mapping, breakclause defaults). This template is NOT the user-editable system prompt.
|
||||
|
||||
#### Scenario: Template loaded at startup
|
||||
|
||||
- **WHEN** the application starts
|
||||
- **THEN** the instruction template is loaded from disk and used as the system message in the extraction ChatHistory
|
||||
|
||||
#### Scenario: Template content
|
||||
|
||||
- **WHEN** the instruction template is loaded
|
||||
- **THEN** it contains the TradeItem field definitions, expected JSON output format, and explicit mapping rules
|
||||
|
||||
### Requirement: ChatHistory assembly with few-shot examples
|
||||
|
||||
The system SHALL provide a `FewShotService` that assembles a reusable ChatHistory prefix at startup: the instruction template as a system message, followed by alternating User (input.html) and Assistant (output.json) messages for each example. Each extraction request SHALL clone this prefix and append the real email as the final user message.
|
||||
|
||||
#### Scenario: ChatHistory prefix structure
|
||||
|
||||
- **WHEN** the service assembles the prefix with 3 examples
|
||||
- **THEN** the ChatHistory contains: 1 system message + 3 user messages + 3 assistant messages (7 messages total)
|
||||
|
||||
#### Scenario: Prefix cached and cloned per request
|
||||
|
||||
- **WHEN** an extraction request arrives
|
||||
- **THEN** the service clones the cached prefix (not re-reading from disk) and appends the email content as a new user message
|
||||
|
||||
### Requirement: Evaluation example folder
|
||||
|
||||
The system SHALL support an `examples/extraction/evaluation/` folder for bulk examples used in offline testing. This folder is NOT loaded at startup and NOT used in the few-shot prompt.
|
||||
|
||||
#### Scenario: Evaluation folder ignored at runtime
|
||||
|
||||
- **WHEN** the application starts
|
||||
- **THEN** it does not load examples from `examples/extraction/evaluation/`
|
||||
47
openspec/specs/prompt-settings-api/spec.md
Normal file
47
openspec/specs/prompt-settings-api/spec.md
Normal file
@@ -0,0 +1,47 @@
|
||||
## Purpose
|
||||
|
||||
Define the shared data models and API contract for system prompt and model settings — ModelSettings class, ChatRequest extensions, and backend handling.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: ModelSettings shared model
|
||||
|
||||
The Shared project SHALL define a `ModelSettings` class with nullable properties: `Temperature` (double?), `TopP` (double?), `MaxTokens` (int?). Null values indicate "use server default".
|
||||
|
||||
#### Scenario: All fields null
|
||||
|
||||
- **WHEN** a ModelSettings instance has all null fields
|
||||
- **THEN** the backend uses Semantic Kernel default values for all parameters
|
||||
|
||||
#### Scenario: Partial override
|
||||
|
||||
- **WHEN** a ModelSettings instance has Temperature set but TopP and MaxTokens null
|
||||
- **THEN** only Temperature is overridden; other parameters use defaults
|
||||
|
||||
### Requirement: System prompt in chat request
|
||||
|
||||
The `ChatRequest` SHALL accept an optional `SystemPrompt` (string?) property. When present and non-empty, the backend SHALL insert it as the first system message in the ChatHistory before user/assistant messages.
|
||||
|
||||
#### Scenario: System prompt provided
|
||||
|
||||
- **WHEN** a ChatRequest includes a non-empty SystemPrompt
|
||||
- **THEN** the ChatHistory starts with a system message containing that text, followed by the conversation messages
|
||||
|
||||
#### Scenario: System prompt absent
|
||||
|
||||
- **WHEN** a ChatRequest has a null or empty SystemPrompt
|
||||
- **THEN** the ChatHistory contains only user and assistant messages (no system message)
|
||||
|
||||
### Requirement: Model settings in chat request
|
||||
|
||||
The `ChatRequest` SHALL accept an optional `Settings` (ModelSettings?) property. When present, the backend SHALL apply non-null values to `OpenAIPromptExecutionSettings` before calling the Semantic Kernel.
|
||||
|
||||
#### Scenario: Temperature override
|
||||
|
||||
- **WHEN** a ChatRequest includes Settings with Temperature = 0.5
|
||||
- **THEN** the OpenAIPromptExecutionSettings.Temperature is set to 0.5
|
||||
|
||||
#### Scenario: No settings provided
|
||||
|
||||
- **WHEN** a ChatRequest has null Settings
|
||||
- **THEN** the backend uses default OpenAIPromptExecutionSettings (only FunctionChoiceBehavior.Auto is set)
|
||||
57
openspec/specs/prompt-settings-ui/spec.md
Normal file
57
openspec/specs/prompt-settings-ui/spec.md
Normal file
@@ -0,0 +1,57 @@
|
||||
## Purpose
|
||||
|
||||
Define the UI controls for configuring system prompt and model parameters — tabbed layout, prompt editor, and settings controls.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: System prompt editor tab
|
||||
|
||||
The chat page SHALL include a "System Prompt" tab with a multi-line text area where the user can enter a system prompt. The system prompt value SHALL persist across tab switches within the same session.
|
||||
|
||||
#### Scenario: User enters a system prompt
|
||||
|
||||
- **WHEN** the user navigates to the System Prompt tab and types text
|
||||
- **THEN** the text is stored in the component state and included in the next chat request
|
||||
|
||||
#### Scenario: System prompt survives tab switch
|
||||
|
||||
- **WHEN** the user enters a system prompt, switches to the Chat tab, then switches back
|
||||
- **THEN** the system prompt text is unchanged
|
||||
|
||||
### Requirement: Model settings tab
|
||||
|
||||
The chat page SHALL include a "Model Settings" tab with controls for Temperature, TopP, and MaxTokens. Each control SHALL display its current value and allow adjustment within valid ranges.
|
||||
|
||||
#### Scenario: Temperature control
|
||||
|
||||
- **WHEN** the user adjusts the Temperature control
|
||||
- **THEN** the value is constrained to 0.0–2.0 and included in the next chat request's settings
|
||||
|
||||
#### Scenario: TopP control
|
||||
|
||||
- **WHEN** the user adjusts the TopP control
|
||||
- **THEN** the value is constrained to 0.0–1.0 and included in the next chat request's settings
|
||||
|
||||
#### Scenario: MaxTokens control
|
||||
|
||||
- **WHEN** the user sets the MaxTokens value
|
||||
- **THEN** the value is constrained to 1–4096 and included in the next chat request's settings
|
||||
|
||||
#### Scenario: Default values
|
||||
|
||||
- **WHEN** the user has not changed any model settings
|
||||
- **THEN** the controls show default values (Temperature: 1.0, TopP: 1.0, MaxTokens: empty/unset) and no overrides are sent to the API
|
||||
|
||||
### Requirement: Tabbed page layout
|
||||
|
||||
The chat page SHALL use MudTabs with three tab panels: "Chat" (the existing conversation UI), "System Prompt" (the prompt editor), and "Model Settings" (the parameter controls).
|
||||
|
||||
#### Scenario: Chat tab is default
|
||||
|
||||
- **WHEN** the page loads
|
||||
- **THEN** the Chat tab is active and the conversation UI is displayed
|
||||
|
||||
#### Scenario: Tab switching
|
||||
|
||||
- **WHEN** the user clicks a different tab
|
||||
- **THEN** the corresponding panel is displayed and the previous panel is hidden but retains its state
|
||||
42
openspec/specs/sidebar-navigation/spec.md
Normal file
42
openspec/specs/sidebar-navigation/spec.md
Normal file
@@ -0,0 +1,42 @@
|
||||
## Purpose
|
||||
|
||||
Define the collapsible sidebar drawer, its hamburger toggle, and the navigation menu with links to application pages.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Collapsible sidebar drawer
|
||||
|
||||
The application SHALL have a MudDrawer in MainLayout that contains a navigation menu. The drawer SHALL be toggleable via a hamburger icon button in the AppBar.
|
||||
|
||||
#### Scenario: Drawer visible on load
|
||||
|
||||
- **WHEN** the application loads
|
||||
- **THEN** the sidebar drawer is displayed in its default open state with navigation links visible
|
||||
|
||||
#### Scenario: Drawer toggles on hamburger click
|
||||
|
||||
- **WHEN** the user clicks the hamburger icon in the AppBar
|
||||
- **THEN** the drawer toggles between open and collapsed states
|
||||
|
||||
### Requirement: Navigation menu with Sales Assistant link
|
||||
|
||||
The sidebar drawer SHALL contain a MudNavMenu with a "Sales Assistant" navigation link that routes to `/sales-assistant`.
|
||||
|
||||
#### Scenario: Sales Assistant link present
|
||||
|
||||
- **WHEN** the drawer is open
|
||||
- **THEN** a "Sales Assistant" link with a SmartToy icon is visible in the navigation menu
|
||||
|
||||
#### Scenario: Clicking Sales Assistant navigates to chat
|
||||
|
||||
- **WHEN** the user clicks the "Sales Assistant" link
|
||||
- **THEN** the browser navigates to `/sales-assistant` and the chat page renders in MudMainContent
|
||||
|
||||
### Requirement: NavMenu is a separate component
|
||||
|
||||
The navigation menu SHALL be implemented as a separate `NavMenu.razor` component in the Layout folder, referenced from MainLayout.
|
||||
|
||||
#### Scenario: NavMenu renders inside drawer
|
||||
|
||||
- **WHEN** MainLayout renders
|
||||
- **THEN** the NavMenu component renders inside the MudDrawer with its navigation links
|
||||
Reference in New Issue
Block a user