feat: add extraction schema, sidebar nav, few-shot prompting, and prompt settings
Overhaul extraction pipeline with new TradeItem model, conversation flow, and dedicated extraction endpoint. Add sidebar navigation with NavMenu component and landing page. Introduce few-shot prompting service and tests. Add prompt settings and email upload specs. Update OpenSpec tooling with improved export-spec and extract-feature commands. Archive completed changes and export full specs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
local
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-06
|
||||
@@ -0,0 +1,73 @@
|
||||
## Context
|
||||
|
||||
The extraction pipeline currently uses a placeholder schema (`ExtractedFields` with Client, Project, Hours, Rate, Currency, Date) and a single in-process validation method. The real domain is CVA trade extraction from sales emails: HTML emails containing swap/leg tables need to be parsed into an array of `TradeItem` objects. Validation and enrichment require calling 3-5 existing external APIs (counterparty lookup, trade validation, etc.), some of which return multiple candidates requiring user disambiguation.
|
||||
|
||||
The `ExtractionPlugin` is already registered with Semantic Kernel and auto-invoked via `FunctionChoiceBehavior.Auto()`. The tool calling loop and SSE streaming infrastructure are in place.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Replace placeholder models with the real TradeItem schema
|
||||
- Replace single validation method with multiple external API tool wrappers
|
||||
- Support disambiguation workflow where tool results return candidate lists
|
||||
- Keep the external API integration configurable and testable
|
||||
|
||||
**Non-Goals:**
|
||||
- Few-shot prompting infrastructure (separate change: `few-shot-prompt-infrastructure`)
|
||||
- Email upload/drag-drop UX (separate change: `email-upload-ux`)
|
||||
- Building the external APIs themselves (they already exist)
|
||||
- Changing the SSE streaming contract or client-side rendering
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Schema structure: wrapper with items array
|
||||
|
||||
**Decision:** Use `ExtractionResult { List<TradeItem> Items }` rather than returning a flat `TradeItem`.
|
||||
|
||||
**Why:** A single email typically contains multiple swaps, each with multiple legs. Each leg becomes a separate `TradeItem`. The wrapper allows returning all items from one extraction pass.
|
||||
|
||||
**Alternative considered:** Returning `List<TradeItem>` directly. Rejected because a wrapper object is more extensible (could add metadata like extraction confidence, email subject, etc. later) and produces cleaner JSON (`{"items": [...]}` vs bare array).
|
||||
|
||||
### 2. TradeItem uses snake_case JSON property names
|
||||
|
||||
**Decision:** Use `[JsonPropertyName("valuedate")]` etc. to produce snake_case JSON output.
|
||||
|
||||
**Why:** The existing external APIs and downstream consumers expect snake_case. The C# properties will use PascalCase per convention, with JSON attributes for serialization.
|
||||
|
||||
### 3. One plugin class with multiple methods vs multiple plugin classes
|
||||
|
||||
**Decision:** Single `ExtractionPlugin` class with 3-5 `[KernelFunction]` methods.
|
||||
|
||||
**Why:** All tools serve the same extraction workflow. SK discovers functions by class, so a single class keeps registration simple (`ImportPluginFromObject`). If tools grow beyond 6-7, split into focused plugin classes.
|
||||
|
||||
**Alternative considered:** Separate plugin class per external API. Rejected as premature — adds registration complexity for no benefit at 3-5 methods.
|
||||
|
||||
### 4. External API calls via typed HttpClients
|
||||
|
||||
**Decision:** Register typed `HttpClient` instances for each external API via `AddHttpClient<T>()` in `Program.cs`. Each plugin method receives its client via constructor injection.
|
||||
|
||||
**Why:** Typed HttpClients are the standard ASP.NET Core pattern. They support per-client base URL configuration, DI, and are easily mockable in tests.
|
||||
|
||||
### 5. Disambiguation via tool return values, not special UI
|
||||
|
||||
**Decision:** When a tool like `lookup_counterparty` finds multiple matches, it returns the candidate list as JSON. The LLM agent sees the candidates and asks the user to choose via natural language in the streamed response.
|
||||
|
||||
**Why:** This uses the existing streaming conversation infrastructure with zero client changes. The agent is already conversational — it simply says "I found 3 matches, which one?" and the user replies.
|
||||
|
||||
**Alternative considered:** Structured disambiguation UI (radio buttons, dropdowns). Rejected for this change — adds client complexity and a new response type. Can be added later as a UX enhancement.
|
||||
|
||||
### 6. ExtractionPlugin conditionally loaded per request
|
||||
|
||||
**Decision:** Continue importing extraction plugins per-request in the controller (`_kernel.ImportPluginFromObject`), not at startup.
|
||||
|
||||
**Why:** General chat requests don't need extraction tools. Per-request import keeps the tool list clean for non-extraction conversations, reducing token usage and avoiding confusing the LLM with irrelevant tools.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
**[External API availability]** → Plugin methods should handle HTTP errors gracefully and return structured error responses that the LLM can reason about (e.g., "Counterparty API unavailable, proceeding without legal entity lookup"). The agent can then inform the user.
|
||||
|
||||
**[Breaking change to ExtractedFields]** → Any code referencing the old schema breaks. Mitigation: this is a small codebase with known consumers. Update all references in the same change. Tests will catch missed spots.
|
||||
|
||||
**[LLM disambiguation quality]** → The agent must correctly interpret candidate lists and present clear choices. Mitigation: tool descriptions must be explicit about what the return values mean. Few-shot examples (next change) will reinforce the pattern.
|
||||
|
||||
**[External API response format coupling]** → Plugin methods parse external API responses. If those APIs change, plugins break. Mitigation: typed response DTOs with defensive deserialization. External APIs are stable and owned by the same team.
|
||||
@@ -0,0 +1,28 @@
|
||||
## Why
|
||||
|
||||
The current extraction schema (`ExtractedFields`) uses placeholder fields (Client, Project, Hours, Rate, Currency, Date) that don't match the real domain. The actual use case is CVA (Credit Valuation Adjustment) trade extraction from sales emails — parsing HTML emails into structured trade items with fields like counterparty, trade_id, pv, and legal_entity. The single `ExtractionPlugin.ValidateExtractedFields()` method also needs to be replaced with multiple tools that wrap existing external APIs for counterparty lookup, trade validation, and other checks.
|
||||
|
||||
## What Changes
|
||||
|
||||
- **Replace `ExtractedFields.cs`** with real domain models: `ExtractionResult` (wrapper) and `TradeItem` (per-trade fields: valuedate, counterparty, legal_entity, trade_id, display_ccy, pv, breakclause)
|
||||
- **Replace `ExtractionPlugin.cs`** single validation method with 3-5 SK plugin methods, each wrapping an existing external API (counterparty lookup, trade validation, currency validation, schema validation)
|
||||
- **Update `ValidationResult.cs`** to support richer results — candidate lists for disambiguation, not just pass/fail
|
||||
- **Add typed HttpClients** for the external validation/lookup APIs, configured via `appsettings.json`
|
||||
- **Update existing tests** that reference the old `ExtractedFields` and `ExtractionPlugin`
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `extraction-schema`: Defines the real TradeItem schema, ExtractionResult wrapper, and the mapping rules from email content to structured output (date format, flattening swap legs, breakclause defaults)
|
||||
- `extraction-tools`: Defines the external API tool plugins — counterparty lookup (with disambiguation), trade validation, currency validation, and final schema validation
|
||||
|
||||
### Modified Capabilities
|
||||
- `agent-extraction`: Update requirements to reference the real schema (TradeItem) instead of generic "predefined fields", and add disambiguation workflow where tool results require user selection (e.g., counterparty/legal_entity tuples)
|
||||
|
||||
## Impact
|
||||
|
||||
- **Shared models**: `ExtractedFields.cs` replaced — **BREAKING** for any code referencing old fields
|
||||
- **API plugins**: `ExtractionPlugin.cs` rewritten with new method signatures — **BREAKING** for existing tool calling behavior
|
||||
- **External dependencies**: New HTTP calls to existing external APIs (counterparty, trade, currency)
|
||||
- **Configuration**: New `appsettings.json` entries for external API base URLs
|
||||
- **Tests**: Existing extraction-related tests need rewriting against new schema and tools
|
||||
@@ -0,0 +1,53 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Predefined extraction schema
|
||||
|
||||
The system SHALL define the extraction schema as a `TradeItem` class with fields: valuedate, counterparty, legal_entity, trade_id, display_ccy, pv, breakclause. Extraction output SHALL be wrapped in an `ExtractionResult` containing a `List<TradeItem>`. All extraction output MUST conform to this schema.
|
||||
|
||||
#### Scenario: Output conforms to schema
|
||||
|
||||
- **WHEN** the agent produces extracted fields from an email
|
||||
- **THEN** every item in the output is a valid TradeItem with all required fields matching expected types
|
||||
|
||||
#### Scenario: Multiple items from one email
|
||||
|
||||
- **WHEN** the agent extracts data from an email containing multiple trade legs
|
||||
- **THEN** the output ExtractionResult contains one TradeItem per trade leg
|
||||
|
||||
### Requirement: Autonomous validation via tool calling
|
||||
|
||||
The agent SHALL validate extracted fields by calling external API tools exposed as Semantic Kernel functions. Validation tools include counterparty lookup, trade validation, currency validation, and schema validation. Each tool returns structured results that the agent reasons about.
|
||||
|
||||
#### Scenario: Validation passes
|
||||
|
||||
- **WHEN** the agent calls the schema validation tool with a complete and correct ExtractionResult
|
||||
- **THEN** the tool returns a success result and the agent returns the final output to the user
|
||||
|
||||
#### Scenario: Validation fails with fixable errors
|
||||
|
||||
- **WHEN** a validation tool returns errors for missing or malformed fields
|
||||
- **THEN** the agent re-reads the source text and attempts to fix the extraction without user intervention
|
||||
|
||||
#### Scenario: Counterparty disambiguation required
|
||||
|
||||
- **WHEN** the counterparty lookup tool returns multiple candidate (counterparty, legal_entity) tuples
|
||||
- **THEN** the agent presents the candidates to the user as a numbered list in the chat and waits for the user to select one before completing the extraction
|
||||
|
||||
### Requirement: Human-in-the-loop clarification
|
||||
|
||||
When the agent escalates to the user, the user SHALL be able to provide the missing information in natural language, and the agent SHALL incorporate the clarification and re-attempt extraction. Disambiguation of counterparty/legal_entity tuples is a specific case of human-in-the-loop clarification.
|
||||
|
||||
#### Scenario: User provides clarification
|
||||
|
||||
- **WHEN** the agent asks for clarification about missing fields and the user responds
|
||||
- **THEN** the agent incorporates the user's response into the conversation context and produces an updated extraction
|
||||
|
||||
#### Scenario: User selects counterparty from candidates
|
||||
|
||||
- **WHEN** the agent presents a numbered list of counterparty/legal_entity candidates and the user replies with a selection
|
||||
- **THEN** the agent populates the `legal_entity` field on all relevant TradeItems and proceeds with validation
|
||||
|
||||
#### Scenario: Clarification via normal chat
|
||||
|
||||
- **WHEN** the agent escalates for clarification
|
||||
- **THEN** the clarification request appears as a regular assistant message in the chat UI, and the user responds via the normal chat input
|
||||
@@ -0,0 +1,63 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: TradeItem schema
|
||||
|
||||
The system SHALL define a `TradeItem` class with the following fields representing a single trade leg extracted from a sales email:
|
||||
- `valuedate` (string, dd/MM/yyyy format)
|
||||
- `counterparty` (string, full legal name as it appears in the email)
|
||||
- `legal_entity` (string, nullable — populated after counterparty disambiguation via lookup tool)
|
||||
- `trade_id` (long, Murex trade identifier)
|
||||
- `display_ccy` (string, ISO currency code e.g. "GBP", "USD")
|
||||
- `pv` (double, present value)
|
||||
- `breakclause` (string, "Y" or "N")
|
||||
|
||||
JSON serialization SHALL use snake_case property names via `[JsonPropertyName]` attributes.
|
||||
|
||||
#### Scenario: All fields populated
|
||||
|
||||
- **WHEN** the extraction agent produces a TradeItem with all fields
|
||||
- **THEN** the JSON output contains all seven fields with snake_case keys and correct types
|
||||
|
||||
#### Scenario: Legal entity null before disambiguation
|
||||
|
||||
- **WHEN** the extraction agent produces a TradeItem before counterparty lookup
|
||||
- **THEN** the `legal_entity` field is null and all other fields are populated
|
||||
|
||||
### Requirement: ExtractionResult wrapper
|
||||
|
||||
The system SHALL define an `ExtractionResult` class containing a `List<TradeItem> Items` property. All extraction output from a single email SHALL be wrapped in this object.
|
||||
|
||||
#### Scenario: Single email with multiple trade legs
|
||||
|
||||
- **WHEN** an email contains two swaps with two legs each (4 trades total)
|
||||
- **THEN** the ExtractionResult contains an `items` array with 4 TradeItem objects
|
||||
|
||||
#### Scenario: JSON output structure
|
||||
|
||||
- **WHEN** the ExtractionResult is serialized to JSON
|
||||
- **THEN** the output has the shape `{"items": [{"valuedate": "...", ...}, ...]}`
|
||||
|
||||
### Requirement: Extraction mapping rules
|
||||
|
||||
The extraction agent SHALL follow these mapping rules when converting email content to TradeItems:
|
||||
- Each swap leg (identified by a unique Murex trade ID) becomes a separate TradeItem
|
||||
- The `valuedate` SHALL be parsed from date references in the email (e.g., "OB 27/11/2025") and formatted as dd/MM/yyyy
|
||||
- The `counterparty` SHALL be the full legal entity name as stated in the email prose
|
||||
- The `display_ccy` SHALL be derived from the currency symbol or code in the email (e.g., "£" or "PV (£)" → "GBP")
|
||||
- The `breakclause` SHALL default to "N" if not explicitly mentioned in the email
|
||||
- The `pv` SHALL be the numeric present value without formatting (no commas, no currency symbols)
|
||||
|
||||
#### Scenario: Flatten multi-leg swap into individual items
|
||||
|
||||
- **WHEN** the email contains a swap with Coupon Leg (Murex 79353083) and APD leg (Murex 79353084)
|
||||
- **THEN** the output contains two separate TradeItems, one per Murex ID
|
||||
|
||||
#### Scenario: Currency symbol to ISO code mapping
|
||||
|
||||
- **WHEN** the email shows PV values in "PV (£)" column
|
||||
- **THEN** the `display_ccy` field is set to "GBP"
|
||||
|
||||
#### Scenario: Default breakclause
|
||||
|
||||
- **WHEN** the email does not mention break clauses
|
||||
- **THEN** all TradeItems have `breakclause` set to "N"
|
||||
@@ -0,0 +1,80 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Counterparty lookup tool
|
||||
|
||||
The extraction plugin SHALL expose a `lookup_counterparty` Semantic Kernel function that accepts a counterparty name string and calls the external counterparty API. The tool SHALL return a list of candidate (counterparty, legal_entity) tuples.
|
||||
|
||||
#### Scenario: Single match found
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches exactly one record
|
||||
- **THEN** the tool returns a single candidate with the counterparty name and legal entity ID
|
||||
|
||||
#### Scenario: Multiple matches found (disambiguation needed)
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches multiple records
|
||||
- **THEN** the tool returns all matching candidates so the agent can present them to the user for selection
|
||||
|
||||
#### Scenario: No match found
|
||||
|
||||
- **WHEN** the tool is called with a counterparty name that matches no records
|
||||
- **THEN** the tool returns an empty list and an informative message so the agent can ask the user for clarification
|
||||
|
||||
### Requirement: Trade validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_trade` Semantic Kernel function that accepts a trade ID and calls the external trade validation API to verify the trade exists.
|
||||
|
||||
#### Scenario: Valid trade ID
|
||||
|
||||
- **WHEN** the tool is called with a known trade ID
|
||||
- **THEN** the tool returns a success result confirming the trade exists
|
||||
|
||||
#### Scenario: Invalid trade ID
|
||||
|
||||
- **WHEN** the tool is called with an unknown trade ID
|
||||
- **THEN** the tool returns an error result so the agent can flag it to the user
|
||||
|
||||
### Requirement: Currency validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_currency` Semantic Kernel function that accepts a currency code and calls the external currency validation API to verify it is a valid ISO currency code.
|
||||
|
||||
#### Scenario: Valid currency code
|
||||
|
||||
- **WHEN** the tool is called with "GBP"
|
||||
- **THEN** the tool returns a success result
|
||||
|
||||
#### Scenario: Invalid currency code
|
||||
|
||||
- **WHEN** the tool is called with an unrecognized code
|
||||
- **THEN** the tool returns an error with suggestions for valid codes
|
||||
|
||||
### Requirement: Schema validation tool
|
||||
|
||||
The extraction plugin SHALL expose a `validate_schema` Semantic Kernel function that accepts the full ExtractionResult JSON and validates that all required fields are present and correctly typed for every TradeItem.
|
||||
|
||||
#### Scenario: Valid extraction result
|
||||
|
||||
- **WHEN** the tool is called with a complete and correctly typed ExtractionResult JSON
|
||||
- **THEN** the tool returns a success result with no errors
|
||||
|
||||
#### Scenario: Missing required fields
|
||||
|
||||
- **WHEN** the tool is called with a TradeItem missing the `trade_id` field
|
||||
- **THEN** the tool returns a failure result listing the missing fields and which item they belong to
|
||||
|
||||
### Requirement: External API configuration
|
||||
|
||||
All external API base URLs SHALL be configurable via `appsettings.json` under an `ExternalApis` section. Each tool's HttpClient SHALL read its base URL from configuration at startup.
|
||||
|
||||
#### Scenario: Configuration at startup
|
||||
|
||||
- **WHEN** the API starts
|
||||
- **THEN** it reads external API base URLs from the `ExternalApis` configuration section and configures typed HttpClients accordingly
|
||||
|
||||
### Requirement: External API error handling
|
||||
|
||||
Each tool SHALL handle HTTP errors from external APIs gracefully, returning a structured error message that the LLM agent can reason about rather than throwing exceptions.
|
||||
|
||||
#### Scenario: External API unavailable
|
||||
|
||||
- **WHEN** a tool calls an external API that is unreachable
|
||||
- **THEN** the tool returns an error result with a descriptive message (e.g., "Counterparty API unavailable") so the agent can inform the user
|
||||
@@ -0,0 +1,38 @@
|
||||
## 1. Replace Shared Models
|
||||
|
||||
- [x] 1.1 Create `TradeItem.cs` in Shared/Models with fields: `Valuedate` (string), `Counterparty` (string), `LegalEntity` (string?), `TradeId` (long), `DisplayCcy` (string), `Pv` (double), `Breakclause` (string). Add `[JsonPropertyName]` attributes for snake_case serialization.
|
||||
- [x] 1.2 Create `ExtractionResult.cs` in Shared/Models with `List<TradeItem> Items` property
|
||||
- [x] 1.3 Delete the old `ExtractedFields.cs`
|
||||
- [x] 1.4 Update `ValidationResult.cs` to support candidate lists — add `List<CandidateMatch>? Candidates` property for disambiguation results (each with `Name` and `LegalEntity` fields)
|
||||
|
||||
## 2. Configure External API HttpClients
|
||||
|
||||
- [x] 2.1 Add `ExternalApis` section to `appsettings.json` with base URLs for counterparty, trade, and currency APIs
|
||||
- [x] 2.2 Create typed HttpClient service `CounterpartyApiClient` with `LookupAsync(string name)` method returning candidate tuples
|
||||
- [x] 2.3 Create typed HttpClient service `TradeApiClient` with `ValidateAsync(long tradeId)` method
|
||||
- [x] 2.4 Create typed HttpClient service `CurrencyApiClient` with `ValidateAsync(string currencyCode)` method
|
||||
- [x] 2.5 Register all typed HttpClients in `Program.cs` via `AddHttpClient<T>()` with base URLs from configuration
|
||||
|
||||
## 3. Rewrite ExtractionPlugin
|
||||
|
||||
- [x] 3.1 Replace `ExtractionPlugin.ValidateExtractedFields()` with `LookupCounterparty(string name)` — calls `CounterpartyApiClient`, returns candidate list as JSON. Include `[KernelFunction]` and `[Description]` attributes explaining the tool returns candidates for disambiguation.
|
||||
- [x] 3.2 Add `ValidateTrade(long tradeId)` method — calls `TradeApiClient`, returns valid/invalid result as JSON
|
||||
- [x] 3.3 Add `ValidateCurrency(string currencyCode)` method — calls `CurrencyApiClient`, returns valid/invalid result as JSON
|
||||
- [x] 3.4 Add `ValidateSchema(string extractionResultJson)` method — validates the full ExtractionResult JSON against TradeItem schema locally (required fields present, correct types, breakclause is Y or N)
|
||||
- [x] 3.5 Inject typed HttpClients into ExtractionPlugin via constructor. Register ExtractionPlugin in DI with its dependencies.
|
||||
- [x] 3.6 Add error handling in each tool method — catch `HttpRequestException`, return structured error JSON instead of throwing
|
||||
|
||||
## 4. Update Controller
|
||||
|
||||
- [x] 4.1 Update `ChatController.Post()` — no changes needed, DI resolution via GetRequiredService works with scoped registration to use the new `ExtractionPlugin` constructor (pass DI-resolved instance with HttpClients)
|
||||
|
||||
## 5. Update Tests
|
||||
|
||||
- [x] 5.1 Remove or rewrite tests referencing old `ExtractedFields` schema
|
||||
- [x] 5.2 Add unit tests for `ValidateSchema` method against TradeItem (valid, missing fields, wrong types)
|
||||
- [x] 5.3 Add unit tests for ExtractionPlugin tool methods with mocked HttpClients (success, error, multi-candidate responses)
|
||||
|
||||
## 6. Build and Verify
|
||||
|
||||
- [x] 6.1 Build the solution (`dotnet build`) and confirm no compilation errors
|
||||
- [x] 6.2 Run all tests (`dotnet test`) and confirm they pass
|
||||
Reference in New Issue
Block a user