family/AgenticCode
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add extraction schema, sidebar nav, few-shot prompting, and prompt settings

Browse Source 
Overhaul extraction pipeline with new TradeItem model, conversation flow,
and dedicated extraction endpoint. Add sidebar navigation with NavMenu
component and landing page. Introduce few-shot prompting service and
tests. Add prompt settings and email upload specs. Update OpenSpec
tooling with improved export-spec and extract-feature commands. Archive
completed changes and export full specs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
local
2026-04-06 23:39:23 +01:00
parent 7a5c22593a
commit 5b027eb0db
83 changed files with 4242 additions and 296 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
openspec/changes/archive/2026-04-06-update-extraction-schema/specs/agent-extraction/spec.md Normal file
View File

@@ -0,0 +1,53 @@
## MODIFIED Requirements
### Requirement: Predefined extraction schema
The system SHALL define the extraction schema as a `TradeItem` class with fields: valuedate, counterparty, legal_entity, trade_id, display_ccy, pv, breakclause. Extraction output SHALL be wrapped in an `ExtractionResult` containing a `List<TradeItem>`. All extraction output MUST conform to this schema.
#### Scenario: Output conforms to schema
- **WHEN** the agent produces extracted fields from an email
- **THEN** every item in the output is a valid TradeItem with all required fields matching expected types
#### Scenario: Multiple items from one email
- **WHEN** the agent extracts data from an email containing multiple trade legs
- **THEN** the output ExtractionResult contains one TradeItem per trade leg
### Requirement: Autonomous validation via tool calling
The agent SHALL validate extracted fields by calling external API tools exposed as Semantic Kernel functions. Validation tools include counterparty lookup, trade validation, currency validation, and schema validation. Each tool returns structured results that the agent reasons about.
#### Scenario: Validation passes
- **WHEN** the agent calls the schema validation tool with a complete and correct ExtractionResult
- **THEN** the tool returns a success result and the agent returns the final output to the user
#### Scenario: Validation fails with fixable errors
- **WHEN** a validation tool returns errors for missing or malformed fields
- **THEN** the agent re-reads the source text and attempts to fix the extraction without user intervention
#### Scenario: Counterparty disambiguation required
- **WHEN** the counterparty lookup tool returns multiple candidate (counterparty, legal_entity) tuples
- **THEN** the agent presents the candidates to the user as a numbered list in the chat and waits for the user to select one before completing the extraction
### Requirement: Human-in-the-loop clarification
When the agent escalates to the user, the user SHALL be able to provide the missing information in natural language, and the agent SHALL incorporate the clarification and re-attempt extraction. Disambiguation of counterparty/legal_entity tuples is a specific case of human-in-the-loop clarification.
#### Scenario: User provides clarification
- **WHEN** the agent asks for clarification about missing fields and the user responds
- **THEN** the agent incorporates the user's response into the conversation context and produces an updated extraction
#### Scenario: User selects counterparty from candidates
- **WHEN** the agent presents a numbered list of counterparty/legal_entity candidates and the user replies with a selection
- **THEN** the agent populates the `legal_entity` field on all relevant TradeItems and proceeds with validation
#### Scenario: Clarification via normal chat
- **WHEN** the agent escalates for clarification
- **THEN** the clarification request appears as a regular assistant message in the chat UI, and the user responds via the normal chat input

63
openspec/changes/archive/2026-04-06-update-extraction-schema/specs/extraction-schema/spec.md Normal file
View File

@@ -0,0 +1,63 @@
## ADDED Requirements
### Requirement: TradeItem schema
The system SHALL define a `TradeItem` class with the following fields representing a single trade leg extracted from a sales email:
- `valuedate` (string, dd/MM/yyyy format)
- `counterparty` (string, full legal name as it appears in the email)
- `legal_entity` (string, nullable — populated after counterparty disambiguation via lookup tool)
- `trade_id` (long, Murex trade identifier)
- `display_ccy` (string, ISO currency code e.g. "GBP", "USD")
- `pv` (double, present value)
- `breakclause` (string, "Y" or "N")
JSON serialization SHALL use snake_case property names via `[JsonPropertyName]` attributes.
#### Scenario: All fields populated
- **WHEN** the extraction agent produces a TradeItem with all fields
- **THEN** the JSON output contains all seven fields with snake_case keys and correct types
#### Scenario: Legal entity null before disambiguation
- **WHEN** the extraction agent produces a TradeItem before counterparty lookup
- **THEN** the `legal_entity` field is null and all other fields are populated
### Requirement: ExtractionResult wrapper
The system SHALL define an `ExtractionResult` class containing a `List<TradeItem> Items` property. All extraction output from a single email SHALL be wrapped in this object.
#### Scenario: Single email with multiple trade legs
- **WHEN** an email contains two swaps with two legs each (4 trades total)
- **THEN** the ExtractionResult contains an `items` array with 4 TradeItem objects
#### Scenario: JSON output structure
- **WHEN** the ExtractionResult is serialized to JSON
- **THEN** the output has the shape `{"items": [{"valuedate": "...", ...}, ...]}`
### Requirement: Extraction mapping rules
The extraction agent SHALL follow these mapping rules when converting email content to TradeItems:
- Each swap leg (identified by a unique Murex trade ID) becomes a separate TradeItem
- The `valuedate` SHALL be parsed from date references in the email (e.g., "OB 27/11/2025") and formatted as dd/MM/yyyy
- The `counterparty` SHALL be the full legal entity name as stated in the email prose
- The `display_ccy` SHALL be derived from the currency symbol or code in the email (e.g., "£" or "PV (£)" → "GBP")
- The `breakclause` SHALL default to "N" if not explicitly mentioned in the email
- The `pv` SHALL be the numeric present value without formatting (no commas, no currency symbols)
#### Scenario: Flatten multi-leg swap into individual items
- **WHEN** the email contains a swap with Coupon Leg (Murex 79353083) and APD leg (Murex 79353084)
- **THEN** the output contains two separate TradeItems, one per Murex ID
#### Scenario: Currency symbol to ISO code mapping
- **WHEN** the email shows PV values in "PV (£)" column
- **THEN** the `display_ccy` field is set to "GBP"
#### Scenario: Default breakclause
- **WHEN** the email does not mention break clauses
- **THEN** all TradeItems have `breakclause` set to "N"

80
openspec/changes/archive/2026-04-06-update-extraction-schema/specs/extraction-tools/spec.md Normal file
View File

@@ -0,0 +1,80 @@
## ADDED Requirements
### Requirement: Counterparty lookup tool
The extraction plugin SHALL expose a `lookup_counterparty` Semantic Kernel function that accepts a counterparty name string and calls the external counterparty API. The tool SHALL return a list of candidate (counterparty, legal_entity) tuples.
#### Scenario: Single match found
- **WHEN** the tool is called with a counterparty name that matches exactly one record
- **THEN** the tool returns a single candidate with the counterparty name and legal entity ID
#### Scenario: Multiple matches found (disambiguation needed)
- **WHEN** the tool is called with a counterparty name that matches multiple records
- **THEN** the tool returns all matching candidates so the agent can present them to the user for selection
#### Scenario: No match found
- **WHEN** the tool is called with a counterparty name that matches no records
- **THEN** the tool returns an empty list and an informative message so the agent can ask the user for clarification
### Requirement: Trade validation tool
The extraction plugin SHALL expose a `validate_trade` Semantic Kernel function that accepts a trade ID and calls the external trade validation API to verify the trade exists.
#### Scenario: Valid trade ID
- **WHEN** the tool is called with a known trade ID
- **THEN** the tool returns a success result confirming the trade exists
#### Scenario: Invalid trade ID
- **WHEN** the tool is called with an unknown trade ID
- **THEN** the tool returns an error result so the agent can flag it to the user
### Requirement: Currency validation tool
The extraction plugin SHALL expose a `validate_currency` Semantic Kernel function that accepts a currency code and calls the external currency validation API to verify it is a valid ISO currency code.
#### Scenario: Valid currency code
- **WHEN** the tool is called with "GBP"
- **THEN** the tool returns a success result
#### Scenario: Invalid currency code
- **WHEN** the tool is called with an unrecognized code
- **THEN** the tool returns an error with suggestions for valid codes
### Requirement: Schema validation tool
The extraction plugin SHALL expose a `validate_schema` Semantic Kernel function that accepts the full ExtractionResult JSON and validates that all required fields are present and correctly typed for every TradeItem.
#### Scenario: Valid extraction result
- **WHEN** the tool is called with a complete and correctly typed ExtractionResult JSON
- **THEN** the tool returns a success result with no errors
#### Scenario: Missing required fields
- **WHEN** the tool is called with a TradeItem missing the `trade_id` field
- **THEN** the tool returns a failure result listing the missing fields and which item they belong to
### Requirement: External API configuration
All external API base URLs SHALL be configurable via `appsettings.json` under an `ExternalApis` section. Each tool's HttpClient SHALL read its base URL from configuration at startup.
#### Scenario: Configuration at startup
- **WHEN** the API starts
- **THEN** it reads external API base URLs from the `ExternalApis` configuration section and configures typed HttpClients accordingly
### Requirement: External API error handling
Each tool SHALL handle HTTP errors from external APIs gracefully, returning a structured error message that the LLM agent can reason about rather than throwing exceptions.
#### Scenario: External API unavailable
- **WHEN** a tool calls an external API that is unreachable
- **THEN** the tool returns an error result with a descriptive message (e.g., "Counterparty API unavailable") so the agent can inform the user