101 lines
4.0 KiB
Markdown
101 lines
4.0 KiB
Markdown
---
|
|
name: wiki-ingest
|
|
description: >
|
|
Ingest sources into the Obsidian wiki vault. Reads a source, extracts entities
|
|
and concepts, creates or updates wiki pages, cross-references, and logs the operation.
|
|
Supports single sources and batch mode. Triggers on: "ingest", "process this source",
|
|
"add this to the wiki", "read and file this", "batch ingest", "ingest all of these".
|
|
allowed-tools: ["Read", "Write", "Edit", "Bash", "Glob", "Grep"]
|
|
---
|
|
|
|
# wiki-ingest — Source Ingestion
|
|
|
|
Read the source. Write the wiki. Cross-reference everything. A single source typically touches 8-15 wiki pages.
|
|
|
|
---
|
|
|
|
## Single Source Ingest
|
|
|
|
Trigger: user drops a file into `.raw/` or pastes content.
|
|
|
|
Steps:
|
|
|
|
1. **Read** the source completely. Do not skim.
|
|
2. **Discuss** key takeaways with the user. Ask: "What should I emphasize? How granular?" Skip this if the user says "just ingest it."
|
|
3. **Create** source summary in `wiki/sources/`. Use the source frontmatter schema from `references/frontmatter.md`.
|
|
4. **Create or update** entity pages for every person, org, product, and repo mentioned. One page per entity.
|
|
5. **Create or update** concept pages for significant ideas and frameworks.
|
|
6. **Update** relevant domain page(s) and their `_index.md` sub-indexes.
|
|
7. **Update** `wiki/overview.md` if the big picture changed.
|
|
8. **Update** `wiki/index.md`. Add entries for all new pages.
|
|
9. **Update** `wiki/hot.md` with this ingest's context.
|
|
10. **Append** to `wiki/log.md` (new entries at the TOP):
|
|
```markdown
|
|
## [YYYY-MM-DD] ingest | Source Title
|
|
- Source: `.raw/articles/filename.md`
|
|
- Summary: [[Source Title]]
|
|
- Pages created: [[Page 1]], [[Page 2]]
|
|
- Pages updated: [[Page 3]], [[Page 4]]
|
|
- Key insight: One sentence on what is new.
|
|
```
|
|
11. **Check for contradictions.** If new info conflicts with existing pages, add `> [!contradiction]` callouts on both pages.
|
|
|
|
---
|
|
|
|
## Batch Ingest
|
|
|
|
Trigger: user drops multiple files or says "ingest all of these."
|
|
|
|
Steps:
|
|
|
|
1. List all files to process. Confirm with user before starting.
|
|
2. Process each source following the single ingest flow. Defer cross-referencing between sources until step 3.
|
|
3. After all sources: do a cross-reference pass. Look for connections between the newly ingested sources.
|
|
4. Update index, hot cache, and log once at the end (not per-source).
|
|
5. Report: "Processed N sources. Created X pages, updated Y pages. Here are the key connections I found."
|
|
|
|
Batch ingest is less interactive. For 30+ sources, expect significant processing time. Check in with the user after every 10 sources.
|
|
|
|
---
|
|
|
|
## Context Window Discipline
|
|
|
|
Token budget matters. Follow these rules during ingest:
|
|
|
|
- Read `wiki/hot.md` first. If it contains the relevant context, don't re-read full pages.
|
|
- Read `wiki/index.md` to find existing pages before creating new ones.
|
|
- Read only 3-5 existing pages per ingest. If you need 10+, you are reading too broadly.
|
|
- Use PATCH for surgical edits. Never re-read an entire file just to update one field.
|
|
- Keep wiki pages short. 100-300 lines max. If a page grows beyond 300 lines, split it.
|
|
- Use search (`/search/simple/`) to find specific content without reading full pages.
|
|
|
|
---
|
|
|
|
## Contradictions
|
|
|
|
When new info contradicts an existing wiki page:
|
|
|
|
On the existing page, add:
|
|
```markdown
|
|
> [!contradiction] Conflict with [[New Source]]
|
|
> [[Existing Page]] claims X. [[New Source]] says Y.
|
|
> Needs resolution — check dates, context, and primary sources.
|
|
```
|
|
|
|
On the new source summary, reference it:
|
|
```markdown
|
|
> [!contradiction] Contradicts [[Existing Page]]
|
|
> This source says Y, but existing wiki says X. See [[Existing Page]] for details.
|
|
```
|
|
|
|
Do not silently overwrite old claims. Flag and let the user decide.
|
|
|
|
---
|
|
|
|
## What Not to Do
|
|
|
|
- Do not modify anything in `.raw/`. These are immutable source documents.
|
|
- Do not create duplicate pages. Always check the index and search before creating.
|
|
- Do not skip the log entry. Every ingest must be recorded.
|
|
- Do not skip the hot cache update. It is what keeps future sessions fast.
|