W37–W38: Continuous Updates (content pipeline)

W37–W38: Continuous Updates (content pipeline)

What you’re doing

You stop letting your knowledge base rot.

Every KB dies the same way:

• people write a few docs
• nobody updates them
• they become wrong
• everyone stops trusting KB

So you build a pipeline that makes updates cheap:

• new docs are easy to add
• old docs get reviewed
• ingestion runs automatically
• retrieval tests catch breakage

Time: 4–6 hours/week
Output: a content pipeline that keeps KB fresh: versioning, review loop, automated ingestion, and quality checks

The promise (what you’ll have by the end)

By the end of W38 you will have:

• A clear “content lifecycle” (draft → reviewed → published → retired)
• A folder structure that supports growth
• Automated ingestion on change
• Retrieval tests as a quality gate
• A “stale content” checklist (what must be reviewed regularly)

The rule: freshness is part of quality

An outdated runbook is worse than no runbook.
Because it creates confident wrong actions.

So you treat content like code:

• versioned
• reviewed
• tested
• released

Step-by-step checklist

1) Define content lifecycle

Simple statuses:

• DRAFT
• REVIEWED
• PUBLISHED
• DEPRECATED

Add this status into metadata for every doc.

2) Add ownership

Every doc must have:

• owner
• last_reviewed_at
• review_frequency (e.g., 90 days)

If there is no owner, it will rot.

3) Create KB folder conventions

Example:

• knowledge/runbooks/
• knowledge/rca/
• knowledge/sop/
• knowledge/mappings/
• knowledge/known-issues/

Keep it boring and consistent.

4) Automate ingestion

On every change to /knowledge:

• regenerate chunks.jsonl
• update index data (even if local for now)

If you have GitHub Actions:

• run ingestion on PR
• store artifact (optional)

If you don’t, at least create one command:

• `make ingest` (or similar)

5) Add retrieval tests as a gate

Use your W31–W32 benchmark.
If top-5 hit rate drops below threshold:

• fail the pipeline / CI
• force fixes before merge

This prevents “docs changed and retrieval broke” silently.

6) Stale content detection

Create a simple script/report:

• list docs not reviewed in 90/180 days
• list docs with deprecated status
• list docs missing owner or metadata

This keeps the KB alive.

Deliverables (you must ship these)

Deliverable A — Content lifecycle rules

• a doc describing statuses and transitions
• metadata fields defined

Deliverable B — Automation command

• one command to run ingestion end-to-end
• output generated consistently

Deliverable C — Quality gate

• retrieval benchmark threshold defined
• failing threshold produces non-zero exit

Deliverable D — Stale content report

• report lists docs to review/retire
• missing metadata is flagged

Common traps (don’t do this)

No. You won’t. Automate it.

• Trap 1: “We’ll remember to update docs.”

No owner = dead content.

• Trap 2: “No ownership.”

If you don’t test retrieval, it will degrade silently.

• Trap 3: “No tests for KB.”

Quick self-check (2 minutes)

Answer yes/no:

• Does every doc have owner + review date?
• Can I run ingestion with one command?
• Do retrieval tests run automatically and block regressions?
• Do I have a stale content report?
• Is content lifecycle documented and used?

If any “no” — fix it before moving on.

Next module preview (W39–W40)

Next: Review & Hardening (quality gates).
We’ll make the whole KB system reliable enough to show in a portfolio.