EXP-0001 — AutoWiki by Factory.ai: a pattern worth borrowing, a product we can't run

23 June 2026#forge#documentation#wiki#patterns#factory-ai#build-failed#open-source

Imagine every time someone updates the code for a piece of software, a fresh, well-organized manual for that software gets written automatically — covering how the parts fit together, where the important pieces are, and how to find your way around. No engineer has to stop coding to write it. The manual just exists, and it's always up to date because it's generated from the same code it describes.

That's the basic idea behind AutoWiki, a feature from a startup called Factory.ai. They've built it as part of a larger paid platform aimed at engineering teams. Every push to a code repository triggers their cloud service to regenerate the documentation, which then appears in several places at once — on GitHub's built-in wiki, on Factory.ai's own viewer, and inside the repository itself.

This is the kind of project that lands in our #development channel with a 🧪 reaction because it's a clever idea worth understanding. Forge is our experiment harness — when a project gets marked, forge tries to clone it, build it in a sandboxed container, run it, and write up what it found. The whole point is to verify the claim before trusting it.

But forge couldn't run this one. AutoWiki is a hosted commercial service — there's no source code to download, no version you can install on your own machine. The actual work happens on Factory.ai's servers. So forge can't bench it the way it benches open-source tools.

Rather than skip it, we wrote up what's interesting about the pattern — the design ideas worth borrowing, even though we can't run the product itself. Three of those patterns map directly onto things we already do or want to do. We also list several open-source projects that implement similar ideas — any one of which forge could run end-to-end, and any one of which we'd happily 🧪 next.

Status: build-failed (not-clonable-proprietary). Forge will not run a hosted SaaS through its sandbox lifecycle, by design. But the pattern underneath this product is one of the most useful things to land in our 🧪 feed this month, and that's worth a post.

TL;DR

What it is. A hosted Factory.ai feature that auto-generates and incrementally refreshes a per-repo developer wiki, keyed off commit-hash diff, on every push to the default branch.
Why forge can't bench it. No public source repo; no self-host option. The whole pipeline (analysis → page generation → multi-surface fan-out) lives behind Factory.ai's API. Forge's sandbox is offline-by-default and refuses to run cloud APIs.
Why we wrote it up anyway. Three reusable patterns are visible from the outside: docs-as-build-artifact, incremental-diff regeneration, and multi-surface output (GitHub Wiki tab + cloud viewer + in-repo droid-wiki/ + Droid session). Each maps directly onto something we already do or want to do in app-state and wiki-*.
Open comparables. Several adjacent open-source projects (MxDkl/AutoWiki, Astro-Han/karpathy-llm-wiki, TheMorpheus407/RepoLens) — any one of which could become a real bench experiment if we 🧪 it.

What AutoWiki does

Factory.ai positions itself as an "agent-native software factory" — agents called Droids work inside their IDE and CLI. AutoWiki is the documentation arm of that platform. On install, it ships a GitHub Actions / GitLab CI workflow (/install-wiki) that runs on every push to the default branch. The workflow:

Sends the changed file set to Factory's analysis API.
Receives a regenerated set of markdown pages organized around how the code actually works (entry points, modules, key flows) rather than as a manual afterthought.
Writes the result to four destinations in parallel: GitHub's built-in Wiki tab, Factory's hosted viewer at app.factory.ai/wiki, a droid-wiki/ directory committed back to the repo, and the active Droid session's context window.

The on-disk shape — markdown organized by code structure — and the multi-surface output are exactly the shape app-state produces. The difference is that AutoWiki spends Factory.ai's compute on the regeneration, whereas app-state runs locally and writes flat files. Architecturally adjacent, governance opposite.

Why this is "build-failed"

Forge's contract is simple: clone the repo, build it inside a node:22 / python:3.12 / etc. container with 4 CPU and 8 GB, run a smoke probe, write up what we found. There are exactly two ways for forge to engage with a project:

There's a public repo → clone, pin a commit, build it. Most experiments end up here.
There isn't → record the reason, write a pattern note, publish.

AutoWiki is case 2. Factory.ai has not open-sourced the product, doesn't offer a self-hostable edition, and the /install-wiki workflow calls back to Factory's analysis API rather than running a local generator. There is no clone target. Forge marks this build-failed with reason: not-clonable-proprietary and routes the experiment straight to the reporter. This is not a complaint about Factory.ai's licensing choices — it's a structural statement: forge cannot verify what it cannot run.

It's worth being explicit that "build-failed" in forge's vocabulary is not the same as "rejected." Several lifecycle states are terminal-with-findings, and a pattern note is the most useful artifact a not-clonable experiment can produce. Recording the rejection cleanly is part of how we keep the queue honest: future 🧪 reactions on similar hosted products will route the same way without anyone re-litigating the rule.

The three reusable patterns

Even without the source, the product surface is described in enough detail to extract the design.

1. Documentation as an automated build artifact

The framing matters. Most teams treat documentation as a manual, side-of-the-desk activity that competes with shipping. AutoWiki recasts it as a build artifact: every commit produces docs the same way every commit produces a compiled binary. The implication is that documentation drift becomes impossible by construction — the docs are always at HEAD because they're generated from HEAD.

This is the same framing app-state uses for status reports, claim drafts, and SC packs in project-state. We didn't borrow it from AutoWiki — we landed in the same place independently — but seeing a venture-funded company adopt the same posture is a useful confirmation that we're not alone in this direction.

2. Incremental regeneration keyed off commit diff

Naively, a wiki generator should re-analyze the entire codebase on every push. That's expensive on a 50 kLOC repo and prohibitive on a 5 MLOC monorepo. AutoWiki sidesteps this by diffing the current commit against the last-generated commit, regenerating only the affected pages, and leaving the rest of the wiki untouched. From the operator's perspective, the wiki refreshes in seconds even on large repos; from the platform's perspective, compute scales with churn rather than total size.

This pattern is directly applicable to app-state's blog drafts (re-synthesize only the sections whose source signals changed), to notella's daily/weekly syntheses (only the days whose source pages were edited), and to forge itself (only re-run the experimenter when the smoke fixture or the build commit changes). It's a cheap optimization that we should bake into the substrate spine.

3. Multi-surface fan-out from a single canonical source

The same regenerated wiki appears in four places: GitHub's Wiki tab, Factory's cloud viewer, an in-repo droid-wiki/ directory, and the Droid session context. All four are read-only renderings of the same canonical markdown produced by the generator. There's no "primary" surface; whichever surface the reader prefers, the content is the same.

We already do a version of this with scsiwyg (blog + newsletter + Confluence) and forge-publisher (gist + blog + work-state event). The lesson from AutoWiki is to be explicit about it: name the canonical source, name every rendering surface, document the fan-out, and resist any pressure to let surfaces drift from each other.

Open comparables worth 🧪-ing

If the patterns above are interesting, here are the open projects that implement adjacent versions of them. Any of these would build cleanly under forge's standard lifecycle:

MxDkl/AutoWiki — namesake hobby project from a different author. Auto-generated wiki, small, hackable. Likely the closest open analogue.
Astro-Han/karpathy-llm-wiki — implements Karpathy's "LLM Wiki" idea against a personal corpus. Already referenced in our #development channel (2026-04-22). Strong starting point for the locally-hosted version.
TheMorpheus407/RepoLens — agent-driven repo-tour generator. More verbose output than AutoWiki, but covers similar surface area.
DeepWiki (Cognition / Devin) — hosted competitor to AutoWiki. Same not-clonable problem, useful as a comparables citation.

If David 🧪s any of these in #development, forge will pick it up automatically and we'll get a real bench experiment to compare against AutoWiki's claims.

Reproducibility


forge phase	researched → build-failed → reported → published
reason	`not-clonable-proprietary`
upstream product	https://factory.ai/news/wiki
docs	https://docs.factory.ai/cli/features/wiki/auto-refresh
example outputs	https://factory.ai/open-source-wikis/bevy, https://factory.ai/open-source-wikis/grafana
build	n/a (no clone target)
smoke probe	n/a (no listener to probe)

There is no companion gist for this experiment because there is no source to anchor. The reproducibility here is the experiment.yaml record itself — a complete audit trail of why forge declined to run this and what we recommend doing instead.