Local AI Cat module
GGUF backend (llama.cpp sidecar)
Current shipping, testing, and product context for this part of Local AI Cat.
Status
Where it stands
- Platforms
- macOS
- Release channels
- Alpha
- Distribution
- Not currently distributed
- AlphamacOS
Description
What it does
macOS ALPHA ONLY since 2026-08-26 (package 121) — it was Outdoor + Beta until then. Sidecar-first (see docs/planning/done/gguf-backend-spike-notes.md): FeatureGGUF's LlamaServerSidecar actor locates a llama-server binary, spawns it on a free loopback port for the selected .gguf model, health-checks /health, detects crashes and restarts (bounded), and terminates gracefully. GGUFChatBackend proxies /v1/chat/completions SSE into LocalAITextGenerationEvents behind the SAME LocalAITextGenerating seam RemoteCatBackend uses, registered through ChatBackendRegistry with a `gguf:`-prefixed model contributor. GGUF-only LM Studio dirs are no longer rejected by ExternalModelSource — they carry format=.gguf and are served here instead of the MLX loader. Physically absent from App Store + iOS (package unlinked, FEATURE_GGUF undefined). Deferred: bundling llama-server (size + signing/notarization) and GGUF model-download UX. Those two deferrals are WHY it is parked to alpha: with no bundled binary the ModelCatalogContributor still contributed every discovered .gguf file, so an Outdoor user with an LM Studio library could pick a model and only learn at send time that it needed `brew install llama.cpp`. `target` records the intended end-state (Outdoor + Beta); package 121 carries the bundling, the binary-gated contributor and the promotion gate.
Headless Local API
Drive it over the loopback API
Logging / observability: provisional grade L0 — structured=0 print=40 sentry=0.
- discover
- not applicable
- execute
- present
- observe
- not applicable
- configure
- not applicable
- cancel/stop
- not applicable
- reset
- not applicable
/v1/local/app/action/v1/local/app/artifacts/{id}/v1/local/app/navigate/v1/local/app/screenshot/v1/local/app/state/v1/local/app/surfaces/v1/local/app/transactions/{id}/v1/local/approvals/pending/v1/local/approvals/{id}/v1/local/artifacts
Media
See it in action
Backlog / improvements
What comes next
No improvements are listed for this module yet.
History
How it got here
No change history has been published for this module yet.

