← All modules

Local AI Cat module

GGUF backend (llama.cpp sidecar)

Current shipping, testing, and product context for this part of Local AI Cat.

Status

Where it stands

Lane: AlphaStatus: BetaTesting: prov. +behavioral
Platforms
macOS
Release channels
Alpha
Distribution
Not currently distributed
  • AlphamacOS

Description

What it does

macOS ALPHA ONLY since 2026-08-26 (package 121) — it was Outdoor + Beta until then. Sidecar-first (see docs/planning/done/gguf-backend-spike-notes.md): FeatureGGUF's LlamaServerSidecar actor locates a llama-server binary, spawns it on a free loopback port for the selected .gguf model, health-checks /health, detects crashes and restarts (bounded), and terminates gracefully. GGUFChatBackend proxies /v1/chat/completions SSE into LocalAITextGenerationEvents behind the SAME LocalAITextGenerating seam RemoteCatBackend uses, registered through ChatBackendRegistry with a `gguf:`-prefixed model contributor. GGUF-only LM Studio dirs are no longer rejected by ExternalModelSource — they carry format=.gguf and are served here instead of the MLX loader. Physically absent from App Store + iOS (package unlinked, FEATURE_GGUF undefined). Deferred: bundling llama-server (size + signing/notarization) and GGUF model-download UX. Those two deferrals are WHY it is parked to alpha: with no bundled binary the ModelCatalogContributor still contributed every discovered .gguf file, so an Outdoor user with an LM Studio library could pick a model and only learn at send time that it needed `brew install llama.cpp`. `target` records the intended end-state (Outdoor + Beta); package 121 carries the bundling, the binary-gated contributor and the promotion gate.

Headless Local API

Drive it over the loopback API

API parity: PartialLogging: L0 · prov.

Logging / observability: provisional grade L0structured=0 print=40 sentry=0.

discover
not applicable
execute
present
observe
not applicable
configure
not applicable
cancel/stop
not applicable
reset
not applicable
localApi
  • /v1/local/app/action
  • /v1/local/app/artifacts/{id}
  • /v1/local/app/navigate
  • /v1/local/app/screenshot
  • /v1/local/app/state
  • /v1/local/app/surfaces
  • /v1/local/app/transactions/{id}
  • /v1/local/approvals/pending
  • /v1/local/approvals/{id}
  • /v1/local/artifacts

Media

See it in action

Walkthrough coming soonUI-test video · GIF fallback
A short walkthrough will appear here when the module's UI test media is ready.

Backlog / improvements

What comes next

No improvements are listed for this module yet.

History

How it got here

No change history has been published for this module yet.