memcache

memcache is a tiny MCP memory server for whatever repository Copilot CLI is working in. It gives the agent repo-local memory: compact writes, fast recall, cited context, and an append-only journal under .agents/.memcache/.

Goals

• Go stdlib first; golang.org/x/... only when explicitly justified.
• No MCP SDK, database, embedded search engine, or reflection schema dependency.
• Stdio MCP transport first.
• Compact, cited, token-budgeted responses by default.
• Repo-local state under tracked .agents/.memcache/, never root .memcache.

Set it up with Copilot CLI

Build the server once, then point Copilot CLI at the binary.

git clone <this-repo-url> ~/src/memcache
cd ~/src/memcache
go test ./...
go build -o ~/.local/bin/memcache ./cmd/memcache

Sanity-check the binary from any repo:

cd /path/to/your/repo
~/.local/bin/memcache inspect --repo .

Add one global MCP entry to Copilot CLI. Start copilot, run /mcp add, then fill:

name: memcache
type: local
command: /home/you/.local/bin/memcache
args: serve

Press Ctrl+S to save. Copilot stores this in ~/.copilot/mcp-config.json by default, or $COPILOT_HOME/mcp-config.json if COPILOT_HOME is set. You can also edit that file directly:

{
  "mcpServers": {
    "memcache": {
      "type": "local",
      "command": "/home/you/.local/bin/memcache",
      "args": ["serve"],
      "tools": [
        "memory.record",
        "memory.search",
        "memory.context",
        "memory.timeline"
      ]
    }
  }
}

Use an absolute path for command. serve speaks MCP over stdio, so it keeps stdout clean and sends diagnostics to stderr. In MCP mode it asks Copilot for the current workspace roots and writes to that repo's .agents/.memcache/; if roots are unavailable, it falls back to the process working directory. If you really want to pin one store, use args: serve --repo /path/to/repo.

Restart copilot in that repo, check /mcp, then try:

Use memory.context for "what did we decide about the cache?"
Record a memory: kind decision, summary "Keep rollups deterministic"

State lands in the active workspace's .agents/.memcache/. Track that directory in the repo so future agents inherit memory.

Local CLI

go run ./cmd/memcache inspect --repo .
go run ./cmd/memcache inspect --repo . --json
go run ./cmd/memcache inspect --repo . --repair
go run ./cmd/memcache reindex --repo .
go run ./cmd/memcache serve
go run ./cmd/memcache record --repo . --kind decision --summary "Keep memory compact"
go run ./cmd/memcache import-jsonl --repo . events.jsonl
go run ./cmd/memcache compact --repo . --dry-run --older-than 720h
go run ./cmd/memcache search --repo . --query "kind:decision memory"
go run ./cmd/memcache context --repo . --task "fix nil pointer" --include-recent
go run ./cmd/memcache timeline --repo . --group-by day --limit 20

The local CLI is handy for health checks, imports, compaction, repair, and debugging outside MCP.

serve is reserved for MCP stdio mode. It must not write non-MCP logs to stdout; diagnostics go to stderr.

Implemented MCP methods:

• initialize
• notifications/initialized
• notifications/resources/list_changed
• notifications/roots/list_changed
• outgoing roots/list request when the client advertises workspace roots
• ping
• tools/list
• tools/call
• prompts/list
• prompts/get
• resources/list
• resources/read
• resources/templates/list
• completion/complete

MCP tools:

• memory.record: append a memory entry. Required: summary. kind defaults to note; use note when unsure.
• memory.search: ranked recall. Required: query.
• memory.context: cited task memory pack. Required: task.
• memory.timeline: chronological/grouped memory view.

MCP resources:

• mem://repo/overview: repo memory card.
• mem://repo/recent: recent timeline.
• mem://entry/{id}: one entry.
• mem://file/{path}: file memory card.

resources/list returns static repo resources plus known entry and file-card resources from the journal. List methods support MCP opaque cursor pagination with nextCursor; invalid cursors return JSON-RPC -32602.

The server advertises resources.listChanged. After initialization, successful memory.record calls emit notifications/resources/list_changed because resource discovery gains a new entry and may gain a new file card.

MCP prompts:

• memory-workflow: teach agents when to call context, search, timeline, and record.
• memory-context: guide the client to build cited task memory with memory.context.
• memory-search: guide the client to search prior work before answering or editing.
• memory-record: guide the client to capture one durable event with memory.record.
• memory-timeline: guide the client to review chronological memory for handoff or recent work.

Tool results include compact text plus structuredContent matching each tool's outputSchema. Search, context, timeline, and record responses include resource_link blocks for cited mem://entry/{id} resources so clients can fetch full entries only when needed.

Search and timeline limits are capped at 100 items. token_budget is capped at 16000 approximate tokens so MCP and CLI responses stay bounded.

Completions are implemented for resource templates:

• mem://entry/{id} completes the id argument from known memory entry IDs.
• mem://file/{path} completes the path argument from repo-relative files seen in memory.
• Prompt completions cover enum-like prompt arguments such as kind, group_by, and booleans; prompt file arguments reuse file-path completion.

Resources, resource templates, resource read contents, and entry resource links include MCP annotations for audience/priority. Entry resources and file/repo memory reads include lastModified when the source timestamp is known.

Unknown tools return JSON-RPC -32602. Tool validation/runtime failures return isError: true. Missing resources return MCP resource error -32002. Reserved MCP _meta fields in params or tool arguments are ignored; other unknown fields stay validation errors.

Store layout

.agents/.memcache/
  manifest.json
  journal/
  index/
    manifest.json
    segments/
  rollups/
    daily/
    files/
  tmp/
  locks/

journal/ holds canonical append-only JSONL memory records and is the only per-entry persistent store. index/ holds dependency-free search structures. rollups/ holds compact scannable memory cards. Older stores may contain a legacy objects/ cache; current writes do not create it, and inspect --repair prunes it.

Memory entries use these fields:

• id: stable base32 SHA-256-derived identifier
• repo: canonical repo root for this store; omit on writes to use --repo
• ts: RFC3339Nano timestamp
• kind: turn, command, edit, test, error, decision, checkpoint, note, or summary; MCP record defaults to note
• summary: required compact human line
• text, files, refs, tags, importance, session_id, actor, source, metadata: optional detail

Writes use a repo-local lock file, then append to the journal. Single-record writes append a small immutable index segment after the journal write, and automatically rebuild the index when incremental segments would exceed 16 files; batch imports rebuild the index once after the batch. The JSONL journal stays source of truth for rebuilds, and the index manifest stores a cheap journal file signature for fast freshness checks. Corrupt segment caches are rebuilt from the journal before recall or after the next write. Journal reads reject malformed, duplicate, wrong-repo, or tampered entries before index, rollup, resource, or context generation uses them.

Harness integration

Harnesses should emit compact, useful records instead of raw noisy transcripts:

• Task start/end: turn or checkpoint.
• Commands: command only when command affects debugging, build, test, or repo state.
• File edits: edit with repo-relative paths and short summary; do not store full diffs by default.
• Test/build failures: error or test with command, exit status, and compact failure snippet.
• Durable choices: decision.

CLI ingestion uses the same record validation, redaction, indexing, and rollup path as MCP memory.record. Batch JSONL import validates every line before writing, prepares/redacts accepted entries once, appends the prepared entries under one store write path, then rebuilds index/rollups once.

Record one event:

go run ./cmd/memcache record \
  --repo . \
  --kind edit \
  --summary "Add ranked search tests" \
  --file internal/index/index_test.go \
  --tag test \
  --ref master \
  --session-id "$SESSION_ID"

Import JSONL events:

{"kind":"command","summary":"Run go test","text":"go test ./... passed","tags":["test"]}
{"kind":"decision","summary":"Keep store under .agents/.memcache","files":["README.md"],"tags":["architecture"]}

go run ./cmd/memcache import-jsonl --repo . events.jsonl

Invalid JSONL lines, invalid records, wrong-repo records, duplicate import entries, and entries already present in the journal are rejected with source and line diagnostics before any import writes happen. Successful imports report imported: N and redacted: N when any records had secret-shaped fields redacted.

Compact old noisy memories:

go run ./cmd/memcache compact --repo . --older-than 720h --limit 50

compact is non-destructive: it appends one summary record with memcache_compaction metadata listing compacted entry IDs, then rebuilds index/rollups. Original journal entries remain for audit and can be searched with --include-compacted, but normal search/context/timeline/rollups suppress compacted originals. By default, candidates are low-importance turn, command, edit, test, error, and note entries older than the cutoff; durable decision, checkpoint, and existing summary entries stay active. Use --dry-run to preview candidates without writing.

Search from CLI:

go run ./cmd/memcache search --repo . --query "nil pointer kind:error session:$SESSION_ID" --file internal/index

Build a cited task context pack from CLI:

go run ./cmd/memcache context \
  --repo . \
  --task "fix nil pointer in memory loader" \
  --file internal/memory \
  --session "$SESSION_ID" \
  --include-recent \
  --token-budget 800

Review timeline memory from CLI:

go run ./cmd/memcache timeline \
  --repo . \
  --since 2026-05-10 \
  --group-by kind \
  --session "$SESSION_ID" \
  --limit 50

Session-aware recall:

• Record harness/session identity with session_id (--session-id in CLI).
• Filter search queries with session: or session_id:.
• Use --session filters on CLI search, context, and timeline.
• Use MCP sessions filters on memory.search, memory.context, and memory.timeline.

Best-effort redaction replaces obvious secret-shaped values before write and reports affected fields in command/tool output. Redaction is conservative and not a substitute for avoiding secret input.

Reliability and maintenance

memcache inspect reports store health:

• manifest schema and repo/store identity status
• journal entry and issue counts
• index manifest entry count and segment count
• index freshness via index_current, index_lag, and index_issue
• segment count and byte size
• quarantine file count
• active lock files and stale-lock status

Use memcache inspect --json for harness-readable health fields with stable snake_case names. manifest_ok is true only when the store manifest schema, repo_root, and store_root match the current layout; mismatches set manifest_issue. index_current is true only when the index manifest is valid, segment files are readable and match the manifest segment count, the journal has no malformed records, indexed entry count matches journal entry count, and the index manifest journal signature matches current journal file metadata. Derived index corruption or drift sets index_issue instead of making inspect fail.

Use memcache inspect --repair to repair derived state from the journal before printing health. Repair prunes legacy object-cache files, rebuilds search index, and regenerates rollups. It does not mutate journal files and stops if journal integrity checks report issues. With --json, repair output wraps repair and post-repair health.

memcache reindex rebuilds from the JSONL journal into a staged temporary index, writes the manifest last, then swaps the staged index into place. It also compacts many incremental record segments back into one full segment. If journal lines are malformed, denormalized, or have entry IDs that no longer match their canonical content, reindex leaves the previous index in place and writes the bad records to .agents/.memcache/quarantine/ for inspection. Journals are never auto-deleted.

On-disk schema changes must go through the migration hook in internal/store/migrate.go; unknown schema versions and mismatched store manifests are rejected instead of guessed or silently rewritten.

Search index

memcache reindex rebuilds search state from the JSONL journal. The index uses immutable JSON segment files under .agents/.memcache/index/segments/. MCP/CLI single-record writes append inc-* segments; reindex writes a compact full-* segment. Search repairs stale, missing, or corrupt index segments before returning hits: it appends journal entries missing from the index, but rebuilds when segment JSON is invalid, segment contents fail validation, or indexed documents are no longer present in the journal. Fresh indexes compare the manifest journal signature with current journal file metadata, then read segment data without parsing the full journal or rewriting the manifest. Malformed journal records still stop search when the journal signature changes and should be handled with memcache inspect/memcache reindex.

Analyzer behavior:

• Unicode lowercase.
• Split text on non-letter/non-digit characters.
• Drop a small built-in stopword set.
• Index weighted fields: summary, text, file paths, refs, tags, session id, and kind.
• File paths add full path plus segment/base/ext tokens, so internal/store/entry.go can match internal, store, entry, go, or the full path.

Query behavior:

• Plain terms use BM25-style scoring with field weights.
• Quoted phrases must occur in the entry text/search fields.
• Filters: kind:, file:, tag:, ref:, session:, session_id:, since:, until:, limit:, token_budget:, include_compacted:.
• Recency and importance apply small boosts after term scoring.
• Result snippets use [[term]] markers, never ANSI.

Rollups and context packs

Rollups are deterministic markdown views rebuilt from active journal entries. Each rebuild removes stale generated daily and file-card markdown that no longer corresponds to current active entries.

• rollups/repo.md: durable decisions/summaries plus recent activity.
• rollups/daily/YYYY-MM-DD.md: chronological daily memory.
• rollups/files/<repo-path>.md: compact memory card for each touched file.

Context packs combine durable facts, ranked search hits, and optional recent activity. Every memory-derived line cites an entry ID in [id] form. Token budgets use a conservative tokens * 4 character cap and truncate at line boundaries. CLI context and timeline use the same memory service as MCP tools.

Dependency policy

All Go imports must be one of:

• standard library packages
• this module: git.sharkk.net/go/memcache/...
• explicitly approved golang.org/x/... packages

go test ./... includes a guard that rejects other external imports.

Benchmarks and fuzzing

Core parsers and hot paths have Go benchmark/fuzz coverage:

• MCP wire decode/encode and raw server input.
• Query parsing, result formatting, analyzer, and search.
• Entry preparation, append path, redaction, rollup generation, and context packs.

Useful commands:

go test ./...
go test ./... -run=^$ -bench=. -benchmem
go test ./internal/mcpwire -run=^$ -fuzz='^FuzzDecodeLine$' -fuzztime=15s
go test ./internal/mcpserver -run=^$ -fuzz='^FuzzServeRawInput$' -fuzztime=15s
go test ./internal/index -run=^$ -fuzz='^FuzzParseQuery$' -fuzztime=15s
go test ./internal/index -run=^$ -fuzz='^FuzzFormatResults$' -fuzztime=15s
go test ./internal/store -run=^$ -fuzz='^FuzzPrepareEntry$' -fuzztime=15s
go test ./internal/memory -run=^$ -fuzz='^FuzzRedactEntry$' -fuzztime=15s
go test ./internal/memory -run=^$ -fuzz='^FuzzCompactSummary$' -fuzztime=15s

Fuzzing found and fixed:

• composite JSON-RPC IDs were accepted; now only string/number IDs are valid.
• limit: and token_budget: query filters bypassed final bounds checks.
• mem://file/... resource reads could traverse outside file rollups.
• Windows-style backslash traversal in file inputs was not normalized before validation.
• adjacent GitHub-style tokens were not redacted.