Watch
1
0
Fork
You've already forked smart-memory
0
mirror of https://github.com/smartmemory/smart-memory.git synced 2026-10-20 14:41:26 +00:00
Find a file
smartmem-dev 85e9df5608 fix(capture): release-held --force really captures, every mode reports only what it queued (CAPTURE-LARGE-RELEASE-NOENQUEUE-1)
A forced release queued the events and reported them released, but the
worker the drain starts re-applied the size knob (or the hold) and parked
them again in the same pass, so --large --force moved nothing and said it
had. A forced release now marks each queued event release_forced, the
worker captures marked events past the hold and the knob (one WARNING
each), and slice continuations and a newer SessionEnd keep the mark.

release-held now checks each event: released only when the ledger
read-back shows it queued; over the knob without --force, a missing
transcript, or a lock timeout leaves it parked with a WARNING, siblings
still release, exit 1.
85e9df5608 · 2026-10-20 22:07:14 +08:00
552 commits

SmartMemory

Give your AI a memory. Your coding assistant forgets everything the moment a session ends. SmartMemory gives it a place to keep what it learned.

Docs PyPI version Python 3.11+

Read the docs

SmartMemory remembers what you and your AI learn while you work: decisions, preferences, facts, and hard-won lessons. Next session, SmartMemory looks up the relevant parts and hands them back to your assistant automatically. It runs on your machine by default, with no Docker and no database to install.

Works out of the box with Claude Code, Cursor, and other MCP-compatible tools, or directly from Python.

(new: export and import your whole memory as portable Open Knowledge Format bundles.)

Get started

pip install smartmemory
smartmemory setup

That's it. Setup asks a couple of questions (arrow keys, sensible defaults) and wires SmartMemory into Claude Code. Using Cursor instead? Run smartmemory setup --for cursor.

Now try it:

sm add "We chose Postgres over MongoDB because of the reporting queries"
sm add "Deploys go through GitHub Actions, never manual"
sm search "which database did we pick"

sm is the short alias for smartmemory. Every command works with either.

First run. SmartMemory runs two small models on your machine: a spaCy language model (about 15 MB) and a local embedding model (about 100 MB). smartmemory setup downloads both. If you skip setup, the first sm add, sm search or sm get downloads them once and tells you what it is downloading and when it is done. If the download fails, for example without a network connection, it stops with a one-line message asking you to run smartmemory setup. The Claude Code hooks never download models. To turn off the download, for example in CI, set SMARTMEMORY_AUTO_DOWNLOAD_MODELS=0.

What you get from here

  • Your AI remembers across sessions. Setup installs hooks that capture context while you work and recall the relevant parts when the next session starts.
  • Ask questions in plain language. sm search is semantic search. "Which database did we pick" finds the Postgres memory even though the words don't match.
  • A knowledge graph you can see. Everything you add gets linked into a graph you can explore in your browser. The tour below shows you around.

A quick tour

Run the guided tour first:

sm tour

It starts an isolated local tour store, opens the graph viewer, seeds a few project facts, searches them, shows a real token receipt, then proves cross-session recall. The tour does not touch your real memories. Use sm tour --no-viewer for headless runs, or sm tour --keep if you want to inspect the temporary store afterward.

Prefer to read the flow instead? The written analog is below.

1. Capture things worth remembering

Decisions, gotchas, preferences. Add them as they happen:

sm add "Staging rate-limits at 100 requests per minute, batch the uploads"
sm add --type procedural "Reset the local DB with docker compose down -v, then ./dev.sh start"

Memory types (semantic, procedural, and more) are optional. Skip the flag and new memories land as episodic, the type for things that happened.

2. Feed it what you already have

Pipe in notes and documents instead of retyping them:

sm add - < meeting-notes.txt        # one memory per line
sm add --all - < project-brief.md   # the whole file as one memory

3. Keep projects separate

Tag memories with any property, then filter searches on it:

sm add --project atlas "Atlas v2 ships on the 15th"
sm search --project atlas "when do we ship"
sm search --top-k 20 "*"            # list more of everything

4. Meet the daemon

SmartMemory runs a small background daemon so the CLI does not cold-start Python on every command. In our own timing on version 1.5.16, a warm sm add took about 210 ms (median of 15 runs) and the first sm add after startup took about 0.9 seconds (3 runs). Setup starts it for you, and on macOS it comes back after login and crashes.

sm status     # daemon health, memory count, enrichment queue
sm restart    # if you ever need a fresh start

In local mode, sm status shows how many memories are saved on this machine and explains that they are outside your cloud account, even when the daemon is stopped or warming. Use sm status --json for machine-readable status. Local output includes local_only and local_memory_count alongside the daemon health fields. Remote output retains the hosted summary fields and reports local_only: false.

The daemon is also where the quality comes from. sm add returns quickly because only a fast entity extraction pass runs before it answers. If you have an LLM API key configured, the daemon then quietly re-reads each memory in the background and adds the entities and relations the fast pass missed. You do not wait for it, and sm status shows the queue draining.

5. Explore your knowledge graph

sm viewer

This opens an interactive graph in your browser. Every memory appears linked to the people, projects, and concepts inside it, and memories that share entities cluster together. After a few days of real use it reads like a map of your work. If the default port is taken, use sm viewer --port 8080.

Over SSH, or when you would rather not leave the terminal:

sm explore                     # opens on the search box
sm explore "amulet"            # opens on the best match for a term

sm explore is a graph browser rather than a graph picture. A terminal cannot lay out a force-directed graph legibly once there are more than a few dozen nodes, so instead you get one focused memory, its relations grouped by type, and a walk. Press up and down to move, Enter to walk to the other end of a relation, Backspace to go back, / to search, ? to ask a question, and q to quit. A breadcrumb across the top shows the path you took. The right pane tails the daemon live, one line per event, and flashes when something touches the memory you are looking at. It needs the daemon running, and says so plainly if it is not.

Use it with Claude Code

If you ran smartmemory setup, this is already working. Setup installs six hooks that follow the rhythm of a coding session:

  • Session starts: SmartMemory recalls what it knows about the directory you're working in.
  • You send a prompt: memories relevant to that prompt are injected as context.
  • Tools run: tool calls are observed and captured.
  • A tool fails: the error pattern is saved so it can be recalled later.
  • Claude finishes responding: the response is paired with the prompt that produced it.
  • Session ends: a session summary is persisted.

The aim is that tomorrow's session can pick up from what today's captured, without you pasting context around.

You also get slash commands inside Claude Code:

/remember <something worth keeping>
/search <query>
/ingest            (current file or a pasted block)
/orient            (what does SmartMemory know about this directory?)

Want Claude to call memory as a tool too? Install the MCP server:

sm mcp install claude-code    # also: cursor, codex

Using Cursor as your editor? smartmemory setup --for cursor configures it in one step.

Use it with Obsidian

The SmartMemory Obsidian plugin brings the same memory to your vault: every note becomes a structured memory with extracted entities, entity chips link every note that mentions the same person or project, a graph pane shows the neighborhood around the active note, and Cmd+Shift+R runs multi-hop semantic search across the whole vault. It can also propose [[wikilinks]] for entity mentions and warn you inline when a note contradicts something newer.

The plugin uses a SmartMemory account (free tier: 1,000 notes, 200 searches per day). For access, write to support@smartmemory.ai, then paste your API key from www.smartmemory.ai into Settings → SmartMemory.

What happens when you add a memory

SmartMemory doesn't just save text. It extracts the people, projects, and concepts inside each memory and links them into a knowledge graph. "Alice leads Project Atlas" becomes Alice, Project Atlas, and the relationship between them. That's why search can answer questions instead of just matching keywords, and why the viewer has a graph to draw.

Use it from Python

from smartmemory.tools.factory import create_lite_memory, lite_context

# Simple usage. Full LLM extraction runs if OPENAI_API_KEY is set.
memory = create_lite_memory()
item_id = memory.ingest("Alice leads Project Atlas")
results = memory.search("who leads Atlas", top_k=5)

# Preferred in scripts: cleans up globals and closes SQLite on exit.
with lite_context() as memory:
    item_id = memory.ingest("Alice leads Project Atlas")
    results = memory.search("who leads Atlas")

# Force no LLM calls (even if OPENAI_API_KEY is set).
from smartmemory.pipeline.config import PipelineConfig
memory = create_lite_memory(pipeline_profile=PipelineConfig.lite(llm_enabled=False))

Everyday commands

sm add "text"                  # Remember something
sm search "query"              # Find memories by meaning
sm search "*"                  # List everything
sm ask "what did we decide?"   # Answer, or labelled excerpts locally without an LLM key
sm viewer                      # Open the knowledge graph in your browser
sm explore                     # Walk the knowledge graph in the terminal
sm status                      # Daemon health and memory count
sm config                      # View settings
sm clear                       # Start over (deletes all memories)

In remote mode, sm search '*' --top-k 20 lists a page of memories in your configured workspace. The output shows the scoped total and a next-page command using --offset. The default page size is 5. Local wildcard listing continues to list all local memories.

Remote sm add "note" --prop project=atlas stores the property in hosted ingest context. Repeat --prop key=value for more properties, or use --project atlas. Producer fields such as origin and memory_type, and hosted scope fields such as workspace_id, are reserved. Write keys must be simple identifiers without __. Use --type to choose the memory type.

Remote sm search '*' --prop project=atlas lists exact matches without semantic ranking. One property filter is supported per request. --prop memory_type=decision uses the list route's type filter. Pagination retains the filter in the next-page command. Property filters with a text query, multiple filters, and semantic or time-window options on wildcard listing are explicitly refused.

Full command reference

Core

smartmemory add "text"                # Add a memory (default type: episodic)
smartmemory add --type semantic "..."  # Add with specific memory type
smartmemory add - < notes.txt          # Add from stdin, one memory per line
smartmemory add --all - < doc.txt      # Add entire stdin as one memory
smartmemory add --project atlas "..."  # Add with arbitrary property flags
smartmemory add "..." --project=atlas  # Equal-form properties also work

smartmemory search "query"             # Semantic search (full item IDs)
smartmemory search "query" --json      # Full IDs, bodies and returned fields as JSON
smartmemory search "query" --full      # Whole bodies instead of 200-character previews
smartmemory get abcdef                 # Local read-only lookup by unique ID prefix (6+ hex chars)
smartmemory help search                # Show command help
smartmemory search "*"                 # List all memories
smartmemory search --top-k 20 "query"  # Control result count (default: 5)
smartmemory search --project atlas "q" # Filter by property
smartmemory ask "what did we decide?"  # Answer, or labelled excerpts locally without an LLM key

smartmemory get <item_id>              # Fetch a single memory by ID
smartmemory retag --content "x" --origin seed:demo --dry-run  # Re-tag matched items' origin
smartmemory recall                     # Session context for Claude Code hooks
smartmemory recall --cwd /path         # Recall with working directory context
smartmemory viewer                     # Open knowledge graph viewer in browser
smartmemory viewer --port 8080         # Custom port for viewer
smartmemory models                     # List available LLM models
smartmemory config                     # View all settings
smartmemory config llm_provider        # View one setting
smartmemory config llm_provider groq   # Change a setting
smartmemory clear                      # Delete all memories and reset vectors

Daemon

smartmemory start                      # Start daemon + enrichment workers
smartmemory stop                       # Stop daemon
smartmemory restart                    # Restart daemon and print phase timings
smartmemory status                     # Daemon health + enrichment stats
smartmemory worker                     # Run enrichment worker (drain and exit)
smartmemory worker --loop              # Run enrichment worker continuously

Search previews mark truncated bodies with …. Copy the full ID into sm get, or use a unique prefix of at least six hexadecimal characters in local mode. Ambiguous prefixes list the full candidate IDs and fail. Remote get requires a full ID. Prefix resolution applies only to read-only get. Quote inline text containing multiple words, or pipe it with sm add --all -. Keyless local semantic hop searches print a notice when the heuristic planner is used.

Setup and lifecycle

smartmemory setup                      # Interactive first-run questionnaire
smartmemory setup --mode local         # Skip questionnaire, set local mode
smartmemory setup --mode remote --api-url https://your-service.example  # SMARTMEMORY_API_KEY in env
smartmemory setup --for cursor         # Configure for Cursor instead of Claude Code
smartmemory server                     # Start MCP server (called by MCP clients)
smartmemory uninstall                  # Remove hooks, skills, plist, and data
smartmemory uninstall --keep-data      # Remove hooks/plist but keep memories

In local mode without an LLM key, sm ask stops with an error that names the missing provider key and the fix (smartmemory setup, or export a supported provider key and run sm restart). sm ask --extractive lists the most relevant stored excerpts with full IDs, types and dates instead. That output is labelled extractive and contains no AI-written answer.

In remote mode, sm ask, add, search, recall, get and status use the hosted service without a local daemon. sm ask uses the service's LLM credentials, so you only need your SmartMemory API key. add accepts user properties, and search '*' supports paginated listing with one exact property filter. Property filters on semantic text searches are explicitly refused. The guided tour, store maintenance, corpus import/export, seed-pack installation and transcript capture commands require local mode. Import dry runs can still validate files in remote mode.

Once a day, any sm command may end with one dim line telling you a newer release is on PyPI:

smartmemory 1.4.87 available — pip install -U smartmemory

The check runs at most once every 24 hours, times out after 2 seconds, and stays quiet when it fails, when output is piped, and on the commands that feed context to a model. Set SMARTMEMORY_NO_UPDATE_CHECK=1 to switch it off. After an upgrade, the next command says Updated to <version>, and suggests re-running sm tour only when the tour itself gained something.

Admin

smartmemory export ./bundle            # Export memories to an OKF bundle directory (alias: admin export)
smartmemory import ./bundle            # Import an OKF bundle directory (alias: admin import)
smartmemory admin export out.jsonl --legacy-jsonl   # Legacy single-file JSONL corpus (read-during-migration)
smartmemory admin import data.jsonl --legacy-jsonl   # Import a legacy JSONL corpus
smartmemory admin reindex              # Re-embed all memories with current model
smartmemory admin reextract            # Re-run entity extraction on all memories
smartmemory admin list-packs           # List available seed packs
smartmemory admin install-pack NAME    # Install a seed pack
smartmemory admin mine                 # Mine Wikidata entities via SPARQL
smartmemory admin convert-rebel        # Convert REBEL dataset to corpus JSONL

Code indexing and MCP

Code indexing reports clean, partial and failed file counts plus acceptance, staging and publication outcomes. Every hosted response derives counts from uploaded file evidence. Uncomputable evidence and missing server counts are displayed as unknown. Localized grammar partials with usable extraction can publish with visible spans and byte coverage. Hard collection or parse failures retain the prior index. Partial 1.x acceptance does not prove G16 complete-generation publication. Successful file extraction checkpoints are stored under SMARTMEMORY_CODE_CHECKPOINT_DIR (default ~/.cache/smartmemory/code-parse) and reused when source, configuration and parser versions match. Hosted CLI failures print diagnostic counts and the server staging and publication outcome, including cleanup failures. Transport loss reports an unconfirmed server outcome.

smartmemory code index <path>          # Index a code repo (AST entities + call graph) into memory
smartmemory code effects <path>        # Print source-qualified Python effects without a store
smartmemory mcp install claude-code    # Write MCP server config for a client (claude-code, cursor, codex)

Lifecycle (hook-driven)

These are invoked automatically by the Claude Code hooks that smartmemory setup installs. You rarely call them by hand.

smartmemory lifecycle orient           # Recall context at session start
smartmemory lifecycle recall           # Inject prompt-relevant context
smartmemory lifecycle observe          # Capture a tool call
smartmemory lifecycle distill          # Pair a response with its stored prompt
smartmemory lifecycle learn            # Capture an error pattern
smartmemory lifecycle persist          # Save a session summary
smartmemory lifecycle status           # Show lifecycle config and session stats

Install options

pip install smartmemory                  # Everything: local memory + MCP server + graph viewer + CLI
pip install smartmemory-core[lite]       # Core library only, no CLI/MCP/viewer
pip install smartmemory-core[gpu]        # Add the torch stack; CUDA is used automatically when present

The default install embeds with all-MiniLM-L6-v2 on ONNX Runtime: about 150 MB and a sub-second cold start, no torch. The [gpu] or [torch] extra swaps the runtime, not the model, so vectors and existing stores are interchangeable.

smartmemory is the distribution package. A single install bundles smartmemory-core[lite] (local SQLite + usearch storage), the unified MCP server, the graph viewer, and the CLI. You pick local or remote mode at smartmemory setup time, not at install time. smartmemory-core is the core library for developers building on top of SmartMemory.

Local mode wires Claude Code hooks, downloads the spaCy language model (about 15MB), and starts a persistent daemon. Remote mode validates your API key and stores it in the OS keychain.

Lite or Service

  • Lite (the default): everything runs on your machine. SQLite graph plus usearch vectors, no Docker, no external services, no account. pip install smartmemory and go.
  • Service: connect to the managed SmartMemory backend instead of running storage locally. Run smartmemory setup --mode remote and paste an API key from www.smartmemory.ai. Nothing to install or operate, and there is a free tier to start on.

You choose Lite or Service at smartmemory setup time, not at install time, and you can switch later by re-running setup.

Going deeper

Everything below is here for the curious and for developers building on top of SmartMemory. You don't need any of it to use the tool.

Memory types

SmartMemory sorts what it stores into 11 curated memory types. Five core types hold knowledge (what's true): Pending, Semantic, Episodic, Procedural, and Zettelkasten. The rest capture expertise (what to do and what not to do): Reasoning, Opinion, Observation, Decision, Constraint, and Learned.

The expertise layer is what makes an agent's memory useful for acting: captured choices, rejected alternatives, hard constraints, and lessons learned. Capture them with mem.add_decision(...), mem.add_constraint(...), and mem.add_learning(...), and recall them with mem.search(query, expertise=True). See Expertise vs Knowledge for the full mapping.

All 11 memory types
  • Pending Memory: Short-term buffer for items awaiting consolidation (formerly "working", routed at ingest by the ConsolidationRouter)
  • Semantic Memory: Facts and concepts with vector embeddings
  • Episodic Memory: Personal experiences and learning history
  • Procedural Memory: Skills, strategies, and learned patterns
  • Zettelkasten Memory: Bidirectional note-taking system with AI-powered knowledge discovery
  • Reasoning Memory: Chain-of-thought traces capturing "why" decisions were made (System 2)
  • Opinion Memory: Beliefs with confidence scores, reinforced or contradicted over time
  • Observation Memory: Synthesized entity summaries from scattered facts
  • Decision Memory: First-class decisions with confidence tracking, provenance chains, and lifecycle management. Structured fields: rejected_alternatives, rationale, constraints. Capture: mem.add_decision(...)
  • Constraint Memory: Hard rules, discovered or imposed. Capture: mem.add_constraint(...)
  • Learned Memory: Lessons learned the hard way. Capture: mem.add_learning(...)

Structural types like code, plan, evaluation, anchor, and tool_call are used internally.

Advanced recall in practice

The quickstart above is capture and simple search. These are the recall patterns a flat memory file cannot do. Every example runs in local Lite mode with no API key (LLM extraction only matters when you want these classified automatically out of raw prose, rather than recorded explicitly). Shared setup:

from datetime import datetime, timezone
from smartmemory.pipeline.config import PipelineConfig
from smartmemory.tools.factory import lite_context
from smartmemory.models.memory_item import MemoryItem

lite = dict(pipeline_profile=PipelineConfig.lite(llm_enabled=False))

Decisions, including what you decided not to do. A decision is a first-class object that keeps the rejected alternatives and the reasoning attached, so you can recall the road not taken.

with lite_context(**lite) as m:
    d = m.add_decision(
        "Use SQLite for local lite storage",
        rejected_alternatives=["Postgres sidecar", "remote-only cloud sync"],
        rationale="Zero-infra installs must work offline",
    )
    print(m.get_decision(d.decision_id).rejected_alternatives)
    # -> ['Postgres sidecar', 'remote-only cloud sync']

    hits = m.search("what did we decide NOT to do for storage", expertise=True)
    print([i.content for i in hits["decision"]])
    # -> ['Use SQLite for local lite storage']

search(expertise=True) returns a dict keyed by expertise type (decision, constraint, learned, ...), not a flat list.

Truth maintenance: supersede a fact, and only the current one comes back.

with lite_context(**lite) as m:
    old = m.add(MemoryItem(content="The incident channel is #ops-old", memory_type="semantic"))
    new = m.add(MemoryItem(content="The incident channel is #ops-war-room", memory_type="semantic"))
    m.supersede(old, new, reason="Team renamed the channel")

    print([r.content for r in m.search("incident channel")])
    # -> ['The incident channel is #ops-war-room']                      (current only)
    print([r.content for r in m.search("incident channel", include_superseded=True)])
    # -> ['The incident channel is #ops-old', 'The incident channel is #ops-war-room']   (full history)

Bi-temporal recall: what did we believe before it changed. Keep the old record's time with reference_time, then ask as of a past moment. A markdown file cannot answer this: once you edit the line, the old value is gone.

with lite_context(**lite) as m:
    old = m.add(MemoryItem(content="The billing provider is Stripe", memory_type="semantic",
                           metadata={"reference_time": datetime.now(timezone.utc).isoformat()}))
    t_before_change = datetime.now(timezone.utc)
    new = m.add(MemoryItem(content="The billing provider is Paddle", memory_type="semantic",
                           metadata={"reference_time": datetime.now(timezone.utc).isoformat()}))
    m.supersede(old, new, reason="Pricing model changed")

    print([r.content for r in m.search("billing provider", as_of_date=t_before_change, include_superseded=True)])
    # -> ['The billing provider is Stripe']      (what you believed then)
    print([r.content for r in m.search("billing provider")])
    # -> ['The billing provider is Paddle']       (what you believe now)

Constraints and lessons, recalled as expertise.

with lite_context(**lite) as m:
    m.add_constraint("Production deploys must use GitHub Actions", domain="release")
    m.add_learning("SQLite WAL avoids writer stalls under concurrent tests")
    hits = m.search("deploy and sqlite guidance", expertise=True)
    print([i.content for i in hits["constraint"]])
    # -> ['Production deploys must use GitHub Actions']
    print([i.content for i in hits["learned"]])
    # -> ['SQLite WAL avoids writer stalls under concurrent tests']

Multi-hop retrieval chains results so each hop informs the next query, reaching facts a single lookup would miss. It pays off once your graph is rich (semantic_hops=True adds LLM-planned hops):

with lite_context(**lite) as m:
    hits = m.search("Redis migration fallout", multi_hop=True, max_hops=3)

Code intelligence: index a repo, then search it by meaning. sm code index builds an AST and call graph into memory. Recall it with search_code, which is semantic, so it finds code that shares no keywords with your query (a plain grep would miss it):

sm code index ./my-project --repo my-project
with lite_context(**lite) as m:
    # "clean up vendor billing records" matches a function actually named
    # reconcile_widget_invoice (no shared keywords, found by meaning).
    for hit in m.search_code("clean up vendor billing records", repo="my-project"):
        print(hit["name"], hit["entity_type"], f'{hit["file_path"]}:{hit["line_number"]}')

The processing pipeline

ingest() runs an 11-stage pipeline:

classify -> coreference -> simplify -> entity_ruler -> llm_extract -> ontology_constrain -> store -> link -> enrich -> ground -> evolve

Each stage implements the StageCommand protocol (execute(state, config) -> state, undo(state) -> state). The pipeline supports breakpoint execution (run_to(), run_from(), undo_to()) for debugging and resumption.

add() is simple storage (normalize -> store -> embed) for internal or derived items.

When an LLM API key is available, the daemon runs two-tier ingestion:

  • Tier 1 (sync, fast): spaCy + EntityRuler extracts entities immediately and returns the item ID
  • Tier 2 (async, slower): A background drain thread runs LLM extraction and adds net-new entities and relations

This means sm add returns after the fast pass while quality improves in the background.

What makes it different

  • Self-learning EntityRuler: Pattern-matching NER that improves with use. LLM discoveries feed back into rules
  • Memory evolution: Built-in evolvers transform memories over time, promoting stable facts, decaying stale ones, and tracking what you actually use
  • Hybrid search: Graph-structured search plus BM25/embedding RRF fusion with query decomposition for compound queries
  • Code indexer: AST-based Python and TypeScript parser with cross-file call resolution, semantic code search, and memory-to-code graph bridging
  • Two-tier ingestion: Instant spaCy extraction plus async LLM enrichment
  • MCP server: Works with Claude Code, Cursor, and other MCP-compatible tools
  • Portable, no lock-in: Export and import your whole memory as Open Knowledge Format bundles (Google's OKF v0.1), a portable format of Markdown files with YAML frontmatter. The Obsidian plugin reads and writes the same format, so you can round-trip a workspace through a vault without a proprietary converter. sm export ./bundle / sm import ./bundle
  • Flexible scoping: Optional ScopeProvider for multi-tenancy or unrestricted usage
  • Plugin security: Sandboxing, permissions, and resource limits for safe plugin execution
Memory evolution in detail

SmartMemory includes built-in evolvers that automatically transform memories. In lite mode, evolution runs incrementally in the background.

Core evolvers (memory type transitions and lifecycle):

  • EpisodicToSemanticEvolver: Promotes stable facts to semantic memory
  • EpisodicToZettelEvolver: Converts episodic events to Zettelkasten notes
  • EpisodicDecayEvolver: Archives old episodic memories
  • SemanticDecayEvolver: Prunes low-relevance semantic facts
  • ZettelPruneEvolver: Merges duplicate or low-quality notes
  • DecisionConfidenceEvolver: Decays confidence on stale decisions, auto-retracts below threshold
  • OpinionSynthesisEvolver: Synthesizes opinions from accumulated observations
  • ObservationSynthesisEvolver: Creates entity summaries from scattered facts
  • OpinionReinforcementEvolver: Adjusts opinion confidence based on new evidence
  • StaleMemoryEvolver: Flags memories as stale when referenced source code changes

Enhanced evolvers (neuroscience-inspired dynamics):

  • ExponentialDecayEvolver: Time-based activation decay with configurable half-life
  • RetrievalBasedStrengtheningEvolver: Memories accessed more frequently become harder to forget
  • HebbianCoRetrievalEvolver: Reinforces edges between memories retrieved together ("neurons that fire together wire together")
  • InterferenceBasedConsolidationEvolver: Similar competing memories interfere, strengthening the dominant one
  • EnhancedWorkingToEpisodicEvolver: Context-aware pending-to-episodic transition with richer metadata

Sleep-cycle and agent evolvers (opt-in background re-derivation, REM/NREM analogs):

  • MemoryConsolidationEvolver: Re-derives consolidated memories across an entity's scattered facts during an idle "sleep cycle"
  • TemporalAgingEvolver: Rewrites future/planned facts into resolved past tense once their date passes ("going to Singapore in July" becomes "went to Singapore in July 2026") via reversible bi-temporal supersession. Triggered by the passage of time, never by re-parsing prose
  • EvaluationEvolver: Writes evidence-derived per-(agent, dimension, domain) performance scores via bi-temporal supersession. Read back with get_evaluation() / list_evaluation_history()
  • SemanticToProceduralEvolver / ProceduralReinforcementEvolver / AnchorReconciliationEvolver: Procedural promotion, usage-based strengthening, and spec-anchor drift reconciliation

Replaced by ConsolidationRouter (CORE-MEMORY-DYNAMICS-1 M1):

  • The former WorkingToEpisodicEvolver and WorkingToProceduralEvolver were retired. Routing from the pending bucket to episodic / procedural now happens at ingest time via the ConsolidationRouter pipeline stage.
Plugin system in detail

SmartMemory features a unified, extensible plugin architecture. All plugins follow a consistent class-based pattern.

Auto-registered by default (loaded by PluginManager._load_builtin_plugins):

  • Extractors: LLMExtractor, LLMSingleExtractor, ConversationAwareLLMExtractor (always), plus SpacyExtractor and GLiNER2Extractor when their optional deps are importable
  • 6 Enrichers: BasicEnricher, LinkExpansionEnricher, SentimentEnricher, TemporalEnricher, ExtractSkillsToolsEnricher, TopicEnricher
  • 9 Evolvers: EpisodicToSemanticEvolver, EpisodicDecayEvolver, SemanticDecayEvolver, EpisodicToZettelEvolver, ZettelPruneEvolver, ExponentialDecayEvolver, InterferenceBasedConsolidationEvolver, RetrievalBasedStrengtheningEvolver, EvaluationEvolver
  • 1 Grounder: WikipediaGrounder

Specialist plugins (selected by pipeline stage, opt-in feature, or the Studio evolver catalog, not auto-run in the idle cycle):

  • Extractors: GroqExtractor, DecisionExtractor, ReasoningExtractor
  • Enrichers: UsageTrackingEnricher
  • Evolvers: DecisionConfidenceEvolver, OpinionSynthesisEvolver, ObservationSynthesisEvolver, OpinionReinforcementEvolver, StaleMemoryEvolver, HebbianCoRetrievalEvolver, MemoryConsolidationEvolver, TemporalAgingEvolver, SemanticToProceduralEvolver, ProceduralReinforcementEvolver, AnchorReconciliationEvolver
  • Grounders: PublicKnowledgeGrounder (Wikidata QIDs)

Two registries coexist by design: a small eager, auto-run set (above) that runs in the idle/batch evolution cycle, and a lazy catalog of ~40 evolvers (evolution/registry.py) that Studio can select into workflow DAGs without importing every class at boot. Auto-run ⊆ catalog is an enforced invariant.

Creating custom plugins:

from smartmemory.plugins.base import EnricherPlugin, PluginMetadata

class MyCustomEnricher(EnricherPlugin):
    @classmethod
    def metadata(cls):
        return PluginMetadata(
            name="my_enricher",
            version="1.0.0",
            author="Your Name",
            description="My custom enricher",
            plugin_type="enricher",
            dependencies=["some-lib>=1.0.0"],
            security_profile="standard",
            requires_network=False,
            requires_llm=False
        )

    def enrich(self, item, node_ids=None):
        item.metadata["custom_field"] = "value"
        return item.metadata

Publishing plugins:

# pyproject.toml
[project.entry-points."smartmemory.plugins.enrichers"]
my_enricher = "my_package:MyCustomEnricher"
pip install my-smartmemory-plugin
# Automatically discovered and loaded!
Python API reference

SmartMemory class

class SmartMemory:
    def __init__(
        self,
        scope_provider=None,
        vector_backend=None,
        cache=None,
        observability: bool = True,
        pipeline_profile=None,
        entity_ruler_patterns=None,
    )

    # Primary API
    def ingest(self, item, sync=True, **kwargs) -> str  # Full pipeline
    def add(self, item, **kwargs) -> str                # Simple storage
    def get(self, item_id: str) -> Optional[MemoryItem]
    def search(
        self,
        query: str,
        top_k: int = 5,
        memory_type: str = None,
        origin: str = None,              # prefix filter, e.g. "user"
        decompose_query: bool = False,   # split compound queries
        multi_hop: bool = False,         # chained recursive retrieval
        semantic_hops: bool = False,     # LLM-driven hop planning
        expertise: bool = False,         # return the expertise channel instead of a flat list
        include_superseded: bool = False,
        as_of_date=None,                 # bi-temporal recall (datetime | ISO-8601); see notes below
    ) -> List[MemoryItem]                # NOTE: with expertise=True, returns dict[str, list] keyed by
                                         # decision/constraint/learned/opinion/reasoning/observation
    def delete(self, item_id: str) -> bool

    # Expertise-layer capture (each returns a typed object, not a str)
    def add_decision(self, content: str, **kwargs) -> "Decision"    # .decision_id, .rejected_alternatives, .rationale, ...
    def add_constraint(self, content: str, **kwargs) -> "Constraint" # .constraint_id, ...
    def add_learning(self, content: str, **kwargs) -> "Learned"      # .learned_id, ...

    # Graph Integrity
    def delete_run(self, run_id: str) -> int
    def rename_entity_type(self, old: str, new: str) -> int
    def merge_entity_types(self, sources: List[str], target: str) -> int

    # Advanced
    def run_clustering(self) -> dict
    def run_evolution_cycle(self) -> None
    def personalize(self, traits: dict = None, preferences: dict = None) -> None
    def get_all_items_debug(self) -> Dict[str, Any]
    def close(self) -> None

MemoryItem class

@dataclass
class MemoryItem:
    content: str
    memory_type: str = 'semantic'
    item_id: str = field(default_factory=lambda: str(uuid.uuid4()))
    valid_start_time: Optional[datetime] = None
    valid_end_time: Optional[datetime] = None
    transaction_time: datetime = field(default_factory=datetime.now)
    embedding: Optional[List[float]] = None
    entities: Optional[list] = None
    relations: Optional[list] = None
    metadata: dict = field(default_factory=dict)

Configuration

Config file: ~/.config/smartmemory/config.toml (XDG on Linux/macOS, %APPDATA%\smartmemory\config.toml on Windows).

On Windows, run sm start after login or a daemon crash. Setup currently starts the daemon for the current session and does not install a login supervisor. Provider keys saved by setup use the Windows credential store for the daemon. Setup also shows PowerShell environment instructions for other CLI sessions. Piped memory text and lifecycle JSON must use UTF-8.

Diagnostic logs rotate with coordination across processes. The daemon's console output is saved separately in daemon-output.log. At daemon start, an output log larger than 5 MiB is moved to daemon-output.log.1 before the child opens it. One previous output log is retained. Override this startup cap with SMARTMEMORY_DAEMON_OUTPUT_MAX_BYTES (1 to 104857600 bytes). Output during one continuous run can exceed the cap until the next start.

API keys are stored in the OS keychain, never in the config file. If Windows Credential Manager is unavailable, setup and MCP login share %APPDATA%\smartmemory\credentials\api_key. Updates serialize writers and verify a protected current-SID-only file ACL before writing the key. An ACL failure preserves the previous key. Legacy MCP credentials migrate to the shared file and are removed. Standalone MCP uses the same protected writer at its legacy path. Set SMARTMEMORY_API_KEY as an env var on other headless systems where the keychain is unavailable. Remote setup uses --api-url, then the effective SMARTMEMORY_API_URL or saved URL, then the hosted default. Doctor and support zip probe this URL in remote mode and skip the SmartMemory API probe in local mode.

Windows and PowerShell

Windows setup validates Git Bash and installs its absolute executable path for all six Claude Code hooks. Install Git for Windows before setup. Setup refuses to install hooks if Git Bash cannot run.

PowerShell 5.1 pipes default to ASCII, which can replace non-ASCII text before Python receives it. Set UTF-8 before piping memory text:

$OutputEncoding = [Console]::OutputEncoding = [Text.UTF8Encoding]::new()
"Remember 中文 and café" | sm add --all -

For PowerShell 5.1 script files containing non-ASCII literals, save the .ps1 file as UTF-8 with BOM. You can also pass text as a quoted argument (sm add "Remember 中文") or supply a UTF-8 file through stdin from a shell that preserves its bytes.

Direct writes wait up to 30 seconds for store ownership. Set SMARTMEMORY_WRITE_LOCK_TIMEOUT to a value greater than 0 and at most 300 seconds. For concurrent CLI use, start the daemon with sm start. A busy store returns a normal error without replaying a write.

Before sm clear --yes, run smartmemory stop and disconnect MCP clients that hold the local store. Reset refuses active owners before deleting graph, FTS or vector files. Detached Windows daemons request breakaway from the terminal's process job. If Windows denies breakaway, startup warns that the process will not survive terminal close.

SOCKS support for Requests and Wikipedia calls is included in the wrapper's requests[socks] dependency.

Non-interactive / CI

# Local
SMARTMEMORY_MODE=local smartmemory server

# Remote
SMARTMEMORY_MODE=remote SMARTMEMORY_API_KEY=sk_... smartmemory server

Env vars always override the config file, which is the correct path for Docker and CI. The setup TUI is automatically disabled in non-interactive environments.

Environment variables
Variable Description
SMARTMEMORY_MODE local or remote, overrides config file
SMARTMEMORY_API_KEY API key for remote mode, bypasses keychain
SMARTMEMORY_API_URL Remote API URL (default: https://api.smartmemory.ai)
SMARTMEMORY_TEAM_ID Team/workspace ID for remote mode
SMARTMEMORY_DATA_DIR Local data directory (default: ~/.smartmemory)
SMARTMEMORY_LLM_PROVIDER LLM provider for local enrichment
SMARTMEMORY_EMBEDDING_PROVIDER Embedding provider (local, openai, ollama)
SMARTMEMORY_DAEMON_PORT Daemon port (default: 9014)
SMARTMEMORY_ASYNC_ENRICHMENT Enable/disable background enrichment
SMARTMEMORY_LOG_LEVEL Logging threshold (default: WARNING). Set to DEBUG for startup context, daemon/fallback routing, and bounded request/response wire detail on stderr; Authorization and other credential-shaped fields are always redacted
SMARTMEMORY_NO_UPDATE_CHECK Set to 1 to turn off the daily update-available hint
CAPTURE_MAX_MB Largest transcript slice a session-end capture reads at once (default 50). A larger transcript, or a larger new tail, is captured one slice per queue pass from where the last capture stopped, so nothing is skipped and other sessions are not held up
CAPTURE_MAX_RSS_MB Memory ceiling for the capture worker (default 3072). Past it, between items, the worker hands the queue to a fresh worker and exits. The waiting item stays queued
SMARTMEMORY_CAPTURE_MAX Most capture workers a session end starts machine-wide (default 2, 0 = no cap). A capped start loses nothing, the queued capture waits for a running worker
SMARTMEMORY_CAPTURE_HOLD Parks every session capture in captures/held-all/ and starts no worker (default 1, 0 = capture as before). Put them back with sm lifecycle release-held, then sm lifecycle drain
SMARTMEMORY_CAPTURE_HOLD_OVER_MB With the hold off, a transcript over this size goes to captures/quarantine-large/ unread (default 50, 0 = off). sm lifecycle release-held --large puts them back
OPENAI_API_KEY OpenAI API key for embeddings and LLM extraction
GROQ_API_KEY Groq API key, an alternative to OpenAI for LLM extraction

Testing

# Run all tests
PYTHONPATH=. pytest -v tests/

# Run specific test categories
PYTHONPATH=. pytest tests/unit/
PYTHONPATH=. pytest tests/integration/
PYTHONPATH=. pytest tests/e2e/

Fresh-install check (CI)

.github/workflows/fresh-install.yml builds the wheel on every wrapper change, installs it into a clean venv with a throwaway HOME and no inherited SMARTMEMORY_* variables, and runs scripts/fresh_install_check.py. The script runs a keyless local setup, sm doctor, an add, search and decision round trip, a setup with poisoned SMARTMEMORY_* variables, and a remote-mode setup against a fake server on 127.0.0.1. It prints one STEP <name> PASS|FAIL|SKIP line per step and needs no secrets. Run it locally on Linux (on macOS, use a Linux container, because sm setup registers a launchd job):

python -m build
python scripts/fresh_install_check.py --wheel dist/smartmemory-*.whl [--core PATH_TO_CORE_SOURCE]

License

SmartMemory Lite is distributed as a compiled runtime under the SmartMemory Lite Runtime License (see LICENSE).

  • Free for personal use, internal tools, evaluation, research, development, and non-commercial applications.
  • Shipping the runtime inside a commercial product requires a Lite Redistribution Agreement, free during early access and for organizations below the revenue threshold in the agreement. Ask at help@smartmemory.ai.
  • No source disclosure is required for your application.

The hosted service and platform are licensed separately.


pip install smartmemory
smartmemory setup

Lexical search migration

Search uses lexical with default weight 0.8. Replace removed contains and keyword-bm25 channel weights explicitly. A zero weight disables lexical, and omission preserves the existing default/profile behavior. Required lexical unavailability fails the whole search. Service callers receive 400 for query/name validation and 503 for unavailable indexes.

Lite is single-tenant. Graph-first retrieval conditionally fills from FTS5 lexical and vector search. Bounded refill can underfill, and regex-text still scans. CORE-FACT-INDEX-1 separately owns deferred fact ranking and path expansion.

Quiesce old writers before first-open indexing. Verify the engine capability pin and use sm rebuild --lexical for recovery. Migration, targets and measured limitations.

Local background work

Local saves (sm add and lifecycle hooks) work without an LLM key, whether the daemon is running or not. Pattern extraction makes memories available immediately; local enrichment runs in the background. With an available LLM route, the core worker also performs deferred LLM extraction. One worker per data directory drains the durable work graph; the daemon, launchd, or an on-demand save starts it.

sm status shows pending, running, and dead work counts. After adding an LLM key, status and setup offer to re-extract memories saved without one.

sm admin reextract           # show count/token estimate, then ask for confirmation
sm admin reextract --yes     # skip confirmation; queue and return immediately
sm admin reextract --all     # include all user memories
sm admin reextract --ruler   # force pattern-only extraction (no LLM)
sm admin reextract --decline # dismiss the re-extraction notice
sm worker requeue-dead       # retry dead and skipped boxes

Flags can be combined. Re-extraction does not require the daemon and does not wait for background processing. The default scope is memories marked ruler_only or llm_failed. Grounding and evolution are off by default in local mode.

Having trouble?

Automatic anonymous crash reports are on by default. They include versions, redacted error frames and bounded log tails. Credentials, memory bodies, private API/proxy hosts and home-directory names are removed, including OS proxy identities and unsaved setup endpoints. Disable them with SMARTMEMORY_CRASH_REPORTS=0 or crash_reports = false in the config file's [smartmemory] section.

Handled installation failures also send anonymous reports and print a relevant doctor fix plus the report ID when sending finishes promptly. Reports are saved after privacy filtering in a durable outbox at ~/.smartmemory/report-outbox before sending. Interactive commands wait at most 400 ms for transport. Queued reports retry on non-lifecycle CLI and daemon starts. Cancelling a prompt or entering an invalid flag does not send a report. A passing first-run native check is cached for the installed version. Hook shell failures are recorded under ~/.smartmemory even with a custom store directory. A non-lifecycle CLI invocation other than doctor reports them, at most once per hook and exit code each day. Lifecycle hooks do not consume markers, run installation probes, auto-diagnose or send reports. Markers are consumed only after their report is durably queued. The outbox retains up to 50 reports for 14 days. A delivery receipt prevents a recovered report from being sent again, and its original hook signature remains deduplicated. Malformed queue files are quarantined with a warning, keeping at most 10 copies, while healthy reports continue. Opting out deletes queued reports without sending them.

smartmemory doctor reads the local SQLite and vector stores, checks vector dimensions and item counts, and inspects owner metadata without acquiring the write lock. A verified healthy SmartMemory daemon or worker owning its configured store is OK. Stale markers and foreign owners remain warnings. Doctor reuses a healthy daemon's successful embedding warmup. Without that result, it tests one embedding using cached files with networking disabled and a 20-second budget. Each diagnostic phase records its own time and budget. A probe timeout means "not checked", including slow embedding cold starts, and does not establish a broken runtime. It also checks keyring, Git Bash on Windows, free disk space, and the configured daemon port. An existing lock file without verified owner metadata shows "ownership unverified". It shows pending hook failures, the queued report count and repair commands. It does not consume markers, flush reports, acquire the store write lock, reindex or reset your store. Its writability check creates and removes a temporary file in the data directory. smartmemory clear --yes is a last resort that deletes all local memories. Back up your data first.

For isolated reporter tests, SMARTMEMORY_CRASH_REPORT_HOST=http://127.0.0.1:<port> redirects the existing reporter to a local capture server. Its default is PostHog. Keep SMARTMEMORY_CRASH_REPORTS=0 for tests that do not explicitly exercise reporting.

Run smartmemory doctor, then smartmemory report "What went wrong" --send to send support diagnostics after confirmation. Use --yes to skip the prompt. Quote the printed report ID to support@smartmemory.ai. If delivery cannot be confirmed within the wait budget, the command reports that the upload is queued and saves a local support zip. With reporting disabled, it only saves the local zip.

smartmemory report --zip creates a local support archive. Existing queued reports may retry on CLI startup, unless reporting is disabled. Email the file at the printed path to support@smartmemory.ai. The archive includes redacted diagnostics, configuration and recent CLI and daemon logs. Use smartmemory report --zip "PATH" to choose where to save it.

For slow model downloads, increase HF_HUB_DOWNLOAD_TIMEOUT and retry smartmemory setup. Set HF_ENDPOINT if you use a Hugging Face mirror. To show a full terminal traceback, set SMARTMEMORY_DEBUG=1 before running the failing command.