Overview
CrewAI provides a unified memory system — a singleMemory class that replaces separate short-term, long-term, entity, and external memory types with one intelligent API. Memory uses an LLM to analyze content when saving (inferring scope, categories, and importance) and supports adaptive-depth recall with composite scoring that blends semantic similarity, recency, and importance.
You can use memory four ways: standalone (scripts, notebooks), with Crews, with Agents, or inside Flows.
Quick Start
Four Ways to Use Memory
Standalone
Use memory in scripts, notebooks, CLI tools, or as a standalone knowledge base — no agents or crews required.With Crews
Passmemory=True for default settings, or pass a configured Memory instance for custom behavior.
memory=True, the crew creates a default Memory() and passes the crew’s embedder configuration through automatically. All agents in the crew share the crew’s memory unless an agent has its own.
After each task, the crew automatically extracts discrete facts from the task output and stores them. Before each task, the agent recalls relevant context from memory and injects it into the task prompt.
With Agents
Agents can use the crew’s shared memory (default) or receive a scoped view for private context.With Flows
Every Flow has built-in memory. Useself.remember(), self.recall(), and self.extract_memories() inside any flow method.
Hierarchical Scopes
What Scopes Are
Memories are organized into a hierarchical tree of scopes, similar to a filesystem. Each scope is a path like/, /project/alpha, or /agent/researcher/findings.
How Scope Inference Works
When you callremember() without specifying a scope, the LLM analyzes the content and the existing scope tree, then suggests the best placement. If no existing scope fits, it creates a new one. Over time, the scope tree grows organically from the content itself — you don’t need to design a schema upfront.
Visualizing the Scope Tree
MemoryScope: Subtree Views
AMemoryScope restricts all operations to a branch of the tree. The agent or code using it can only see and write within that subtree.
Best Practices for Scope Design
-
Start flat, let the LLM organize. Don’t over-engineer your scope hierarchy upfront. Begin with
memory.remember(content)and let the LLM’s scope inference create structure as content accumulates. -
Use
/{entity_type}/{identifier}patterns. Natural hierarchies emerge from patterns like/project/alpha,/agent/researcher,/company/engineering,/customer/acme-corp. -
Scope by concern, not by data type. Use
/project/alpha/decisionsrather than/decisions/project/alpha. This keeps related content together. -
Keep depth shallow (2-3 levels). Deeply nested scopes become too sparse.
/project/alpha/architectureis good;/project/alpha/architecture/decisions/databases/postgresqlis too deep. -
Use explicit scopes when you know, let the LLM infer when you don’t. If you’re storing a known project decision, pass
scope="/project/alpha/decisions". If you’re storing freeform agent output, omit the scope and let the LLM figure it out.
Use Case Examples
Multi-project team:Memory Slices
What Slices Are
AMemorySlice is a view across multiple, possibly disjoint scopes. Unlike a scope (which restricts to one subtree), a slice lets you recall from several branches simultaneously.
When to Use Slices vs Scopes
- Scope: Use when an agent or code block should be restricted to a single subtree. Example: an agent that only sees
/agent/researcher. - Slice: Use when you need to combine context from multiple branches. Example: an agent that reads from its own scope plus shared company knowledge.
Read-Only Slices
The most common pattern: give an agent read access to multiple branches without letting it write to shared areas.Read-Write Slices
When read-only is disabled, you can write to any of the included scopes, but you must specify which scope explicitly.Composite Scoring
Recall results are ranked by a weighted combination of three signals:- similarity =
1 / (1 + distance)from the vector index (0 to 1) - decay =
0.5^(age_days / half_life_days)— exponential decay (1.0 for today, 0.5 at half-life) - importance = the record’s importance score (0 to 1), set at encoding time
Memory constructor:
MemoryMatch includes a match_reasons list so you can see why a result ranked where it did (e.g. ["semantic", "recency", "importance"]).
LLM Analysis Layer
Memory uses the LLM in three ways:- On save — When you omit scope, categories, or importance, the LLM analyzes the content and suggests scope, categories, importance, and metadata (entities, dates, topics).
- On recall — For deep/auto recall, the LLM analyzes the query (keywords, time hints, suggested scopes, complexity) to guide retrieval.
- Extract memories —
extract_memories(content)breaks raw text (e.g. task output) into discrete memory statements. Agents use this before callingremember()on each statement so that atomic facts are stored instead of one large blob.
Memory Consolidation
When saving new content, the encoding pipeline automatically checks for similar existing records in storage. If the similarity is aboveconsolidation_threshold (default 0.85), the LLM decides what to do:
- keep — The existing record is still accurate and not redundant.
- update — The existing record should be updated with new information (LLM provides the merged content).
- delete — The existing record is outdated, superseded, or contradicted.
- insert_new — Whether the new content should also be inserted as a separate record.
Intra-batch Dedup
When usingremember_many(), items within the same batch are compared against each other before hitting storage. If two items have cosine similarity >= batch_dedup_threshold (default 0.98), the later one is silently dropped. This catches exact or near-exact duplicates within a single batch without any LLM calls (pure vector math).
Non-blocking Saves
remember_many() is non-blocking — it submits the encoding pipeline to a background thread and returns immediately. This means the agent can continue to the next task while memories are being saved.
Read Barrier
Everyrecall() call automatically calls drain_writes() before searching, ensuring the query always sees the latest persisted records. This is transparent — you never need to think about it.
Crew Shutdown
When a crew finishes,kickoff() drains all pending memory saves in its finally block, so no saves are lost even if the crew completes while background saves are in flight.
Standalone Usage
For scripts or notebooks where there’s no crew lifecycle, calldrain_writes() or close() explicitly:
Source and Privacy
Every memory record can carry asource tag for provenance tracking and a private flag for access control.
Source Tracking
Thesource parameter identifies where a memory came from:
Private Memories
Private memories are only visible to recall when thesource matches:
RecallFlow (Deep Recall)
recall() supports two depths:
depth="shallow"— Direct vector search with composite scoring. Fast (~200ms), no LLM calls.depth="deep"(default) — Runs a multi-step RecallFlow: query analysis, scope selection, parallel vector search, confidence-based routing, and optional recursive exploration when confidence is low.
query_analysis_threshold (default 200 characters) skip the LLM query analysis entirely, even in deep mode. Short queries like “What database do we use?” are already good search phrases — the LLM analysis adds little value. This saves ~1-3s per recall for typical short queries. Only longer queries (e.g. full task descriptions) go through LLM distillation into targeted sub-queries.
Embedder Configuration
Memory needs an embedding model to convert text into vectors for semantic search. You can configure this in three ways.Passing to Memory Directly
Via Crew Embedder Config
When usingmemory=True, the crew’s embedder config is passed through:
Provider Examples
OpenAI (default)
OpenAI (default)
Ollama (local, private)
Ollama (local, private)
Azure OpenAI
Azure OpenAI
Google AI
Google AI
Google Vertex AI
Google Vertex AI
Cohere
Cohere
VoyageAI
VoyageAI
AWS Bedrock
AWS Bedrock
Hugging Face
Hugging Face
Jina
Jina
IBM WatsonX
IBM WatsonX
Custom Embedder
Custom Embedder
Provider Reference
| Provider | Key | Typical Model | Notes |
|---|---|---|---|
| OpenAI | openai | text-embedding-3-small | Default. Set OPENAI_API_KEY. |
| Ollama | ollama | mxbai-embed-large | Local, no API key needed. |
| Azure OpenAI | azure | text-embedding-ada-002 | Requires deployment_id. |
| Google AI | google-generativeai | gemini-embedding-001 | Set GOOGLE_API_KEY. |
| Google Vertex | google-vertex | gemini-embedding-001 | Requires project_id. |
| Cohere | cohere | embed-english-v3.0 | Strong multilingual support. |
| VoyageAI | voyageai | voyage-3 | Optimized for retrieval. |
| AWS Bedrock | amazon-bedrock | amazon.titan-embed-text-v1 | Uses boto3 credentials. |
| Hugging Face | huggingface | all-MiniLM-L6-v2 | Local sentence-transformers. |
| Jina | jina | jina-embeddings-v2-base-en | Set JINA_API_KEY. |
| IBM WatsonX | watsonx | ibm/slate-30m-english-rtrvr | Requires project_id. |
| Sentence Transformer | sentence-transformer | all-MiniLM-L6-v2 | Local, no API key. |
| Custom | custom | — | Requires embedding_callable. |
LLM Configuration
Memory uses an LLM for save analysis (scope, categories, importance inference), consolidation decisions, and deep recall query analysis. You can configure which model to use.Memory() never fails at construction time, even if API keys aren’t set. Errors only surface when the LLM is actually called (e.g. when saving without explicit scope/categories, or during deep recall).
For fully offline/private operation, use a local model for both the LLM and embedder:
Storage Backend
- Default: LanceDB, stored under
./.crewai/memory(or$CREWAI_STORAGE_DIR/memoryif the env var is set, or the path you pass asstorage="path/to/dir"). - Custom backend: Implement the
StorageBackendprotocol (seecrewai.memory.storage.backend) and pass an instance toMemory(storage=your_backend).
Discovery
Inspect the scope hierarchy, categories, and records:Failure Behavior
If the LLM fails during analysis (network error, rate limit, invalid response), memory degrades gracefully:- Save analysis — A warning is logged and the memory is still stored with default scope
/, empty categories, and importance0.5. - Extract memories — The full content is stored as a single memory so nothing is dropped.
- Query analysis — Recall falls back to simple scope selection and vector search so you still get results.
Privacy Note
Memory content is sent to the configured LLM for analysis (scope/categories/importance on save, query analysis and optional deep recall). For sensitive data, use a local LLM (e.g. Ollama) or ensure your provider meets your compliance requirements.Memory Events
All memory operations emit events withsource_type="unified_memory". You can listen for timing, errors, and content.
| Event | Description | Key Properties |
|---|---|---|
| MemoryQueryStartedEvent | Query begins | query, limit |
| MemoryQueryCompletedEvent | Query succeeds | query, results, query_time_ms |
| MemoryQueryFailedEvent | Query fails | query, error |
| MemorySaveStartedEvent | Save begins | value, metadata |
| MemorySaveCompletedEvent | Save succeeds | value, save_time_ms |
| MemorySaveFailedEvent | Save fails | value, error |
| MemoryRetrievalStartedEvent | Agent retrieval starts | task_id |
| MemoryRetrievalCompletedEvent | Agent retrieval done | task_id, memory_content, retrieval_time_ms |
Troubleshooting
Memory not persisting?- Ensure the storage path is writable (default
./.crewai/memory). Passstorage="./your_path"to use a different directory, or set theCREWAI_STORAGE_DIRenvironment variable. - When using a crew, confirm
memory=Trueormemory=Memory(...)is set.
- Use
depth="shallow"for routine agent context. Reservedepth="deep"for complex queries. - Increase
query_analysis_thresholdto skip LLM analysis for more queries.
- Memory still saves/recalls with safe defaults. Check API keys, rate limits, and model availability if you want full LLM analysis.
- Memory saves run in a background thread. Errors are emitted as
MemorySaveFailedEventbut don’t crash the agent. Check logs for the root cause (usually LLM or embedder connection issues).
- LanceDB operations are serialized with a shared lock and retried automatically on conflict. This handles multiple
Memoryinstances pointing at the same database (e.g. agent memory + crew memory). No action needed.
Configuration Reference
All configuration is passed as keyword arguments toMemory(...). Every parameter has a sensible default.
| Parameter | Default | Description |
|---|---|---|
llm | "gpt-4o-mini" | LLM for analysis (model name or BaseLLM instance). |
storage | "lancedb" | Storage backend ("lancedb", a path string, or a StorageBackend instance). |
embedder | None (OpenAI default) | Embedder (config dict, callable, or None for default OpenAI). |
recency_weight | 0.3 | Weight for recency in composite score. |
semantic_weight | 0.5 | Weight for semantic similarity in composite score. |
importance_weight | 0.2 | Weight for importance in composite score. |
recency_half_life_days | 30 | Days for recency score to halve (exponential decay). |
consolidation_threshold | 0.85 | Similarity above which consolidation is triggered on save. Set to 1.0 to disable. |
consolidation_limit | 5 | Max existing records to compare during consolidation. |
default_importance | 0.5 | Importance assigned when not provided and LLM analysis is skipped. |
batch_dedup_threshold | 0.98 | Cosine similarity for dropping near-duplicates within a remember_many() batch. |
confidence_threshold_high | 0.8 | Recall confidence above which results are returned directly. |
confidence_threshold_low | 0.5 | Recall confidence below which deeper exploration is triggered. |
complex_query_threshold | 0.7 | For complex queries, explore deeper below this confidence. |
exploration_budget | 1 | Number of LLM-driven exploration rounds during deep recall. |
query_analysis_threshold | 200 | Queries shorter than this (in characters) skip LLM analysis during deep recall. |
