Allowing an agent to update its own memory in-band (while actively working on your tasks) solved day-to-day continuity. Yet as my usage scaled to marathon multi-thousand-step sessions across homelab servers (nas1, nas2, nas3), smart home automation, and Signal/Meshtastic bots, a deeper architectural limitation surfaced:
When an agent is focused on debugging a broken production service, asking it to simultaneously act as a meticulous archivist splits its attention, bloats the active context window, and misses critical side-discoveries.
Recently, Lamis Mukta from Anthropic’s Applied AI team presented a breakthrough architectural paradigm at AI-Native DevCon titled “Learning While You Sleep: Beyond Memory to Dreaming.”
In this post, I will break down Lamis Mukta’s Dreaming concept and show how I engineered and deployed a complete out-of-band REM memory consolidation engine (/home/gvoina/scripts/antigravity_dream.py) for Antigravity directly on top of my Obsidian vault.
1. The Paradigm Shift: From In-Band Note-Taking to Out-of-Band “Dreaming”
In cognitive neuroscience, the human brain does not permanently wire every episodic experience into the neocortex in real time. During wakefulness, the hippocampus rapidly buffers raw episodic traces. Then, while we sleep—across NREM replay and REM synthesis—the brain replays the day’s transcripts, extracts structural patterns into long-term neocortical memory, and prunes redundant synaptic connections.
Lamis Mukta applies this exact biological division of labor to AI agent systems:
Instead of forcing the live agent to do heavy memory reorganization mid-conversation, Dreaming runs as a periodic out-of-band batch process:
It ingests the transcripts from the agent’s daily sessions alongside the current memory state.
It runs an asynchronous Dreaming consolidation pass while the system is idle.
It outputs an updated memory state with new insights and organized structure—so that “next day’s agent sessions are automatically more intelligent.”
A Unified Two-Speed Memory System
Crucially, Dreaming does not replace real-time in-band memory; it complements it in a dual-speed architecture:
Left Loop — Real-Time Memory (Wakeful / In-Band): As Agent A, Agent B, and Agent C work on live tasks (in my setup: interactive CLI sessions, the signal-bot.service daemon, and the meshtastic_bot.py LoRa gateway), they read and write fast updates to shared Markdown files (Meta/antigravity-memory.md, Meta/states/antigravity.md, Meta/agent-log.md).
Right Loop — Dreaming (Sleep / Out-of-Band): Between sessions, a background consolidator harvests the raw session transcripts (transcript.jsonl) and performs three deep operations on the shared memory store:
Verify: Cross-check facts, hashes, and live states so stale or conflicting assumptions are eliminated.
Organize: Deduplicate overlapping notes, compact bloated lists, and keep context windows lean (mitigating proactive interference).
Enrich: Rescue unlogged discoveries buried deep inside raw session transcripts and promote them into structured long-term memory.
2. Inside a Dreaming Pass: $MEM -> clone -> $MEM_OUT
One of the sharpest systems-engineering insights in Lamis Mukta’s presentation is how a dreaming pass isolates its work from the live memory store:
Look closely at the three numbered steps in the slide above:
Step 1 (clone): The Input memory store ($MEM) is cloned into an isolated staging Output memory store ($MEM_OUT), while recording cryptographic hashes of the input files.
Step 2 (Orchestrator -> one per session -> Subagent): The Orchestrator iterates over unconsolidated session transcripts (Session 1 transcript, Session 2 transcript, Session 3 transcript).
Step 3 (read / write to reorganise): Each session pass reads and reorganizes $MEM_OUT—distilling new facts, merging duplicates, and pruning obsolete entries—before atomically promoting $MEM_OUT back to $MEM.
Translating This Architecture to Antigravity + Obsidian
Here is how our local implementation maps every component of Anthropic’s diagram onto Linux, SQLite, Obsidian, and Python:
Let’s walk through the four production engineering problems we solved in /home/gvoina/scripts/antigravity_dream.py.
Problem 1: Incremental Watermark Harvesting (NREM Replay)
Antigravity stores metadata for every conversation in ~/.gemini/antigravity-cli/conversation_summaries.db and writes step-by-step execution trajectories to ~/.gemini/antigravity-cli/brain/<conversation_id>/.system_generated/logs/transcript.jsonl.
Some conversations are short 4-step checks; others (like session e7c5ebab, which spanned weeks of homelab work) contain 3,938 steps and megabytes of raw run_command outputs. Feeding raw .jsonl files directly into an LLM would blow up the context window with noisy stdout dumps.
To solve this, antigravity_dream.py maintains an incremental watermark ledger in Meta/states/dream-state.json:
When a dream cycle starts, discover_unconsolidated_sessions() opens SQLite in read-only URI mode (mode=ro) so it never locks the active CLI, filters for sessions where current_step_count > watermark_step_count, and parses transcript.jsonlstarting strictly from prev_steps. It strips out intermediate tool outputs and extracts clean (User Request -> Final Planner Resolution) pairs.
Even better, to prevent recursive self-dreaming (where a headless LLM dream pass creates a new session in conversation_summaries.db, which the next dream pass then tries to consolidate), every dream prompt is stamped with [DREAM_WORKER_CONSOLIDATION] and automatically filtered out by the harvester.
Problem 2: Staging $MEM -> $MEM_OUT & Dual-Engine Synthesis
Following Slide 2 of Lamis Mukta’s architecture, clone_memory_store() copies Meta/antigravity-memory.md, Meta/states/antigravity.md, and Meta/user-profile.md into /tmp/antigravity-dream-mem-out-<id> and records their pre-dream SHA-256 digests:
def clone_memory_store() -> tuple[Path, dict]:
"""
Step 1 of Dreaming Pass (Lamis Mukta Architecture):
Clone Input Memory Store ($MEM = Meta/) to an isolated staging directory ($MEM_OUT)
while recording pre-dream SHA-256 hashes for Optimistic Concurrency Control.
"""
stage_dir = Path(tempfile.mkdtemp(prefix="antigravity-dream-mem-out-"))
(stage_dir / "states").mkdir(parents=True, exist_ok=True)
hashes = {}
for rel in ["antigravity-memory.md", "states/antigravity.md", "user-profile.md"]:
src = META_DIR / rel
dst = stage_dir / rel
if src.exists():
content = src.read_text(encoding="utf-8")
hashes[rel] = sha256_text(content)
dst.write_text(content, encoding="utf-8")
else:
hashes[rel] = sha256_text("")
return stage_dir, hashes
For synthesis (Verify, Organize, Enrich), antigravity_dream.py supports two engines:
LLM Agentic Consolidator (--engine llm / auto): Invokes agy headlessly with --disable-slash-commands --effort low --output-format json --json-schema <schema>, forcing the model to return a strict JSON object containing new_learnings, active_focus, new_milestone, and dream_summary.
Deterministic Heuristic Consolidator (--engine heuristic): A fast, zero-API-cost pattern engine that extracts domain facts (SMB shares, SSO ports, hardware/audio gear, CVE patch states, and final resolution conclusions from multi-step sessions) when offline or running quick dry-runs.
Problem 3: Mitigating Proactive Interference (Pruning & Deduplication)
If you only append to a memory file, older or duplicated notes eventually degrade retrieval accuracy—a phenomenon cognitive scientists call proactive interference.
During every dream pass, deduplicate_learnings_section() scans ## 5. Notes & Learnings Log in Meta/antigravity-memory.mdin reverse chronological order (newest to oldest). Using difflib.SequenceMatcher on normalized token strings, any older bullet with $\ge 82%$ similarity to a newer verified learning (or substring containment) is automatically compacted away. Simultaneously, prune_state_milestones() bounds Recent Milestones in Meta/states/antigravity.md to the top 8 items so the working-memory sticky note never grows unbounded.
Problem 4: Hash-Based Optimistic Concurrency Control (OCC)
What happens if a background dream pass takes 30 seconds to synthesize 6 session transcripts, and during those 30 seconds I send a message via Signal Bot that appends a new note to Meta/antigravity-memory.md?
If the dream job blindly overwrote Meta/antigravity-memory.md with $MEM_OUT, the live in-band update would be wiped out.
atomic_write_with_occ() prevents this using fcntl.flock + SHA-256 verification:
def atomic_write_with_occ(file_path: Path, new_content: str, expected_sha256: str, fallback_lines: list[str] = None) -> tuple[bool, bool]:
"""
Phase 3: Hash-Based Optimistic Concurrency Control + flock.
Returns (written_ok, conflict_resolved).
"""
LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
with open(LOCK_FILE, "w") as lock_fd:
fcntl.flock(lock_fd, fcntl.LOCK_EX)
try:
current_content = file_path.read_text(encoding="utf-8") if file_path.exists() else ""
current_sha = sha256_text(current_content)
conflict_resolved = False
if current_sha != expected_sha256:
# Concurrent in-band edit occurred during dreaming!
# Preserve live file and append only net-new distilled lines
conflict_resolved = True
merged = current_content.rstrip() + "\n"
if fallback_lines:
for line in fallback_lines:
if normalize_for_dedup(line) not in normalize_for_dedup(current_content):
merged += f"{line}\n"
new_content = merged
tmp_path = file_path.with_suffix(".dream.tmp")
tmp_path.write_text(new_content, encoding="utf-8")
os.replace(tmp_path, file_path)
return True, conflict_resolved
finally:
fcntl.flock(lock_fd, fcntl.LOCK_UN)
If the on-disk SHA-256 matches expected_sha256, the reorganized $MEM_OUT file is swapped in atomically via os.replace(). If the hash changed mid-dream, the consolidator switches to a non-destructive additive merge—preserving every concurrent in-band edit while still appending the newly distilled dream insights!
4. Real-World Proof: What Dream Cycle #1 Rescued from 4,500+ Steps
To test the system on real production data, I ran Dream Cycle #1 across my recent unconsolidated session transcripts:
When I opened the newly created Meta/dream-journal.md in Obsidian, the value of out-of-band dreaming was unmistakable. Earlier in the week, I had spent hours across sessions a17238b6 (167 steps) and e7c5ebab (3,938 steps) troubleshooting services on nas3 (192.168.1.50), configuring Samba shares, and checking PHP CVEs—none of which had been written to Meta/antigravity-memory.md at the time.
Dream Cycle #1 replayed all 6 sessions in the background and automatically distilled those exact missing facts into Meta/antigravity-memory.md and Meta/dream-journal.md:
When I ran python3 /home/gvoina/scripts/obsidian_memory.py dream a second time 10 seconds later, the watermark check completed in under 50 milliseconds:
Dream check complete: All session watermarks are current and memory files are clean.
5. Scheduling Nightly REM Sleep with systemd --user
An agent shouldn’t have to be told to go to sleep. To make Dreaming completely autonomous, I wired /home/gvoina/scripts/antigravity_dream.py into two systemd --user units:
~/.config/systemd/user/antigravity-dream.service
[Unit]
Description=Antigravity Out-of-Band Memory Dreaming & Consolidation Service
After=default.target
[Service]
Type=oneshot
ExecStart=/usr/bin/python3 /home/gvoina/scripts/antigravity_dream.py --engine auto --limit 8
StandardOutput=journal
StandardError=journal
~/.config/systemd/user/antigravity-dream.timer
[Unit]
Description=Nightly REM Dreaming Consolidation Timer for Antigravity Memory
[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=10min
Persistent=true
Unit=antigravity-dream.service
[Install]
WantedBy=timers.target
Setting Persistent=true is essential on a Linux workstation: if my laptop is suspended or powered off at 02:30 AM, systemd immediately triggers the missed dream cycle the moment I open the lid in the morning.
6. Conclusion: The Complete 4-Layer Cognitive Stack
With Out-of-Band Dreaming in place, my local AI assistant now operates on a complete, biologically inspired cognitive architecture inside Obsidian:
Every day I work with agy, it takes fast operational notes in real time. And every night at 2:30 AM, it replays our transcripts, extracts what matters, prunes what doesn’t, and wakes up sharper the next morning.
