In a recent post I made the case that MongoDB can serve as the “brain” of a modern AI application by combining durable state, retrieval, and application data in one place. That framing still holds, but brains are only useful if they can recall the right thing at the right time. I wanted to dig into agentic application development in more detail in a series of posts, so for the first real entry in this series, I want to start one layer below “agents” and one layer above raw storage: the context layer.

That might sound slightly less glamorous than “multi-agent orchestration,” which is exactly why it matters. Most enterprise AI systems do not fail because they lack a clever planner. They fail because the model sees the wrong document, too much irrelevant text, or none of the operational data that actually matters.

To make this concrete, the application thread for this series will be an enterprise support escalation copilot for a B2B SaaS team. By the end of the series, it should be able to answer questions about incidents, remember previous escalations, pull account context, and coordinate specialized agents when needed. Today, though, we’re giving it its first useful skill: finding the right context for the job.

Think about the kind of question a real support engineer asks:

“Acme’s enterprise tenant started seeing INV-4421 after upgrading to 3.8. Did we see this before, is there a known workaround, and does it affect EU clusters only?”

That is not a pure semantic search problem. It is part natural language, part exact identifier lookup, part metadata filtering, and part ranking problem. Error codes matter. Version numbers matter. Tenant boundaries matter. Timing matters. That’s why this is such a good place to start - and to solve this problem we’ll dig in with MongoDB and Voyage AI.

MongoDB Vector Search is built to search vector data alongside the rest of your operational data, supports filtering on other fields in the collection, and can be combined with full-text search for hybrid retrieval. MongoDB’s hybrid search documentation explicitly describes combining semantic and full-text search results with Reciprocal Rank Fusion, which is exactly what you want when a query mixes fuzzy intent with exact strings like issue IDs, SKUs, or feature flags.

On the retrieval-model side, Voyage provides high-accuracy embedding and reranking models, including newer capabilities like contextualized chunk embeddings, multimodal embeddings, and rerankers designed to refine the top candidate set after initial retrieval. MongoDB Atlas now also exposes Voyage models through its Embedding and Reranking API, currently in preview, which means you can either call Voyage models directly or keep retrieval models, vector search, and operational data closer together under Atlas.

So what does the retrieval core for our support copilot actually do?

First, it stores source material in MongoDB: runbooks, release notes, KB articles, previous incident reviews, ticket summaries, and whatever structured account data the support flow needs. Then it chunks the long-form content, embeds it with Voyage, and stores the vectors with the source text and metadata. At query time, it narrows scope using metadata like tenant, product, region, or severity; retrieves candidates semantically and lexically; reranks the best matches; and only then hands a compact, relevant context window to the LLM. In other words: don’t ask the model to be psychic when the database can be specific.

There are a lot of AI frameworks right now, and they absolutely do not all feel the same. But this is the first important pattern in the series: the framework should shape the developer experience, not force you to redesign the data layer every six months. The retrieval architecture is the stable part. MongoDB and Voyage AI are the stable parts. LangChain, LlamaIndex, Haystack, LangGraph, CrewAI, or whatever comes next should be able to sit on top of that foundation.

A framework-agnostic mental model

Before jumping into code, here is the mental model I’d keep fixed no matter which framework you prefer:

1. Put source documents and operational records in MongoDB.
2. Generate embeddings with Voyage.
3. Index vector fields and filter fields in MongoDB.
4. Use semantic retrieval for meaning.
5. Use full-text retrieval for exact strings.
6. Rerank the candidate set before generation.
7. Return only the context the model actually needs.

That shape maps cleanly to both MongoDB Vector Search and Voyage’s model stack. MongoDB handles vector indexes, full-text search, filterable metadata, and live application data; Voyage handles embeddings and reranking; the framework becomes the control surface.

Approach 1: LangChain for the shortest path from data to grounded answers

If the goal is to get a retrieval-backed application running quickly, LangChain remains a very practical starting point. MongoDB’s LangChain integration supports vector search, full-text search, and a hybrid retriever that combines both with Reciprocal Rank Fusion. It also supports pre-filtering with MQL expressions, which matters immediately for tenant scoping and product boundaries.

An illustrative version for our support copilot looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import os

from langchain_voyageai import VoyageAIEmbeddings
from langchain_mongodb.vectorstores import MongoDBAtlasVectorSearch
from langchain_mongodb.retrievers.hybrid_search import MongoDBAtlasHybridSearchRetriever

embeddings = VoyageAIEmbeddings(
    api_key=os.environ["VOYAGE_API_KEY"],
    model="voyage-4",
)

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
    connection_string=os.environ["MONGODB_URI"],
    namespace="support.context",
    embedding=embeddings,
    index_name="support_vector_index",
)

retriever = MongoDBAtlasHybridSearchRetriever(
    vectorstore=vector_store,
    search_index_name="support_search_index",
    k=8,
    fulltext_penalty=60.0,
    vector_penalty=60.0,
)

docs = retriever.invoke(
    "Acme tenant seeing INV-4421 after upgrading to 3.8"
)

In a production version, I’d pair that with metadata filters on fields like tenant_id, product, region, and severity, then pass the top candidates through a Voyage reranker before generation. The point is not that LangChain is magical. The point is that the MongoDB + Voyage retrieval story already fits the way LangChain applications are commonly assembled.

Approach 2: LlamaIndex when the center of gravity is the data itself

If LangChain often feels application-first, LlamaIndex tends to feel data-first. That makes it a very natural fit when you want to spend more time shaping ingestion, chunking, metadata, and query behavior.

Using MongoDB’s LlamaIndex integration we can use VoyageEmbedding alongside MongoDBAtlasVectorSearch to make metadata filters very explicit, which is helpful for real enterprise retrieval where “give me the right answer” usually means “give me the right answer for this tenant, this region, and this product line.”

The shape is roughly:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import os

from pymongo import MongoClient
from llama_index.embeddings.voyageai import VoyageEmbedding
from llama_index.vector_stores.mongodb import MongoDBAtlasVectorSearch
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.core.retrievers import VectorIndexRetriever
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

embed_model = VoyageEmbedding(
    voyage_api_key=os.environ["VOYAGE_API_KEY"],
    model_name="voyage-4",
)

mongo_client = MongoClient(os.environ["MONGODB_URI"])
vector_store = MongoDBAtlasVectorSearch(
    mongo_client,
    db_name="support",
    collection_name="context",
    vector_index_name="support_vector_index",
)

storage_context = StorageContext.from_defaults(vector_store=vector_store)

# docs is your loaded support corpus, such as runbooks, incident reviews,
# release notes, and ticket summaries.
vector_index = VectorStoreIndex.from_documents(
    docs,
    storage_context=storage_context,
    embed_model=embed_model,
)

filters = MetadataFilters(
    filters=[ExactMatchFilter(key="tenant_id", value="acme")]
)

retriever = VectorIndexRetriever(
    index=vector_index,
    filters=filters,
    similarity_top_k=10,
)

nodes = retriever.retrieve("Known workaround for INV-4421?")

What I like about this path is that it keeps the retrieval pipeline honest. You can see the data model. You can see the filter model. You can see how chunking choices affect what comes back. For article one in a series like this, that clarity is useful because it keeps us focused on context quality before we get distracted by agent loops.

Approach 3: Haystack when you want explicit, composable pipelines

Haystack is a nice fit for teams that prefer explicit components over higher-level abstractions. MongoDB’s Haystack integration uses a MongoDBAtlasDocumentStore with MongoDB retrievers, and the official tutorial pairs that with Voyage embedders. Haystack’s MongoDB integration also has separate semantic and full-text retrievers, which is useful when you want to make the retrieval strategy itself a first-class part of the pipeline.

A trimmed-down version looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from haystack import Pipeline
from haystack.utils import Secret
from haystack_integrations.components.embedders.voyage_embedders import VoyageTextEmbedder
from haystack_integrations.document_stores.mongodb_atlas import MongoDBAtlasDocumentStore
from haystack_integrations.components.retrievers.mongodb_atlas import MongoDBAtlasEmbeddingRetriever

document_store = MongoDBAtlasDocumentStore(
    mongo_connection_string=Secret.from_env_var("MONGODB_URI"),
    database_name="support",
    collection_name="context",
    vector_search_index="support_vector_index",
    full_text_search_index="support_search_index",
)

pipeline = Pipeline()
pipeline.add_component("query_embedder", VoyageTextEmbedder(model="voyage-4"))
pipeline.add_component(
    "retriever",
    MongoDBAtlasEmbeddingRetriever(document_store=document_store, top_k=10),
)
pipeline.connect("query_embedder.embedding", "retriever.query_embedding")

result = pipeline.run(
    {"query_embedder": {"text": "Known workaround for INV-4421?"}}
)

This is probably the most “pipe and fitting” version of the three, and that is a compliment. For enterprise teams, explicit systems are often easier to debug, evaluate, and explain. And once again, the interesting part is not that the framework is different. It is that the same MongoDB + Voyage retrieval core still fits.

Why MongoDB

The support copilot does not just need chunks in a vector store. It needs chunks, source documents, tenant metadata, ticket references, account records, release versions, and eventually execution state. MongoDB Vector Search lets you search semantic meaning alongside that operational data, pre-filter the search space using indexed fields, and combine vector and full-text retrieval when exact terms matter. Change streams then give you a way to react to new or updated records in real time, which is exactly what you want when incidents, tickets, or KB articles change during the workday.

And if you want an even tighter platform story, MongoDB Atlas now exposes Voyage models directly through the Embedding and Reranking API. That API is database-agnostic, but it pairs especially well with Atlas because it reduces the number of moving pieces needed to stand up a modern retrieval pipeline. Fewer services, fewer credentials, less trying to debug “why is this top result here?”.

This is also where the framework story becomes easier to reason about. LangChain, LlamaIndex, and Haystack all give you different ergonomics. MongoDB stays the system where the data lives. Voyage stays the retrieval layer that improves what gets surfaced. That is a much more durable architecture than betting everything on whichever orchestration framework happens to be loudest this quarter.

What next?

Once the retrieval core is solid, adding agents becomes a lot more interesting.

In the next post, I’d take this same support copilot and add short-term execution state and long-term memory. LangGraph is a natural next step as it separates persistence into checkpoints for thread state and stores for long-term memory, and MongoDB already has first-class integrations for both the LangGraph checkpointer and the long-term store. That is where the earlier “brain” idea becomes concrete: not just retrieval, but retrieval plus memory plus durable execution.

The broader trend line is pretty clear, too. Agent frameworks are converging on durable state and memory. Retrieval models are getting richer with contextualized chunk embeddings, multimodal embeddings, and better rerankers. MongoDB Atlas is moving retrieval models and database capabilities closer together. The winning application architecture is the one that can absorb those changes without forcing you to rebuild your data layer every few months. MongoDB and Voyage AI fit that direction unusually well.

The OpenAI Agents SDK has shipped session backends for SQLite, SQLAlchemy, Redis, and Dapr for a while now. With 0.14.2 (April 2026), MongoDBSession joined that list, and 0.14.6 added the docs page. If you’re already running MongoDB for application data, this is the moment to stop standing up a second store just to remember what the agent said three turns ago. The demo for this walkthrough is a small e-commerce support app with three handoff-connected agents and one MongoDB instance behind everything: customers, orders, support articles, and the conversation history.

Repo: https://github.com/alexbevi/mongodb-openai-agents-sdk-example.

What you’ll build

A CLI customer-support agent that identifies the user from MongoDB, hands off between a triage agent, an order-support agent, and a knowledge-base agent, and persists every turn (user message, tool call, tool output, assistant reply, handoff) to MongoDB via MongoDBSession. You quit, restart the process, log in with the same email, and the agent picks up the thread — no re-explaining the return you started yesterday.

Why MongoDB for sessions

A session backend has three jobs: store one item per turn, return them in order on the next run, and not corrupt itself when two processes write at once. The interesting part for MongoDB is how naturally each of those maps to things the database already does.

Items in a session are heterogeneous. A turn can be a user message, a tool call, a tool result, an assistant message, or a handoff record — each with its own shape. A document store takes those payloads as-is. There’s no messages table you have to migrate every time the SDK adds a new run-item type, and no JSON column to parse around.

Ordering is the hard part, and $inc is built for it. MongoDBSession stamps each message with a monotonically increasing seq counter — the SDK docs call this out explicitly: it preserves ordering across concurrent writers and processes. That’s a single-document atomic increment, not a distributed lock or an optimistic-retry loop. Two FastAPI workers handling the same session_id won’t interleave.

One store, one connection pool. This is the angle the demo actually showcases. The ecommerce_support database holds customers, orders, and support_articles next to agent_sessions and agent_messages. Tools query operational data, the SDK persists turns, and they share the same AsyncMongoClient. Adding session memory cost zero new infrastructure.

Walkthrough

1. Prerequisites

Python 3.10+, an OpenAI API key, and either a local mongod or a MongoDB Atlas cluster. Nothing in the demo requires Atlas-only features — a 27017 on localhost is fine.

2. Install

requirements.txt pins the new extra:

1
2
3
openai-agents[mongodb]>=0.14.2
python-dotenv>=1.0.0
pymongo>=4.13

1
pip install -r requirements.txt

The [mongodb] extra pulls in pymongo’s async client; the MongoDBSession class lives at agents.extensions.memory.MongoDBSession.

3. Connect

The demo uses one shared AsyncMongoClient per process (the right pattern — sessions don’t own the client, they share its pool):

1
2
3
4
5
6
7
8
9
10
11
12
13
from pymongo.asynchronous.mongo_client import AsyncMongoClient

MONGODB_URI = os.environ.get("MONGODB_URI", "mongodb://localhost:27017")
DB_NAME = "ecommerce_support"

mongo_client = AsyncMongoClient(MONGODB_URI)
db = mongo_client[DB_NAME]

try:
    await mongo_client.admin.command("ping")
except Exception as exc:
    print(f"\nCannot connect to MongoDB ({MONGODB_URI}):\n  {exc}")
    return

4. Seed and identify

python seed_data.py loads three demo customers, five products, five orders with embedded line items, and seven support articles indexed for $text search. Then main.py looks the customer up so the triage agent doesn’t have to ask for an email it already knows.

5. Instantiate the session

This is the integration:

1
2
3
4
5
6
7
8
9
10
11
session_id = f"support_{email.replace('@', '_at_').replace('.', '_')}"
session = MongoDBSession(
    session_id=session_id,
    client=mongo_client,
    database=DB_NAME,
)

if not await session.ping():
    print("Warning: MongoDB session storage is unavailable.")

existing = await session.get_items()

Constructing with client= (rather than MongoDBSession.from_uri(...)) means the session shares the app’s connection pool and session.close() becomes a no-op — the lifecycle stays with you. session.ping() is a real round-trip against MongoDB, useful for liveness probes.

6. Run

Pass session= to the runner. Everything else is the same SDK you already know:

1
2
3
4
5
6
7
with trace("Customer Support", group_id=conversation_id):
    result = await Runner.run(
        current_agent,
        input=user_input,
        context=ctx,
        session=session,   # MongoDB stores every turn automatically
    )

Have a conversation, quit, run python main.py again with the same email, and the next message gets the full prior context prepended automatically.

What MongoDB actually stored

After a few turns with alice@example.com, two collections show up in the ecommerce_support database. The interesting one is agent_messages. A representative document, abridged:

1
2
3
4
5
6
7
8
9
10
11
{
  _id: ObjectId("6620d1f4..."),
  session_id: "support_alice_at_example_com",   // partition key for this conversation
  seq: 7,                                        // monotonically increasing turn order
  message_data: {                                // the SDK's run-item, stored as-is
    type: "function_call_output",
    call_id: "call_8b2...",
    output: "Return initiated for order ORD-1001.\nReason: Not powerful enough...\nEstimated refund: $1,484.98 (includes 10% Gold loyalty bonus)"
  },
  created_at: ISODate("2026-04-26T19:14:08.221Z")
}

Three fields earn their keep:

• session_id is the only field every read filters on. It’s the partition key for “this conversation.”
• seq is the integer that makes ordering deterministic. The SDK reads with sort({ seq: 1 }) and writes with an atomic $inc against the matching agent_sessions document, which is what makes concurrent workers safe without a distributed lock.
• message_data is the SDK’s run-item — a user message, tool call, tool output, assistant message, or handoff. Different shape every time. The document model just stores it.

agent_sessions holds one document per session_id with the current high-water seq and timestamps — that’s the counter $inc operates on.

The SDK creates its indexes on first use (per the sessions docs). You’ll see a compound index on (session_id, seq) on agent_messages (the only access pattern the SDK has — fetch ordered history for one session) and a unique index on session_id in agent_sessions.

Production notes

For Atlas, swap the URI for mongodb+srv://... — MongoDBSession accepts it without any other change. If abandoned conversations accumulate, add a TTL index on agent_messages.created_at and old turns retire on their own.

Connection lifetime matters: keep one AsyncMongoClient per process, construct MongoDBSession(client=...) per request, and let the Runner do the rest. Don’t reach for MongoDBSession.from_uri(...) in a web handler — it builds and tears down a client every call. The session needs read/write on the two configured collections (defaults agent_sessions and agent_messages, both overridable via sessions_collection= and messages_collection=). The seq counter keeps concurrent writers safe, but fanning the same session_id across processes will interleave their turns — safe, but probably not what the user meant.

Try it yourself

1
2
3
4
5
6
git clone https://github.com/alexbevi/mongodb-openai-agents-sdk-example
cd mongodb-openai-agents-sdk-example
pip install -r requirements.txt
cp env.example .env          # set OPENAI_API_KEY and MONGODB_URI
python seed_data.py
python main.py

Required env vars: OPENAI_API_KEY, MONGODB_URI (defaults to mongodb://localhost:27017). Demo accounts: alice@example.com (Gold), bob@example.com (Standard), carol@example.com (Platinum).

Where to go next

• The full session API surface — get_items, add_items, pop_item, clear_session, ping — is documented in the Sessions overview, including the MongoDB-specific notes on collection naming and Atlas URIs.
• Wrap your MongoDBSession in OpenAIResponsesCompactionSession once threads grow long; it summarizes old turns server-side and rewrites the underlying session.
• The natural next MongoDB feature for this demo is Atlas Vector Search — store embeddings on support_articles and replace the $text query in search_knowledge_base with $vectorSearch. Same database, same client, one new index.

Durable memory is not raw chat replay. Vertex AI Memory Bank is built for identity-scoped, cross-session personalization and LLM-driven knowledge extraction, and Google’s ADK memory write-up describes Memory Bank as extracting key information from session data rather than replaying every turn. LangChain’s memory model is equally explicit: long-term memory can be semantic (facts), episodic (past actions), or procedural (rules and prompts).

Agent memory should be structured data, not opaque blobs. LangChain stores persist long-term memory as JSON documents, and ADK’s DatabaseSessionService migration moved session serialization from pickle-based storage to JSON-based storage in v1.22.0. MongoDB’s document model matches that reality directly.

MongoDB is a strong fit because retrieval lives in the same system as the memory. MongoDB Vector Search supports both approximate and exact nearest-neighbor search, and the default index type is HNSW. Vector search pre-filters let you scope recall by fields like user_id, tenant_id, or memory_type before embeddings are compared. Hybrid search combines vector and full-text retrieval with reciprocal rank fusion, which is exactly what memory needs when the data mixes natural language with exact identifiers like invoice IDs, feature flags, or product SKUs.

Memory also needs retention and update policy. TTL indexes automatically expire session artifacts, scratchpads, or short-lived summaries. Change streams give you a real-time feed of inserts and updates, which is the right trigger for summarization, entity extraction, or memory distillation jobs. When the data is relationship-heavy instead of chunk-heavy, GraphRAG on MongoDB uses entities, edges, and $graphLookup for relationship-aware, multi-hop retrieval.

This is not a narrow LangChain story. MongoDB publishes integrations for LangGraph, LangChain, LlamaIndex, Semantic Kernel, Haystack, Spring AI, CrewAI, and Vertex AI. LlamaIndex can use MongoDB for the vector store, document store, and index store. Mem0 also supports MongoDB as a memory backend. MongoDB fits the storage contract these frameworks keep converging on: structured documents plus semantic, lexical, and graph-based retrieval.

LangGraph: short-term checkpoints and long-term memory in one database

MongoDB’s LangGraph integration exposes MongoDBSaver for checkpoints and MongoDBStore for durable memory, with optional vector indexing and TTL-based expiry. That maps directly to LangGraph’s own split between thread persistence and store-backed recall.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
from pymongo import MongoClient
from langgraph.checkpoint.mongodb import MongoDBSaver
from langgraph.store.mongodb import MongoDBStore, create_vector_index_config
from langchain_openai import OpenAIEmbeddings

MONGODB_URI = "<connection-string>"

# Short-term memory: thread checkpoints
client = MongoClient(MONGODB_URI)
checkpointer = MongoDBSaver(client)

# Long-term memory: semantic store with metadata filters
index_config = create_vector_index_config(
    embed=OpenAIEmbeddings(model="text-embedding-3-small"),
    dims=1536,
    fields=["content"],
    filters=["user_id", "memory_type"],
)

# Assume `builder` is an existing LangGraph StateGraph
with MongoDBStore.from_conn_string(
    conn_string=MONGODB_URI,
    db_name="agent_memory",
    collection_name="memories",
    index_config=index_config,
    ttl_config={
        "default_ttl": 60 * 60 * 24 * 30,   # 30 days
        "refresh_on_read": True,
    },
) as store:
    graph = builder.compile(checkpointer=checkpointer, store=store)

    store.put(
        namespace=("user-42", "memories"),
        key="pref:vegan:soho",
        value={
            "content": "User prefers vegan restaurants near SoHo.",
            "user_id": "user-42",
            "memory_type": "semantic",
        },
    )

    results = store.search(
        ("user-42", "memories"),
        query="Where should I book dinner tonight?",
        limit=3,
    )

    for result in results:
        print(result.value)

This is the clean MongoDB story: checkpoints for working memory, a store for long-term memory, vector retrieval for recall, metadata filters for isolation, and TTL for automatic cleanup. The same database handles all of it.

LangChain: chat history plus hybrid recall

At the LangChain layer, MongoDB covers both conversation state and retrieval. MongoDBChatMessageHistory persists per-session message history, MongoDBAtlasVectorSearch stores semantic memories, and MongoDBAtlasHybridSearchRetriever fuses lexical and semantic recall.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
from langchain_core.documents import Document
from langchain_mongodb.chat_message_histories import MongoDBChatMessageHistory
from langchain_mongodb.retrievers.hybrid_search import MongoDBAtlasHybridSearchRetriever
from langchain_mongodb.vectorstores import MongoDBAtlasVectorSearch
from langchain_openai import OpenAIEmbeddings

MONGODB_URI = "<connection-string>"

history = MongoDBChatMessageHistory(
    session_id="user-42:thread-7",
    connection_string=MONGODB_URI,
    database_name="agent_memory",
    collection_name="chat_history",
    history_size=20,
)

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
    connection_string=MONGODB_URI,
    namespace="agent_memory.user_memories",
    embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
    index_name="vector_index",
)

vector_store.add_documents(
    [
        Document(
            page_content="User prefers vegan restaurants near SoHo.",
            metadata={"user_id": "user-42", "memory_type": "semantic"},
        ),
        Document(
            page_content="Invoice 8419 was disputed last month.",
            metadata={"user_id": "user-42", "memory_type": "episodic"},
        ),
    ]
)

history.add_user_message("Find dinner options for me in SoHo.")

retriever = MongoDBAtlasHybridSearchRetriever(
    vectorstore=vector_store,
    search_index_name="search_index",
    top_k=5,
    fulltext_penalty=50,
    vector_penalty=50,
)

docs = retriever.invoke("Find dinner options for a vegan user in SoHo")
for doc in docs:
    print(doc.page_content, doc.metadata)

Hybrid retrieval is not optional in a serious memory system. “Prefers vegan restaurants in SoHo” is semantic. “Invoice 8419” is lexical. MongoDB’s hybrid retriever exists because production memory contains both.

Google ADK: the Sessions-and-Memory model maps cleanly to MongoDB

Google ADK makes the architecture explicit. SessionService manages session objects, applies state_delta, and appends event history. MemoryService manages long-term semantic memory across sessions. The Sessions docs currently list InMemorySessionService, VertexAiSessionService, and DatabaseSessionService, so MongoDB is not a built-in backend today. But ADK exposes base session and memory service abstractions, which makes MongoDB a natural implementation target rather than a workaround. Memory Bank then adds the identity-scoped memory semantics on top.

A MongoDB-backed ADK deployment should separate sessions, events, and distilled memories into dedicated collections. That mirrors ADK’s documented split between mutable session state, append-only event history, and searchable long-term memory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
from datetime import datetime, timezone
from pymongo import ASCENDING, MongoClient

MONGODB_URI = "<connection-string>"

client = MongoClient(MONGODB_URI)
db = client["agent_memory"]

sessions = db["adk_sessions"]
events = db["adk_events"]
memories = db["adk_memories"]

sessions.create_index(
    [("app_name", ASCENDING), ("user_id", ASCENDING), ("session_id", ASCENDING)],
    unique=True,
)

events.create_index(
    [
        ("app_name", ASCENDING),
        ("user_id", ASCENDING),
        ("session_id", ASCENDING),
        ("timestamp", ASCENDING),
    ]
)

memories.create_index([("user_id", ASCENDING), ("memory_type", ASCENDING)])
# Create a MongoDB Vector Search index on memories.embedding
# Mark user_id and memory_type as filter fields in the index definition.

def persist_session_turn(app_name: str, user_id: str, session_id: str, event: dict, state: dict) -> None:
    now = datetime.now(timezone.utc)

    sessions.update_one(
        {"app_name": app_name, "user_id": user_id, "session_id": session_id},
        {
            "$set": {"state": state, "updated_at": now},
            "$setOnInsert": {"created_at": now},
        },
        upsert=True,
    )

    events.insert_one(
        {
            "app_name": app_name,
            "user_id": user_id,
            "session_id": session_id,
            "timestamp": event["timestamp"],
            "event": event,
        }
    )

def store_memory(user_id: str, content: str, embedding: list[float], source_session_id: str) -> None:
    memories.insert_one(
        {
            "user_id": user_id,
            "memory_type": "semantic",
            "content": content,
            "embedding": embedding,
            "source_session_ids": [source_session_id],
            "created_at": datetime.now(timezone.utc),
        }
    )

def search_memories(user_id: str, query_embedding: list[float]):
    pipeline = [
        {
            "$vectorSearch": {
                "index": "memory_vector_index",
                "path": "embedding",
                "queryVector": query_embedding,
                "numCandidates": 100,
                "limit": 5,
                "filter": {"user_id": user_id, "memory_type": "semantic"},
            }
        },
        {
            "$project": {
                "content": 1,
                "memory_type": 1,
                "score": {"$meta": "vectorSearchScore"},
            }
        },
    ]
    return list(memories.aggregate(pipeline))

This is a design sketch, not an official ADK adapter. The point is that ADK’s abstractions already describe a storage model MongoDB handles well: mutable session state, append-only events, and searchable long-term memory. Once a MemoryService exists, ADK’s built-in PreloadMemory and LoadMemory tools can use it.

Dedicated memory layers also fit: Mem0 on MongoDB

MongoDB is not only useful when memory is native to the agent framework. Mem0’s MongoDB backend supports MongoDB directly as a vector database for memory storage and retrieval. That matters because it shows MongoDB works both as the application database and as the substrate beneath a dedicated memory layer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from mem0 import Memory

config = {
    "vector_store": {
        "provider": "mongodb",
        "config": {
            "db_name": "mem0_db",
            "collection_name": "mem0_collection",
            "mongo_uri": "<connection-string>",
        },
    }
}

memory = Memory.from_config(config)

messages = [
    {"role": "user", "content": "I'm planning to watch a movie tonight."},
    {"role": "assistant", "content": "What genres do you like?"},
    {"role": "user", "content": "I love sci-fi, not thrillers."},
]

memory.add(messages, user_id="alice", metadata={"category": "movies"})

The architectural value is the same as in LangGraph and LangChain: persistent memory objects, vector retrieval, and application data can live in one operational system instead of being spread across separate services.

Why MongoDB is the best choice here

MongoDB is the best choice when you want one system to hold agent state, long-term memory, retrieval data, and the application records the agent reasons over. The document model matches how current frameworks persist memory. Vector Search and Search cover recall. TTL indexes and change streams cover retention and event-driven memory extraction. GraphRAG covers relationship-heavy data. The result is not “a vector store with extra features.” It is a memory layer that can also be the system of record. That is why MongoDB works as the brain of a modern AI application.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 5: Debugging Audio Issues
• Reverse Engineering Harvester with Ghidra and Codex - Part 6: Timers

Article 6 of 6 in this series.

The Harvester game engine supports time-based functionality through the same scripting system that drives room transitions, object interactions, NPC state changes, and cutscenes.

While re-implementing the engine, one of the main challenges has been figuring out how to debug those systems in a way that gives future Codex prompts useful context. Logs are great after something fires, but timers have a different failure mode: you often need to know whether a timer exists, whether it is currently enabled, how much time is left, and what command chain it will run before it expires.

That is especially important in Harvester because timers are not just visual delays. They can damage the player, monsterfy NPCs, advance dialogue staging, unlock doors, and trigger authored room events.

Timer records in HARVEST.SCR

In part 4 I covered the command-opcode side of the script. Timers sit beside those COMMAND records as their own world records. Once HARVEST.SCR is XOR-decoded, a timer looks like this:

1
initialSeconds TIMER timerName roomName actionTag enabled looping global

For example:

1
2
3
133 TIMER "ACID_TIMER3" "EYEHALL" "HURT_PC_ACIDA" "F" "F" "T"
266 TIMER "ACID_TIMER2" "EYEHALL" "HURT_PC_ACIDB" "F" "F" "T"
400 TIMER "ACID_TIMER"  "EYEHALL" "KILL_PC_ACID"  "F" "F" "T"

The fields map cleanly to the runtime data structure:

The decoded script contains 76 real TIMER records. Most are initially disabled and are started by a nearby COMMAND, but a few start enabled as part of room setup.

At runtime, these timers are materialized as invisible runtime entities. That turned out to be an important detail: timer state is not only a field in the parsed script record. A live countdown exists in the room entity list, and global timers can survive room changes by preserving that live entity instead of destroying it with the rest of the room.

When a timer entity expires, the room loop records the expired timer name, resolves the backing TimerRecord, and dispatches the record’s actionTag. In other words, the timer name is just the lookup key. The action is whatever command chain is stored in the timer record.

Starting and stopping timers

Script command chains control timers with SET_TIMER and KILL_TIMER:

1
2
3
COMMAND "SET_HALL_TIMER"   "SET_TIMER" "ACID_TIMER"  "ON" "" "KILL_TRIG_TIMER"
COMMAND "SET_HALL_2TIMER"  "SET_TIMER" "ACID_TIMER2" "ON" "" "SET_HALL_2TIMER2"
COMMAND "SET_HALL_2TIMER2" "SET_TIMER" "ACID_TIMER3" "ON" "" ""

SET_TIMER ... ON enables a timer. If the timer was previously disabled, the current value is reset back to the initial value. SET_TIMER ... OFF and KILL_TIMER disable it.

The subtle part is that timer commands often live in longer command chains. They are not isolated API calls. Starting a timer might be one step in a room-entry sequence, and the timer’s expiry might run another command chain that branches on flags, shows text, adjusts HP, or changes rooms.

In the MAINHALL with DEBUG_TIMERS and DEBUG_ROOM toggled

A good example is when you enter the MAINHALL on Disc 3, because it uses multiple global timers with staggered deadlines.

1
2
3
4
5
6
7
8
9
133 TIMER "ACID_TIMER3" "EYEHALL" "HURT_PC_ACIDA" "F" "F" "T"
266 TIMER "ACID_TIMER2" "EYEHALL" "HURT_PC_ACIDB" "F" "F" "T"
400 TIMER "ACID_TIMER"  "EYEHALL" "KILL_PC_ACID"  "F" "F" "T"

COMMAND "HURT_PC_ACIDA"   "KILL_TIMER" "ACID_TIMER3" "" "" "HURT_PC_ACID"
COMMAND "HURT_PC_ACIDB"   "KILL_TIMER" "ACID_TIMER3" "" "" "HURT_PC_ACID"
COMMAND "HURT_PC_ACID"    "CHECK_FLAG" "CLEANED_CLOTHES" "" "HURT_PC_ACID_1" ""
COMMAND "HURT_PC_ACID_1"  "SHOW_TEXT"  "ACID_TEXT2" "" "" "HURT_PC_ACID_2"
COMMAND "HURT_PC_ACID_2"  "ADJ_HP"     "-7" "" "" ""

The first two timers are warning/damage stages. When they expire, they enter HURT_PC_ACID, which checks the CLEANED_CLOTHES flag. If the clothes have not been cleaned, the game shows acid text and subtracts 7 HP. If the flag has been set, the branch target is empty and the command chain stops.

The last timer, ACID_TIMER, is the hard fail path:

1
400 TIMER "ACID_TIMER" "EYEHALL" "KILL_PC_ACID" "F" "F" "T"

Cleaning the clothes is itself just another scripted interaction. Using the money on the cloakroom attendant starts CLEAN_CLOTHES, which shows text, sets the flag, removes the money, and disables the lethal acid timer:

1
2
3
4
5
6
7
8
USEITEM "BARCASHFIVE" "CLOAKROOM" "CLOAK_ATND" "CLEAN_CLOTHES"

COMMAND "CLEAN_CLOTHES"   "SHOW_TEXT" "CLEANED_CLOTHS" "" "" "CLEAN_CLOTHES_1"
COMMAND "CLEAN_CLOTHES_1" "SET_FLAG"  "CLEANED_CLOTHES" "T" "" "CLEAN_CLOTHES_2"
COMMAND "CLEAN_CLOTHES_2" "DELETE"    "CLOAKROOM" "BARCASHFIVE" "" "CLEAN_CLOTHES_3"
COMMAND "CLEAN_CLOTHES_3" "DELETE"    "INVENTORY" "BARCASHFIVE" "" "CLEAN_CLOTHES_4"
COMMAND "CLEAN_CLOTHES_4" "SET_TIMER" "ACID_TIMER" "OFF" "" "CLEAN_CLOTHES_5"
COMMAND "CLEAN_CLOTHES_5" "KILL_TIMER" "ACID_TIMER" "" "" ""

This is the kind of script graph that is hard to reason about from static records alone. Some of the authored data is also a little odd: both intermediate acid chains kill ACID_TIMER3, even though the second one is entered by ACID_TIMER2. Watching the live timers makes it much easier to tell whether that is harmless authored data, a reimplementation bug, or a broken state sync.

Adding a timer overlay

To make this easier to debug, I asked Codex to add a console command that would render active timers directly over the room:

the game manages timers periodically. I want to introduce a DEBUG_TIMERS command that when enabled it will overlay text on the screen that will be the timer name, starting value, current value, action to take when timer expires.

If multiple timers are active, they should appear one after the other. Draw this near the middle-left of the screen. It should be white text on a black background

Instead of wiring this up myself, Codex was able to do it in about 5 minutes.

The implementation is intentionally small:

• DEBUG_TIMERS toggles a boolean on the Harvester engine.
• The room renderer checks that boolean after drawing the room and other debug overlays.
• The overlay walks the known timer records, finds matching live timer entities, filters out disabled timers, and formats each row as name start=initial current=current action=tag.
• Text is drawn near the middle-left of the screen using white text over a black rectangle.

This means the overlay is showing live runtime state, not just parsed script data. If a timer is missing from the live entity list, disabled, or no longer counting down, it disappears.

The label format is deliberately boring:

1
2
3
ACID_TIMER3 start=133 current=128 action=HURT_PC_ACIDA
ACID_TIMER2 start=266 current=261 action=HURT_PC_ACIDB
ACID_TIMER start=400 current=395 action=KILL_PC_ACID

That is exactly the information I need while debugging: what is active, how long it has left, and what will happen when it fires.

Having a visual indicator for timer progress, and whether those timers were triggering the correct actions, made it much easier to discover some more obscure bugs.

These log lines are all coming from the game’s scripts. In this run, the clothes have been cleaned, so the acid timers still reach their action chains, but the CLEANED_CLOTHES flag prevents the HP penalty path from continuing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Harvester: action tag 'CLEAN_CLOTHES' step=0 tag='CLEAN_CLOTHES' opcode='SHOW_TEXT' args=['CLEANED_CLOTHS','','','CLEAN_CLOTHES_1']
Harvester: action tag 'CLEAN_CLOTHES_1' step=0 tag='CLEAN_CLOTHES_1' opcode='SET_FLAG' args=['CLEANED_CLOTHES','T','','CLEAN_CLOTHES_2']
Harvester: action tag 'CLEAN_CLOTHES_1' SET_FLAG 'CLEANED_CLOTHES' 0 -> 1 existed=1 changed=1
Harvester: action tag 'CLEAN_CLOTHES_1' step=1 tag='CLEAN_CLOTHES_2' opcode='DELETE' args=['CLOAKROOM','BARCASHFIVE','','CLEAN_CLOTHES_3']
Harvester: action tag 'CLEAN_CLOTHES_1' step=2 tag='CLEAN_CLOTHES_3' opcode='DELETE' args=['INVENTORY','BARCASHFIVE','','CLEAN_CLOTHES_4']
Harvester: action tag 'CLEAN_CLOTHES_1' step=3 tag='CLEAN_CLOTHES_4' opcode='SET_TIMER' args=['ACID_TIMER','OFF','','CLEAN_CLOTHES_5']
Harvester: action tag 'CLEAN_CLOTHES_1' step=4 tag='CLEAN_CLOTHES_5' opcode='KILL_TIMER' args=['ACID_TIMER','','','']

Harvester: timer command 'ACID_TIMER3' step=0 tag='HURT_PC_ACIDA' opcode='KILL_TIMER' args=['ACID_TIMER3','','','HURT_PC_ACID']
Harvester: timer command 'ACID_TIMER3' step=1 tag='HURT_PC_ACID' opcode='CHECK_FLAG' args=['CLEANED_CLOTHES','','HURT_PC_ACID_1','']
Harvester: timer command 'ACID_TIMER3' flag 'CLEANED_CLOTHES' -> 1

Harvester: timer command 'ACID_TIMER2' step=0 tag='HURT_PC_ACIDB' opcode='KILL_TIMER' args=['ACID_TIMER3','','','HURT_PC_ACID']
Harvester: timer command 'ACID_TIMER2' step=1 tag='HURT_PC_ACID' opcode='CHECK_FLAG' args=['CLEANED_CLOTHES','','HURT_PC_ACID_1','']
Harvester: timer command 'ACID_TIMER2' flag 'CLEANED_CLOTHES' -> 1

The overlay was also useful outside this one hallway. Some NPC state changes are timer-driven too:

1
2
3
4
5
COMMAND "START_INQ_TIM" "SET_TIMER" "INQUIST_ATTACK_TIMER" "ON" "" ""
120 TIMER "INQUIST_ATTACK_TIMER" "PAIN" "MNSTFY_INQUIST" "F" "F" "F"

COMMAND "START_MERCY_TIMR" "SET_TIMER" "GLADIATOR_TIMER" "ON" "" ""
30 TIMER "GLADIATOR_TIMER" "MERCY" "MNST_GLAD" "F" "F" "F"

Those are awkward to verify with logs alone because nothing visible happens until the timer expires. With DEBUG_TIMERS enabled, I can start the dialogue branch, see the countdown appear, wait for it to hit zero, and then check whether the expected monsterfy command chain ran.

That has become the general debugging pattern for this engine work: use Ghidra and the decoded script to understand the original data model, add small runtime instrumentation when the model is too indirect to observe comfortably, then feed the resulting logs, screenshots, and videos back into the next prompt.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 4: Command Opcodes
• Reverse Engineering Harvester with Ghidra and Codex - Part 5: Debugging Audio Issues
• → Reverse Engineering Harvester with Ghidra and Codex - Part 6: Timers

Article 5 of 6 in this series.

An issue I’ve noticed with the harvester engine implementation is that every time there’s an audio sample played, it starts with a distinct “popping” sound. I thought this could be an issue with how I’d approached the re-implementation or disassembly, but after a few passes at decompiling the audio code I’d always end up with the same result.

While doing some unrelated research I stumbled on Kostya’s Boring Codec World and a post he’d made on FutureVision formats, discussing Harvester audio/video specifically:

“Music format is based on IMA ADPCM with minimalistic file header and the default predictor. Which means it needs some time to adapt to the actual coming signal amplitudes. In result the old format description based on reverse engineering recommends skipping first 7-57 bytes right after the header because of the garbage sound it produces. When I looked into the binary reference (an adventure that will be described in an upcoming post), it turned out that in certain cases they simply zero out the first 50 decoded samples.”

Since our approach to do everything with Ghidra/Codex, I thought this would be a good opportunity to try just prompting GPT-5.4 with a bit more context to see what it would come back with. I tried feeding the following prompt as a result:

1
2
3
based on https://codecs.multimedia.cx/2021/01/a-look-on-futurevision-formats/ "Music format is based on IMA ADPCM with minimalistic file header and the default predictor. Which means it needs some time to adapt to the actual coming signal amplitudes. In result the old format description based on reverse engineering recommends skipping first 7-57 bytes right after the header because of the garbage sound it produces. When I looked into the binary reference (an adventure that will be described in an upcoming post), it turned out that in certain cases they simply zero out the first 50 decoded samples."

Can we look into how realistic this may be based on what's in ghidra? The audio still produces a popping sound on each sample being played, so maybe this might help address that

After processing for a few minutes, the response back was promising:

1
2
3
4
5
6
7
8
Yes. Based on live Ghidra, the article is not just plausible, it matches what HARVEST.LE actually does.

start_music_stream at 0x1bfc0 decodes the first FCMP chunk and then zeroes the first 0x64 decoded bytes before queueing it. On the common 16-bit streams, that is 50 samples.

load_sound_sample at 0x18470 and load_dialogue_voice_sample at 0x191d0 are even more aggressive: they queue from decoded_pcm + 0x1f4, so they discard the first 500 decoded bytes of one-shot/dialogue FCMP playback.
The current ScummVM Harvester decoder was doing neither, so a clip-start pop on CMP-backed audio is consistent with that mismatch.

I patched media_manager.cpp (line 127) to mirror the native split: music now mutes the first 100 decoded bytes, while startup speech/other FCMP playback trims the first 500 decoded bytes. WAV handling is unchanged. 

The applyStartupFcmpWarmup function that this prompt resulted in effectively addressed the issue we were experiencing.

There’s still a slight pop on the first FST video being loaded, but all scene sound effects and dialogue interactions are now pop-free.

The patch is below to show how this was incorporated into the engine’s media manager.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
diff --git a/engines/harvester/media_manager.cpp b/engines/harvester/media_manager.cpp
index 691863468fa6..e953c0226cba 100644
--- a/engines/harvester/media_manager.cpp
+++ b/engines/harvester/media_manager.cpp
@@ -41,6 +41,11 @@ namespace Harvester {
 
 namespace {
 
+enum class FcmpWarmupMode {
+	kMusic,
+	kSample
+};
+
 static const int8 kHarvesterImaIndexAdjustTable[16] = {
 	-1, -1, -1, -1, 2, 4, 6, 8,
 	-1, -1, -1, -1, 2, 4, 6, 8
@@ -119,7 +124,29 @@ static uint32 decodeHarvesterFcmp(byte *dest, const byte *src, uint32 srcSize, u
 	return decodedSize;
 }
 
-static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableReadStream *stream) {
+static void applyStartupFcmpWarmup(byte *decodedPcm, uint32 &decodedSize, FcmpWarmupMode warmupMode) {
+	if (!decodedPcm || decodedSize == 0)
+		return;
+
+	if (warmupMode == FcmpWarmupMode::kMusic) {
+		// Native start_music_stream zeros the first 0x64 decoded bytes of the first FCMP chunk.
+		memset(decodedPcm, 0, MIN<uint32>(decodedSize, 100));
+		return;
+	}
+
+	// Native load_sound_sample/load_dialogue_voice_sample queue from decoded_pcm + 0x1f4.
+	const uint32 trimBytes = MIN<uint32>(decodedSize, 500);
+	if (trimBytes == decodedSize) {
+		memset(decodedPcm, 0, decodedSize);
+		return;
+	}
+
+	memmove(decodedPcm, decodedPcm + trimBytes, decodedSize - trimBytes);
+	decodedSize -= trimBytes;
+}
+
+static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableReadStream *stream,
+		FcmpWarmupMode warmupMode) {
 	if (!stream)
 		return nullptr;
 
@@ -156,7 +183,7 @@ static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableRead
 				return nullptr;
 			}
 
-			const uint32 decodedSize = payloadSize * (bitsPerSample >> 2);
+			uint32 decodedSize = payloadSize * (bitsPerSample >> 2);
 			byte *decodedPcm = (byte *)malloc(decodedSize);
 			if (!decodedPcm) {
 				free(compressedPayload);
@@ -165,6 +192,7 @@ static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableRead
 			}
 
 			decodeHarvesterFcmp(decodedPcm, compressedPayload, payloadSize, bitsPerSample);
+			applyStartupFcmpWarmup(decodedPcm, decodedSize, warmupMode);
 			free(compressedPayload);
 			delete stream;
 			return Audio::makeRawStream(decodedPcm, decodedSize, sampleRate,
@@ -177,8 +205,9 @@ static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableRead
 	return Audio::makeWAVStream(stream, DisposeAfterUse::YES);
 }
 
-static Audio::SeekableAudioStream *openStartupAudioStream(ResourceManager &resources, const Common::String &path) {
-	return decodeStartupAudioStream(resources.openFile(path));
+static Audio::SeekableAudioStream *openStartupAudioStream(ResourceManager &resources,
+		const Common::String &path, FcmpWarmupMode warmupMode) {
+	return decodeStartupAudioStream(resources.openFile(path), warmupMode);
 }
 
 } // End of anonymous namespace
@@ -268,7 +297,8 @@ bool MediaManager::playMusic(const Common::String &path) {
 		return true;
 	}
 
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kMusic);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup music '%s'", path.c_str());
 		return false;
@@ -323,7 +353,8 @@ bool MediaManager::playSound(const Common::String &path) {
 	_soundSlotIndex = (_soundSlotIndex + 1) % ARRAYSIZE(_soundHandles);
 	stopSoundHandle(_soundHandles[_soundSlotIndex]);
 
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup sound '%s'", path.c_str());
 		return false;
@@ -340,7 +371,8 @@ bool MediaManager::playSingleSound(const Common::String &path) {
 		return false;
 
 	stopSoundHandle(_singleSoundHandle);
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup sound '%s'", path.c_str());
 		return false;
@@ -364,7 +396,8 @@ bool MediaManager::playSpeech(const Common::String &path) {
 		return false;
 
 	stopSoundHandle(_speechHandle);
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup speech '%s'", path.c_str());
 		return false;
@@ -410,7 +443,7 @@ bool MediaManager::playLoadedSound(int slot) {
 
 	Common::SeekableReadStream *stream = new Common::MemoryReadStream(
 		_loadedSoundData[slot].data(), _loadedSoundData[slot].size(), DisposeAfterUse::NO);
-	Audio::SeekableAudioStream *audioStream = decodeStartupAudioStream(stream);
+	Audio::SeekableAudioStream *audioStream = decodeStartupAudioStream(stream, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup sound slot %d ('%s')",
 			slot, _loadedSoundPaths[slot].c_str());

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 3: File Formats
• Reverse Engineering Harvester with Ghidra and Codex - Part 4: Command Opcodes
• → Reverse Engineering Harvester with Ghidra and Codex - Part 5: Debugging Audio Issues

Article 4 of 6 in this series.

Harvester’s startup / world script is not bytecode. It is XOR-obfuscated text, and opcode dispatch happens through COMMAND records in HARVEST.SCR:

1
COMMAND triggerTag opcodeName arg1 arg2 arg3 [arg4]

In the original game and in ScummVM, these opcode names come from the data pipeline, not from a compiled bytecode table:

• In ScummVM, Script::load() reads HARVEST.SCR, decode() XOR-deobfuscates it, and parseTownRecords() turns it into typed startup records.
• Within parseTownRecords(), COMMAND lines are parsed into StartupCommandRecord entries alongside ROOM, OBJECT, REGION, TIMER, USEITEM, and related world records.
• Those other records provide the entry labels for command chains through room setup/exit, interactions, and timers.
• findCommandRecord() and executeCommandChain() resolve the current label to a COMMAND record, decode the opcode name on that line, and dispatch into engine handlers for room flow, media, inventory, actor state, and other subsystems.

Command Labels

triggerTag

triggerTag is the label attached to one COMMAND record. It is the string used to find that record later.

• The parser stores it from the token immediately after COMMAND in parseTownRecords().
• Script::findCommandRecord() resolves a command by comparing the requested tag string against command.triggerTag.

So triggerTag is not a condition and not an opcode argument in the behavioral sense. It is the command node’s name.

currentTag

currentTag is the interpreter’s working variable while it walks a command chain.

• executeCommandChain() initializes currentTag from the caller-supplied starting tag.
• It then resolves the current command with findCommandRecord(currentTag).
• After each opcode runs, currentTag is updated to the next label:
  • branch opcodes like CHECK_FLAG and CHECK_PERC set it from arg2 or arg3
  • most linear opcodes continue to arg4
  • deferred opcodes may stash arg4 as a continuation tag and return to the caller instead of immediately continuing

If you think of the script as a graph, triggerTag is the node name stored in the file, and currentTag is the interpreter’s current node pointer.

Where Starting Tags Come From

The interpreter does not enter command chains automatically just because a COMMAND record exists. Some other game record must point at its label.

Common entry points in the current engine:

• object interaction uses object.actionTag
• region interaction uses region.actionTag
• use-item interaction uses useItem.actionTag
• room enter / exit uses room->onEnterCommand and room->onExitCommand
• timer execution starts from timer->arg2

That means a more precise reading of the format is:

1
COMMAND label opcodeName arg1 arg2 arg3 [nextLabel]

with the caveat that arg2, arg3, and arg4 are opcode-specific, so only some of them are actually labels for a given opcode.

Examples

Example 1: straight-line chain

1
2
COMMAND "OPEN_GATE" "SET_FLAG" "GATE_OPEN" "T" "" "OPEN_GATE_TEXT"
COMMAND "OPEN_GATE_TEXT" "SHOW_TEXT" "Gate_Is_Open" "" "" ""

If an object’s actionTag is "OPEN_GATE":

1. executeCommandChain() starts with currentTag = "OPEN_GATE".
2. findCommandRecord("OPEN_GATE") resolves the first line because its triggerTag is "OPEN_GATE".
3. SET_FLAG runs and then sets currentTag = arg4, which is "OPEN_GATE_TEXT".
4. findCommandRecord("OPEN_GATE_TEXT") resolves the second line.
5. SHOW_TEXT runs. Because it is deferred, the interpreter returns control to the caller instead of continuing immediately.

Example 2: branch on a flag

1
2
3
COMMAND "TRY_SHED" "CHECK_FLAG" "HAS_SHED_KEY" "SHED_OPEN" "SHED_LOCKED"
COMMAND "SHED_OPEN" "CHANGE_ROOM" "SHED_INT" "" "" ""
COMMAND "SHED_LOCKED" "SHOW_TEXT" "Need_A_Key" "" "" ""

If the chain starts at "TRY_SHED":

1. currentTag starts as "TRY_SHED".
2. CHECK_FLAG looks up HAS_SHED_KEY.
3. If the flag is true, currentTag becomes arg2, so the next lookup is "SHED_OPEN".
4. If the flag is false, currentTag becomes arg3, so the next lookup is "SHED_LOCKED".

So here the first COMMAND line is acting like a named branch node.

Example 3: deferred opcode with continuation

1
2
COMMAND "POTTS_EVENT" "GOFLIC" "GRAPHIC/FST/C001B.FST" "" "" "POTTS_AFTER_MOVIE"
COMMAND "POTTS_AFTER_MOVIE" "SET_FLAG" "STEPH_MIDGAME_PLAYED" "T" "" ""

When currentTag reaches "POTTS_EVENT":

1. GOFLIC does not immediately jump to "POTTS_AFTER_MOVIE".
2. Instead, it stores arg4 as a continuation tag and returns the movie request to the caller.
3. After the cutscene finishes, room/dialogue code can resume by starting another command-chain execution at "POTTS_AFTER_MOVIE".

That is why arg4 is often best read as “the next tag after this opcode completes”, not just “the next line”.

Most of the opcode recognition below lives in Script::executeCommandChain(), while deferred outputs such as modal text, dialogue continuations, lighting changes, player moves, and follow-up tags are consumed by the room interaction processor in room.cpp.

Control Flow And Transitions

World And Runtime State

Player And UI

Audio

The audio opcodes all share the same queueing path through appendStartupAudioCommand() and are later handed off to Flow::executeStartupAudioCommands().

Observed Engine-Side Aliases And Shared Paths

• HEAL_PC and ADJ_HP currently share the exact same implementation.
• KILL_NPC and MONSTERFY share the same base handler; MONSTERFY additionally activates the linked monster target when present.
• CLOSEUP and CHANGE_ROOM share the same transition-output path, differing only in the transition kind reported to room logic.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 2
• Reverse Engineering Harvester with Ghidra and Codex - Part 3: File Formats
• → Reverse Engineering Harvester with Ghidra and Codex - Part 4: Command Opcodes

Article 3 of 6 in this series.

File Formats

This document tracks file-format facts that are confirmed by the current Harvester reverse-engineering work. It is intentionally conservative: if a field or behavior is not supported by code or verified runtime notes, it is left out.

The sections below combine three sources of evidence: the current ScummVM Harvester engine code, named native functions and data types in Ghidra, and spot-checks against extracted sample files from the Harvester CD image. Unknown or only partially understood fields stay marked as such.

DAT (XFILE archive payload)

Harvester’s .DAT files act as payload containers for the game’s numbered XFILE resource sets. The important wrinkle is that the .DAT file is not the whole format by itself: the loader depends on a sidecar INDEX.00N file for the directory. In other words, the .DAT holds the bytes, while the INDEX.00N records tell the engine where each member starts, how large it is, and whether it must be unpacked after reading.

At confirmed cold start, the original game mounts these pairs in this order:

Summary

.DAT payload layout

This is sparse compared with many archive formats. The format’s structure is directory-driven rather than self-describing. All meaningful per-file metadata currently comes from the matching INDEX.00N file.

INDEX.00N directory record layout

Each confirmed sidecar index is read as a flat array of 0x94-byte records:

Packed entry stream

When packed_flag != 0, the loader reads stored_size bytes from the .DAT and expands them into an unpacked_size output buffer using a simple control-byte stream:

The decode loop stops once either the compressed input is exhausted or the output buffer reaches unpacked_size. Entries that fail to produce exactly unpacked_size bytes are treated as invalid.

Path handling notes

Practical interpretation

For extraction or tooling work, the current evidence supports treating Harvester’s .DAT archives as a two-file format:

1. Read the matching INDEX.00N file as a sequence of 0x94-byte XFLE records.
2. Use each record’s archive_offset and stored_size to slice bytes from the companion .DAT.
3. If packed_flag is zero, the slice is the final file.
4. If packed_flag is nonzero, expand the slice with the literal/repeat decoder above until unpacked_size bytes are produced.

That model is enough to explain the startup resource mounts already confirmed in the reverse-engineering notes, and it matches the current ScummVM-side archive loader.

RCS (Quick-tip text lists)

The only confirmed .RCS use so far is ADJHEAD.RCS, the quick-tips file shown by the startup/options overlay. Both native run_quick_tips_screen and the ScummVM Flow::loadQuickTips path treat it as plain text rather than as a binary container.

Summary

File layout

Notes

• The sampled ADJHEAD.RCS begins directly with readable text: Double click the left mouse button....
• The native quick-tips path advances through the file with read_line_from_file_stream, wraps back to the start on EOF, and chooses a random starting point the first time it runs.
• This is a good example of Harvester’s resource pipeline staying pragmatic: not every gameplay-facing asset is wrapped in a custom binary format.

DIALOG.RSP (Dialogue response and keyword text)

DIALOG.RSP is a plaintext line table used by the dialogue UI. It does not appear to contain ids, offsets, or a binary header; the engine indexes it by zero-based line number and then uses the line text directly.

Summary

File layout

Notes

• The native loader reads a requested zero-based line, strips the trailing CR/LF, and returns the shared text buffer directly.
• run_dialogue_response_menu and run_dialogue_keyword_menu split visible options on /; there is no confirmed hidden topic-id layer behind those labels.
• load_dialogue_index also pulls zero-based line 13 from DIALOG.RSP to seed the default keyword topic; in the sampled file that line is BYE (one-based line 14).
• DIALOG.RSP shows Harvester leaning on designer-editable text tables even inside a fairly custom dialogue stack.

DIALOGUE.IDX (Subtitle index)

DIALOGUE.IDX is an XOR-obfuscated text file that maps voice sample ids to subtitle strings. The native load_dialogue_index path and the ScummVM Text::loadDialogueIndex reimplementation both decode it with XOR 0xAA while leaving CR/LF intact.

Summary

Decoded stream layout

Derived index entry

Notes

• After XOR decode, the sampled file starts with a simple alternating pattern: 1, "junk init", 7, "Yes?", 11, "I need some help...", and so on.
• The native loader builds a 3000-entry in-memory table and keeps the decoded text blob around so play_dialogue_line can seek back into it cheaply.
• Architecturally, this splits dialogue cleanly in two: DIALOGUE.IDX carries spoken-line subtitles keyed by numeric wav ids, while DIALOG.RSP carries menu-facing response text.

1
2
3
4
5
6
7
8
9
10
11
python3 -c 'import sys,pathlib,signal; signal.signal(signal.SIGPIPE, signal.SIG_DFL); d=pathlib.Path(sys.argv[1]).read_bytes(); sys.stdout.buffer.write(bytes(b if b in (10,13) else b ^ 0xAA for b in d))' DIALOGUE.IDX | head -n 10
1
"junk init"
7
"Yes?"
11
"I need some help... Mister...?"
15
"Postmaster Boyle.  What can I do you for today?"
25
"Sorry, youngster, we're out of applications right now."

SCR (Town/world startup script)

HARVEST.SCR is the central world-definition and startup-script file. It is XOR-obfuscated text, not bytecode. The native load_xor_obfuscated_town_script path and the ScummVM Script::decode path both XOR each non-CR/LF byte with 0xAA, then parse whitespace-separated records with quoted strings preserved.

Summary

Confirmed record forms

Notes

• The sampled file starts with readable structural records immediately after XOR decode, for example ENTRANCE, OBJECT, and later ROOM, TEXT, HEAD, and COMMAND.
• Paths embedded in script records are not uniform: archive-backed resources use 1:\..., 2:\..., 3:\..., while some direct-file assets are bare relative paths such as dialogue.idx.
• GOFLIC and GODEATHFLIC are especially revealing command names. In the sampled script, those opcodes point to .FST paths, which suggests the script vocabulary preserved older naming while the shipping runtime movie path used FST files.
• HARVEST.SCR shows that Harvester’s resource architecture is data-driven at the top level: rooms, objects, dialogue portraits, commands, music, palettes, and cutscene triggers all converge here.

1
2
3
4
5
6
7
8
9
10
11
python3 -c 'import sys,pathlib,signal; signal.signal(signal.SIGPIPE, signal.SIG_DFL); d=pathlib.Path(sys.argv[1]).read_bytes(); sys.stdout.buffer.write(bytes(b if b in (10,13) else b ^ 0xAA for b in d))' HARVEST.SCR | head -n 10
{// HARVESTER (c) 1995-96 Scripting Language
{// Town
        0   0   0   ENTRANCE "BACK" "TOWN_2_LODGE" "SERGEANT_2_LODGE"
        0   0   0   ENTRANCE "FRONT" "" "SAVE_GAME"
        492 67  554 96   0  1 OBJECT "INVENTORY" "INV_EXIT" "" "" "" "" "" "X" "F" "T" "" "Inventory"
        299 0   0   0    2  1 OBJECT "NULL_ID"          "EXIT_BM"  "1:\GRAPHIC\OTHER\EXITSIGN.BM" "" "" "" ""  "" "F" "T" "" "exit"
        299 0   375 61   0  1 OBJECT "NULL_ID"          "EXIT_HS"  ""                             "" "" "" "X" "" "T" "T" "" "exit"

{// inventory health indicator
     72 314 0   0  -12  1 OBJECT "INVENTORY" "INV_STAT1" "1:\graphic\other\head-a1.bm" "" "" "INV_STAT_ST" "" "" "F" "F" "" "Health_Indicator"

BM (Static indexed bitmap)

.BM is Harvester’s simplest custom image format: a small fixed header followed by raw 8-bit indexed pixels. It is used for UI panels, portraits, inventory art, help screens, and other non-animated images.

Summary

File layout

Notes

• The native binary reuses the same RawBitmap/BitmapBuffer shape in memory: { width, height, pixels }.
• The format itself does not encode transparency, but many callers render it with palette index 0 treated as transparent.
• Compared with many later adventure engines, this is an aggressively direct format: no chunking, no palette sidecar inside the file, and no per-row metadata.

PAL (Standalone palette)

Harvester’s .PAL files are raw palette payloads with no header. The native palette upload helpers and ScummVM Art::loadPalette both treat them as 256 RGB triplets.

Summary

File layout

Notes

• Native upload_palette_to_vga forces palette index 0 to black at upload time, applies brightness scaling, and then shifts the stored 8-bit channels down to VGA’s 0..63 DAC range.
• The sampled INVHELP.PAL contains full-range byte values up to 0xff, which matches the native analysis that .PAL is stored as 8-bit RGB, not pre-divided 6-bit VGA values.
• This makes .PAL a nice contrast with FST’s embedded movie palettes, which are stored in 6-bit VGA form inside each frame payload.

TRY IT OUT!

1
2
python3 -m pip install pillow
python3 -c 'from PIL import Image; import sys,struct; d=open(sys.argv[1],"rb").read(); w,h=struct.unpack_from("<II",d,0); i=Image.frombytes("P",(w,h),d[12:12+w*h]); p=open(sys.argv[2],"rb").read()[:768] if len(sys.argv)>2 else bytes(c for n in range(256) for c in (n,n,n)); i.putpalette(p); i.show()' /path/to/*.BM [/path/to/*.PAL]

python3 -c '...' GRAPHIC/OTHER/INVHELP.BM

python3 -c '...' GRAPHIC/OTHER/INVHELP.BM GRAPHIC/PAL/INVHELP.PAL

CFT (Bitmap font)

.CFT packages a bitmap font as a fixed metrics header plus one raw 8-bit atlas image. The native load_font_resource and the ScummVM Text::loadFont / HarvesterCftFont code agree on the basic structure.

Summary

File layout

Notes

• The renderer derives each glyph by slicing width pixels from x inside the shared atlas. There is no per-glyph bitmap chunking.
• drawHeight is effectively atlasHeight - 1 in the current font renderer, which matches the native behavior of treating the last row as non-drawing padding.
• The font renderer treats both ' ' and '_' as space-width characters. That ties neatly back to the script/text resources, where underscore-heavy identifiers and UI labels coexist with visible text.

TRY IT OUT!

1
python3 -c 'from PIL import Image; import sys,struct; d=open(sys.argv[1],"rb").read(); aw,ah=struct.unpack_from("<II",d,0x448); h=max(1,min(struct.unpack_from("<H",d,0x40)[0] or ah-1,ah-1)); sw=struct.unpack_from("<H",d,0x442)[0] or 1; s=[struct.unpack_from("<H",d,0x42+i*2)[0] for i in range(256)]; w=[struct.unpack_from("<H",d,0x242+i*2)[0] for i in range(256)]; a=Image.frombytes("L",(aw,ah),d[0x454:0x454+aw*ah]); t=sys.argv[2]; W=sum(sw if ord(c)>255 or c in " _" or w[ord(c)]<=0 else w[ord(c)] for c in t); i=Image.new("L",(max(1,W),h),0); x=0; exec("for c in t:\n o=ord(c)\n gw=sw if o>255 or c in \" _\" or w[o]<=0 or s[o]>=aw else w[o]\n if gw and o<=255 and c not in \" _\" and w[o]>0 and s[o]<aw: i.paste(255,(x,0,x+gw,h),a.crop((s[o],0,s[o]+gw,h)))\n x+=gw"); i.resize((max(1,i.width*4),max(1,i.height*4)),Image.NEAREST).show()' "GRAPHIC/FONT/HARVFNT2.CFT" "Hello World"

ABM (Animated bitmap / sprite strip)

.ABM is Harvester’s main custom sprite/animation format. It backs cursor art, actor sprites, room animations, combat entities, and wait-overlay animation frames.

Summary

File header

Per-frame record

Compressed frame stream

Notes

• Each frame decodes to exactly width * height bytes of indexed pixels.
• The sampled BLOOD.ABM starts with 5 frames; its first frame is offset (4, 0), size 38 x 50, and marked compressed.

Try it out!

1
python3 -c $'from PIL import Image\nimport sys,struct,tempfile,pathlib,webbrowser\n\ndef dec(s,n):\n i=0; o=bytearray()\n while i<len(s) and len(o)<n:\n  k=s[i]; i+=1\n  if k<128:\n   c=min(k,len(s)-i,n-len(o)); o+=s[i:i+c]; i+=c\n  elif i<len(s):\n   o.extend([s[i]]*min(k&127,n-len(o))); i+=1\n return bytes(o)\n\nd=open(sys.argv[1],"rb").read()\npal=open(sys.argv[2],"rb").read()[:768] if len(sys.argv)>2 else bytes(c for n in range(256) for c in (n,n,n))\nfc=struct.unpack_from("<I",d,0)[0]; off=8; F=[]; minx=miny=10**9; maxx=maxy=-10**9\nfor _ in range(fc):\n x,y,w,h=struct.unpack_from("<iiii",d,off); c=d[off+16]; n=struct.unpack_from("<I",d,off+17)[0]; s=d[off+25:off+25+n]; off+=25+n; p=s[:w*h] if not c else dec(s,w*h); F.append((x,y,w,h,p)); minx=min(minx,x); miny=min(miny,y); maxx=max(maxx,x+w); maxy=max(maxy,y+h)\nminx=min(0,minx); miny=min(0,miny); W=maxx-minx; H=maxy-miny; G=[]\nfor x,y,w,h,p in F:\n src=Image.frombytes("P",(w,h),p); src.putpalette(pal); m=src.point(lambda v:0 if v==0 else 255,"L"); fr=Image.new("RGBA",(W,H),(0,0,0,0)); fr.paste(src.convert("RGBA"),(x-minx,y-miny),m); G.append(fr)\nf=tempfile.NamedTemporaryFile(suffix=".gif",delete=False).name; G[0].save(f,save_all=True,append_images=G[1:],duration=100,loop=0,disposal=2,transparency=0); webbrowser.open(pathlib.Path(f).as_uri())' /path/to/.ABM [/path/to/.PAL]

For example:
python3 -c $'...' "GRAPHIC/ROOMANIM/CLOAK.ABM"

python3 -c $'...' "GRAPHIC/ROOMANIM/CLOAK.ABM" "GRAPHIC/PAL/INVHELP.PAL"

CMP (FutureVision compressed audio)

See FutureVision audio formats as well.

.CMP is Harvester’s custom compressed audio wrapper. The sampled files begin with FCMP, and the native load_sound_sample / load_dialogue_voice_sample code plus the ScummVM decodeHarvesterFcmp path all treat that payload as IMA-ADPCM-like compressed audio.

Summary

File layout

Notes

• Sampled MENACE.CMP starts with FCMP, a payload size of 0x0db48b, sample rate 22050, and 16 bits per sample.
• The loader family is tolerant: some call paths accept either FCMP or raw WAVE data. The .CMP extension itself points to the compressed path, but the runtime checks the actual file signature before decoding.
• From an architectural perspective, this is the audio-side equivalent of Harvester’s XFILE abstraction: one wrapper format reused across music, speech, and effects.

FST (Cutscene / streamed animation)

.FST is Harvester’s custom streamed movie format. It combines a file header, a compact per-frame index table, block-coded video payloads, optional per-frame palettes, and per-frame audio chunks.

Summary

File header

Frame index entry

Video payload for one frame

Block coding

Notes

• Sampled VIRGLOGO.FST is 320 x 200, stores 131 frames, plays at 15 fps, and carries 22050 Hz / 16-bit audio.
• The native and ScummVM players both treat FST as a streaming format: read one frame, queue its audio, decode its video, and advance without loading the whole movie at once.
• HARVEST.SCR uses GOFLIC/GODEATHFLIC opcodes to trigger .FST files. That mismatch between opcode names and file extension is a strong architectural clue that FST replaced or wrapped an older movie concept without rewriting the script vocabulary.
• The censorship path also shows how self-contained the format is: FST can carry its own palette updates frame-by-frame, while the player temporarily swaps in an external CENSORED.PCX overlay when gore is disabled.

PCX (Standard indexed still image)

Harvester does use standard .PCX files in at least one confirmed place: the censorship overlay shown during certain FST sequences when gore is disabled.

Summary

Confirmed header fields

Notes

• Sampled CENSORED.PCX is a textbook single-plane 320 x 200 PCX, which is why the ScummVM port can use the generic PCX decoder for it.
• The native loader contains two Harvester-specific wrinkles: it trims the optional 0x0c palette marker by jumping straight to the last 0x300 bytes, and when the logical width from xMax is one pixel smaller than bytesPerLine, it trims the padded stride byte after decode.

FLC (Standard Autodesk FLIC animation)

Harvester does ship .FLC files, and the sampled files match the standard Autodesk FLIC/FLC header rather than a Harvester-specific wrapper. The current Harvester engine work has not yet recovered the exact native Harvester-side call path that consumes them, so this section stays limited to what is directly supported by sample bytes, binary strings, and the generic ScummVM FLIC decoder.

Summary

Confirmed standard header fields

Notes

• Sampled HARVPNTR.FLC starts with a valid FLC header: size 0x214a, magic 0xaf12, 10 frames, 26 x 26, and 8-bit color.
• The decoded HARVEST.SCR sample does not reference .FLC files directly; its GOFLIC opcodes currently point to .FST files instead. That suggests .FLC belongs to a parallel or older asset path rather than the main shipping story-transition pipeline. This last point is an inference from the sampled script and should stay provisional.
• Taken together, the resource set shows three animation strata: lightweight sprite ABMs, streamed FST movies, and a residual standard FLC layer. That is a strong narrative hook for explaining how mixed-tool resource pipelines often survive into shipped games.

TRY IT OUT

1
2
ffplay -loop -1 "GRAPHIC/ROOMANIM/HARVPNTR.FLC"
ffmpeg -i "GRAPHIC/ROOMANIM/HARVPNTR.FLC" out.gif

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering a DOS Game with Ghidra and Codex
• Reverse Engineering Harvester with Ghidra and Codex - Part 2
• → Reverse Engineering Harvester with Ghidra and Codex - Part 3: File Formats

Article 2 of 6 in this series.

According to the Harvester manual, your health is represented by a picture of Steve in the inventory that gets bloodier the more damage you take. Since we’re using a Ghidra disassembly and Codex to build out our ScummVM engine, let’s have a look at how easy adding this functionality can be.

First let’s confirm that our implementation doesn’t support this functionality just yet. This is easy enough to do as we’ve already started working on the inventory system, so clicking on Steve should bring up the expected view - missing photo and all.

Comparing this with the original via DOSBox (left) and our current engine (right), we not only can clearly see what the expected outcome should be. The prompt we’re going to use is as follows:

when you click on the inventory view, Steve has his health tracked in the photo in the bottom left. Check back in Ghidra to trace health management, and if there are any unnamed functions or data structures that apply, rename them once you have high confidence. Implement these changes engine-side and commit changes when ready.

Just like magic, after about 13 minutes of processing Codex will revisit our existing disassembly in Ghidra, clarify any gaps there first, carry that logic back to Visual Studio Code and propose engine changes to implement the requested functionality.

The first pass is honestly pretty impressive, as other than a visual anomaly in the form or a black box seemingly being rendered out of place, the health indicator is working already!

Since we’re taking side-by-side screenshots of a reference implementation (the original running in DOSBox) to compare our re-implementation with, we can actually attach that screenshot as context to steer our next prompt:

you can see from the attached screenshot of dosbox (left) and our engine (right) that the health indicator photo is correctly placed, but there’s a black box under it that appears to be either an incorrect transparency or a misrotated container element. Revist Ghidra for inventory and health drawing and determine how to fix this

Codex is making a number of tool calls, so I’ve got rtk installed to compress some of the output to extend my context window slightly. It only takes a couple minutes, but the model is able to determine the cause of the issue based on the state of our reversed engineered executable in Ghidra.

Having invested the time to map out the data structures and function calls in Ghidra really makes it a lot more efficient to work with it to reimplement game logic, and the Ghidra MCP bridge allows us to keep doing this quickly and effectively in Visual Studio Code.

What this practically translates to is a fairly good implementation (see full commit).

Ten years ago while I was still muddling my way through asylum engine updates in IDA Pro, making this type of progress would have taken me weeks. Now, in about 30 minutes I can achieve exponentially more!

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• Reverse Engineering a DOS Game with Ghidra and Codex
• → Reverse Engineering Harvester with Ghidra and Codex - Part 2

Article 1 of 6 in this series.

As part of my series called “Let’s Adventure!” I had a chance to revisit a lot of classic adventure games I loved growing up, but also experience some I’d missed or never heard of. One of those games was Harvester, which was just the right mix of weird characters and settings with a bizarre plot tieing everything together.

Though I wouldn’t say the game was great, it was interesting enough to keep me engaged and even after playing through it still remains top of mind.

With the advent of agentic development tools like Claude Code or Codex, I figured I’d take a crack at reverse engineering Harvester with the goal of eventually reimplementing it as a game engine in ScummVM.

Tools

• Visual Studio Code
• Codex (used via Visual Studio Code)
• Ghidra
• Ghidra MCP
• Ghidra Loader for the LX/LE executable file format

Getting Started

The first step is almost always to just load the EXE into Ghidra and kicking off an initial analysis. Once the analysis is complete, I like to get a sense of what types of string constants are exposed that I may be able to anchor some logic to by inspecting. the Defined Strings.

What I’m looking for on a first pass is:

• resource filenames as constants (ex: HARVEST.DAT) as these can help point to where file I/O routines are
• error messages
• inventory item labels
• dialogue text
• names that might identify internal data structures
• character names

Since I didn’t see any of these, I tried to decompress the executable in DOSBox using UNP.EXE. This didn’t do anything as compression wasn’t what was obfuscating the strings, but what I could see looked like memory management details - which lead me to believe the game was using a DOS extender like DOS/4GW.

DOS/4GW was the Watcom-packaged subset of Tenberry’s DOS/4G: a 32-bit DOS extender customized for the Watcom toolchain. Its job was to let a DOS program run in 386 protected mode with a flat, zero-based memory model, so developers could stop fighting 16-bit segmented memory and work with much larger address spaces. Just as importantly for games, DOS/4GW still mapped the first megabyte of physical memory into a shared linear address space, which meant code could still reach things like video RAM and BIOS data when needed.

Unbinding the Extended EXE

To be able to make any meaningful progress, we need Ghidra to be able to analyze teh un-exetended EXE. After looking around I found a blog describing how to “Disassemble DOS/4GW” which hinted at being able to use the SUNSYS Bind Utility to extract a Linear Executable from the extended EXE.

You can find a copy of the bind utility (SB.EXE) with the open source DOS32A DOS extender. Once downloaded, you can use this directly via DOSBox to unbind the EXE.

When running this against HARVEST.EXE it will produce a HARVEST.LE, which is the linear executable. Once we have this the next logical step would be to try and load it in Ghidra, but doing this doesn’t quite work as Ghidra thinks the LE file is raw data.

Decompiling a Linear Executable

To successfully start analyzing HARVEST.LE, we’ll first need to install the Ultimate Ghidra Loader for the LX/LE executable file format extension.

After doing this and restarting Ghidra, the LE EXE format should be properly detected, and we can load this and begin our analysis. Since the goal is to develop a working ScummVM game engine, we’ll be working on a harvester branch of a ScummVM fork.

Working with Ghidra via MCP

The HARVEST.LE executable has now been analyzed and we have a much more informative listing of defined strings to anchor our reverse engineering efforts on. Since we’ll be using Codex directly via Visual Studio Code, the next step is to setup Ghidra and Ghidra MCP.

This is fairly straightforward:

• Download Ghidra and unzip
• In vscode install the openai.chatgpt extension
• Open the Codex panel and just start chatting:

  1
  2
  3
    I've installed Ghidra at /path/to/extracted/ghidra and I want to work with it using 
    MCP. Download GhidraMCP from https://github.com/bethington/ghidra-mcp and install 
    it. Verify this is working once complete.
  

  

With GhidraMCP installed and working, we can dig into our initial reverse engineering attempt. I started off by developing an AGENTS.md I could use specifically for this initiative.

To make it easier to port logic from the original game EXE to a ScummVM engine we’ll need to identify functions, data structures, subsystems, state management, resource handling and many other components that made up the game - so we kick things off with:

1
2
3
4
5
Read the AGENTS.md file, start at the application entry point in Ghidra and begin 
scanning the available functions (FUN_*). Based on the xrefs to those functions, 
as well as how defined strings are being used and where DOS interrupts are being 
invoked, when you have high confidence in a function's purpose, rename it with a 
meaningful name.

The AGENTS.md file outlines operating parameters such as keeping track of our findings in an ARCHITECTURE.md file and ongoing progress in a TRACKER.md file. As we progress with subsequent prompts, more of the linear executable in Ghidra will be decompiled, making it easier to work with as a source of truth for our re-implementation efforts.

Using Ghidra via the Ghidra MCP bridge makes it very easy to incrementally analyze the game executable and start piecing together how it works.

Scaffolding a Game Engine

Once we’ve made sufficient progress unwinding some of the original main game loop and resource handling, we can start wiring up an engine using additional prompts. The ARCHITECTURE.md we’re evolving along with direct access to Ghidra via MCP can be used to first:

• create a skeletal engine (based on ScummVM’s HOWTO-Engines guidance)
• add basic resource handling
• add basic audio/video decoding
• play our first video resources

The Cutting Room Floor has an entry on Harvester, as well as some additional notes that can be shared with Codex to inform file format disassembly and re-implementation.

I was actually pleasantly surprised with how easy it was to make progress using this configuration, as targeted prompts more often than not produced positive, actionable results. Within a day of starting this project I was able to get the intro videos playing, the first scene to render and the background music and sound effects working.

Plenty more to do (such as fixing the palette), the main thing to highlight here is that understanding assembly language doesn’t need to be a barrier to undertaking these types of reverse engineering initiatives anymore.

1
Error: Dynamic require of "punycode/" is not supported

The TL;DR is there is an issue with how @cloudflare/vite-plugin is processing an import with a trailing slash within the tr46 library, which is a transitive dependency of the MongoDB Node.js driver. The current solution is to patch this out until a proper fix is in place.

Reproduction

Let’s begin with a new application we can use as a minimum reproduction. Chances are you’ve already got an application ready that’s hitting this issue, but if not we can verify this behavior by simply creating a new React Router app using create-cloudflare as follows, then adding the MongoDB Node.js driver as a dependency and importing it.

1
2
3
4
5
6
7
8
9
10
# create a new react router app
npm create cloudflare@latest -- my-react-router-app --framework=react-router
cd my-react-router-app
# install mongodb
npm install mongodb --save
# prepend an import to the workers/app.ts file
printf 'import { MongoClient } from "mongodb";\n%s' "$(cat workers/app.ts)" > workers/app.ts
# update wrangler.jsonc with compatibility flags to support SSR
sed -i '' '/"compatibility_date": "2025-04-04"/a\
  "compatibility_flags": ["nodejs_compat"],' wrangler.jsonc

With a freshly bootstrapped application, let’s try running it to see what happens.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
npm run dev
> dev
> react-router dev

11:15:07 AM [vite] (ssr) Re-optimizing dependencies because vite config has changed
11:15:08 AM [vite] (ssr) ✨ new dependencies optimized: mongodb
11:15:08 AM [vite] (ssr) ✨ optimized dependencies changed. reloading
[vite] program reload
Error: Dynamic require of "punycode/" is not supported
    at null.<anonymous> (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:11:9)
    at node_modules/tr46/index.js (/Users/alex/Temp/my-react-router-app/node_modules/tr46/index.js:3:18)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/lib/url-state-machine.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/lib/url-state-machine.js:2:14)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/lib/URL-impl.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/lib/URL-impl.js:2:13)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/lib/URL.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/lib/URL.js:499:14)
    at __require2 (/Users/alex/Temp/my-react-router-app/node_modules/.vite/deps_ssr/chunk-PLDDJCW6.js:17:50)
    at node_modules/whatwg-url/webidl2js-wrapper.js (/Users/alex/Temp/my-react-router-app/node_modules/whatwg-url/webidl2js-wrapper.js:3:13) {
  [cause]: undefined
}

Vite is complaining that the dynamic require of “punycode/” is not supported. The trailing slash following “punycode” is interesting, but we should first see where it’s being imported. We can do this by using npm ls to quickly narrow down usage of punycode to the tr46 library:

1
2
3
4
5
6
7
npm ls punycode
my-react-router-app@ /Users/alex/Temp/my-react-router-app
└─┬ mongodb@7.0.0
  └─┬ mongodb-connection-string-url@7.0.0
    └─┬ whatwg-url@14.2.0
      └─┬ tr46@5.1.1
        └── punycode@2.3.1

Inspecting the tr46 library at https://github.com/jsdom/tr46/blob/main/index.js shows the trailing slash on the import as well:

1
2
3
4
5
6
7
"use strict";

const punycode = require("punycode/"); // <--- this is the line in question
const regexes = require("./lib/regexes.js");
const mappingTable = require("./lib/mappingTable.json");
const { STATUS_MAPPING } = require("./lib/statusMapping.js");
// ...

I initially tried to open a PR at https://github.com/jsdom/tr46/pull/73 to sort this out, but the maintainer points out that the issue is with Vite, so we’ll need to look elsewhere for a solution. This change (introduced in commit fef6e95) was likely done to address punycode deprecation warnings such as that described in https://github.com/jsdom/tr46/issues/63. For more info on those deprecations see “Solving the "Punycode Module is Deprecated" Issue in Node.js”.

Patching

We’re going to solve this issue in a roundabout fashion using patch-package to modify the punycode import directly in our node_packages and then have a postinstall script that will ensure the patch is consistently applied.

1
2
3
4
5
6
7
8
9
10
# install patch-package
npm install patch-package
# update package.json to run patch-package as well as cf-typegen (which is there by default)
npm pkg set scripts.postinstall="patch-package && npm run cf-typegen"
# update node_modules/tr46/index.js to remove the trailing slash from the import
sed -i '' 's/require("punycode\/")/require("punycode")/g' node_modules/tr46/index.js
# create a patch for the tr46 package based on the above change
npx patch-package tr46
# reinstall and apply patches
npm install

That should do it! When we run npm install it will also run the postinstall, which will apply the patch we just created.

Summary

Though patching transient dependencies to work around an issue like this is not ideal, it does offer a path forward for anyone hitting this specific error. To summarize what we did to address the issue:

1. Install the patch-package library (npm install patch-package)
2. Update your package.json’s scripts.postinstall to prepend a patch-package script to any postinstall scripts that may already be present
3. Modify node_modules/tr46/index.js to remove the trailing / from require("punycode/")
4. Create the patch by running npx patch-package tr46
5. Ensure the patch is applied by running npm install

Hopefully we can get this sorted out more cleanly (reported as https://github.com/cloudflare/workers-sdk/issues/11751), but in the meantime feel free to use this approach if you find it suitable.