They are not.

A RIFF AVI file begins with a RIFF header; these files begin with IAVF2.00.

The extension describes the role of the file, not its format. What looks like a video file is actually a packet stream containing audio data, video setup records, compressed video frames, presentation coordinates, timing gates, and display commands.

Getting these files to play correctly required more than identifying a codec. I had to recover the container’s packet language, determine which clock controlled presentation timing, reconstruct standard Smacker streams from fragmented payloads, and reproduce display operations that occurred between segments.

The result is an engine-local demultiplexer that translates the IAVF format into services ScummVM already provides. The engine’s media format detector now makes the initial routing decision from the stream signature, distinguishing IAVF2.00 from SMK2 and SMK4 without trusting the filename extension.

Prior work provided the map

I did not arrive at the IAVF model entirely from scratch. Three earlier investigations supplied important starting points.

Kostya’s “Ripper: Smackered AVIs” first highlighted that RIPPER’s .AVI files are not Microsoft AVI containers. The post identified the stream as a sequence of commands with 14-byte headers and observed that one command carries a complete Smacker header. That suggested the central model used here: IAVF does not merely contain a movie; it contains instructions for constructing and playing one.

IAVFExtract turned that observation into a practical extractor capable of producing Smacker video and PCM audio from IAVF assets. It provided a useful executable reference for the container layout, demonstrated that a single IAVF file can contain multiple media segments, and confirmed that the embedded streams can be reconstructed into conventional .SMK and .WAV files.

The MultimediaWiki IAVF page provided a third map. It documented the 145-byte file header, the 14-byte command descriptors, and a broader opcode inventory that included both FLIC and Smacker paths. Several fields and commands were deliberately left tentative. That made the page useful not only as a reference, but also as a list of questions to take back to the executable.

Just as importantly, IAVFExtract documented an unresolved problem: extracted video could drift out of sync with its audio even when the expected frames appeared to be present. That observation helped focus the reverse-engineering work on timing rather than decoding. The Ghidra analysis described below builds on that question, recovering the packet scheduler, the distinct timing semantics of audio commands 0x66 and 0x67, and the use of audio playback as the master clock.

These projects provided the conceptual model and a practical proof of extraction. The ScummVM implementation then independently checked the runtime behavior against RIPPER.EXE and the original assets, filling in the scheduling, silence-padding, display-reset, palette, and scene-restoration behavior needed for in-engine playback.

Finding the player in Ghidra

The main packet processing function in RIPPER.EXE is now identified in Ghidra as:

1
RunPacketizedMediaPlaybackCore @ 0x5b592

Its callers eventually lead back to the game’s higher-level presentation wrapper:

1
RunMediaPresentation @ 0x168af

The core function opens a packet stream, validates an IAVF header with version 2.00, configures audio and display state, then enters an opcode dispatch loop.

The decompiled function is large because it supports more than one presentation path. Its branches include audio submission, audio control, display resets, FLIC setup and decoding, custom packet setup, palette processing, queued payloads, and stream prebuffering.

That was an important constraint on the re-implementation. The goal was not to declare that every IAVF command had been understood. It was to isolate the commands exercised by the files needed for the current game path and give those commands the narrowest meanings supported by the executable and the assets.

IAVF is a packet program

After the initial header, the observed stream is made up of 14-byte descriptors:

1
2
3
4
5
6
struct IavfDescriptor {
	uint16 opcode;
	uint32 arg0;
	uint32 arg1;
	uint32 arg2;
};

Some descriptors are followed by a variable-sized payload. The three arguments change meaning according to the opcode.

For the Smacker-backed files currently handled by the engine, the useful subset looks like this:

This is not intended to be a complete public specification for IAVF. It is the behavior confirmed for the assets exercised by the current ScummVM implementation.

The parser first reads a 0x91-byte header. An exhaustive scan of the 475 IAVF files in the retail data set clarified several fields:

1
2
3
4
5
6
7
8
0x10  number of opcode 0x67 playback gates
0x1c  sample rate
0x1e  channel count
0x1f  bits per sample
0x20  audio bytes per second
0x24  audio block alignment
0x2f  presentation height
0x31  presentation width

The value at 0x10 is especially easy to misread as a video frame count. It exactly matches the number of 0x67 commands in every file in the corpus, including gates used around setup and shutdown rather than frame rendering. For example, LOGO.AVI declares 378 gates but contains 369 rendered frames.

The corpus also confirmed that the dimension order is height followed by width. PROINT.AVI stores 200 at 0x2f and 320 at 0x31. One field remains a useful warning against overfitting: the 32-bit value at 0x14 is zero in 474 files but contains 22050 in PROINT.AVI. Its purpose is still unresolved.

The 16-bit value at 0x2d also looks like a nominal frame rate because the observed values are 10, 20, and 30. The original player does not use it to schedule video, however. Ghidra shows it being passed into InitializePacketizedManagedAudioSession as audio-session context, while the no-audio pacing path reads its rate from each 0x67 command. The field may still describe a nominal media rate, but it is not the clock that controls normal playback.

The startup movies use mono, 16-bit PCM. Their video data is more interesting because a complete Smacker file does not appear contiguously anywhere in the container.

The executable contains additional branches for FLIC setup at 0x69, custom packet loading at 0x6b, and FLIC frame decoding at 0x76. None of the 475 files in this data set uses those commands. They are confirmed capabilities of the original player, but not yet justified implementation targets for the ScummVM engine.

There is also data after the logical end of many files. All 475 files in the corpus contain 0x70, which stops the original dispatch loop. In 425 files it is followed by a 0x79 record and a variable-sized payload; the other 50 end immediately. Ghidra confirms that 0x79 is never dispatched because packet processing has already stopped. In every observed trailer, arg1 is the remaining payload size and arg2 is arg1 - arg0, but the payload’s purpose remains unknown.

Rebuilding Smacker instead of replacing it

Smacker is already supported by ScummVM through Video::SmackerDecoder. Writing another Smacker decoder inside the RIPPER engine would duplicate mature shared code and create another implementation to maintain.

The better boundary was to reconstruct the stream expected by the existing decoder.

An IAVF segment contains enough information to do that:

1
2
3
4
5
6
7
8
9
10
struct IavfSegment {
	Common::Array<byte> setup;
	Common::Array<uint32> frameSizes;
	Common::Array<Common::Array<byte> > framePayloads;
	Common::Array<uint32> frameAudioOffsets;
	uint32 expectedFrames;
	int x;
	int y;
	bool clearDisplayBefore;
};

The setup data begins with SMK2 and contains the normal 104-byte Smacker header, frame-type information, and Huffman tree data. IAVF stores the frame sizes and compressed frame bodies separately.

The engine rebuilds a conventional stream in this order:

1
2
3
4
5
6
7
Smacker header
frame-size table
frame-type and tree data
compressed frame 0
compressed frame 1
...
compressed frame N

That reconstructed stream is placed in a Common::MemoryReadStream and handed through the RIPPER media player to Video::SmackerDecoder.

Conceptually, the adapter is small:

1
2
3
4
5
6
7
8
9
10
Common::SeekableReadStream *smacker = rebuildSmackerStream(segment);

Video::SmackerDecoder decoder;
decoder.loadStream(smacker);
decoder.start();

while (!decoder.endOfVideo()) {
	const Graphics::Surface *frame = decoder.decodeNextFrame();
	// Present the decoded frame using RIPPER's position and timing rules.
}

Most of the difficult work stays outside the codec. The RIPPER engine is responsible for determining when a frame is due, where it belongs on the display, which palette rules apply, and what must happen between segments.

Loading a frame is not presenting it

The wider opcode survey exposed a subtle mistake in the first parser. It treated 0x6c as the complete frame command and ignored 0x77. The resulting movies still played because every observed 0x6c is immediately followed by one 0x77, but that collapsed two operations that the original player keeps separate.

Ghidra shows the actual dispatch boundary:

1
2
0x6c -> LoadCustomPacketPaletteStateBlock @ 0x6c430
0x77 -> RenderCustomPacketFrameAndOverlays @ 0x6c486

The engine now retains the compressed payload as pending state when it encounters 0x6c. It commits that payload as a reconstructed Smacker frame only when the matching 0x77 arrives. Across the retail corpus there are 155,486 load/render pairs with no mismatches.

This distinction does not change the pixels produced by the current assets, but it restores an important property of the packet language: decoding state may be prepared before the command that makes it visible.

The current implementation separates IAVF parsing and Smacker reconstruction from presentation, input, display, and timing.

The video frame rate was the wrong clock

IAVFExtract had already demonstrated that the embedded Smacker and PCM streams could be extracted, but it also reported persistent A/V drift. That was an important clue: the remaining problem was probably not missing media data, but timing information encoded in the IAVF command stream.

Reconstructing the Smacker streams made the pictures appear, but it did not make the presentation correct.

The first implementation allowed each reconstructed segment to advance according to its embedded Smacker timing. That is the natural behavior for an ordinary .SMK file. In IAVF, however, the audio stream controls the presentation.

The important evidence came from commands 0x66 and 0x67.

Audio command 0x66

For the observed files, the arguments describe:

1
2
3
arg0 = descriptor tag
arg1 = number of bytes represented on the presentation timeline
arg2 = number of bytes physically stored after the descriptor

Those last two values are not always equal.

When arg1 is larger than arg2, the original player still advances the audio timeline by arg1 bytes. The missing portion represents silence. Reading only the stored payload would produce a shorter audio stream and cause every later frame boundary to arrive early.

The reimplementation therefore appends the stored bytes and fills the remainder with zeroes:

1
2
append(payload, arg2);
appendSilence(arg1 - arg2);

The complete PCM buffer now has the same duration as the original presentation timeline.

Audio gate 0x67

Command 0x67 refers back to a tagged audio descriptor and supplies its cumulative effective byte offset. It is a playback gate, not inherently a video-frame command. Some gates surround segment setup or the end of a presentation, which is why the header’s gate count can be larger than the number of rendered frames.

Ghidra shows the original player servicing this through:

1
ServiceManagedAudioTriggerEntry840ControlLoop @ 0x48ad3

The loop waits for the managed-audio state to advance far enough before allowing the corresponding presentation work to continue.

When a gate precedes a custom frame packet, it establishes the scheduling relationship: the frame is due when audio playback reaches the recorded byte offset.

The third argument also describes fallback pacing when managed audio is unavailable. The original code interprets the low byte as the integer rate and the high byte as hundredths. A value of 0x3207, for example, represents 7.50 updates per second. Normal playback still follows the tagged audio descriptors rather than this fallback clock.

The ScummVM implementation converts the byte offset into milliseconds:

1
2
3
4
5
audioByteRate =
    sampleRate * channels * bitsPerSample / 8

targetMilliseconds =
    frameAudioOffset * 1000 / audioByteRate

During playback it compares that target against the mixer’s elapsed time:

1
2
audioElapsedMs = mixer->getSoundElapsedTime(audioHandle);
frameDue = audioElapsedMs >= targetAudioMs;

If audio could not be started, the engine falls back to a system timer using the same target offsets. Under normal conditions, the mixer is authoritative.

This fixed a class of errors that could not be solved by adjusting the Smacker frame rate. The video segments did not own independent clocks. They were visual events attached to positions in one continuous audio timeline.

The final packet is not the end of playback

A later scene exposed one more timing boundary. Opcode 0x70 exits the packet dispatch loop, but the original player continues polling GetManagedAudioTriggerActiveDescriptor until the final managed-audio descriptor completes. KA_BOOK.AVI, for example, has several seconds of audio after its last custom-video frame.

The ScummVM implementation therefore retains the terminal decoded frame while the remaining PCM plays. Escape and Space remain active during that wait. Only after the audio tail completes, or the player skips it, does the presentation release its media state. In other words, 0x70 marks the end of the packet program, not necessarily the end of the visible and audible presentation.

High-verbosity logging made the result measurable:

1
2
3
4
5
Ripper: Smacker 'PROINT.AVI#7'
        frame=...
        audioTargetMs=...
        audioElapsedMs=...
        driftMs=...

Per-frame diagnostics are intentionally kept out of normal debug levels, but they were useful while establishing the clock relationship.

One file can contain several movies

PROINT.AVI is not one Smacker stream. It contains 16 segments. PROLOG1.AVI contains two.

Segments can have different dimensions and positions. The scale decision is made separately for each embedded Smacker branch, matching the behavior selected by:

1
InitializeMediaPresentationDisplayModeCallback @ 0x163a8

PreparePacketizedMediaPlaybackBranchSetup invokes that callback for every branch. A branch smaller than 321×201 receives a 2:1 display descriptor, and its effective scaled extents are centered. This allows a 320×200 segment to occupy the full 640×400 presentation surface while preserving its original aspect ratio.

This distinction matters for PROLOG2.AVI: its IAVF canvas declares 640×400, but its embedded Smacker branches are 320×200 and must still be doubled. Full-sized branches switch to the full display context rather than inheriting the scene viewport’s 50-pixel vertical origin.

The segment boundaries also explain another visible defect from the early implementation. A smaller segment could be drawn over a larger preceding segment while leaving old pixels around its edges.

The video decoder was working correctly. The display state was not.

Display commands are part of the format

IAVF command 0x68 reaches two display services in the original executable:

1
2
DispatchDisplayServiceCommand(0x1d)
DispatchDisplayServiceCommand(0x14)

The second command resolves to:

1
ClearGenericVideoLogicalPage @ 0x45ed8

In PROINT.AVI, command 0x68 appears before transitions between several differently sized segments. The active logical page must be cleared before the next segment’s palette and pixels are presented.

Without that operation, pixels from the previous segment remain visible. Because the new segment may also install a different palette, those leftover pixel indices can change colour even though their values in the framebuffer never changed.

The reimplementation records the display boundary on the following segment:

1
segment.clearDisplayBefore = pendingDisplayClear;

Before presenting that segment, it clears the ScummVM surface and submits the update. If 0x68 is followed by 0x70 instead of another segment, the engine records a final display clear and applies it after the presentation and any managed-audio tail complete.

Palette handling required similar care. RIPPER has another Smacker path that preserves palette ranges used by the toolbar and other interface elements. IAVF does not use that same path. The original packet renderer installs the packet’s active palette directly, so applying the interface patch universally would modify the introductory movies incorrectly.

This produced a useful rule:

1
2
3
4
5
Direct scene Smacker:
    apply the shared interface palette bands

Reconstructed IAVF segment:
    preserve the complete decoded packet palette

The distinction is based on two separate original call paths, not on a general preference about how palettes should behave.

Returning control to the scene

Playing the movie was not the final boundary.

Some IAVF presentations are launched from scene scripts. On the keyboard-controlled path, the original RunMediaPresentation wrapper preserves the logical display state while the packetized presentation temporarily owns the screen. When playback ends, it restores the previous page, submits a full display update, restores the palette, and reactivates selection presentation.

ScummVM has one active framebuffer rather than the original display-page arrangement. The engine therefore snapshots both indexed pixels and all 256 palette entries before keyboard-controlled IAVF playback.

After the movie finishes, it restores them together:

1
2
3
4
5
6
capture indexed framebuffer
capture 256-colour palette
play IAVF presentation
restore indexed framebuffer
restore palette
update screen

Restoring only the pixels is insufficient because the same byte values can represent entirely different colours under the movie’s final palette.

This became especially important once dialogue response videos and other scripted presentations began returning directly to interactive scenes. The transition had to restore the scene the player left, not retain the last frame or palette from the movie.

That restoration is a property of the presentation route, not a universal IAVF rule. An uncontrolled presentation intentionally leaves its final page visible. The Virtual Herald path depends on this behavior: after VH_1.AVI completes, the game enters its interface frame using the movie’s retained display rather than restoring the black pre-roll page.

Controls and explicit extensions

The original controlled-presentation callback recognizes Escape and Space. Escape stops the whole presentation, Space pauses or resumes both video and managed audio, and mouse buttons do not skip the packetized path.

To make initial debugging and play-testing easier, I added Right Arrow handling as an explicit convenience for segmented IAVF files. It leaves the current Smacker branch, seeks the shared PCM stream to the next branch’s first absolute audio offset, and rebases that branch’s frame gates to the restarted mixer clock. Right Arrow on the final or only branch reports successful completion so the surrounding script still performs its normal played-state update.

Escape also needs a deliberate translation at the engine boundary. ScummVM rebuilds and presents the terminal Smacker frame before returning through the normal completion path, ensuring later script commands still run and delta-coded surface or palette state is complete. Retail stops the presentation instead of visibly decoding forward, so this is completion behavior rather than a claim of literal input fidelity.

Keeping the implementation narrow

There were several opportunities to solve a local problem in the wrong layer:

• teach the shared Smacker decoder about IAVF;
• add RIPPER palette rules to generic video code;
• change shared display behavior to accommodate one game’s page model;
• implement a new codec rather than reconstruct a standard stream;
• assume every branch in the original packet engine had the same semantics as the observed assets.

The current design avoids those shortcuts.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Media format detector
    identifies IAVF2.00, SMK2, and SMK4 by signature
    preserves the stream position before dispatch

IAVF adapter
    parses IAVF descriptors
    constructs the PCM timeline
    records segment and frame-gate boundaries
    rebuilds Smacker streams

RIPPER MediaPlayer
    maps audio offsets to frame deadlines
    applies placement, input, palette, and display rules
    restores or retains the final display for the active route

ScummVM services
    decode Smacker frames
    play PCM audio
    manage surfaces and palettes
    provide events and timing

This boundary keeps the proprietary behavior auditable against RIPPER.EXE while leaving shared ScummVM components game-agnostic.

Each route now expresses its decoder policy through a SmackerPlaybackPlan, grouping placement, input, timeline, palette, frame range, looping, callbacks, and rendering. The player emits one stable trace of that plan before playback, and controlled IAVF routes use the engine’s shared IndexedDisplaySnapshot service for pixels and palette state. These are structural changes rather than new IAVF semantics, but they make later refactoring easier to compare with the original call paths.

It also leaves room for future work. Ghidra shows FLIC and other custom packet branches in RunPacketizedMediaPlaybackCore, but the complete scan of this retail data set found no IAVF file that exercises them. The current engine does not claim to implement every branch merely because it exists in the executable. Those paths should be added when an asset or reachable scene supplies enough data to verify them.

What this work established

Reconstructing IAVF produced several broader lessons for the RIPPER engine:

1. File extensions are weak evidence. Inspect the bytes and follow the native dispatch path.
2. A proprietary container may wrap a standard codec rather than replace it.
3. Stored payload length and presentation duration are separate concepts.
4. A header’s count may describe scheduler gates rather than decoded frames.
5. Loading decoder state and presenting it may be separate commands.
6. The correct clock may live outside the video stream.
7. Display clears and palette changes are part of media semantics.
8. Framebuffer state and palette state must be restored together.
9. The end of packet dispatch may precede the end of the presentation timeline.
10. Shared codecs should remain shared; game-specific scheduling belongs in the engine.

The same pipeline now supports startup movies and later scene, dialogue, WAC, inventory, and puzzle presentations. It preserves the original packet ordering, reconstructed Smacker segments, continuous PCM timeline, audio-master frame scheduling, segment display boundaries, and route-specific palette and display behavior.

Prior work established the essential breakthrough: RIPPER’s misleadingly named .AVI files are command streams around standard Smacker and PCM data. Reconstructing the original player in Ghidra supplied the remaining piece: how those commands schedule, synchronize, reset, and restore playback inside the game.

What initially looked like an unsupported AVI codec turned out to be a custom multimedia scheduler built around familiar components. Once that distinction was clear, the reimplementation became less about decoding video and more about recovering the rules that connected audio, frames, palettes, and display state.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 6: Timers
• Reverse Engineering Harvester with Ghidra and Codex - Part 7: Demo Support

Article 7 of 7 in this series.

Until recently, the ScummVM harvester engine had been developed against the retail DOS release. That was enough to recover the resource formats, room system, script interpreter, dialogue, combat, timers, and most of the surrounding game flow. It did not tell me whether the earlier playable demo was simply a smaller data set or a meaningfully different build of the engine.

Supporting the demo gave me a useful way to answer that question. I loaded the retail HARVEST.LE and demo HARVEST.EXE into separate Ghidra projects, exposed both through Ghidra MCP, and used Codex to compare the two programs while working through the demo in ScummVM.

Establishing the comparison baseline

The first result was that the demo is much closer to retail than its different executable name suggests.

Ghidra recovered 745 functions from the demo and 928 from retail. After normalizing the opcode streams, 672 demo functions had an exact hash match in the retail executable. That left 73 without an immediate match, but 43 of those were dialogue handlers whose embedded dialogue identifiers differed between builds. The instructions around those identifiers were otherwise equivalent.

The resource layer tells the same story. The demo uses the same XFile archive format described in part 3. Its first two archive sets contain 354 entries, and the missing third set is a normal consequence of shipping a smaller slice of the game. ScummVM’s ResourceManager already tolerated that absent set.

This gave me a practical implementation rule: keep one Harvester engine and introduce demo-specific behavior only where the binaries or data demonstrate a real boundary. A single isDemo() query, backed by the detector flags, became the gate for those cases.

The detector entry itself uses the demo executable’s size and MD5:

1
2
3
4
HARVEST.EXE
size: 1,167,247 bytes
first 5,000 bytes MD5: 787e43b868ebfaca614010af3ab66b6d
full-file MD5: 4990e3cfffe190285500b9d297948a3e

Once detected, the variant is marked with Advanced Detector game entry flags ADGF_DEMO | ADGF_UNSTABLE. The retail detector entry remains separate, which lets the engine expose different configuration options without adding checks throughout the runtime. For example, the retail build can ask whether CD-change prompts should be displayed, while that option is hidden for the single-disc demo.

The menu is compiled into the demo

The first visible difference appears before entering a room. Retail Harvester loads its menu labels from MENU.INI. The demo does not contain that file because the equivalent strings are built into HARVEST.EXE:

1
2
3
4
5
6
NEW GAME
SAVE GAME
LOAD GAME
OPTIONS
HELP
QUIT GAME

Treating the missing file as a generic resource failure produced an empty or unusable menu. The fix was small: if MENU.INI is absent and isDemo() is true, populate those six confirmed labels. Retail continues to use its data file.

The same distinction applies to several prompts. The demo executable supplies labels such as Click, START A NEW GAME?, and QUIT HARVESTER?. These are details, but they are also exactly the sort of details that disappear if demo support is reduced to “make the first room load.”

Similar files can still have different layouts

Dialogue exposed a less obvious variant boundary. The demo includes DIALOG.RSP, just as retail does, but the two files do not have the same layout.

The engine already had recovery logic for an awkward retail response layout associated with Hank’s dialogue. If the expected structure was missing, it searched the game path for a better candidate. That was useful while developing against retail files, but it was actively harmful when a demo directory lived below a retail installation: valid demo data could be replaced by a parent-directory retail DIALOG.RSP.

Comparing the two builds showed that the demo layout was intentional. Its only active scripted conversation is LODGE_CHEF, and the shared CLOAK_ATND handler is already represented by the engine. The correction was therefore to keep the retail recovery search away from demo data rather than trying to force both files into the same shape.

Comparing the script surface

The demo’s world script is HARVDEMO.SCR. Like the retail HARVEST.SCR covered in part 4, it is XOR-obfuscated text made up of records and named command chains.

After decoding it, I found 359 live COMMAND records using 27 opcodes. ScummVM already implemented 26 of them. All active command branches, flags, and resource paths resolved. The single missing opcode was END_DEMO.

That is a much narrower problem than implementing a second script interpreter. In the native parser, END_DEMO maps to opcode 0x34. It appears at the end of the demo’s final interaction, after SHOW_TEXT ART_STEF_SCREAM. The interpreter needs to preserve that continuation, turn END_DEMO into a terminal request, and stop walking the command chain.

I carried that request through the same InteractionResult structure used for room changes, cutscenes, modal text, and other deferred actions. This keeps the script layer responsible for interpreting the opcode while the flow layer owns the presentation and shutdown sequence.

1
2
3
4
5
HARVDEMO.SCR
359 live COMMAND records
27 distinct opcodes
26 already implemented
1 demo-specific opcode: END_DEMO

Restoring the native introduction

Getting to the first room is not the start of the native demo. Before START, the original executable presents a short sequence built from ordinary demo resources.

It displays the confirmed DEMOTEXT paragraph over MAINMENU.BM using MAINMENU.PAL, with LODGERB.CMP playing underneath. The native code waits for 1,700 centiseconds, or 17 seconds, while allowing the player to skip. It then plays GRAPHIC/FST/PCFALL.FST before entering the initial room.

This sequence belongs behind isDemo() because retail has its own startup flow. The input handling is also important. A player should be able to skip the text using the same keyboard or mouse inputs expected by the original without accidentally bypassing later game input.

Restoring the native ending

END_DEMO does more than quit the executable. The room presentation stops, the cursor is hidden, and SOUND/MUSIC/ROCKFITE.CMP begins playing. The demo then displays five bitmap and palette pairs in this order:

1
2
3
4
5
SCREEN1
SCREEN2
SCREEN3
SCREEN4
SCREEN6

The numbering is worth calling out because it is easy to “correct” it into a consecutive sequence that the native executable never uses. Each screen remains visible for up to 1,000 centiseconds, or 10 seconds. Space, Escape, Enter, or a primary mouse click advances to the next image. The screens fade between one another, the music fades during the final screen, resources are released, and the engine terminates cleanly.

This is the largest demo-only flow added for parity, but it still reuses the existing bitmap, palette, audio, event, and fade machinery. The special behavior is mostly the confirmed ordering and timing.

What a complete play-through found

Static comparison established that the demo did not need a separate engine. Runtime progression still mattered because matching functions and resolving script paths cannot prove that every deferred action is applied at the right point in the room loop.

The demo is particularly good at exercising those edges. Its opening route uses timer-based player locking in BOWLSNTRY1. Later sequences depend on room-entry actions after leaving closeups, region and lighting continuations, monster activation, scripted weapon changes, entity depth alignment, and player movement along the Z axis.

Working through those scenarios exposed several missing or incomplete shared behaviors:

• PAUSE_PC and RESUME_PC had to control player input for the full scripted interval.
• Room-entry actions had to run after returning from a closeup.
• PC_CHANGE_WEAPON had to update the scripted weapon state.
• MOVE_BM2PCZ had to align a scripted entity with the player’s depth.
• PC_GOTO_Z had to move the player along Z while preserving the current X position.

These are not reasons to fork demo behavior. They are parts of the common script model that the shorter demo happens to reach quickly and predictably. Once corrected, the same semantics are available to retail scripts.

This was also a good continuation of the timer work from part 6. With the resources, scene, dialogue, and combat debug channels enabled, I could follow each transition from decoded command record to live runtime state and compare the result against the original demo.

Keeping the variants honest

The main risk in adding demo support was not the amount of missing code. It was letting a handful of confirmed differences turn into broad isDemo() branches that would be difficult to verify later.

The current split is intentionally small:

• detection identifies the demo as its own variant;
• built-in menu labels and prompts replace files the demo does not ship;
• dialogue recovery respects the demo’s valid response layout;
• the introduction and ending reproduce demo-only presentation flows;
• the CD-change prompt option remains retail-only;
• shared script and room behavior stays shared.

The remaining parity work is scenario-driven. Save and load need round-trip checks during demo progression. Lodge Chef dialogue, lighting continuations, monster activation, and the final TO_ART_GAL2 route need to remain reproducible. Any additional divergence should start with the same test: compare the demo and retail binaries in Ghidra, reproduce the behavior in a specific runtime scenario, and only then decide whether it belongs behind isDemo().

Supporting the demo turned out to be a compact audit of the engine as a whole. The binary comparison showed that most of the original implementation was shared. The places that differed were concrete and explainable: compiled-in menu text, a distinct dialogue response layout, an introductory sequence, one terminal script opcode, and an ending slideshow. Everything else became a test of whether the common Harvester model had been implemented faithfully enough to run a second build of the game.

The Fork in the Road: Two Problems, Two Projects

By early 2023, the practical challenge of building LLM-powered applications had split into two distinct problems that nobody had cleanly solved.

The first was enterprise integration: how do you embed an LLM into an existing application with proper dependency injection, logging, telemetry, and reusable tool abstractions? The second was multi-agent coordination: how do you get several specialized agents to collaborate, argue, hand off work, and converge on a goal without you writing all the orchestration logic by hand?

Microsoft ended up with two separate projects, built by two separate teams, each solving one of these problems. They developed in parallel, later aligned, and eventually converged into one successor framework.

Semantic Kernel (March 2023): The Enterprise SDK

Semantic Kernel launched publicly on March 17, 2023, with C#/.NET as the primary supported path and Python already available in an experimental preview branch. Java support followed later. Its central abstraction was the Kernel object — a hub that wired together plugins, planners, memory, and model connectors.

Plugins (originally called “skills”) were collections of native functions and prompt templates that an LLM could invoke. Planners asked the model to choose which plugin functions to compose in order to satisfy a user goal — though as Microsoft’s own documentation notes, the framework has since moved away from prompt-based planning toward native function calling. Around this core sat the plumbing enterprise .NET teams expected: dependency injection, filters, telemetry hooks, and extensive connector support for vector stores and embedding services.

This was genuinely useful. If you had an existing enterprise application and wanted to add LLM capabilities without throwing away your architecture, Semantic Kernel gave you a structured, testable way to do it.

What early Semantic Kernel didn’t give you was any first-class concept of multiple agents. A Kernel plus a planner could compose tools, but it did not originally model agents negotiating with each other, a coding agent handing a result to a review agent, or a team of agents coordinating around a shared task.

That changed over time. Microsoft introduced agent abstractions and multi-agent support in Semantic Kernel during 2024, and Semantic Kernel Agents became generally available in April 2025. By then, Semantic Kernel had sequential, concurrent, group-chat, handoff, and Magentic-style orchestration patterns. The better historical claim is not that Semantic Kernel never supported agents, but that its original center of gravity was enterprise application integration rather than conversational multi-agent orchestration.

AutoGen (August 2023): Conversation as a Programming Model

AutoGen came from a completely different direction. The foundational paper — “AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation” by Qingyun Wu, Gagan Bansal, Chi Wang, and eleven co-authors — landed on arXiv on August 16, 2023. Its documented proposition was flexible programming of multi-agent conversation patterns and reusable infrastructure for building LLM applications. In practice, the big idea felt simple: conversation could become a powerful programming model.

Instead of writing imperative orchestration logic, you defined agents (AssistantAgent, UserProxyAgent) with system prompts and tools, dropped them into a GroupChat with a GroupChatManager, and let them reach a goal by exchanging messages. Code execution was first-class: a UserProxyAgent could run model-generated Python in a Docker sandbox and feed the results back into the conversation. The pattern was surprisingly expressive — two-agent debate, critic-and-coder loops, and tool-use chains all fell out naturally from the same model.

AutoGen spread fast in the research community precisely because it was easy to prototype with. But it had real production weaknesses: the GroupChat manager’s speaker-selection logic was opaque, the synchronous execution model didn’t scale, and the architecture made it hard to add proper observability or async workflows.

The Fork: AG2 Splits from AutoGen

By late 2024, the original AutoGen lineage had split. AG2, led by Chi Wang and Qingyun Wu, the paper’s primary authors, positioned itself as a spin-off and evolution of AutoGen with an independent organization and open governance. It lives at ag2ai/ag2 on GitHub. Its modifications and additions are Apache 2.0, while inherited AutoGen code remains under the original MIT license.

Meanwhile, Microsoft shipped AutoGen v0.4 in January 2025 as a complete rewrite. The new architecture was built around an asynchronous, event-driven actor model with a clean three-layer API:

• autogen-core — the async runtime, message passing, and actor primitives
• autogen-agentchat — the high-level AssistantAgent, GroupChat, and team abstractions familiar from v0.2
• autogen-ext — extensions for specific models, tools, and integrations

The layered design let you drop down to autogen-core for fine-grained control or stay at autogen-agentchat for the familiar conversational API. But it was a breaking change from v0.2, and teams that had built on the original AutoGen now had to choose: follow Microsoft to v0.4, or follow the original authors to AG2.

⚠️ The pip install autogen Packaging Hazard

Before going further, there is a concrete gotcha you will hit if you’re not careful.

The PyPI package name autogen is not Microsoft’s current AutoGen. The name existed years before the split, but as of June 2026 it is controlled and used as an alias for AG2. Installing it silently gives you AG2’s API, not Microsoft’s rewritten AutoGen packages, with no error or warning. If you’re following Microsoft’s docs, you’ll get an API mismatch that can be genuinely confusing to debug.

The correct install commands are:

1
2
3
4
5
6
7
8
# Microsoft AutoGen v0.4 (AgentChat plus OpenAI extension)
pip install -U "autogen-agentchat" "autogen-ext[openai]"

# AG2 (the community fork, maintained by the original authors)
pip install ag2

# Microsoft Agent Framework (the current successor to both)
pip install agent-framework

This has caught real developers. If your imports are failing or the API doesn’t match the docs, check pip show autogen first — there’s a good chance you have the wrong package entirely. The AG2 migration guide and AutoGen v0.4 docs both call this out explicitly, but it’s easy to miss if you’re moving fast.

The Microsoft Agent Framework (October 2025 to April 2026): The Merger

On October 1–2, 2025, Microsoft announced the public preview of the Microsoft Agent Framework — agent-framework on PyPI, Microsoft.Agents.AI on NuGet. On April 3, 2026, Microsoft released Agent Framework 1.0 for both Python and .NET, describing it as production-ready, with stable APIs and a commitment to long-term support. Microsoft’s own documentation is explicit about what this is: “the direct successor” to both Semantic Kernel and AutoGen, “created by the same teams.”

The merger is architectural, not just organizational. The Agent Framework takes Semantic Kernel’s enterprise plumbing — session-based state management, type safety, filters, telemetry, and the full connector ecosystem — and combines it with AutoGen’s multi-agent orchestration patterns. On top of that foundation it unifies, productizes, and stabilizes capabilities that had previously been scattered across the predecessor projects:

Graph-based workflows with type-safe routing. Workflows connect executors and edges. Executors may wrap agents, ordinary functions, or deterministic business logic. Transitions are typed, so the compiler (in .NET) or runtime (in Python) can catch routing errors before they surface as mysterious failures at 2am.

Checkpointing and resumability. Long-running workflows can be persisted and resumed. This is the feature that makes human-in-the-loop genuinely practical — a workflow can pause, wait for human input or approval, and pick up where it left off.

First-class human-in-the-loop. Rather than relying only on a UserProxyAgent-style workaround, the framework supports human interaction through request/response mechanisms such as RequestPort, ctx.request_info(), response handlers, and approval-required tools that can suspend and later resume a workflow.

Unified observability. Semantic Kernel’s telemetry hooks and AutoGen’s tracing capabilities are merged into a single instrumentation layer, with OpenTelemetry support throughout.

Agent Framework 1.0 is available for both Python and .NET. Some adjacent features remain preview, but the core agent and workflow APIs are now the production path Microsoft is supporting.

What Should You Use Today?

The landscape has four real options. Here’s the honest breakdown:

Microsoft Agent Framework 1.0 — Use this for new production projects, especially if you’re in a Microsoft/Azure ecosystem or need stable APIs, Microsoft support, graph workflows, telemetry, checkpointing, or .NET/Python parity. It reached 1.0 on April 3, 2026 and is now the production-ready, long-term-supported successor to Semantic Kernel and AutoGen. The quickstart is the right place to start.

AG2 — Use this if you want to stay on the conversational ConversableAgent model, you have existing AutoGen 0.2.x-style code, or you want a framework governed outside Microsoft’s roadmap. It is actively maintained and the community is engaged, but do not treat the current API as a fully settled 1.0 surface: AG2’s own roadmap says the project is on the path to v1.0, the current framework will go through deprecations and move into maintenance mode, and the beta framework will become the official v1.0 API. You can install it as either pip install ag2 or pip install autogen — both resolve to the same package — but ag2 is the canonical name.

Microsoft AutoGen v0.4 — The async rewrite is solid engineering, but the Microsoft AutoGen repository now says the project is in maintenance mode, will receive no new features or enhancements, and is community-managed going forward. Existing deployments can keep running, and the concepts map cleanly to the Agent Framework, but new strategic projects should start with Agent Framework and existing users should plan migration.

Semantic Kernel 1.x standalone — Still supported and still useful for existing applications, especially plugin-heavy .NET systems. But Microsoft has said most new agent-development investment will go into Agent Framework, so new multi-agent development should usually start there rather than in Semantic Kernel directly.

Sources: Semantic Kernel launch post · Semantic Kernel agents announcement · AutoGen arXiv paper · AutoGen v0.4 announcement · Microsoft Agent Framework overview · Microsoft Agent Framework 1.0 announcement · Agent Framework workflows · Agent Framework human-in-the-loop docs · Microsoft AutoGen repository · AG2 repository · PyPI autogen package · PyPI autogen 0.1.0

In a recent post I made the case that MongoDB can serve as the “brain” of a modern AI application by combining durable state, retrieval, and application data in one place. That framing still holds, but brains are only useful if they can recall the right thing at the right time. I wanted to dig into agentic application development in more detail in a series of posts, so for the first real entry in this series, I want to start one layer below “agents” and one layer above raw storage: the context layer.

That might sound slightly less glamorous than “multi-agent orchestration,” which is exactly why it matters. Most enterprise AI systems do not fail because they lack a clever planner. They fail because the model sees the wrong document, too much irrelevant text, or none of the operational data that actually matters.

To make this concrete, the application thread for this series will be an enterprise support escalation copilot for a B2B SaaS team. By the end of the series, it should be able to answer questions about incidents, remember previous escalations, pull account context, and coordinate specialized agents when needed. Today, though, we’re giving it its first useful skill: finding the right context for the job.

Think about the kind of question a real support engineer asks:

“Acme’s enterprise tenant started seeing INV-4421 after upgrading to 3.8. Did we see this before, is there a known workaround, and does it affect EU clusters only?”

That is not a pure semantic search problem. It is part natural language, part exact identifier lookup, part metadata filtering, and part ranking problem. Error codes matter. Version numbers matter. Tenant boundaries matter. Timing matters. That’s why this is such a good place to start - and to solve this problem we’ll dig in with MongoDB and Voyage AI.

MongoDB Vector Search is built to search vector data alongside the rest of your operational data, supports filtering on other fields in the collection, and can be combined with full-text search for hybrid retrieval. MongoDB’s hybrid search documentation explicitly describes combining semantic and full-text search results with Reciprocal Rank Fusion, which is exactly what you want when a query mixes fuzzy intent with exact strings like issue IDs, SKUs, or feature flags.

On the retrieval-model side, Voyage provides high-accuracy embedding and reranking models, including newer capabilities like contextualized chunk embeddings, multimodal embeddings, and rerankers designed to refine the top candidate set after initial retrieval. MongoDB Atlas now also exposes Voyage models through its Embedding and Reranking API, currently in preview, which means you can either call Voyage models directly or keep retrieval models, vector search, and operational data closer together under Atlas.

So what does the retrieval core for our support copilot actually do?

First, it stores source material in MongoDB: runbooks, release notes, KB articles, previous incident reviews, ticket summaries, and whatever structured account data the support flow needs. Then it chunks the long-form content, embeds it with Voyage, and stores the vectors with the source text and metadata. At query time, it narrows scope using metadata like tenant, product, region, or severity; retrieves candidates semantically and lexically; reranks the best matches; and only then hands a compact, relevant context window to the LLM. In other words: don’t ask the model to be psychic when the database can be specific.

There are a lot of AI frameworks right now, and they absolutely do not all feel the same. But this is the first important pattern in the series: the framework should shape the developer experience, not force you to redesign the data layer every six months. The retrieval architecture is the stable part. MongoDB and Voyage AI are the stable parts. LangChain, LlamaIndex, Haystack, LangGraph, CrewAI, or whatever comes next should be able to sit on top of that foundation.

A framework-agnostic mental model

Before jumping into code, here is the mental model I’d keep fixed no matter which framework you prefer:

1. Put source documents and operational records in MongoDB.
2. Generate embeddings with Voyage.
3. Index vector fields and filter fields in MongoDB.
4. Use semantic retrieval for meaning.
5. Use full-text retrieval for exact strings.
6. Rerank the candidate set before generation.
7. Return only the context the model actually needs.

That shape maps cleanly to both MongoDB Vector Search and Voyage’s model stack. MongoDB handles vector indexes, full-text search, filterable metadata, and live application data; Voyage handles embeddings and reranking; the framework becomes the control surface.

Approach 1: LangChain for the shortest path from data to grounded answers

If the goal is to get a retrieval-backed application running quickly, LangChain remains a very practical starting point. MongoDB’s LangChain integration supports vector search, full-text search, and a hybrid retriever that combines both with Reciprocal Rank Fusion. It also supports pre-filtering with MQL expressions, which matters immediately for tenant scoping and product boundaries.

An illustrative version for our support copilot looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import os

from langchain_voyageai import VoyageAIEmbeddings
from langchain_mongodb.vectorstores import MongoDBAtlasVectorSearch
from langchain_mongodb.retrievers.hybrid_search import MongoDBAtlasHybridSearchRetriever

embeddings = VoyageAIEmbeddings(
    api_key=os.environ["VOYAGE_API_KEY"],
    model="voyage-4",
)

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
    connection_string=os.environ["MONGODB_URI"],
    namespace="support.context",
    embedding=embeddings,
    index_name="support_vector_index",
)

retriever = MongoDBAtlasHybridSearchRetriever(
    vectorstore=vector_store,
    search_index_name="support_search_index",
    k=8,
    fulltext_penalty=60.0,
    vector_penalty=60.0,
)

docs = retriever.invoke(
    "Acme tenant seeing INV-4421 after upgrading to 3.8"
)

In a production version, I’d pair that with metadata filters on fields like tenant_id, product, region, and severity, then pass the top candidates through a Voyage reranker before generation. The point is not that LangChain is magical. The point is that the MongoDB + Voyage retrieval story already fits the way LangChain applications are commonly assembled.

Approach 2: LlamaIndex when the center of gravity is the data itself

If LangChain often feels application-first, LlamaIndex tends to feel data-first. That makes it a very natural fit when you want to spend more time shaping ingestion, chunking, metadata, and query behavior.

Using MongoDB’s LlamaIndex integration we can use VoyageEmbedding alongside MongoDBAtlasVectorSearch to make metadata filters very explicit, which is helpful for real enterprise retrieval where “give me the right answer” usually means “give me the right answer for this tenant, this region, and this product line.”

The shape is roughly:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import os

from pymongo import MongoClient
from llama_index.embeddings.voyageai import VoyageEmbedding
from llama_index.vector_stores.mongodb import MongoDBAtlasVectorSearch
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.core.retrievers import VectorIndexRetriever
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

embed_model = VoyageEmbedding(
    voyage_api_key=os.environ["VOYAGE_API_KEY"],
    model_name="voyage-4",
)

mongo_client = MongoClient(os.environ["MONGODB_URI"])
vector_store = MongoDBAtlasVectorSearch(
    mongo_client,
    db_name="support",
    collection_name="context",
    vector_index_name="support_vector_index",
)

storage_context = StorageContext.from_defaults(vector_store=vector_store)

# docs is your loaded support corpus, such as runbooks, incident reviews,
# release notes, and ticket summaries.
vector_index = VectorStoreIndex.from_documents(
    docs,
    storage_context=storage_context,
    embed_model=embed_model,
)

filters = MetadataFilters(
    filters=[ExactMatchFilter(key="tenant_id", value="acme")]
)

retriever = VectorIndexRetriever(
    index=vector_index,
    filters=filters,
    similarity_top_k=10,
)

nodes = retriever.retrieve("Known workaround for INV-4421?")

What I like about this path is that it keeps the retrieval pipeline honest. You can see the data model. You can see the filter model. You can see how chunking choices affect what comes back. For article one in a series like this, that clarity is useful because it keeps us focused on context quality before we get distracted by agent loops.

Approach 3: Haystack when you want explicit, composable pipelines

Haystack is a nice fit for teams that prefer explicit components over higher-level abstractions. MongoDB’s Haystack integration uses a MongoDBAtlasDocumentStore with MongoDB retrievers, and the official tutorial pairs that with Voyage embedders. Haystack’s MongoDB integration also has separate semantic and full-text retrievers, which is useful when you want to make the retrieval strategy itself a first-class part of the pipeline.

A trimmed-down version looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from haystack import Pipeline
from haystack.utils import Secret
from haystack_integrations.components.embedders.voyage_embedders import VoyageTextEmbedder
from haystack_integrations.document_stores.mongodb_atlas import MongoDBAtlasDocumentStore
from haystack_integrations.components.retrievers.mongodb_atlas import MongoDBAtlasEmbeddingRetriever

document_store = MongoDBAtlasDocumentStore(
    mongo_connection_string=Secret.from_env_var("MONGODB_URI"),
    database_name="support",
    collection_name="context",
    vector_search_index="support_vector_index",
    full_text_search_index="support_search_index",
)

pipeline = Pipeline()
pipeline.add_component("query_embedder", VoyageTextEmbedder(model="voyage-4"))
pipeline.add_component(
    "retriever",
    MongoDBAtlasEmbeddingRetriever(document_store=document_store, top_k=10),
)
pipeline.connect("query_embedder.embedding", "retriever.query_embedding")

result = pipeline.run(
    {"query_embedder": {"text": "Known workaround for INV-4421?"}}
)

This is probably the most “pipe and fitting” version of the three, and that is a compliment. For enterprise teams, explicit systems are often easier to debug, evaluate, and explain. And once again, the interesting part is not that the framework is different. It is that the same MongoDB + Voyage retrieval core still fits.

Why MongoDB

The support copilot does not just need chunks in a vector store. It needs chunks, source documents, tenant metadata, ticket references, account records, release versions, and eventually execution state. MongoDB Vector Search lets you search semantic meaning alongside that operational data, pre-filter the search space using indexed fields, and combine vector and full-text retrieval when exact terms matter. Change streams then give you a way to react to new or updated records in real time, which is exactly what you want when incidents, tickets, or KB articles change during the workday.

And if you want an even tighter platform story, MongoDB Atlas now exposes Voyage models directly through the Embedding and Reranking API. That API is database-agnostic, but it pairs especially well with Atlas because it reduces the number of moving pieces needed to stand up a modern retrieval pipeline. Fewer services, fewer credentials, less trying to debug “why is this top result here?”.

This is also where the framework story becomes easier to reason about. LangChain, LlamaIndex, and Haystack all give you different ergonomics. MongoDB stays the system where the data lives. Voyage stays the retrieval layer that improves what gets surfaced. That is a much more durable architecture than betting everything on whichever orchestration framework happens to be loudest this quarter.

What next?

Once the retrieval core is solid, adding agents becomes a lot more interesting.

In the next post, I’d take this same support copilot and add short-term execution state and long-term memory. LangGraph is a natural next step as it separates persistence into checkpoints for thread state and stores for long-term memory, and MongoDB already has first-class integrations for both the LangGraph checkpointer and the long-term store. That is where the earlier “brain” idea becomes concrete: not just retrieval, but retrieval plus memory plus durable execution.

The broader trend line is pretty clear, too. Agent frameworks are converging on durable state and memory. Retrieval models are getting richer with contextualized chunk embeddings, multimodal embeddings, and better rerankers. MongoDB Atlas is moving retrieval models and database capabilities closer together. The winning application architecture is the one that can absorb those changes without forcing you to rebuild your data layer every few months. MongoDB and Voyage AI fit that direction unusually well.

The OpenAI Agents SDK has shipped session backends for SQLite, SQLAlchemy, Redis, and Dapr for a while now. With 0.14.2 (April 2026), MongoDBSession joined that list, and 0.14.6 added the docs page. If you’re already running MongoDB for application data, this is the moment to stop standing up a second store just to remember what the agent said three turns ago. The demo for this walkthrough is a small e-commerce support app with three handoff-connected agents and one MongoDB instance behind everything: customers, orders, support articles, and the conversation history.

Repo: https://github.com/alexbevi/mongodb-openai-agents-sdk-example.

What you’ll build

A CLI customer-support agent that identifies the user from MongoDB, hands off between a triage agent, an order-support agent, and a knowledge-base agent, and persists every turn (user message, tool call, tool output, assistant reply, handoff) to MongoDB via MongoDBSession. You quit, restart the process, log in with the same email, and the agent picks up the thread — no re-explaining the return you started yesterday.

Why MongoDB for sessions

A session backend has three jobs: store one item per turn, return them in order on the next run, and not corrupt itself when two processes write at once. The interesting part for MongoDB is how naturally each of those maps to things the database already does.

Items in a session are heterogeneous. A turn can be a user message, a tool call, a tool result, an assistant message, or a handoff record — each with its own shape. A document store takes those payloads as-is. There’s no messages table you have to migrate every time the SDK adds a new run-item type, and no JSON column to parse around.

Ordering is the hard part, and $inc is built for it. MongoDBSession stamps each message with a monotonically increasing seq counter — the SDK docs call this out explicitly: it preserves ordering across concurrent writers and processes. That’s a single-document atomic increment, not a distributed lock or an optimistic-retry loop. Two FastAPI workers handling the same session_id won’t interleave.

One store, one connection pool. This is the angle the demo actually showcases. The ecommerce_support database holds customers, orders, and support_articles next to agent_sessions and agent_messages. Tools query operational data, the SDK persists turns, and they share the same AsyncMongoClient. Adding session memory cost zero new infrastructure.

Walkthrough

1. Prerequisites

Python 3.10+, an OpenAI API key, and either a local mongod or a MongoDB Atlas cluster. Nothing in the demo requires Atlas-only features — a 27017 on localhost is fine.

2. Install

requirements.txt pins the new extra:

1
2
3
openai-agents[mongodb]>=0.14.2
python-dotenv>=1.0.0
pymongo>=4.13

1
pip install -r requirements.txt

The [mongodb] extra pulls in pymongo’s async client; the MongoDBSession class lives at agents.extensions.memory.MongoDBSession.

3. Connect

The demo uses one shared AsyncMongoClient per process (the right pattern — sessions don’t own the client, they share its pool):

1
2
3
4
5
6
7
8
9
10
11
12
13
from pymongo.asynchronous.mongo_client import AsyncMongoClient

MONGODB_URI = os.environ.get("MONGODB_URI", "mongodb://localhost:27017")
DB_NAME = "ecommerce_support"

mongo_client = AsyncMongoClient(MONGODB_URI)
db = mongo_client[DB_NAME]

try:
    await mongo_client.admin.command("ping")
except Exception as exc:
    print(f"\nCannot connect to MongoDB ({MONGODB_URI}):\n  {exc}")
    return

4. Seed and identify

python seed_data.py loads three demo customers, five products, five orders with embedded line items, and seven support articles indexed for $text search. Then main.py looks the customer up so the triage agent doesn’t have to ask for an email it already knows.

5. Instantiate the session

This is the integration:

1
2
3
4
5
6
7
8
9
10
11
session_id = f"support_{email.replace('@', '_at_').replace('.', '_')}"
session = MongoDBSession(
    session_id=session_id,
    client=mongo_client,
    database=DB_NAME,
)

if not await session.ping():
    print("Warning: MongoDB session storage is unavailable.")

existing = await session.get_items()

Constructing with client= (rather than MongoDBSession.from_uri(...)) means the session shares the app’s connection pool and session.close() becomes a no-op — the lifecycle stays with you. session.ping() is a real round-trip against MongoDB, useful for liveness probes.

6. Run

Pass session= to the runner. Everything else is the same SDK you already know:

1
2
3
4
5
6
7
with trace("Customer Support", group_id=conversation_id):
    result = await Runner.run(
        current_agent,
        input=user_input,
        context=ctx,
        session=session,   # MongoDB stores every turn automatically
    )

Have a conversation, quit, run python main.py again with the same email, and the next message gets the full prior context prepended automatically.

What MongoDB actually stored

After a few turns with alice@example.com, two collections show up in the ecommerce_support database. The interesting one is agent_messages. A representative document, abridged:

1
2
3
4
5
6
7
8
9
10
11
{
  _id: ObjectId("6620d1f4..."),
  session_id: "support_alice_at_example_com",   // partition key for this conversation
  seq: 7,                                        // monotonically increasing turn order
  message_data: {                                // the SDK's run-item, stored as-is
    type: "function_call_output",
    call_id: "call_8b2...",
    output: "Return initiated for order ORD-1001.\nReason: Not powerful enough...\nEstimated refund: $1,484.98 (includes 10% Gold loyalty bonus)"
  },
  created_at: ISODate("2026-04-26T19:14:08.221Z")
}

Three fields earn their keep:

• session_id is the only field every read filters on. It’s the partition key for “this conversation.”
• seq is the integer that makes ordering deterministic. The SDK reads with sort({ seq: 1 }) and writes with an atomic $inc against the matching agent_sessions document, which is what makes concurrent workers safe without a distributed lock.
• message_data is the SDK’s run-item — a user message, tool call, tool output, assistant message, or handoff. Different shape every time. The document model just stores it.

agent_sessions holds one document per session_id with the current high-water seq and timestamps — that’s the counter $inc operates on.

The SDK creates its indexes on first use (per the sessions docs). You’ll see a compound index on (session_id, seq) on agent_messages (the only access pattern the SDK has — fetch ordered history for one session) and a unique index on session_id in agent_sessions.

Production notes

For Atlas, swap the URI for mongodb+srv://... — MongoDBSession accepts it without any other change. If abandoned conversations accumulate, add a TTL index on agent_messages.created_at and old turns retire on their own.

Connection lifetime matters: keep one AsyncMongoClient per process, construct MongoDBSession(client=...) per request, and let the Runner do the rest. Don’t reach for MongoDBSession.from_uri(...) in a web handler — it builds and tears down a client every call. The session needs read/write on the two configured collections (defaults agent_sessions and agent_messages, both overridable via sessions_collection= and messages_collection=). The seq counter keeps concurrent writers safe, but fanning the same session_id across processes will interleave their turns — safe, but probably not what the user meant.

Try it yourself

1
2
3
4
5
6
git clone https://github.com/alexbevi/mongodb-openai-agents-sdk-example
cd mongodb-openai-agents-sdk-example
pip install -r requirements.txt
cp env.example .env          # set OPENAI_API_KEY and MONGODB_URI
python seed_data.py
python main.py

Required env vars: OPENAI_API_KEY, MONGODB_URI (defaults to mongodb://localhost:27017). Demo accounts: alice@example.com (Gold), bob@example.com (Standard), carol@example.com (Platinum).

Where to go next

• The full session API surface — get_items, add_items, pop_item, clear_session, ping — is documented in the Sessions overview, including the MongoDB-specific notes on collection naming and Atlas URIs.
• Wrap your MongoDBSession in OpenAIResponsesCompactionSession once threads grow long; it summarizes old turns server-side and rewrites the underlying session.
• The natural next MongoDB feature for this demo is Atlas Vector Search — store embeddings on support_articles and replace the $text query in search_knowledge_base with $vectorSearch. Same database, same client, one new index.

Durable memory is not raw chat replay. Vertex AI Memory Bank is built for identity-scoped, cross-session personalization and LLM-driven knowledge extraction, and Google’s ADK memory write-up describes Memory Bank as extracting key information from session data rather than replaying every turn. LangChain’s memory model is equally explicit: long-term memory can be semantic (facts), episodic (past actions), or procedural (rules and prompts).

Agent memory should be structured data, not opaque blobs. LangChain stores persist long-term memory as JSON documents, and ADK’s DatabaseSessionService migration moved session serialization from pickle-based storage to JSON-based storage in v1.22.0. MongoDB’s document model matches that reality directly.

MongoDB is a strong fit because retrieval lives in the same system as the memory. MongoDB Vector Search supports both approximate and exact nearest-neighbor search, and the default index type is HNSW. Vector search pre-filters let you scope recall by fields like user_id, tenant_id, or memory_type before embeddings are compared. Hybrid search combines vector and full-text retrieval with reciprocal rank fusion, which is exactly what memory needs when the data mixes natural language with exact identifiers like invoice IDs, feature flags, or product SKUs.

Memory also needs retention and update policy. TTL indexes automatically expire session artifacts, scratchpads, or short-lived summaries. Change streams give you a real-time feed of inserts and updates, which is the right trigger for summarization, entity extraction, or memory distillation jobs. When the data is relationship-heavy instead of chunk-heavy, GraphRAG on MongoDB uses entities, edges, and $graphLookup for relationship-aware, multi-hop retrieval.

This is not a narrow LangChain story. MongoDB publishes integrations for LangGraph, LangChain, LlamaIndex, Semantic Kernel, Haystack, Spring AI, CrewAI, and Vertex AI. LlamaIndex can use MongoDB for the vector store, document store, and index store. Mem0 also supports MongoDB as a memory backend. MongoDB fits the storage contract these frameworks keep converging on: structured documents plus semantic, lexical, and graph-based retrieval.

LangGraph: short-term checkpoints and long-term memory in one database

MongoDB’s LangGraph integration exposes MongoDBSaver for checkpoints and MongoDBStore for durable memory, with optional vector indexing and TTL-based expiry. That maps directly to LangGraph’s own split between thread persistence and store-backed recall.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
from pymongo import MongoClient
from langgraph.checkpoint.mongodb import MongoDBSaver
from langgraph.store.mongodb import MongoDBStore, create_vector_index_config
from langchain_openai import OpenAIEmbeddings

MONGODB_URI = "<connection-string>"

# Short-term memory: thread checkpoints
client = MongoClient(MONGODB_URI)
checkpointer = MongoDBSaver(client)

# Long-term memory: semantic store with metadata filters
index_config = create_vector_index_config(
    embed=OpenAIEmbeddings(model="text-embedding-3-small"),
    dims=1536,
    fields=["content"],
    filters=["user_id", "memory_type"],
)

# Assume `builder` is an existing LangGraph StateGraph
with MongoDBStore.from_conn_string(
    conn_string=MONGODB_URI,
    db_name="agent_memory",
    collection_name="memories",
    index_config=index_config,
    ttl_config={
        "default_ttl": 60 * 60 * 24 * 30,   # 30 days
        "refresh_on_read": True,
    },
) as store:
    graph = builder.compile(checkpointer=checkpointer, store=store)

    store.put(
        namespace=("user-42", "memories"),
        key="pref:vegan:soho",
        value={
            "content": "User prefers vegan restaurants near SoHo.",
            "user_id": "user-42",
            "memory_type": "semantic",
        },
    )

    results = store.search(
        ("user-42", "memories"),
        query="Where should I book dinner tonight?",
        limit=3,
    )

    for result in results:
        print(result.value)

This is the clean MongoDB story: checkpoints for working memory, a store for long-term memory, vector retrieval for recall, metadata filters for isolation, and TTL for automatic cleanup. The same database handles all of it.

LangChain: chat history plus hybrid recall

At the LangChain layer, MongoDB covers both conversation state and retrieval. MongoDBChatMessageHistory persists per-session message history, MongoDBAtlasVectorSearch stores semantic memories, and MongoDBAtlasHybridSearchRetriever fuses lexical and semantic recall.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
from langchain_core.documents import Document
from langchain_mongodb.chat_message_histories import MongoDBChatMessageHistory
from langchain_mongodb.retrievers.hybrid_search import MongoDBAtlasHybridSearchRetriever
from langchain_mongodb.vectorstores import MongoDBAtlasVectorSearch
from langchain_openai import OpenAIEmbeddings

MONGODB_URI = "<connection-string>"

history = MongoDBChatMessageHistory(
    session_id="user-42:thread-7",
    connection_string=MONGODB_URI,
    database_name="agent_memory",
    collection_name="chat_history",
    history_size=20,
)

vector_store = MongoDBAtlasVectorSearch.from_connection_string(
    connection_string=MONGODB_URI,
    namespace="agent_memory.user_memories",
    embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
    index_name="vector_index",
)

vector_store.add_documents(
    [
        Document(
            page_content="User prefers vegan restaurants near SoHo.",
            metadata={"user_id": "user-42", "memory_type": "semantic"},
        ),
        Document(
            page_content="Invoice 8419 was disputed last month.",
            metadata={"user_id": "user-42", "memory_type": "episodic"},
        ),
    ]
)

history.add_user_message("Find dinner options for me in SoHo.")

retriever = MongoDBAtlasHybridSearchRetriever(
    vectorstore=vector_store,
    search_index_name="search_index",
    top_k=5,
    fulltext_penalty=50,
    vector_penalty=50,
)

docs = retriever.invoke("Find dinner options for a vegan user in SoHo")
for doc in docs:
    print(doc.page_content, doc.metadata)

Hybrid retrieval is not optional in a serious memory system. “Prefers vegan restaurants in SoHo” is semantic. “Invoice 8419” is lexical. MongoDB’s hybrid retriever exists because production memory contains both.

Google ADK: the Sessions-and-Memory model maps cleanly to MongoDB

Google ADK makes the architecture explicit. SessionService manages session objects, applies state_delta, and appends event history. MemoryService manages long-term semantic memory across sessions. The Sessions docs currently list InMemorySessionService, VertexAiSessionService, and DatabaseSessionService, so MongoDB is not a built-in backend today. But ADK exposes base session and memory service abstractions, which makes MongoDB a natural implementation target rather than a workaround. Memory Bank then adds the identity-scoped memory semantics on top.

A MongoDB-backed ADK deployment should separate sessions, events, and distilled memories into dedicated collections. That mirrors ADK’s documented split between mutable session state, append-only event history, and searchable long-term memory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
from datetime import datetime, timezone
from pymongo import ASCENDING, MongoClient

MONGODB_URI = "<connection-string>"

client = MongoClient(MONGODB_URI)
db = client["agent_memory"]

sessions = db["adk_sessions"]
events = db["adk_events"]
memories = db["adk_memories"]

sessions.create_index(
    [("app_name", ASCENDING), ("user_id", ASCENDING), ("session_id", ASCENDING)],
    unique=True,
)

events.create_index(
    [
        ("app_name", ASCENDING),
        ("user_id", ASCENDING),
        ("session_id", ASCENDING),
        ("timestamp", ASCENDING),
    ]
)

memories.create_index([("user_id", ASCENDING), ("memory_type", ASCENDING)])
# Create a MongoDB Vector Search index on memories.embedding
# Mark user_id and memory_type as filter fields in the index definition.

def persist_session_turn(app_name: str, user_id: str, session_id: str, event: dict, state: dict) -> None:
    now = datetime.now(timezone.utc)

    sessions.update_one(
        {"app_name": app_name, "user_id": user_id, "session_id": session_id},
        {
            "$set": {"state": state, "updated_at": now},
            "$setOnInsert": {"created_at": now},
        },
        upsert=True,
    )

    events.insert_one(
        {
            "app_name": app_name,
            "user_id": user_id,
            "session_id": session_id,
            "timestamp": event["timestamp"],
            "event": event,
        }
    )

def store_memory(user_id: str, content: str, embedding: list[float], source_session_id: str) -> None:
    memories.insert_one(
        {
            "user_id": user_id,
            "memory_type": "semantic",
            "content": content,
            "embedding": embedding,
            "source_session_ids": [source_session_id],
            "created_at": datetime.now(timezone.utc),
        }
    )

def search_memories(user_id: str, query_embedding: list[float]):
    pipeline = [
        {
            "$vectorSearch": {
                "index": "memory_vector_index",
                "path": "embedding",
                "queryVector": query_embedding,
                "numCandidates": 100,
                "limit": 5,
                "filter": {"user_id": user_id, "memory_type": "semantic"},
            }
        },
        {
            "$project": {
                "content": 1,
                "memory_type": 1,
                "score": {"$meta": "vectorSearchScore"},
            }
        },
    ]
    return list(memories.aggregate(pipeline))

This is a design sketch, not an official ADK adapter. The point is that ADK’s abstractions already describe a storage model MongoDB handles well: mutable session state, append-only events, and searchable long-term memory. Once a MemoryService exists, ADK’s built-in PreloadMemory and LoadMemory tools can use it.

Dedicated memory layers also fit: Mem0 on MongoDB

MongoDB is not only useful when memory is native to the agent framework. Mem0’s MongoDB backend supports MongoDB directly as a vector database for memory storage and retrieval. That matters because it shows MongoDB works both as the application database and as the substrate beneath a dedicated memory layer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from mem0 import Memory

config = {
    "vector_store": {
        "provider": "mongodb",
        "config": {
            "db_name": "mem0_db",
            "collection_name": "mem0_collection",
            "mongo_uri": "<connection-string>",
        },
    }
}

memory = Memory.from_config(config)

messages = [
    {"role": "user", "content": "I'm planning to watch a movie tonight."},
    {"role": "assistant", "content": "What genres do you like?"},
    {"role": "user", "content": "I love sci-fi, not thrillers."},
]

memory.add(messages, user_id="alice", metadata={"category": "movies"})

The architectural value is the same as in LangGraph and LangChain: persistent memory objects, vector retrieval, and application data can live in one operational system instead of being spread across separate services.

Why MongoDB is the best choice here

MongoDB is the best choice when you want one system to hold agent state, long-term memory, retrieval data, and the application records the agent reasons over. The document model matches how current frameworks persist memory. Vector Search and Search cover recall. TTL indexes and change streams cover retention and event-driven memory extraction. GraphRAG covers relationship-heavy data. The result is not “a vector store with extra features.” It is a memory layer that can also be the system of record. That is why MongoDB works as the brain of a modern AI application.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 5: Debugging Audio Issues
• Reverse Engineering Harvester with Ghidra and Codex - Part 6: Timers
• → Reverse Engineering Harvester with Ghidra and Codex - Part 7: Demo Support

Article 6 of 7 in this series.

The Harvester game engine supports time-based functionality through the same scripting system that drives room transitions, object interactions, NPC state changes, and cutscenes.

While re-implementing the engine, one of the main challenges has been figuring out how to debug those systems in a way that gives future Codex prompts useful context. Logs are great after something fires, but timers have a different failure mode: you often need to know whether a timer exists, whether it is currently enabled, how much time is left, and what command chain it will run before it expires.

That is especially important in Harvester because timers are not just visual delays. They can damage the player, monsterfy NPCs, advance dialogue staging, unlock doors, and trigger authored room events.

Timer records in HARVEST.SCR

In part 4 I covered the command-opcode side of the script. Timers sit beside those COMMAND records as their own world records. Once HARVEST.SCR is XOR-decoded, a timer looks like this:

1
initialSeconds TIMER timerName roomName actionTag enabled looping global

For example:

1
2
3
133 TIMER "ACID_TIMER3" "EYEHALL" "HURT_PC_ACIDA" "F" "F" "T"
266 TIMER "ACID_TIMER2" "EYEHALL" "HURT_PC_ACIDB" "F" "F" "T"
400 TIMER "ACID_TIMER"  "EYEHALL" "KILL_PC_ACID"  "F" "F" "T"

The fields map cleanly to the runtime data structure:

The decoded script contains 76 real TIMER records. Most are initially disabled and are started by a nearby COMMAND, but a few start enabled as part of room setup.

At runtime, these timers are materialized as invisible runtime entities. That turned out to be an important detail: timer state is not only a field in the parsed script record. A live countdown exists in the room entity list, and global timers can survive room changes by preserving that live entity instead of destroying it with the rest of the room.

When a timer entity expires, the room loop records the expired timer name, resolves the backing TimerRecord, and dispatches the record’s actionTag. In other words, the timer name is just the lookup key. The action is whatever command chain is stored in the timer record.

Starting and stopping timers

Script command chains control timers with SET_TIMER and KILL_TIMER:

1
2
3
COMMAND "SET_HALL_TIMER"   "SET_TIMER" "ACID_TIMER"  "ON" "" "KILL_TRIG_TIMER"
COMMAND "SET_HALL_2TIMER"  "SET_TIMER" "ACID_TIMER2" "ON" "" "SET_HALL_2TIMER2"
COMMAND "SET_HALL_2TIMER2" "SET_TIMER" "ACID_TIMER3" "ON" "" ""

SET_TIMER ... ON enables a timer. If the timer was previously disabled, the current value is reset back to the initial value. SET_TIMER ... OFF and KILL_TIMER disable it.

The subtle part is that timer commands often live in longer command chains. They are not isolated API calls. Starting a timer might be one step in a room-entry sequence, and the timer’s expiry might run another command chain that branches on flags, shows text, adjusts HP, or changes rooms.

In the MAINHALL with DEBUG_TIMERS and DEBUG_ROOM toggled

A good example is when you enter the MAINHALL on Disc 3, because it uses multiple global timers with staggered deadlines.

1
2
3
4
5
6
7
8
9
133 TIMER "ACID_TIMER3" "EYEHALL" "HURT_PC_ACIDA" "F" "F" "T"
266 TIMER "ACID_TIMER2" "EYEHALL" "HURT_PC_ACIDB" "F" "F" "T"
400 TIMER "ACID_TIMER"  "EYEHALL" "KILL_PC_ACID"  "F" "F" "T"

COMMAND "HURT_PC_ACIDA"   "KILL_TIMER" "ACID_TIMER3" "" "" "HURT_PC_ACID"
COMMAND "HURT_PC_ACIDB"   "KILL_TIMER" "ACID_TIMER3" "" "" "HURT_PC_ACID"
COMMAND "HURT_PC_ACID"    "CHECK_FLAG" "CLEANED_CLOTHES" "" "HURT_PC_ACID_1" ""
COMMAND "HURT_PC_ACID_1"  "SHOW_TEXT"  "ACID_TEXT2" "" "" "HURT_PC_ACID_2"
COMMAND "HURT_PC_ACID_2"  "ADJ_HP"     "-7" "" "" ""

The first two timers are warning/damage stages. When they expire, they enter HURT_PC_ACID, which checks the CLEANED_CLOTHES flag. If the clothes have not been cleaned, the game shows acid text and subtracts 7 HP. If the flag has been set, the branch target is empty and the command chain stops.

The last timer, ACID_TIMER, is the hard fail path:

1
400 TIMER "ACID_TIMER" "EYEHALL" "KILL_PC_ACID" "F" "F" "T"

Cleaning the clothes is itself just another scripted interaction. Using the money on the cloakroom attendant starts CLEAN_CLOTHES, which shows text, sets the flag, removes the money, and disables the lethal acid timer:

1
2
3
4
5
6
7
8
USEITEM "BARCASHFIVE" "CLOAKROOM" "CLOAK_ATND" "CLEAN_CLOTHES"

COMMAND "CLEAN_CLOTHES"   "SHOW_TEXT" "CLEANED_CLOTHS" "" "" "CLEAN_CLOTHES_1"
COMMAND "CLEAN_CLOTHES_1" "SET_FLAG"  "CLEANED_CLOTHES" "T" "" "CLEAN_CLOTHES_2"
COMMAND "CLEAN_CLOTHES_2" "DELETE"    "CLOAKROOM" "BARCASHFIVE" "" "CLEAN_CLOTHES_3"
COMMAND "CLEAN_CLOTHES_3" "DELETE"    "INVENTORY" "BARCASHFIVE" "" "CLEAN_CLOTHES_4"
COMMAND "CLEAN_CLOTHES_4" "SET_TIMER" "ACID_TIMER" "OFF" "" "CLEAN_CLOTHES_5"
COMMAND "CLEAN_CLOTHES_5" "KILL_TIMER" "ACID_TIMER" "" "" ""

This is the kind of script graph that is hard to reason about from static records alone. Some of the authored data is also a little odd: both intermediate acid chains kill ACID_TIMER3, even though the second one is entered by ACID_TIMER2. Watching the live timers makes it much easier to tell whether that is harmless authored data, a reimplementation bug, or a broken state sync.

Adding a timer overlay

To make this easier to debug, I asked Codex to add a console command that would render active timers directly over the room:

the game manages timers periodically. I want to introduce a DEBUG_TIMERS command that when enabled it will overlay text on the screen that will be the timer name, starting value, current value, action to take when timer expires.

If multiple timers are active, they should appear one after the other. Draw this near the middle-left of the screen. It should be white text on a black background

Instead of wiring this up myself, Codex was able to do it in about 5 minutes.

The implementation is intentionally small:

• DEBUG_TIMERS toggles a boolean on the Harvester engine.
• The room renderer checks that boolean after drawing the room and other debug overlays.
• The overlay walks the known timer records, finds matching live timer entities, filters out disabled timers, and formats each row as name start=initial current=current action=tag.
• Text is drawn near the middle-left of the screen using white text over a black rectangle.

This means the overlay is showing live runtime state, not just parsed script data. If a timer is missing from the live entity list, disabled, or no longer counting down, it disappears.

The label format is deliberately boring:

1
2
3
ACID_TIMER3 start=133 current=128 action=HURT_PC_ACIDA
ACID_TIMER2 start=266 current=261 action=HURT_PC_ACIDB
ACID_TIMER start=400 current=395 action=KILL_PC_ACID

That is exactly the information I need while debugging: what is active, how long it has left, and what will happen when it fires.

Having a visual indicator for timer progress, and whether those timers were triggering the correct actions, made it much easier to discover some more obscure bugs.

These log lines are all coming from the game’s scripts. In this run, the clothes have been cleaned, so the acid timers still reach their action chains, but the CLEANED_CLOTHES flag prevents the HP penalty path from continuing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Harvester: action tag 'CLEAN_CLOTHES' step=0 tag='CLEAN_CLOTHES' opcode='SHOW_TEXT' args=['CLEANED_CLOTHS','','','CLEAN_CLOTHES_1']
Harvester: action tag 'CLEAN_CLOTHES_1' step=0 tag='CLEAN_CLOTHES_1' opcode='SET_FLAG' args=['CLEANED_CLOTHES','T','','CLEAN_CLOTHES_2']
Harvester: action tag 'CLEAN_CLOTHES_1' SET_FLAG 'CLEANED_CLOTHES' 0 -> 1 existed=1 changed=1
Harvester: action tag 'CLEAN_CLOTHES_1' step=1 tag='CLEAN_CLOTHES_2' opcode='DELETE' args=['CLOAKROOM','BARCASHFIVE','','CLEAN_CLOTHES_3']
Harvester: action tag 'CLEAN_CLOTHES_1' step=2 tag='CLEAN_CLOTHES_3' opcode='DELETE' args=['INVENTORY','BARCASHFIVE','','CLEAN_CLOTHES_4']
Harvester: action tag 'CLEAN_CLOTHES_1' step=3 tag='CLEAN_CLOTHES_4' opcode='SET_TIMER' args=['ACID_TIMER','OFF','','CLEAN_CLOTHES_5']
Harvester: action tag 'CLEAN_CLOTHES_1' step=4 tag='CLEAN_CLOTHES_5' opcode='KILL_TIMER' args=['ACID_TIMER','','','']

Harvester: timer command 'ACID_TIMER3' step=0 tag='HURT_PC_ACIDA' opcode='KILL_TIMER' args=['ACID_TIMER3','','','HURT_PC_ACID']
Harvester: timer command 'ACID_TIMER3' step=1 tag='HURT_PC_ACID' opcode='CHECK_FLAG' args=['CLEANED_CLOTHES','','HURT_PC_ACID_1','']
Harvester: timer command 'ACID_TIMER3' flag 'CLEANED_CLOTHES' -> 1

Harvester: timer command 'ACID_TIMER2' step=0 tag='HURT_PC_ACIDB' opcode='KILL_TIMER' args=['ACID_TIMER3','','','HURT_PC_ACID']
Harvester: timer command 'ACID_TIMER2' step=1 tag='HURT_PC_ACID' opcode='CHECK_FLAG' args=['CLEANED_CLOTHES','','HURT_PC_ACID_1','']
Harvester: timer command 'ACID_TIMER2' flag 'CLEANED_CLOTHES' -> 1

The overlay was also useful outside this one hallway. Some NPC state changes are timer-driven too:

1
2
3
4
5
COMMAND "START_INQ_TIM" "SET_TIMER" "INQUIST_ATTACK_TIMER" "ON" "" ""
120 TIMER "INQUIST_ATTACK_TIMER" "PAIN" "MNSTFY_INQUIST" "F" "F" "F"

COMMAND "START_MERCY_TIMR" "SET_TIMER" "GLADIATOR_TIMER" "ON" "" ""
30 TIMER "GLADIATOR_TIMER" "MERCY" "MNST_GLAD" "F" "F" "F"

Those are awkward to verify with logs alone because nothing visible happens until the timer expires. With DEBUG_TIMERS enabled, I can start the dialogue branch, see the countdown appear, wait for it to hit zero, and then check whether the expected monsterfy command chain ran.

That has become the general debugging pattern for this engine work: use Ghidra and the decoded script to understand the original data model, add small runtime instrumentation when the model is too indirect to observe comfortably, then feed the resulting logs, screenshots, and videos back into the next prompt.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 4: Command Opcodes
• Reverse Engineering Harvester with Ghidra and Codex - Part 5: Debugging Audio Issues
• → Reverse Engineering Harvester with Ghidra and Codex - Part 6: Timers

Article 5 of 7 in this series.

An issue I’ve noticed with the harvester engine implementation is that every time there’s an audio sample played, it starts with a distinct “popping” sound. I thought this could be an issue with how I’d approached the re-implementation or disassembly, but after a few passes at decompiling the audio code I’d always end up with the same result.

While doing some unrelated research I stumbled on Kostya’s Boring Codec World and a post he’d made on FutureVision formats, discussing Harvester audio/video specifically:

“Music format is based on IMA ADPCM with minimalistic file header and the default predictor. Which means it needs some time to adapt to the actual coming signal amplitudes. In result the old format description based on reverse engineering recommends skipping first 7-57 bytes right after the header because of the garbage sound it produces. When I looked into the binary reference (an adventure that will be described in an upcoming post), it turned out that in certain cases they simply zero out the first 50 decoded samples.”

Since our approach to do everything with Ghidra/Codex, I thought this would be a good opportunity to try just prompting GPT-5.4 with a bit more context to see what it would come back with. I tried feeding the following prompt as a result:

1
2
3
based on https://codecs.multimedia.cx/2021/01/a-look-on-futurevision-formats/ "Music format is based on IMA ADPCM with minimalistic file header and the default predictor. Which means it needs some time to adapt to the actual coming signal amplitudes. In result the old format description based on reverse engineering recommends skipping first 7-57 bytes right after the header because of the garbage sound it produces. When I looked into the binary reference (an adventure that will be described in an upcoming post), it turned out that in certain cases they simply zero out the first 50 decoded samples."

Can we look into how realistic this may be based on what's in ghidra? The audio still produces a popping sound on each sample being played, so maybe this might help address that

After processing for a few minutes, the response back was promising:

1
2
3
4
5
6
7
8
Yes. Based on live Ghidra, the article is not just plausible, it matches what HARVEST.LE actually does.

start_music_stream at 0x1bfc0 decodes the first FCMP chunk and then zeroes the first 0x64 decoded bytes before queueing it. On the common 16-bit streams, that is 50 samples.

load_sound_sample at 0x18470 and load_dialogue_voice_sample at 0x191d0 are even more aggressive: they queue from decoded_pcm + 0x1f4, so they discard the first 500 decoded bytes of one-shot/dialogue FCMP playback.
The current ScummVM Harvester decoder was doing neither, so a clip-start pop on CMP-backed audio is consistent with that mismatch.

I patched media_manager.cpp (line 127) to mirror the native split: music now mutes the first 100 decoded bytes, while startup speech/other FCMP playback trims the first 500 decoded bytes. WAV handling is unchanged. 

The applyStartupFcmpWarmup function that this prompt resulted in effectively addressed the issue we were experiencing.

There’s still a slight pop on the first FST video being loaded, but all scene sound effects and dialogue interactions are now pop-free.

The patch is below to show how this was incorporated into the engine’s media manager.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
diff --git a/engines/harvester/media_manager.cpp b/engines/harvester/media_manager.cpp
index 691863468fa6..e953c0226cba 100644
--- a/engines/harvester/media_manager.cpp
+++ b/engines/harvester/media_manager.cpp
@@ -41,6 +41,11 @@ namespace Harvester {
 
 namespace {
 
+enum class FcmpWarmupMode {
+	kMusic,
+	kSample
+};
+
 static const int8 kHarvesterImaIndexAdjustTable[16] = {
 	-1, -1, -1, -1, 2, 4, 6, 8,
 	-1, -1, -1, -1, 2, 4, 6, 8
@@ -119,7 +124,29 @@ static uint32 decodeHarvesterFcmp(byte *dest, const byte *src, uint32 srcSize, u
 	return decodedSize;
 }
 
-static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableReadStream *stream) {
+static void applyStartupFcmpWarmup(byte *decodedPcm, uint32 &decodedSize, FcmpWarmupMode warmupMode) {
+	if (!decodedPcm || decodedSize == 0)
+		return;
+
+	if (warmupMode == FcmpWarmupMode::kMusic) {
+		// Native start_music_stream zeros the first 0x64 decoded bytes of the first FCMP chunk.
+		memset(decodedPcm, 0, MIN<uint32>(decodedSize, 100));
+		return;
+	}
+
+	// Native load_sound_sample/load_dialogue_voice_sample queue from decoded_pcm + 0x1f4.
+	const uint32 trimBytes = MIN<uint32>(decodedSize, 500);
+	if (trimBytes == decodedSize) {
+		memset(decodedPcm, 0, decodedSize);
+		return;
+	}
+
+	memmove(decodedPcm, decodedPcm + trimBytes, decodedSize - trimBytes);
+	decodedSize -= trimBytes;
+}
+
+static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableReadStream *stream,
+		FcmpWarmupMode warmupMode) {
 	if (!stream)
 		return nullptr;
 
@@ -156,7 +183,7 @@ static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableRead
 				return nullptr;
 			}
 
-			const uint32 decodedSize = payloadSize * (bitsPerSample >> 2);
+			uint32 decodedSize = payloadSize * (bitsPerSample >> 2);
 			byte *decodedPcm = (byte *)malloc(decodedSize);
 			if (!decodedPcm) {
 				free(compressedPayload);
@@ -165,6 +192,7 @@ static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableRead
 			}
 
 			decodeHarvesterFcmp(decodedPcm, compressedPayload, payloadSize, bitsPerSample);
+			applyStartupFcmpWarmup(decodedPcm, decodedSize, warmupMode);
 			free(compressedPayload);
 			delete stream;
 			return Audio::makeRawStream(decodedPcm, decodedSize, sampleRate,
@@ -177,8 +205,9 @@ static Audio::SeekableAudioStream *decodeStartupAudioStream(Common::SeekableRead
 	return Audio::makeWAVStream(stream, DisposeAfterUse::YES);
 }
 
-static Audio::SeekableAudioStream *openStartupAudioStream(ResourceManager &resources, const Common::String &path) {
-	return decodeStartupAudioStream(resources.openFile(path));
+static Audio::SeekableAudioStream *openStartupAudioStream(ResourceManager &resources,
+		const Common::String &path, FcmpWarmupMode warmupMode) {
+	return decodeStartupAudioStream(resources.openFile(path), warmupMode);
 }
 
 } // End of anonymous namespace
@@ -268,7 +297,8 @@ bool MediaManager::playMusic(const Common::String &path) {
 		return true;
 	}
 
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kMusic);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup music '%s'", path.c_str());
 		return false;
@@ -323,7 +353,8 @@ bool MediaManager::playSound(const Common::String &path) {
 	_soundSlotIndex = (_soundSlotIndex + 1) % ARRAYSIZE(_soundHandles);
 	stopSoundHandle(_soundHandles[_soundSlotIndex]);
 
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup sound '%s'", path.c_str());
 		return false;
@@ -340,7 +371,8 @@ bool MediaManager::playSingleSound(const Common::String &path) {
 		return false;
 
 	stopSoundHandle(_singleSoundHandle);
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup sound '%s'", path.c_str());
 		return false;
@@ -364,7 +396,8 @@ bool MediaManager::playSpeech(const Common::String &path) {
 		return false;
 
 	stopSoundHandle(_speechHandle);
-	Audio::SeekableAudioStream *audioStream = openStartupAudioStream(_resources, path);
+	Audio::SeekableAudioStream *audioStream =
+		openStartupAudioStream(_resources, path, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup speech '%s'", path.c_str());
 		return false;
@@ -410,7 +443,7 @@ bool MediaManager::playLoadedSound(int slot) {
 
 	Common::SeekableReadStream *stream = new Common::MemoryReadStream(
 		_loadedSoundData[slot].data(), _loadedSoundData[slot].size(), DisposeAfterUse::NO);
-	Audio::SeekableAudioStream *audioStream = decodeStartupAudioStream(stream);
+	Audio::SeekableAudioStream *audioStream = decodeStartupAudioStream(stream, FcmpWarmupMode::kSample);
 	if (!audioStream) {
 		warning("Harvester: unable to decode startup sound slot %d ('%s')",
 			slot, _loadedSoundPaths[slot].c_str());

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 3: File Formats
• Reverse Engineering Harvester with Ghidra and Codex - Part 4: Command Opcodes
• → Reverse Engineering Harvester with Ghidra and Codex - Part 5: Debugging Audio Issues

Article 4 of 7 in this series.

Harvester’s startup / world script is not bytecode. It is XOR-obfuscated text, and opcode dispatch happens through COMMAND records in HARVEST.SCR:

1
COMMAND triggerTag opcodeName arg1 arg2 arg3 [arg4]

In the original game and in ScummVM, these opcode names come from the data pipeline, not from a compiled bytecode table:

• In ScummVM, Script::load() reads HARVEST.SCR, decode() XOR-deobfuscates it, and parseTownRecords() turns it into typed startup records.
• Within parseTownRecords(), COMMAND lines are parsed into StartupCommandRecord entries alongside ROOM, OBJECT, REGION, TIMER, USEITEM, and related world records.
• Those other records provide the entry labels for command chains through room setup/exit, interactions, and timers.
• findCommandRecord() and executeCommandChain() resolve the current label to a COMMAND record, decode the opcode name on that line, and dispatch into engine handlers for room flow, media, inventory, actor state, and other subsystems.

Command Labels

triggerTag

triggerTag is the label attached to one COMMAND record. It is the string used to find that record later.

• The parser stores it from the token immediately after COMMAND in parseTownRecords().
• Script::findCommandRecord() resolves a command by comparing the requested tag string against command.triggerTag.

So triggerTag is not a condition and not an opcode argument in the behavioral sense. It is the command node’s name.

currentTag

currentTag is the interpreter’s working variable while it walks a command chain.

• executeCommandChain() initializes currentTag from the caller-supplied starting tag.
• It then resolves the current command with findCommandRecord(currentTag).
• After each opcode runs, currentTag is updated to the next label:
  • branch opcodes like CHECK_FLAG and CHECK_PERC set it from arg2 or arg3
  • most linear opcodes continue to arg4
  • deferred opcodes may stash arg4 as a continuation tag and return to the caller instead of immediately continuing

If you think of the script as a graph, triggerTag is the node name stored in the file, and currentTag is the interpreter’s current node pointer.

Where Starting Tags Come From

The interpreter does not enter command chains automatically just because a COMMAND record exists. Some other game record must point at its label.

Common entry points in the current engine:

• object interaction uses object.actionTag
• region interaction uses region.actionTag
• use-item interaction uses useItem.actionTag
• room enter / exit uses room->onEnterCommand and room->onExitCommand
• timer execution starts from timer->arg2

That means a more precise reading of the format is:

1
COMMAND label opcodeName arg1 arg2 arg3 [nextLabel]

with the caveat that arg2, arg3, and arg4 are opcode-specific, so only some of them are actually labels for a given opcode.

Examples

Example 1: straight-line chain

1
2
COMMAND "OPEN_GATE" "SET_FLAG" "GATE_OPEN" "T" "" "OPEN_GATE_TEXT"
COMMAND "OPEN_GATE_TEXT" "SHOW_TEXT" "Gate_Is_Open" "" "" ""

If an object’s actionTag is "OPEN_GATE":

1. executeCommandChain() starts with currentTag = "OPEN_GATE".
2. findCommandRecord("OPEN_GATE") resolves the first line because its triggerTag is "OPEN_GATE".
3. SET_FLAG runs and then sets currentTag = arg4, which is "OPEN_GATE_TEXT".
4. findCommandRecord("OPEN_GATE_TEXT") resolves the second line.
5. SHOW_TEXT runs. Because it is deferred, the interpreter returns control to the caller instead of continuing immediately.

Example 2: branch on a flag

1
2
3
COMMAND "TRY_SHED" "CHECK_FLAG" "HAS_SHED_KEY" "SHED_OPEN" "SHED_LOCKED"
COMMAND "SHED_OPEN" "CHANGE_ROOM" "SHED_INT" "" "" ""
COMMAND "SHED_LOCKED" "SHOW_TEXT" "Need_A_Key" "" "" ""

If the chain starts at "TRY_SHED":

1. currentTag starts as "TRY_SHED".
2. CHECK_FLAG looks up HAS_SHED_KEY.
3. If the flag is true, currentTag becomes arg2, so the next lookup is "SHED_OPEN".
4. If the flag is false, currentTag becomes arg3, so the next lookup is "SHED_LOCKED".

So here the first COMMAND line is acting like a named branch node.

Example 3: deferred opcode with continuation

1
2
COMMAND "POTTS_EVENT" "GOFLIC" "GRAPHIC/FST/C001B.FST" "" "" "POTTS_AFTER_MOVIE"
COMMAND "POTTS_AFTER_MOVIE" "SET_FLAG" "STEPH_MIDGAME_PLAYED" "T" "" ""

When currentTag reaches "POTTS_EVENT":

1. GOFLIC does not immediately jump to "POTTS_AFTER_MOVIE".
2. Instead, it stores arg4 as a continuation tag and returns the movie request to the caller.
3. After the cutscene finishes, room/dialogue code can resume by starting another command-chain execution at "POTTS_AFTER_MOVIE".

That is why arg4 is often best read as “the next tag after this opcode completes”, not just “the next line”.

Most of the opcode recognition below lives in Script::executeCommandChain(), while deferred outputs such as modal text, dialogue continuations, lighting changes, player moves, and follow-up tags are consumed by the room interaction processor in room.cpp.

Control Flow And Transitions

World And Runtime State

Player And UI

Audio

The audio opcodes all share the same queueing path through appendStartupAudioCommand() and are later handed off to Flow::executeStartupAudioCommands().

Observed Engine-Side Aliases And Shared Paths

• HEAL_PC and ADJ_HP currently share the exact same implementation.
• KILL_NPC and MONSTERFY share the same base handler; MONSTERFY additionally activates the linked monster target when present.
• CLOSEUP and CHANGE_ROOM share the same transition-output path, differing only in the transition kind reported to room logic.

This review is part of the Reverse Engineering Harvester series, where I document my journey of reverse engineering the 1996 DOS game Harvester to re-implement its game engine in ScummVM.

• ← Reverse Engineering Harvester with Ghidra and Codex - Part 2
• Reverse Engineering Harvester with Ghidra and Codex - Part 3: File Formats
• → Reverse Engineering Harvester with Ghidra and Codex - Part 4: Command Opcodes

Article 3 of 7 in this series.

File Formats

This document tracks file-format facts that are confirmed by the current Harvester reverse-engineering work. It is intentionally conservative: if a field or behavior is not supported by code or verified runtime notes, it is left out.

The sections below combine three sources of evidence: the current ScummVM Harvester engine code, named native functions and data types in Ghidra, and spot-checks against extracted sample files from the Harvester CD image. Unknown or only partially understood fields stay marked as such.

DAT (XFILE archive payload)

Harvester’s .DAT files act as payload containers for the game’s numbered XFILE resource sets. The important wrinkle is that the .DAT file is not the whole format by itself: the loader depends on a sidecar INDEX.00N file for the directory. In other words, the .DAT holds the bytes, while the INDEX.00N records tell the engine where each member starts, how large it is, and whether it must be unpacked after reading.

At confirmed cold start, the original game mounts these pairs in this order:

Summary

.DAT payload layout

This is sparse compared with many archive formats. The format’s structure is directory-driven rather than self-describing. All meaningful per-file metadata currently comes from the matching INDEX.00N file.

INDEX.00N directory record layout

Each confirmed sidecar index is read as a flat array of 0x94-byte records:

Packed entry stream

When packed_flag != 0, the loader reads stored_size bytes from the .DAT and expands them into an unpacked_size output buffer using a simple control-byte stream:

The decode loop stops once either the compressed input is exhausted or the output buffer reaches unpacked_size. Entries that fail to produce exactly unpacked_size bytes are treated as invalid.

Path handling notes

Practical interpretation

For extraction or tooling work, the current evidence supports treating Harvester’s .DAT archives as a two-file format:

1. Read the matching INDEX.00N file as a sequence of 0x94-byte XFLE records.
2. Use each record’s archive_offset and stored_size to slice bytes from the companion .DAT.
3. If packed_flag is zero, the slice is the final file.
4. If packed_flag is nonzero, expand the slice with the literal/repeat decoder above until unpacked_size bytes are produced.

That model is enough to explain the startup resource mounts already confirmed in the reverse-engineering notes, and it matches the current ScummVM-side archive loader.

RCS (Quick-tip text lists)

The only confirmed .RCS use so far is ADJHEAD.RCS, the quick-tips file shown by the startup/options overlay. Both native run_quick_tips_screen and the ScummVM Flow::loadQuickTips path treat it as plain text rather than as a binary container.

Summary

File layout

Notes

• The sampled ADJHEAD.RCS begins directly with readable text: Double click the left mouse button....
• The native quick-tips path advances through the file with read_line_from_file_stream, wraps back to the start on EOF, and chooses a random starting point the first time it runs.
• This is a good example of Harvester’s resource pipeline staying pragmatic: not every gameplay-facing asset is wrapped in a custom binary format.

DIALOG.RSP (Dialogue response and keyword text)

DIALOG.RSP is a plaintext line table used by the dialogue UI. It does not appear to contain ids, offsets, or a binary header; the engine indexes it by zero-based line number and then uses the line text directly.

Summary

File layout

Notes

• The native loader reads a requested zero-based line, strips the trailing CR/LF, and returns the shared text buffer directly.
• run_dialogue_response_menu and run_dialogue_keyword_menu split visible options on /; there is no confirmed hidden topic-id layer behind those labels.
• load_dialogue_index also pulls zero-based line 13 from DIALOG.RSP to seed the default keyword topic; in the sampled file that line is BYE (one-based line 14).
• DIALOG.RSP shows Harvester leaning on designer-editable text tables even inside a fairly custom dialogue stack.

DIALOGUE.IDX (Subtitle index)

DIALOGUE.IDX is an XOR-obfuscated text file that maps voice sample ids to subtitle strings. The native load_dialogue_index path and the ScummVM Text::loadDialogueIndex reimplementation both decode it with XOR 0xAA while leaving CR/LF intact.

Summary

Decoded stream layout

Derived index entry

Notes

• After XOR decode, the sampled file starts with a simple alternating pattern: 1, "junk init", 7, "Yes?", 11, "I need some help...", and so on.
• The native loader builds a 3000-entry in-memory table and keeps the decoded text blob around so play_dialogue_line can seek back into it cheaply.
• Architecturally, this splits dialogue cleanly in two: DIALOGUE.IDX carries spoken-line subtitles keyed by numeric wav ids, while DIALOG.RSP carries menu-facing response text.

1
2
3
4
5
6
7
8
9
10
11
python3 -c 'import sys,pathlib,signal; signal.signal(signal.SIGPIPE, signal.SIG_DFL); d=pathlib.Path(sys.argv[1]).read_bytes(); sys.stdout.buffer.write(bytes(b if b in (10,13) else b ^ 0xAA for b in d))' DIALOGUE.IDX | head -n 10
1
"junk init"
7
"Yes?"
11
"I need some help... Mister...?"
15
"Postmaster Boyle.  What can I do you for today?"
25
"Sorry, youngster, we're out of applications right now."

SCR (Town/world startup script)

HARVEST.SCR is the central world-definition and startup-script file. It is XOR-obfuscated text, not bytecode. The native load_xor_obfuscated_town_script path and the ScummVM Script::decode path both XOR each non-CR/LF byte with 0xAA, then parse whitespace-separated records with quoted strings preserved.

Summary

Confirmed record forms

Notes

• The sampled file starts with readable structural records immediately after XOR decode, for example ENTRANCE, OBJECT, and later ROOM, TEXT, HEAD, and COMMAND.
• Paths embedded in script records are not uniform: archive-backed resources use 1:\..., 2:\..., 3:\..., while some direct-file assets are bare relative paths such as dialogue.idx.
• GOFLIC and GODEATHFLIC are especially revealing command names. In the sampled script, those opcodes point to .FST paths, which suggests the script vocabulary preserved older naming while the shipping runtime movie path used FST files.
• HARVEST.SCR shows that Harvester’s resource architecture is data-driven at the top level: rooms, objects, dialogue portraits, commands, music, palettes, and cutscene triggers all converge here.

1
2
3
4
5
6
7
8
9
10
11
python3 -c 'import sys,pathlib,signal; signal.signal(signal.SIGPIPE, signal.SIG_DFL); d=pathlib.Path(sys.argv[1]).read_bytes(); sys.stdout.buffer.write(bytes(b if b in (10,13) else b ^ 0xAA for b in d))' HARVEST.SCR | head -n 10
{// HARVESTER (c) 1995-96 Scripting Language
{// Town
        0   0   0   ENTRANCE "BACK" "TOWN_2_LODGE" "SERGEANT_2_LODGE"
        0   0   0   ENTRANCE "FRONT" "" "SAVE_GAME"
        492 67  554 96   0  1 OBJECT "INVENTORY" "INV_EXIT" "" "" "" "" "" "X" "F" "T" "" "Inventory"
        299 0   0   0    2  1 OBJECT "NULL_ID"          "EXIT_BM"  "1:\GRAPHIC\OTHER\EXITSIGN.BM" "" "" "" ""  "" "F" "T" "" "exit"
        299 0   375 61   0  1 OBJECT "NULL_ID"          "EXIT_HS"  ""                             "" "" "" "X" "" "T" "T" "" "exit"

{// inventory health indicator
     72 314 0   0  -12  1 OBJECT "INVENTORY" "INV_STAT1" "1:\graphic\other\head-a1.bm" "" "" "INV_STAT_ST" "" "" "F" "F" "" "Health_Indicator"

BM (Static indexed bitmap)

.BM is Harvester’s simplest custom image format: a small fixed header followed by raw 8-bit indexed pixels. It is used for UI panels, portraits, inventory art, help screens, and other non-animated images.

Summary

File layout

Notes

• The native binary reuses the same RawBitmap/BitmapBuffer shape in memory: { width, height, pixels }.
• The format itself does not encode transparency, but many callers render it with palette index 0 treated as transparent.
• Compared with many later adventure engines, this is an aggressively direct format: no chunking, no palette sidecar inside the file, and no per-row metadata.

PAL (Standalone palette)

Harvester’s .PAL files are raw palette payloads with no header. The native palette upload helpers and ScummVM Art::loadPalette both treat them as 256 RGB triplets.

Summary

File layout

Notes

• Native upload_palette_to_vga forces palette index 0 to black at upload time, applies brightness scaling, and then shifts the stored 8-bit channels down to VGA’s 0..63 DAC range.
• The sampled INVHELP.PAL contains full-range byte values up to 0xff, which matches the native analysis that .PAL is stored as 8-bit RGB, not pre-divided 6-bit VGA values.
• This makes .PAL a nice contrast with FST’s embedded movie palettes, which are stored in 6-bit VGA form inside each frame payload.

TRY IT OUT!

1
2
python3 -m pip install pillow
python3 -c 'from PIL import Image; import sys,struct; d=open(sys.argv[1],"rb").read(); w,h=struct.unpack_from("<II",d,0); i=Image.frombytes("P",(w,h),d[12:12+w*h]); p=open(sys.argv[2],"rb").read()[:768] if len(sys.argv)>2 else bytes(c for n in range(256) for c in (n,n,n)); i.putpalette(p); i.show()' /path/to/*.BM [/path/to/*.PAL]

python3 -c '...' GRAPHIC/OTHER/INVHELP.BM

python3 -c '...' GRAPHIC/OTHER/INVHELP.BM GRAPHIC/PAL/INVHELP.PAL

CFT (Bitmap font)

.CFT packages a bitmap font as a fixed metrics header plus one raw 8-bit atlas image. The native load_font_resource and the ScummVM Text::loadFont / HarvesterCftFont code agree on the basic structure.

Summary

File layout

Notes

• The renderer derives each glyph by slicing width pixels from x inside the shared atlas. There is no per-glyph bitmap chunking.
• drawHeight is effectively atlasHeight - 1 in the current font renderer, which matches the native behavior of treating the last row as non-drawing padding.
• The font renderer treats both ' ' and '_' as space-width characters. That ties neatly back to the script/text resources, where underscore-heavy identifiers and UI labels coexist with visible text.

TRY IT OUT!

1
python3 -c 'from PIL import Image; import sys,struct; d=open(sys.argv[1],"rb").read(); aw,ah=struct.unpack_from("<II",d,0x448); h=max(1,min(struct.unpack_from("<H",d,0x40)[0] or ah-1,ah-1)); sw=struct.unpack_from("<H",d,0x442)[0] or 1; s=[struct.unpack_from("<H",d,0x42+i*2)[0] for i in range(256)]; w=[struct.unpack_from("<H",d,0x242+i*2)[0] for i in range(256)]; a=Image.frombytes("L",(aw,ah),d[0x454:0x454+aw*ah]); t=sys.argv[2]; W=sum(sw if ord(c)>255 or c in " _" or w[ord(c)]<=0 else w[ord(c)] for c in t); i=Image.new("L",(max(1,W),h),0); x=0; exec("for c in t:\n o=ord(c)\n gw=sw if o>255 or c in \" _\" or w[o]<=0 or s[o]>=aw else w[o]\n if gw and o<=255 and c not in \" _\" and w[o]>0 and s[o]<aw: i.paste(255,(x,0,x+gw,h),a.crop((s[o],0,s[o]+gw,h)))\n x+=gw"); i.resize((max(1,i.width*4),max(1,i.height*4)),Image.NEAREST).show()' "GRAPHIC/FONT/HARVFNT2.CFT" "Hello World"

ABM (Animated bitmap / sprite strip)

.ABM is Harvester’s main custom sprite/animation format. It backs cursor art, actor sprites, room animations, combat entities, and wait-overlay animation frames.

Summary

File header

Per-frame record

Compressed frame stream

Notes

• Each frame decodes to exactly width * height bytes of indexed pixels.
• The sampled BLOOD.ABM starts with 5 frames; its first frame is offset (4, 0), size 38 x 50, and marked compressed.

Try it out!

1
python3 -c $'from PIL import Image\nimport sys,struct,tempfile,pathlib,webbrowser\n\ndef dec(s,n):\n i=0; o=bytearray()\n while i<len(s) and len(o)<n:\n  k=s[i]; i+=1\n  if k<128:\n   c=min(k,len(s)-i,n-len(o)); o+=s[i:i+c]; i+=c\n  elif i<len(s):\n   o.extend([s[i]]*min(k&127,n-len(o))); i+=1\n return bytes(o)\n\nd=open(sys.argv[1],"rb").read()\npal=open(sys.argv[2],"rb").read()[:768] if len(sys.argv)>2 else bytes(c for n in range(256) for c in (n,n,n))\nfc=struct.unpack_from("<I",d,0)[0]; off=8; F=[]; minx=miny=10**9; maxx=maxy=-10**9\nfor _ in range(fc):\n x,y,w,h=struct.unpack_from("<iiii",d,off); c=d[off+16]; n=struct.unpack_from("<I",d,off+17)[0]; s=d[off+25:off+25+n]; off+=25+n; p=s[:w*h] if not c else dec(s,w*h); F.append((x,y,w,h,p)); minx=min(minx,x); miny=min(miny,y); maxx=max(maxx,x+w); maxy=max(maxy,y+h)\nminx=min(0,minx); miny=min(0,miny); W=maxx-minx; H=maxy-miny; G=[]\nfor x,y,w,h,p in F:\n src=Image.frombytes("P",(w,h),p); src.putpalette(pal); m=src.point(lambda v:0 if v==0 else 255,"L"); fr=Image.new("RGBA",(W,H),(0,0,0,0)); fr.paste(src.convert("RGBA"),(x-minx,y-miny),m); G.append(fr)\nf=tempfile.NamedTemporaryFile(suffix=".gif",delete=False).name; G[0].save(f,save_all=True,append_images=G[1:],duration=100,loop=0,disposal=2,transparency=0); webbrowser.open(pathlib.Path(f).as_uri())' /path/to/.ABM [/path/to/.PAL]

For example:
python3 -c $'...' "GRAPHIC/ROOMANIM/CLOAK.ABM"

python3 -c $'...' "GRAPHIC/ROOMANIM/CLOAK.ABM" "GRAPHIC/PAL/INVHELP.PAL"

CMP (FutureVision compressed audio)

See FutureVision audio formats as well.

.CMP is Harvester’s custom compressed audio wrapper. The sampled files begin with FCMP, and the native load_sound_sample / load_dialogue_voice_sample code plus the ScummVM decodeHarvesterFcmp path all treat that payload as IMA-ADPCM-like compressed audio.

Summary

File layout

Notes

• Sampled MENACE.CMP starts with FCMP, a payload size of 0x0db48b, sample rate 22050, and 16 bits per sample.
• The loader family is tolerant: some call paths accept either FCMP or raw WAVE data. The .CMP extension itself points to the compressed path, but the runtime checks the actual file signature before decoding.
• From an architectural perspective, this is the audio-side equivalent of Harvester’s XFILE abstraction: one wrapper format reused across music, speech, and effects.

FST (Cutscene / streamed animation)

.FST is Harvester’s custom streamed movie format. It combines a file header, a compact per-frame index table, block-coded video payloads, optional per-frame palettes, and per-frame audio chunks.

Summary

File header

Frame index entry

Video payload for one frame

Block coding

Notes

• Sampled VIRGLOGO.FST is 320 x 200, stores 131 frames, plays at 15 fps, and carries 22050 Hz / 16-bit audio.
• The native and ScummVM players both treat FST as a streaming format: read one frame, queue its audio, decode its video, and advance without loading the whole movie at once.
• HARVEST.SCR uses GOFLIC/GODEATHFLIC opcodes to trigger .FST files. That mismatch between opcode names and file extension is a strong architectural clue that FST replaced or wrapped an older movie concept without rewriting the script vocabulary.
• The censorship path also shows how self-contained the format is: FST can carry its own palette updates frame-by-frame, while the player temporarily swaps in an external CENSORED.PCX overlay when gore is disabled.

PCX (Standard indexed still image)

Harvester does use standard .PCX files in at least one confirmed place: the censorship overlay shown during certain FST sequences when gore is disabled.

Summary

Confirmed header fields

Notes

• Sampled CENSORED.PCX is a textbook single-plane 320 x 200 PCX, which is why the ScummVM port can use the generic PCX decoder for it.
• The native loader contains two Harvester-specific wrinkles: it trims the optional 0x0c palette marker by jumping straight to the last 0x300 bytes, and when the logical width from xMax is one pixel smaller than bytesPerLine, it trims the padded stride byte after decode.

FLC (Standard Autodesk FLIC animation)

Harvester does ship .FLC files, and the sampled files match the standard Autodesk FLIC/FLC header rather than a Harvester-specific wrapper. The current Harvester engine work has not yet recovered the exact native Harvester-side call path that consumes them, so this section stays limited to what is directly supported by sample bytes, binary strings, and the generic ScummVM FLIC decoder.

Summary

Confirmed standard header fields

Notes

• Sampled HARVPNTR.FLC starts with a valid FLC header: size 0x214a, magic 0xaf12, 10 frames, 26 x 26, and 8-bit color.
• The decoded HARVEST.SCR sample does not reference .FLC files directly; its GOFLIC opcodes currently point to .FST files instead. That suggests .FLC belongs to a parallel or older asset path rather than the main shipping story-transition pipeline. This last point is an inference from the sampled script and should stay provisional.
• Taken together, the resource set shows three animation strata: lightweight sprite ABMs, streamed FST movies, and a residual standard FLC layer. That is a strong narrative hook for explaining how mixed-tool resource pipelines often survive into shipped games.

TRY IT OUT

1
2
ffplay -loop -1 "GRAPHIC/ROOMANIM/HARVPNTR.FLC"
ffmpeg -i "GRAPHIC/ROOMANIM/HARVPNTR.FLC" out.gif