Making Hive-Engine Nodes Do Less Waiting

I have been continuing the small, careful performance work on the Hive-Engine node.

This time the change is less about adding a new operator setting and more about removing two bits of avoidable waiting from the normal path.

The short version is:

Transaction index writes are now batched, and the JavaScript VM pool is warmed up once on first use.

But the node has now been running this branch on a live witness for more than 72 hours, and it has been doing very well.

The First Part: Fewer MongoDB Writes

When a sidechain block is saved, the node also creates small transaction-index documents so it can find transaction information later.

Previously, it wrote those documents one at a time:

insertOne()
insertOne()
insertOne()
...

That works, but every individual insert is another trip through the MongoDB driver.

The change builds the complete set of transaction-index documents for the block and sends them with one insertMany() call instead.

The insert is explicitly ordered.

That matters because this is a consensus-adjacent codebase, and the transaction indexes contain the transaction positions within the block. I did not want to make a reviewer infer that ordering was still intentional, so the code says ordered: true directly.

Empty blocks are handled separately too. The node does not call insertMany([]) for a block with no transactions.

The Second Part: Warming the VM Pool

Hive-Engine runs smart contract code inside isolated JavaScript virtual machines.

The node already had a configurable maxJSVMs setting from the previous work. Before this change, those isolates were created one at a time as contract execution needed them.

That meant the first contract execution after startup paid the allocation cost.

When the first contract needs to run, the node allocates the configured VM pool in one eager-once warmup. The first contract still runs normally, and later executions can reuse the already-created pool.

So the behavior is:

• no VM allocation for a node that never executes contracts
• one pool warmup on the first contract execution
• reuse afterward
• the existing maxJSVMs value remains the limit

Why These Two Changes Together?

The original proposal contains much bigger ideas: caching, faster sync, snapshots, lite-node improvements, RPC changes, and dependency upgrades.

Those are still interesting, but the first goal was to build a contribution that is easy to explain and easy to review.

The safety claim for this bundle is simple:

The node processes the same transactions in the same order. It just batches one database write pattern and moves VM allocation to a predictable one-time warmup.

Testing It

I added focused tests for the transaction batching behavior:

• the generated documents preserve block order
• ordered: true is passed
• the active MongoDB session is preserved
• empty batches do not issue an insert

The focused database tests passed, and the Smart Contracts suite passed all 25 tests under Node 22.

The Live Test

After merging the branch into my custom Hive-Engine node setup, I put it on a live witness node.

It has now been running for more than 72 hours without a recurring issue.

Where It Is

The branch is here:

feature/knobs-b-batching-vm-prewarm

The commit is 6e0c24d.

The next step is upstream review.

I am especially interested in feedback on the VM memory tradeoff. Prewarming the configured pool makes first-use timing more predictable, but a larger pool also means more memory is committed sooner once contracts begin executing.

That is the sort of tradeoff that should be visible to operators, not hidden behind a vague performance claim.

For now, though, the live result is good.

A little less database chatter. A predictable VM warmup. The same processing order.

Nothing fancy.

As always,
Michael Garcia a.k.a. TheCrazyGM