We replaced our ledger with two functions

# read the truth
def get_balances(account)
  :: {:ok, Balances.t()} | {:error, term()}

# change the truth by recording what happened
def record_event(event, balances)
  :: {:ok, Balances.t()} | {:error, term()}

This post covers why we decided to redesign our ledger, what’s interesting about the new design, and how we leveraged it to perform a zero-downtime migration through our highest-volume trading days.

River is a Bitcoin financial institution providing people and businesses with a safe, easy way to buy, sell, secure, and use bitcoin. We offer everyday banking services for bitcoin and dollars in one app.

When we designed our previous ledger, the company was operating at a much smaller scale. We offered basic Bitcoin brokerage features (buy, sell, send, receive) but did not have the full suite of banking services we offer today (bitcoin interest on cash, direct deposit, bill pay, etc.).

As our product suite grew, the ledger became increasingly complex, revealing several shortcomings:

• An imperative API that bloated to ~40 functions.
• Coupling of ledger accounting with business logic and fraud risk semantics (e.g. is your bitcoin withdrawable yet).
• Insufficient granularity in tracking intermediate dollar states (e.g. is the cash in transit from your bank to ours).
• Query inefficiencies requiring joins across dozens of tables for chronological transaction history.

It was time to “day-zero” the system.

The new design simplifies the API to two primary functions: fetching current balances and recording money movement events (e.g., buy_completed). Each event type implements BalanceRules, a pure, versioned function that transforms inputs and balances into ledger entries using declarative rules.

defprotocol BalanceRules do
  # pure & deterministic: facts + current balances in, ledger entries out.
  def apply(inputs, current_balances)
end

# a simplified buy completion: decrease USD, increase BTC
# returns {debit, credit, amount} tuples
def apply(%BuyCompleted.Inputs{} = i, balances) do
  [
    {:usd_liability, :fbo,           i.amount_usd},
    {:received_btc,  :btc_liability, i.amount_btc}
  ]
end

This design naturally makes every event testable, composable, and auditable. Versioning pins historical rules to their inputs while managing compatibility as we iterate.

Events are grouped into flows that represent the activity (e.g. wire_transfer, buy_order, ach_withdrawal) and these flows can reference each other to signify some semantic relation. This gives us an efficient way to query transaction history, forming a tree-like data structure e.g. a return that points at the deposit it reverses or a deposit that’s linked to the chain of recurring orders that it is a part of.

The previous data model was framed around the funds’ availability. We replaced that with a more precise model that tracks not only “how much”, but also “where” and “what kind”. Is the money sitting at our partner bank on behalf of the client, in transit, with one of our trading partners, something that we’re fronting to the client, etc.

Policy decisions, like whether a client may spend funds based on payment risk, live in a layer on top of get_balances. Other teams can now adjust those parameters (fraud detection, new features like instant withdrawal, etc.) without inheriting the blast radius of the accounting layer.

We enforce accounting invariants at the database level, rather than just relying on application-level conventions. The ledger_entries table structurally requires double entry. Each row has a debit balance name, a credit balance name, and a single amount column. All our accounting invariants (assets must equal liabilities, balances cannot go negative, we only front what’s receivable, etc.) are guarded by a single Postgres CHECK constraint. So, any transaction that breaks the books cannot be committed.

The system is entirely append-only. Only the denormalized current balances row for an account is updated via optimistic locking, resulting in constant-time API performance (~10ms p95, ~20ms p99) regardless of an account’s transaction volume.

To manage complex trade-offs across engineering, product, finance, and legal, we used a decision log. We broke large, ambiguous challenges into smaller, isolated, and explicit requirements. This allowed us to treat foundational choices as building blocks, letting us stack solutions and gain momentum.

We made the decision to decouple the historical data backfill from the rollout of the new ledger. The standard playbook for data migrations is to: copy the old history, replicate the live delta to catch up, and when the lag is zero, pause writes and cut over. But we did it in reverse: atomic snapshot and cutover, replicate the live data, and independently backfill the history later.

The live replication could fail because of latent bugs in the new ledger, or the translation of legacy data could have issues and block the rollout. It was an open question whether the historical data could even satisfy all the new constraints and it could have taken weeks or months to finish successfully.

So, we decoupled these failure modes, allowing each to iterate and roll out in parallel. The trade-off was the extra machinery needed to do the cutover without pausing writes and having the backfill start from zero and mesh perfectly in the middle for each account.

The pure function balance rules are trivially unit testable with extensive table-driven tests and we have the typical integration tests that cover happy paths and failure cases. Beyond those, we introduced two purpose-built harnesses.

The first is a new scenario testing suite. This addressed the “combination of scenarios” gap that neither unit tests nor integration tests cover: a deposit that’s spent before it returns, a chargeback that lands while a bill pay is still outstanding, a returned deposit whose retry has to cover a buy that already credited its bitcoin.

Because the rules are simple in isolation, it becomes critical to ensure that they still play well together when composed into complex scenarios. So we built a custom DSL that can express these in a simple manner and have hundreds of fuzzed scenarios that run in a handful of seconds as part of CI:

test "return with buy completed then retry covers the purchase" do
  # An ACH-funded buy. The bitcoin is credited, then the deposit
  # is returned and then retried.
  scenario("return with buy then retry")
  |> ach_initiated(100)
  |> buy_completed(100)
  |> ach_returned()
  |> assert_balances(%{...})
  |> ach_retried(100)
  |> ach_received()
  |> assert_balances(%{...})
end

We’re also big fans of TigerBeetle and took inspiration from them to build our own simulator. It drives our real APIs with fleets of simulated users doing millions of transactions in (un)usual account configurations. Unlike tests with known assertions, a simulator is only useful with a good oracle to detect badness.

In our case we could lean on two things. Primarily, the various invariants of our ledger. All of the simulated actions are legitimate client actions; if any of them end up violating one of our constraints, then it’s a real signal. We also have a known-correct reference implementation in the old ledger, so for every transaction we could do a parity check.

In the month before rollout we simulated the rollout itself hundreds of times and ironed out the kinks. Overall, the simulator caught a dozen real bugs across River’s systems, a handful of them in the new ledger. Each of those prevented a potential production incident.

ledger simulator TUI

The rollout was executed on a per-account basis. The first time an eligible account transacted, we atomically snapshotted its balances from the old ledger into the new and recorded the precise point in the event stream where the new world is spawned from.

From then on, every transaction was written to both ledgers, and we did parity checks before starting the DB transaction and before committing it. Any failure automatically disabled the new ledger for that account and failed open to the legacy system to ensure zero client-facing impact.

// every event writes to both ledgers, then compares the books.

During this whole effort, the rest of the business, product roadmaps, engineering, couldn’t be put on pause. We shipped a full app redesign, moved parts of our frontend to React, upgraded Postgres to the latest major version and more while the two ledgers ran side by side, even through our highest-volume trading days.

Features that the new ledger was designed to support, like transaction APIs, revamped statements, Direct Deposit, Bill Pay, etc., shipped before or alongside the rollout. For some projects we decided it was worth paying the both-worlds tax and for others we paused.

Backfilling six years of history was arguably the most challenging part of this project. It required consolidating data from ~40 tables into a canonical event stream that satisfies all the new constraints and meshes perfectly at each account’s cutover point.

Instead of using balance rules, we mapped historical data points as “evidence” for these legacy events. We tried our best to semantically match different states to our current superset of events which resulted in a much richer view of history than we originally had.

The structural constraints of the new ledger ensured that any data inconsistency resulted in immediate failure during the staging phase. Then, our Finance team re-ran their month-end audits across many historical month-ends and compared the books looking for unexplained discrepancies. Once they signed off, we materialized the data into the real production tables.

With the history in place and a couple of months of zero parity failures, we performed authoritative read cutovers across all our systems (wallet, trading system, statements, interest workflows, admin tooling, analytics, etc.). Then, the easy change of deleting the feature flag (Kent Beck style—”make the change easy, then make the easy change”).

The final phase was the most satisfying: deleting the old code and temporary scaffolding. This was still a careful maneuver because we didn’t want the old ledger to crash the party because of some partial state. The tally:

With all the new features in the ledger: asset tracking, richer events, single timeline etc. we ended up with fewer lines of client-facing code, much simpler abstractions, and an API that makes it very easy to build upon and extend our product.

The success of this 16-month effort was rooted in these core principles:

1. Make invariants structural. Enforcing constraints at the schema layer beats application-level convention.
2. Move complexity out of the API. Narrow-waist APIs provide a strong contract and can be tested and verified in one place.
3. Separate truth from policy. The system that determines and enforces policy should be built on top of the system that records the truth.
4. Automate your resiliency. Running in shadow mode, parity checking, automatic mitigations, extensive monitoring, reset/retry tooling, are all worthwhile investments when resiliency is paramount.
5. Decouple the ways you can fail. Parallelize workstreams that could each sink the project and don’t run them in series.
6. Leverage AI smartly. Autonomy is safe where the design is sound and the failure is loud.