rdb — an offline-first client for a remote database

One header, src/include/rdb/RemoteDatabase.hpp. Describe a table through the API, point it at a PocketBase server, and you get a local SQLite mirror, authentication with an optional persisted session, and two-way sync.

Nothing is read from the environment. Every knob is on rdb::Config.

Every public type and method, with signatures, is in DOCUMENTATION.html — a single self-contained file, open it in any browser. This README is the tour; that is the reference.

#include <rdb/RemoteDatabase.hpp>

rdb::Config config;
config.setBaseUrl("https://db.example.com")
      .setAuthCollection("users")
      .setDataDirectory("/home/me/.local/state/myapp");

rdb::Schema schema("transactions");
schema.setOwnerField("owner")
      .addStringField("description", true)   // true == required
      .addIntField("amount_cents")
      .addStringField("occurred_on", true)
      .addBoolField("cleared")
      .addFloatField("rate")
      .addDateTimeField("due_at")
      .addJsonField("tags")
      .addDateTimeField("edited_at")
      .setEditedField("edited_at");          // see Conflicts, below

rdb::Database db(config, schema);

db.login("me@example.com", "hunter2", /* remember = */ true);

rdb::Record row;
row.setString("description", "Coffee").setInt("amount_cents", 350)
   .setString("occurred_on", "2026-01-02");
db.insert(row);

const rdb::SyncResult result = db.sync();

Everything throws rdb::Error on failure. error.type() separates config, schema, network, http, auth, database, parse and usage problems; error.httpStatus() carries the status code when there is one.

Building

You need a C++17 compiler, CMake 3.20 or newer, SQLite and OpenSSL. zlib is optional. cpp-httplib and nlohmann/json are vendored under src/include/vendor, so configuring needs no network access and there is nothing else to fetch.

The repository has two CMake files with different jobs. src/CMakeLists.txt defines the library, and is what your own project should use (see below). The top-level CMakeLists.txt is the workspace for this repo: it pulls in src/ and builds the example client from src/test/main.cpp as C++23, with hardening flags in Release.

It is not header-only any more

It was, and that made every file that included it slow: cpp-httplib is some 23,000 lines of templates, and each translation unit compiled all of them. Measured on one machine with GCC, a file that merely included the header:

	before	now
include the header, -O0	3831 ms	702 ms
include the header, -O2	8182 ms	695 ms

So src/rdb.cpp now compiles the httplib-facing part once into a static library, and the public header doesn't include httplib.h at all. The API is unchanged, and so is the CMake you write as a consumer — rdb::rdb still names the target.

Two things to know:

• Files that use a lot of the API still pay for nlohmann/json, which the header does include: about 1.8 s at -O0 for one driving Database, sync and the JSON conversions, against 5.0 s before.
• If any of your code relied on rdb's header to supply httplib — the test client's fake server did — include <vendor/httplib.h> yourself now.

-DRDB_HEADER_ONLY=ON restores the old arrangement: nothing to build or link, every definition inline, and the old compile times with it.

The commands below build into bin/, which is what the repo's .gitignore and .clangd expect — clangd reads bin/compile_commands.json.

Arch Linux

sudo pacman -S --needed base-devel cmake sqlite openssl zlib
cmake -B bin -DCMAKE_BUILD_TYPE=Release
cmake --build bin -j$(nproc)

No -dev split to worry about: sqlite, openssl and zlib all ship their headers alongside the libraries. openssl and zlib are almost certainly installed already, so in practice cmake sqlite is the list.

macOS

xcode-select --install
brew install cmake sqlite openssl@3

cmake -B bin -DCMAKE_BUILD_TYPE=Release \
      -DOPENSSL_ROOT_DIR="$(brew --prefix openssl@3)"
cmake --build bin -j$(sysctl -n hw.ncpu)

OPENSSL_ROOT_DIR is not optional. The macOS SDK has shipped without usable OpenSSL headers for years, so find_package(OpenSSL) finds nothing without being pointed at Homebrew's copy. $(brew --prefix …) resolves to /opt/homebrew on Apple Silicon and /usr/local on Intel, so the same line works on both.

SQLite and zlib both come with the SDK, so brew install sqlite only buys you a newer SQLite. Add -DSQLite3_ROOT="$(brew --prefix sqlite)" if you want it.

Windows — MSYS2, UCRT64

Open the UCRT64 shell, not the plain MSYS one. The prompt says UCRT64; get this wrong and the mingw toolchain isn't on PATH and nothing below resolves.

pacman -S --needed \
    mingw-w64-ucrt-x86_64-toolchain \
    mingw-w64-ucrt-x86_64-cmake \
    mingw-w64-ucrt-x86_64-ninja \
    mingw-w64-ucrt-x86_64-sqlite3 \
    mingw-w64-ucrt-x86_64-openssl \
    mingw-w64-ucrt-x86_64-zlib

cmake -B bin -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build bin -j$(nproc)

The binary links against the UCRT64 runtime DLLs, so it runs from that shell as-is; to run it anywhere else, ship those DLLs beside it or link them statically.

If TLS handshakes fail with a verification error, OpenSSL has no trust store it can find. Point the library at one rather than turning verification off:

config.setCaCertPath("C:/msys64/ucrt64/etc/ssl/certs/ca-bundle.crt");

Windows — MSVC

With vcpkg (vcpkg install sqlite3 openssl zlib) everything is found normally. Without it, when find_package(SQLite3) comes up empty, src/CMakeLists.txt falls back to linking a library called sqlite3 by name — so sqlite3.lib and sqlite3.h then have to be on the linker and include paths by other means.

The example client's Release flags (-D_FORTIFY_SOURCE=3, -fstack-protector-strong, …) are GCC/Clang flags. Under MSVC, build it as Debug, or skip it and consume the library from your own project.

Using it in your own project

Point CMake at src/, not at the repository root — the root is the workspace above and would add the example client to your build.

add_subdirectory(external/rdb/src rdb)
target_link_libraries(myapp PRIVATE rdb::rdb)

Or without copying the repository into your tree:

include(FetchContent)
FetchContent_Declare(rdb
    GIT_REPOSITORY https://git.keylightpiano.com/sylar/rdb.git
    GIT_TAG main
    SOURCE_SUBDIR src)
FetchContent_MakeAvailable(rdb)
target_link_libraries(myapp PRIVATE rdb::rdb)

rdb::rdb is a static library (an interface target under RDB_HEADER_ONLY), and everything it carries passes on to whoever links it —

• the include path, so #include <rdb/RemoteDatabase.hpp> works;
• C++17 as a minimum (a floor, not a setting — your project can build higher);
• SQLite, and OpenSSL;
• zlib, when zlib is found;
• ws2_32 and crypt32 on Windows, Threads::Threads on Linux.

The CPPHTTPLIB_* feature defines are private to the library in a compiled build, since httplib is compiled only there; under RDB_HEADER_ONLY they propagate to you, because then you compile it too.

It is safe to reach the same src/ by two routes in one build; the second add_subdirectory is a no-op rather than a duplicate-target error.

Option	Default
RDB_ENABLE_SSL	ON	Off builds a plain-HTTP client: an https:// base url is then refused with Error::eType::Config instead of failing obscurely later
RDB_ENABLE_COMPRESSION	ON	Accept gzip/deflate responses. Needs zlib; when it isn't found, compression is quietly left off
RDB_HEADER_ONLY	OFF	On, nothing is compiled or linked: the header carries the library, as it used to, and including it costs ~3.8 s a file again

What a collection must have

Two fields are mandatory, whatever the collection is for:

id	automatic in PocketBase — nothing to do
updated	autodate, on create and on update

updated earns its place four times over: it is the incremental pull filter (updated > "<last_sync>"), the high-water marker that advances it, the signal that a record changed remotely since you last looked (which is how a conflict is detected at all), and the check that skips rewriting a version you already hold. setUpdatedField renames it; nothing removes the need for it.

created is not used anywhere in this library. Keep it if you find it useful, delete it if you don't.

Two more are conditional, each tied to a feature you choose:

field	type	needed when
owner	relation → users, required	the collection's API rules scope rows per user. setOwnerField("") for a shared or global collection
edited_at	date	PreferNewest should mean newest edit and not last arrival — see Conflicts

Outside the collection you need an auth collection with password auth (users by default, setAuthCollection to change) and API rules that let an authenticated user list, view, create, update and delete. It has to be a base collection: view collections expose a records endpoint but are read-only, so every write fails.

On this side, Schema::validate() asks only for a collection name, at least one field, and — if you named an edited field — that it is declared and is a DateTime or String.

Mapping PocketBase's types onto the six here

PocketBase	declare as	what you get
text, editor, email, url	addStringField	the string
number	addIntField / addFloatField
bool	addBoolField
date	addDateTimeField
select, relation, file — single	addStringField	the value, the related id, the filename
select, relation, file — multi	addJsonField	the array, as raw JSON text
json, geoPoint	addJsonField
autodate	don't declare it — reserved

Required-ness is enforced in two places and they are not linked: this library checks its own required flags before writing locally, the server checks its own on push. A field the server requires and your schema doesn't will fail as a 400 part-way through a push rather than as a clean local error, so keep the two in step.

Describing the table

rdb::Schema replaces the hand-written struct. Adding a column is one line: the SQLite mirror grows one to match, and sync carries it both ways.

Method	Column	JSON
addStringField(name, required)	TEXT	string
addIntField(name, required)	INTEGER	number
addFloatField(name, required)	REAL	number
addBoolField(name, required)	INTEGER	bool
addDateTimeField(name, required)	TEXT	string
addJsonField(name, required)	TEXT	parsed structure

addField(rdb::Field(...)) takes the long way round when you want setIndexed(true) or setDefault(rdb::Value::makeInt(0)).

id, created, updated, local_id, remote_id and the sync_* columns are reserved — adding one is refused up front rather than silently dropped by the server or collided with in the mirror.

The mirror table is created once and then extended in place: a schema field with no matching column is added by ALTER TABLE. Columns for fields you removed stay behind, unused, so an older build keeps working against the same file.

Records

rdb::Record is a dynamic row. Values are typed but forgiving: reading an int column as a string works, and so does the reverse where it makes sense.

row.setString("description", "Coffee");
row.setInt("amount_cents", 350);
row.setNull("due_at");

row.getString("description");
row.getInt("amount_cents", /* fallback = */ 0);
row.has("cleared");
row.isNull("due_at");

Plus the sync bookkeeping, all read-only in practice: localId(), remoteId(), updated(), localUpdated(), revision(), isDirty(), isDeleted().

Authentication

db.login(identity, password, /* remember = */ true);
db.restoreSession();   // load the saved token and prove it still works
db.logout();           // forget it here and delete it from disk

remember == false keeps the token in memory only. Nothing is written, and any previously saved session is removed so it cannot outlive this one.

remember == true writes session.json in the data directory, owner-only (0600 on POSIX; on Windows the file inherits the directory ACL). logout() deletes it. Mirrored rows survive a logout — call clearLocal() for those.

Opening a database adopts a saved session without touching the network, so isAuthenticated() is true immediately. restoreSession() is what trades the stored token for a fresh one and tells you whether it was still valid; it returns false, having cleaned up, when the server rejects it.

When a different user logs in, the mirror is wiped and the sync marker reset, so one account never sees another's cached rows. Turn that off with setResetLocalOnUserChange(false).

Reading and writing

db.insert(row);                       // fills row.localId()
db.update(local_id, values);          // writes only the fields `values` carries
db.remove(local_id);                  // tombstone, deleted remotely on next sync
db.all(include_deleted, "occurred_on, local_id");
db.where("amount_cents < 0", "occurred_on DESC");
db.pending();
db.find(local_id);
db.findByRemoteId(remote_id);
db.count();
db.clearLocal();

A partial update leaves the columns it doesn't mention alone, so read-modify-write of one field doesn't need the rest of the row.

discardPending() is the undo: it throws away every change the server hasn't seen — or just one row's, with discardPending(local_id) — putting edited and removed rows back to the server's current version and deleting rows that were never pushed. Restoring needs the server, because the mirror keeps no copy of a row's last synced values; everything is fetched before anything is written, so a failed request changes nothing, and discarding only never-pushed rows works offline. clearLocal() is the blunt version: it drops every row, synced or not.

fetchRemote(filter, sort) goes straight to the server and skips the mirror entirely. db.client() is the escape hatch to the raw REST calls.

Sync

const rdb::SyncResult r = db.sync();
r.created_remote; r.updated_remote; r.deleted_remote;
r.pulled; r.pruned_local; r.conflicts;

Three phases:

1. Pull — GET filtered on updated > "<last_sync>", so only what changed comes down. The marker advances only after the batch commits, so an interrupted sync re-fetches the same window and the upsert makes that harmless. Records already held at that exact version are skipped rather than rewritten — including your own push coming back around while the row has been edited again since, which builds on that version and is not a conflict. This runs first so that conflicts are visible; see below.
2. Push — every still-dirty row goes up: POST without a remote id, PATCH with one, DELETE for tombstones.
3. Reconcile — fetch every remote id (?fields=id) and drop local rows that are gone. This phase exists because a deletion leaves nothing behind for an incremental query to find; without it a device that had a copy keeps it forever. Turn it off with setReconcileEnabled(false).

pushOnly() and pullOnly() run the halves separately.

Dry run

const rdb::SyncPlan plan = db.previewSync();
if (!plan.empty())
    std::cout << plan.toString() << '\n';

previewSync() works out everything sync() would do and does none of it. It makes the same two read requests a real sync starts with — the incremental list and the id list — and writes nothing on either side: not the records, not the sync marker, not the clock offset. pbsync sync --dry-run prints one:

pushed 1 created, 2 updated, 1 deleted; pulled 3; pruned 1; conflicts 3 (1 local, 1 remote, 0 merged)
  pull update            #3      rec3              fields: qty
  conflict, take remote  #6      rec6              fields: qty  (local edit 2026-09-13 09:07:33.210Z, remote edit 2026-09-13 09:07:33.240Z)
  conflict, keep local   #7      rec7              fields: qty  (local edit 2026-09-13 09:07:33.210Z, remote edit 2026-09-13 09:07:33.180Z)
  pull new               -       rec8
  push update            #1      rec1
  push delete            #2      rec2
  discard local          #5      rec5              edited here, deleted on the server
  push create            #8      -
  prune local            #4      rec4              deleted on the server

Each line is an rdb::SyncChange: what would happen, the local and remote ids, the local row and the server's version where they exist, the fields that would change, and for a conflict both edit times. changed_fields is empty for pushes, because the server's copy isn't fetched for those. plan.predicted is the SyncResult a sync() run straight afterwards would return, worked out by the same rules.

Three limits. It is a snapshot: another client writing between the preview and the sync changes what the sync does. A conflict resolver set on the Config is really called, just as sync() would call it, so one with side effects will have them. And it previews a full sync() only — there is no preview of pushOnly() or pullOnly().

Conflicts

A conflict is a row that changed on both sides since the last sync. The pull above is exactly the query that finds them — "changed remotely since we last looked" — and it runs before the push for that reason. With the push first, it would overwrite the server's copy before anything had a chance to notice, and the local side would silently win every time regardless of policy.

setConflictPolicy() picks the winner:

PreferNewest	default: whichever edit happened later
PreferLocal	keep the local edit and push it
PreferRemote	take the server's version and discard the local edit

Either way the row is counted in conflicts, and broken down across conflicts_kept_local, conflicts_took_remote and conflicts_merged.

What "newer" has to mean

PreferNewest needs a timestamp saying when an edit was made. The server's own updated is not that: it says when the edit arrived. A laptop that has been shut for a week syncs, and its stale edits are stamped as the newest thing the server has ever seen — so the machine that was away longest wins. That is the opposite of what you want.

So the edit time has to travel with the record. Declare a field for it and name it:

schema.addDateTimeField("edited_at").setEditedField("edited_at");

Add the matching field to the collection on the server (type date, not required). From then on every push records when that client made the change, and conflicts compare those to each other. Without setEditedField the resolution falls back to the server's arrival time, with the failure above.

The library stamps local edits itself, in sync_local_updated, readable as Record::localUpdated().

Clocks

Two machines' clocks disagree, so comparing a timestamp one wrote against a timestamp the other wrote is only meaningful once they are on the same scale. Every response re-estimates the offset between this machine's clock and the server's from the Date header, and local edits are stamped with that correction applied. The estimate is kept in the mirror, so a client that starts up offline still stamps in roughly server time. serverClockOffsetMilliseconds() reports it; setClockSkewCorrection(false) turns it off for machines you know are already disciplined to a common clock.

Date has one-second resolution, which bounds the estimate's error at about ±500 ms per client. Two edits on different machines less than a second apart cannot be ordered reliably. If that matters, you need version vectors, not better timestamps.

When neither side is simply right

config.setConflictResolver([](const rdb::Record& local, const rdb::Record& remote,
                              rdb::Record& merged) {
    merged.setString("notes", local.getString("notes") + "\n" + remote.getString("notes"));
    merged.setInt("amount_cents", remote.getInt("amount_cents"));
    return rdb::eConflictChoice::UseMerged;
});

Set, it overrides the policy entirely. Return KeepLocal, TakeRemote, or UseMerged after filling merged — a merged record is written locally and queued for the next push, so both sides converge on it. The ids are filled in for you.

What is still not handled

Another client can write in the window between this client's pull and its push, and that write is overwritten. Closing it needs optimistic concurrency — compare-and-swap on a version the server checks — which PocketBase does not expose. The window is milliseconds wide and the next sync converges, but it is not zero.

pushOnly() skips the pull, and therefore skips conflict detection entirely.

Threads and background sync

rdb::Database does no locking of its own. It holds one SQLite connection, one HTTP client, and a token and clock offset that every request updates, so it must be used from one thread at a time.

rdb::SyncedDatabase is how you share one. It owns a Database behind a mutex, and can run a worker thread that syncs it in the background:

rdb::SyncedDatabase db(config, schema);

db.with([](rdb::Database& d) {
    if (!d.restoreSession()) d.login("me@example.com", "hunter2", true);
});

db.start(rdb::SyncOptions()
             .setInterval(std::chrono::seconds(30))
             .setCallback([](const rdb::SyncEvent& e) {
                 std::cerr << e.toString() << '\n';   // on the worker thread
             }));

// From any thread:
rdb::Record row;
row.setString("description", "Coffee").setInt("amount_cents", 350);
db.with([&](rdb::Database& d) { d.insert(row); });
db.requestSync();                       // sooner than the timer

Every call on the database goes through with(), which runs a function under the lock and returns what it returns, or through lock(), which holds the lock for a group of calls that must not have a sync land between them:

{
    auto d = db.lock();
    const rdb::i64 id = d->insert(row);
    d->update(id, changes);
}   // released here

The worker:

start(options)	starts it, or restarts it with new options
stop()	stops it, waiting for a sync in progress to finish; the destructor does this too
requestSync()	sync as soon as possible; requests made during a sync add up to one more
syncAndWait(timeout)	request a sync and block until it has run; returns the SyncEvent
status()	a SyncStatus: counters, the last attempt, the last success, when the next one is due

SyncOptions:

setInterval	time between automatic syncs, default 1 min; 0 syncs only when asked
setMode	eSyncMode::Full (default), PushOnly or PullOnly
setSyncOnStart	default on: the first sync starts right away
setRetryBackoff(first, max)	default 5 s doubling to 5 min; (0ms, 0ms) turns retrying off
setCallback	called after every attempt, on the worker thread, with no lock held

Each attempt is a SyncEvent that Succeeded (with the SyncResult), Failed (with the rdb::Error) or was Skipped — not logged in, or the database is closed. A skipped attempt is not a failure. A failure is retried on the backoff, regardless of the interval, until one succeeds. The exception is an Auth failure: the same token would fail the same way, so the worker waits for the interval or a request instead. Log in again through with(), then call requestSync().

The callback may use the SyncedDatabase, including stop(). It must not call syncAndWait() or start(), which would wait on themselves, and those throw Usage instead. Anything the callback throws is swallowed so the worker keeps running. A conflict resolver set on the Config also runs on the worker thread, under the lock.

Two connections

The worker holds the lock for a whole sync, network time included, so calls from other threads wait for it. When that is too long for a UI thread, give that thread its own Database on the same mirror file. WAL mode lets it read while the sync writes, and the busy timeout (5 s) queues its writes behind the sync's short transactions.

Its writes are safe as well. Each local write bumps the row's revision(), and a push marks a row clean only if the revision hasn't moved since the row was read. An edit, removal or discard that lands on a row while it is on its way to the server is therefore kept and goes up on the next push, rather than being mistaken for the version that was sent. Two rules for this setup:

• Only one connection syncs. Two syncs running at once each push the same pending rows, and every new row is created twice.
• Each connection is still single-threaded. Lock it yourself, or put it in a SyncedDatabase without starting the worker.

pbsync watch [seconds] runs the worker while you add rows from the prompt.

Without the mirror

setLocalMirrorEnabled(false) drops SQLite from the picture: insert, update, remove and all go straight to the server, no file is created, and sync() throws Error::eType::Usage because there is nothing to sync. Records are then keyed by remoteId() rather than localId().

Configuration reference

setBaseUrl	scheme included: https://host or http://127.0.0.1:8090
setAuthCollection	default users
setAuthHeaderPrefix	default empty (PocketBase wants the raw token); "Bearer " elsewhere
setUserAgent, addHeader	sent on every request
setConnectionTimeout, setReadTimeout, setWriteTimeout	seconds
setFollowRedirects	default on
setVerifyCertificate	default on, and it should stay on
setCaCertPath	explicit trust store, usually needed on Windows
setDataDirectory	parent of the mirror and the session file; created on open
setDatabaseFileName, setSessionFileName	defaults local.db, session.json
setDatabasePath, setSessionPath	full paths, overriding the pair above
setLocalMirrorEnabled	default on
setConflictPolicy	PreferNewest (default), PreferLocal or PreferRemote
setConflictResolver	callback overriding the policy
setClockSkewCorrection	default on; steers local edit stamps onto server time
setReconcileEnabled	default on
setResetLocalOnUserChange	default on
setPageSize, setIdPageSize	1–500, default 200 and 500

rdb::Config::platformDataDirectory("myapp") is the one function that looks at the environment, and only because you called it: $XDG_STATE_HOME or ~/.local/state on Linux, ~/Library/Application Support on macOS, %LOCALAPPDATA% on Windows. Pass a literal path instead if you'd rather it didn't.