SQLite Wasm in the browser backed by the Origin Private File System

Use SQLite to handle all your storage needs performantly on the web.

Thomas Steiner

SQLite is a popular, open-source, lightweight, embedded relational database management system. Many developers use it to store data in a structured, easy-to-use manner. Because of its small size and low memory requirements, SQLite is often leveraged as a database engine in mobile devices, desktop applications, and web browsers.

One of the key features of SQLite is that it is a serverless database, which means that it does not require a separate server process to operate. Instead, the database is stored in a single file on the user's device, making it easy to integrate into applications.

SQLite based on Web Assembly

There are a number of unofficial SQLite versions based on Web Assembly (Wasm), allowing it to be used in web browsers, for example, sql.js. The sqlite3 WASM/JS subproject is the first effort that is officially associated with the SQLite project making Wasm builds of the library established members of the family of supported SQLite deliverables. The concrete goals of this project include:

Binding a low-level sqlite3 API which is as close to the C one as feasible in terms of usage.
A higher-level object-oriented API, more akin to sql.js and Node.js-style implementations, that speaks directly to the low-level API. This API must be used from the same thread as the low-level API.
A Worker-based API which speaks to the previous APIs via Worker messages. This one is intended for use in the main thread, with the lower-level APIs installed in a Worker thread, and talking to them via Worker messages.
A Promise-based variant of the Worker API which entirely hides the cross-thread communication aspects from the user.
Support for persistent client-side storage using available JavaScript APIs, including the Origin Private File System (OPFS).

Using SQLite Wasm with the Origin Private File System persistence backend

Installing the library from npm

Install the @sqlite.org/sqlite-wasm package from npm with the following command:

npm install @sqlite.org/sqlite-wasm

The Origin Private File System

The Origin Private File System (OPFS, part of the File System Access API) is augmented with a special surface that brings very performant access to data. This new surface differs from existing ones by offering in-place and exclusive write access to a file's content. This change, along with the ability to consistently read unflushed modifications and the availability of a synchronous variant on dedicated workers, significantly improves performance and unblocks new use cases.

As you can imagine, the last point of the project's goals, Support for persistent client-side storage using available JavaScript APIs, comes with strict performance requirements regarding persisting data to the database file. This is where the Origin Private File System, and, more specifically, the createSyncAccessHandle() method of FileSystemFileHandle objects comes into play. This method returns a Promise which resolves to a FileSystemSyncAccessHandle object that can be used to synchronously read from and write to a file. The synchronous nature of this method brings performance advantages, but therefore it is only usable inside dedicated Web Workers for files within the Origin Private File System so the main thread can't be blocked.

Setting the required headers

Among other files, the downloaded SQLite Wasm archive contains the sqlite3.js and sqlite3.wasm files, which make up the sqlite3 WASM/JS build. The jswasm directory contains the core sqlite3 deliverables and the top-level directory contains demonstration and test apps. Browsers will not serve Wasm files from file:// URLs, so any apps you build with this require a web server and that server must include the following headers in its response when serving the files:

Cross-Origin-Opener-Policy set to the same-origin directive, which isolates the browsing context exclusively to same-origin documents. Cross-origin documents are not loaded in the same browsing context.
Cross-Origin-Embedder-Policy set to the require-corp directive, so a document can only load resources from the same origin, or resources explicitly marked as loadable from another origin.

The reason for these headers is that SQLite Wasm depends on SharedArrayBuffer, and setting these headers is part of its security requirements.

If you inspect the traffic with DevTools, you should find the following information:

The two headers mentioned above, Cross-Origin-Embedder-Policy and Cross-Origin-Opener-Policy, highlighted in Chrome DevTools.

Speedtest

The SQLite team have run some benchmarks on their WebAssembly implementation compared to the deprecated Web SQL. These benchmarks show that SQLite Wasm is generally about as fast as Web SQL. Sometimes it's a little slower, sometimes it's a little faster. See all details on the results page.

Getting started code sample

As mentioned previously, SQLite Wasm with the Origin Private File System persistence backend needs to run from a Worker context. The good news is that the library automatically takes care of all of this for you and you can use it right from the main thread.

import { sqlite3Worker1Promiser } from '@sqlite.org/sqlite-wasm';

(async () => {
  try {
    console.log('Loading and initializing SQLite3 module...');

    const promiser = await new Promise((resolve) => {
      const _promiser = sqlite3Worker1Promiser({
        onready: () => {
          resolve(_promiser);
        },
      });
    });

    console.log('Done initializing. Running demo...');

    let response;

    response = await promiser('config-get', {});
    console.log('Running SQLite3 version', response.result.version.libVersion);

    response = await promiser('open', {
      filename: 'file:worker-promiser.sqlite3?vfs=opfs',
    });
    const { dbId } = response;
    console.log(
      'OPFS is available, created persisted database at',
      response.result.filename.replace(/^file:(.*?)\?vfs=opfs$/, '$1'),
    );

    await promiser('exec', { dbId, sql: 'CREATE TABLE IF NOT EXISTS t(a,b)' });
    console.log('Creating a table...');

    console.log('Insert some data using exec()...');
    for (let i = 20; i <= 25; ++i) {
      await promiser('exec', {
        dbId,
        sql: 'INSERT INTO t(a,b) VALUES (?,?)',
        bind: [i, i * 2],
      });
    }

    console.log('Query data with exec()');
    await promiser('exec', {
      dbId,
      sql: 'SELECT a FROM t ORDER BY a LIMIT 3',
      callback: (result) => {
        if (!result.row) {
          return;
        }
        console.log(result.row);
      },
    });

    await promiser('close', { dbId });
  } catch (err) {
    if (!(err instanceof Error)) {
      err = new Error(err.result.message);
    }
    console.error(err.name, err.message);
  }
})();

Demo

See the above code in action in the demo. Be sure to check out the source code on Glitch. Note how the embedded version below does not use the OPFS backend, but when you open the demo in a separate tab it does.

Debug the Origin Private File System

To debug SQLite Wasm's Origin Private File System output, use the OPFS Explorer Chrome extension.

OPFS Explorer in the Chrome Web Store.

After installing the extension, open the Chrome DevTools, select the OPFS Explorer tab, and you're then ready to inspect what SQLite Wasm writes to the Origin Private File System.

OPFS Explorer Chrome extension showing the Origin Private File System structure of the demo app.

If you select any of the files in the OPFS Explorer window in DevTools, you can save it to the local disk. You can then use an app like SQLite Viewer to inspect the database, so you can assure yourself that SQLite Wasm actually works as promised.

SQLite Viewer app used to open a database file from the SQLite Wasm demo.

Get help and provide feedback

SQLite Wasm is developed and maintained by the SQLite community. Get help and provide feedback by searching in and posting to the support forum. The full documentation is available on the SQLite site.