TinyBase is a great JS package to deal with data for local-first apps with reactivity. With it, you can create store objects that contain key-value pairs or even tabular data. And then you can query with it, you can create metrics and use indexes to keep track of data in an optmized way. There’s also support for persisting data in a set of different ways, including SQLite databases, and a lot more.
It’s pretty hot right now, and I took it for a ride on an Expo SDK 49 project.
This combination (TinyBase + Expo 49) was a bit tricky at first, since the new release of Expo comes with a new version of expo-sqlite that at the moment of this writing was still lacking documentation while also had its API changed. On top of that, TinyBase tends to be very silent on errors, so it took me some time to figure out how to get around to persist TinyBase store data into a SQLite database in Expo workflow.
Say I had a very simple TinyBase store:
const store = createStore().setTablesSchema({ textCard: { sourceIcon: { type: "string" }, sourceName: { type: "string" }, category: { type: "string" }, cardId: { type: "string" }, heading: { type: "string" }, subheading: { type: "string" }, }, mediaCard: { cardId: { type: "string" }, sourceIcon: { type: "string" }, sourceName: { type: "string" }, category: { type: "string" }, imgUri: { type: "string" }, avatarImg: { type: "string" }, avatarText: { type: "string" }, }, })
When first trying to persist it with SQLite, I considered using one of the off-the-shelf persisters provided by TinyBase and the problem was that Expo wouldn’t let me. It won’t work with other SQLite APIs and there’s no expo-sqlite persister specific implementation as of yet. So I turned to custom persisters. Their implementation is quite simple, as you can read from the docs:
To build a custom
Persister, you should provide four functions:
- `getPersisted`, an asynchronous function which will fetch content from the persistence layer (or `null` or `undefined` if not present).
- `setPersisted`, an asynchronous function which will send content to the persistence layer.
- `addPersisterListener`, a function that will register a `listener` listener on underlying changes to the persistence layer. You can return a listening handle that will be provided again when `delPersisterListener` is called.
- `delPersisterListener`, a function that will unregister the listener from the underlying
changes to the persistence layer. It receives whatever was returned from your `addPersisterListener` implementation. @returns A reference to the new [`Persister`](https://tinybase.org/api/persisters/interfaces/persister/persister/) object.
With the creating function to be called like this:
createCustomPersister<ListeningHandle>( store:Store, getPersisted: () => Promise<undefined | [Tables,Values]>, setPersisted: (getContent: () => [Tables,Values], getTransactionChanges?:GetTransactionChanges) => Promise<void>, addPersisterListener: (listener:PersisterListener) => ListeningHandle, delPersisterListener: (listeningHandle: ListeningHandle) => void, ):Persister
But while still getting used to how TinyBase v4.0 works in general, having the documented example to be so much simpler than what I set out to do, I had some trouble with the implementations of both getPersisted and setPersisted function arguments. So now that I got it working, I’m gonna explain below how I did it in the hope that it helps somebody else.
First things first, then. For that store schema I showed earlier, I created the following tables with expo-sqlite:
CREATE TABLE IF NOT EXISTS media_card ( card_id TEXT PRIMARY KEY NOT NULL, source_icon TEXT NOT NULL, source_name TEXT NOT NULL, category TEXT NOT NULL, img_uri TEXT NOT NULL, avatar_img TEXT NOT NULL, avatar_text TEXT NOT NULL )
CREATE TABLE IF NOT EXISTS text_card ( card_id TEXT PRIMARY KEY NOT NULL, source_icon TEXT NOT NULL, source_name TEXT NOT NULL, category TEXT NOT NULL, heading TEXT NOT NULL, subheading TEXT NOT NULL )
These statements were executed asynchronously with the following function:
async function asyncQueryTx<ExpectedData = any[]>([ db: SQLite.SQLiteDatabase, // note that the database type now is different on expo-sqlite v11.3.1 sqlStatement: string, args?: (string | number)[] ) { try { if (!db) return []; const result = await new Promise<ExpectedData>((resolve, reject) => { db.transactionAsync(async (tx) => { try { const txResult = await tx.executeSqlAsync(sqlStatement, args || []); resolve(txResult.rows as ExpectedData); } catch (error) { reject(error); } }); }); return result; } catch (error) { console.log("Error querying database:", error); throw error; } }
This, of course, over a valid open database connection db.
Now, with both the TinyBase store and the SQLite tables existing and being able to be mapped from one to the other, we can move forward to creating our persister.
For the getPersisted function, we must get the data from the SQLite tables and return a promise of a tuple [Tables, Values]. Since we don’t have any key-value pairs in this implementation, our returned promise will be of type [Tables, {}].
async function getPersisted() { // object to be structured as a Tables type const result = { mediaCard: {}, textCard: {} }; // get data from table media_cards const persistedMediaCardContents = (await asyncQueryTx( "SELECT * FROM media_card" ).catch((error) => console.log(error))) as any[]; // map the data from table to store persistedMediaCardContents.forEach(async (row) => { result.mediaCard[row.card_id] = { cardId: row.card_id, sourceIcon: row.source_icon, sourceName: row.source_name category: row.category, imgUri: row.img_ur, avatarImg: row.avatar_im, avatarText: row.avatar_text }; }); // get data from table text_cards const persistedTextCardContents = (await asyncQueryTx( "SELECT * FROM text_card" ).catch((error) => console.log(error))) as any[]; // map the data from table to store persistedTextCardContents.forEach(async (row) => { result.textCard[row.card_id] = { cardId: row.card_id, sourceIcon: row.source_icon, sourceName: row.source_name, category: row.category, heading: row.heading, subheading: row.subheading }; }); return [result, {}] as [Tables, Values]; }
It’s pretty straight-forward here. We get the data from both tables (media_card and text_card) and format them into a result object that is structured as a TinyBase’s Tables type. This function will be called whenever we have the persister loaded.
With the setPersisted function, we go the other way around: we get the data from the Tinybase store and save it into SQLite tables. It’s a bit more complex because I decided to verify which items should be added based on if they are already stored, as well as I also decided to delete the items that are in the database but not on the TinyBase store anymore. The code is as follows.
async function setPersisted(getContent) { // get the data from the mediaCards table in tinybase store const currentMediaCardRows = Object.values(getContent()[0].mediaCard); // get the Ids of the mediaCards that are in store const currentMediaCardIds = currentMediaCardRows.map( (row: PersistedMediaCard) => row.cardId ); // get Ids of the media cards that are already persisted const mediaCardIdsAlreadyPersisted = ( await asyncQueryTx(`SELECT card_id FROM media_card`) ).map((row) => row.card_id); // filter the cards to be persisted const mediaCardsToPersist = currentMediaCardRows.filter( (row: PersistedMediaCard) => !mediaCardIdsAlreadyPersisted.includes(row.cardId) ); // filter the cards to be deleted from persistence const mediaCardsToDelete = mediaCardIdsAlreadyPersisted.filter( (id) => !currentMediaCardIds.includes(id) ); // delete them mediaCardsToDelete.forEach(async (cardId) => { await asyncQueryTx(`DELETE FROM media_card WHERE card_id = ?`, [ cardId, ]).catch((error) => console.log(error)); }); // persist the ones that should be persisted mediaCardsToPersist.forEach(async (row: PersistedMediaCard) => { await asyncQueryTx( `INSERT INTO media_card (card_id, source_icon, source_name, category, img_uri, avatar_img, avatar_text) VALUES (?, ?, ?, ?, ?, ?, ?)`, [ row.cardId, await row.sourceIcon, await row.sourceName, await row.category, await row.imgUri, await row.avatarImg, await row.avatarText ] ).catch((error) => console.log(error)); }); // do the same thing with textCards const currentTextCardRows = Object.values(getContent()[0].textCard); const currentTextCardIds = currentTextCardRows.map( (row: PersistedTextCard) => row.cardId ); const textCardIdsAlreadyPersisted = ( await asyncQueryTx(`SELECT card_id FROM text_card`) ).map((row) => row.card_id); const textCardsToPersist = currentTextCardRows.filter( (row: PersistedTextCard) => !textCardIdsAlreadyPersisted.includes(row.cardId) ); const textCardsToDelete = textCardIdsAlreadyPersisted.filter( (id) => !currentTextCardIds.includes(id) ); textCardsToDelete.forEach(async (cardId) => { await asyncQueryTx(`DELETE FROM text_card WHERE card_id = ?`, [ cardId, ]).catch((error) => console.log(error)); }); textCardsToPersist.forEach(async (row: PersistedTextCard) => { await asyncQueryTx( `INSERT INTO text_card (card_id, source_icon, source_name, category, heading, subheading) VALUES (?, ?, ?, ?, ?, ?)`, [ row.cardId, await row.sourceIco, await row.sourceName, await row.category, await row.heading, await row.subheading ] ).catch((error) => console.log(error)); }); }
And this function will be the one called when the persister is to be saved.
As for the addPersisterListener and the removePersisterListener functions, for the purpose of this case I’m working on, the SQLite database should always just reflect changes on TinyBase’s store, which is the source of all truth for this app. So I figure I don’t really need to listen to the persister here, and will do nothing inside of these functions.
const addPersisterListener = (listener) => { // do nothing }; const removePersisterListener = () => { //do nothing };
Now, with the persister defined as below, we’re only left with the task of managing the TinyBase store and deciding when to load from and when to save to the persistence layer.
const persister = createCustomPersister( store, getPersisted, setPersisted, addPersisterListener, removePersisterListener )
I decided to save to the persister each time there’s some change in the store. So for that, I added a tables listener on the TinyBase store like:
async function savePersister() { await persister.save() } store.addTablesListener(savePersister);
And I only load from the persister once, while mounting my app with:
persister.load()
Of course there’s some simplification here, as I’m actually doing all this inside a react provider, and envolving some data encryption — which are topics I’ll be covering on future writings — but for now I think this is enough.
💡 Check this post on Elliptic Curve encryption within Expo workflow for local-first apps if you also have to deal with encrypting data with Expo.
This is an open learning experience for me. If you have any suggestions or questions, feel free to reach me at frederico@mulungood.com