Show HN: InstantDB – A Modern Firebase

I always assumed that an architectural decision had prevented relational queries in Firebase.

It was jarring to find out that indexes are required for every combination of filters your app applies, but then you quickly realize that Firebase solves a particular problem and you're attempted to shoehorn into a problem-space better solved by something like Supabase.

It's not too dissimilar to DynamoDB vs RDB.

Thanks for creating Firebase!

It's really the definition of an managed database/datastore.

Do you see InstantDB as a drop in replacement ?

To be honest I don't want to have to worry about my backend. I want a place to effectively drop JSON docs and retract them later.

This is more than enough for a hobbyist project, though I imagine at scale things get might not work as well.

This is an aside but “trifecta but with four” actually has an awesome name: “Superfecta”!

You probably heard this a million times but I still remember trying that simple firebase demo of draw in one box; see the results in another and being amazed. That was one of my pushes out of boring enterprise software death by configuration and into software creation based on modern OSS products.

Was pretty neat to see your investment/involvement!

Made me feel quite old that Firebase is no longer "modern" though...

Awesome to see this launch and to see James Tamplin backing this project.

Thank you James!

If we only had doSQL() for everything.

Thank you for the feedback, this makes sense!

I updated the example to include the imports:

```

import { init, tx, id } from "@instantdb/react";

const db = init({ appId: process.env.NEXT_PUBLIC_APP_ID, });

function Chat() {

  // 1. Read
  const { isLoading, error, data } = db.useQuery({
    messages: {},
  });

  // 2. Write
  const addMessage = (message) => {
    db.transact(tx.messages[id()].update(message));
  };

  // 3. Render!
  return <UI data={data} onAdd={addMessage} />;

```

What do you think?

Yes. This gives users the vibe of “ this is obvious, if you don’t know it , you are dumb “ .

lol i was wondering the same thing

this is my question too, lol

Co-founder of PowerSync here. Would love to hear what you would like to see improved in PowerSync :) Thanks!

> CouchDB / PocketDB aren’t really up to date anymore.

Source? I’ve been using CouchDB as my game world DB for years, works fine for me?

We hear you on the pain for evolving NoSQL schemas. [1]

For what it's worth, we're built on top of Aurora and support relations so evolution should be much easier!

[1] https://mdp.github.io/2017/10/29/prototyping-in-the-age-of-n...

This is why I’ve always stayed well behind the bleeding edge but still within earshot if anything comes along that sounds like it’s of interest to me. I usually code to work and not for pleasure, although I do a little web programming for friends, but I still use jQuery and Typescript for that, I think the only “new”thing I use is tailwind, which is a bit of a game changer for what I like to do, I never liked CSS but it worked well enough for my needs.

Did they say schemas aren’t supported, or is that implied by the firebase label?

Good point on the update frequency, I believe it is a must to batch the requests and responds for any of this type of lib/service to work in a production environment, a performance report/comparison is still required for ppl to get the idea if this is good to support their business model.

About the synchronized data though I think it's not about the database but the data types designed to sync the data? I worked on multiple-player canvas games and we didn't really care that much about relational db or document db, they worked both fine. I would love to know what's the difference and the challanges.

We do indeed batch frequent updates! Still many opportunities for improvements there, but we have a working demo of a team-oriented tldraw [1]

[1] https://github.com/jsventures/instldraw

We would love to hear more about the architecture you used at Framer. Would you be up for a coffee? My email is stopa@instantdb.com

Would love to hear how you went about doing things at Framer!

Noted about the refresh tokens, thank you!

> One thing I was curious about is how well the system currently supports users with multiple emails? GitHub popularized this pattern, and these days it's pretty much table stakes in the dev tools space to be able to sign in once and use the same account across personal accounts and orgs associated with different emails

Right now there is an assumption of 1 `user` object per email. You could create an entity like `workspace` inside Instant, and tie multiple users together this way for now.

However, making the `user` support multiple identities, and creating recipes for common data models (like workspaces) is on the near-term roadmap.

I really appreciate your message Slava. Your essays were really influential for us.

It's been great iterating with you AJ! Can't wait for what's ahead.

> What's the short summary of how the authorization system works for this?

We built a permission system on top of Google's CEL [1]. Every object returned in a query is filtered by a 'view' rule. Similarly, every modification of an object goes through a 'create/update/delete' rule.

The docs: https://www.instantdb.com/docs/permissions

The experience is similar to Firebase in three ways:

1. Both languages are based on CEL 2. There's a distinct separation between data retrieval and access policy 3. You can start on a happy path when developing, and lock down later.

AFAIK, Firebase Realtime can be more efficient, as it can tell if a permission check has passed statically. I am not sure if Firestore works this way. We wanted to be more dynamic, to support more nuanced rules down the road (stuff like 'check this http endpoint if an object has permissions'). We took inspiration Facebook's 'EntPrivacy' rules in this respect.

It’s not quite the same thing but nearby:

I built a EAV secondary index system on top of Postgres to accelerate Notion’s user-defined-schema “Databases” feature about a year ago. By secondary index, I mean the EAV table was used for queries that returned IDs, and we hydrated the full objects from another store.

We’d heard that “EAV in Postgres is bad” but wanted to find out for ourselves. Our strategy was to push the whole query down to Postgres and avoid doing query planning in our application code.

When we first turned it on in our dogfood environment, the results looked quite promising; large improvement compared to the baseline system at p75, but above that things looked rough, and at p95 queries would never complete (time out after 60s).

It worked great if you want to filter and sort on the same single attribute. The problem queries were when we tried to query and sort on multiple different attributes. We spent a few weeks fixing the most obviously broken classes of query and learned a lot about common table expressions, all the different join types, and strategies for hinting the Postgres query planner. Performance up to p95 was looking good, but after p95 we still had a lot of timeout queries.

It turns out using an EAV table means Postgres statistics system is totally oblivious to the shape of objects, so the query planner will be very silly sometimes when you JOIN. Things like forget about the Value index and just use a primary key scan for some arms of the join because the index doesn’t look effective enough.

It was clear we’d need to move a lot of query planning to the application, maintain our own “table” statistics, and do app joins instead of Postgres joins if Postgres was going to mess it up. That last part was the last nail in the coffin - we really couldn’t lean on join in PG at all because we had no way to know when the query planner was going to be silly.

It was worth doing for the learning! I merged a PR deleting the EAV code about a month ago, and we rolled out a totally different design to production last week :)

I've also found triple stores to have terrible performance, but it looks like the intended use-case for this (like Firebase) is rapid development, prototyping, and startups. You aren't going to generate enough traffic when you're building an MVP for this to be an issue.

And it's a hosted service, so the performance issues are for the InstantDB team to worry about, and they can fold it into the price they charge. It does mean that your application architecture will get locked in to something that costs a fortune in server bills when it gets big, but from InstantDB's POV, that's a feature not a bug. From your POV as a startup it may be a feature as well, since if you get to that point you'll like have VC to blow on server bills or use to rewrite your backend.

So far we haven't hit intractable problems with query performance. One approach that we could evolving too down the road is similar to Tao [1]. In Tao, there are two tables: objects and references. This has scaled well for Facebook.

We're also working on an individual Postgres adapter. This would replace the underlying triple store with a fully relational Postgres database.

[1] https://www.usenix.org/system/files/conference/atc13/atc13-b...

Absolutely. Instant has similar design goals to Rails and ActiveRecord

Here are some parallels your example:

A. ActiveRecord:

```

post = Post.find_by(author: "John Smith") post.author.email = "john@example.com" post.save

```

B. Instant:

```

db.transact( tx.users[lookup('author', 'John Smith')].update({ email: 'john@example.com' }), );

```

> In React/Vue/Solid, I want to say express things like this:

Here's what the React/Vue code would look like:

```

function BlogPostDetailComponent(props) {

  // `useQuery` is equivelant to the `subscribe` that you mentioned:

  const { isLoading, data, error } = db.useQuery({posts: {author: {}, $: {where: { id: props.id }, } })
  
  if (isLoading) return ...
  
  if (error) return .. 
  
  function updateAuthorName(newName) {
  
    // `db.transact` does what you mentioned: 
    // it attempts to persist any pending changes to browser storage, then
    // sync to remote db, rolling back changes if there's a failure, and
    // gives an easy way to show an error toast if the update failed. (it's awaitable)
  
    db.transact(
      tx.authors[author.id].update({name: newName})
    )
  
  }

  return (
    <>
      ...
    </>
  )

```

From what you say, seems like Meteor + React would deliver almost the exact syntax you want, although it's MongoDB instead of SQL.

Reference: https://react-tutorial.meteor.com/simple-todos/02-collection...

Every day, we get closer to what Ember.js did/does.

> Is the datalog engine exposed? Is there any way to cache parsed queries?

We don't currently expose the datalog engine. You _technically_ could use it, but that part of the query system changes much more quickly.

Queries results are also cached by default on the client.

> Other datalog engines support recursive queries, which makes my life so much easier. Can I do that now with this?

There's no shorthand for recursive queries yet, but it's on the roadmap. Today if you had a data model like 'blocks have child blocks', you wanted to get 3 levels deep, you could write:

```

useQuery({ blocks: { child: { child: {} } } });

```

> Also, why the same name as the (Lutris) Enhydra java database?

When we first thought of the idea for this project, our 'codename' was Instant. We didn't actually think we could get `instantdb.com` as a real domain name. But, after some sleuthing, we found that the email server for instantdb.com went to a gentleman in New Zealand. Seems like he nabbed it after Lutris shut down. We were about to buy the domain after.

> Given that it's implemented clojure and some other datalog engines are in clojure, can you say anything about antecedents?

Certainly. Datomic has had a huge influence on us. I first used it at a startup in 2014 (wit.ai) and enjoyed it.

Datalog and triples were critical for shipping Instant. The datalog syntax was simple enough that we could write a small query engine for the client. Triples were flexible enough to let us support relations. We wrote a bit about how helpful this was in this essay: https://www.instantdb.com/essays/next_firebase#another-appro...

We studied just about all the codebases you mentioned as we built Instant. Fun fact, datascript actually supports our in-memory cache on the server:

https://github.com/instantdb/instant/blob/main/server/src/in...

This is from me. I didn't realize the connection to Lutris + Enhydra. It should be listed as a "Acquired Company" + "Abandoned Project". Wikipedia also says that it lasted until 2001. Usage is different from development/maintenance. I will update the entry for the old InstantDB and add an entry for this new InstantDB.

I think given that the original InstantDB died over two decades okay and is not widely known/remembered, reusing the name is fine.