Why do we need yet another database?
You like to spend a lot of time defining a great interface for your RPC.
This is why you picked gRPC + Protocol Buffers instead of a JSON REST API.
Protobuf is great! Messages are extremely small and quick to parse, and then gRPC takes care of sending them over HTTP/2, you even get the bidirectional stream coolness!
But then, you want to store your messages, so you have to convert them to SQL… Not cool!
ProfaneDB wants to address this. It adds a simple key option in your Protobuf definitions, and then you can store your messages as they are, and retrieve them by their unique key.
What is ProfaneDB?
This makes ProfaneDB a Key-Value database, that does its best not to reinvent the wheel.
Keys are very important: as a Protobuf message can contain many others, nested messages can’t be duplicated.
Why Protobuf? What about
your IDL of choice?
ProfaneDB revolves around Protocol Buffers because I like it very much.
However, because of the way it’s designed, most IDLs (eg Apache Thrift, Cap’n Proto, even plain JSON) could probably be adapted to the same structure.
ProfaneDB core does nothing more than getting a unique key for a message and all its nested messages, and store them in a flat KV storage (in this case RocksDB, but this as well, could be LevelDB, Redis, etc…)
As with Protobuf, I liked it and used it before.
However I’m considering plain old SQL databases as well (yes, the same I was despising at the beginning of this FAQ: I like structured storage, I just don’t like writing SQL CREATE queries when I have my Protobuf definition already)
RocksDB can be embedded, do I really need gRPC overhead?
As with the two previous questions, I think this can be avoided too.
The build process already makes a
libprofanedb.so object that doesn’t depend on gRPC.
I can see many cases where one would want to embed ProfaneDB (just like SQLite),
however, I’m focusing on gRPC too because I think we might get to a point where one could call ProfaneDB straight from a mobile app.
Does it scale? What about replication? Can you add
Likely not on my own.
So should you have any great idea, would like to implement it, or have suggestions and fixes, I encourage you to help me out!
I’ll be very grateful, and would love to learn more!