Zetabase Overview: Use Cases

  • By Jason Victor
  • July 9, 2020

What are the coolest use cases of Zetabase? You tell us! We designed a system to allow new types of architectures and software, and we're excited to see what inventions may result from that. But, of course, we had some thoughts in mind at time of writing about what we wanted to use it for, and that's what this post is about.

Web applications and services

The three-tier architecture — HTML/JS, REST service, database — has dominated the landscape. In fact, it has so dominated the way we think about web development that job titles have arisen around it. The title "frontend engineer" only makes sense if you assume only HTML/JS for a single job, no backend experience needed; "dev ops engineer" only makes sense if you need to scale backend services; and so on.

It is our very humble opinion that the three-tier architecture is garbage. This architecture might be easy to get going, but it suffers from scalability woes — any backend service must be scaled, and that's complicated.

Zetabase enables a wide variety of alternatives. The first, and perhaps most important, is this:

In this setting, the database connection is established from the web client; background workers can consume and publish data as necessary. This setup works in a very wide variety of settings, and it makes scaling such applications trivial.

Or, there's the decentralized model: something that is only now possible using Zetabase. You can think of this kind of like a blockchain application, but without the blockchain. In this setting, the database is a shared resource among multiple parties. Clients can be developed by anyone, in theory, using this architecture; but most importantly, the parties can all use the database without trusting each other necessarily. We'll do a deeper dive on these features in a later post.

Trading and finance

A number of our team members have grown up around trading and finance, and Zetabase is set up to handle for the idiosyncrasies of this very important use case.

To start, Zetabase has security properties that make it desirable for financial data: a leaked password or internal mischief will not lead to any kind of financial impact, since nobody would have access to private keys but the person in charge.

It also works well for a number of financial use cases, like time-series data. The Python client allows users to query data directly into a Pandas data frame for subsequent analysis:

from zbpy import queries

age = Field('age')
name = Field('name')

query = ((age == 19) | ((age > 25) & (age <= 27))) & (name == 'Austin')
result = client.query('TABLE ID', query)
df = result.to_dataframe()

And boom, you're ready to analyze it.

This is very useful for storing data sets. For example, one could create a table for storing snapshots of markets at a given point in time. If each one is inserted with a key of the form symbol/timestamp, then querying for all objects of the form symbol/% (the % is the wildcard operator) gives you the time series for that particular symbol.

Storing a security master or asset universe? Fear not! The semi-structured approach of Zetabase will allow you to index the fields that are common to many instruments (like symbol or expiry/maturity) while adding whatever extra fields are necessary for individual instruments (e.g. an option would have strike, type, etc.).

Mobile apps

The Great Mobile App REST API is one of the largest sources of waste in modern computing. Swift developers are more than capable of writing an entire application front-to-back. Their only need for backend developers arises because machine code can be decompiled and inspected, and is therefore unsuitable for containing a critical database secret; thus, they are beholden to REST services that don't suffer from that issue.

Zetabase answers the question: why can't mobile devs just hit the database directly from their frontend code? The answer used to be "security," but Zetabase's system of permissions and constraints enables precisely this behavior. By restricting access to the database based on the content of the data involved, we have a simple model wherein the client application can also be the client to the database.

For example, imagine a mobile app that allows users to talk on a message board. When a user wants to send a message, normally, a JSON body like this would be posted to a REST endpoint:

{
  "user_id": "...",
  "message": "..."
}

Really, all we're saying here is that a user can post a message, and we require (or "constrain") that the user_id field contain the user's actual ID (so that users can't forge requests). So, we can create a table to do just that: it only allows well-formed messages that the user is "allowed" to write. This way, the mobile app talk directly to the database, avoiding the need for scaling a backend REST service. Indeed, this entire mobile app could run with just the frontend binary and Zetabase — no backend needed. More on that is contained in this document.

Machine learning

Zetabase is great for storing data sets, pre-processed data, and trained models. Furthermore, it can be used to great effect in settings where the data cannot be easily held in RAM at once: in these settings, the data can be streamed and the model trained in online fashion. A wide variety of algorithms support this kind of behavior, from the deep nets used in image recognition or NLP to Follow the Regularized Leader.

Critically, the Zetabase client for Python is deeply integrated into this toolchain. It has built-ins for storing and loading data as NumPy arrays and as Pandas data frames. For example, to create a new table with indexed fields and populate it all from a Pandas dataframe, one would use something like:

client.put_dataframe_new_table('Table ID', yourDataFrame, 'DF KEY', specify_fields=['age', 'height'])

A table has now been created with the specified fields indexed. (By default, all fields are indexed.) Never before has it been so simple to manipulate large data sets in Python!

Get started!

Hopefully you're as excited to get started working with Zetabase as we are to present it to you! We look forward to hearing more from you. Please consider joining our community Slack or following us on Twitter — and of course, get started with Zetabase!