fbpx
The best IoT Databases for the Edge – an overview and compact guide

The best IoT Databases for the Edge – an overview and compact guide

For many IoT projects, relying on the cloud for data storage and analysis is inefficient and costly. Some IoT projects, however, cannot work cloud-based due to hardware or network/bandwidth limitations or a need for realtime response rates. Then Edge Computing is the only solution. And in order to bring computing closer to the source of the data, you need an IoT database optimized for the edge

What to look for in an IoT Database for the Edge

There are several things to consider when choosing an IoT database for the edge. The five most important criteria to take into account are the edge-capability, performance, ACID-support, language support and data type support.

Edge-capability

Of course, in order to qualify as edge-capable, the database needs to run directly on a broad spectrum of edge devices – either embedded or in-memory. Many IoT devices are physically small and have limited resources, so a database for the edge needs to have a small footprint. For that reason, the list below does not include databases with a core library larger than 10MB.

Performance

Many edge cases have a need for speed; for example: In additive manufacturing making necessary adjustments to the next layer added to an asset needs to happen in near real-time. Because this decision is based on a multitude of environmental factors from the factory floor, tons of data from sensors need to be processed extremely quickly.

ACID-compliance

Depending on whether you can afford to lose some of your data sometimes, you need to check if the database is fully ACID compliant – and under which conditions any benchmarks have been run. What does ACID mean? In the database world, this popular acronym refers to how data in transactions is handled by a database and stands for: Atomicity, Consistency, Isolation, Durability. In short, a fully ACID-compliant database is transactionally safe and ensures that, despite errors, power failures etc., no data is lost and the transactions are always executed in a valid way.

Language support

Another important criteria obviously is the language used to implement the database: Does it match your project’s language and developer skills? Generally speaking it is more efficient to keep to one language; this is also why many developers love to avoid dealing with SQL.

Data Type support

Finally, you need to decide on the overall structure data shall be stored in. In this article, we will only focus on full databases that enable complex computing on small devices – thus it only includes traditional databases, i.e. those that are relational, object-oriented or graph-based. Databases that are limited to time-series data only (e.g. InfluxDB, TimescaleDB) or any ORMs will not be discussed here.

IoT Databases for the Edge

In order to help you in choosing the best IoT database for your next project swiftly, we had a look around and compared available databases. Here is a list of IoT databases for use on the edge:

Badger calls itself a distributed, fast graph database. It is an ACID-compliant, NoSQL, LSM tree-based key-value store written fully in and available only for Go. As Badger does not focus on being run on IoT devices, it supports easy horizontal scaling, synchronous replication to prevent data loss, load balancing and using the full capacity of SSDs instead of the RAM. Central or P2P synchronization are not available.

Berkeley DB is an ACID compliant embedded key-value store. Due to a static library size of less than 1 MB, and runtime dynamic memory requirements of only a few KB, it is suitable for a variety of edge devices. The database is usable in many different languages, such as C++, C#, Java, Perl, PHP, Python, Ruby, Smalltalk and Tcl. Berkeley DB does not offer any synchronization support.

LevelDB is a key-value storage library that provides an ordered mapping from string keys to string values. It is written in C++ and has bindings for languages such as C, Go, NodeJS and Java. LevelDB runs on-disk and is queried without SQL. Applications need to use it as a library, as the database does not provide any server or command line interface. Indexes and synchronization are not supported.

ObjectBox is a fast, object-oriented, ACID-compliant database with strong relation support. It was designed specifically for Edge IoT and embedded and mobile applications. ObjectBox has a memory footprint of less than 1 MB. Language support includes C, Go, Java, Kotlin, Swift, Python (Beta), and Dart. Centralized synchronization support is currently in its alpha stage and P2P synchronization is a work in progress.

Realm, which has been acquired by MongoDB in summer 2019, is an ACID-compliant NoSQL database. It has been strongly focused on mobile platforms from its start and is only beginning to move into IoT. It offers central as well as P2P synchronization. Supported languages include Java, Kotlin, Swift, C# and NodeJS.

Redis is a key-value database, which is per default not ACID-compliant. However, it offers an optional durability transaction concept, which when turned on reduces the database performance significantly. In contrast to other database systems, it works in memory with user commands not being data queries, but specific operations to be performed on abstract data types. Redis has many different client bindings, e.g. C, C++, Dart, Go, Java, NodeJS, Python and Rust.

RocksDB is a persistent key-value store for SSD and RAM storage. It is not ACID-compliant, but works using concurrent transactions with conflict resolution. This embedded NoSQL database supports Java, Python, NodeJS, Go, PHP and Rust, to name only a few languages. Synchronization in any form is not natively possible with RocksDB.

SQLite is the only fully SQL-based relational database library in this list. SQLite comes with a small footprint and is fully ACID-compliant. It offers encryption as a paid service. There is no support for synchronization.

Let us know your thoughts!

Different use cases call for different databases, and we hope that this list gives you a good starting point for your edge computing project. Let us know your thoughts in the comments below – what is your favorite database work with with and why?

How to set up ObjectBox Go on Raspberry Pi

How to set up ObjectBox Go on Raspberry Pi

ObjectBox is a fast object-oriented database for IoT and Mobile edge computing. With its Go support, ObjectBox is a perfect match for Raspberry Pi and will add instant speed to your next Raspberry project.

To get started right away, jump straight to the Installation section.

Local data persistence

In general, databases are great to add data persistence to your project. If you need to turn off the device or if there is a power outage, your sensor statistics or other interesting information are safe. The next time you boot up the application it’s there again as if nothing happened. Another rather obvious advantage is speed and independence from an Internet connection. Storing data on a local drive is just faster than sending it to the cloud and back. And when you store your data locally, your application works with or without an Internet connection. This also means, you have full control over all your data, especially regarding who shall be able to access it… Finally, this reduces cloud costs.

For many databases (e.g. SQLite, PostgreSQL or MySQL), you need to know SQL. And for managing complex datasets, you need to be really good at it. Some like it, but for many developers this is annoying. Also, you might worry about SQL injection attacks.

Advantages of ObjectBox

ObjectBox is a database uniquely designed to sit on IoT and Mobile devices and fix these issues.

Firstly, you do not formulate your queries in SQL. ObjectBox comes with easy APIs allowing you to use the constructs and syntax native to the language you are using anyway. In this way ObjectBox is a NoSQL database. Secondly, benchmarks show that ObjectBox is faster than alternatives. So, even if you’d like to insert the data of a lot of sensors simultaneously, you don’t need to worry about speed anymore. This has the nice side effect that more speed means less CPU and thus is resourceful in every respect. Last not least, because ObjectBox is designed for small devices, it just needs about 1 MB of disk space. If you need something smaller, you can check out the ObjectBox client for Azure Sphere.

If you want to use ObjectBox in your next Raspberry Pi project, here is how you get started:

Installation

We’ll talk about the Go version of ObjectBox here. There are bindings for C, Java and Swift as well – and we will publish articles in the future about them too.

First, you need to have access to the shell (i.e. command line) of your Raspberry Pi. It does not matter if you’re connected via SSH or directly, just make sure that your Raspberry Pi has a working internet connection. Then, just execute the following command to get started right away:

This creates a folder named go in your home directory with the following subdirectories:

  • go with Go 1.12.7 installation, unless you already had Go 1.12+ installed
  • objectbox with the shell script update-objectbox.sh you can execute to easily update ObjectBox upon a new release
  • projects/objectbox-go-test mainly with the file main.go which contains a demo application, with an entity model in its own directory.

Note: The script will install libobjectbox.so globally (confirm by typing “y”), so it might prompt you to enter your root password.

Working with ObjectBox Go

The first tool you will come across is objectbox-gogen. It generates the correct Go files out of a Go structure to make it usable with ObjectBox. Assuming your main.go looks something like this:

As long as you only change the program’s logic, you don’t need to do anything. If you change the model, simply execute go genereate ./... to update the auto-generated files for the ObjectBox model interface. Afterwards, you can run your program using go run . .

As a result, you’ll get a directory called objectbox with the files data.mdb and lock.mdb; these are the underlying database files and they will be reused when you execute the program again.

Summary

So, that’s basically it! Setting up ObjectBox and using ObjectBox is easy. Check it out yourself and please share your thoughts with us; we appreciate any feedback. 🙂

Until we release the next part of this tutorial discussing the features of ObjectBox in Go more thoroughly, feel free to check out the fully-fledged demo application in its official repository. Also, if you’d like to learn more about all other features of ObjectBox Go, have a look at the official documentation!

If you encounter any problems during this process, please reach out to us. The best way is to ask a question and tag it “ObjectBox” on Stack Overflow. Finally, we would love hearing your thoughts on ObjectBox via TwitterFacebook, or Mail (contact [at] objectbox . io).

Azure Sphere & ObjectBox: IoT Sensor Demo

Azure Sphere & ObjectBox: IoT Sensor Demo

A few weeks ago, we published ObjectBox for Azure Sphere. Today, we are adding IoT sensor data collection to this. It’s a working demonstration that you can run on your machine along with an Azure Sphere board. To jump straight to it, here is the repository.

Forwarding Sensor Data to ObjectBox Database

We added an example to the Azure Sphere ObjectBox repository illustrating how to read IoT sensor data and forward them to an ObjectBox database. For integrating the sensors, the example relies on the Grove Shield library from the manufacturers of Azure Sphere, Seeed Studio. This library makes reading sensor information super simple. Next, the collected sensor data is sent via an ObjectBox REST Client to a device running a ObjectBox database. Here, sensor data can be processed further.

The demo setup comes with a ready-made ObjectBox HTTP server for download. The Azure Sphere client application has to be built from sources as the IP of the server needs to be white-listed here. Please find a step-by-step guide in the repository.

Setup Overview

Have a look at the general application architecture to get the gist of the demo:

Browsing collected Sensor Data

Once you collected sensor information like light intensity, temperature, and humidity, you may want to view it. The most simple option is the “ObjectBox Browser” that comes embedded with the ObjectBox HTTP server.

Let us know what you think

So, of course we are looking forward to your feedback again! Do you have a use case in mind that you want to discuss with us? Also, please feel free to open a new GitHub issue if you run into a problem or ask a question tagged ObjectBox on Stack Overflow. And finally, don’t hesitate to share your thoughts on ObjectBox / Azure Sphere with us via Twitter, Facebook, or Mail (contact [at] objectbox . io).