fbpx
The best IoT Databases for the Edge – an overview and compact guide

The best IoT Databases for the Edge – an overview and compact guide

For many IoT projects, relying on the cloud for data storage and analysis is inefficient and costly. Some IoT projects, however, cannot work cloud-based due to hardware or network/bandwidth limitations or a need for realtime response rates. Then Edge Computing is the only solution. And in order to bring computing closer to the source of the data, you need an IoT database optimized for the edge

What to look for in an IoT Database for the Edge

There are several things to consider when choosing an IoT database for the edge. The five most important criteria to take into account are the edge-capability, performance, ACID-support, language support and data type support.

Edge-capability

Of course, in order to qualify as edge-capable, the database needs to run directly on a broad spectrum of edge devices – either embedded or in-memory. Many IoT devices are physically small and have limited resources, so a database for the edge needs to have a small footprint. For that reason, the list below does not include databases with a core library larger than 10MB.

Performance

Many edge cases have a need for speed; for example: In additive manufacturing making necessary adjustments to the next layer added to an asset needs to happen in near real-time. Because this decision is based on a multitude of environmental factors from the factory floor, tons of data from sensors need to be processed extremely quickly.

ACID-compliance

Depending on whether you can afford to lose some of your data sometimes, you need to check if the database is fully ACID compliant – and under which conditions any benchmarks have been run. What does ACID mean? In the database world, this popular acronym refers to how data in transactions is handled by a database and stands for: Atomicity, Consistency, Isolation, Durability. In short, a fully ACID-compliant database is transactionally safe and ensures that, despite errors, power failures etc., no data is lost and the transactions are always executed in a valid way.

Language support

Another important criteria obviously is the language used to implement the database: Does it match your project’s language and developer skills? Generally speaking it is more efficient to keep to one language; this is also why many developers love to avoid dealing with SQL.

Data Type support

Finally, you need to decide on the overall structure data shall be stored in. In this article, we will only focus on full databases that enable complex computing on small devices – thus it only includes traditional databases, i.e. those that are relational, object-oriented or graph-based. Databases that are limited to time-series data only (e.g. InfluxDB, TimescaleDB) or any ORMs will not be discussed here.

IoT Databases for the Edge

In order to help you in choosing the best IoT database for your next project swiftly, we had a look around and compared available databases. Here is a list of IoT databases for use on the edge:

Badger calls itself a distributed, fast graph database. It is an ACID-compliant, NoSQL, LSM tree-based key-value store written fully in and available only for Go. As Badger does not focus on being run on IoT devices, it supports easy horizontal scaling, synchronous replication to prevent data loss, load balancing and using the full capacity of SSDs instead of the RAM. Central or P2P synchronization are not available.

Berkeley DB is an ACID compliant embedded key-value store. Due to a static library size of less than 1 MB, and runtime dynamic memory requirements of only a few KB, it is suitable for a variety of edge devices. The database is usable in many different languages, such as C++, C#, Java, Perl, PHP, Python, Ruby, Smalltalk and Tcl. Berkeley DB does not offer any synchronization support.

LevelDB is a key-value storage library that provides an ordered mapping from string keys to string values. It is written in C++ and has bindings for languages such as C, Go, NodeJS and Java. LevelDB runs on-disk and is queried without SQL. Applications need to use it as a library, as the database does not provide any server or command line interface. Indexes and synchronization are not supported.

ObjectBox is a fast, object-oriented, ACID-compliant database with strong relation support. It was designed specifically for Edge IoT and embedded and mobile applications. ObjectBox has a memory footprint of less than 1 MB. Language support includes C, Go, Java, Kotlin, Swift, Python (Beta), and Dart. Centralized synchronization support is currently in its alpha stage and P2P synchronization is a work in progress.

Realm, which has been acquired by MongoDB in summer 2019, is an ACID-compliant NoSQL database. It has been strongly focused on mobile platforms from its start and is only beginning to move into IoT. It offers central as well as P2P synchronization. Supported languages include Java, Kotlin, Swift, C# and NodeJS.

Redis is a key-value database, which is per default not ACID-compliant. However, it offers an optional durability transaction concept, which when turned on reduces the database performance significantly. In contrast to other database systems, it works in memory with user commands not being data queries, but specific operations to be performed on abstract data types. Redis has many different client bindings, e.g. C, C++, Dart, Go, Java, NodeJS, Python and Rust.

RocksDB is a persistent key-value store for SSD and RAM storage. It is not ACID-compliant, but works using concurrent transactions with conflict resolution. This embedded NoSQL database supports Java, Python, NodeJS, Go, PHP and Rust, to name only a few languages. Synchronization in any form is not natively possible with RocksDB.

SQLite is the only fully SQL-based relational database library in this list. SQLite comes with a small footprint and is fully ACID-compliant. It offers encryption as a paid service. There is no support for synchronization.

Let us know your thoughts!

Different use cases call for different databases, and we hope that this list gives you a good starting point for your edge computing project. Let us know your thoughts in the comments below – what is your favorite database work with with and why?

Car Tolling – A case for Edge Computing

Car Tolling – A case for Edge Computing

Governments often face tight budgets on infrastructure development; car tolling is increasingly seen as the answer for raising funds¹, making it more and more prevalent. From 2008 to 2018 the total length of tolled roads in Europe increased by 23%² and tolling revenue in Europe increased by 37%³ to €31.3 bn. per year; similarly, from 2010 to 2015 the United States experienced a 63% increase in transponders and 52% more tolling revenue, resulting in $13.8 bn. in 2015. On top, despite car sharing efforts, car ownership and traffic is still increasing in many countries, e.g. Germany, France and India. Increasing amounts of traffic, devices, and data points bring current tolling solutions to their limits. Taking data to the edge in new and existing tolling solutions, for example with the ObjectBox data storage and synchronization solution, can make tolling more efficient and reliable.

Setting the stage: a typical car tolling situation

A national infrastructure company has deployed several hundred car tolling stations all over the country. These stations automatically recognize passing cars by detecting licence plates, using visual recognition or wirelessly, e.g. by receiving data from an RFID transponder in the car. In order to ensure that only eligible cars are passing through the tolling station and violators are fined, it is necessary for the tolling station software to look up the gathered vehicle information – among millions of entries – as fast as possible. If the data look-up is not  fast enough, or the data on the roadsides/tolling stations isn’t up to date and in sync with the central data, the tolling station loses money.

“The importance of mobile apps is increasing for Kapsch TrafficCom so that we see ObjectBox’ edge computing database solution as an interesting future base technology for all types of mobility apps.”

Peter Ummenhofer

Executive VP Solution Management, Kapsch TrafficCom

Why edge computing and fast lookup is key to today’s car tolling systems

In general, modern nationwide tolling infrastructure consists of three systems: tolling stations operated by the respective agencies, central open road, also called mobile tolling, and central transaction clearing houses. Within this infrastructure, all data related to violators and other operational information needs to be synchronized between these three systems in a consistent way, with as little delay as possible. If this is not the case, together with other problems, car tolling system operators are faced with high monetary losses every day.

Today’s car tolling systems are based on the fundamental idea that cars do not need to stop to be checked or charged. Thus, as the cars move quickly through the scanning area, the challenge of implementing a car tolling system directly relates to the amount of data that needs to be searched within a very short time frame.  To be successful, this process needs to happen in near real-time. From a development perspective, these problems are rooted in:

  • accessing data from a remote location (speed of communication, speed of network)
  • keeping data in synchronization with car tolling stations that are closer to the drivers and/or roadside units
  • database speed on remote servers
  • database speed on roadside units (car tolling edge devices)
  • limitations of existing hardware as some systems are quite old, and rolling out new hardware is expensive

Furthermore, it is possible that stations shut down from time to time, due to the weather, power outages, vandalism or simply technical failures. However, tolling providers generally need to provide strict uptime guarantees and thus service level agreements often include penalty fees in case of excessive downtime. Such events cost the providers substantial amounts of money – and data loss, i.e. undetected violators, even more so.

Adding to this, privacy and legal requirements differ from country to country and increase the complexity of the systems and timings. For example, in Austria the pictures and derived license plate information may only be used for checking, but in case no violation was detected, they need to be removed in an unrecoverable manner¹⁰. On the other hand, the data of potential violators may be stored for the sole purpose of toll collection or prosecution, but only for a maximum of three years.

How fast data storage and syncing can help in car tolling

To solve these problems, a data storage and data synchronization solution like ObjectBox can be deployed on every type of tolling station, i.e. open and static stations, as well as on the central server. From a technical point of view, this is not a problem, because the ObjectBox library supports virtually all platforms and operating systems. Financially, it is considerably cheaper to update software, than it is to upgrade hardware.

Having the library installed, with ObjectBox Sync, it is guaranteed that the vehicle data in the internal stations’ memory is always up-to-date with the central server, so the station will make a decision based on the most accurate data every time. Additionally, the other systems involved in the tolling infrastructure consistently receive the most recent information with no further effort required.

Deploying the synchronization solution also means, because ObjectBox is particularly reliable (ACID compliant) and well-tested, that station shutdowns or internet connection issues are not a problem anymore. The stations’ operating company will no longer lose violator’s information due to technical reasons.

Summary – Car tolling is moving to the edge

As this case study shows, the use of edge computing is a perfect fit for modern infrastructure. In the context of car tolling, speed, reliable data storage and synchronization are indispensable, resulting in ObjectBox being an effective solution for today’s and future technological advancements.

If you are interested in learning more, feel free to get in touch with us! We appreciate any kind of feedback.

How to benchmark database performance – and ObjectBox

How to benchmark database performance – and ObjectBox

Benchmarking the performance of databases is a science in and of itself and it’s hard to get reliable and comparable results. Therefore, we decided to note down some standard patterns and pitfalls when doing database benchmarks we learned about over the years. We are including specific notes and things to consider when benchmarking ObjectBox.

Designing a benchmarking test

Phrase your research question

When you want to benchmark databases, you will usually have a specific use case in mind and want to answer questions regarding that case. Therefore, before writing your first line of code, have a look at what it is you want to benchmark, why, and what statement you want to be able to make at the end. Just writing down a simple question like “Is X faster than Y doing A?” can help you verify that the finished benchmark actually measures what you want it to.

Start documenting

Whatever you do, document each step diligently. Benchmarks that are not properly documented are challenging to reproduce, and thus of limited worth. So, the mantra in benchmarking really is: Document, document, document. That way it is also easier to go back later and make adjustments if needed.

Select the sample

If you have a clear use case and goal in mind, this probably determines the type of databases you are going to consider for your benchmark. Generally speaking, benchmarks should compare databases of similar type and not mix approaches that are too different. A database might be designed to run on a cluster of servers and not, like ObjectBox, on a constrained device (e.g. a smartphone, Raspberry Pi or IoT device). Or data may be stored as documents (e.g. NoSQL) and not in a relational table-like structure (e.g. SQL); or on disk and not in-memory. Consistency guarantees can also make a difference (ACID-compliance vs. not transactionally safe). Document why you chose these databases for comparison.

Become an expert

Once you have decided on what databases and features to compare, familiarize yourself with the APIs of the products, e.g. by reading the documentation or looking at code examples. Making wrong assumptions about how a feature works can skew your results and make a product appear much faster or slower than it actually is. If the code is open source, it can also help to dive into the code to see how things actually work. For example: an insert function runs asynchronously so the function call returns immediately, but the actual insert still executes in the background. To correctly measure the actual time it takes to complete the insert, you need to do some additional work.

Things to watch out for when coding benchmarks

Let’s move on to some specific coding tips. A benchmark is only as good as its time measurement. Check what APIs your test platform offers to get the time; they might be affected by how and from which thread they are called. Make sure, you find an accurate way to measure the time that does not skew up your results. For example, on Android there are numerous possibilities beyond using currentTimeMillis().

If a measured code block executes so fast it is near the available precision of your clock (e.g. time is in milliseconds, but code takes microseconds), consider running it multiple times and measure the total time of all runs.

Next, before starting each measurement, make sure to free up memory and clear references no longer in use by the previous measurement or setup code. If the environment your benchmark is written for uses a garbage collector, check if it can be triggered manually to free memory (e.g. System.gc() in Java). Otherwise each consecutive measurement might be skewed due to less and less memory being available or a garbage collector halting execution to free memory. In your benchmarking results, you should look out for strange results like a continual decrease in performance from run to run.

Furthermore, take into account that some runtime environments, for example the Java Virtual Machine, do just-in-time compilation. This can cause a delay the first time code is executed, but provide better performance on subsequent executions. The effect of this on the final result can however be minimized by proper testing procedure, e.g. by running a code block multiple times instead of once and measuring the total execution time.

Then something obvious, but easily overlooked, is to ensure that between the start and end of a measurement only the functionality that you actually want to compare is executed. So avoid or turn off logging (a seemingly innocent string concatenation can skew results) and construct test data outside of a measured code block.

To be absolutely sure that your code is doing what you intended it to, use a profiler to inspect resource usage during a trial run. IDEs like Android Studio and Xcode come with an embedded profiler, and there are also several standalone profilers to choose from.

Optimizing benchmarks for meaningful results – with ObjectBox examples

Before benchmarking the chosen databases, make sure you understand their differences and default settings to adjust all settings to be comparable. In the following section, we will go through the most important differences and settings you need to look out for based on ObjectBox as an example.

Transactions, Durability, Consistency, ACID – how to make your benchmarks comparable

First, be aware of the impact the use of transactions or lack thereof can have. For databases, committing a transaction is an expensive operation as it requires waiting until the disk has safely stored the data. If possible, group multiple operations into a single transaction. For example, in the ObjectBox code snippet below, there is a notable speed-up when wrapping multiple box operations into a transaction block.

Speaking of transactions, also check if using bulk operations is possible. These also use transactions to speed up execution. E.g. instead of performing a put on each entity in a list, put the whole list.

The ObjectBox transactions docs provide more details and are available for Java, Swift or Go – though the basic principle is the same across languages.

Second, and closely related to transactions, are durability guarantees when writing data. This is about the “D” in the popular ACID acronym (Atomic, Consistent, Isolated and Durable). ObjectBox transactions and standard (non-async) operations are fully ACID-compliant.

Thus, pay close attention to what durability modes other tested databases guarantee, or respectively, which durability mode you want to measure. Most NoSQL databases don’t give hard durability guarantees. Some provide an extra command or special mode to enforce durability. Therefore, if your use case needs to ensure data is actually stored safely after a write operation, you would need to enable this durability for other databases when comparing to ObjectBox.

On the other hand, if you are interested in scenarios that emphasize performance over durability, you should look into the OjectBox async APIs. Those don’t come with durability guarantees unless you define “checkpoints” in your code to wait for async operations.

Indexing – how to make your database queries efficient

Third, when measuring query operations, see if you can use indexes, another typical database optimization. If the database has an index on a property that is used in a query condition, it can find matches much faster.

An index makes queries “scalable” – the more objects are stored, the more an index makes sense. Without an index, a database has to do a “full scan” over all potential results.

Number of objects and the disk bottleneck – how to measure the database and not the disk

And lastly, keep an eye on the number of objects you operate on. For example, if you put a single object, something like 99.99% percent will be spent on disk I/O. Thus, if you test this on several databases, the chance of getting about the same results on all databases is quite high. The limiting factor is always the disk. So if you want to measure the efficiency of converting objects into their persisted counterparts instead, you should look at much higher object counts to factor the disk out of the equation. Depending on the disk and device speed, a bulk “put” of 10K, 100K or 1 million objects will make more sense to measure in this context. 

Multi threaded tests – how to set writers and readers

ObjectBox is build upon a multiversion concurrency control foundation, and thus is ready for multi threaded access. Each thread will have a consistent transactional view of the data. ObjectBox differentiates between “readers” (a thread currently in a read transaction) and “writers” (a thread currently in a write transaction). Readers never block; no matter what goes on in other threads. However, a writer will block other writers when using standard transactions. Writers are sequential; only a single writer can be run at any time. Thus, if your load is write-heavy in multiple threads, you may want to look at the asynchronous APIs of ObjectBox. These handle write operations very efficiently; no matter how many threads are involved.

Extending the database size limit – size matters

By default, ObjectBox limits the database size to 1 GB to avoid filling up the disk by accident (e.g. your code has a bug in a data insertion loop). So if you use large data sets to benchmark, we recommended increasing the maximum database size when building the box store.

Preparing the benchmarks: devices and platforms

Once your benchmark code is ready, it’s time to set up the test device(s). The first step is to ensure other apps or processes are not doing (too much) work while your measurements run. Ideally, you would use a clean device with no apps installed or services configured. This is especially true for mobile devices: once you connect them and they start charging, the operating system might wake numerous apps to perform background work or network updates. You can somewhat avoid this by switching on airplane mode. And to be safe, wait a few seconds after connecting the device.

Also ensure, this is again important for mobile devices, that your device does not enter a low power mode which reduces performance. For example on Android, keeping the screen on typically prevents that. When running on a laptop, check whether the power supply is plugged in and which power mode your operating system is set to. You may explicitly want to test in a certain power state or with the default behavior.

And just to mention it, make sure to turn off any profiling or monitoring tools that you used during building your benchmark. They can significantly skew your results.

When running on multiple devices, pay attention to the differences in hardware, like available memory and processor model, but also in software, like the operating system version. Different hardware and software might have different optimizations that can skew your results. Again, do not forget to document your device and software setup, so results can be properly interpreted and reproduced.

Running the database benchmark and collecting results

When it is time to run the benchmark it’s good to run it not once, not twice, but many times. This minimizes the impact of the various side effects we discussed above. Output the results of each run into a comma separated (CSV) or tabulator separated (TSV) format, that can easily be imported in a spreadsheet application for analysis. You can look at how the ObjectBox benchmark does it.

Once you have collected some data, verify its quality. You might have already spotted outliers while perusing the results. Alternatively, calculate the average of all measurements and see if it deviates a lot from their median value. If you are familiar with it, look for a high variance value instead. Too many outliers or a high variance might hint at side effects of your benchmark code or device setup you have not considered. Better double check to be sure.

How to publish meaningful database performance benchmarks

The final step is to share your results with the world (or just your team). Make sure it’s clear what exactly you have measured and how you have arrived at those results. So include which device and software was used, how they were configured, what you did before running the benchmark and how the benchmark was run.

If possible, share the generated raw data so others can verify that your calculations, and remember to lead to the published results. Even better, publish the source code as well, so others can run it on other devices or help you spot and fix issues.

Last not least: Be careful to draw conclusions. Rather let the data speak for itself. Respond to questions and feedback. Be honest, if you learn your benchmarks may be skewed and update them. In the end, everyone wins by getting more meaningful results.

ObjectBox Go 1.0 release and performance benchmarks

ObjectBox Go 1.0 release and performance benchmarks

ObjectBox 1.0 is here. Late last year, we introduced ObjectBox to the world of Go – and were awestruck by the GitHub traction and feedback we got. So we doubled down on our efforts and are thrilled to announce 1.0 with many new features, a stabilized API, and improved performance. From now on your projects are safely able to upgrade.

For those of you new to ObjectBox, here is how ObjectBox persists Go structs (objects):

ObjectBox Go 1.0 – Highlights

Let’s recap what ObjectBox Go brings to the table:

  • automatic schema migration (e.g. adding new structs or fields just works)
  • type-safe APIs, e.g. no interface{} arguments
  • embedded database – no server needed, all data stored on the device
  • Go structs as data (also embedded structs); no ORM, no SQL
  • relations: to-one, to-many (eager and lazy fetching)
  • powerful queries with indexes for scalable lookups
  • ACID compliant with implicit (automatic) and explicit (user defined) transactions
  • superfast bulk/batch operations
  • low memory usage
  • runs on 64-bit Linux, macOS, Windows, small 32-bit ARM-based Linux devices (e.g. Raspberry Pi)

How fast is ObjectBox Go? Performance Benchmarks

To see how ObjectBox compares performance-wise, we looked at libraries with comparable levels of storage abstraction – so not just plain SQL/Key-value storage but also ORM features. Finding embedded data storage libraries satisfying those simple criteria wasn’t so easy, but we have picked a few based on GitHub popularity:

  • GORM using SQLite – GORM supports multiple backends including the embedded SQLite database.
  • Storm using bbolt – while Bolt DB is not maintained anymore, it’s fork bbolt seems to be picking up traction.

What we tested

To get an overview of the databases, we tested CRUD (create, read, update, delete), indexed and non-indexed queries. Additionally, we have compared a peak memory usage of each database during a test run. Each test was run multiple times, with garbage collection turned off and executed manually outside of the measured time. Data preparation and evaluation was also done outside of the measured time.

We tried to keep the test implementations as close as possible to each other while picking the approaches recommended by the docs for each database. The test code is available at https://github.com/objectbox/objectbox-go-performance.

Test setup

The benchmarks were the only foreground application running on the PC and the given results are an average over 20 runs.

Specs: i7-8850H laptop CPU, Manjaro 18, Linux 4.19.56
Disk speeds (LUKS encrypted SSD): ~ 480 MB/s read, ~120 MB/s write 

 

CRUD

All operations were executed in batches, not one-operation-per-transaction. For GORM and Storm it was necessary to do a batch insert, update and remove inside a manual transaction because they don’t support batch operations out of the box.

The numbers were measured by running benchmarks for 100.000 object-sets for ObjectBox and GORM, while we have chosen to use just 10.000 objects with Storm because it was achieving better operations per second with a smaller set. We will follow up with a post comparing how each database fares with different data sets.

The standard operations you would be using all the time with any database show between 6.8 times speedup (update ObjectBox vs Storm) up to 124 times speedup (Create ObjectBox vs Storm). 

Queries

Next we looked at query performance by executing two simple queries. The first test uses an index (actually the “primary-key”) to fetch 100 objects. The second one to find 1111 objects based on a string prefix match (not indexed). The tests were executed 20 times and the average times (ms) are shown below. Note, the arbitrary number, 1111, is just based on how the test code prepares the data (strings contain an object ID, thus 1111 objects out of 10.000 start with the number “1”).

GORM/SQLite performance is very good for a primary-key based query but falls off when it has to use “LIKE” over the whole dataset without an index. Storm doesn’t catch up in either case.

Memory usage

Lastly, we compared the memory usage of each solution. The peak memory usage was measured externally, executing the benchmark binary using /usr/bin/time -v, taking the “Maximum resident set size”. The memory-usage test was run on 10.000 objects with one “test-run” (as opposed to 20 runs in previous benchmarks), repeated manually to make sure there’s no reasonable difference between the runs.

This benchmark shows massive difference in memory usage, which is a good thing to think about not only for small devices but also for cloud-based resources (where RAM is comparatively pricey).

Of course any results you get from performance benchmarks will always be just that, benchmarks. You should consider your application needs and, ideally, benchmark the critical scenarios yourself. If you do, here you can read up a couple of tips on how to do good benchmarks. And because sharing is caring: Do let us know what you find out. Having said that, ObjectBox does show clear lead across the board so why don’t you go ahead and start using it now 🙂

Getting started

Check the ObjectBox Go docs which come with setup instructions, an introduction and information on many more topics. The ObjectBox Go GitHub repository contains all sources if you are interested in internals. Also, it comes with an example application.

Outlook

With version 1.0 we hit an important milestone. We’re looking forward to your feedback and we’ll progress with additional features around queries, relations, and async operations. And of course, data synchronization that will allow you to share data among servers and clients. Stay tuned and let us know your thoughts.

 

 

 

Edge Computing Case Study with easyGOband: How ObjectBox enables compatibility from Android, iOS, and Raspbian to Linux

Edge Computing Case Study with easyGOband: How ObjectBox enables compatibility from Android, iOS, and Raspbian to Linux

Christian Bongardt

Christian Bongardt

CTO, easyGOband

In this case study, we talked with easyGOband CTO Christian Bongart about their implementation of ObjectBox in a cashless payment and access control solution, which spans across devices from Android to Raspberry Pi.

Alyssa – Hi Christian, thanks for joining me. Can you quickly introduce yourself and easyGOband?

Christian – Hi Alyssa, thanks. I am the co-founder of easyGOband and the CTO. We founded easyGOband back in 2017 as a product for music festivals. We introduced RFID wristbands as an access control system and as a payment solution for music festivals, since they have issues with connectivity.

Normally festivals only accept cash or they have a plastic token system. easyGOband, is a cashless system where you link your entry ticket over web application with your barcode. You can prepay your Near-Field Communication (NFC) wristband with lets say €20, for example. We activate and validate your ticket through the access control, and we hand you over the activated NFC wristband which would then contain the €20. Each seller then has an android device, which is like a small POS device, where you can enter the products you want to charge and the balance gets removed from the wrist band.

A – So, tell me a bit more about where the data sits.

C – The actual data is all stored in the wrist band and on the POS device. So it works in low connectivity environment because music festivals the massification of all the people together adds connectivity issues. Antennas can’t make it. Wifi is also a problem because of interference with audio devices, microphone and stuff like this – it is very hard to have a good connectivity. Other music festivals have invested online solutions with WiFi and they always have big problems with it because music festivals have 60,000 – 70,000 attendants and then the whole payment system goes down – it’s a catastrophe. That many people – no beer, that’s not good at a music event. And that’s how our company was initially born. We have been working in music festivals all over the world, in small music festivals, in bigger music festivals, in Argentina and Mexico, in Ecuador, and in Spain mostly. 

A – Are music festivals still your primary target group? 

C – Well, we noticed that this music festival business is not the best business we could pick up because it is very hard. Every year, you have to rearrange the agreement with the music festivals. It is quite hard for them to pay and then we noticed that our product could be well suited as well for hotels and resorts. And then we started to work with some large hotel customers, for example, in Spain we worked with Globalia which is the owner of Air Europa. Or Grupo Piñero, and in Cancun working with hotel chain called Oasis and now we are quite far into the hotel business and it’s working quite well.

A – Okay, that makes sense, hotels are a big market. So, tell me a bit about how you use ObjectBox, what does our solution solve for easyGOband?

C – The thing is, the low connectivity environment for us plays a pretty important role in our product. And that means we have to store a lot of data locally on the devices. For example, when the device makes a transaction, it tries to make the notification through the application server but if it can’t notify it then it just stores the data locally and tries again afterwards. For example, all transactions that are made during the event or hotel will store it locally on every single device so that device – as long as it has any connectivity during the operation even if the connectivity breaks at a single moment – can still see all the data: which transaction has been made, what’s the balance, what room is this wristband related to, what access group has it. We combine the data that we synchronize with the application server with ObjectBox, and the data that we can get real time with the NFC wristbands, we can operate 100% even if we are out of connectivity.

We first started with just SQLite. The thing is, we have to work on different devices. We have to work on Android devices, on Linux devices, we have to work on Windows PC and other devices. Something happened in the Android versions I think with the JDBC driver for SQLite and then we needed two different implementations. One with the native SQLite driver from Android and one with just the JDBC driver. That wasn’t ideal for us, more maintenance. After that we tried H2 but there were some issues with corrupting the DBs and stuff like this. And then I found ObjectBox and we give it a try and it worked quite well. And we are now using ObjectBox on all our devices – Windows PC, Linux PC, we are even using Raspberry Pi. 

We have to work on different devices. We have to work on Android devices, on Linux devices, we have to work on Windows PC and other devices.

A – Very cool. What’s the use case for the Raspberry Pi?

C – We have a system where we integrate with gatekeeper devices, like automatic doors, and we have one single Raspberry Pi for each gatekeeper. You scan the wrist band, the Raspberry Pi makes the connection with the gatekeeper and opens the door, for example. Or in general we use it for access control system for example, camping or resorts where you have access to the gym, it’s an electro magnetic door and we connect the Raspberry Pi to it and with a relay to open the door for it. And the Raspberry Pi is perfect for this. The newer Raspberry Pis run java based applications very well. Even with a user interface, we found it works well. ObjectBox is just perfect for us, since we can use it on all the different devices, one single implementation for all the repositories. For us, it’s perfect.

A – I believe it. So, in terms of implementation, was it fairly easy to do so across the different devices, were there any challenges?

C – It was quite easy. There was some smaller workarounds. For example we had to stick to number IDs, but the IDs on our system are UIDs. Because data is generated on the devices, we have to use UIDs, we cannot just use a non-sequential ID for this. Just some smaller workarounds – I think you are already working on different solutions that would fix our minor issues. Performance is very good. Implementation was done by one week or so, so yeah, it was quite good.

A – What are some big picture goals for your company, in terms of your road map, product road map?

C – Our next goal is a whole new product for hotels. Because, when we started doing business with hotels and we began seeing what our customers need. Now we have learned enough so we can do a single product for our hotel customers. We are going to do a web page and connect to peripherals over websockets. This means, for example, you as an operator in a hotel, as a receptionist, you login to your web panel, and there’s a button that says, let’s say – “Activate RFID Wristband” and we can connect to the device and execute the order that was initiated by the receptionist. The peripherals in Android devices, and in general would all be using ObjectBox to sync and store on the later. 

ObjectBox is just perfect for us, since we can use it on all the different devices, one single implementation for all the repositories.

A – Great that you are able to solve a specific customer pain point. What are you using as a synchronization solution, is that built in house?

C – Yes, yes. On the app server, we use MySQL, I think Aurora Serverless from Amazon and we use JOOQ, a query builder on top of it to build our queries and stuff like this, and then we have an SDK on the client size which uses ObjectBox to store the data on the device.

A – Okay, that’s interesting. Maybe, if you’re familiar or not, we have a synchronization solution for ObjectBox as well.

C – Yes, I’ve been looking into it. Looks good, we will definitely try it out when it’s released. We generate data on different devices and all devices need to sync data that is generated by all the other different devices. 

A – So, did you look at ObjectBox because of performance at all?

C – Not really, we were mostly having issues in terms of compatibility. That was the main reason we switched from SQLite or H2 to ObjectBox. It wasn’t only performance related. For example, with SQLite, the performance we were getting was okay. Because the data was stored on every single device, it’s not that much data volume that you have. For example, even at the largest music festival, maybe the biggest we make 1000 or 2000 transactions in minutes or at most. We don’t generate that much data. It was much more relevant with the different compatibility, on the different devices, and that code-base was usable on the most devices possible. That was very important for us. Obviously performance is also important – but it’s not the most important thing for us. 

A – Sure, so performance wasn’t necessarily a driver there. Anything else you would like to share about using ObjectBox?

CYou solved a lot of issues that we were facing. And the thing is, we are very happy that every time we have an issue, for example, we found an issue that we couldn’t use it on 32-bit windows devices, that was also almost a year ago, it was fixed within just a few weeks and that is very nice. We have never found such a good and quick response from 3rd party and free solution. Later on we had the issue with the Raspberry Pi where we couldn’t use it because of some issue with your continuous integration – also it was solved by you. That was amazing, I don’t know how to thank you. 

A – That’s great to hear. Our community is extremely important to us, it’s a large part of why we’re building ObjectBox. Thank you for sharing your case study, it will be nice to be able to give other users an idea of how ObjectBox can be implememented in so many different applications.

Industrial IoT (IIoT) edge solution for railway operators – a Kapsch ObjectBox Case Study

Industrial IoT (IIoT) edge solution for railway operators – a Kapsch ObjectBox Case Study

Executive Summary of IIoT Edge Case Study

The biggest challenge railway providers face today is digitization to increase operational efficiency. Issues like unscheduled downtimes or track repairs are very costly and have a strong impact on customer satisfaction. On top, railway providers constantly need to work on ensuring and improving passenger security. The problem behind these issues is that railway operators are often still lacking data when it comes to knowing what is happening on the tracks, in the tunnels and trains.

This case study looks at how Kapsch and ObjectBox collaborated on a Kapsch Industrial Internet of Things (IIoT) edge solution for the railway industry. The project enables railway operators to optimize their operational efficiency and asset management via rapid processing of real time mission critical data, ensuring extremely reliable asset operations and timely decision-making.

Kapsch

Founded in 1892, today this family-owned company headquartered in Vienna is a globally-operating technology group with offices and subsidiaries on all continents. The Kapsch Group focuses on peoples’ requirements in the fields of communication and mobility. With innovative products and solutions, Kapsch BusinessCom and Kapsch TrafficCom make a significant contribution to the digital transformation and a sustainable future in public and private transportation.

ObjectBox

ObjectBox makes real time data consistently accessible from sensor to server – including sensors (client), Android and iOS devices, IoT gateways, on-premise and cloud servers. ObjectBox is 10 times faster than any alternative, smaller than 1 MB and uniquely designed for IoT and Mobile settings.

Kapsch and ObjectBox

Kapsch is a longtime partner of railway operators and is thus helping the industry with digitization. By integrating ObjectBox’ database and synchronization solution into the Kapsch railway offering, Kapsch can provide superior speed and data continuity to their railway customers. This means critical data is available when needed and can be interacted with in real-time. This heightens operational efficiency and passenger security, because speed matters in that environment. It also decreases networking costs significantly. Last not least, keeping data locally as much as possible increases data security.

The challenge: Having data up to date – fast, reliably, across devices

As a long-term partner of railway providers across the world, Kapsch addresses their major pain points with its new IIoT solution. Central to optimizing railway providers’ operational efficiency and security is having real-time information about tracks and trains available when needed where needed.

Project requirements at a glance:

    • Performance and operation on all kinds of platforms from sensor to IoT gateway to server to iOS and Android devices
    • Reliable data synchronization between devices
    • Fast (near real-time) and network-connection-independent operation on all devices, meaning on the edge. Therefore, a superfast database across entities is required.
    • Possibility of opening the APIs for external developer projects in the future.

What we developed in the course of our digitalization initiative, was an end to end solution for IIoT, and with this we have an edge computing solution. We were looking for possible partners and one of them was ObjectBox. When we saw their synchronization methodology, we understood it was a perfect fit.

Jochen Nowotny

Vice President of Product Management, R&D, Delivery & Support, Kapsch

The project environment

Developed in a co-creation process with railway operators, the Kapsch IIoT-solution is uniquely tailored to the main needs and challenges of the railway industry: Passenger experience, operational efficiency and safety / security.

The Kapsch railway cross-platform solution applies mission-critical data to avoid costly downtimes and repairs, reducing maintenance times and delays. Thus, timetables can be kept more accurately up-to-date, making travelling a better experience for end users.

At the core of the solution lies the Kapsch mission-critical network. Another essential part of this solution is the storage, processing and delivery of data, so data is accessible where it is needed, when it is needed. Doing this efficiently provides a competitive edge to railway operators. Having the data needed to make decisions in real time – across any part of the railway network – improves operational management and the experience for both employees and customers.

ObjectBox’ solution

This is where ObjectBox comes in. ObjectBox offers a fast data storage and synchronization solution that works seamlessly across devices, from sensors to mobile (iOS and Android) to server to cloud. Because of its out-of-the-box synchronization solution, it also saves time to market and costs. On top, ObjectBox’ easy APIs could easily be extended to external developers. Therefore, Kapsch decided to implement and benchmark the startup’s solution.

 

How did Kapsch find ObjectBox’ solution?

Sourcing new innovative approaches is like finding a needle in the haystack. That is why Kapsch runs the later stage accelerator program Factory1. Factory1 focuses on piloting startup solutions at Kapsch. In a rigorous process, Kapsch’ top management and experts from around the world assess startups and define pilot projects together with them. In the final evaluation step, the startups pitch these pilots to the Kapsch board. After going through this thorough process, ObjectBox convinced the board of their solution’s great potential and usefulness for Kapsch.

The solution: A unique hardware/software stack with out-of-the-box synchronization

While the rigorous sourcing process already ensured a good paper and personal fit, in software projects it always comes down to the nitty-gritty details of the technology stack.

Working together – from technology fit to project-setup

The Kapsch project was developed mainly in Java for the operating systems centOS, Android and iOS. ObjectBox supports all platforms and languages used.

So the first step was to integrate the ObjectBox database for storing data, which meant replacing the persistence layer with ObjectBox. When exchanging a database (the data persistence layer) in an existing project, four cases can be distinguished:

SQL database with abstraction layerNoSQL database with abstraction layer
SQL database without abstraction layerNoSQL database without abstraction layer

In this project, a NoSQL database with an abstraction layer needed to be exchanged.

The second step was to synchronize data between IoT edge gateways and central servers.

Each IoT edge gateway (located along the tracks and in the trains) collects local sensor data and makes it accessible to local devices like smartphones, as well as centralized locations (e.g. on the headquarter’s servers). This setup allows high speed operations independent of the availability and quality of network connections.

This is not a simple case of “sending data one way” or “full replication”, rather a more sophisticated technology referred to as “data synchronization”. It is transactionally safe (aka ACID compliant), meaning no data is lost in transit. The network and gateway can go down any time – once everything is up again, data is safely transmitted. This data synchronization is a built-in feature of ObjectBox and therefore does not require any additional, costly software development efforts from Kapsch.

The third step was to compare ObjectBox to alternatives. Therefore, KPIs were defined and a benchmarking application was set up. The KPIs used in this evaluation were: internal project goals, ease of use, the performance of the database, the speed and efficiency of synchronization, and data consistency across devices.

Pilot project results

Together, ObjectBox and Kapsch achieved their goals and created a solution that adds a competitive edge.

“ObjectBox integration in our IIoT solution has enabled significant performances improvement … far beyond other databases. More, we’ve learn a lot on how to operate efficiently a database from the collaboration with ObjectBox team.”

Farid Bazizi

Head of R&D Development - Mission Critical Communication & Industrial IoT Technologies, Kapsch

Overall goals

More specifically, the following goals were met in the 3 month project:

  • There was a successful integration of the ObjectBox database, demonstrating how fast and easy it can be implemented. Because ObjectBox is flexible and easy to use, it can help ensure that will be usable and maintainable for 10+ years.
  • The performance benchmarks done in the project clearly showed that ObjectBox outperformed the alternative solution, which was based on Couchbase, with regards to speed (CRUD operations), as well as CPU and memory (RAM) usage.
  • Last but not least, ObjectBox Synchronization proved to be easy to implement, transferring data seamlessly and reliably between devices. Benchmarked against the existing solution, measurements showed that ObjectBox synchronized the data 61 to 94 times faster.

Benchmarks

Please note, the benchmarks done here are project-specific. The benchmarks simply compare the internal existing Kapsch implementation, which is based on Couchbase, to the ObjectBox implementation. Thus, they should be read as project-specific results only.

Database Performance (CRUD), higher is better

ObjectBox is faster on all basic database operations within this project. This means you can process more data faster, and run more complex applications, like artificial intelligence or machine learning applications on the device – and respond to incidents faster, before it gets costly or dangerous.

 

CPU & Memory Usage, lower is better

All hardware comes with limited CPU and memory resources. These resources are shared between all the apps running on the device. The less CPU and memory that the data storage solution and sync uses, the more is left for other uses.

This means it is possible to:

  • install smaller (and often cheaper) hardware
  • run more applications and do more complex computing (e.g. edge AI) on the existing hardware, allowing you to capitalize on existing infrastructure.

Both are huge cost factors in the project settings.

Synchronization speed

ObjectBox’ synchronization implementation was much faster than the alternative, processing 37,629 objects in one second, as compared to the 400 synchronized by the alternative. In practice this means that in the given setup, ObjectBox supports 10 times more clients (nodes) for every server. This translates to huge cost savings on hardware as servers are expensive and only one server is now required where before 10 were needed.

Startup-Corporate Collaboration: A win-win situation

As this case study shows, it pays off for projects to evaluate solutions independent from their provider’s company maturity level.

On the journey, both companies learned and benefited from each other’s expertise. Apart from solving the concrete technological challenges, the project yielded further positive side effects. Sebastian Opitz, Head of Controlling and ObjectBox’ project mentor, gives an example: “ObjectBox helped to quickly identify a faster way forward with a new technical solution. It would have taken us much longer doing it on our own and probably only would have been discovered much later.”

From the startup perspective, the collaboration gave ObjectBox access to new departments and new markets. “Kapsch has a deep market knowledge and a lot of experience with digitization projects. This experience helped ObjectBox to reach the next level”, concludes Vivien Dollinger, ObjectBox CEO.