ObjectBox Go 1.0 release and performance benchmarks

ObjectBox Go 1.0 release and performance benchmarks

ObjectBox 1.0 is here. Late last year, we introduced ObjectBox to the world of Go – and were awestruck by the GitHub traction and feedback we got. So we doubled down on our efforts and are thrilled to announce 1.0 with many new features, a stabilized API, and improved performance. From now on your projects are safely able to upgrade.

For those of you new to ObjectBox, here is how ObjectBox persists Go structs (objects):

ObjectBox Go 1.0 – Highlights

Let’s recap what ObjectBox Go brings to the table:

  • automatic schema migration (e.g. adding new structs or fields just works)
  • type-safe APIs, e.g. no interface{} arguments
  • embedded database – no server needed, all data stored on the device
  • Go structs as data (also embedded structs); no ORM, no SQL
  • relations: to-one, to-many (eager and lazy fetching)
  • powerful queries with indexes for scalable lookups
  • ACID compliant with implicit (automatic) and explicit (user defined) transactions
  • superfast bulk/batch operations
  • low memory usage
  • runs on 64-bit Linux, macOS, Windows, small 32-bit ARM-based Linux devices (e.g. Raspberry Pi)

How fast is ObjectBox Go? Performance Benchmarks

To see how ObjectBox compares performance-wise, we looked at libraries with comparable levels of storage abstraction – so not just plain SQL/Key-value storage but also ORM features. Finding embedded data storage libraries satisfying those simple criteria wasn’t so easy, but we have picked a few based on GitHub popularity:

  • GORM using SQLite – GORM supports multiple backends including the embedded SQLite database.
  • Storm using bbolt – while Bolt DB is not maintained anymore, it’s fork bbolt seems to be picking up traction.

What we tested

To get an overview of the databases, we tested CRUD (create, read, update, delete), indexed and non-indexed queries. Additionally, we have compared a peak memory usage of each database during a test run. Each test was run multiple times, with garbage collection turned off and executed manually outside of the measured time. Data preparation and evaluation was also done outside of the measured time.

We tried to keep the test implementations as close as possible to each other while picking the approaches recommended by the docs for each database. The test code is available at https://github.com/objectbox/objectbox-go-performance.

Test setup

The benchmarks were the only foreground application running on the PC and the given results are an average over 20 runs.

Specs: i7-8850H laptop CPU, Manjaro 18, Linux 4.19.56
Disk speeds (LUKS encrypted SSD): ~ 480 MB/s read, ~120 MB/s write 

 

CRUD

All operations were executed in batches, not one-operation-per-transaction. For GORM and Storm it was necessary to do a batch insert, update and remove inside a manual transaction because they don’t support batch operations out of the box.

The numbers were measured by running benchmarks for 100.000 object-sets for ObjectBox and GORM, while we have chosen to use just 10.000 objects with Storm because it was achieving better operations per second with a smaller set. We will follow up with a post comparing how each database fares with different data sets.

The standard operations you would be using all the time with any database show between 6.8 times speedup (update ObjectBox vs Storm) up to 124 times speedup (Create ObjectBox vs Storm). 

Go database benchmarks CRUD

Queries

 Next we looked at query performance by executing two simple queries. The first test uses an index (actually the “primary-key”) to fetch 100 objects. The second one to find 1111 objects based on a string prefix match (not indexed). The tests were executed 20 times and the average times (ms) are shown below. Note, the arbitrary number, 1111, is just based on how the test code prepares the data (strings contain an object ID, thus 1111 objects out of 10.000 start with the number “1”).

GORM/SQLite performance is very good for a primary-key based query but falls off when it has to use “LIKE” over the whole dataset without an index. Storm doesn’t catch up in either case.

Go-database-benchmarks-10kQueries
 

Memory usage

Lastly, we compared the memory usage of each solution. The peak memory usage was measured externally, executing the benchmark binary using /usr/bin/time -v, taking the “Maximum resident set size”. The memory-usage test was run on 10.000 objects with one “test-run” (as opposed to 20 runs in previous benchmarks), repeated manually to make sure there’s no reasonable difference between the runs.

This benchmark shows massive difference in memory usage, which is a good thing to think about not only for small devices but also for cloud-based resources (where RAM is comparatively pricey).  

Go database memory performance benchmarks

Of course any results you get from performance benchmarks will always be just that, benchmarks. You should consider your application needs and, ideally, benchmark the critical scenarios yourself. If you do, here you can read up a couple of tips on how to do good benchmarks. And because sharing is caring: Do let us know what you find out. Having said that, ObjectBox does show clear lead across the board so why don’t you go ahead and start using it now 🙂

 

Getting started

Check the ObjectBox Go docs which come with setup instructions, an introduction and information on many more topics. The ObjectBox Go GitHub repository contains all sources if you are interested in internals. Also, it comes with an example application.

 

Outlook

 With version 1.0 we hit an important milestone. We’re looking forward to your feedback and we’ll progress with additional features around queries, relations, and async operations. And of course, data synchronization that will allow you to share data among servers and clients. Stay tuned and let us know your thoughts. 

Objectively Swifter Database: How Swift + C outperform Objective-C

Objectively Swifter Database: How Swift + C outperform Objective-C

We’ve been optimizing ObjectBox for speed right from the beginning, making it the fastest database for edge computing around. However, the fastest library in the world is useless if its interface to your language isn’t optimized for that language’s strengths and weaknesses. The 0.8.0 release of ObjectBox for Swift gives you a nearly 70% performance boost out of the box by improving the way we integrate with Swift:

ObjectBox Swift

How did we Achieve this Speed-Up?

When we created ObjectBox’s Swift binding, we did not have our C API yet. So, to bridge between the C++ database core and Swift, we implemented a number of classes using Objective-C++. Objective-C++ is neat: it looks to Swift like Objective-C (so like a Swift class), but will happily call into C++ code.

Objective-C is also a very different language from Swift, and particularly generics and structs don’t have a direct equivalent in Objective-C. So when we realized that many reactive frameworks in Swift were built around structs, we decided to change gears. We rewrote a number of core Objective-C classes in the Swift binding as Swift classes on top of the C API that we now use in all our newer language bindings, eliminating a lot of Objective-C’s dynamic dispatch, and generally improving upon algorithms here and there.

The main goal was to give our users struct support and lay the groundwork for taking advantage of Swift 5.1’s upcoming UTF8 strings by eliminating use of the UTF16-based NSString. We also wanted to bring ObjectBox’s Objective-C-beholden error handling in line with Swift conventions. So we expected modest speed-ups here and there, but a speed increase this noticeable even before Swift 5.1 was a pleasant surprise, and we wanted to get these improvements into our users’ hands as quickly as possible.

Using structs with ObjectBox

One rather unique aspect of Swift compared to other languages is how Swift defines the term struct as value type and class as reference type. That makes structs incredibly handy for cases where you want to guarantee an object can never change.

Thread-safety, for instance: if you know an object is unchangeable, there is no chance of race conditions while editing. You need no locks, no copies; your threads can all safely operate on the same memory.

However, when you put an object into your ObjectBox database, put(_:)updates the object’s id property to establish the association between the database entry and your object in memory. ObjectBox can’t do that with unchangeable structs, so we needed to make a slight adjustment to ObjectBox’s usual simple put(_:) flow:

Missed the difference? It’s tiny: You use put(struct:) instead of the regular put(_:). put(struct:) will create a copy of the struct you gave it with the ID changed to whatever ID the object was assigned in the database (the copy is what we store in savedUser in the above example).

So, what if you want to make changes? The way you change immutable structs is to make a copy with the one thing you wanted to change set to a different value. So while you could save it to the database using put(struct:), you already made a copy of the object, and it will not change after being saved, because it already has an ID. Won’t that second copy be wasteful?

That’s why Box now also offers putImmutable(_:). If you know that your object has already been saved, and you don’t need a copy. Just call putImmutable(_:). instead of put(struct:).

This will return the ID for your convenience, but you can always ignore it, if you do not need it.

What else has changed?

While we’re always improving things under the hood, not much should change for your existing ObjectBox code. Apart from the new ObjectBoxError enum replacing the janky old Objective-C-style OBXError... classes, your existing code should just compile. All the changes are in the generated code.

Go give it a try, and let us know how we’re doing.

ObjectBox Python Alpha

ObjectBox Python Alpha

As of today, you’ll be able to start using ObjectBox from the comfort of Python.

Whether you’re building an IoT solution, a desktop utility or a webservice, you’ll find the ObjectBox database handy to help you with data persistence on the edge. Among other features, ObjectBox provides implicit ACID transactions so there’s one less thing you have to worry about.

Let’s take a look at an example to see how ObjectBox lets you work with your data using Python:

Aside from x64 CPUs, ObjectBox supports ARMv6 and ARMv7, allowing you to benefit from a super-fast and scalable database on IoT devices, for example, the Raspberry Pi family (from the minimalistic Pi Zero to the high-spec Pi 3B+). You can also target desktop and server apps running on Linux, MacOS, or Windows using Python.

ObjectBox Python: How to get started

While this is still an early release in Python, it’s based on the proven ObjectBox core so you can already start building your applications today. Just install using

and head over to our GitHub repo at http://github.com/objectbox/objectbox-python to see some examples.

Your feedback helps us to improve ObjectBox, please share your thoughts and comments 🙂

ObjectBox EdgeX Release (Beta)

ObjectBox EdgeX Release (Beta)

ObjectBox now runs on the EdgeX Foundry IoT Edge Platform. Utilizing ObjectBox’ speed and ease of use, EdgeX users can now compute millions of data points on the edge with minimal latency. ObjectBox’ small footprint of less than 1MB makes the database uniquely optimized for high performance on IoT edge devices and gateways, as well as fog nodes. Combining the speed and size advantages of ObjectBox on the EdgeX platform, we empower companies to analyze more data locally on the machine, faster than ever before.

As an object oriented database, ObjectBox is the perfect solution for structuring data within complex data models, from artificial intelligence and machine learning applications, as well as image recognition software that is beginning to be used more commonly on the edge.

With ObjectBox-backed EdgeX we’re bringing the efficiency, performance and small footprint of ObjectBox to all EdgeX applications. It is fully compatible, so you can use it as a drop-in replacement. And if you call against existing REST or Go EdgeX APIs, you do not need to change the code. Our version is based on the latest sources leading to  EdgeX 1.0, which is scheduled for June. And once EdgeX 1.0 is finalized, we will be ready to upgrade the ObjectBox Edition to 1.0 right away.

Get started with ObjectBox EdgeX

The simplest way to get started is to fetch the latest docker-compose.yml and start the containers:

You can check the status of your running services by going to http://localhost:8500/. At this point, you have the REST services running at their respective ports, available to access from your EdgeX applications.

Find more details and sources in our GitHub repo at  https://github.com/objectbox/edgex-objectbox.

If you’re new to EdgeX, you can visit the EdgeX Foundry IoT Edge Platform home page to find out more.

Feedback

Last not least, we are always happy to hear from you. Post any questions you may have on stack overflow tagged ObjectBox. Please share your thoughts on ObjectBox EdgeX with us via TwitterFacebook, or Mail (contact [at] objectbox . io).

The ObjectBox Swift Beta is Here!

The ObjectBox Swift Beta is Here!

If you’ve ever needed a database for your iOS app, you’ve probably had to manage schemas, tables, query strings and all sorts of overhead. Moreover, whenever you wanted to modify the structure of your database, you had to write migration code so that your users’ data would be upgraded to fit the new structure.

Wouldn’t it be nice if your database just did all that for you, automatically? That was what we thought when we designed ObjectBox. ObjectBox thinks the way a Swift developer does: You take your objects and stick them in a box. You use regular Swift methods and operators to search for objects in your database. You add or remove fields and the database just copes with it. And you can use it right now.

(more…)

ObjectBox Swift for iOS and macOS

ObjectBox Swift for iOS and macOS

We’re happy to share our first Swift version of the ObjectBox database for iOS and macOS! We want to give you a Swift “native” API without any ObjectiveC legacy shining through, which is why we decided to put Swift, with its unique language features, first. We really want your feedback on this to improve swiftly, for example on our query API (more details below):

(more…)