fbpx
ObjectBox Go 1.1

ObjectBox Go 1.1

The 1.1 release of ObjectBox for Go is now available, bringing new features such as Box insert() and update() semantics, a new AsyncBox with all write operations (put, insert, update, delete), improved Queries with order and aliases; as well as some fixes and quality of life improvements, such as time. Time support or more forgiving generator code validation. For the full list of changes see the changelog.

To upgrade to the latest version, run go get -u in your project and don’t forget to re-run the generator to make sure all the code is in sync and you get the new features:

Async Box

The new AsyncBox gives you asynchronous processing for write operations such as Put, Insert, Update, Remove, RemoveId.

First a quick reminder how a standard (synchronous) Box works:

Now, let’s have a look at the new AsyncBox. Let’s say tasks are processed in multiple iterations by calling a “WorkOn(*Task)”. Let’s also assume that WorkOn() sets a “finished” flag on the object if it was able to complete a task in an iteration. In that case, the task can be removed from the database. Otherwise, partial progress on the task should be saved for the next iteration.

So, what’s the advantage of using AsyncBox in this example? Because we don’t wait for updating or removing a task, we just created an efficient pipeline: we can spend all computational resources on WorkOn(), while AsyncBox performs persistence in the background. Both steps never have to wait for each other.

The second advantage of AsyncBox is “transaction merging.” Because “WorkOn” takes some time, we operate on a single object at a time. A synchronous solution would require a transaction per object, introducing significant disk overhead. AsyncBox can reduce the amount of transactions required and thus dramatically improve throughput.

You may also have noted the usage of “Update()” instead of the standard “Put”. An update is different from a put as it only persists the object if it already exists in the database. Let’s say our example has another process that removes Tasks; a standard put operation might “resurrect” a task previously removed by the other process. If we don’t want that to happen, we can use update semantics. The new update and insert operations are also available in the standard Box API.

Please let us, and everyone else, know what you like about this release and ObjectBox in general. We’d love to hear from you to know what you’d like to see next.


Looking for an easy way to sync data between devices? Check out ObjectBox Sync, sign up for early access, and look out for the release early 2020!

ObjectBox EdgeX v1.1 – database with ARM32 support

ObjectBox EdgeX v1.1 – database with ARM32 support

With EdgeX Foundry just reaching v1.1, we continue to provide ObjectBox as an embedded high-performance database backend so you can start using ObjectBox EdgeX v1.1 right away. If you need data reliability and high-speed database operations, ObjectBox is for you. Additionally, starting with ObjectBox EdgeX 1.1, you can use it on 32-bit ARM devices.

Combining the speed and size advantages of ObjectBox on the EdgeX platform, we empower companies to analyze more data locally on the machine, enabling new use cases.

With ObjectBox-backed EdgeX we’re bringing the efficiency, performance and small footprint of the ObjectBox database to all EdgeX applications. It is fully compatible, so you can use it as a drop-in replacement: you call against the same REST and Go EdgeX APIs. As simple as that;no need to change any code.

Performance comparison of EdgeX database backends

EdgeX Foundry comes with a choice of two database engines: MongoDB and Redis. ObjectBox EdgeX brings an alternative to Redis and MongoDB to the table.  Because ObjectBox is an embedded database, optimized for high speed and ease of use while also delivering data reliability, it enables a new set of use cases. As we all know, benchmarks are hard to do. This is why all our benchmarks are open source and we invite you to check them out for yourself. To give you a quick impression of how you could benefit from using ObjectBox, let’s have a look at how each compares in basic database operations on “Device Readings”, one of the most performance intensive data points.

Note: The Read and Write operations (all CRUD (Create, Read, Update, Delete) operations are measured in objects / second). The benchmarks test internal EdgeX database layer performance, not the REST APIs throughput.

These benchmarks provide a good perspective why you should consider ObjectBox with EdgeX. Benchmark sources are available publicly in ObjectBox EdgeX github repo.

So, why is ObjectBox EdgeX faster?

First of all, you are probably aware of the phrase “Lies, damned lies, and statistics benchmarks”. Of course, you should look at performance for yourself and consider based on your specific use case needs. That’s why we make our benchmarks available as open source. It is a good starting point.

To make it easier to compare ObjectBox (in addition to our open source benchmarks) here are some of the high-level “unfair advantages” that make ObjectBox fast:

  • Objects: As you can derive from its name, ObjectBox is all about for objects. It’s highly optimized for persisting objects. The EdgeX architecture and Go sources are a great fit here as it puts Go’s objects (structs) in the center of its interface. This means, we can omit costly transformations and this helps with speed.
  • Embedded database: Redis and MongoDB are client/server databases running in separate processes. ObjectBox, however, is running in the same process as EdgeX itself (each EdgeX microservice, to be precise). This has definite efficiency advantages, but it also comes with some restrictions: Whereas you can put Redis/MongoDB in separate Dockers or machines, this option is not available for ObjectBox yet.
  • Transaction merging: ObjectBox can execute individual write operations in a common database transaction. This means, we can reduce the costly transactions for a number of write operations. This is a great way to add on performance, delaying the transaction end by single digit milliseconds.

Get started with ObjectBox EdgeX

The simplest way to get started is to fetch the latest docker-compose.yml and start the containers:

You can check the status of your running services by going to http://localhost:8500/. At this point, you have the REST services running at their respective ports, available to access from your EdgeX applications.

Find more details, instructions for ARM32, and sources in our GitHub repo at  https://github.com/objectbox/edgex-objectbox.

If you’re new to EdgeX, find out all about the open source  IoT Edge Platform here. The EdgeX project is led by the Linux Foundation and supported by many industry players, including Dell, IBM, and Fujitsu.

We love to hear from you 🙏

We’re very interested to hear about the challenges you are facing on the edge and in IoT. As performance experts, we are always embracing a tough challenge. Reach out to us to set up a pilot project.

Is there something you are missing? Please do reach out to us. We want to make ObjectBox the best edge data persistence layer available. We love to receive your feedback.

What next?

Find out more about ObjectBox EdgeX and get started, go directly to GitHub or download the snap on Snapcraft.

ObjectBox Java 2.4

ObjectBox Java 2.4

Update: newer versions were released; check the changelog for details. 

The 2.4.0 update of ObjectBox for Java (and Kotlin) is here. We encourage everyone to update to this release, as it includes quite a few quality of life improvements and resolves many of the issues that you have reported, so thank you for that!

This is also the first release where the ObjectBox LiveData and Paging integration has migrated from Android Support Libraries, to Jetpack (AndroidX) Libraries. If you are using those features, check the upgrade notes for possible changes that you need to make to your app.

Also note that this version makes some changes to the generated MyObjectBox and JSON model file. Make sure to commit changes to the model file after building your app. Also, if you are using a library that ships with a pre-generated MyObjectBox file, that library needs to be updated to 2.4.0 as well.

Besides those improvements, we were also fine-tuning performance a bit. While fixing a performance regression for 32 bit CPUs related to ordered queries, we were able to do additional optimizations. Now ordered queries using a limit run up to three times faster than before.

For a list of all the changes, please check the changelog.

Last not least, let us share some related ObjectBox’ developments in the mobile space. Today, we also released version 0.3 for ObjectBox Dart. So, if you are interested in creating Flutter apps, you will be able to use ObjectBox soon. Last month, we released ObjectBox Swift 1.0. Therefore, you can build native apps with ObjectBox for the two prominent mobile platforms, Android and iOS. Additionally, we’re also making great progress with data synchronization; sign up for sync updates to be notified sync related news and to be part of the upcoming early releases.

ObjectBox Swift 1.0 Released

ObjectBox Swift 1.0 Released

Update: newer versions have been released. Check the changelog.

ObjectBox Swift 1.0 is here! Since the first public alpha released 10 months ago, we’ve worked hard and made major changes to put Swift first, tune the performance, and iterate on the API. We hope you will love the result.

All of this, to bring you the features you expect from a database, but more importantly – the features that we think delight developers and sets ObjectBox apart from other databases out there. Let’s swiftly (cheap pun intended) dive into ObjectBox Swift 1.0:

Built with Swift in Mind

ObjectBox isn’t just a database bolted onto Swift. Your database entities are regular Swift classes or structs that you devise. No need to subclass a particular class (as with CoreData’s NSManagedObject), nor to write tedious serialization code. 😊

All you need to do is add one property for the unique ID, build your project, and ObjectBox’s code generator will write a little bit of code for you, just like the Swift compiler does for Codable objects. All that’s left then, is to call a simple method like put() on the object to write it out:

We’ve tried to keep this simplicity throughout the Swift binding, e.g. making it very easy to use any RawRepresentable enum without writing any conversion code.

Automatic Schema Migrations

A common chore with databases is schema migration. ObjectBox takes care of that. If you add a new property or class there are no additional migration steps required. Old objects will keep working, and new objects will be saved with the additional fields. Similarly, adding new classes will add them to the database without any error-prone migration steps.

Moreover, you do not need to maintain a dedicated schema, because your classes and structs are the schema in the first place.

Relations

To ensure ObjectBox knows how to save object references, you use a wrapper class. Either ToOne or ToMany, instead of a straight reference or an array. This lets ObjectBox lazily load the related objects from the database, only when you’re actually accessing a related object.

The Swift 1.0 release brings you our complete set of relations: One-to-many, many-to-many, and their corresponding back-links. ToMany behaves just like any other Swift collection, you can add or remove objects as you please with your familiar methods like append().

Queries

Of course ObjectBox lets you perform queries to collect data; either complete objects or individual properties (basic Swift data types).

But with ObjectBox you don’t mess around with query strings or unpack data from cursors. You simply write Swift expressions with function calls and operators you’re already used to.

Also, you get to keep the type-safety guarantees and compile-time checking. So you don’t have to spend hours figuring out why your query doesn’t return the proper results, just to discover you made a typo in a field name in a query string.

ObjectBox lets you then operate on these objects, watch a query for changes, retrieve the results, delete the objects matching a query etc. The source code even contains a file that adds Combine support so you can integrate with its pipelines to take advantage of Apple’s newest technology.

Open Source Swift Binding

If you’re curious how things work behind the scenes, feel free to check out the Swift source code. The source code for the Swift binding, as well as our code generator based on Sourcery, are available among other projects through our Github account.

How-to Get Started

It’s a matter of minutes to get started with ObjectBox. Check our setup instructions (based on CocoaPods) and jump right into code with the getting started guide.

Your Feedback. And what’s Next?

As always, we would love to hear your feedback! Do you like ObjectBox as much as we do? We put our hearts in this product and are excited to learn your thoughts: What features are you most excited about, what are we missing?

We haven’t written much about a topic very dear to us: performance. We will cover this in a follow up post. Also, look forward to our ObjectBox Swift 1.0 benchmarks, which we will release soon including the sources.

——

Looking for a fast and simple data synchronization solution?


ObjectBox Swift Binding Open Sourced

ObjectBox Swift Binding Open Sourced

Today’s 0.9 release is not just a step closer to 1.0, but also an important milestone for us: for the first time, all Swift binding source code is available on GitHub. We’ve also improved performance (again) and added useful features like type converters.

ObjectBox’ lightning-fast database offers a cross-platform C API for language-specific bindings. These bindings provide a thin and nimble wrapper around the C calls and expose ObjectBox functionality in the way that feels most natural to the host language (i.e. Swift).

An open-source binding allows you to make changes to ObjectBox and build it on your own, backport it to older OS or Swift versions, and contribute your changes back to us if you feel so inclined.

Is this the binding I’ve been using all this time?

Although it’s one of our newer language bindings, our Swift binding already has a storied and colorful history. The first Swift binding actually predates our C library and was built directly against the core C++ codebase. As such, it used Objective-C to bridge Swift and C++.

Once we had the C library, we set about eliminating the intermediary and started rewriting the central Objective-C portions as pure Swift on top of the C API. The immediate result when we released ObjectBox for Swift 0.8, was a significant speed-up in write operations due to Swift’s new native UTF-8 strings, and struct support.

But for the open source release, we completed this rewrite, and ObjectBox Swift got even faster:

Not having to bridge to Objective-C meant we could take advantage of Swift’s associated types to resolve class/entity associations instead of a runtime look-up, and eliminate a few duplicated classes necessitated by adding Swift features to Objective-C classes. This in turn allowed removing some unnecessary data copying, which translated directly to increased speed.

While we enjoy a good performance increase as much as the next database developer, we also added new features and fixes, like more query comparison operators for scalar types, and improved behavior when adding properties to an entity.

Property Converters

One major new feature of this release is support for converting simple user-defined types into types the database understands. For example, you can mark any RawRepresentable enum with an annotation to tell ObjectBox how to serialize it. Let’s say we have an enum like

In your class, you simply add the following annotation:

But of course we also let you work with your own types, or system types. Let’s say you wanted to save a window’s dimensions into your database. You do this using a converter class that implements convert() methods that convert your custom type into one of ObjectBox’s built-in types, and back:

And then you tell ObjectBox to use that converter using an annotation:

That’s it.

And that’s just a selection of the changes you’ll find in the new open-source release. It contains the entire binding, but also the fork of Sourcery that does its duties as our code generator, and a performance test app. It is faster, more feature-full, and we’ve also squashed some bugs in the binding.

To take a look and get started, simply go to our ObjectBox Swift Git repository, where, in addition to our example application, you can now also find the binding’s source code. Or pull version 0.9 via CocoaPods.

We can’t wait for you to try it out and let us know what you think.

ObjectBox Go 1.0 release and performance benchmarks

ObjectBox Go 1.0 release and performance benchmarks

ObjectBox 1.0 is here. Late last year, we introduced ObjectBox to the world of Go – and were awestruck by the GitHub traction and feedback we got. So we doubled down on our efforts and are thrilled to announce 1.0 with many new features, a stabilized API, and improved performance. From now on your projects are safely able to upgrade.

For those of you new to ObjectBox, here is how ObjectBox persists Go structs (objects):

ObjectBox Go 1.0 – Highlights

Let’s recap what ObjectBox Go brings to the table:

  • automatic schema migration (e.g. adding new structs or fields just works)
  • type-safe APIs, e.g. no interface{} arguments
  • embedded database – no server needed, all data stored on the device
  • Go structs as data (also embedded structs); no ORM, no SQL
  • relations: to-one, to-many (eager and lazy fetching)
  • powerful queries with indexes for scalable lookups
  • ACID compliant with implicit (automatic) and explicit (user defined) transactions
  • superfast bulk/batch operations
  • low memory usage
  • runs on 64-bit Linux, macOS, Windows, small 32-bit ARM-based Linux devices (e.g. Raspberry Pi)

How fast is ObjectBox Go? Performance Benchmarks

To see how ObjectBox compares performance-wise, we looked at libraries with comparable levels of storage abstraction – so not just plain SQL/Key-value storage but also ORM features. Finding embedded data storage libraries satisfying those simple criteria wasn’t so easy, but we have picked a few based on GitHub popularity:

  • GORM using SQLite – GORM supports multiple backends including the embedded SQLite database.
  • Storm using bbolt – while Bolt DB is not maintained anymore, it’s fork bbolt seems to be picking up traction.

What we tested

To get an overview of the databases, we tested CRUD (create, read, update, delete), indexed and non-indexed queries. Additionally, we have compared a peak memory usage of each database during a test run. Each test was run multiple times, with garbage collection turned off and executed manually outside of the measured time. Data preparation and evaluation was also done outside of the measured time.

We tried to keep the test implementations as close as possible to each other while picking the approaches recommended by the docs for each database. The test code is available at https://github.com/objectbox/objectbox-go-performance.

Test setup

The benchmarks were the only foreground application running on the PC and the given results are an average over 20 runs.

Specs: i7-8850H laptop CPU, Manjaro 18, Linux 4.19.56
Disk speeds (LUKS encrypted SSD): ~ 480 MB/s read, ~120 MB/s write 

 

CRUD

All operations were executed in batches, not one-operation-per-transaction. For GORM and Storm it was necessary to do a batch insert, update and remove inside a manual transaction because they don’t support batch operations out of the box.

The numbers were measured by running benchmarks for 100.000 object-sets for ObjectBox and GORM, while we have chosen to use just 10.000 objects with Storm because it was achieving better operations per second with a smaller set. We will follow up with a post comparing how each database fares with different data sets.

The standard operations you would be using all the time with any database show between 6.8 times speedup (update ObjectBox vs Storm) up to 124 times speedup (Create ObjectBox vs Storm). 

Queries

Next we looked at query performance by executing two simple queries. The first test uses an index (actually the “primary-key”) to fetch 100 objects. The second one to find 1111 objects based on a string prefix match (not indexed). The tests were executed 20 times and the average times (ms) are shown below. Note, the arbitrary number, 1111, is just based on how the test code prepares the data (strings contain an object ID, thus 1111 objects out of 10.000 start with the number “1”).

GORM/SQLite performance is very good for a primary-key based query but falls off when it has to use “LIKE” over the whole dataset without an index. Storm doesn’t catch up in either case.

Memory usage

Lastly, we compared the memory usage of each solution. The peak memory usage was measured externally, executing the benchmark binary using /usr/bin/time -v, taking the “Maximum resident set size”. The memory-usage test was run on 10.000 objects with one “test-run” (as opposed to 20 runs in previous benchmarks), repeated manually to make sure there’s no reasonable difference between the runs.

This benchmark shows massive difference in memory usage, which is a good thing to think about not only for small devices but also for cloud-based resources (where RAM is comparatively pricey).

Of course any results you get from performance benchmarks will always be just that, benchmarks. You should consider your application needs and, ideally, benchmark the critical scenarios yourself. If you do, here you can read up a couple of tips on how to do good benchmarks. And because sharing is caring: Do let us know what you find out. Having said that, ObjectBox does show clear lead across the board so why don’t you go ahead and start using it now 🙂

Getting started

Check the ObjectBox Go docs which come with setup instructions, an introduction and information on many more topics. The ObjectBox Go GitHub repository contains all sources if you are interested in internals. Also, it comes with an example application.

Outlook

With version 1.0 we hit an important milestone. We’re looking forward to your feedback and we’ll progress with additional features around queries, relations, and async operations. And of course, data synchronization that will allow you to share data among servers and clients. Stay tuned and let us know your thoughts.