The 1.1 release of ObjectBox for Go is now available, bringing new features such as Box insert() and update() semantics, a new AsyncBox with all write operations (put, insert, update, delete), improved Queries with order and aliases; as well as some fixes and quality of life improvements, such as time.Time support or more forgiving generator code validation. For the full list of changes see the changelog.
To upgrade to the latest version, run go get -u in your project and don’t forget to re-run the generator to make sure all the code is in sync and you get the new features:
1
2
go get-ugithub.com/objectbox/objectbox-go
go generate./…
Async Box
The new AsyncBox gives you asynchronous processing for write operations such as Put, Insert, Update, Remove, RemoveId.
First a quick reminder how a standard (synchronous) Box works:
1
2
3
4
5
6
7
8
9
10
11
box:=model.BoxForTask(ob)
// Create
id,_:=box.Put(&model.Task{
Text:"Buy milk",
})
task,_:=box.Get(id)// Read
task.Text+=" & some bread"
box.Put(task)// Update
box.Remove(task)// Delete
Now, let’s have a look at the new AsyncBox. Let’s say tasks are processed in multiple iterations by calling a “WorkOn(*Task)”. Let’s also assume that WorkOn() sets a “finished” flag on the object if it was able to complete a task in an iteration. In that case, the task can be removed from the database. Otherwise, partial progress on the task should be saved for the next iteration.
1
2
3
4
5
6
7
8
9
10
11
12
13
asyncBox:=box.Async()
vartasks=QueryTasksToWorkOn()
for_,task:=rangetasks{
WorkOn(task)// Consumes some CPU time
iftask.finished{
// Remove a finished task
asyncBox.Remove(task)
}else{
// Save progress on task
asyncBox.Update(task)
}
}
So, what’s the advantage of using AsyncBox in this example? Because we don’t wait for updating or removing a task, we just created an efficient pipeline: we can spend all computational resources on WorkOn(), while AsyncBox performs persistence in the background. Both steps never have to wait for each other.
The second advantage of AsyncBox is “transaction merging.” Because “WorkOn” takes some time, we operate on a single object at a time. A synchronous solution would require a transaction per object, introducing significant disk overhead. AsyncBox can reduce the amount of transactions required and thus dramatically improve throughput.
You may also have noted the usage of “Update()” instead of the standard “Put”. An update is different from a put as it only persists the object if it already exists in the database. Let’s say our example has another process that removes Tasks; a standard put operation might “resurrect” a task previously removed by the other process. If we don’t want that to happen, we can use update semantics. The new update and insert operations are also available in the standard Box API.
Please let us, and everyone else, know what you like about this release and ObjectBox in general. We’d love to hear from you to know what you’d like to see next.
Looking for an easy way to sync data between devices? Check out ObjectBox Sync, sign up for early access, and look out for the release early 2020!
ObjectBox 1.0 is here. Late last year, we introduced ObjectBox to the world of Go – and were awestruck by the GitHub traction and feedback we got. So we doubled down on our efforts and are thrilled to announce 1.0 with many new features, a stabilized API, and improved performance. From now on your projects are safely able to upgrade.
For those of you new to ObjectBox, here is how ObjectBox persists Go structs (objects):
Let’s recap what ObjectBox Go brings to the table:
automatic schema migration (e.g. adding new structs or fields just works)
type-safe APIs, e.g. no interface{} arguments
embedded database – no server needed, all data stored on the device
Go structs as data (also embedded structs); no ORM, no SQL
relations: to-one, to-many (eager and lazy fetching)
powerful queries with indexes for scalable lookups
ACID compliant with implicit (automatic) and explicit (user defined) transactions
superfast bulk/batch operations
low memory usage
runs on 64-bit Linux, macOS, Windows, small 32-bit ARM-based Linux devices (e.g. Raspberry Pi)
How fast is ObjectBox Go? Performance Benchmarks
To see how ObjectBox compares performance-wise, we looked at libraries with comparable levels of storage abstraction – so not just plain SQL/Key-value storage but also ORM features. Finding embedded data storage libraries satisfying those simple criteria wasn’t so easy, but we have picked a few based on GitHub popularity:
GORM using SQLite – GORM supports multiple backends including the embedded SQLite database.
Storm using bbolt – while Bolt DB is not maintained anymore, it’s fork bbolt seems to be picking up traction.
What we tested
To get an overview of the databases, we tested CRUD (create, read, update, delete), indexed and non-indexed queries. Additionally, we have compared a peak memory usage of each database during a test run. Each test was run multiple times, with garbage collection turned off and executed manually outside of the measured time. Data preparation and evaluation was also done outside of the measured time.
We tried to keep the test implementations as close as possible to each other while picking the approaches recommended by the docs for each database. The test code is available at https://github.com/objectbox/objectbox-go-performance.
Test setup
The benchmarks were the only foreground application running on the PC and the given results are an average over 20 runs.
Specs: i7-8850H laptop CPU, Manjaro 18, Linux 4.19.56 Disk speeds (LUKS encrypted SSD): ~ 480 MB/s read, ~120 MB/s write
CRUD
All operations were executed in batches, not one-operation-per-transaction. For GORM and Storm it was necessary to do a batch insert, update and remove inside a manual transaction because they don’t support batch operations out of the box.
The numbers were measured by running benchmarks for 100.000 object-sets for ObjectBox and GORM, while we have chosen to use just 10.000 objects with Storm because it was achieving better operations per second with a smaller set. We will follow up with a post comparing how each database fares with different data sets.
The standard operations you would be using all the time with any database show between 6.8 times speedup (update ObjectBox vs Storm) up to 124 times speedup (Create ObjectBox vs Storm).
Queries
Next we looked at query performance by executing two simple queries. The first test uses an index (actually the “primary-key”) to fetch 100 objects. The second one to find 1111 objects based on a string prefix match (not indexed). The tests were executed 20 times and the average times (ms) are shown below. Note, the arbitrary number, 1111, is just based on how the test code prepares the data (strings contain an object ID, thus 1111 objects out of 10.000 start with the number “1”).
GORM/SQLite performance is very good for a primary-key based query but falls off when it has to use “LIKE” over the whole dataset without an index. Storm doesn’t catch up in either case.
Memory usage
Lastly, we compared the memory usage of each solution. The peak memory usage was measured externally, executing the benchmark binary using /usr/bin/time -v, taking the “Maximum resident set size”. The memory-usage test was run on 10.000 objects with one “test-run” (as opposed to 20 runs in previous benchmarks), repeated manually to make sure there’s no reasonable difference between the runs.
This benchmark shows massive difference in memory usage, which is a good thing to think about not only for small devices but also for cloud-based resources (where RAM is comparatively pricey).
Of course any results you get from performance benchmarks will always be just that, benchmarks. You should consider your application needs and, ideally, benchmark the critical scenarios yourself. If you do, here you can read up a couple of tips on how to do good benchmarks.And because sharing is caring: Do let us know what you find out. Having said that, ObjectBox does show clear lead across the board so why don’t you go ahead and start using it now 🙂
With version 1.0 we hit an important milestone. We’re looking forward to your feedback and we’ll progress with additional features around queries, relations, and async operations. And of course, data synchronization that will allow you to share data among servers and clients. Stay tuned and let us know your thoughts.
Today, we are bringing the power of ObjectBox to Golang. Whatever solution you are building, be it a web service, an IoT/IIoT solution, or any data-driven application, you will benefit from the efficiency and speed of ObjectBox (see benchmarks below). Let us know what you think!
Let’s look at some code, and see how ObjectBox persists Go structs (objects):
The ObjectBox Go API allows you to create data-driven cross-platform apps. ObjectBox supports x64 and 32 bit ARM (ARMv6 and ARMv7) CPUs, enabling you to benefit from a super-fast scalable database on IoT devices, industrial edge gateways or, for example, the Raspberry Pi family (from the minimalistic Pi Zero to the high-spec Pi 3B+). At the same time, you can now target desktop and server apps running on Linux, MacOS, or Windows using Go.
How to get started
To get started, please have a look at the ObjectBox Go docs. Here’s the TL;DR version for your bash:
go install github.com/objectbox/objectbox-go/cmd/objectbox-gogen/
go test github.com/objectbox/objectbox-go/...
This will clone the repository to your GOPATH, download the objectbox-c library and install the ObjectBox code generator.
At this point you can start using ObjectBox in your project – just define a struct you want to persist, add
//go:generate objectbox-gogen comment in the same file and run go generate ./… on your project. See Getting started for more details.
Performance benchmarks
We have done some quick CRUD performance tests to give you an impression about the speed ObjectBox provides. On a i7-8850H CPU, ObjectBox consistently processes over 1.5 million CRUD operations per second with 4.4 million object reads per second:
To put these numbers into perspective, we also did a comparison using an edge computing platform and two leading NoSQL databases. We won’t spill the names and final numbers just yet, as we are going to release all the details very soon. ?
Disclaimer: ObjectBox was the only database running in embedded mode, which is not supported by the others in this setup. Obviously this can have a significant impact on performance. On the other hand, this is the most resource friendly mode of operation (RAM and CPU), which might be very relevant if you target restricted ARM32 devices.
Your feedback
This is the first public version of ObjectBox for Go and we are more than excited to get your feedback on it. We have prepared a short questionnaire for you, which should only take 1-2 minutes. Your feedback is extremely valuable to make ObjectBox a fun tool to use.
Future work
While this initial release covers all basic features, the native core of ObjectBox offers much more than what is currently exposed to Go. For example, a complete set of query conditions and relations between objects. Also, we will introduce a client/server operation mode. This will allow ObjectBox to run in Containers (e.g. Docker) and in classic server setups. As a major theme, the ObjectBox team is also working on data synchronization across devices. This keeps edge devices and gateways “in-sync”. Sync also enables seamless integration with mobile apps (Android and iOS) pushing relevant data to mobile clients and back. Of course sync will also be available for Go. You can sign up here for updates on sync.
