Go, ObjectBox Golang!

Go, ObjectBox Golang!

Today, we are bringing the power of ObjectBox to Go. Whatever solution you are building, be it a web service, an IoT/IIoT solution, or any data-driven application, you will benefit from the efficiency and speed of ObjectBox (see benchmarks below). Let us know what you think!

Let’s look at some code, and see how ObjectBox persists Go structs (objects):

The ObjectBox Go API allows you to create data-driven cross-platform apps. ObjectBox supports x64 and 32 bit ARM (ARMv6 and ARMv7) CPUs, enabling you to benefit from a super-fast scalable database on IoT devices, industrial edge gateways or, for example, the Raspberry Pi family (from the minimalistic Pi Zero to the high-spec Pi 3B+). At the same time, you can now target desktop and server apps running on Linux, MacOS, or Windows using Go.

How to get started

To get started, please have a look at the ObjectBox Go docs. Here’s the TL;DR version for your bash:

This will clone the repository to your GOPATH, download the objectbox-c library and install the ObjectBox code generator.

At this point you can start using ObjectBox in your project – just define a struct you want to persist, add  //go:generate objectbox-gogen  comment in the same file and run go generate ./… on your project. See Getting started for more details.

Performance benchmarks

We have done some quick CRUD performance tests to give you an impression about the speed ObjectBox provides. On a i7-8850H CPU, ObjectBox consistently processes over 1.5 million CRUD operations per second with 4.4 million object reads per second:

The sources for the performance tests are part of the GitHub repository.

To put these numbers into perspective, we also did a comparison using an edge computing platform and two leading NoSQL databases. We won’t spill the names and final numbers just yet, as we are going to release all the details very soon. 😉

Disclaimer: ObjectBox was the only database running in embedded mode, which is not supported by the others in this setup. Obviously this can have a significant impact on performance. On the other hand, this is the most resource friendly mode of operation (RAM and CPU), which might be very relevant if you target restricted ARM32 devices.

Your feedback

This is the first public version of ObjectBox for Go and we are more than excited to get your feedback on it. We have prepared a short questionnaire for you, which should only take 1-2 minutes. Your feedback is extremely valuable to make ObjectBox a fun tool to use.

Future work

While this initial release covers all basic features, the native core of ObjectBox offers much more than what is currently exposed to Go. For example, a complete set of query conditions and relations between objects. Also, we will introduce a client/server operation mode. This will allow ObjectBox to run in Containers (e.g. Docker) and in classic server setups. As a major theme, the ObjectBox team is also working on data synchronization across devices. This keeps edge devices and gateways “in-sync”. Sync also enables seamless integration with mobile apps (Android and iOS) pushing relevant data to mobile clients and back.  Of course sync will also be available for Go. You can sign up here for updates on sync.

 

EdgeX Interview: Why open source is key for IoT and Edge Computing

EdgeX Interview: Why open source is key for IoT and Edge Computing

We were thrilled to speak with Michael Hall, Brett Preston (the Linux foundation) and Jim White (Dell) about their open source platform EdgeX.

Jim White

Jim White

Dell, EdgeX

Jim is Vice Chair of the Technical Steering Committee of EdgeX. He is also Team Lead of the IoT Platform Development and IoT Solutions Division at Dell.

Michael Hall

Michael Hall

The Linux Foundation for EdgeX Foundry

Michael is Developer Advocate and Community Manager at the Linux Foundation.

 

Brett Preston

Brett Preston

The Linux Foundation for EdgeX Foundry

Brett is Developer Advocate and Community Manager at The Linux Foundation for EdgeX Foundry.

 

Vivien: What is EdgeX and where are you at with it?

Michael: EdgeX Foundry is a vendor neutral open source platform containing a collection of micro services that take care of different aspects of what you’re going to need to have an edge computing platform. If you’re making IoT devices, you don’t want to reinvent that layer of the stack. Having that common platform for IoT is something that is going to benefit everybody. The Linux Foundation is a neutral umbrella over EdgeX. Inside the project are all the member companies who are actually funding the development, putting developers and marketing resources in, to make it an actual, usable product for everybody. That’s the model of the project and the actual code itself. The main goal of EdgeX is processing and transporting data between IoT devices and sensors and things in the cloud and on the backend. The focus is on being able to respond locally as much as you can, so that you don’t have the latency of going on the cloud and back. And also, being able to continue working, if you loose that connection.

Jim: EdgeX is a an open source platform containing a collection of micro services that take care of different aspects of what you’re gonna have to do to have an edge computing platform. Any of those are semi-dependent: You can replace anything you need to replace. For the status of the project: We just had a year of fast pace growth and we have rewritten everything in Go, so all of our processes are a lot smaller and more efficient now.

Vivien: How are you currently tackling local on-device data persistence?

Jim: We currently use Mongo DB as the persistence engine although we could support almost any kind of persistence store at the edge as long as it was small enough. We also have used SQLite in the past for a couple of customers. However, Mongo DB is the largest element in our portfolio of services. There are a couple of reasons why we are probably going to offer an alternative to Mongo with our next big release in spring 2019: Footprint, licensing, and lack of support for ARM32.

Michael: As we are a collection of microservices, you can always swap out individual pieces depending on what your needs are.

Vivien: Is EdgeX and its components restricted to certain licenses?

EdgeX builds on Apache 2

Jim: EdgeX is an Apache 2 license open source project, so we prefer Apache 2 level or at least a compatible license, because we want to be very business friendly. We want people to take the application and use it in all sort of settings, including actually embeddeding it in gateways. We also want to be very decoupled at discrete points.

For example, if I’m a company like Dell and I use EdgeX. If some of my customers have an absolute demand that a certain database be at the heart, then I want to be able to choose the database, depending upon the customers, the use cases, and the environment that they find themselves in. EdgeX is all about the flexibility. So, for this example, we offer what we call a reference implementation database. Customers or users could take EdgeX and replace elements with their own technology, which may not be open source even.

Michael: You can take what’s open source and add proprietary file systems or hardware depending on what your specific needs are. EdgeX tries to be that common open source base. It provides all of the functionality in a open source license but that still lets you replace bits as needed with whatever it is that you want to run.

Vivien: Can you give us an example how and where EdgeX is currently used? 

Jim: There are over 70 companies now that are part of the EdgeX community and each group is using it differently. There are some that are serving EdgeX as the Red Hat model: they are providing distribution, services and support behind EdgeX. A company like mine, Dell, we’re trying to find a platform that actually goes on our gateway. So we’re going to build a commercial version of EdgeX for our own platform. There will be pieces that we will replace based on better performing mechanism to some of our cloud based products. Then you have other groups out there that are proving particular services for EdgeX, for example edge analytics. There are lots of different service capabilities where we see potential replacements. Then there are companies like Samsung that uses EdgeX in their factory floor to help run their automation. So, they are users, but they also want to make sure  EdgeX meets their needs. Our community is made of snowflakes, they are all very special *laughs* – common goals but different use cases for almost everybody that is part of the organization.

 

Vivien: That sounds really cool. In your opinion, moving the data to the edge, what is the edge, where do you see the data ending up, for example more on the sensor level or gateway level?

Latency concerns, cost of shipping up the data, and the ability to actuate locally are key reasons why you have to have edge software.

Jim White

Vice Chair of the Technical Steering Committee, EdgeX

Jim: We absolutely believe EdgeX is a mechanism for the edge. While you could run pieces of EdgeX on the cloud, we do not believe is what the future holds. There are gonna be certain use cases where that works, but latency concerns, the cost of shipping up the data, and the ability to actuate locally are all key ingredients and reasons why you have to have edge software and edge platforms. Now, these are gonna get smaller. At Dell, we are manufacturing gateways of different sizes, because we know that certain use cases are gonna dictate a larger box and other are going to dictate something like a Raspberry Pi or even smaller. We have companies in our foundry that are looking at running parts of EdgeX in things like PLCs, to help address their realtime needs. So, we absolutely believe that the edge is very much going to be a part of our IoT environments. There are going to be use cases that dictate different levels of compute all the way up from sensors to cloud.

 

Michael: And all of our member companies see a need for that platform, but that platform is not going to be their product or their service. So everybody wants it to exist, so everybody is gonna work together to make it exist, so that they can build their own value-add on top of that or below their device level. Everyone agrees that this is an important thing, that we have to have a solution there for all the innovation that people see on the horizon.

Markus: So, small device level versus gateway – would you say your current focus is on the gateway?

Levels of Edge Computing

Jim: I would say that it really isn’t that one or the other is more important. You’re gonna have situations, as we know from a Dell perspective, where what we call a brownfield device (e.g. a 1979 modbus engine) needs a gateway, because it doesn’t have the ability to communicate into any kind network otherwise. So there has to be a gateway that provides that first level of compute. There are other things that are evolving in the industry: think of say windmill generators, where there is lots of capability right there at the device level, there is a lot of compute built right into those systems. So things will run at that level, and then you have everything in between. Even something like BLE or Zigbee type of environments where there is wifi and ability to connect directly to a network. Typically, we’re finding organizations are reluctant to allow those kind of things to connect into their major networks without some security, apparatus and analytics to see what’s going on, so as not to create problems in their larger networks. So even there, a gateway may be necessary, not because of hardwiring or physical connections, but because you want some insurances in place at the edge before that data leaks on up to your enterprise.

It’s the worst way to build our product except for all others.

Jim White

Vice Chair of the Technical Steering Committee, EdgeX

Vivien: What’s the worse about open source that you’ve experienced?

Jim: *laughs* Now you are going to make me say some things in front of Brett and Michael as members of the Linux foundation… There is a quote by Winston Churchill that talks about democracy, saying it’s the worst form of government except all others. I kind of feel the same way about open source development. It’s the worst way to build our product except for all others. Because it does take time. It’s a community effort and anything done by a community automatically seeks a ground where it’s going to be the best and brightest product. So you get the best input from everybody, but it takes time. It’s easier for say something like Dell to go marching off and build a software solution that they think is the best. It will get there faster but it’s not necessarily going to get there in a way that the world and communities accept more easily. So anything built by many hands is going to take a little bit more time and a little bit more process. But it ends up getting a lot better results I think in the end.

Michael: Whenever you have a community building something you can’t just come in and say “This is what you’re gonna build” because they don’t have to do what you say. And that’s true even with EdgeX. Everybody who is working on it is working for a company invested in it, but there is no one person who can say this is what you’re all going to do. So it’s not enough to say just what you want done. You have to explain and justify why and get people to buy into that. And that takes more effort, but you have to know that what you’re proposing is the right solution, that it’s going to work. If you can’t explain that, you can’t communicate that to the community then it’s not going to get done. As Jim said, it takes time but at the end product is going to be better.

Jim: In this case with IoT, I will tell you that no one company will be able to provide it all. As Dell, we would love to be the company providing it all… (laughter) We have learnt the hard way that in an IoT landscape there are going to be certain things in the company that you can’t touch and IoT has to touch everything. Maybe it’s the network, hardware or operating systems, particular sensors and protocols. You can help to persuade customers to do some things in your way, but you’re never going to be able to get them to do everything in your way and that’s why IoT takes an ecosystem. Which is why we think the second part of EdgeX is so important; our product is important, but just as vital is the ecosystems. We have a collection of companies all trying to work together to provide for interoperability. That is just as important as the actual end product we develop.

No one company will be able to provide it all.

Jim White

Vice Chair of the Technical Steering Committee, EdgeX

Vendor lock-in is not going to work in IoT.

Michael Hall

Developer Advocate, Linux Foundation

Michael: Vendor lock-in is not going to work in IoT. There is no way any company is gonna be able to provide all the needs of somebody. So having an equal playing field for everybody, having that common ground that anybody can come to and interact with anybody else, is what’s going to allow us to fulfill the promise of IoT in general.

P.S. You can meet Brett, Michael, and Jim at the Open IoT Summit Europe in Edinburgh in October.

Please help us learn more about the data persistence landscape on the edge by filling out our small questionnaire (literally 2 minutes). Thanks!

Top 5 reasons why edge computing is crucial for IoT

Top 5 reasons why edge computing is crucial for IoT

IoT is growing at a very rapid rate and with it the vast amount of data it produces. Handling these amounts of data is an unresolved challenge. Edge Computing could be part of the solution. 
According to Dave Evans of Cisco, in 2010 the number of IoT devices connected to the internet passed the world population, with a device to person ratio of 1.84.1 By 2020 there will be up to ten Web-connected devices per person, collectively producing over 40 zettabytes of data.2, 4 

The graph on the right shows the estimated number of IoT devices from 2015-2025, Statista Gmbh.5 

Downsides of pure cloud computing in IoT use cases

Today, most IoT devices constantly push all data generated to the cloud and use little of the on-device capacities. There are some downsides to that:

 

  1. Data security
    When data is constantly being sent from device to the cloud, the risk of the data being compromised is huge. “As a centralized resource out of users’ control, the cloud resents an ever-present opportunity to violate privacy.”3 
      
  2. Realtime requirement
    IoT applications have a “need for speed”.2 The response time, however, inevitably decreases as the distance between device and the place where data is stored and computed (in the cloud) increases.2
      
  3. Cloud costs
    Pushing, storing, and processing all data in the cloud is associated with high cloud costs. These costs increase as data volumes increase.15

  4. Wastefulness
    Our current bandwidth infrastructure does not support this rapid growth: “With global Internet traffic growing by an estimated 22% per year, the demand for bandwidth is fast outstripping providers’ best efforts to supply it”.Even worse, most data stored in the cloud is of no value to the company and never used.
That is why analysts predict data will move to the edge. 2, 6, 10, 11

What is edge computing?

While there are varying definitions, a simple pragmatic definition is: computing data close to where it is produced, at the edge of the network, instead of a central point far away.1, 9 More technically, it is decentralized data persistence that happens on or near the devices that produce the data.

Moving data to the edge does not mean that data will be solely stored on the edge instead of the cloud. There is just a shift to more data being processed on the edge and less data stored in the cloud.1

Advantages of Edge Computing

Storing and processing data locally on device (e.g. on the IoT gateway) has some advantages:

 

  1. Privacy and Security
    If you are working with sensitive personal data that is not needed centrally / on the cloud, it is easier to keep data secure by storing it where it belongs.18 This can also ease up on GDPR compliance.12

  2. Latency / Speed
    If the data is being stored and processed in a local database, then computing can be done in real time rather than having to communicate back and forth with the cloud for every interaction.5 

  3. Offline-capability
    The more you compute on the edge, the more your app is independent from a constant network connection.19

  4. Costs
    Because you only store the data that is needed centrally or the data you really want to backup, your cloud costs will go down.12

  5. Resourcefulness
    Storing and processing data on the edge and only sending out to the cloud what will be used and useful saves bandwidth and server space.18
Practical Examples

Currently the main focus industries for IoT Edge Computing are Smart Cities, Autonomous Vehicles; Drones, and Industrial IoT.17

A simple case for an IoT edge solution is wearable health monitors. They locally analyze data like heart rate or sleep patterns and provide recommendations without the need for a constant cloud connection.16 It makes sense to be able to get health recommendations in any situation, no matter if there is an internet connection available. Also, not every patient may want all his/her personal health data stored online.
Straightforward IoT use cases, which can only work with local data processing, are (semi-)-autonomous cars. Data needs to be processed in real-time and independent of a network connection as no one would like to crash, because of lagging. Also, edge computing enables cars to process more sensor data faster and find patterns.5 
 
There is a trend to move data to the edge

When you look around the internet, you will find many studies predicting the rise of edge computing, for example: Peter Livine, partner at Andreessen Horowitz, predicted “the end of cloud computing” in favor of edge computing. Livine believes that the bulk of processing will soon take place at the device level.13 Transparency Market Research (TMR) forecasted the global edge computing market will be worth US $13.3 billion by the end of 2022.14 The IDC’s Global IoT Decision Maker Survey showed that 43% of IoT decision makers want to build on edge computing.4

For us edge computing makes a lot of sense, because it is resourceful, efficient, and data stays where it belongs. That’s why we built ObjectBox. Everything else, we’ll see… 😉

 

ObjectBox is an object oriented embedded database solution for IoT-devices (10x faster than SQLite, 1 million+ entities/second400kb native core, cross platform compatible, ACID-compliant). 

Sources

[1] Cisco (2011) https://www.cisco.com/c/dam/en_us/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf
[2] Ieee (2018) http://innovationatwork.ieee.org/how-edge-computing-will-drive-5g-technology/
[3] Usenix (2015) https://www.usenix.org/system/files/conference/hotcloud15/hotcloud15-zhang.pdf
[4] IDC (2017) https://www.ge.com/de/sites/www.ge.com.de/files/IDC%20MarketScape_Worldwide%20IoT%20Platforms_Software%20Vendors_US42033517%5B1%5D.pdf
[5] Data Makes Possible (2018) https://www.datamakespossible.com/edge-computing-transform-autonomous-cars/
[6] IDG (2017) https://www.idgconnect.com/abstract/28926/iot-set-push-computing-edge-2018
[7] Scientific American (2016) https://www.scientificamerican.com/article/the-bandwidth-bottleneck-that-is-throttling-the-internet/
[9] Gartner (2017) https://www.gartner.com/smarterwithgartner/what-edge-computing-means-for-infrastructure-and-operations-leaders/
[10] Business Insider (2016) https://www.businessinsider.de/edge-computing-in-the-iot-forecasts-key-benefits-and-top-industries-adopting-an-analytics-model-that-improves-processing-and-cuts-costs-2016-7?r=US&IR=T
[11] Ieee (2016)  https://spectrum.ieee.org/tech-talk/telecom/internet/popular-internet-of-things-forecast-of-50-billion-devices-by-2020-is-outdated – as always with predictions. However, talking to many CTOs of IoT companies, we see there definitely is a need for edge computing.
[12] IoT Agenda (2018) https://internetofthingsagenda.techtarget.com/blog/IoT-Agenda/Living-on-the-edge-Why-IoT-demands-a-new-approach-to-data
[13] Andreessen Horowitz (2016) https://a16z.com/2016/12/16/the-end-of-cloud-computing/
[14] Transparency Market Research (2017) https://www.transparencymarketresearch.com/edge-computing-market.html
[15] SysGen (2017) https://sysgen.ca/cloud-vs-in-house-servers/
[16] Gartner (2017) https://www.gartner.com/smarterwithgartner/what-edge-computing-means-for-infrastructure-and-operations-leaders/
[17] Bowery Capital (2016) https://bowerycap.com/blog/insights/4-impacts-edge-computing/
[18] Hubraum (2017) https://www.slideshare.net/hubraum_iotacademy/edge-computing-and-5g-a-powerful-digital-mix-for-iot-ait-83646157
[19] HPE (2018) https://www.hpe.com/us/en/insights/articles/iot-analytics-strategy-cloud-edge-or-both-1803.html