fbpx
Introducing: ObjectBox Generator, plus C++ API [Request for Feedback!]

Introducing: ObjectBox Generator, plus C++ API [Request for Feedback!]

We are introducing the ObjectBox Generator today to simplify ObjectBox development for more programming languages, starting with C/C++. Additionally, we are releasing a brand new C++ API that goes hand in hand with the new generator. Historically, our C API was rather low level as it was focused on providing the foundation for our Swift and Go APIs. With this release we want to provide C/C++ developers with ObjectBox convenience and ease of use. 

ObjectBox Generator takes over the burden of writing the binding code and data model declaration. Based on a single input file, it generates the code for you, so you can focus on the actual application logic.

Generator Example

ObjectBox let’s you handle data as FlatBuffers. For example, you can put and get data objects as FlatBuffers encoded bytes. To work with FlatBuffers, you need to define a FlatBuffer schema file (.fbs). And this file is also the input for ObjectBox Generator. This way, everything is defined in a single location.

Let’s say we have a FlatBuffers schema file “task.fbs” with the following content:

Now, we can tell ObjectBox Generator to use this file to generate C++ sources:

This makes ObjectBox Generator to generate the following files:

  • objectbox-model.h: source code to build the internal data model, that you need to pass when creating a store.
  • objectbox-model.json: keeps track of internal schema IDs; you don’t need to worry about this except that you should put it in your source control.
  • task-cpp.obx.h: the C++ value structs (data objects), binding code for FlatBuffers and the new Box class.

C++ API Example

Now, let’s use the previously generated code and the new C++ API around the Store and Box classes. A simple CRUD application boils down to a few lines:

Note that the generated code is header-only and compatible with the existing ObjectBox C-API, allowing both to be used from the same application. The C and C++ APIs both have unique advantages: the C++ API uses RAII so you do not need to worry about cleaning up, while the C API has additional features, e.g. queries.

Open Source, Docs

ObjectBox Generator is open source and available on GitHub. The repository comes with a readme file that also serves as a documentation. Among other things, you will find ObjectBox specific annotations there, which are used in fbs files to express ObjectBox-specific concerns. For example, in the definition of Task above, we used ulong as a FlatBuffers type to store dates. However, FlatBuffers does not know what a date is and we use ObjectBox annotations to express this:

For our initial release of ObjectBox Generator and the public C++ API we decided on labeling it as version 0.9. Although we are already very close to a 1.0 and we wanted to gather some feedback before our first major release. As we can still change the API or smooth out any rough edges you may find, we cannot stress enough how much we welcome and appreciate your feedback at this point. Thank you!

ObjectBox EdgeX v1.1 – database with ARM32 support

ObjectBox EdgeX v1.1 – database with ARM32 support

With EdgeX Foundry just reaching v1.1, we continue to provide ObjectBox as an embedded high-performance database backend so you can start using ObjectBox EdgeX v1.1 right away. If you need data reliability and high-speed database operations, ObjectBox is for you. Additionally, starting with ObjectBox EdgeX 1.1, you can use it on 32-bit ARM devices.

Combining the speed and size advantages of ObjectBox on the EdgeX platform, we empower companies to analyze more data locally on the machine, enabling new use cases.

With ObjectBox-backed EdgeX we’re bringing the efficiency, performance and small footprint of the ObjectBox database to all EdgeX applications. It is fully compatible, so you can use it as a drop-in replacement: you call against the same REST and Go EdgeX APIs. As simple as that;no need to change any code.

Performance comparison of EdgeX database backends

EdgeX Foundry comes with a choice of two database engines: MongoDB and Redis. ObjectBox EdgeX brings an alternative to Redis and MongoDB to the table.  Because ObjectBox is an embedded database, optimized for high speed and ease of use while also delivering data reliability, it enables a new set of use cases. As we all know, benchmarks are hard to do. This is why all our benchmarks are open source and we invite you to check them out for yourself. To give you a quick impression of how you could benefit from using ObjectBox, let’s have a look at how each compares in basic database operations on “Device Readings”, one of the most performance intensive data points.

Note: The Read and Write operations (all CRUD (Create, Read, Update, Delete) operations are measured in objects / second). The benchmarks test internal EdgeX database layer performance, not the REST APIs throughput.

These benchmarks provide a good perspective why you should consider ObjectBox with EdgeX. Benchmark sources are available publicly in ObjectBox EdgeX github repo.

So, why is ObjectBox EdgeX faster?

First of all, you are probably aware of the phrase “Lies, damned lies, and statistics benchmarks”. Of course, you should look at performance for yourself and consider based on your specific use case needs. That’s why we make our benchmarks available as open source. It is a good starting point.

To make it easier to compare ObjectBox (in addition to our open source benchmarks) here are some of the high-level “unfair advantages” that make ObjectBox fast:

  • Objects: As you can derive from its name, ObjectBox is all about for objects. It’s highly optimized for persisting objects. The EdgeX architecture and Go sources are a great fit here as it puts Go’s objects (structs) in the center of its interface. This means, we can omit costly transformations and this helps with speed.
  • Embedded database: Redis and MongoDB are client/server databases running in separate processes. ObjectBox, however, is running in the same process as EdgeX itself (each EdgeX microservice, to be precise). This has definite efficiency advantages, but it also comes with some restrictions: Whereas you can put Redis/MongoDB in separate Dockers or machines, this option is not available for ObjectBox yet.
  • Transaction merging: ObjectBox can execute individual write operations in a common database transaction. This means, we can reduce the costly transactions for a number of write operations. This is a great way to add on performance, delaying the transaction end by single digit milliseconds.

Get started with ObjectBox EdgeX

The simplest way to get started is to fetch the latest docker-compose.yml and start the containers:

You can check the status of your running services by going to http://localhost:8500/. At this point, you have the REST services running at their respective ports, available to access from your EdgeX applications.

Find more details, instructions for ARM32, and sources in our GitHub repo at  https://github.com/objectbox/edgex-objectbox.

If you’re new to EdgeX, find out all about the open source  IoT Edge Platform here. The EdgeX project is led by the Linux Foundation and supported by many industry players, including Dell, IBM, and Fujitsu.

We love to hear from you 🙏

We’re very interested to hear about the challenges you are facing on the edge and in IoT. As performance experts, we are always embracing a tough challenge. Reach out to us to set up a pilot project.

Is there something you are missing? Please do reach out to us. We want to make ObjectBox the best edge data persistence layer available. We love to receive your feedback.

What next?

Find out more about ObjectBox EdgeX and get started, go directly to GitHub or download the snap on Snapcraft.

Sync.Drone: a drone project based on ObjectBox

Sync.Drone: a drone project based on ObjectBox

This spring, a student group from Augsburg University of Applied Sciences build a drone application based on ObjectBox Database and Sync. This is a guest blog post by Michelle from the sync.drone project group, describing the project from start to finish and sharing the results. 

The goal: Showcasing the ObjectBox database and Synchronization solution with drones

The goal of the project was to synchronize the colors and flight patterns between two drones to coordinate themselves autonomously in formation flight to showcase the ObjectBox solution. In the future, this technology could be used in many drone applications. First, due to ObjectBox’ speed, more data can be processed faster on each drone, saving resources, specifically battery. This allows drones to fly longer. At the same time, going beyond the scope of this initial showcase, the technology could be used to synchronize swarms of drones, making their use more reliable and flexible – and less dependant upon a constant Internet connection. For example, as an artistic installation, or in emergency situations during a large-scale search for missing persons. Drones can also be used in large warehouses to facilitate the organization of different parts, and pass on the position of a particular part.

 In this article, we will explore the process we used to build our self-synchronizing drones, sharing our software and hardware, so you can try it out yourself.

Hardware: Raspberry Pi, 3D printing, and more

In order to build our drones and turn our vision into reality, we had to consider a number of hardware options. It was important that our drone was compatible and programmable with ObjectBox. The drone had to be localizable and airworthy, so that a safe autonomous flight was possible. All parts had to be compatible so we could easily swap parts if something did not meet our requirements.

We built the drone frame from scratch, using 3D printers. The housing was created in the 3D program Autodesk Inventor and the parts were assembled to a drone frame. We used NeoPixel RGB LED sticks to make the drone glow in color. We chose the following components. 

Microcomputer

A Raspberry Pi was the most suitable central computer on our drone. It offered both performance and size. We chose the Raspberry Pi 3 B+, which would later control the processes of our drone independently.

Tracking system 

After looking at different tracking systems, we chose the “POZYX” UWB tracking system. This ensured an accurate and user-friendly handling.

Accumulator

We had to make sure that the drone’s battery would last long enough to power a Raspberry Pi, LEDs and a POZYX tag in addition to the flight hardware. First started with a 6 cell LiPo battery with 5000mAh. However, later in the project, we replaced the battery with a lighter and more compact 6-cell LiPo battery with 1800mAh.

Engines

The engines (1750 kV) from the Drone-Racing sector had enough power to make the drone fly. Motors with even lower kV would have given the drone more power, but are much more expensive.

Flight controller 

As flight controller we chose the “Omnibus F4 V6” chip, which ran with the open source software “Beta Flight” and was accessible via the so-called Multiwii Serial Protocol (MSP). This allowed us to use the advantages of a proven flight software, and also transfer the flight instructions via USB directly to the flight controller using the MSP.

Electronic Speed Controller (ESC) 

For the ESC , which implements the instructions of the flight controller by direct voltage changes at the motors, we chose a 4-in-1 model. With only one connection cable to the flight controller, all four motors can be controlled at the same time. Usually one ESC is required per motor. It was also compatible with our hardware.

Software – Tracking, Flying and Syncing the Drones

Except for a start signal, the drone was supposed to operate without a remote control. Several drones would coordinate themselves at the same time according to the instructions. We decided to develop the code in three separate “cores”, which were merged at the end of the project. These were divided into “tracking”, “flying” and “syncing”. Using the university git lab as a repository, we were able to simplify development and share the code with the group. This allowed structured work on the code. With the help of ObjectBox and Prof. Dr. -Ing. Alexandra Teynor we were able to assemble the following code parts.

Tracking 

For collision avoidance it was important to implement tracking, so that the drone knows it’s own position. We solved this by using the position calculated by the POZYX tag, which was then transmitted to the Raspberry Pi in the tracking core.  We read the coordinates from the IMU sensors (“inertial measurement unit” = unit of measurement based on multiple sensors ) from the POZYX tag, but not the exact positioning.

The so called “heading”, or yaw of the drone, is read out by a magnetometer. However, this internal “compass” reacts to disruptive factors and can deliver inaccurate results. We solved the correction of the heading via an algorithm using OpenCV. This algorithm uses a small camera module on the drone and special markings on the ground to detect its orientation. This allows the direction vectors of the drones to be calculated more accurately.

Flying

In the flying core, the flight instructions were developed based on the tracking core data, and then implemented by passing this data on to the flight controller. First of all the drones have to be lifted off the ground. For this purpose we used a laptop keyboard control, which forwarded flight instructions to the drone via a web socket.

Flight control

The Raspberry Pi establishes a serial connection to the flight controller via USB. As soon as this connection is established, flight instructions are transmitted in the form of inclination values for roll, pitch, yaw and throttle (thrust). These values may lie between 1000 and 2000. In a neutral position, roll, pitch and yaw are at an average value of 1500.  

Using Python, we calculated the required roll, pitch, yaw and throttle values and assembled them using the Multiwii Serial Protocol. This was translated into pure byte code and sent to the flight controller via the USB cable. The flight controller now tries to reach the corresponding values. In order to turn to the right, the left motors are turned slightly up and the right motors slightly down. The ESC received the commands for the desired motor speed from the flight controller. It then applied the required voltage to each motor according to its instructions. The communication between the flight controller and ESC happened either by an analog (PWM) signal or a digital signal (D-Shot).

Keyboard control 

The computer runs a Python script that registers keystrokes and converts them into instructions. For example, pressing the right arrow key creates the command “raise-roll” and pressing the left arrow key triggers the command “lower-roll”.

The drone also runs a Python script that opens a web socket to which the PC script connects. Each time a key is pressed on the laptop, a corresponding command string is generated (e.g. “raise-yaw”) and sent to the drone via the web socket. As soon as a string arrives, the relevant value (roll, pitch, yaw, throttle) is increased or decreased.

To prevent the drone from crashing if a connection is lost, the values are flattened algorithmically.

ObjectBox Database and Sync Drone Implementation  

In the syncing core, the position data of all drones as well as the LED color, should be exchanged and commands passed on. The RGB color space of the LEDs was mapped to the x-, y- and z-position. In this way, the sync features of the drones could be displayed without them flying. For the implementation we used the ObjectBox database and the ObjectBox Sync Server.

Originally, we had planned to use the ObjectBox Go Binding because it is precompileable and very fast. However, the POZYX system we choose used Python. There was also already a Python implementation available for our flight controller, but none available for Go. Luckily, ObjectBox offered to develop and provide a small Python binding of their database according to our needs. This included all ObjectBox functions that were relevant to us. It was officially released in version 0.1.0 specifically for our project. As a result, the ObjectBox database could be easily integrated into our code.

Realization of syncing

In Python version 0.1.0, ObjectBox incorporated the basic features we needed. For our application the simple CRUD functions and the Sync feature, which synchronizes the data in near real time, were sufficient. The database is compact and the speed and ease of use is optimized for restricted IoT devices, for example the Raspberry Pi used in this project.

The sync server is started by running the init-server.py script on the master drone. At first, an empty database (model) was initialized. The master drone then communicates with the other drones via WLAN network connection and synchronizes the ObjectBox database between the respective devices.

Three entities (classes) are stored:
– the identification and position data of the anchors
– the identification and position data of the tags
– the color values of the LEDs.

The drone stored it’s position and LED color in the database. The master drone then reads out this information and overwrites it  with the values calculated by the master drone (e.g. LED color or target position in the future).

Thank you!

At the end of our project, we had three drones. Depending on the position of the master drone, all drones could synchronize their LED colors. Unfortunately we were not able to finish the flight due to a defect in the flight controller and a delayed delivery of parts. Finally we decided to publish the code for the drone control on GitHub. Additionally, you can get inspired on our website as well as on our social media platforms. 

Furthermore, we would be happy, if the project would be continued by another group of students in the future. With our work we have created a basis for many more ideas. In summary, our project still has a lot of ambitious potential for the future.

Thanks to ObjectBox for this great opportunity – we mastered many problems along the way and learned a lot. Thanks for the constant support.We also thank our professors Prof. Dr. -Ing. Alexandra Teynor, Prof. KP Ludwig John and our coach Sandra Hobelsberger for their professional advice and patience. Finally, we would like to thank HSA_Innolab for their additional financial support and FabLab for their advice and resources.

In collaboration with interactive media students of the University of Applied Sciences in Augsburg.

 

ObjectBox Swift Binding Open Sourced

ObjectBox Swift Binding Open Sourced

Today’s 0.9 release is not just a step closer to 1.0, but also an important milestone for us: for the first time, all Swift binding source code is available on GitHub. We’ve also improved performance (again) and added useful features like type converters.

ObjectBox’ lightning-fast database offers a cross-platform C API for language-specific bindings. These bindings provide a thin and nimble wrapper around the C calls and expose ObjectBox functionality in the way that feels most natural to the host language (i.e. Swift).

An open-source binding allows you to make changes to ObjectBox and build it on your own, backport it to older OS or Swift versions, and contribute your changes back to us if you feel so inclined.

Is this the binding I’ve been using all this time?

Although it’s one of our newer language bindings, our Swift binding already has a storied and colorful history. The first Swift binding actually predates our C library and was built directly against the core C++ codebase. As such, it used Objective-C to bridge Swift and C++.

Once we had the C library, we set about eliminating the intermediary and started rewriting the central Objective-C portions as pure Swift on top of the C API. The immediate result when we released ObjectBox for Swift 0.8, was a significant speed-up in write operations due to Swift’s new native UTF-8 strings, and struct support.

But for the open source release, we completed this rewrite, and ObjectBox Swift got even faster:

Not having to bridge to Objective-C meant we could take advantage of Swift’s associated types to resolve class/entity associations instead of a runtime look-up, and eliminate a few duplicated classes necessitated by adding Swift features to Objective-C classes. This in turn allowed removing some unnecessary data copying, which translated directly to increased speed.

While we enjoy a good performance increase as much as the next database developer, we also added new features and fixes, like more query comparison operators for scalar types, and improved behavior when adding properties to an entity.

Property Converters

One major new feature of this release is support for converting simple user-defined types into types the database understands. For example, you can mark any RawRepresentable enum with an annotation to tell ObjectBox how to serialize it. Let’s say we have an enum like

In your class, you simply add the following annotation:

But of course we also let you work with your own types, or system types. Let’s say you wanted to save a window’s dimensions into your database. You do this using a converter class that implements convert() methods that convert your custom type into one of ObjectBox’s built-in types, and back:

And then you tell ObjectBox to use that converter using an annotation:

That’s it.

And that’s just a selection of the changes you’ll find in the new open-source release. It contains the entire binding, but also the fork of Sourcery that does its duties as our code generator, and a performance test app. It is faster, more feature-full, and we’ve also squashed some bugs in the binding.

To take a look and get started, simply go to our ObjectBox Swift Git repository, where, in addition to our example application, you can now also find the binding’s source code. Or pull version 0.9 via CocoaPods.

We can’t wait for you to try it out and let us know what you think.

GitCoin: Harnessing the Power of Blockchain for Open Source

GitCoin: Harnessing the Power of Blockchain for Open Source

Vivien had the chance to chat with Kevin Owocki, about his latest project GitCoin, and the intersection of open sourcce and blockchain.
Kevin Owocki

Kevin Owocki

Startup entrepreneur, founder of GitCoin
Vivien: Hi Kevin, thanks for talking with me today. Why don’t you start by telling me about yourself and your current open source project, GitCoin?

Kevin: I’ve been a startup entrepreneur over the past ten years, and all of the businesses that I’ve built on or built have been based off of open source software. One of the things that I’ve learned is that value accrual goes to the application layer, to the data that you’re collecting about marketing, and the services that are built on top of the software. Gitcoin is my current project, and its mission is to grow open source. Our thesis is that Blockchain is a game changer for open source funding. There’s so much money in Blockchain right now because of the 2017 Bull market… billions of dollars chasing too few developers. And that creates an opportunity for developers to monetize their work in open source software. Gitcoin is our core product and it’s basically a double sided market: people who want to augment their team can hire talent on a per-issue basis. So, it’s less commitment than hiring someone for a contract. On the other side, developers get to work on issues in open source, learn new skills, meet new people, and get paid using cryptocurrency. We incentivize developers to work on their open source software. The second product that we just launched is called CodeFund, kind of like a sub brand that we’re working on. Basically, it is ads on documentation sites that are aimed at open source developers. So the idea is that if you have a following for your open source repo, you can create passive income by putting ethical advertisements. These advertisements don’t track your users on your documentation site or on your Github and they sustain your open source with passive income. This is a long way of saying that we think Blockchain is a game changer for open source funding and are trying to build the rails for capital deployment to open source projects.

Vivien: We work in the worldwide market where laws are different, e.g. EU vs US. Do you have support for the contracts?

Kevin: Each project kind of has its own license, and we have a repo setup guide where we ask people to add contributor guidelines, licenses, stuff like that. If you look at the Gitcoin terms of service, it basically says that if you develop for this repo, then you are assigning rights according to their license. But the worldwide nature of all of this stuff is very interesting because enforcing IP agreements, from Germany over to the US, there’s not really a court of law that handles that, so we’re kind of waiting for the legal system to catch up in that respect.

Vivien: You’ve built all your business on open source. What do you recommend to other people looking to build on open source?

Kevin: I think it depends on the nature of your business and whether or not open source can accelerate it. For me, I’ve built a lot of B2C companies, and leveraging open source data stores and open source web servers worked for me, but if I was a risk averse financial institution, then I think it’d be different, right?

We think Blockchain is a game changer for open source funding and are trying to build the rails for capital deployment to open source projects.

Kevin Owocki

Founder, GitCoin

Vivien: Why did you choose to always build on open source? What do you think is best about open source?

Kevin: I think that open source tools with an active community are the ones that are going to have the need for developers, because they have support, and support isn’t monopolized by one company. I think that because of the scale of development today, if you have a problem with an open source repo, you can likely search it and find someone else who’s had that problem in the past. I actually just watched Revolution OS, which is an interesting documentary about the free software movement. Then later the open source movement, and it made a really interesting case. Just focusing on creating great software, and not worrying about the RM and licensing of that software, you can get it in the most hands possible. This includes people who are focused on creating working software over comprehensive documentation or licensing, and therefore that tends to create the best software over time. I think those are all reasons that I’ve been doing it, but it’s very much a cultural thing in startups these days. I think if I’d been born a generation earlier, maybe we’d be singing the praises of Microsoft Visual Studio or something like that, but it’s a generational thing, you know?

If you’re an open source developer… it’s about more than good compensation, it’s about doing work that aligns with your values, and has an impact on the world.
Kevin Owocki

Founder, GitCoin

Vivien: So, you use open source projects as a basis – what is your business model?

Kevin: Well, in my prior businesses, we would build off open source but our software itself was proprietary. We sold it with various degrees of success (laughing). I think for Gitcoin we’re still figuring out the business model, so I’m in awe a little bit, about that, but it’s basically a question about where the value accrues in the network. Our core thesis is that by focusing on being open source, we can make our product as good as possible because we have the most contributors. We think that the value accrues in the brand, and the relationships with the customers. So, while anyone can fork Gitcoin and copy it if they wanted to, then they wouldn’t have our network of developers, they wouldn’t have our mission and our ethos, and they wouldn’t have the relationships with the people who are going to deploy the capital. So, I think it’s very much a work in progress, but that’s a strategy at least, Gitcoin is less than a year old so that’s the strategy in year 0.

Vivien: But you’re already quite big, right? How many users do you have?

Kevin: We’ve got about 7,000. We’ve benefited from being well funded by consensus, but I wouldn’t say we’re super big yet, the open collectives of the world have hundreds of thousands of users and so I think, that to me is big. I think we’re still a small startup.

Vivien: What types of projects do you think you’ll be mainly supporting? Like big company type of open source projects?

Kevin: If you picture a Venn diagram of open source software and Blockchain, we’re in the center of that Venn diagram, and it means that a lot of the people we fund are kind of like Blockchain hipster types, to be honest. I think eventually we would like to go into the broader open source community, and traditionally that’s been monetized with big corporate types, and so I think there’ll be a cultural shift and maybe a little bit of a brand shift as we do that. But that’s very much a year 2 or year 3 thing, there’s plenty of money in Blockchain, so we’re in no hurry to exit that niche at this point.

Vivien: So if I have a small open source project, often I struggle devoting my time to it, especially if it’s more of a side project, could I expect to be getting help in your community, and what would I need to expect to pay?

Kevin: One of the things that we found about open source software developers and their motivation is that it’s about more than good compensation. It’s about doing work that aligns with your values, and has an impact on the world. So, if you’re a repo maintainer that wants to incentivise people to work on your repo, having a mission that aligns with your core contributors, and giving them an ability to impact your project are two axes that you can modify in addition to the compensation axis. When you’re seed stage, you want to be working with people that are intrinsically motivated to work on your project. Balancing those intrinsic and extrinsic motivations is really important. If I were talking to a repo maintainer who wants help on their project, I would help articulate their mission, and identify where people can have an impact. Only then would I start thinking about putting Gitcoin balances on their repo.

“There is this paradox with software, that strategic value doesn’t always mean economic value for the authors(…)”
Kevin Owocki

CEO, Gitcoin

Vivien: There’s a whole ecosystem of great libraries that are used by thousands of big companies, yet struggle to maintain their projects due to restrained resources. How can they get paid? Can they get paid?

Kevin: I’ll hedge my answer a little bit and say that I am at the intersection of Blockchain and open source, and I think that Blockchain changes the dynamics of funding for open source. This is sort of outside of my area of expertise, but I do have ten years of experience in working on software prior to doing Blockchain stuff, and so for that reason I empathize with the contributors of these project. They built something that by all accounts brought lots of value for the world. There is this paradox with software, that strategic value doesn’t always mean economic value for the authors, but my hope is that we can solve that problem moving forward. For right now, I would say these contributors could try to put some CodeFund adds up, because if you can earn a couple hundred to a thousand dollars per month by putting ethical advertising in front of your audience, then I think that at least helps ease some of the pain associated with those support requests and feature requests. Right now I think that we’re sort of in the early stages of the Blockchain revolution, and so I can’t say that we have anything immediately up our sleeve, but I hope that things change and turn around for projects like this.

Kevin can be seen speaking at Coinvention in Philidelphia, August 31, 2018.

EdgeX Interview: Why open source is key for IoT and Edge Computing

EdgeX Interview: Why open source is key for IoT and Edge Computing

We were thrilled to speak with Michael Hall, Brett Preston (the Linux foundation) and Jim White (Dell) about their open source platform EdgeX.

Jim White

Jim White

Dell, EdgeX

Jim is Vice Chair of the Technical Steering Committee of EdgeX. He is also Team Lead of the IoT Platform Development and IoT Solutions Division at Dell.
Michael Hall

Michael Hall

The Linux Foundation for EdgeX Foundry

Michael is Developer Advocate and Community Manager at the Linux Foundation.

 

Brett Preston

Brett Preston

The Linux Foundation for EdgeX Foundry

Brett is Developer Advocate and Community Manager at The Linux Foundation for EdgeX Foundry.

 

Vivien: What is EdgeX and where are you at with it?

Michael: EdgeX Foundry is a vendor neutral open source platform containing a collection of micro services that take care of different aspects of what you’re going to need to have an edge computing platform. If you’re making IoT devices, you don’t want to reinvent that layer of the stack. Having that common platform for IoT is something that is going to benefit everybody. The Linux Foundation is a neutral umbrella over EdgeX. Inside the project are all the member companies who are actually funding the development, putting developers and marketing resources in, to make it an actual, usable product for everybody. That’s the model of the project and the actual code itself. The main goal of EdgeX is processing and transporting data between IoT devices and sensors and things in the cloud and on the backend. The focus is on being able to respond locally as much as you can, so that you don’t have the latency of going on the cloud and back. And also, being able to continue working, if you loose that connection.

Jim: EdgeX is a an open source platform containing a collection of micro services that take care of different aspects of what you’re gonna have to do to have an edge computing platform. Any of those are semi-dependent: You can replace anything you need to replace. For the status of the project: We just had a year of fast pace growth and we have rewritten everything in Go, so all of our processes are a lot smaller and more efficient now.

Vivien: How are you currently tackling local on-device data persistence?

Jim: We currently use Mongo DB as the persistence engine although we could support almost any kind of persistence store at the edge as long as it was small enough. We also have used SQLite in the past for a couple of customers. However, Mongo DB is the largest element in our portfolio of services. There are a couple of reasons why we are probably going to offer an alternative to Mongo with our next big release in spring 2019: Footprint, licensing, and lack of support for ARM32.

Michael: As we are a collection of microservices, you can always swap out individual pieces depending on what your needs are.

Vivien: Is EdgeX and its components restricted to certain licenses?

EdgeX builds on Apache 2

Jim: EdgeX is an Apache 2 license open source project, so we prefer Apache 2 level or at least a compatible license, because we want to be very business friendly. We want people to take the application and use it in all sort of settings, including actually embeddeding it in gateways. We also want to be very decoupled at discrete points.

For example, if I’m a company like Dell and I use EdgeX. If some of my customers have an absolute demand that a certain database be at the heart, then I want to be able to choose the database, depending upon the customers, the use cases, and the environment that they find themselves in. EdgeX is all about the flexibility. So, for this example, we offer what we call a reference implementation database. Customers or users could take EdgeX and replace elements with their own technology, which may not be open source even.

Michael: You can take what’s open source and add proprietary file systems or hardware depending on what your specific needs are. EdgeX tries to be that common open source base. It provides all of the functionality in a open source license but that still lets you replace bits as needed with whatever it is that you want to run.
Vivien: Can you give us an example how and where EdgeX is currently used? 

Jim: There are over 70 companies now that are part of the EdgeX community and each group is using it differently. There are some that are serving EdgeX as the Red Hat model: they are providing distribution, services and support behind EdgeX. A company like mine, Dell, we’re trying to find a platform that actually goes on our gateway. So we’re going to build a commercial version of EdgeX for our own platform. There will be pieces that we will replace based on better performing mechanism to some of our cloud based products. Then you have other groups out there that are proving particular services for EdgeX, for example edge analytics. There are lots of different service capabilities where we see potential replacements. Then there are companies like Samsung that uses EdgeX in their factory floor to help run their automation. So, they are users, but they also want to make sure  EdgeX meets their needs. Our community is made of snowflakes, they are all very special *laughs* – common goals but different use cases for almost everybody that is part of the organization.

 

Vivien: That sounds really cool. In your opinion, moving the data to the edge, what is the edge, where do you see the data ending up, for example more on the sensor level or gateway level?
Latency concerns, cost of shipping up the data, and the ability to actuate locally are key reasons why you have to have edge software.
Jim White

Vice Chair of the Technical Steering Committee, EdgeX

Jim: We absolutely believe EdgeX is a mechanism for the edge. While you could run pieces of EdgeX on the cloud, we do not believe is what the future holds. There are gonna be certain use cases where that works, but latency concerns, the cost of shipping up the data, and the ability to actuate locally are all key ingredients and reasons why you have to have edge software and edge platforms. Now, these are gonna get smaller. At Dell, we are manufacturing gateways of different sizes, because we know that certain use cases are gonna dictate a larger box and other are going to dictate something like a Raspberry Pi or even smaller. We have companies in our foundry that are looking at running parts of EdgeX in things like PLCs, to help address their realtime needs. So, we absolutely believe that the edge is very much going to be a part of our IoT environments. There are going to be use cases that dictate different levels of compute all the way up from sensors to cloud.

 

Michael: And all of our member companies see a need for that platform, but that platform is not going to be their product or their service. So everybody wants it to exist, so everybody is gonna work together to make it exist, so that they can build their own value-add on top of that or below their device level. Everyone agrees that this is an important thing, that we have to have a solution there for all the innovation that people see on the horizon.
Markus: So, small device level versus gateway – would you say your current focus is on the gateway?

Levels of Edge Computing

Jim: I would say that it really isn’t that one or the other is more important. You’re gonna have situations, as we know from a Dell perspective, where what we call a brownfield device (e.g. a 1979 modbus engine) needs a gateway, because it doesn’t have the ability to communicate into any kind network otherwise. So there has to be a gateway that provides that first level of compute. There are other things that are evolving in the industry: think of say windmill generators, where there is lots of capability right there at the device level, there is a lot of compute built right into those systems. So things will run at that level, and then you have everything in between. Even something like BLE or Zigbee type of environments where there is wifi and ability to connect directly to a network. Typically, we’re finding organizations are reluctant to allow those kind of things to connect into their major networks without some security, apparatus and analytics to see what’s going on, so as not to create problems in their larger networks. So even there, a gateway may be necessary, not because of hardwiring or physical connections, but because you want some insurances in place at the edge before that data leaks on up to your enterprise.

It’s the worst way to build our product except for all others.
Jim White

Vice Chair of the Technical Steering Committee, EdgeX

Vivien: What’s the worse about open source that you’ve experienced?

Jim: *laughs* Now you are going to make me say some things in front of Brett and Michael as members of the Linux foundation… There is a quote by Winston Churchill that talks about democracy, saying it’s the worst form of government except all others. I kind of feel the same way about open source development. It’s the worst way to build our product except for all others. Because it does take time. It’s a community effort and anything done by a community automatically seeks a ground where it’s going to be the best and brightest product. So you get the best input from everybody, but it takes time. It’s easier for say something like Dell to go marching off and build a software solution that they think is the best. It will get there faster but it’s not necessarily going to get there in a way that the world and communities accept more easily. So anything built by many hands is going to take a little bit more time and a little bit more process. But it ends up getting a lot better results I think in the end.

Michael: Whenever you have a community building something you can’t just come in and say “This is what you’re gonna build” because they don’t have to do what you say. And that’s true even with EdgeX. Everybody who is working on it is working for a company invested in it, but there is no one person who can say this is what you’re all going to do. So it’s not enough to say just what you want done. You have to explain and justify why and get people to buy into that. And that takes more effort, but you have to know that what you’re proposing is the right solution, that it’s going to work. If you can’t explain that, you can’t communicate that to the community then it’s not going to get done. As Jim said, it takes time but at the end product is going to be better.
Jim: In this case with IoT, I will tell you that no one company will be able to provide it all. As Dell, we would love to be the company providing it all… (laughter) We have learnt the hard way that in an IoT landscape there are going to be certain things in the company that you can’t touch and IoT has to touch everything. Maybe it’s the network, hardware or operating systems, particular sensors and protocols. You can help to persuade customers to do some things in your way, but you’re never going to be able to get them to do everything in your way and that’s why IoT takes an ecosystem. Which is why we think the second part of EdgeX is so important; our product is important, but just as vital is the ecosystems. We have a collection of companies all trying to work together to provide for interoperability. That is just as important as the actual end product we develop.
No one company will be able to provide it all.
Jim White

Vice Chair of the Technical Steering Committee, EdgeX

Vendor lock-in is not going to work in IoT.
Michael Hall

Developer Advocate, Linux Foundation

Michael: Vendor lock-in is not going to work in IoT. There is no way any company is gonna be able to provide all the needs of somebody. So having an equal playing field for everybody, having that common ground that anybody can come to and interact with anybody else, is what’s going to allow us to fulfill the promise of IoT in general.
P.S. You can meet Brett, Michael, and Jim at the Open IoT Summit Europe in Edinburgh in October.
Please help us learn more about the data persistence landscape on the edge by filling out our small questionnaire (literally 2 minutes). Thanks!