If you’ve ever needed a database for your iOS app, you’ve probably had to manage schemas, tables, query strings and all sorts of overhead. Moreover, whenever you wanted to modify the structure of your database, you had to write migration code so that your users’ data would be upgraded to fit the new structure.
Wouldn’t it be nice if your database just did all that for you, automatically? That was what we thought when we designed ObjectBox. ObjectBox thinks the way a Swift developer does: You take your objects and stick them in a box. You use regular Swift methods and operators to search for objects in your database. You add or remove fields and the database just copes with it. And you can use it right now.
Today, we are bringing the power of ObjectBox to Go. Whatever solution you are building, be it a web service, an IoT/IIoT solution, or any data-driven application, you will benefit from the efficiency and speed of ObjectBox (see benchmarks below). Let us know what you think!
Let’s look at some code, and see how ObjectBox persists Go structs (objects):
The ObjectBox Go API allows you to create data-driven cross-platform apps. ObjectBox supports x64 and 32 bit ARM (ARMv6 and ARMv7) CPUs, enabling you to benefit from a super-fast scalable database on IoT devices, industrial edge gateways or, for example, the Raspberry Pi family (from the minimalistic Pi Zero to the high-spec Pi 3B+). At the same time, you can now target desktop and server apps running on Linux, MacOS, or Windows using Go.
How to get started
To get started, please have a look at the ObjectBox Go docs. Here’s the TL;DR version for your bash:
go install github.com/objectbox/objectbox-go/cmd/objectbox-gogen/
go test github.com/objectbox/objectbox-go/...
This will clone the repository to your GOPATH, download the objectbox-c library and install the ObjectBox code generator.
At this point you can start using ObjectBox in your project – just define a struct you want to persist, add //go:generate objectbox-gogen comment in the same file and run go generate ./… on your project. See Getting started for more details.
We have done some quick CRUD performance tests to give you an impression about the speed ObjectBox provides. On a i7-8850H CPU, ObjectBox consistently processes over 1.5 million CRUD operations per second with 4.4 million object reads per second:
To put these numbers into perspective, we also did a comparison using an edge computing platform and two leading NoSQL databases. We won’t spill the names and final numbers just yet, as we are going to release all the details very soon. 😉
Disclaimer: ObjectBox was the only database running in embedded mode, which is not supported by the others in this setup. Obviously this can have a significant impact on performance. On the other hand, this is the most resource friendly mode of operation (RAM and CPU), which might be very relevant if you target restricted ARM32 devices.
This is the first public version of ObjectBox for Go and we are more than excited to get your feedback on it. We have prepared a short questionnaire for you, which should only take 1-2 minutes. Your feedback is extremely valuable to make ObjectBox a fun tool to use.
While this initial release covers all basic features, the native core of ObjectBox offers much more than what is currently exposed to Go. For example, a complete set of query conditions and relations between objects. Also, we will introduce a client/server operation mode. This will allow ObjectBox to run in Containers (e.g. Docker) and in classic server setups. As a major theme, the ObjectBox team is also working on data synchronization across devices. This keeps edge devices and gateways “in-sync”. Sync also enables seamless integration with mobile apps (Android and iOS) pushing relevant data to mobile clients and back. Of course sync will also be available for Go. You can sign up here for updates on sync.
Vivien: Hi Kevin, thanks for talking with me today. Why don’t you start by telling me about yourself and your current open source project, GitCoin?
Kevin: I’ve been a startup entrepreneur over the past ten years, and all of the businesses that I’ve built on or built have been based off of open source software. One of the things that I’ve learned is that value accrual goes to the application layer, to the data that you’re collecting about marketing, and the services that are built on top of the software. Gitcoin is my current project, and its mission is to grow open source. Our thesis is that Blockchain is a game changer for open source funding. There’s so much money in Blockchain right now because of the 2017 Bull market… billions of dollars chasing too few developers. And that creates an opportunity for developers to monetize their work in open source software. Gitcoin is our core product and it’s basically a double sided market: people who want to augment their team can hire talent on a per-issue basis. So, it’s less commitment than hiring someone for a contract. On the other side, developers get to work on issues in open source, learn new skills, meet new people, and get paid using cryptocurrency. We incentivize developers to work on their open source software. The second product that we just launched is called CodeFund, kind of like a sub brand that we’re working on. Basically, it is ads on documentation sites that are aimed at open source developers. So the idea is that if you have a following for your open source repo, you can create passive income by putting ethical advertisements. These advertisements don’t track your users on your documentation site or on your Github and they sustain your open source with passive income. This is a long way of saying that we think Blockchain is a game changer for open source funding and are trying to build the rails for capital deployment to open source projects.
Vivien: We work in the worldwide market where laws are different, e.g. EU vs US. Do you have support for the contracts?
Kevin: Each project kind of has its own license, and we have a repo setup guide where we ask people to add contributor guidelines, licenses, stuff like that. If you look at the Gitcoin terms of service, it basically says that if you develop for this repo, then you are assigning rights according to their license. But the worldwide nature of all of this stuff is very interesting because enforcing IP agreements, from Germany over to the US, there’s not really a court of law that handles that, so we’re kind of waiting for the legal system to catch up in that respect.
Vivien: You’ve built all your business on open source. What do you recommend to other people looking to build on open source?
Kevin: I think it depends on the nature of your business and whether or not open source can accelerate it. For me, I’ve built a lot of B2C companies, and leveraging open source data stores and open source web servers worked for me, but if I was a risk averse financial institution, then I think it’d be different, right?
We think Blockchain is a game changer for open source funding and are trying to build the rails for capital deployment to open source projects.
Vivien: Why did you choose to always build on open source? What do you think is best about open source?
Kevin: I think that open source tools with an active community are the ones that are going to have the need for developers, because they have support, and support isn’t monopolized by one company. I think that because of the scale of development today, if you have a problem with an open source repo, you can likely search it and find someone else who’s had that problem in the past. I actually just watched Revolution OS, which is an interesting documentary about the free software movement. Then later the open source movement, and it made a really interesting case. Just focusing on creating great software, and not worrying about the RM and licensing of that software, you can get it in the most hands possible. This includes people who are focused on creating working software over comprehensive documentation or licensing, and therefore that tends to create the best software over time. I think those are all reasons that I’ve been doing it, but it’s very much a cultural thing in startups these days. I think if I’d been born a generation earlier, maybe we’d be singing the praises of Microsoft Visual Studio or something like that, but it’s a generational thing, you know?
If you’re an open source developer… it’s about more than good compensation, it’s about doing work that aligns with your values, and has an impact on the world.
Vivien: So, you use open source projects as a basis – what is your business model?
Kevin: Well, in my prior businesses, we would build off open source but our software itself was proprietary. We sold it with various degrees of success (laughing). I think for Gitcoin we’re still figuring out the business model, so I’m in awe a little bit, about that, but it’s basically a question about where the value accrues in the network. Our core thesis is that by focusing on being open source, we can make our product as good as possible because we have the most contributors. We think that the value accrues in the brand, and the relationships with the customers. So, while anyone can fork Gitcoin and copy it if they wanted to, then they wouldn’t have our network of developers, they wouldn’t have our mission and our ethos, and they wouldn’t have the relationships with the people who are going to deploy the capital. So, I think it’s very much a work in progress, but that’s a strategy at least, Gitcoin is less than a year old so that’s the strategy in year 0.
Vivien: But you’re already quite big, right? How many users do you have?
Kevin: We’ve got about 7,000. We’ve benefited from being well funded by consensus, but I wouldn’t say we’re super big yet, the open collectives of the world have hundreds of thousands of users and so I think, that to me is big. I think we’re still a small startup.
Vivien: What types of projects do you think you’ll be mainly supporting? Like big company type of open source projects?
Kevin: If you picture a Venn diagram of open source software and Blockchain, we’re in the center of that Venn diagram, and it means that a lot of the people we fund are kind of like Blockchain hipster types, to be honest. I think eventually we would like to go into the broader open source community, and traditionally that’s been monetized with big corporate types, and so I think there’ll be a cultural shift and maybe a little bit of a brand shift as we do that. But that’s very much a year 2 or year 3 thing, there’s plenty of money in Blockchain, so we’re in no hurry to exit that niche at this point.
Vivien: So if I have a small open source project, often I struggle devoting my time to it, especially if it’s more of a side project, could I expect to be getting help in your community, and what would I need to expect to pay?
Kevin: One of the things that we found about open source software developers and their motivation is that it’s about more than good compensation. It’s about doing work that aligns with your values, and has an impact on the world. So, if you’re a repo maintainer that wants to incentivise people to work on your repo, having a mission that aligns with your core contributors, and giving them an ability to impact your project are two axes that you can modify in addition to the compensation axis. When you’re seed stage, you want to be working with people that are intrinsically motivated to work on your project. Balancing those intrinsic and extrinsic motivations is really important. If I were talking to a repo maintainer who wants help on their project, I would help articulate their mission, and identify where people can have an impact. Only then would I start thinking about putting Gitcoin balances on their repo.
“There is this paradox with software, that strategic value doesn’t always mean economic value for the authors(…)”
Vivien: There’s a whole ecosystem of great libraries that are used by thousands of big companies, yet struggle to maintain their projects due to restrained resources. How can they get paid? Can they get paid?
Kevin: I’ll hedge my answer a little bit and say that I am at the intersection of Blockchain and open source, and I think that Blockchain changes the dynamics of funding for open source. This is sort of outside of my area of expertise, but I do have ten years of experience in working on software prior to doing Blockchain stuff, and so for that reason I empathize with the contributors of these project. They built something that by all accounts brought lots of value for the world. There is this paradox with software, that strategic value doesn’t always mean economic value for the authors, but my hope is that we can solve that problem moving forward. For right now, I would say these contributors could try to put some CodeFund adds up, because if you can earn a couple hundred to a thousand dollars per month by putting ethical advertising in front of your audience, then I think that at least helps ease some of the pain associated with those support requests and feature requests. Right now I think that we’re sort of in the early stages of the Blockchain revolution, and so I can’t say that we have anything immediately up our sleeve, but I hope that things change and turn around for projects like this.
Kevin can be seen speaking at Coinvention in Philidelphia, August 31, 2018.
Uwe: That’s a very good question. I started in school in 2004 because a friend of mine coded a basic vocabulary training program in Visual Basic. I started to improve that program and later enrolled in a programming class in school, which was very new at the time. Right now it’s common to have computer science classes in school, but back then it was not the standard. I started doing other side projects for fun, but only got serious about coding when I started studying computer science at University in 2008.
Dorian: Do you think going to University helped your projects?
Uwe: Yes, absolutely. There’s a lot of technical and theoretical background that you are not exposed to, if you just program. At University, the coding exercises were very easy for me, because I had a lot of practice beforehand. However, I learned so much about other things like designing systems, Software Engineering, how to manage projects. Also the theoretical background, like data structures, how to do things efficiently. Also things I never would have looked at, like computer architecture, how stuff works on the lower levels, etc. Which is interesting to know, but it doesn’t really help with the stuff I am currently working on.
Dorian: But that knowledge of the lower levels has maybe helped you solve some bugs?
Uwe: It has a little bit. I think the only relevant thing that I still come across now is bit operations. But things like registers, or programming in assembler, no. This is probably more helpful for the C guys, which program at a lower level. I like working at a higher level. So I program in Java, Kotlin, for Android, and Windows back in the day. These all abstract the lower level away and I didn’t have to worry about memory management, which I didn’t really like doing back then. It’s complicated and very easy to get wrong. I don’t know if you have heard about pointers, but they are one of the biggest problems with programming in C. If you get a pointer wrong everything breaks, you have security issues, and so on. If you talk to C developers about pointers, they could go on forever.
Uwe Trottmann at Droidcon Berlin 2018.
Dorian: How did you start up with Open Source? Did you start right away out of University or did you do work for other companies and do Open Source on the side?
Uwe: In the beginning I had my own projects and didn’t know about Open Source. The first thing that I open sourced was SeriesGuide. My work on SeriesGuide started as a hobby because I had recently switched to Android and was missing the series management app I previously used on another platform. I first posted SeriesGuide to Google Code and then the Google Play Store in 2010. I later moved it to GitHub 2011. My work got more serious during my computer science studies at University. Once I started using GitHub, I added more libraries that SeriesGuide uses and I contributed to other projects on there as well. GitHub really accelerated the SeriesGuide development.
SeriesGuide Statistics Screen
Dorian: Did having access to other Open Source projects help you develop SeriesGuide?
Uwe: Of course, it tremendously helped. I guess SeriesGuide is probably 80% other libraries, it’s Open Source so you can go check. Some of the API integration with the TV show database I used back in the day was an Open Source library, so I built on top of that. I then replaced it with my own later. Also if you know Jake Wharton, the Keynote speaker for Droidcon Berlin this year, he was one of the early contributors to SeriesGuide and I used some of his libraries in the beginning like ActionBarSherlock. Then later the networking and logging libraries he was working on. Obviously I used the Android Open Source libraries once they came along. SeriesGuide would not exist without Open Source software.
Dorian: Why did you choose Open Source as the foundation for your paid app?
Uwe: It started as free and Open Source at the beginning on the Google Play Store, because I was just a computer science student and didn’t have the need to make money. It was also rather complicated to charge money, because you have to set up your own company. Around three years after I posted it to the Play Store, due to people suggesting that I charge money, I started developing extra paid features, such as notifications, theming, and then later the syncing component.
Shows on SeriesGuide
Dorian: Is the core of your project still Open Source and will it stay like this forever?
Uwe: The whole app besides the small server component is public domain. And the spec for the server component is open, so you could build your own if you wanted to.
Uwe: Honestly, I’d have to say the community. Back in the day I had a few people who really pushed me to improve the app, like Jake Wharton, Craig, Chris, and other friends. They really helped me get SeriesGuide off the ground. Without them I would have fixed a bug or two, and then would have moved on. Because of the continuous requests and a growing community, I had to keep going and make it better. And without the community paying for the app, I wouldn’t have the means necessary to continue working on it.
Dorian: What do you dislike about Open Source?
Uwe: Honestly, in the beginning I thought open sourcing would be risky because someone could copy my app, slightly modify it and then publish it, taking credit for my hard work. Because of that I have a separate private repository with new features that I hold off publishing for a few days. But if I remember correctly, this only happened once and was not an issue because the developer didn’t continue development. I am not 100% certain it was a copy, but the code looked very similar to mine. Right now, I don’t think this is an issue anymore. If people use my app they expect my updates because I maintain the app. If somebody copies it, they don’t have my work power and knowledge.
Uwe’s contribution to SeriesGuide on GitHub as of July 6th, 2018.
Dorian: How do you manage your time between developing Open Source projects and your work?
Uwe: I work Mondays and Tuesdays on my main job, kind of as a backup. The rest of the week whenever I have time, I work on SeriesGuide and related Open Source projects. Of course it never ends up being three days. If somebody wants to contribute, I welcome pull requests. I also allow people to translate the app into their language. If somebody has a bug to report, the easiest way to do that is on the website. If people want to contribute code, they can go to the GitHub contribute file which lists everything you need to know.
Dorian: Do you have more features in mind right now?
Uwe: There’s a feedback site where people can request changes, and there’s no shortage of suggestions! I also have ideas in mind, but some things are bigger and I don’t want to do them right away. It’s a constant struggle to get things done. I am not out of ideas; it’s more of a time problem right now.
Dorian: You have a very good time structure, which is encouraging. How hard is it to turn an Open Source project into a profitable business? How did the community react that you have built a business with paid features on top of an Open Source project?
Uwe: The overall reaction was positive. Obviously there are some people that aren’t happy with the presence of paid features. I always tell them that the app has no ads in the free version and you can do pretty much everything except for some convenience features. I would say that for most people this is not an issue because the basic functionality of the app is still freely available. One of the early complaints was that I charge too much money. So in the beginning I was charging 1€ as a one time payment for all access, then increased prices to 3€ per year or a 7€ one time payment. I explained this was to support myself and future updates, basically development time, and most people understood that. I clearly mention that inside the app, paying is not only to get some features but also to support me and future updates. So far most people like the idea and buy into it. Obviously, if you look at the share of paid vs free users it’s nowhere near something like 50/50, it’s way less. But I have a core set of users who believe in the app and support it financially. So far this method has gone well. I am also pushing people towards subscriptions now to secure future development.
SeriesGuide X Pass
Dorian: Do you think the complaints came mostly during the shift from free to some paid features?
Uwe: Yes. In the beginning some people were upset, but as I only made convenience features paid, it wasn’t a huge issue. That is also a takeaway for me: don’t have a free app and then suddenly make half of the features paid, because then users will get upset. I was very careful to only make new or some convenience features paid. Or when I made the previously free theming features paid, I made an effort to spruce them up before. But features like notifications and server syncing were new and weren’t available for free before I implemented them.
Dorian: Do you have any tips for developers that are starting with their Open Source projects or are thinking about open sourcing their code?
Uwe: The best thing you can do for your project is to Open Source it, or in short: put it on GitHub. If you don’t put it online, people won’t find it. Another good way to get traction is to contribute to other projects so people know you exist. And then if you ask for help or feedback about their library you are using in your project, people will get to know your project and maybe look at it. Contributing is a good way to get into the community and be noticed. Give and be given, right?
Dorian: Now we have some fun fact questions. First, what do you think about Microsoft’s acquisition of GitHub?
Uwe: I don’t really care, because Microsoft has done some great things for the Open Source community. Azure runs Linux now and they have their own Linux operating system. It’s still good to know that there is the Open Source alternative GitLab that people can switch to if needed. However, it’s sad that things get monopolized into those big companies. In the end, we are maybe going to end up with only Microsoft, Google and Amazon. GitHub sold out, but we’ll have to see what happens.
Dorian: Do you have a favorite book, film, or game?
IoT is growing at a very rapid rate and with it the vast amount of data it produces. Handling these amounts of data is an unresolved challenge. Edge Computing could be part of the solution.
According to Dave Evans of Cisco, in 2010 the number of IoT devices connected to the internet passed the world population, with a device to person ratio of 1.84.1 By 2020 there will be up to ten Web-connected devices per person, collectively producing over 40 zettabytes of data.2, 4
The graph on the right shows the estimated number of IoT devices from 2015-2025, Statista Gmbh.5
Downsides of pure cloud computing in IoT use cases
Today, most IoT devices constantly push all data generated to the cloud and use little of the on-device capacities. There are some downsides to that:
Data security When data is constantly being sent from device to the cloud, the risk of the data being compromised is huge. “As a centralized resource out of users’ control, the cloud resents an ever-present opportunity to violate privacy.”3
Realtime requirement IoT applications have a “need for speed”.2The response time, however, inevitably decreases as the distance between device and the place where data is stored and computed (in the cloud) increases.2
Cloud costs Pushing, storing, and processing all data in the cloud is associated with high cloud costs. These costs increase as data volumes increase.15
Wastefulness Our current bandwidth infrastructure does not support this rapid growth: “With global Internet traffic growing by an estimated 22% per year, the demand for bandwidth is fast outstripping providers’ best efforts to supply it”.7 Even worse, most data stored in the cloud is of no value to the company and never used.
That is why analysts predict data will move to the edge. 2, 6, 10, 11
What is edge computing?
While there are varying definitions, a simple pragmatic definition is: computing data close to where it is produced, at the edge of the network, instead of a central point far away.1, 9 More technically, it is decentralized data persistence that happens on or near the devices that produce the data.
Moving data to the edge does not mean that data will be solely stored on the edge instead of the cloud. There is just a shift to more data being processed on the edge and less data stored in the cloud.1
Advantages of Edge Computing
Storing and processing data locally on device (e.g. on the IoT gateway) has some advantages:
Privacy and Security If you are working with sensitive personal data that is not needed centrally / on the cloud, it is easier to keep data secure by storing it where it belongs.18This can also ease up on GDPR compliance.12
Latency / Speed If the data is being stored and processed in a local database, then computing can be done in real time rather than having to communicate back and forth with the cloud for every interaction.5
Offline-capability The more you compute on the edge, the more your app is independent from a constant network connection.19
Costs Because you only store the data that is needed centrally or the data you really want to backup, your cloud costs will go down.12
Resourcefulness Storing and processing data on the edge and only sending out to the cloud what will be used and useful saves bandwidth and server space.18
Currently the main focus industries for IoT Edge Computing are Smart Cities, Autonomous Vehicles; Drones, and Industrial IoT.17
A simple case for an IoT edge solution is wearable health monitors. They locally analyze data like heart rate or sleep patterns and provide recommendations without the need for a constant cloud connection.16 It makes sense to be able to get health recommendations in any situation, no matter if there is an internet connection available. Also, not every patient may want all his/her personal health data stored online.
Straightforward IoT use cases, which can only work with local data processing, are (semi-)-autonomous cars. Data needs to be processed in real-time and independent of a network connection as no one would like to crash, because of lagging. Also, edge computing enables cars to process more sensor data faster and find patterns.5
There is a trend to move data to the edge
When you look around the internet, you will find many studies predicting the rise of edge computing, for example: Peter Livine, partner at Andreessen Horowitz, predicted “the end of cloud computing” in favor of edge computing. Livine believes that the bulk of processing will soon take place at the device level.13 Transparency Market Research (TMR) forecasted the global edge computing market will be worth US $13.3 billion by the end of 2022.14 The IDC’s Global IoT Decision Maker Survey showed that 43% of IoT decision makers want to build on edge computing.4
For us edge computing makes a lot of sense, because it is resourceful, efficient, and data stays where it belongs. That’s why we built ObjectBox. Everything else, we’ll see… 😉
ObjectBox is an object oriented embedded database solution for IoT-devices (10x faster than SQLite, 1 million+ entities/second, 400kb native core, cross platform compatible, ACID-compliant).