Building a Business on Open Source

Building a Business on Open Source

What is open source software?

For the sake of unambiguity: Open source software (OSS) primarily means that the source code of the software is accessible and users are free to use the code as they please. Depending on the license, you might be expected to attribute the source code to the authors and / or commit code enhancements back. Note: It’s “free” as in “freedom” not as in “free beer”. 

opensourceneedsmorebalance

Open Source and Commercialisation?

The origins of open source did not entail commercialization thoughts. However, in the last 20 years a lot of things have changed, and open source projects have seen commercial successes – though not always by the creators and maintainers… Open source is in its core tied to a philosophy and value set for many people. Simplified: For the developer community by and large open source is considered to be “good”  versus proprietory source code is considered to be “evil”.

In any case, open source is one way to keep up an active vibrant developer ecosystem that empowers individual developers as well as startups and smaller players. Open Source is actually one piece of the IT ecosystem that helps balance the Big Tech and drive overall innovation. However, we also believe the open source ecosystem needs more balance to be successful longterm. If widely used open source repos cannot even sustain the half or full developer resource needed to maintain them, then there might well be a flaw in the system. If startups cannot build a business around their widely used open source code to sustain it longterm, it is to the disadvantage of the community, especially for the individual developers and SMEs. And likely, the learning at some point will be to keep the source closed instead.

In the following we will share, why we believe now is the unique opportunity to add fairness and balance for the value creators to the open source ecosystem to keep that ecosystem thriving and successful longterm.

What do we mean with “building a business on open source”?

In many talks with many people, we found there’s at least two diametric conceptions of building a business on open source:

1) using open source software for free and building something around it to earn money
2) developing a solution and open sourcing it or parts of it as part of the business model

In this article, we mean the latter and it inherently entails contributing a useful part of a solution to open source. For some open source enthusiasts a company needs to open source everything to be an open source company, and that’s ok. It is just our definition for this article.

A look at the market – the struggle of open source businesses

The Open Source Gold Rush: Success Stories

In the last years there have been many open source success stories, e.g. MongoDB, elastic, Cloudera all IPOd very successfully. There seemingly is a lot of money in open source businesses, e.g. a study by Fraunhofer concluded that “the EU economy is hugely benefiting from global OSS.” [1] Also, companies and big corporations are way more open to work with open source software, indeed 2020 was the first year where open source databases were on par with closed-source databases with regards to corporate adoption (see chart). [2]

And a recent (2021) report showed that across 17 industries, from 1,546 codebases 98% contained open source code. [3] There even is a bit of a hype that open source is the path to success. Now that it’s clear that it is possible to build a business with open source software, VCs also are more open to funding open source businesses. An Andreessen Horowitz report reveals that OSS companies have raised over $10B in capital with a trend towards bigger and bigger deals. [4] Annual invested capital in open-source and related dev tools has increased at around 10% CAGR over the last 5 years. [5] In the years 2018 and 2019 acquisitions, mergers, and IPOs from open-source companies generated over 80USD billion liquidity value according to Bessemer Venture Partners. [6]

The struggle of turning Open Source into a Business

GitHub Sponsorship fail Historically, open source companies have struggled with turning open source adoption into monetary success, “less than a decade ago open source was considered almost impossible to monetize.” [7] Sadly, that’s still a reality today for many open source maintainers and companies alike. Lots of open source maintainers with widely used open source code (“successful open source”), cannot get enough financial support to maintain the code. Of course, there are some successes, but in the end that might also be a question of ratios. For example, in 2020 GitHub reported having more than 190 million repositories. Even if only 10% of those do want to build a business on top of their code, how many of those see a financial reward? Gut feel: Far less than typical startup success odds. On top: What looks successful from the outside, might not really be a viable self-sustained business. Despite its many users, MongoDB spent $100M on development, and it took them more than 10 years to become profitable according to their own statements. [8] 

db-enginesMariaDBvsMySQL A lot of tech companies struggle with – and spend a lot of time on – all the decisions around an open source business model. It isn’t easy, read up how GitLab struggled with finding a business model, or look closer into the MySQL story, and the MariaDB journey (which is a MySQL fork by the founders and original authors of MySQL); look at blog posts from CockroachDB, MongoDB, or elastic on open source – and what you see is a constant re-positioning of open source strategies.

As Mike Volpi from Index Ventures noted at the Index Open Source Summit (2021): “It took Mongo DB 10 years to derive the business model they run now and monetize successfully…” Wow, 10 years to somewhat successful monetization – and that is one of the major open source success stories.

Open sourcing your main technology as a strategy

In this article, we take a deeper look at open source as a pro-active business strategy.

open-source-traction-growth-business Open Source to Build Traction

Traction is the most obvious reason to open source your product. It works like Freemium in the Mobile Games market – or more generally the Mobile Apps market. It’s a great way to evaluate product-market-fit and build traction. When you have that, you can think about monetization.

However, there is a big difference between giving something away for free and open sourcing it. If we stay in the mobile app world: Would open sourcing the app help with traction? Would it jeopardize the business model? Unless the main target users are developers, at least in the beginning likely not – less than making the app / game available for free in any case. However, once the app grows at amazing pace, open source availability could become a challenge in several respects.  

The most obvious would be fast followers entering with that same game and potentially much bigger marketing budgets and better customer access (e.g. on the apps store). Think what would have happened if WhatsApp would have open sourced all its code from day 1 on top of giving the app away for free? It is a legit hyothesis that a fast follower could have scraped some of the market, changing the whole story. On the other hand, if they would open source all their code base now, how much would it harm them? At some point, it beame all about the traction, brand, customer access, so, I would think, it wouldn’t harm them at all at this point. So, driving traction with open source is probably only a viable idea if you address developers or engineers. It’s clearly a phenomenon of the developer-led landscape, and acts as a developer distribution channel. This being said, the price of open source traction is commercialization. It’s a straight forward trade-off: The more open and free your license is, the harder it is to monetize later on. 

building-trust-open-source Open Source to Build Trust

Trust is something that is likely more important for certain software types (e.g. B2B and core tech).

ObjectBox is a database and with that it is a data-centric “core technology” / software infrastructure, sitting at the heart of a company’s solution. Anything that gets used at the heart of other companies or their solutions needs a lot of trust. Trust is easier to come by with size, “no one was ever fired for choosing SAP.” Being a small startup lies at the opposite on that spectrum for many decision makers. Open Source can be a way to overcome this specific challenge and build trust in three ways:

  1. Transparency: The freedom to verify what the code enables; the internal developer team can check the code and vouch for the solution 
  2. Risk-reduction: The freedom to change and maintain the code oneself gives independence from the authors and the success of the solution
  3. Quality: If an open source solution is actively used by a large number of developers quality inevitably goes up 

So, if you are looking for adoption from big players in heavily regulated or security-concerned industries, e.g. medical, manufacturing, automotive, anything with mission-critical networks, open source can help you overcome many of the adoption hurdles you are facing.

open-source-ip Open Source as an IP Strategy

Seems counter-intuitive, right? Well, if you are not aiming to patent your technology, you still might not want someone else (who has been working on the same problem) to patent the same technology harming your freedom to operate. You can protect yourself from that risk by open sourcing it. This can come in the form of a copyleft license, designed to encourage further innovation advancements to the benefit of all, but also limiting the commercial exploitation opportunities for everyone. Or, you can choose a more permissive license, allowing people with commercial interests to keep any advancements they make to themselves. 

Note: Open source code is not a blueprint with exact instructions; there are no obligations to provide clear docs or explanations. While a majority of open source projects strive to deliver a code base that is readable by others, it is not controlled. So, while open sourcing a technology harms patenting it, unfortunately, a way to still protect it, is making it hard to understand. On the other hand, a patent must have an extensive explanation. This makes it easily repeatable by others in the future, after the end of the patent protection, or as a basis for further research (and ways to tweak it in a novel enough way). 

Although it often feels like open source is on the other spectrum of patents, a patent has a limited timeframe and people can learn from it even before it expires. The deal is basically an exchange of knowledge (to be used in the future) for protection (for commercially exploiting it). Keeping it a trade secret has other risks, but could mean that an invention wouldn’t be shared with others for a truly long time. And of course the protection encourages big companies to invest big budgets in R&D too. Delayed open source actually has many similarities with a patent, in both cases the tech is only made available for advancements and unrestricted use after a certain time frame has ended. 

Open Source for the sake of it

There are a lot of ideas floating around open source, and some pressure from the developer community to open source everything. Among developers, open sourcing is considered to be good, social, fair, transparent, and worthy. While there are many advantages in open source, it has turned into a kind of “political tool”, and that’s a downside – and probably the opposite of the original idea. 

Consideration 1: How is a great software supposed to be maintained and advanced without anyone providing funds? When MMOGs (Massive Multiplayer Online Games) became a thing, people understood that there was a constant cost associated with it and were willing to switch from a one-off fee to monthly payments. Software typically needs to be maintained too. So, there are ongoing development costs associated with a piece of software, even if it is not hosted. So, who benefits from open source in the end, if the original creators cannot keep up their work (assuming they need to eat and sleep)? Before pushing everyone to open source, maybe read here, here, here, or here about open source maintainers struggling under the pressure and dealing with burnout.  On the flip side, if a company markets itself heavily as an “open source company”, they should give considerable parts of their own value creating solution back to the community. Using open source tools and building on top of open source code (and even committing back to these solutions) does not mean you are an open source company: If you want to reap the marketing benefits of calling yourself an “open source company” then you should truly be one and commit your value back to open source.

Consideration 2: Who benefits if another company pulls the repo, adds “sparkles”, maybe even some “missing features”, or merely a big “brand name”, or the “marketing budget” and makes a ton of money selling the solution? This is of course assuming a permissive license was used. Well, from an open source perspective that is perfectly fine, and part of the intention of open source. So, it’s great, right? We think, it is easy to understand that some authors who have put all their “free time” / unpaid time into that code struggle to accept when this happens, especially if they have a hard time supporting themselves. But we also understand that big companies with investors (stakeholders…) that have invested heavily in R&D and might or might not yet have reached profitability, don’t really like to see this happen. Unless you are really in it for the fun and driven by altruism and will be in perfect harmony with other people using your code to make money, you should look closely if and how you want to open source your code.

Open Source to save development costs

There is the idea floating around that you can develop your project for free using the open source community. We doubt it works out for many. Of course, if Google maintains a repo that is a base technology used by many developers, developers might want to commit something (anything really) for fame, to be part of it, maybe to get noticed. However, the “anything really” is already a problem: Someone needs to review the submission, respond, potentially rework it and so on… Most other repos will probably not get too many commit requests (let alone from the best tech talent around). Even then, onboarding a large community of unknown developers and letting them commit to your code has its challenges – especially if you are quality-conscious and / or trying to build a business. It creates a lot of work to review commits and reject / merge them. And on top of that from a legal perspective you need to have a waterproof contributors license signed by anyone committing. There clearly is some work involved in the process, maybe more than what it is worth sometimes. 

Also consider this: Most successful open source projects that turned into a business success have limited contributors and / or only internal (contracted) contributors. For example, SQLite 99% of the code was done by Richard Hipp (author and founder of SQLite), and MongoDB stated that about 98-99% of the code was done internally. Redis was almost exclusively coded by Salvatore Sanfilippo. In a presentation from Index Ventures (one of the most renowned open source VCs), one criteria for potentially successful open source businesses was that at least 90% of the code base was developed internally – and of course that the team owned all the IP. If you are after cheap development and external help with your project, maybe take a closer look if open source is the right path.

What open source business models exist? 

The following open source business models are common, but typically used in combination and not as pure models, e.g. most open source companies offer paid support, but rarely only paid support. Note: With time the examples may become wrong/outdated, because once you look into it, you will notice that companies adapt / change their model regularly. If you need to understand one specific company’s model you need to dig into it individually at that time.

There are three basic open source licenses to be distinguished: permissive, weak copyleft and copyleft.

A quick high-level note on the major license effects

Copyleft – major point is that derived works must be open sourced with a compatible copyleft license, meaning any advancements and changes to the work will be contributed back to the community and freely available for unrestricted use.

Weak Copyleft – the weaker copyleft refers to licenses where not all derived works inherit the just described copyleft effect; typically used in software libraries, e.g. a database library used in app development, so the library can be used in a mobile app without needing to contribute the whole app to open source; only changes to the database library itself would carry the copyleft effect.

Permissive – a permissive open source license allows you to do anything with the source code including keeping derived works to yourself and commercialising on it

Description Examples Note
Paid Support Providing paid support, trainings, certificates RedHat Where has this approach been working – as a pure paid support approach – ever since Red Hat?
Open Core The core product is free and open source, extra features are paid; have an open-source core and sell closed-source features on top of it SugarCRM,
MySQL
It is basically the widely successful freemium model just with open source; typically you expect the large majority of users to use it for free. The open source part of course enables anyone to build the same features as you
Dual Licencing The free open source sw uses a copyleft license, whereas the paid license is a commercial license without copyleft effects MySQL,
elastic
This kind of license enables you to monetize your commercial (typically bigger users) and still enables the community to expand the product landscape and innovate based on the code base
Delayed Open Source All code will be fully open sourced with a time delay (details and timings vary) MariaDB,
Cockroach DB
The effect depends also on the licenses used, but typically it protects you from competition for a given time frame, so only you can exploit your development commercially and gain market share / develop an advantage based on market entry time. At the same time it reduces the risk for adopters, because they know the code will become available to them
Open SaaS Offering the software open source and hosted as a service (SaaS), which is the primary source of revenue allowing anyone to do the same with the software with a permissive license (self-host or host for others) WordPress,
Sharetribe,
MySQL,
MariaDB
This model has been the major point of discussion in the last 3 years and is seen by many as the holy grail for monetizing open source software; it also triggered many companies to move away from an open source licensing model as large cloud providers can easily host an open source product at better rates
“Closed SaaS” Strictly speaking / officially not “open source”. Offering the solution open source and hosting it as a service (SaaS) while NOT allowing anyone to host it, often times unless they contribute the whole solution back to open source (copyleft effect)) MongoDB,
elastic,
Cockroach DB
The first license that built this specific copyleft-effect into its license was MongoDB (SPSL). The license has since been adopted by e.g. elastic, …. Since then similar licenses have been developed. OSI did not approve the license as an official open source license.
“Ad model” For lack of a better name, I called it “Ad model”; it’s really having so much reach and traction that companies pay for customer access through your solution or similar co-operations AdBlock Plus,
Firefox
Can take many variations: For instance, the open-source application AdBlock Plus gets paid by Google for letting whitelisted acceptable Ads bypass the browser ad remover.
Or, in 2014 Yahoo struck a deal with the Mozilla Corporation to make Yahoo the default search engine in Firefox

 

A look at the open source market

Name Founding Year Funding Summary Started with Open Source (license) Open Source Evolvement Devtool Open to contributions / CLA HQ* Notes / Story synopsis
MongoDB 2007

6 funding rounds with a total of $311M

IPO was in autumn 2017; valuation $1.6B

started with AGPL Created SSPL in 2018 causing much debate in the community. SSPL is not an open source license Database “we own 100% of the IP”; 99.9% developed in house and the few contributions accepted were from people who signed a CLA US-based According to statements fromMongoDB, adoption went up after the license change (15 mill dwlds, more than in the prior 10 years together). In 2016 they launched their database-as-a-service offering, which is considered the game changer w. regards to building a business. Until Oct 2017 MongoDb downloads were >30M with 10M from the prior 21 months.
Data Bricks 2013 Total funding 1.9B; last round: Series G; Feb 2021 $1B proprietary PaaS their main service is proprietary, but they use a lot of open source software and have a strong footprint in the open source community Backend NA US-based “Databricks is the original creator of some of the world’s most popular Open Source data technologies” – open source is a large part of their positioning and marketing. However, it seems their main offering, while based on open source, is proprietary. So, not an open source business as defined here.
elastic predecessor released in 2004; first elasticsearch released in 2010; incorporation only in 2012 Total funding $162M; last round was a series D; elastic did IPO in autumn 2018 started with Apache 2 for for elastic search (which was the original main product) Last license change in 2021: You can now choose between the proprietary elastic license or SSPL; so stritly spaking not open source anymore Devtool CLA US-based 2018: elastic IPO –> shares doubled the first day. Note: With so many different products (not a single product company), the open source strategy is harder to grasp.
Confluent 2011 Total Funding Amount $455.9M, last round: series E Unlike Apache Kafka which is available under the Apache 2.0 license, the Confluent Community License is not open source and has a few restrictions Kafka is open source,
Confluent isn’t
Devtool NA US-based “Founded by the team that originally created Apache Kafka” – the team behind Confluent contributed a lot to open source prior to Confluent, but the Confluent code itself isn’t open source as far as we understand. They heavily rely on other open source software for their tech stack though.
RealmDB 2011, before the founders did “TightDB” on which the Realm DB was based 4 investment rounds. Then MongoDB acquired them for $39M on Apr 24, 2019 started out closed; then open sourced the database and went for the open core model, then subsequently open sourced the Sync solution too, going for the hosted (SaaS) model from closed to open core to open SaaS; acquired by Mongo to push their backend offerings and complement with an edge and sync (serving Mobile and IoT better) Database looks like they accepted contributions Started in Europe, but HQ went to the US when joining YC 2014; it was since bought bei MongoDB The founders both left the company the year before it was acquired by MongoDB. The acquisition prize was a little less than what Realm had raised in the years before. The Sync solution is now tied to using the Mongo servers / cloud and a huge part of their push for the IoT market.
SQLite 2000 Bootstrapped Public Domain, which we always considered one of the most “open source” ways to open source but in the light of recent discussions around the SSPL license, strictly speaking it is at least not OSI-approved Public Domain, mainly monetize big corporates for being in a Consortium; also offers services and since xxxx? encryption (basically paid feature); our guess is that this is not really a repeatable business model Database Richard Hipp owns all IP, 99% is developed by himself; very limited outside support (2 part-time freelancers that we are aware of, both don’t have any rights to the IP) US-based (privately held by Hipp, Wyrick & Company, Inc (author: Richard Hipp and all stock held by his wife G. Wyrick; both work for the company)), HQ The company has always been and still is run by Richard Hipp and his wife; from a development perspective it is a one-man-show. Richard wrote SQLite himself, as far as we are aware they have no other employees apart from 2-3 part-time supporters for specific versions; very special Open Source Story.
Couchbase Lite 2009 – Couchbase, Inc. is a merger of Membase + CouchOne in 02.2011; both former companies were started 2009 and had funding 251 million USD total funding; 8 rounds with latest Series G for $105 million Apache 2 Delayed Open Source Database US-based (both entities were US-based already before the merger) Couchbase now mainly sells Couchbase Servers; Couchbase Lite is the smallest part of their business; in 2020 there seemed to be a shift towards the Sync Gateway and Edge Computing market in communication; however, the main business still seems be on the server side and based on cloud lock-in.
redis 2009 Total Funding Amount $246.6M redis the database itself is and always was BSD; redislabs is the company that has secured certain rights for redis and sells extensions and add-ons under several licenses, they changed from APGL to Apache 2.0 with Common Clause to a proprietary license called “Redis Source Available License” redis itself is BSD but features / extensions around it from RedisLabs are licensed uner prorietary licenses Database Any contribution needs a CLA that is provided by redislabs; we believe anything committed under this CLA could also be used in redislabs proprietary products (which typically is the same for anything committed under a permissive license, but which has attracted some criticism from the OSS community) Redislabs is US-based. Salvatore Sanfillipo (antirez) was always bsaed in Europe; redislabs originated in Israel RedisLabs is the commercial entity that markets redis; redis was largely developed by Salvatore Sanfilippo. He left redis as a maintainer in 2020.
RedHat 1993 bought by IBM in 2019 for $34 billion; before that they had raised $240.7M Linux, which was the core of the success of RedHat, is GPL (though of course not the company’s decision) RedHat is a huge company, definetely not a single product company, and thus also does not really fit into this matrix, however, it is THE example for successful commercialisation of open source and we feel the matrix would lack without it Backend / Data centric we believe you can contribute to most (all?) projects without a CLA US-based Read here why there will never be another Red Hat (and there is no “Red Hat Model”). Note that of course the Red Hat founders did not write Linux (on which the majority of their success is based), but at the very least they (as well as VA Linux) gave option shares to Linus Torvald out of gratitude (at lest not out of obligation). When both companies successfully IPOd, Linus made 20 Mill USD (in total) from both sales.
MySQL 1995 (development started already in 1994) Total Funding Amount $39.8M, sold to Sun in 2008 for 1 USD billion started out with AGPL; several license adaptions and changes in the open source business model over the years, e.g. for a long time they had a 2 year delay for the open source version, but changed that to no delay at some point. Dual Licensing and Paid Support Database Yes, even though called OCA (Oracle Contributor Agreement) Sweedish company until it was acquired by Sun Microsystems in 2008 (who then were acquired by Oracle) The founders forked the latest MySQL version when Oracle acquired it. Most of the original database code base was developed by Michael Widenius; with regards to database technologies a pattern emerges: Often the core / most of the base technology is developed by one person – as building a database is a rather huge endeavor that’s kind of striking, isn’t it? BTW: MySQL is named after Monty Widenius daughter (“My”)
Hyper 2010 (academic research project at TUM) undisclosed proprietary, not open source None Database NA EU-based; German “university spinoff” acquired by Tableau very early 2016: HyPer acquired by Tableau. Terms of the deal were undisclosed
ParStream 2011 acquired by CISCO in November 3, 2015 proprietary, not open source NA Database NA Originally EU-based (German), then moved to US in 2012, acquired by Cisco in 2015 Cisco ParStream is no longer offered as a stand-alone product. The functionality of Cisco ParStream is now part of Cisco Kinetic.
Cockroach DB 2015 Series E in Jan 2021 for $160M Apache 2.0, plus a proprietary license for enterprise features Started as open core, now a form of closed SaaS with delayed open source: They changed to a proprietary license in 2019, called BSL, which prohibits users from offering CockroachDB as a service (DBaaS, SaaS), and each release converts to an open source license after three years. CockroachDB is therefore officially not considered open sorce anymore Database CockroachDB received significant contributions from the community (“we have had over 1590 commits from over 320 external contributors across all our open source repositories” (2020)), CLA: Yes US-based In June 2019, Cockroach Labs announced that CockroachDB would change its license from the free software license Apache License 2.0 to their own proprietary license, known as the Business Source License (BSL), which forbids “offer[ing] a commercial version of CockroachDB as a service without buying a license”, while remaining free for community use.
Berkeley DB 1994 Acquired by Oracle in 2006 BSD and Sleepycat Public License (a permissive OSS license) Oracle changed to dual licensing with APGL and a commercial license Database NA US-based It is still used in many routers and gutfeel is that the market share in that specific area is good. Unfortunately, no numbers available.
GitHub 2008 In 2018 Microsoft bought GITHUB for $7.5 billion. proprietary, not open source NA Backend / Data centric NA US-based Microsoft bought GitHub for the developer access; that would not have changed if it would have been open source and I do wonder what would have happened to GitHub if it would have been open source; one thing is for sure: GitLab wouldn’t have been able to position themselves as the open source alternative; however: the closed source model worked for them well, even though it is a developer tool.
GitLab development started in 2011; incorporated only in 2014 $434.2M Series E completely open source (MIT license) Now: Open Core Model; Community Edition: MIT License
Enterprise Edition: Source-available proprietary software
Backend / Data centric Originally CLA, now dropped and instead the code must be committed under the same license as the feature is (mainly Apache 2.0) plus a DCO US-based (development was started in Europe, the founders incorporated in the US in 2014 when joining YC) GitLab used being open source as a strong positioning factor against GitHub (which was never open source). It was an odyssey to find a sustainable business model (and it seems it is not SaaS). Note: The pure service model and the donation model did not work for them. Again: The code base of the core system was by and large developed by one person.
MariaDB 2009 Total Funding Amount $123.2M Dual licenscing with GPL license, version 2 and a prorietary source available license for some parts They evolved their dual licensing approach to using the proprietary source avaiable license (BSL) Database Yes, and the CLA is shared under a creative commons license that allows you to use it as you like https://mariadb.com/kb/en/mca/ Sweedish company 10 years after it was forked, MariaDB has 20M users, a fast growing database business and has >€100m backing. Note: The pure service model as well as the donation model did not work for them.

Building an Open Source business Exec Summary – TL; DR

  • There is a lot of evidence that open source companies struggle with open source models and licenses – this is also true for successful companies
  • There is no “Red Hat Model” – just selling services has rarely worked
  • The donation model typically hasn’t worked for open source companies, e.g. GitLab and MariaDB, so it is not astonishing that GitHub sponsorships don’t work out great for most maintainers. Also note: GitHub sponsorships may put you in a bad legal position depending on where you are based
  • There is a trend from successful open source companies towards Source Available licenses instead of “official Open Source licenses”, e.g. MongoDB, elastic, CockroachDB, …
  • There is an indication that successful open source companies are US-based (even if founded / started in Europe), which we believe is due to the funding opportunities provided in the US: 1) the US provides generally more funding (more and bigger funding opportunities; there is lots of market research on that), 2) US VCs and Silicon Valley have the reputation to also fund at earlier stages, e.g. idea stage, and companies with traction (instead of revenue), investing in a longterm perspective. Traditionally, European investors don’t.
  • Public domain is strictly speaking also not considered to be an open source license 😮 (at least not if it needs OSI-approval; does it? 🤔) 
  • While Open and Closed SaaS seem at this moment to have been the most successful models, it is no holy grail and definetely does not work for everyone, e.g. it didn’t work as the sole business model for GitLab

Conclusion

The open source market lacks flexibility and transparency from a licencing / legal perspective, and ever more Source Available licenses don’t help: A “license stack” with building blocks like the Creative Commons would be helpful to mark software easily and clearly with regards to the main terms, e.g. “source available”, “free for commercial use”, “attribution necessary” etc. It would help maintainers and users alike, but needs bigger entities to drive this (like an OSI).

The open source market also needs more balance, at the very least more understanding and “love” towards maintainers. More finanical support as well as other ways of giving back to demonstrate the appreciation of well-maintained repos and great free software, will keep the ecosystem healthy and thriving. That’s a community effort; everyone can contribute.

Dart Flutter Database ObjectBox 1.0 Release

Dart Flutter Database ObjectBox 1.0 Release

In 2019 we first introduced the ObjectBox database v0.1 for Flutter/Dart. Our team has loved the engagement and feedback we’ve received from the developer community since, and we’re thrilled to announce the first stable version 1.0 for ObjectBox Dart/Flutter today.

With this release we bring you the fast and easy to use ObjectBox database for Dart objects: optimized for high performance on mobile and desktop devices. ObjectBox persists your Dart objects (null safe, of course) and comes with relations, queries, transactions, and Data Sync. For a feature list and more, please also check the pub.dev page.

ObjectBox by Example

For those of you new to ObjectBox, here is how you can use it (or check the docs if you want to dive deep right away). By annotating a class with @Entity you tell ObjectBox that you want to persist its objects, which is done putting the object in a Box:

What’s new with the 1.0?

Version 1.0 delivers a stabilized API and adds new essential features like async writes and query streams. We’ve also extended support for Flutter desktop. Let’s look at queries and how they can be used depending on the use case:

There are two new approaches to do async puts for asynchronous database writes: putAsync() returns a Future to check if the call was successful.

Or you can use a background queue if you don’t need individual Futures, the following code inserts 100 objects and only waits once:

If you are interested in further improvements we made to 1.0, please check out the full changelog.

Dart Flutter Database Benchmarks

ObjectBox Dart v1.0 also comes with considerable optimizations bringing a new level of database performance to Flutter apps. ObjectBox enables data-heavy apps that were not possible on Flutter before. Consider this a first sneak-peek; stay tuned for detailed performance benchmarks to be released including queries (hint: they are really fast) along with updated benchmarking code.

What we tested

We looked at some two popular approaches: sqflite, a SQLite wrapper for Flutter (no Dart Native support), and Hive, a key-value store with Class-adapters which seems still popular although its creator abandoned it for architectural shortcomings (it has memory problems and does not support queries). In the previous benchmark we’ve also had a look at Firestore, but being an online-only database it was thousands of times slower than the rest so we’ve left it to rest this time around. Check our previous benchmark if you’re interested.

To get an overview of the databases, we tested CRUD operations (create, read, update, delete). Each test was run multiple times and executed manually outside of the measured time. Data preparation and evaluation were also done outside of the measured time.

ObjectBox, sqflite, Hive performance comparison across CRUD

Looking at the results, we can see ObjectBox performing significantly faster than sqflite across the board, with up to 100 time speed-up in case of create & update operations. Compared to Hive, the results are a little closer in some cases (read) though ObjectBox still comes out on top in all the metrics. Considering that Hive keeps all Dart objects in memory (!) while ObjectBox does not, should give you a good impression of how fast object persistence with ObjectBox is.

ObjectBox Database for Flutter/Dart Highlights

For those of you new to ObjectBox, here’s a quick summary of what our super-fast embedded database offers, out of the box:

  • automatic schema migration: adding new classes or fields just works
  • type-safe APIs, e.g. no interface{} arguments
  • embedded edge database – no server needed, store all data directly on the device
  • no ORM, no SQL
  • relations: to-one, to-many (eager and lazy fetching)
  • robust query support, including indexes for scalable lookups
  • Support for implicit (automatic) and explicit (user defined)
  • transactions: ACID compliant with superfast bulk/batch operations
  • low memory usage
  • runs across operating systems: 64-bit Linux, macOS, Windows, small 32-bit ARM-based Linux devices (e.g. Raspberry Pi)
  • Data Sync: an efficient and easy way to synchronize data between your app and the cloud

Getting Started with ObjectBox for Flutter/Dart Today

ObjectBox is free to use and you can get started right now with the Flutter database on GitHub, via the docs, pub.dev, or this getting-started video tutorial, or getting-started article.

We ❤️ your Feedback

Now it’s your turn: let us know what you love, what you don’t, what do you want to see next? Share your feedback with us, or check out GitHub and up-vote the features you’d like to see next in ObjectBox.

Why database performance creates business value

Why database performance creates business value

“Why does database performance matter?” “What is the business value of database speed?” “Why should I care about the performance of a database?”

Why database performance matters in a nutshell

fast-database-business-decisions

Make Faster Decisions

Database speed is the key to compute more data faster and make data-based decisions quickly. Faster decision-making drives business value.

fast-database-speed-saves-costs

Save Resources & Costs

Database speed translates into resource efficiency. Saving resources (like battery, CPU, memory) saves money and reduces waste.

improve-ux-response-times-database

Better UX & Response Rates

Database speed affects end user response rates significantly – smooth and fast user experiences keep people happy and more productive.

As a developer, it seems clear that database performance matters. At the very least, a fast database that gives you out-of-the-box speed saves time and nerves during development. Any piece of the tech stack that works super-fast makes a developer’s job easier. But there is more to it. Learn, why and how database performance impacts business value and get ideas on how to quantify this for your business case.

Data should be available when need where needed

We all dream of a future transformed by data. Cars that drive themselves to be repaired before a failure occurs. Fridges that are restocked while we are at work. Reducing resource waste to an absolute minimum. Building sustainable cities and communities.[1] It is truly amazing what is possible today…

database performance business value

Then reality hits: Before you can implement amazing solutions to make the world a better place for everyone, someone needs to solve the technical challenges, including hidden requirements. For example: you need the necessary data, and you need it available when needed where needed. This often isn’t that simple. Data persistence, database speed, and data synchronization are typical non-functional or “hidden” requirements. These are prerequisite technologies to allow the application to access, process and possibly depict the data required to answer a request (from another application or from a user), and thus enable the functionalities /  features. All in all, this is a pretty fundamental requirement. And it pays off to build your app on top of a solid foundation. Because, if you built your application on a solid foundation, every feature you dream up, no matter when,  and any next feature will be easier and faster to implement. 

Functional and non-functional requirements – the hidden challenges of your IoT project

IoT project hidden challenge

While you need data in any application, most often no one will write down where and how to handle it  as a user story or requirement. As opposed to features, e.g. “being able to search for names in the address book”, data persistence, database speed, and often even data synchronization are “hidden requirements”. Data is just expected to be available where needed when needed. Whether  the data you need really will be available when you need it, depends strongly on the database the application is using and and where this database runs. On top, the mechanisms you employ to exchange data between different devices (end devices, servers, ….) matter.

Hidden requirements are one of the major reasons why the Industry 4.0 dream is still in many respects a dream and not a reality – in Europe at least. Despite it being a topic for more than 10 years. [2]

Database performance 

What is a database?

A database is a piece of software that allows the storage and systematic use of digital information. A database typically allows developers to store, access, search, update, query, and otherwise manipulate data in the database via a developer language or API. These types of operations are done within an application, in the background, typically hidden from end users. Most applications need a database as part of their technology stack.

What is database performance?

We like and therefore use the following definition from Craig Mullins (2002): “Database performance can be defined as the optimization of resource use to increase throughput and minimize contention, enabling the largest possible workload to be processed.” [3]

Why does it matter if the database runs on the edge or in the cloud?

An edge database holds data on the (end) devices, where the data is used – and typically additionally sends some parts of the data to a central place like an on-premise server or the cloud. As opposed to this, a server / cloud-based database holds all data on the server / in the cloud. Where the data sits, determines from where, when and how it can be accessed. If all data is on a central server or the cloud, the prerequisite to accessing this data is a working network connection.

Online

Offline

It follows that edge applications are based upon a distributed computing paradigm, allowing edge devices to be autonomous. On the other hand, cloud-based applications are based on the centralized computing paradigm, where one central instance is in charge, with all other devices being dependent upon this central instance. This significantly affects the response time of the application, the availability of the application, and last not least the bandwidth needed for the application, which also translates into cloud costs.

database performance business value

Location matters: while a fast database gives you fast response times, if the database sits in the cloud and needs to be called from edge devices, you need to factor in  the duration it takes to request the data and get a response. And with any networking you cannot guarantee response times or ensure it is always available. While this is not the database performance itself, it highly affects application performance. 

The impact of database performance on your business

Database performance matters. Whether your solution needs the speed, because of the necessity to re-act in (near) realtime, or to keep your users (customers, employees, …) happy, productive, buying, or just to save costs for stronger edge hardware and the cloud. “Considering that even a single moment of latency or downtime can cost companies thousands of dollars, the speed advantages of edge computing cannot be overlooked.” [4]

The necessity of database speed for mission-critical, security relevant, (near) real-time functionalities 

If you need near real time functionalities, every piece in the tech stack matters, but the database has a particularly strong impact on the response rates of your application. Consider autonomous driving, healthcare and security applications, or IIoT solutions for production lines: Any application supporting such a scenario needs to respond reliably with speed. “This is not the same as a lag in loading your favorite cat pictures. A lag in a moving vehicle scenario is a matter of life and death.” [5]

Accordingly, if end devices like cars, smartphones, health trackers, machines on the factory floor are involved, a purely cloud-based application is not an option. Data needs to be stored and used on the devices directly. Thus, an edge database is necessary. Ideally, an extremely fast one.

Examples of use cases with a need for database speed

Anything running on a car really needs to be highly ressource efficient and fast. Ressources on the car are highly limited and database speed translates into ressource-efficency. Autonomous driving capabilities are a special case requiring significant compute power to run the algorithms in real-time within the control unit of the car. As can be easily deducted from first-hand driving experience, during this kind of constant information processing and instantaneous decision making, every fraction of a millisecond counts. Information processing speed and reliability (guaranteed QoS parameters)  is of the essence for driver assistance and autonomous driving.

Moving to a purely monetary example, let’s consider roadside tolling. In roadside tolling, the edge devices on the side of the road need to process the information from a moving vehicle in order to identify the car, bill according to usage, and detect violators. Ideally, it even informs the car owner of the result. As the car is constantly moving and can be going fast, all of this needs to happen in a very short amount of time. A super fast database lookup on the edge is key to avoid money loss and deliver good customer service. 

For a final example,  let us look at an Industrial IoT (IIoT) application: Additive manufacturing. 3D printers use layering techniques with a variety of materials to quickly create custom designed parts. During the layering process, the controller needs to quickly and efficiently incorporate small changes in the environment (e.g. an increase in temperature) to ensure quality and accuracy of the part. Faster and more precise manufacturing is currently limited by the I/O throughput. With a fast database, the I/O throughput is higher, allowing for more complex and finite production.

In short: A superfast database is not a nice to-have, it is a must-have. The database speed a database brings out-of-the-box is critical for such an application.

 

The impact of database speed on Sales, Conversions, Retention (or at least, nerves) 

There is a reason Google forces companies to optimize their websites and mobile applications for performance: There is a wealth of research and evidence that suggests response rates of websites and mobile applications impact user behavior significantly.[6] Even more, there are several studies providing evidence that response rates impact actual buying behavior. [7] While there is less research on other digital applications like e.g. a desktop app or workplace software, some studies have shown that needing to work with slow applications decreases employee satisfaction and productivity. [8]

The impact of database speed on battery, CPU, hardware and related resources

Another hidden requirement typically is resource-efficiency with regards to CPU, RAM, Disc space and battery / electricity. For any application running in the cloud, these requirements are balanced in the backend as the cloud scales vertically. It “only” adds to cloud costs (and is a waste of energy – not to mention all the infrastructure / hardware enabling that waste). 

On the edge, you typically work with restricted devices, meaning you can only use the devices’ resources, which can be pretty limited. Therefore, inefficient applications can push a device to its limits, leading to e.g. slow response rates, crashes, and battery drain. Security is a very necessary cross-the-stack functionality that often impacts performance. While data that stays on the edge is challenging to hack, edge data needs to be protected just like data in the cloud.

How database performance impacts the business value of your IoT application

All applications on one device share the available hardware capabilities; resource allocation is managed by the operating system. Accordingly, the more resources an application or the database uses, the less resources are available for other uses. The faster a database executes its operations, the less CPU it uses, the less battery / electricity, and typically also memory. In practice that means there are more resources available on the device to run e.g. Edge AI or Edge ML applications.

database

From a business value perspective that means:

  • You can save on hardware costs (CPU, RAM, Disc, Memory, …): either do more on existing / chosen hardware, upgrade hardware later or choose smaller and thus less expensive hardware. 
  • You can save on energy and cloud costs: The more efficient, the less electricity, the less cloud costs. This can add up tremendously as projects scale.
  • You can add more features, deliver more functionalities, make your application more secure within a given environment. 
  • You can deliver a smooth, fast user experience, enabling applications that deliver in near-realtime. 

    In sum, it clearly impacts the cost structure and value you can deliver.

database performance business value

Database performance impacts business value, directly and indirectly

As projects scale in size and scope, hidden requirements like database performance often become clear. At scale, small issues like delayed data, or data volumes, become big headaches. Ideally, these sorts of requirements would be at the heart of the design stage of any project – and budgeted for at the beginning. The choice of database clearly has a huge impact on the business success of IoT applications.

[1] See https://www.weforum.org/agenda/2018/01/effect-technology-sustainability-sdgs-internet-things-iot/ for IoT impact on Sustainable Development Goals (SDG)
[2] https://restart-project.eu/much-know-industry-4-0/
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=2ahUKEwiGidSA6trnAhVQY8AKHTpSDUIQFjAMegQICBAB&url=https%3A%2F%2Fwww.mdpi.com%2F2076-3387%2F9%2F3%2F71%2Fpdf&usg=AOvVaw3cx44OOMfNzJ_BJlCG8Gfj
[3] Database Administration: The Complete Guide to Practices and Procedures By Craig Mullins 2002
[4] https://www.vxchnge.com/blog/the-5-best-benefits-of-edge-computing
[5] https://www.zdnet.com/article/why-autonomous-vehicles-will-rely-on-edge-computing-and-not-the-cloud/
[6] https://developers.google.com/web/fundamentals/performance/why-performance-matters https://www.thinkwithgoogle.com/intl/en-154/insights-inspiration/research-data/need-mobile-speed-how-mobile-latency-impacts-publisher-revenue/
https://www.machmetrics.com/speed-blog/how-does-page-load-time-affect-your-site-revenue
https://datadome.co/bot-management-protection/website-performance-how-to-increase-your-business-by-blocking-bots/
[7] https://developers.google.com/web/fundamentals/performance/why-performance-matters
https://www.thinkwithgoogle.com/intl/en-154/insights-inspiration/research-data/need-mobile-speed-how-mobile-latency-impacts-publisher-revenue/
https://www.machmetrics.com/speed-blog/how-does-page-load-time-affect-your-site-revenue
https://datadome.co/bot-management-protection/website-performance-how-to-increase-your-business-by-blocking-bots/
[8] https://drum.lib.umd.edu/handle/1903/1233
https://www.tandfonline.com/doi/abs/10.1080/01449290500196963

 

Flutter databases –  hive, ObjectBox, sqflite (+ Drift, floor)

Flutter databases – hive, ObjectBox, sqflite (+ Drift, floor)

Flutter, the renowned cross-platform mobile framework, has been gaining immense popularity among developers worldwide. As the Flutter community expands, the demand for efficient Flutter databases is also increasing. Developers now have access to a range of Flutter database options that cater to various needs and preferences.

In this article, we’ll focus specifically on local storage solutions, as these are essential for enabling offline functionality, improving performance, ensuring data persistence, enhancing data privacy and security, and supporting edge computing capabilities. Furthermore, local data storage is needed to promote sustainability. Let’s dive into the current local database landscape for Flutter and compare the most popular options.

Flutter databases / Flutter Dart data persistence

While the database market is huge and dynamic,  there are only few options to choose from if you are a Flutter / Dart app developer. Before we dive into the Flutter database options, advantages and disadvantages, we’re taking a very quick look at databases to make sure, we share a common ground. 

What is a database?

A database is a piece of software that allows the storage and systematic use of digital information, in other words: data persistence. As opposed to mere caching, data is reliably stored and available to work with unless actively deleted. A database typically allows developers to store, access, search, update, query, and otherwise manipulate data in the database via a developer language or API. These types of operations are done within an application, in the background, typically hidden from end users. Many applications need a database as part of their technology stack. The most typical database operations are CRUD: Create, Read, Update, Delete.

What are the major types of databases?

There are many types of databases. For our purpose, the most important differentiations are non-relational (NoSQL) versus relational databases (SQL), cloud databases versus edge databases, and maybe embedded versus in-memory. However, databases can be further distinguished by additional criteria e.g. the data types they support, or the way they scale – and definitions can vary.

What is an ORM?

An Object relational Mapper (ORM) is not a database. We’re bringing this up mainly, because we see it confused often. It is a layer that sits on top of a database and makes it easier to use. This is typically especially relevant when the database is a relational database (SQL) and the programming language used is object-oriented. As noted above, Dart is an object-oriuented programming language.

The Flutter local data persistence landscape

There are several Flutter databases that provide offline support, offering the ability to store and access data locally even without an internet connection. Here are some of the notable options:

  • Hive is a lightweight key-value database written in Dart for Flutter applications, inspired by Bitcask.
  • ObjectBox DB is a highly performant lightweight NoSQL database with an integrated Data Sync. It stores objects.
  • sqflite is a wrapper around SQLite, which is a relational database without direct support for Dart objects. 
  • Drift is a reactive persistence library for Flutter and Dart, built ontop of SQLite. 
  • Floor is another ORM on top of SQLite.

 

What is the best offline Flutter Dart database?

This of course depends… Make up your own mind with the following comparison matrix as a starting point. Note: With very few options to choose from, the following overview is sometimes a bit like comparing apples 🍎 and pears 🍐.

Name Description Primary Model Language License Data Sync
Hive Lightweight key-value database NoSQL Dart Apache 2.0
ObjectBox Lightweight NoSQL database with integrated Data Sync NoSQL Dart Bindings are Apache 2.0
Drift ORM on top of SQLite relational SQL SQLite is public domain, Drift is MIT
Floor ORM on top of SQLite relational SQL SQLite is public domain, Floor is Apache 2.0
sqflite SQLite plugin for Flutter relational SQL SQLite is public domain, sqflite lib is MIT
<body> <p>Diese Seite verwendet Frames. Frames werden von Ihrem Browser aber nicht unterstützt.</p> </body>

Flutter Database performance benchmarks

As with any benchmark, you need to take a look at the details. We take benchmarking very serious and strive to get accurate results. Therefore, we also always open source the benchmarking code and encourage you to check it out. If you note anything that does not even out in your oppinion, do let us know. We have a long history of updating and improving our benchmarks continually and are happy to take any recommendations.

Performance Benchmark Test Setup

We used an Android 10 device with a Kirin 980 CPU to run the benchmarks as a Flutter app. The app executed all operations (ops) in batches of 10.000 objects. Each batch formed a single transaction. We ran each test 50 times. The results you see in the diagram are averages across all runs. We set it up that way to ensure that neither the Virtual Machine warmup during the first run nor the garbage collections affect the overall result significantly. 

Flutter Databases CRUD Performance Results

Summary of the Flutter Dart DB Benchmarks

Hive and ObjectBox clearly outperform sqflite across all CRUD operations. The results show ObjectBox performing with up to 70 times the speedup for create and update operations. With regards to comparing Hive and ObjectBox, the results vary more. Hive can be faster at reading objects than ObjectBox. However, strictly speaking it’s not a fair comparison, because in Hive, the high read numbers result from Dart objects already cached in memory. If the objects are fetched using the async API from disk, the numbers drop by factor 1000.

Drift and Floor were not part of the benchmarking as they are ORMs. However, it is very likely they will perform similarly to sqflite, reflecting primarily the performance of SQLite.

Flutter Data persistence – Conclusion

Recently, the Flutter database landscape has experienced significant growth and diversification. With Flutter’s increasing popularity, developers now have a number of database options available. In this article, we focused on the best local databases, comparing their features in a comprehensive matrix, and showcasing performance benchmarks. In the end, the best choice depends on the specific needs of each project. The Flutter database landscape in 2023 is a thriving ecosystem, continuously evolving to meet the changing needs of Flutter app development. One upcoming change that we can see is the rise of vector databases for AI. So, we encourage you to keep an eye on the lively market of Flutter databases not to miss any important updates.

If you want to get started learning how to use a database, we suggest you check out this video tutorial series that teaches you how to build a Flutter app with ObjectBox from scratch.

 

Firebase alternatives for Data Storage and Data Sync

Firebase alternatives for Data Storage and Data Sync

Data Sync is a typical recurring and typically non-trivial developer challenge. Synchronizing data in offline/online settings, like for example across eventually connected devices, is simply hard. While JSON / REST is great, building Data Sync yourself is time-consuming, risky, and typically considered no fun. Therefore, today, we take a look at the out-of-the-box Data Sync market. If you are rather interested learning about Data Sync in general, check out this article about why data sync technology is more necessary than ever.

Introduction

One of the most well-known Data Sync solutions is Firebase. However, Firebase is purely cloud based and offers no support for local data storage ( as in “data persistence above caching”) and therefore offline usage. With a huge shift happening in computing from the cloud to the edge, offline-first approaches and Edge Computing are getting more and more important. Therefore, we’ve recently taken a comprehensive look at mobile database and edge database offerings on the market. But what options do Mobile and IoT developers working on the edge have for out-of-the-box Data Sync solutions? Very few. While there are more and more cloud-based Firebase alternatives springing up nearly daily (e.g. appwrite and supabase) forcing the user into a centralized cloud setup, there is almost nothing that supports offline Data Sync and / or persistent local data storage. As our focus is on offline / edge Data Sync and local storage, in the following we add all edge / offline Data Sync solutions we know of, but spare you the wealth of cloud options only adding the established ones.

Firebase

Firebase is a cloud backend service ((Mobile) Backend as a Service ((M)BaaS)) that enables developers to build mobile or web applications without needing to take care of the backend. This includes the data synchronization, scalability, network, infrastructure challenges etc. Indeed, Firebase, today, offers many different services (e.g. analytics, crashlytics) and goes well beyond Data Sync. We are looking at Firebase from the Data Sync perspective only. Firebase was one of the first Data Sync solutions available on the market together with Parse and Couchbase, which all started in 2011 (Couchbase through a merger of CouchOne and Membase). In 2014, Firebase was acquired by Google. Incidentally, the same year Parse was acquired by Facebook to be subsequently shut down, and Couchbase raised significant funding. All three are still in use today. 

Firebase Pros and Cons

In the following, we will first look at the advantages and disadvantages of Firebase. Then, we will compare Firebase with Firebase alternatives like Couchbase, Parse and ObjectBox in a comprehensive matrix.  

Firebase Advantages ++

Firebase Disadvantages —

Cloud based Purely cloud based
Google: large team that supports and maintains it; very low risk of the company failing; however, Google has a reputation of discontinuing products / services, so there is no guarantee Google: vendor lock-in (no migration tools prevents you from making your app portable), you cannot access your data as it is hosted on the Firebase server
Backend as a service (ease of use) Less flexibility: You cannot optimize the backend to match your app’s needs

The Firebase Realtime Database has its own advantages:

  • hosted, powered by Google
  • for pure online use cases rather fast
  • great if you do not have a strong DB background

The Firebase Realtime Database has its own drawbacks:

  • the whole DB is a huge JSON file
  • limited querying capabilities
  • no way to efficiently filter data
  • Easily disorganized, hard to navigate and search
Pay as you go, price scales with usage Cost insecurities, hard to impossible to predict
Less iOS support (stronger focus on Android) Less iOS support (stronger focus on Android)
Doesn’t work in countries that don’t allow Google
User privacy concerns***

 

*** “Firebase has been claimed to be used by Google to track users without their knowledge. On July 14, 2020, a lawsuit was filed accusing Google of (…) logging what the users are looking at in many types of apps, despite the user following Google’s own instructions to turn off the web and app activity collected by the company.” (https://en.wikipedia.org/wiki/Firebase)anced settings.

Firebase Advantages ++

Cloud based
– Google: large team that supports and maintains it; very low risk of the company failing; however, Google has a reputation of discontinuing products / services, so there is no guarantee
– Backend as a service (ease of use)
– The Firebase Real-time Database has its own advantages:
– Pay as you go, price scales with usage
– Less iOS support (stronger focus on Android)

Firebase Disadvantages —

– Purely cloud based
Google: vendor lock-in (no migration tools prevents you from making your app portable), you cannot access your data as it is hosted on the Firebase server
– Less flexibility: You cannot optimize the backend to match your app’s needs
The Firebase Real-time Database has its own drawbacks
the whole DB is a huge JSON file
limited querying capabilities
no way to efficiently filter data
Easily disorganized, hard to navigate and search
– Cost insecurities, hard – impossible to predict
– Less iOS support (stronger focus on Android)
– Doesn’t work in the countries that don’t allow Google
User privacy concerns: “Firebase has been claimed to be used by Google to track users without their knowledge. On July 14, 2020, a lawsuit was filed accusing Google of (…) logging what the users are looking at in many types of apps, despite the user following Google’s own instructions to turn off the web and app activity collected by the company.” (https://en.wikipedia.org/wiki/Firebase)

Firebase alternatives: A look at out-of-the-box data sync solutions

The majority of offerings for developers that handle Data Sync as defined here, are cloud-based and fall into the category of BaaS (can also be MBaaS (Mobile Backend as a Service) or PaaS (platform as a Service) or DBaaS (Database as a Service). This means that data synchronisation is only a specific part of the whole offering. 

Data Sync Solution comparison matrix – Firebase and its alternatives

Solution name Company Category Data Sync IoT / Mobile Database Type of DB Cloud OS / Platforms Languages License
Cloudant Sync
IBM (Cloudant was acquired in 2014) DBaaS
(Cloud DB and Cloud Sync)
Two-way
cloud data replication (called “sync”)
IoT
& Mobile
Cloud
database based on Couch DB
NoSQL;
distributed JSON document database
Cloud-based
replication to and from on-device data (CouchDB <> cloud service)
hosted
service
C#,
Java, JavaScript, Objective-C, PHP, Ruby
Proprietary
(CouchDB is Apache 2.0 and they integrate with several open source libraries)
Couchbase server &  Sync Gateway
Couchbase (a merger of Couch One and Membase) Cloud
DB and Cloud Sync
Sync
needs a Couchbase Server
IoT
& Mobile
Edge:
Couchbase Lite; Server: Couchbase
NoSQL;
document database
Always
needs Couchbase Server (originally Membase)
mainly
used as hosted service;
iOS, Android, .NET (Desktop/Server), .NET UWP, Xamarin
Swift,
Objective-C, Java (Android), Java (Non-Android), Kotlin, C#, JavaScript, C
Apache
2.0, delayed open source
Firebase**
Google (Firebase was acquired by Google in 2014) BaaS
(Cloud)
Cloud
Sync via Google servers
Mobile Cloud:
Firebase Realtime Database; Edge: Caching only (Firestore)
Document
store
hosted
only
APIS
for iOS & Android
JavaScript API
RESTful HTTP API
Java
JavaScript
Objective-C
proprietory
Mongo Realm Sync
MongoDB
(Realm was acquired in 2019)
Cloud
DB and Cloud Sync
Sync
(in Alpha); only via Mongo Cloud
IoT
& Mobile
Cloud:
MongoDB, Edge: Mongo Realm
MongoDB:
NoSQL document store; RealmDB: Embedded NoSQL DB
hosted
service
MongoDB:
Linux, OS X, Solaris, Windows
Mongo Realm DB:
Android, iOS
20+
languages, e.g. Java, C, C#, C++
Mongo
DB changed its license from open source (GNU) to MongoDB Inc.’s Server Side
Public License (SSPL) in 2018.
ObjectBox
Sync
ObjectBox DB
and Sync

Offline
Sync, on-premise Sync, Cloud Sync

p2p Sync is planned

IoT
& Mobile
ObjectBox Object-oriented
embedded NoSQL DB
Self-hosted
/ on-premise; hosted service upon request only
iOS,
Android, Linux, Windows, MacOS, any POSIX-system
C,
C++
Java
Kotlin
Swift
Go
Flutter / Dart
Python
DB:
Open source bindings, Apache 2.0, proprietary core
Parse
Originally
Parse, acquired by Facebook, closed down and open sourced, unmaintained
MBaaS
(Cloud)
Cloud
Sync, self-hosted or via a provider that offers Parse hosting
Mobile Both,
PostgreSQL* and MongoDB, can be used as a database for Parse
MongoDB:
NoSQL document store; PostgreSQL:
Only
Cloud, only self-hosted or via a provider that offers Parse hosting
Server: REST
API lets you interact with Parse Server from anything that can send an HTTP
request
open
source, BSD
Syncstudio
HandApps Cloud-based
sync between SQLite and MS SQL Server based in the MS Sync Framework
Sync Mobile Edge:
SQLite or MSSQL (including LocalDB or Express); Server:
Microsoft SQL
relational
/ SQL
SQL
Server; Sync / replication works via cloud only
Android
Java, Basic4Android, Windows Forms, UWP, Windows Mobile, Xamarin
proprietory,
4 licenses available: Community/Free, Subscription, Perpetual and Royalty
Free
Zumero
Zumero
LLC
Cloud-based
replication of SQL data for Mobile
Sync Mobile Edge:
SQLite; Server: Microsoft SQL
relational
/ SQL
SQL
Server; Sync / replication works via cloud only
Mobile
only (iOS, Android, Xamarin, PhoneGap)

proprietory,
annual license scaling with the number of devices

 

 

 

 

Notes: Microsoft Sync Framework (renamed Sync Framework Toolkit at some point) is a legacy open source product which MS no longer supports

* PostgreSQL vs Postgres
** There are many Cloud Sync alternatives to Firebase, we added the more prominent options and any service that also serves Edge Computing

Data Sync is no standardized term and though it seems to be in use by many big companies and most dvelopers will have a notion of what it is, the devil is in the details. So, we might have missed an important solution or taken an angle someone else would not agree with. Please feel free to let us know what to improve.

👉 Want to save this info for later? Watch the Firebase alternatives matrix on GitHub to find it easily wherever you need it.

ObjectBox DB and Sync – designed to keep data up to date across time and space

ObjectBox is a high performance NoSQL fully ACID-compliant edge database built from scratch for efficient data on and across restricted and occasionally connected devices, taking care of keeping data in sync reliably. ObjectBox developer tools are easy to use, quick to implement, and optimized for high-performance and frugal resource-use on edge devices running mobile, desktop, server, and IoT applications. ObjectBox helps developers to focus on what they love and build great applications without needing to take care of the boilerplate code for resilient connectivity, synchronizing data, and tedious DB optimizations. This cuts down initial implementation efforts, ongoing maintenance efforts, undesired problems, and data loss – therefore reducing costs and time to market tremendously. We are dedicated to bring joy and delight to Mobile and IoT application developers.