Firebase alternatives for Data Storage and Data Sync

Firebase alternatives for Data Storage and Data Sync

Data Sync is a typical recurring and typically non-trivial developer challenge. Synchronizing data in offline/online settings, like for example across eventually connected devices, is simply hard. While JSON / REST is great, building Data Sync yourself is time-consuming, risky, and typically considered no fun. Therefore, today, we take a look at the out-of-the-box Data Sync market. If you are rather interested learning about Data Sync in general, check out this article about why data sync technology is more necessary than ever.

Introduction

One of the most well-known Data Sync solutions is Firebase. However, Firebase is purely cloud based and offers no support for local data storage ( as in “data persistence above caching”) and therefore offline usage. With a huge shift happening in computing from the cloud to the edge, offline-first approaches and Edge Computing are getting more and more important. Therefore, we’ve recently taken a comprehensive look at mobile database and edge database offerings on the market. But what options do Mobile and IoT developers working on the edge have for out-of-the-box Data Sync solutions? Very few. While there are more and more cloud-based Firebase alternatives springing up nearly daily (e.g. appwrite and supabase) forcing the user into a centralized cloud setup, there is almost nothing that supports offline Data Sync and / or persistent local data storage. As our focus is on offline / edge Data Sync and local storage, in the following we add all edge / offline Data Sync solutions we know of, but spare you the wealth of cloud options only adding the established ones.

Firebase

Firebase is a cloud backend service ((Mobile) Backend as a Service ((M)BaaS)) that enables developers to build mobile or web applications without needing to take care of the backend. This includes the data synchronization, scalability, network, infrastructure challenges etc. Indeed, Firebase, today, offers many different services (e.g. analytics, crashlytics) and goes well beyond Data Sync. We are looking at Firebase from the Data Sync perspective only. Firebase was one of the first Data Sync solutions available on the market together with Parse and Couchbase, which all started in 2011 (Couchbase through a merger of CouchOne and Membase). In 2014, Firebase was acquired by Google. Incidentally, the same year Parse was acquired by Facebook to be subsequently shut down, and Couchbase raised significant funding. All three are still in use today. 

Firebase Pros and Cons

In the following, we will first look at the advantages and disadvantages of Firebase. Then, we will compare Firebase with Firebase alternatives like Couchbase, Parse and ObjectBox in a comprehensive matrix.  

Firebase Advantages ++

Firebase Disadvantages —

Cloud based Purely cloud based
Google: large team that supports and maintains it; very low risk of the company failing; however, Google has a reputation of discontinuing products / services, so there is no guarantee Google: vendor lock-in (no migration tools prevents you from making your app portable), you cannot access your data as it is hosted on the Firebase server
Backend as a service (ease of use) Less flexibility: You cannot optimize the backend to match your app’s needs

The Firebase Realtime Database has its own advantages:

  • hosted, powered by Google
  • for pure online use cases rather fast
  • great if you do not have a strong DB background

The Firebase Realtime Database has its own drawbacks:

  • the whole DB is a huge JSON file
  • limited querying capabilities
  • no way to efficiently filter data
  • Easily disorganized, hard to navigate and search
Pay as you go, price scales with usage Cost insecurities, hard to impossible to predict
Less iOS support (stronger focus on Android) Less iOS support (stronger focus on Android)
Doesn’t work in countries that don’t allow Google
User privacy concerns***

 

*** “Firebase has been claimed to be used by Google to track users without their knowledge. On July 14, 2020, a lawsuit was filed accusing Google of (…) logging what the users are looking at in many types of apps, despite the user following Google’s own instructions to turn off the web and app activity collected by the company.” (https://en.wikipedia.org/wiki/Firebase)anced settings.

Firebase Advantages ++

Cloud based
– Google: large team that supports and maintains it; very low risk of the company failing; however, Google has a reputation of discontinuing products / services, so there is no guarantee
– Backend as a service (ease of use)
– The Firebase Real-time Database has its own advantages:
– Pay as you go, price scales with usage
– Less iOS support (stronger focus on Android)

Firebase Disadvantages —

– Purely cloud based
Google: vendor lock-in (no migration tools prevents you from making your app portable), you cannot access your data as it is hosted on the Firebase server
– Less flexibility: You cannot optimize the backend to match your app’s needs
The Firebase Real-time Database has its own drawbacks
the whole DB is a huge JSON file
limited querying capabilities
no way to efficiently filter data
Easily disorganized, hard to navigate and search
– Cost insecurities, hard – impossible to predict
– Less iOS support (stronger focus on Android)
– Doesn’t work in the countries that don’t allow Google
User privacy concerns: “Firebase has been claimed to be used by Google to track users without their knowledge. On July 14, 2020, a lawsuit was filed accusing Google of (…) logging what the users are looking at in many types of apps, despite the user following Google’s own instructions to turn off the web and app activity collected by the company.” (https://en.wikipedia.org/wiki/Firebase)

Firebase alternatives: A look at out-of-the-box data sync solutions

The majority of offerings for developers that handle Data Sync as defined here, are cloud-based and fall into the category of BaaS (can also be MBaaS (Mobile Backend as a Service) or PaaS (platform as a Service) or DBaaS (Database as a Service). This means that data synchronisation is only a specific part of the whole offering. 

Data Sync Solution comparison matrix – Firebase and its alternatives

Solution name Company Category Data Sync IoT / Mobile Database Type of DB Cloud OS / Platforms Languages License
Cloudant Sync
IBM (Cloudant was acquired in 2014) DBaaS
(Cloud DB and Cloud Sync)
Two-way
cloud data replication (called “sync”)
IoT
& Mobile
Cloud
database based on Couch DB
NoSQL;
distributed JSON document database
Cloud-based
replication to and from on-device data (CouchDB <> cloud service)
hosted
service
C#,
Java, JavaScript, Objective-C, PHP, Ruby
Proprietary
(CouchDB is Apache 2.0 and they integrate with several open source libraries)
Couchbase server &  Sync Gateway
Couchbase (a merger of Couch One and Membase) Cloud
DB and Cloud Sync
Sync
needs a Couchbase Server
IoT
& Mobile
Edge:
Couchbase Lite; Server: Couchbase
NoSQL;
document database
Always
needs Couchbase Server (originally Membase)
mainly
used as hosted service;
iOS, Android, .NET (Desktop/Server), .NET UWP, Xamarin
Swift,
Objective-C, Java (Android), Java (Non-Android), Kotlin, C#, JavaScript, C
Apache
2.0, delayed open source
Firebase**
Google (Firebase was acquired by Google in 2014) BaaS
(Cloud)
Cloud
Sync via Google servers
Mobile Cloud:
Firebase Realtime Database; Edge: Caching only (Firestore)
Document
store
hosted
only
APIS
for iOS & Android
JavaScript API
RESTful HTTP API
Java
JavaScript
Objective-C
proprietory
Mongo Realm Sync
MongoDB
(Realm was acquired in 2019)
Cloud
DB and Cloud Sync
Sync
(in Alpha); only via Mongo Cloud
IoT
& Mobile
Cloud:
MongoDB, Edge: Mongo Realm
MongoDB:
NoSQL document store; RealmDB: Embedded NoSQL DB
hosted
service
MongoDB:
Linux, OS X, Solaris, Windows
Mongo Realm DB:
Android, iOS
20+
languages, e.g. Java, C, C#, C++
Mongo
DB changed its license from open source (GNU) to MongoDB Inc.’s Server Side
Public License (SSPL) in 2018.
ObjectBox
Sync
ObjectBox DB
and Sync

Offline
Sync, on-premise Sync, Cloud Sync

p2p Sync is planned

IoT
& Mobile
ObjectBox Object-oriented
embedded NoSQL DB
Self-hosted
/ on-premise; hosted service upon request only
iOS,
Android, Linux, Windows, MacOS, any POSIX-system
C,
C++
Java
Kotlin
Swift
Go
Flutter / Dart
Python
DB:
Open source bindings, Apache 2.0, proprietary core
Parse
Originally
Parse, acquired by Facebook, closed down and open sourced, unmaintained
MBaaS
(Cloud)
Cloud
Sync, self-hosted or via a provider that offers Parse hosting
Mobile Both,
PostgreSQL* and MongoDB, can be used as a database for Parse
MongoDB:
NoSQL document store; PostgreSQL:
Only
Cloud, only self-hosted or via a provider that offers Parse hosting
Server: REST
API lets you interact with Parse Server from anything that can send an HTTP
request
open
source, BSD
Syncstudio
HandApps Cloud-based
sync between SQLite and MS SQL Server based in the MS Sync Framework
Sync Mobile Edge:
SQLite or MSSQL (including LocalDB or Express); Server:
Microsoft SQL
relational
/ SQL
SQL
Server; Sync / replication works via cloud only
Android
Java, Basic4Android, Windows Forms, UWP, Windows Mobile, Xamarin
proprietory,
4 licenses available: Community/Free, Subscription, Perpetual and Royalty
Free
Zumero
Zumero
LLC
Cloud-based
replication of SQL data for Mobile
Sync Mobile Edge:
SQLite; Server: Microsoft SQL
relational
/ SQL
SQL
Server; Sync / replication works via cloud only
Mobile
only (iOS, Android, Xamarin, PhoneGap)

proprietory,
annual license scaling with the number of devices

 

 

 

 

Notes: Microsoft Sync Framework (renamed Sync Framework Toolkit at some point) is a legacy open source product which MS no longer supports

* PostgreSQL vs Postgres
** There are many Cloud Sync alternatives to Firebase, we added the more prominent options and any service that also serves Edge Computing

Data Sync is no standardized term and though it seems to be in use by many big companies and most dvelopers will have a notion of what it is, the devil is in the details. So, we might have missed an important solution or taken an angle someone else would not agree with. Please feel free to let us know what to improve.

👉 Want to save this info for later? Watch the Firebase alternatives matrix on GitHub to find it easily wherever you need it.

ObjectBox DB and Sync – designed to keep data up to date across time and space

ObjectBox is a high performance NoSQL fully ACID-compliant edge database built from scratch for efficient data on and across restricted and occasionally connected devices, taking care of keeping data in sync reliably. ObjectBox developer tools are easy to use, quick to implement, and optimized for high-performance and frugal resource-use on edge devices running mobile, desktop, server, and IoT applications. ObjectBox helps developers to focus on what they love and build great applications without needing to take care of the boilerplate code for resilient connectivity, synchronizing data, and tedious DB optimizations. This cuts down initial implementation efforts, ongoing maintenance efforts, undesired problems, and data loss – therefore reducing costs and time to market tremendously. We are dedicated to bring joy and delight to Mobile and IoT application developers.

How Building Green IoT Solutions on the Edge Can Help Save Energy and CO2

How Building Green IoT Solutions on the Edge Can Help Save Energy and CO2

The internet of things (IoT) has a huge potential to reduce carbon emissions, as it enables new ways of operating, living, and working [1] that are more efficient and sustainable. However, IoT’s huge and growing electricity demands are a challenge. This demand is due primarily to the transmission and storage of data in cloud data centers. [2] While data center efficiency and the use of green energy will reduce the CO2 emissions needed for this practice, it is not addressing the problem directly. [3

iot-data-cloud-energy-waste

With ObjectBox, we address this unseen and fast-growing CO2 source at the root: ObjectBox empowers edge computing, reducing the volume of data transmitted to central data storage, while at the same time, heightening data transmission and storage efficiency. [4] We’ve talked before about how edge computing is necessary for a sustainable future, below we dive into the numbers a bit deeper. TLRD: ObjectBox enables companies to cut the power consumption of their IoT applications, and thus their emissions, by 50 – 90%. For 2025, the potential impact of ObjectBox is a carbon emission reduction of 594 million metric tons (see calculations below).

How ObjectBox’ Technology Reduces Overall Data Transmission

 ObjectBox reduces data transmission in two ways: 1. ObjectBox reduces the need for data transmission, 2. ObjectBox makes data transmission more efficient. ObjectBox’ database solution allows companies to build products that store and process data on edge devices and work with that data offline (as well as online). This

Green IoT Solution

not only improves performance and customer experience, it also reduces the overall volume of data that is being sent to the cloud, and thus the energy needed to transfer the data as well as store it in the cloud. ObjectBox’ Synchronization solution makes it easy for companies to transmit only the data that needs to be transmitted through 1) selective two-way syncing and 2) differential delta syncing. Synchronizing select data reduces the energy required for unnecessarily transmitting all data to the cloud.

We have demonstrated in exemplary case studies that ObjectBox can reduce total data transmissions by 70-90%, depending on the case. There will, however, typically be value in transmitting some parts of data to a central data center (cloud); ObjectBox Sync combines efficient compression based on standard and proprietary edge compression methods to keep this data small. ObjectBox also has very little overhead. Comparing the transmission of the same data sets, ObjectBox saves 40-60% on transmission data volume through the delta syncing and compression, and thus saves equivalent CO2 emissions for data transmissions. Additional studies support these results, and have shown that moving from a centralized to a distributed data structure, saves between 32 and 93% of transmission data. [5

sync-sustainable-data-save-energy

Calculations: How Does ObjectBox Save CO2?

Physically using a device consumes little energy directly; it is the wireless cloud infrastructure in the backend (data center storage and data transmission) that is responsible for the high carbon footprint of mobile phones [6] and IoT devices. Estimates say that IoT devices will produce around 2,8 ZB of data in 2020 (or 2,823,000,000,000  GB), globally. [7] Only a small portion of that data actually gets stored and used; we chose to use a conservative estimate of 5% [8] (141,150,000,000 GB) and of that portion, 90% is transferred to the cloud [9] (127,035,000,000 GB). Transferring 1 GB of data to the cloud and storing it there costs between 3 and 7 kWh. [10] Assuming an average of 5 kWh this means a 127,035,000,000 GB multiplied by 5kWh, resulting in a total energy expenditure of 635,175,000,000 kWh. Depending on the energy generation used, CO2 emissions vary. We are using a global average of 0,475 kgCO2 / 1 kwH. [11] In total this means that there will be 301,708,125,000 KG of CO2, or roughly 301 million metric tons of CO2 produced to transfer data to the cloud and store it there in 2020. 

Projections for 2025 have data volumes as high as 79.4 ZB. [12] Following the same calculations as above, IoT devices would be responsible for 8 billion metric tons of CO2 in 2025.* We estimate that using ObjectBox can cut CO2 caused by data transmission and data centers by 50-90%, by keeping the majority of data on the device, and transmitting data efficiently. It will take time for ObjectBox to enter the market, so assuming a 10% market saturation by 2025 and an average energy reduction of 70%, using ObjectBox could cut projected CO2 emissions by 594 million metric tons in 2025.

ObjectBox is on a mission to reduce digital waste which unnecessarily burdens bandwidth infrastructure and fills cloud servers, forcing the expansion of cloud farms and in turn, contributing to the pollution of the environment. As our digital world grows, we all need to give some thought to how we should structure our digital environments to optimize and support useful, beneficial solutions, while also keeping them efficient and sustainable. 

*Of course, in that time, the technologies will all be more efficient and thus use less electricity while at the same time CO2 emissions / kWh will have dropped too. Thus, we are aware that this projection is an oversimplification of a highly complex and constantly changing system.

[1] https://www.theclimategroup.org/sites/default/files/archive/files/Smart2020Report.pdf
[2] https://www.iea.org/reports/tracking-buildings/data-centres-and-data-transmission-networks
[3]“Data centres… have eaten into any progress we made to achieving Ireland’s 40% carbon emissions reduction target.” from https://www.climatechangenews.com/2017/12/11/tsunami-data-consume-one-fifth-global-electricity-2025/
[4] https://medium.com/stanford-magazine/carbon-and-the-cloud-d6f481b79dfe
[5] https://www.researchgate.net/publication/323867714_The_carbon_footprint_of_distributed_cloud_storage
[6] https://www.resilience.org/stories/2020-01-07/the-invisible-and-growing-ecological-footprint-of-digital-technology/
[7] https://www.idc.com/getdoc.jsp?containerId=prUS45213219, https://priceonomics.com/the-iot-data-explosion-how-big-is-the-iot-data/, https://www.gartner.com/en/newsroom/press-releases/2018-11-07-gartner-identifies-top-10-strategic-iot-technologies-and-trends, https://www.iotjournaal.nl/wp-content/uploads/2017/02/white-paper-c11-738085.pdf, ObjectBox research
[8] Forrester (https://internetofthingsagenda.techtarget.com/blog/IoT-Agenda/Preventing-IoT-data-waste-with-the-intelligent-edge), Harvard BR (https://hbr.org/2017/05/whats-your-data-strategy), IBM (http://www.redbooks.ibm.com/redbooks/pdfs/sg248435.pdf), McKinsey (https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/the-internet-of-things-the-value-of-digitizing-the-physical-world)
[9] https://www.gartner.com/smarterwithgartner/what-edge-computing-means-for-infrastructure-and-operations-leaders/
[10] According to the American Council for an Energy-Efficient Economy: 5,12 kWh of electricity / GB of transferred data. According to a Carnegie Mellon University study: 7 kWh / GB. The American Council for an Energy-Efficient Economy concluded: 3.1 kWh / GB.
[11] https://www.iea.org/reports/global-energy-co2-status-report-2019/emissions
[12] https://www.idc.com/getdoc.jsp?containerId=prUS45213219