SQLite and SQLite alternatives – a comprehensive overview
SQLite and SQLite alternatives - databases for the Mobile and IoT edge
Overview of SQLite and SQLite alternatives as part of the mobile / edge database market with a comprehensive comparison matrix (last updated autumn 2024)Digitalization is still on the rise, as is the number of connected devices (from 13 billion connected IoT devices + 15 billion mobile devices operating in 2021 already). Data volumes are growing accordingly ( 3.5 quintillion bytes of data is produced daily in 2023), and centralised (typically cloud-based) computing canbot support all the current needs. This has led to a shift from the cloud to the edge.Â
Therefore, there is a renewed need for on-device databases like SQLite and SQLite alternatives to persist and manage data on edge devices. On top, due to the distributed nature of the edge, there is a need to manage data flows to / from and between edge devices. This can be done with Edge Databases that provide a Data Sync functionality (SQLite alternatives only, as SQLite doesn’t support this). Below, we’ll take a close look at SQLite and its alternatives with consideration of today’s needs.
Databases for the Edge
While being quite an established market with many players, the database market is still growing consistently and significantly. The reason is that databases are at the core of almost any digital solution, and directly impact business value and therefore never going out of fashion. With the rapid evolvements in the tech industry, however, databases evolve too. This, in turn, yields new database types and categories. We have seen the rise of NoSQL databases in the last 20 years, and more recently some novel database technologies, like graph databases and time-series databases, and vector databases.
With AI and accordingly vector databases being all the hype since 2022/2023, the database market is indeed experiencing fresh attention. Due to the speed with which AI is evolving, we’re however already leaving the “mainframe era of AI” and entering the distributed Edge AI space. With SQLite not supporting vector search and related vector database functions, this adds a new dimension to this ever-present topic. There is a need for local, on-device vector databases to support on-device AI that’s independent of an Internet connection, reliably fast, and keeps data on the device (100% private).Â
We’re expecting vector databases that run locally on a wide variety of devices (aka Edge Vector Databases) to become the next big thing, surpassing even what we have seen happening in the server vector database space. And we wouldn’t be astonished if the synchronizing of vector data is a game changer for Edge AI. Time will tell đ
Both, the shift back from a centralised towards a decentralised paradigm, and the growing number of restricted devices call for a “new type” of an established database paradigm. SQLite has been around for more than 20 years and for good reason, but the current market shift back to decentralized computing happens in a new environment with new requirements. Hence, the need for a “new” database type, based on a well-established database type: “Edge databases”. Accordingly, a need for SQLite alternatives that consider the need for decentralized data flows and AI functionalities (depending on the use case of course; after all SQLite is a great database).
What is an Edge Database?
Edge databases are a type of databases that are optimised for local data storage on restricted devices, like embedded devices, Mobile, and IoT. Because they run on-device, they need to be especially resource-efficient (e.g. with regards to battery use, CPU consumption, memory, and footprint). The term âedge databaseâ is becoming more widely-used every year, especially in the IoT industry. In IoT, the difference between cloud-based databases and ones that run locally (and therefore support Edge Computing) is crucial.
What is a Mobile Database?
We look at mobile databases as a subset of edge databases that run on mobile devices. The difference between the two terms lies mainly in the supported operating systems / types of devices. Unless Android and iOS are supported, an edge database is not really suited for the mobile device / smartphone market. In this article, we will use the term âmobile databaseâ only as âdatabase that runs locally on a mobile (edge) device and stores data on the device”. Therefore, we also refer to it as an âon-deviceâ database.
What are the advantages and disadvantages of working with SQLite?
SQLite is a relational database that is clearly the most established database suitable to run on edge devices. Moreover, it is probably the only âestablishedâ mobile database. It was designed in 2000 by Richard Hipp and has been embedded with iOS and Android since the beginning. Now letâs have a quick look at its main advantages and disadvantages:
Advantages | Â Disadvantages |
|
|
What are the SQLite alternatives?
There are a bunch of options for making your life easier, if you want to use SQLite. You can use an object abstraction on top of it, an object-Relational-Mapper (ORM), for instance greenDAO, to avoid writing lots of SQL. However, you will typically still need to learn SQL and SQLite at some point. So what you really want is a full blown database alternative, like any of these: Couchbase Lite, Interbase, LevelDB, ObjectBox, Oracle Berkeley DB, Mongo Realm, SnappyDB, SQL Anywhere, or UnQLite.
While SQLite really is designed for small devices, people do run it on the server / cloud too. Actually, any database that runs efficiently locally, will be highly efficient on big servers too, making them a sustainable lightweight choice for some scenarios. However, for server / cloud databases, there are a lot of alternatives you can use as a replacement like e.g. MySQL, MongoDB, or Cloud Firestore.
Bear in mind that, if you are looking to host your database in the cloud with apps running on small distributed devices (e.g. mobile apps, IoT apps, any apps on embedded devices etc.), there are some difficulties. Firstly, this will result in higher latency, i.e. slow response-rates. Secondly, the offline capabilities will be highly limited or absent. As a result, you might have to deal with increased networking costs, which is not only reflected in dollars, but also CO2 emissions. On top, it means all the data from all the different app users is stored in one central place. This means that any kind of data breach will affect all your and your usersâ data. Most importantly, you will likely be giving your cloud / database provider rights to that data. (Consider reading the general terms diligently). If you care about privacy and data ownership, you might therefore want to consider a local database option, as in an Edge Database. This way you can decide, possibly limit, what data you sync to a central instance (like the cloud or an on-premise server).
SQLite alternatives Comparison Matrix
To give you an overview, we have compiled a comparison table including SQLite and SQLite alternatives. In this matrix we look at databases that we believe are apt to run on edge devices. Our rule of thumb is the databasesâ ability to run on Raspberry Pi type size devices. If you’re reading this on mobile, click here to view the full matrix.
Edge Database | Short description | License / business model | Android / iOS* | Type of data stored | Central Data Sync | P2P Data Sync | Offline Sync (Edge) | Data level encryption | Flutter / Dart support | Vector Database (AI support) | Minimum Footprint size | Company |
SQLite | C programming library; probably still 90% market share in the small devices space (personal assumption) | Public domain | embedded on iOS and Android | Relational | No | No | No | No, but option to use SQLCipher to encrypt SQLite | Flutter plugins (ORMs) for SQLite, but nothing from Hwaci | No, but various early & unofficial extensions are available | < 1 MB | Hwaci |
Couchbase Mobile / Lite | Embedded / portable database with P2P and central synchronization (sync) support;Â pricing upon request;Â some restrictions apply for the free version. Secure SSL. | Partly proprietary, partly open-source, Couchbase Lite is BSL 1.1 | Android / iOS | JSON Documents / NoSQL db | Yes | Yes | No | Database encryption with SQLCipher (256-bit AES) | Unofficial Flutter plugin for Couchbase Lite Community Edition | No | < 3,5 MB | Couchbase |
InterBase ToGo / IBLite | Embeddable SQL database. | Proprietary | Android / iOS | Relational | No | No | No | 256 bit AES strength encryption | No | No | < 1 MB | Embarcadero |
LevelDB | Portable lightweight key-value store, NoSQL, no index support; benchmarks from 2011 have been removed unfortunately | New BSD | Android / iOS | Key-value pairs / NoSQL db | No | No | No | No | Unofficial client that is very badly rated | No | < 1 MB | LevelDB Team |
LiteDB | A .Net embedded NoSQL database | MIT license | Android / iOS (with Xamarin only) | NoSQL document store, fully wirtten in .Net | No | No | No | Salted AES | No | No | < 1 MB | LiteDB team |
Realm DB | Embedded object database | Apache 2.0 | Android / iOS | Object Database | deprecated | No | deprecated | Yes | Yes | No | 5 MB+ | Acquired by MongoDB in 2019, Data Sync deprecated in 2024; DB still available as open source, not maintained |
ObjectBox | NoSQL Edge Vector Database with out-of-the-box Data Sync for Mobile and IoT; fully ACID compliant; benchmarks available as open source. | Open Core (plus Apache 2.0 bindings) | Android / iOS / Linux / Windows / any POSIX | Object-oriented NoSQL edge database for high-performance on edge devices in Mobile and IoT | Yes | WIP | Yes | transport encryption; additional encryption upon request | Yes | First local vector database fo on-device Edge AI released May 2024 | < 1 MB | ObjectBox |
Oracle Database Lite | Portable with P2P and central sync support as well as support for sync with SQLite | Proprietary | Android / iOS | Relational | Yes | Yes | No | 128-bit AES Standard encrytion | No | No | < 1 MB | Oracle Corporation |
SQL Anywhere | Embedded / portable database with central snyc support with a stationary database, pricing now available here | Proprietary | Android / iOS | Relational | Yes, tied to using other SAP tech though (we believe) | No | No | AES-FIPS cipher encryption for full database or selected tables | No | No |  | SAP (originally Sybase) |
UnQLite | Portable lightweight embedded db; self-contained C library without dependency. | 2-Clause BSD | Android / iOS | Key-value pairs / JSON store / NoSQL db | No | No | No | 128-bit or 256-bit AES standard encryption | not yet; might be coming though; there was a 0.0.1 released some time ago | No | ~ 1.5 MB | Symisc systems |
extremeDB | Embedded relational database | Proprietary | iOS | In-memory relational DB, hybrid persistence | No | No | No | AES encryption | No | No | < 1 MB | McObject LLC |
redis DB | High-performance in-memory Key Value store with optional durability | Three clause BSD license, RSAL and Proprietary | No | K/V in-memory store, typically used as cache | No | No | No | TLS/SSL-based encryption can be enabled for data in motion. | Unofficial redis Dart client available | No on-device vector database, but cloud vector support | An empty instance uses ~ 3MB of memory | redislabs (the original author of redis left in 2020) |
Azure SQL Edge | Designed as a SQL database for the IoT edge; however, due to the footprint it is no Edge Database | Proprietary | No | Relational DB for IoT | No | No | No | will provide encryption | No | Not on-device | 500 MB+ | Microsoft |
If you are interested in an indication of the diffusion rate of databases, check out the following database popularity ranking: http://db-engines.com/en/ran. If you are interested to learn more about SQLite, there is a great Podcast interview with Richard Hipp that is worthwhile listening to.
Is there anything weâve missed? What do you agree and disagree with? Please share your thoughts with us via Twitter or email us on contact[at]objectbox.io.Â
Make sure to check out the ObjectBox Database & try out ObjectBox Sync. You can get started in minutes and itâs perfect if you are using an object-oriented programming language, as it empowers you to work with your objects within the database. More than 1,000,000 developers already use this Edge Database designed specifically for high performance on small, connected, embedded devices.