45.8k
Get started
Data Sync Alternatives: Offline vs. Online Solutions

Data Sync Alternatives: Offline vs. Online Solutions

by | Feb 5, 2025 | Data Sync, Edge Database, Mobile Database, ObjectBox, Open Source, SQlite

Ever waited to order or pay with a waiter holding their ordering device in the air for a signal? These moments show why offline-first Data Sync is essential. With more and more services relying on the availability of on-device apps and the IoT market projected to hit $1.1 trillion by 2026, choosing the right solution – particularly online-only or offline-first data sync – is more crucial than ever. In this blog, we discuss their differences and highlight common Data Sync alternatives.

What is Data Sync?

Data synchronization (Sync) aligns data between two or more devices to maintain consistency over time. It is an essential component in applications ranging from IoT and mobile apps to cloud computing. Challenges in data synchronization include asynchrony, conflicts, and managing data across flaky networks.

Data Sync vs. Data Replication

Data Synchronization is often confused with Data Replication. Nevertheless, they serve different purposes:

  • Data Replication: A unidirectional process (works in one direction only) that duplicates data across storage locations to ensure availability and prevent loss. It is simple but limited in its application, and efficiency, and lacks conflict management.
  • Data Synchronization: A bidirectional process that harmonizes all or a subset of data between two or more devices. It ensures consistency across devices and entails conflict resolution. It is inherently more complex but also more flexible.

Online vs Offline Solutions: Why Offline Sync Matters

Online-only synchronization solutions rely entirely on cloud infrastructure, requiring a stable internet connection to function. While these tools offer simplicity and scalability, their dependency on constant cloud connectivity brings limitations: Online Data Sync solutions cannot guarantee response rates and their speed varies depending on the network. They do not work when offline or in on-premise settings. Using an Online Sync solution often entails sharing the data and might not comply with data privacy requirements. So, do read the terms and conditions.

Offline-first solutions (offline Sync) focus on local data storage and processing, ensuring the app remains fully functional even without an internet connection. When a network is available, the app synchronizes seamlessly with a server, the cloud, or other devices as needed. These solutions are ideal for on-premise scenarios with unreliable or no internet access, mission-critical applications that must always operate, real-time and high-performance use cases, as well as situations requiring high data privacy and data security compliance.

A less discussed, but in our view also relevant point, is sustainability. While there might be exceptions depending on the use case, for most applications offline-first solutions are more resourceful and therefore more sustainable. If CO2 footprint or battery usage is of concern to you, you might want to look into offline-first Data Sync alternatives.

Now, let’s have a look at current options:

Data Sync Alternatives

(If you are on mobile, click here for a view that’s optimized for mobile)

Solution Company Type Offline Support Self-hosted Sync Decentralized Sync Database Type of DB OS/Platforms Languages Open-Source Component License Other Considerations Country
Firebase Google
 (Firebase was acquired by Google in 2014)		 Online Local cache only, no persistence, syncs when online ❌ ❌ Cloud: Firebase Realtime Database; Edge: Only caching, no DB (called Firestore) Document store iOS, Android, Web Java
JavaScript
Objective-C
Swift
Kotlin
C++
Dart
C#
Python, Go, Node.js		 ❌ proprietory Tied to Google Cloud, requires internet connectivity 🇺🇸
Supabase Supabase Online Limited ✅ ❌ Cloud DB: PostgreSQL Relational document store Primarily a cloud solution JavaScript/TypeScript
Flutter/Dart
C#
Swift
Kotlin
Python		 ✅ Apache License 2.0 Supabase is mainly designed as a SaaS, for use cases with constant connectivity 🇸🇬
ObjectBox Sync ObjectBox Offline-first ✅ ✅ In development ObjectBox Object-oriented embedded NoSQL DB Android, Linux, Ubuntu,
Windows,
macOS, iOS,
QNX, Raspbian,
any POSIX system really,
any cloud (e.g. AWS/Azure/Google Cloud),
bare metal		 C
C++
Java
Kotlin
Swift
Go
Flutter / Dart
Python		 ✅ DB: Open source bindings, Apache 2.0, closed core Highly efficient (saves CPU, Memory, battery, and bandwidth); fully offline-first, supports on-premise settings, 100% cloud optional 🇩🇪
Couchbase (Lite + Couchbase Sync Gateway) Couchbase (a merger of Couch One and Membase) Online ✅ The CE Sync is a bare minimum and typically not usable; Self-hosted Sync with Couchbase Servers is available as part of their Enterprise offering ✅ as part of the Enterprise offering; gets expensive quickly Edge: Couchbase Lite; Server: Couchbase Multi-model NoSQL document-oriented database Couchbase Lite: iOS, Android, macOS, Linux, Windows, Raspbian and Raspberry Pi OS

Couchbase Sync Gateway: Red Hat Enterprise Linux (RHEL) 9.x, Alma Linux 9.x, Rocky Linux 9.x, Ubuntu, Debian (11.x, 12.x), Windows Server 2022		 .Net
C
Go
Java
JavaScript info
Kotlin
PHP
Python
Ruby
Scala		 ✅ Couchbase Lite is available under different licenses; the open source Community Edition does not get regular updates and misses many features especially around Sync (e.g. it does not include Delta Sync making it slow and expensive) Typically requires Couchbase servers, quickly gets expensive 🇺🇸
MongoDB Realm + Atlas Device Sync MongoDB
 (Realm was acquired by MongoDB in 2019)		 Offline-First ✅ Cloud-based sync only ❌ Cloud: MongoDB, Edge: Mongo Realm DB MongoDB: NoSQL document store; RealmDB: Embedded NoSQL DB MongoDB: Linux
OS X
Solaris
Windows
Mongo Realm DB:
Android, iOS		 more than 20 languages, e.g. Java, C, C#, C++ ✅ MongoDB changed its license from open source (AGPL) to MongoDB Inc.’s Server Side Public License (SSPL) in 2018. RealmDB is open source under the Apache 2.0 License. The Data Sync was proprietary.  Deprecated (in Sep 2024); End-of-life in Sep 2025; ObjectBox offers a migration option 🇺🇸

While SQLite does not offer a sync solution out-of-the-box, various vendors have built something on top, or integrated with SQLite giving them offline persistence.

Key Considerations for Choosing a Data Sync Solution

When selecting a synchronization solution, consider:

  1. Connectivity Requirements: Will the application function in offline environments; how will it work with flaky network conditions; how is the user experience when there is intermittent connectivity?
  2. Data Privacy & Security: How critical is it to ensure sensitive data remains local? Data compliance? How important is it that data is not breached?
  3. Scalability and Performance: What are the expected data loads and network constraints? How important is speed for the users? Is there any need to guarantee QoS parameters? How much will the cloud and networking costs be?
  4. Conflict Resolution: How does the solution handle data conflicts?
  5. Delta Sync: Does the solution always synchronize all data or only changes (data delta)? Can a subset of data be synchronized? How efficient is the Sync protocol (affecting costs and speed)?

The Shift Towards Edge Computing

The trend toward Edge Computing highlights the growing preference for offline-first solutions. By processing and storing data closer to its source, Edge Computing reduces cloud dependency, enhances privacy, and improves efficiency. Data synchronization plays an important role in this shift, ensuring seamless operation across decentralized networks.

Offline and online synchronization solutions each have their merits, but the rise of edge computing and data privacy concerns has propelled offline Sync to the forefront. Developers must assess their application’s unique requirements to select the most appropriate synchronization method. As the industry evolves, hybrid and offline-first solutions are going to dominate, offering the best balance of functionality, privacy, and performance.

Top Small Language Models (SLMs) and how local vector databases add power

Top Small Language Models (SLMs) and how local vector databases add power

by | Jan 20, 2025 | AI, Edge AI, Edge Database, Mobile Database, vector database

Can Small Language Models (SLMs) really do more with less? In this article, we discuss the unique strengths of SLMs, the top SLMs, their integration with local vector databases, and how SLMs + local vector databases are shaping the future of AI, prioritizing privacy, immediacy, and sustainability.

The Evolution of Language Models

In the world of artificial intelligence (AI), bigger models were once seen as better. Large Language Models (LLMs) amazed everyone with their ability to write, translate, and analyze complex texts. But they come with big problems too: high costs, slow processing, and huge energy demands. For example, OpenAI’s latest GPT-o3 model can cost up to $6,000 per task. The annual energy consumption of GPT-3.5 is equivalent to powering more than 4000 US households for a year. That’s a huge price to pay, both financially and environmentally.

Now, Small Language Models (SLMs) are stepping into the spotlight, enabling sophisticated AI to run directly on devices (local AI) like your phone, laptop, or even a smart home assistant. These models not only reduce costs and energy consumption but also bring the power of AI closer to the user, ensuring privacy and real-time performance.

What Are Small Language Models (SLMs)?

LLMs are designed to understand and generate human language. Small Language Models (SLMs) are compact versions of LLMs. So, the key difference between SLMs and LLMs is their size. While LLMs like GPT-4 are designed with hundreds of billions of parameters, SLMs use only a fraction of that. There is no strict definition of SLM vs. LLM yet. At this moment, SLM sizes can be as small as single-digit million parameters and go up to several billion parameters. Some authors suggest 8B parameters as the limit for SLMs. However, in our view that opens up the question if we need a definition for Tiny Language Models (TLMs)?

Advantages and disadvantages of SLM

According to Deloitte’s latest tech trends report, SLMs are gaining increasing importance in the AI landscape due to their cost-effectiveness, efficiency, and privacy advantages. Small Language Models (SLMs) bring a range of benefits, particularly for local AI applications, but they also come with trade-offs.

Benefits of SLMs

  • Privacy: By running on-device, SLMs keep sensitive information local, eliminating the need to send data to the cloud.
  • Offline Capabilities: Local AI powered by SLMs functions seamlessly without internet connectivity.
  • Speed: SLMs require less computational power, enabling faster inference and smoother performance.
  • Sustainability: With lower energy demands for both training and operation, SLMs are more environmentally friendly.
  • Accessibility: Affordable training and deployment, combined with minimal hardware requirements, make SLMs accessible to users and businesses of all sizes.

Limitations of SLMs

The main disadvantage is the flexibility and quality of SLM responses: SLMs typically cannot tackle the same broad range of tasks as LLMs in the same quality. However, in certain areas, they already match their larger counterparts. For example, Artificial Analysis AI Review 2024 highlights that GPT-4o-mini (July 2024) has a similar Quality Index to GPT-4 (March 2023), while being 100x cheaper in price.

Small Language Models vs LLMs
Small Language Models vs LLMs

A recent study comparing various SLMs highlights the growing competitiveness of these models, demonstrating that in specific tasks, SLMs can achieve performance levels comparable to much larger models.

Overcoming limitations of SLMs

A combination of SLMs with local vector databases is a game-changer. With a local vector database, the variety of tasks and the quality of answers cannot only be enhanced but also for the areas that are actually relevant to the use case you are solving. E.g. you can add internal company knowledge, specific product manuals, or personal files to the SLM. In short, you can provide the SLM with context and additional knowledge that has not been part of its training via a local vector database. In this combination, an SLM can already today be as powerful as an LLM for your specific case and context (your tasks, your apps, your business). We’ll dive into this a bit more later.

In the following, we’ll have a look at the current landscape of SLMs – including the top SLMs – in a handy comparison matrix.

Top SLM Models

Model NameSize (Parameters)Company/
Team		LicenseSourceQuality claims
DistilBERT66 MHugging FaceApache 2Hugging Face"40% less parameters than google-bert/bert-base-uncased, runs 60% faster while preserving over 95% of BERT’s performances"
MobileLLM1.5 BMetaPre-training code for MobileLLM open sourced (Attribution-NonCommercial 4.0 International)Arxiv.org"2.7%/4.3% accuracy boost over preceding
125M/350M state-of-the-art models"
"close correctness to LLaMA-v2 7B in API
calling tasks"
TinyGiant (xLAM-1B)1.3 BSalesforceTraining set open sourced (Creative Commons Public Licenses); trained model will be open sourcedAnnouncement

Related Research on Arxiv.org		"outperforming models 7x its size, including GPT-3.5 & Claude"
Gemma 2B2 BGoogleGemma license (not open source per definition, but seemingly pretty much unrestricted use), training data not sharedHuggingface"The Gemma performs well on the Open LLM leaderboard. But if we compare Gemma-2b (2.51 B) with PHI-2 (2.7 B) on the same benchmarks, PHI-2 easily beats Gemma-2b."
Phi-33.8 B, 7 BMicrosoftMIT LicenseMicrosoft NewsiPhone 14: Phi-3-mini processing speed of 12 tokens per second.
From the H2O Danube3 benchmarks you can see that the Phi-3 model shows top performance compared to similar size models, oftentimes beating the Danube3
OpenELM270M, 450M, 1.1B, 3BAppleApple License, but pretty much reads like you can do as much with it as a permissive oss license (of course not use their logo)Huggingface

GitHub		OpenELM 1.1 B shows 1.28% (Zero Shot Tasks), 2.36% (OpenLLM Leaderboard), and 1.72% (LLM360) higher accuracy compared to OLMo 1.2 B, while using 2× less pretraining data
H2O Danube33-500M, 3-4BH2O.aiApache 2.0Arvix.org

Huggingface		"competitive performance compared to popular models of similar size across a wide variety of benchmarks including academic benchmarks, chat benchmarks, as well as fine-tuning benchmarks"
GPT-4o mini~8B (rumoured)OpenAIProprietaryAnnouncementGPT-4o mini scores 82% on MMLU and currently outperforms GPT-4 on chat preferences in LMSYS leaderboard⁠. GPT-4o mini surpasses GPT-3.5 Turbo and other small models on academic benchmarks across both textual intelligence and multimodal reasoning, and supports the same range of languages as GPT-4o
Gemini 1.5 Flash 8B8BGoogleProprietaryAnnouncement on Google for DevelopersSmaller and faster variant of 1.5 Flash features half the price, twice the rate limits, and lower latency on small prompts compared to its forerunner. Nearly matches 1.5 Flash on many key benchmarks.
Llama 3.1 8B8BMetaLlama 3.1 CommunityHuggingface

Artificial Analysis		MMLU score of 69.4% and a Quality Index across evaluations of 53. Faster compared to average, with a output speed of 157.7 tokens per second. Low latency (0.37s TTFT), small context window (128k).
Mistral-7B7BMistralApache 2.0Huggingface

Artificial Analysis		MMLU score 60.1%. Mistral 7B significantly outperforms Llama 2 13B on all metrics, and is on par with Llama 34B (since Llama 2 34B was not released, we report results on Llama 34B). It is also vastly superior in code and reasoning benchmarks. Was the best model for its size in autumn 2023.
Ministral3B, 8BMistralMistral Research LicenseHuggingface

Artificial Analysis		Claimed (by Mistral) to be the world's best Edge models.

Ministral 3B has MMLU score of 58% and Quality index across evaluations of 51. Ministral 8B has MMLU score of 59% and Quality index across evaluations of 53.
Granite2B, 8BIBMApache 2.0Huggingface

IBM Announcement		Granite 3.0 8B Instruct matches leading similarly-sized open models on academic benchmarks while outperforming those peers on benchmarks for enterprise tasks and safety.
Qwen 2.50.5B, 1.5B, 3B, 7BAlibaba CloudApache 2.0 (0.5B, 1.5B, 7B)
Qwen Research (3B)		Huggingface

Qwen Announcement		Models specializing in coding and solving Math problems. For 7B model, MMLU score 74.2%, context window (128k).
Phi-414 BMicrosoftMIT LicenseHuggingface

Artificial Analysis		Quality Index across evaluations of 77, MMLU 85%, Supports a 16K token context window, ideal for long-text processing. Outperforms Phi3 and outperforms on many metrices or is comparable with Qwen 2.5 , and GPT-4o-mini

SLM Use Cases – best choice for running local AI

SLMs are perfect for on-device or local AI applications. On-device / local AI is needed in scenarios that involve hardware constraints, demand real-time or guaranteed response rates, require offline functionality or need to comply with strict data privacy and security needs. Here are some examples:

  • Mobile Applications: Chatbots or translation tools that work seamlessly on phones even when not connected to the internet.
  • IoT Devices: Voice assistants, smart appliances, and wearable tech running language models directly on the device.
  • Healthcare: Embedded in medical devices, SLMs allow patient data to be analyzed locally, preserving privacy while delivering real-time diagnostics.
  • Industrial Automation: SLMs process language on edge devices, increasing uptime and reducing latency in robotics and control systems.

By processing data locally, SLMs not only enhance privacy but also ensure reliable performance in environments where connectivity may be limited.

On-device Vector Databases and SLMs: A Perfect Match

Imagine a digital assistant on your phone that goes beyond generic answers, leveraging your company’s (and/or your personal) data to deliver precise, context-aware responses – without sharing this private data with any cloud or AI provider. This becomes possible when Small Language Models are paired with local vector databases. Using a technique called Retrieval-Augmented Generation (RAG), SLMs access the additional knowledge stored in the vector database, enabling them to provide personalized, up-to-date answers. Whether you’re troubleshooting a problem, exploring business insights, or staying informed on the latest developments, this combination ensures tailored and relevant responses.

Key Benefits of using a local tech stack with SLMs and a local vector database

  • Privacy. SLMs inherently provide privacy advantages by operating on-device, unlike larger models that rely on cloud infrastructure. To maintain this privacy advantage when integrating additional data, a local vector database is essential. ObjectBox is a leading example of a local database that ensures sensitive data remains local. 
  • Personalization. Vector databases give you a way to enhance the capabilities of SLMs and adapt them to your needs. For instance, you can integrate internal company data or personal device information to offer highly contextualized outputs.
  • Quality. Using additional context-relevant knowledge reduces hallucinations and increases the quality of the responses.
  • Traceability. As long as you store your metadata alongside the vector embeddings, all the knowledge you use from the local vector database can give the sources.
  • Offline-capability. Deploying SLMs directly on edge devices removes the need for internet access, making them ideal for scenarios with limited or no connectivity.
  • Cost-Effectiveness. By retrieving and caching the most relevant data to enhance the response of the SLM, vector databases reduce the workload of the SLM, saving computational resources. This makes them ideal for edge devices, like smartphones, where power and computing resources are limited.

Use case: Combining SLMs and local Vector Databases in Robotics

Imagine a warehouse robot that organizes inventory, assists workers, and ensures smooth operations. By integrating SLMs with local vector databases, the robot can process natural language commands, retrieve relevant context, and adapt its actions in real time – all without relying on cloud-based systems.

For example:

  • A worker says, “Can you bring me the red toolbox from section B?”
  • The SLM processes the request and consults the vector database, which stores information about the warehouse layout, inventory locations, and specific task history.
  • Using this context, the robot identifies the correct toolbox, navigates to section B, and delivers it to the worker.

The future of AI is – literally – in our hands

AI is becoming more personal, efficient, and accessible, and Small Language Models are driving this transformation. By enabling sophisticated local AI, SLMs deliver privacy, speed, and adaptability in ways that larger models cannot. Combined with technologies like vector databases, they make it possible to provide affordable, tailored, real-time solutions without compromising data security. The future of AI is not just about doing more – it’s about doing more where it matters most: right in your hands.

IoT, Edge Computing, and Digitalization in Healthcare

IoT, Edge Computing, and Digitalization in Healthcare

by | Nov 22, 2024 | Case Study, Edge Computing, Insights, IoT

The healthcare industry is experiencing an unprecedented surge in data generation, responsible for approximately 30% of the world’s total data volume. This vast and fast-growing amount of health data is the primary force behind the digital transformation of healthcare. Only through the adoption of advanced technologies can healthcare providers manage, analyze, and secure this information. While COVID-19 accelerated this shift, contributing to the explosion of health data, the ongoing demand for real-time patient insights, personalized treatment, and improved operational efficiency continues to drive the sector toward digitalization and AI. Simultaneously, growing data privacy concerns, increasing costs, and heavier regulatory requirements are challenging the use of cloud computing to manage this data. A megashift to Edge Computing and Edge AI is addressing these challenges, enabling a faster, safer, and more reliable digital healthcare infrastructure.

The digital healthcare market 2024 and beyond, a high-speed revolution

Prior to COVID, growth in digital health adoption stalled. However, digitalization in the healthcare industry has sky-rocketed since the start of the pandemic. Reflecting this market turnaround, followed by the rise of advanced digital tools like AI, recent years have been record-breaking for investments in healthcare companies. A trend that will continue in the next years, as analysts predict rapid growth across digital healthcare market sectors:

Healthcare market overview

Drivers of growth and change in digital healthcare

 

Digital Healthcare Growth Driver 1: Growing Medical IoT Device Adoption

There will be a projected 40 billion IoT devices by 2030. IoMT devices already accounted for 30% of the entire IoT device market in 2020. Internet of Medical Things (IoMT) are hardware devices designed to process, collect, and/or transmit health–related data via a network. According to Gartner, 79% of healthcare providers are already using IoT in their processes, i.e. remote health monitoring via wearables, ingestible sensors, disinfection robots, or closed-loop insulin delivery systems. IoMT devices increase safety and efficiency in healthcare, and future technical applications, like smart ambulances or augmented reality glasses that assist during surgery, are limitless.

IoMT devices accounted for 30% of the IoT device market

health-care-edge-computing

Digital Healthcare Growth Driver 2: The Explosion of Health Data

Growing IoMT adoption is subsequently driving a rapid increase in the amount of collected health data. According to the RBC study, the healthcare industry is now responsible for approximately 30% of the world’s total data volume. By 2025, healthcare data is expected to continue growing at a 36% CAGR, outpacing data volumes from sectors like manufacturing, financial services, and media. Big health data sets are being used to revolutionize healthcare, bringing new insights into fields like oncology, and improving patient experience, care, and diagnosis. According to the Journal of Big Data: “taken together, big data will facilitate healthcare by introducing prediction of epidemics (in relation to population health), providing early warnings of disease conditions, and helping in the discovery of novel biomarkers and intelligent therapeutic intervention strategies for an improved quality of life.” In fact, the healthcare analytics market is projected to reach $129.7 billion by 2028, growing at a 23.5% CAGR​. This growth is driven by the need for real-time data processing, personalized medicine, and predictive analytics to manage chronic conditions and optimize hospital operations.

health-care-edge-computing

Healthcare data occupies ~30% of the world’s total data volume

Digital Healthcare Growth Driver 3: Artificial Intelligence

The increase in healthcare data opens up new opportunities and challenges to apply advanced technologies like big data analytics and artificial intelligence (AI) to improve healthcare delivery, patient outcomes, and operational efficiency. For instance, AI is being used to analyze medical imaging data, identify patterns in electronic health records, and predict patient outcomes, contributing to improved patient care. By 2026, AI is projected to save the global healthcare industry over $150 billion annually, by answering “20 percent of unmet clinical demand.” 

Generative AI, which includes Large Language Models (LLMs) such as GPT-4, is playing a crucial role in this transformation. According to the survey from McKinsey, 70% of surveyed healthcare organizations are either currently testing or actively using generative AI tools for both clinical and administrative applications​. This is unsurprising, as LLM Chatbots can reduce waiting times by 80% in healthcare facilities. In diagnostics, LLMs are being applied to interpret electronic health records and assist with predictive analytics, leading to a reduction in hospital readmissions by up to 22%. Additionally, LLMs have helped improve medication adherence rates by 60%, demonstrating their impact on patient care quality​.

70% of healthcare organizations plan or use AI

health-care-edge-computing

Digital Healthcare Growth Driver 4: Artificial Intelligence

With the rise of IoMT and the boost in healthcare data, Edge Computing is becoming a key driver of healthcare digitalization. The majority of IoMT devices (55.3 %) currently operate on-premise rather than in the cloud, ensuring faster, more secure real-time data processing. This shift to Edge Computing enhances data privacy and reduces latency, which is critical in life-critical medical applications. Additionally, the development of Small Language Models (SLMs) for on-device AI (Edge AI) allows healthcare providers to deploy AI-powered solutions directly on medical devices. This helps with tasks like remote monitoring and diagnostics without the need for cloud connectivity, which is particularly beneficial in environments with limited internet access​. 

As IoMT continues to evolve, Edge Computing will play an essential role in supporting healthcare’s increasing demand for real-time data processing. By 2025, it is projected that 75% of the healthcare data will be generated at the Edge, further driving the adoption of these technologies across the industry​.

AI-Health-Icon

75% of the healthcare  data will be generated at the Edge in 2025

Digital Healthcare Growth Driver 5: Underlying Social Megatrends

The global population is growing; global life expectancy is rising. Accordingly, by 2030 the world needs more energy, more food, and more water. Explosive population growth in some areas versus declines in others contributes to shifts in economic power, resource allocation, societal habits, and norms. Many Western populations are aging rapidly. E.g. in America, the number of people 65+ is expected to nearly double to 72.1 million by 2034. Because the population is shrinking at the same time, elder care is a growing challenge and researchers are looking to robots to solve it. 

Health megatrends focus not only on the prevention of disease, but also on the perception of wellness, and new forms of living and working. Over this decade more resources will be spent on health and longevity, leading to artificially and technologically enhanced human capabilities. More lifestyle-related disorders and diseases are expected to emerge in the future.

A focus on health and longevity will
lead to artificial & tech-enhanced
human capabilities

health-care-edge-computing

The Challenges of Healthtech

Along with more data, more devices, and more opportunities also comes more responsibility and more costs for healthcare providers.

health-care-edge-computing

Data Volume and Availability With the growing number of digital healthcare and medical devices, a dazzling volume of health data is created and collected across many different channels. It will be vital for the healthcare industry to reliably synchronize and combine data across devices and channels. Due to the sheer volume, reliable collection and analysis of this data is a major challenge. After it’s been processed, data needs to be available on demand, i.e. in emergency situations that require reliable, fast, available data.

health-care-edge-computing

Reliability, Privacy, and Data Security are extremely important in health technology; 70% of healthcare consumers are concerned about data privacy. Data use is often governed by increasingly strict national regulations, i.e. HIPAA (USA) and/or GDPR (Europe). With the number of cyber-attacks in the healthcare industry on the rise, healthcare professionals must be even more diligent about the storage and processing of data. In addition, healthtech must be extremely well vetted; failures can cost lives – typical “banana products”, which ripen with the customers, are a no-go.

health-care-edge-computing

IT Costs Medical devices contribute a large portion to healthcare budgets. However as data volumes grow, data costs will also become a relevant cost point. Sending all health data to the cloud to be stored and processed is not only slow and insecure, it is also extremely costly. To curb mobile network and cloud costs, much health data can be stored and processed at the edge, on local devices, with only necessary data being synced to a cloud or central server. By building resilient data architecture now, healthcare providers (e.g. hospitals, clinics, research centers) can avoid future costs and headaches.

Edge Computing is Integral to Data-driven Healthcare Ecosystems

With big data volumes, industries like healthcare need to seek out resilient information architectures to accommodate growing numbers of data and devices. To build resilient and secure digital infrastructure, healthcare providers will need to utilize both cloud computing and edge computing models, exploiting the strengths of both systems.

Cloud & Edge: What’s the Difference?

Cloud Computing information is sent to a centralized data center, to be stored, processed and sent back to the edge. This causes latency and a higher risk of data breaches. Centralized data is useful for large-scale data analysis and the distribution of data between i.e. hospitals and doctors’ offices.

Edge Computing Data is stored and processed on or near the device it was created on. Edge Computing works without an internet connection, and thus is reliable and robust in any scenario. It is ideal for time-sensitive data (real-time), and improved data privacy and security.

health-care-edge-computing

Edge Computing contributes to resilient and secure healthcare data systems

health-care-edge-computing

Transforming Healthcare with Edge Computing

Use Case: Secure and Up to Date Digital Record Keeping in Doctors Offices

For private doctors’ offices, embracing digitalization comes with different hurdles than larger healthcare providers. Often, offices do not keep a dedicated IT professional on staff, and must find digital solutions that serve their needs, while allowing them to comply with ever-increasing data regulations. As an industry used to legislative challenges, GPs know that sensitive patient data must be handled with care.

Solution providers serving private doctors’ offices are using edge databases to help keep patient data secure. An edge database allows private GPs to collect and store digital data locally. In newer practice setups, doctors use tablets, like iPads, throughout their practice to collect and track patient data, take notes and improve flexibility. This patient data should not be sent or stored in a central cloud server as this increases the risk of data breaches and opens up regulatory challenges. In a cloud-centered setup, the doctor also always needs to rely on a constant internet connection being available, making this also a matter of data availability

health-care-edge-computing

Accordingly, the patient data is stored locally, on the iPads, accessible only by the doctor treating the patient. Some of the data is synchronized to a local, in-office computer at the front desk for billing and administration. Other data is only synchronized for backup purposes and encrypted. Such a setup also allows synchronizing data between iPads, enabling doctors to share data in an instant.

Use Case: Connected Ambulances – Real-Time Edge Data from Home to Hospital

health-care-edge-computing

Between an incidence location and the hospital, a lot can happen. What if everything that happened in the ambulance was reliably and securely tracked and shared with the hospital, seamlessly? There are already trials using 5G technology to stream real-time data to hospitals, allowing ambulance medics to access patient data while in transit. Looking to the future, Edge Computing will enable digital healthcare applications to function in real-time and reliably anywhere and anytime, e.g. a moving ambulance, in the tunnel, or a remote area, enabling ambulance teams and doctors to give the best treatment instantly / on-site, while using available bandwidth and networks when available to seamlessly synchronize the relevant information to the relevant healthcare units, e.g. the next hospital. This will decrease friction, enhance operational processes, and improve time to treatment.

Digital Healthcare: Key Take-Aways

Digital healthcare is a fast-growing industry; more data and devices alongside new tech are empowering rapid advances. Finding ways to utilize growing healthcare data, while ensuring data privacy, security and availability are key challenges ahead for healthcare providers. The healthcare industry must find the right mix of technologies to manage this data, utilizing cloud for global data exchange and big data analytics, while embracing Edge Computing for it’s speed, security, and resilience.

health-care-edge-computing

Underutilized data plays a major role in health-tech innovation, data is the lifeline of future healthcare offerings; however, there is still much work to be done to improve the collection, management, and analysis of this data.

health-care-edge-computing

It’s all about data availability. Either in emergency situations, or simply to provide a smooth patient experience, data needs to be fast, reliable, and available: when you need it where you need it.

health-care-edge-computing

Edge computing alongside other developing technologies like 5G or Artificial Intelligence will empower a new and powerful digital healthcare ecosystem.

ObjectBox provides edge data software, to empower scalable and resilient digital innovation on the edge in healthcare, automotive, and manufacturing. ObjectBox’ edge database and data synchronization solution is 10x faster than any alternative, and empowers applications
that respond in real-time (low-latency), work offline without a connection to the cloud, reduce energy needs, keep data secure, and lower mobile network and cloud costs.

Get in touch to learn more about ObjectBox' Edge Technology

Sources

  1. https://www.marketsandmarkets.com/Market-Reports/iot-healthcare-market-160082804.html
  2. https://www.marketsandmarkets.com/Market-Reports/artificial-intelligence-healthcare-market-54679303.html
  3. https://www.grandviewresearch.com/press-release/global-mhealth-app-market
  4. https://www.grandviewresearch.com/industry-analysis/wearable-medical-devices-market
  5. https://www.fortunebusinessinsights.com/industry-reports/digital-health-market-100227

The rise of small language models (“small LLMs”)

The rise of small language models (“small LLMs”)

by | Oct 2, 2024 | AI, Edge AI, Edge Computing, Edge Database, Mobile Database, vector database

As artificial intelligence (AI) continues to evolve, companies, researchers, and developers are recognizing that bigger isn’t always better. Therefore, the era of ever-expanding model sizes is giving way to more efficient, compact models, so-called Small Language Models (SLMs). SLMs offer several key advantages that address both the growing complexity of AI and the practical challenges of deploying large-scale models. In this article, we’ll explore why the race for larger models is slowing down and how SLMs are emerging as the sustainable solution for the future of AI.

 

 

From Bigger to Better: The End of the Large Model Race

Up until 2023, the focus was on expanding models to unprecedented scales. But the era of creating ever-larger models appears to be coming to an end. Many newer models like Grok or Llama 3 are smaller in size yet maintain or even improve performance compared to models from just a year ago. The drive now is to reduce model size, optimize resources, and maintain power.

The Plateau of Large Language Models (LLMs)

 

2024_12_16_SLMs_2

Why Bigger No Longer Equals Better

As models become larger, developers are realizing that the performance improvements aren’t always worth the additional computational cost. Breakthroughs in knowledge distillation and fine-tuning enable smaller models to compete with and even outperform their larger predecessors in specific tasks. For example, medium-sized models like Llama with 70B parameters and Gemma-2 with 27B parameters are among the top 30 models in the chatbot arena, outperforming even much larger models like GPT-3.5 with 175B parameters.

 

The Shift Towards Small Language Models (SLMs)

In parallel with the optimization of LLMs, the rise of SLMs presents a new trend (see Figure). These models require fewer computational resources, offer faster inference times, and have the potential to run directly on devices. In combination with an on-device database, this enables powerful local GenAI and on-device RAG apps on all kinds of embedded devices, like on mobile phones, Raspberry Pis, commodity laptops, IoT, and robotics.

 

Advantages of SLMs

Despite the growing complexity of AI systems, SLMs offer several key advantages that make them essential in today’s AI landscape:

 

speed-icon

Efficiency and Speed
SLMs are significantly more efficient, needing less computational power to operate. This makes them perfect for resource-constrained environments like edge computing, mobile phones, and IoT systems. This enables quicker response times and more real-time applications. For example, studies show that small models like DistilBERT can retain over 95% of the performance of larger models in some tasks while being 60% smaller and faster to execute.

Accessibility
As SLMs are less resource-hungry (less hardware requirements, less CPU, memory, power needs), they are more accessible for companies and developers with smaller budgets. Because the model and data can be used locally, on-device / on-premise, there is no need for cloud infatstructure and they are also usable for use cases with high privacy requirements. All in all, SLMs democratize AI development and empower smaller teams and individual developers to deploy advanced models on more affordable hardware.

Cost Reduction and Sustainability
Training and deploying large models require immense computational and financial resources, and comes with high operational costs. SLMs drastically reduce the cost of training, deployment, and operation as well as the carbon footprint, making AI more financially and environmentally sustainable.

Gear

Specialization and Fine-tuning
SLMs can be fine-tuned more efficiently for specific applications. They excel in domain-specific tasks because their smaller size allows for faster and more efficient retraining. It makes them ideal for sectors like healthcare, legal document analysis, or customer service automation. For instance, using the ‘distilling step-by-step’ mechanism, a 770M parameter T5 model outperformed a 540B parameter PaLM model using 80% of the benchmark dataset, showcasing the power of specialized training techniques with a much smaller model size

Gear

On-Device AI for Privacy and Security
SLMs are becoming compact enough for deployment on edge devices like smartphones, IoT sensors, and wearable tech. This reduces the need for sensitive data to be sent to external servers, ensuring that user data stays local. With the rise of on-device vector databases, SLMs can now handle use-case-specific, personal, and private data directly on the device. This allows more advanced AI apps, like those using RAG, to interact with personal documents and perform tasks without sending data to the cloud. With a local, on-device  vector database users get personalized, secure AI experiences while keeping their data private.

 The Future: Fit-for-Purpose Models: From Tiny to Small to Large Language models

 The future of AI will likely see the rise of models that are neither massive nor minimal but fit-for-purpose. This “right-sizing” reflects a broader shift toward models that balance scale with practicality. SLMs are becoming the go-to choice for environments where specialization is key and resources are limited. Medium-sized models (20-70 billion parameters) are becoming the standard choice for balancing computational efficiency and performance on general AI tasks. At the same time, SLMs are proving their worth in areas that require low latency and high privacy.

Innovations in model compression, parameter-efficient fine-tuning, and new architecture designs are enabling these smaller models to match or even outperform their predecessors. The focus on optimization rather than expansion will continue to be the driving force behind AI development in the coming years.

 

 Conclusion: Scaling Smart is the New Paradigm

 

As the field of AI moves beyond the era of “bigger is better,” SLMs and medium-sized models are becoming more important than ever. These models represent the future of scalable and efficient AI. They serve as the workhorses of an industry that is looking to balance performance with sustainability and efficiency. The focus on smaller, more optimized models demonstrates that innovation in AI isn’t just about scaling up; it’s about scaling smart.

Local AI – what it is and why we need it

Local AI – what it is and why we need it

by | Sep 11, 2024 | AI, Edge AI, Edge Computing, Mobile Database, vector database

Artificial Intelligence (AI) has become an integral part of our daily lives in recent years. However, it has been tied to running in huge, centralized cloud data centers. This year, “local AI”, also known as “on-device AI” or “Edge AI”, is gaining momentum. Local vector databases, efficient language models (so-called Small Language Models, SLMs), and AI algorithms are becoming smaller, more efficient, and less compute-heavy. As a result, they can now run on a wide variety of devices, locally.

Figure 1. Evolution of language model’s size with time. Large language models (LLMs) are marked as celadon circles, and small language models (SLMs) as blue ones.

What is Local AI (on-device AI, Edge AI)?

Local AI refers to running AI applications directly on a device, locally, instead of relying on (distant) cloud servers. Such an on-device AI works in real-time on commodity hardware (e.g. old PCs), consumer devices (e.g. smartphones, wearables), and other types of embedded devices (e.g. robots and point-of-sale (POS) systems used in shops and restaurants). An interest in local Artificial Intelligence is growing (see Figure 2).

Figure 2. Interest over time according to Google Trends.

Why use Local AI: Benefits

Local AI addresses many of the concerns and challenges of current cloud-based AI applications. The main reasons for the advancement of local AI are: 

On top, local AI reduces:

  • latency, enabling real-time apps
  • data transmission and cloud costs, enabling commodity business cases

In short: By leveraging the power of Edge Computing and on-device processing, local AI can unlock new possibilities for a wide range of applications, from consumer applications to industrial automation to healthcare.

Privacy: Keeping Data Secure

In a world where data privacy concerns are increasing, local AI offers a solution. Since data is processed directly on the device, sensitive information remains local, minimizing the risk of breaches or misuse of personal data. No need for data sharing and data ownership is clear. This is the key to using AI responsibly in industries like healthcare, where sensitive data needs to be processed and used without being sent to external servers. For example, medical data analysis or diagnostic tools can run locally on a doctor’s device and be synchronized to other on-premise, local devices (like e.g. PCs, on-premise servers, specific medical equipment) as needed. This ensures that patient data never leaves the clinic, and data processing is compliant with strict privacy regulations like GDPR or HIPAA.

Accessibility: AI for Anyone, Anytime

One of the most significant advantages of local AI is its ability to function without an internet connection. This opens up a world of opportunities for users in remote locations or those with unreliable connectivity. Imagine having access to language translation, image recognition, or predictive text tools on your phone without needing to connect to the internet. Or a point-of-sale (POS) system in a retail store that operates seamlessly, even when there’s no internet. These AI-powered systems can still analyze customer buying habits, manage inventory, or suggest product recommendations offline, ensuring businesses don’t lose operational efficiency due to connectivity issues. Local AI makes this a reality. In combination with little hardware requirements, it makes AI accessible to anyone, anytime. Therefore, local AI is an integral ingredient in making AI more inclusive and to democratize AI.

Sustainability: Energy Efficiency

Cloud-based AI requires massive server farms that consume enormous amounts of energy. Despite strong efficiency improvements, in 2022, data centers globally consumed between 240 and 340 terawatt-hours (TWh) of electricity. To put this in perspective, data centers now use more electricity than entire countries like Argentina or Egypt. This growing energy demand places considerable pressure on global energy resources and contributes to around 1% of energy-related CO2 emissions.

The rise of AI has amplified these trends. According to McKinsey, the demand for data center capacity is projected to grow by over 20% annually, reaching approximately 300GW by 2030, with 70% of this capacity dedicated to hosting AI workloads. Gartner even predicts that by 2025, “AI will consume more energy than the human workforce”. AI workloads alone could drive a 160% increase in data center energy demand by 2030, with some estimates suggesting that AI could consume 500% more energy in the UK than it does today. By that time, data centers may account for up to 8% of total energy consumption in the United States.

In contrast, local AI presents a more sustainable alternative, e.g. by leveraging Small Language Models, which require less power to train and run. Since computations happen directly on the device, local AI significantly reduces the need for constant data transmission and large-scale server infrastructure. This not only lowers energy use but also helps decrease the overall carbon footprint. Additionally, integrating a local vector database can further enhance efficiency by minimizing reliance on power-hungry data centers, contributing to more energy-efficient and environmentally friendly technology solutions.

When to use local AI: Use case examples

Local AI enables an infinite number of new use cases. Thanks to advancements in AI models and vector databases, AI apps can be run cost-effectively on less capable hardware, e.g. commodity PCs, without the need for an internet connection and data sharing. This opens up the opportunity for offline AI, real-time AI, and private AI applications on a wide variety of devices. From smartphones and smartwatches to industrial equipment and even cars, local AI is becoming accessible to a broad range of users. 

  • Consumer Use Cases (B2C): Everyday apps like photo editors, voice assistants, and fitness trackers can integrate AI to offer faster and more personalized services (local RAG), or integrate generative AI capabilities. 
  • Business Use Cases (B2B): Retailers, manufacturers, and service providers can use local AI for data analysis, process automation, and real-time decision-making, even in offline environments. This improves efficiency and user experience without needing constant cloud connectivity.

Conclusion

Local AI is a powerful alternative to cloud-based solutions, making AI more accessible, private, and sustainable. With Small Language Models and on-device vector databases like ObjectBox, it is now possible to bring AI onto everyday devices. From the individual user who is looking for convenient, always-available tools to large businesses seeking to improve operations and create new services without relying on the cloud – local AI is transforming how we interact with technology everywhere.