Cross platform Data Sync: a simple example

by Anna | Mar 16, 2022 | Edge Computing, IoT, Mobile Database, Sync, Tutorial

Cross platform data sync can be simple: In this tutorial we will show you how you can easily sync data across devices.

Built for fast and effortless data access on and across embedded devices from Mobile to IoT, ObjectBox keeps data in sync between devices for you. The Database and Data Snyc works across platforms (iOS, Android, Linux, Rasbian, Windows, MacOS) and supports a variety of languages with easy native APIs (Swift, Java, Kotlin, C / C++, Flutter / Dart, Golang).

For example, you can sync between an Industrial IoT sensor app in Go and a C++ monitoring application – and a mobile Android app written in Kotlin or Java – and of course an iOS app written in Swift – and… you get the drift 😉

ObjectBox is a high-performance embedded database for Edge Computing with integrated Data Sync. The ObjectBox database is quick to set up and free and easy to use. Our powerful and intuitive APIs are a great match for multiplatform development environments.

Syncing data across devices – a task-list app example

In this tutorial, we are going to sync data across three instances of an example task-list app (written in C++, Go and Java).

With the task-list app, users can create simple text-based tasks and mark them as done. It stores tasks together with their creation dates. There is also a parameter to store the date when the task was completed. It acts as a filter for users to only see unfinished tasks.

This app is a standard cross platform ObjectBox example that is available for all language bindings. Here are the repositories of each example app that we will be looking at today:

Overview of the example code

In this section, we’ll quickly review how the the task-list example app uses ObjectBox Sync. For a more detailed description, check out the Sync docs. If you want to see how each of these steps were incorporated into the example code, go to the next section.

Note: The basic use of the database and its sync features is the same for all programming languages. If you haven’t used the ObjectBox DB yet, please refer to the corresponding documentation: C/C++ Docs, Java/Kotlin/Dart Docs, Go Docs, Swift Docs.

For sync to work in any app, we generally only need four things:

The sync-enabled library — this is not the same as the general ObjectBox library and has to be downloaded separately.
Database objects enabled for sync — for this we need include the sync annotation in the ObjectBox schema file.
ObjectBox Sync Server — please apply for a free Sync Trial here to get your own copy of the Sync Server (available for Linux and Docker). Note that this will only have to be started once and in this tutorial we’ll show you how to run the server on Linux. If you are using Docker, follow the steps outlined here.
Start a Sync Client in the app — as one can see from the Sync Client docs, creating and starting a sync client is just a matter of a couple of lines of code.

Important: When syncing between different apps, please make sure that the UIDs in the model JSON file (e.g. objectbox-default.json) are the same everywhere.

How to run the examples

Here you’ll find requirements and step-by-step guides for running the task-list example app in each of the three languages.

C++ example app

Requirements

New to C++? Check out our beginner C++ ObjectBox installation tutorial.

WSL Ubuntu
CMake
Git
C++
Clang

Step-by-step guide

1.Start by creating a CMakelists.txt file:

include(FetchContent)

FetchContent_Declare(

objectbox

GIT_REPOSITORY https://github.com/objectbox/objectbox-c.git

GIT_TAG v0.15.2

)

FetchContent_MakeAvailable(objectbox)

add_executable(myapp main.cpp)

target_link_libraries(myapp objectbox-sync)

Now configure and build the project via CMake: Configure (Clang), CMake: Build.

2. Sync-enabled objects: note the first line in tasklist.fbs.

3. [if not running a server already] Start the ObjectBox Sync Server on Linux by running ./sync-server --model build/_deps/objectbox-src/examples/cpp-gen/objectbox-model.json --unsecured-no-authentication

where sync-server is the path to your sync server executable. You can find more information about the server in the Sync Server docs.

4. Sync Client: launch [objectbox-c-examples-cpp-gen-sync], and the Sync Client will start automatically. You can see how it was implemented in main.cpp.

As this is just an example, we opted for no authentication to make things simple. This is not what you would use in production. We currently offer two authentication methods: shared secret and Google Sign-In. Here is the relevant Sync docs section on authentication options that explains how to use these.

5. Let’s add a first task, called “task-cpp” (new task-cpp-1), to check if our C++ app syncs correctly. The output should look like this:

Output of the C++ tasklist example app, showing a newly added task

6. You can finally open the Admin UI to check if the task appears there. This is most easily done by opening http://127.0.0.1:9980/ in any web browser. For a more detailed description of what this can do, check out the Admin UI docs.

Go example app

Requirements

WSL Ubuntu
Go (see how to configure it for VS Code here)
Git

Step-by-step guide

1. First, clone the objectbox-go repository to your VS Code project. Make sure the current directory is objectbox-go.

2. Sync-enabled objects. There are two versions of the task-list example: with and without sync. To run the one with sync, we need to enable our Task object for syncing. To do this, simply put the sync annotation on a new line in examples/tasks/internal/model/task.go:

// Put this on a new line to enable sync:

// `objectbox:"sync"`

type Task struct {

Id uint64

Text string

DateCreated time.Time `objectbox:"date"`

// DateFinished is initially set to unix epoch (value 0 in ObjectBox DB) to tag the task as "unfinished"

DateFinished time.Time `objectbox:"date"`

}

Then run the generator: go generate examples/tasks/internal/model/task.go to update the schema.

3. [if not running a server already] Now start the ObjectBox Sync Server: ./sync-server --model=examples/tasks/internal/model/objectbox-model.json --unsecured-no-authentication,

where sync-server is the path to your sync server file. You can find more information about the server in the Sync Server docs.

4. Run go run examples/tasks/main.go. The Sync Client will start within the app; check main.go to see how this was implemented.

5. Now we can add our first task (new task-go) – if it synced correctly, you should already see that from the output of the app. In particular, there will be a message from the change listener (“received 1 changes”):

Output of the Go task-list example app after adding a first task

6. Lastly, open the Admin UI to check if the task appears there. This is most easily done by opening http://127.0.0.1:9980/ in any web browser. For a more detailed description of what this can do, check out the Admin UI docs.

Admin UI showing a task created with the Go example app

Java (Android) example app

Requirements

Java
Android Studio

Step-by-step guide

First of all, open Android Studio and clone the objectbox-examples repository via File → New → Project from Version Control. Use this URL: https://github.com/objectbox/objectbox-examples.git
Sync-enabled objects: check out Task.java to see how this was done (note the @Sync annotation).
[if not running a server already] Start the ObjectBox Sync Server:

./sync-server --model android-app-sync/objectbox-models/default.json --unsecured-no-authentication,

where sync-server is the path to your sync server file. You can find more information about the server in the Sync Server docs.

Now you can run “android-app-sync” on a device of your choice. The Sync Client will start in the app.

As this is just an example, we opted for no authentication to make things simple. This is not what you would use in production. We currently offer two authentication methods: shared secret and Google Sign-In (only for Java, Kotlin, Dart, C & Go). Here is the relevant Sync docs section on authentication options that explains how to use these.

5. Add a new task called “task-java”.

6. Finally, open the Admin UI to check if the task appears there. This is most easily done by opening http://127.0.0.1:9980/ in any web browser. For a more detailed description of what this can do, check out the Admin UI docs.

Next Steps

How easy was that? Now that you’ve run your first ObjectBox Sync example, why not build something yourself? Use any combination of the supported languages to build your own cross platform app.

We’re eager to see your use case examples! Don’t hesitate to share your results with us by posting on Social Media and tagging @objectbox_io, or simply sending us an email on contact[at]objectbox.io.

If you want to learn more about how ObjectBox can be used in IoT, here is an overview of different use cases.

What is Data Synchronization + How to Keep Data in Sync

by Vivien | Dec 29, 2021 | Edge Computing, Insights, IoT, Sync

What is Data Sync / Data Synchronization in app development?

Data Synchronization (Sync) is the process of establishing consistency and consolidation of data between different devices. It is fundamental to most IT solutions, especially in IoT and Mobile. Data Sync entails the continuous harmonization of data over time and typically is a complex, non-trivial process. Even corporates struggle with its implementation and had to roll back Data Sync solutions due to technical challenges.

The question Data Sync answers is

phone-data-sync-with-machine-payment-automatic-data

How do you keep data sets from two (or more) data stores / databases – separated by space and time – mirrored with one another as closely as possible, in the most efficient way?

Data Sync challenges include asynchrony, conflicts, slow bandwidth, flaky networks, third-party applications, and file systems that have different semantics.

Data Sync versus Data Replication in Databases

Data replication is the process of storing the same data in several locations to prevent data loss and improve data availability and accessibility. Typically, data replication means that all data is fully mirrored / backed up / replicated on another instance (device/server). This way, all data is stored at least twice. Replication typically works in one direction only (unidirectional); there is no additional logic to it and no possibility of conflicts.

In contrast, Data Sync typically relates to a subset of the data (selection) and works in two directions (bi-directional). This adds a layer of complexity, because now conflicts can arise. Of course, if you select all data for synchronisation into one direction, it will yield the same result as replication. However, replication cannot replace synchronization.

Why do you need to keep data in sync?

Think about it – if clocks were not in sync, everyone would live on a different time. While I can see an upside to this, it would result in many inefficiencies as you could not rely on schedules. When business data is not in sync (up-to-date everywhere), it harms the efficiency of the organization due to:

Isolated data silos
Conflicting data / information states
Duplicate data / double effort
Outdated information states / incorrect data

In the end, the members of such an organization would not be able to communicate and collaborate efficiently with each other. They would instead be spending a lot of time on unnecessary work and “conflict resolution”. On top, management would miss an accurate overview and data-driven insights to prioritize and steer the company. The underlying mechanism that keeps data up-to-date across devices is a technical process called data synchronization (Sync). And while we expect these processes to “just work”, someone needs to implement and maintain them, which is a non-trivial task.

Growing data masses and shifts in data privacy requirements call for sensible usage of network bandwidth and the cloud. Edge computing with selective data synchronization is an effective way to manage which data is sent to the cloud, and which data stays on the device. Keeping data on the edge and synchronizing selective data sets effectively, reduces the data volume that is transferred via the network and stored in the cloud. Accordingly, this means lower mobile networking and cloud costs. On top, it also enables higher data security and data privacy, because it makes it easy to store personal and private data with the user. When data stays with the user, data ownership is clear too.

Unidirectional Data Replication

Bidirectional Data Synchronization

Out-of-the-box Sync magic: Syncing is hard

Almost every Mobile or IoT application needs to sync data, so every developer is aware of the basic concept and challenges. This is why many experienced developers appreciate out-of-the-box solutions. While JSON / REST offers a great concept to transfer data, there is more to Data Sync than what it looks like at a glance. Of course, the complexity of Sync varies widely depending on the use case. For example, the amount of data, data changes, synchronous / asynchronous sync, and number of devices (connections), and what kind of client-server or peer-to-peer setup is needed, all affect the complexity.

What looks easy in practice hides a complex bit of coding and opens a can of worms for testing. For an application to work seamlessly across devices – independent of the network, which can be offline, flaky, or only occasionally connected – an app developer must anticipate and handle a host of local and network failures to ensure data consistency. Moreover, for devices with restricted memory, battery and/or CPU resources (i.e. Mobile and IoT devices), resource sensitivity is also essential. Data storage and synchronization solutions must be both effective / efficient, and sustainable.

How to Keep Data in Sync Without the Headache?

Thankfully, there are out-of-the-box data synchronization solutions available on the market, which solve data syncing for developers. They fall broadly into two categories: cloud-dependent data synchronization, and independent, “edge” data synchronization. Cloud-based solutions, like Firebase, require a connection to the internet to function. Data is sent to and requested from the cloud constantly. Edge solutions, like ObjectBox, also offer “Offline Sync”: Data is stored in an efficient on-device database, synchronization on and between edge devices can be done continually without an Internet connection, and Dat Sync with a cloud or a backend that is not located on premise occurs once the device(s) goes online. Below, we summarize the most popular market offerings for data synchronization (offline and cloud based):

Couchbase

Couchbase is a Cloud DB, Edge DB and Sync offering that requires the use of Couchbase servers.

Firebase

Firebase is a Backend as a Service (BaaS) offering from Google (acquired). Google offers it as a cloud hosted solution for mobile developers.

Mongo Realm

Realm was acquired by MongoDB in 2019; the Mongo Realm Sync solution (Atlas Device Sync) used Realm DB on edge devices and synchronized with a MongoDB hosted in the cloud. However, MongoDB recently announced end-of-life for it.

ObjectBox

ObjectBox is a DB for any device, from restricted edge devices to servers, and offers an out-of-the-box Sync solution. ObjectBox enables self-hosting on-premise / in the cloud, as well as Offline Sync.

Parse

Parse is a BaaS offering that Facebook acquired and shut down. Facebook open sourced the code. The GitHub repository is not officially maintained. You can host Parse yourself or use a Parse hosting service.

Data Sync, Edge Computing, and the Future of Data

There is a megashift happening in computing from centralized cloud computing to Edge Computing. Edge computing is a decentralized topology entailing storing and using data as close to the source of the data as possible, i.e. directly on edge devices. Accordingly, the market is growing rapidly with projections estimating continuing growth with a 34% CAGR for the next five years. The move from the cloud to the edge is strongly driven by new use cases and growing data masses. Edge data persistence and Data Sync (managing decentralized data flows), especially “Offline Sync”, are the key technologies needed for Edge Computing. Using edge data persistence, data can be stored and processed on the edge. This means application always work, independent from a network connection, offline. Faster response times can be guaranteed. With Offline Sync, data can be synchronized between several edge devices in any location independant from an Internet connection. Once a connection becomes available, selected data can be synchronized with a central server. By exchanging less data with the cloud or a central instance, data synchronization reduces the burden on the network. This brings down mobile network and cloud costs, and reduces the amount of energy used: a win-win-win solution. It also enables data privacy by design.

Try the fastest Edge Sync solution today

Why database performance creates business value

by Vivien | Feb 20, 2021 | Edge Computing, Insights, IoT

“Why does database performance matter?” “What is the business value of database speed?” “Why should I care about the performance of a database?”

Why database performance matters in a nutshell

Make Faster Decisions

Database speed is the key to compute more data faster and make data-based decisions quickly. Faster decision-making drives business value.

Save Resources & Costs

Database speed translates into resource efficiency. Saving resources (like battery, CPU, memory) saves money and reduces waste.

Better UX & Response Rates

Database speed affects end user response rates significantly – smooth and fast user experiences keep people happy and more productive.

As a developer, it seems clear that database performance matters. At the very least, a fast database that gives you out-of-the-box speed saves time and nerves during development. Any piece of the tech stack that works super-fast makes a developer’s job easier. But there is more to it. Learn, why and how database performance impacts business value and get ideas on how to quantify this for your business case.

Data should be available when need where needed

We all dream of a future transformed by data. Cars that drive themselves to be repaired before a failure occurs. Fridges that are restocked while we are at work. Reducing resource waste to an absolute minimum. Building sustainable cities and communities.[1] It is truly amazing what is possible today…

Then reality hits: Before you can implement amazing solutions to make the world a better place for everyone, someone needs to solve the technical challenges, including hidden requirements. For example: you need the necessary data, and you need it available when needed where needed. This often isn’t that simple. Data persistence, database speed, and data synchronization are typical non-functional or “hidden” requirements. These are prerequisite technologies to allow the application to access, process and possibly depict the data required to answer a request (from another application or from a user), and thus enable the functionalities / features. All in all, this is a pretty fundamental requirement. And it pays off to build your app on top of a solid foundation. Because, if you built your application on a solid foundation, every feature you dream up, no matter when, and any next feature will be easier and faster to implement.

Functional and non-functional requirements – the hidden challenges of your IoT project

While you need data in any application, most often no one will write down where and how to handle it as a user story or requirement. As opposed to features, e.g. “being able to search for names in the address book”, data persistence, database speed, and often even data synchronization are “hidden requirements”. Data is just expected to be available where needed when needed. Whether the data you need really will be available when you need it, depends strongly on the database the application is using and and where this database runs. On top, the mechanisms you employ to exchange data between different devices (end devices, servers, ….) matter.

Hidden requirements are one of the major reasons why the Industry 4.0 dream is still in many respects a dream and not a reality – in Europe at least. Despite it being a topic for more than 10 years. [2]

Database performance

What is a database?

A database is a piece of software that allows the storage and systematic use of digital information. A database typically allows developers to store, access, search, update, query, and otherwise manipulate data in the database via a developer language or API. These types of operations are done within an application, in the background, typically hidden from end users. Most applications need a database as part of their technology stack.

What is database performance?

We like and therefore use the following definition from Craig Mullins (2002): “Database performance can be defined as the optimization of resource use to increase throughput and minimize contention, enabling the largest possible workload to be processed.” [3]

Why does it matter if the database runs on the edge or in the cloud?

An edge database holds data on the (end) devices, where the data is used – and typically additionally sends some parts of the data to a central place like an on-premise server or the cloud. As opposed to this, a server / cloud-based database holds all data on the server / in the cloud. Where the data sits, determines from where, when and how it can be accessed. If all data is on a central server or the cloud, the prerequisite to accessing this data is a working network connection.

Online

Offline

It follows that edge applications are based upon a distributed computing paradigm, allowing edge devices to be autonomous. On the other hand, cloud-based applications are based on the centralized computing paradigm, where one central instance is in charge, with all other devices being dependent upon this central instance. This significantly affects the response time of the application, the availability of the application, and last not least the bandwidth needed for the application, which also translates into cloud costs.

Location matters: while a fast database gives you fast response times, if the database sits in the cloud and needs to be called from edge devices, you need to factor in the duration it takes to request the data and get a response. And with any networking you cannot guarantee response times or ensure it is always available. While this is not the database performance itself, it highly affects application performance.

The impact of database performance on your business

Database performance matters. Whether your solution needs the speed, because of the necessity to re-act in (near) realtime, or to keep your users (customers, employees, …) happy, productive, buying, or just to save costs for stronger edge hardware and the cloud. “Considering that even a single moment of latency or downtime can cost companies thousands of dollars, the speed advantages of edge computing cannot be overlooked.” [4]

The necessity of database speed for mission-critical, security relevant, (near) real-time functionalities

If you need near real time functionalities, every piece in the tech stack matters, but the database has a particularly strong impact on the response rates of your application. Consider autonomous driving, healthcare and security applications, or IIoT solutions for production lines: Any application supporting such a scenario needs to respond reliably with speed. “This is not the same as a lag in loading your favorite cat pictures. A lag in a moving vehicle scenario is a matter of life and death.” [5]

Accordingly, if end devices like cars, smartphones, health trackers, machines on the factory floor are involved, a purely cloud-based application is not an option. Data needs to be stored and used on the devices directly. Thus, an edge database is necessary. Ideally, an extremely fast one.

Examples of use cases with a need for database speed

Anything running on a car really needs to be highly ressource efficient and fast. Ressources on the car are highly limited and database speed translates into ressource-efficency. Autonomous driving capabilities are a special case requiring significant compute power to run the algorithms in real-time within the control unit of the car. As can be easily deducted from first-hand driving experience, during this kind of constant information processing and instantaneous decision making, every fraction of a millisecond counts. Information processing speed and reliability (guaranteed QoS parameters) is of the essence for driver assistance and autonomous driving.

Moving to a purely monetary example, let’s consider roadside tolling. In roadside tolling, the edge devices on the side of the road need to process the information from a moving vehicle in order to identify the car, bill according to usage, and detect violators. Ideally, it even informs the car owner of the result. As the car is constantly moving and can be going fast, all of this needs to happen in a very short amount of time. A super fast database lookup on the edge is key to avoid money loss and deliver good customer service.

For a final example, let us look at an Industrial IoT (IIoT) application: Additive manufacturing. 3D printers use layering techniques with a variety of materials to quickly create custom designed parts. During the layering process, the controller needs to quickly and efficiently incorporate small changes in the environment (e.g. an increase in temperature) to ensure quality and accuracy of the part. Faster and more precise manufacturing is currently limited by the I/O throughput. With a fast database, the I/O throughput is higher, allowing for more complex and finite production.

In short: A superfast database is not a nice to-have, it is a must-have. The database speed a database brings out-of-the-box is critical for such an application.

The impact of database speed on Sales, Conversions, Retention (or at least, nerves)

There is a reason Google forces companies to optimize their websites and mobile applications for performance: There is a wealth of research and evidence that suggests response rates of websites and mobile applications impact user behavior significantly.[6] Even more, there are several studies providing evidence that response rates impact actual buying behavior. [7] While there is less research on other digital applications like e.g. a desktop app or workplace software, some studies have shown that needing to work with slow applications decreases employee satisfaction and productivity. [8]

The impact of database speed on battery, CPU, hardware and related resources

Another hidden requirement typically is resource-efficiency with regards to CPU, RAM, Disc space and battery / electricity. For any application running in the cloud, these requirements are balanced in the backend as the cloud scales vertically. It “only” adds to cloud costs (and is a waste of energy – not to mention all the infrastructure / hardware enabling that waste).

On the edge, you typically work with restricted devices, meaning you can only use the devices’ resources, which can be pretty limited. Therefore, inefficient applications can push a device to its limits, leading to e.g. slow response rates, crashes, and battery drain. Security is a very necessary cross-the-stack functionality that often impacts performance. While data that stays on the edge is challenging to hack, edge data needs to be protected just like data in the cloud.

How database performance impacts the business value of your IoT application

All applications on one device share the available hardware capabilities; resource allocation is managed by the operating system. Accordingly, the more resources an application or the database uses, the less resources are available for other uses. The faster a database executes its operations, the less CPU it uses, the less battery / electricity, and typically also memory. In practice that means there are more resources available on the device to run e.g. Edge AI or Edge ML applications.

From a business value perspective that means:

You can save on hardware costs (CPU, RAM, Disc, Memory, …): either do more on existing / chosen hardware, upgrade hardware later or choose smaller and thus less expensive hardware.
You can save on energy and cloud costs: The more efficient, the less electricity, the less cloud costs. This can add up tremendously as projects scale.
You can add more features, deliver more functionalities, make your application more secure within a given environment.
You can deliver a smooth, fast user experience, enabling applications that deliver in near-realtime.
In sum, it clearly impacts the cost structure and value you can deliver.

Database performance impacts business value, directly and indirectly

As projects scale in size and scope, hidden requirements like database performance often become clear. At scale, small issues like delayed data, or data volumes, become big headaches. Ideally, these sorts of requirements would be at the heart of the design stage of any project – and budgeted for at the beginning. The choice of database clearly has a huge impact on the business success of IoT applications.

[1] See https://www.weforum.org/agenda/2018/01/effect-technology-sustainability-sdgs-internet-things-iot/ for IoT impact on Sustainable Development Goals (SDG)
[2] https://restart-project.eu/much-know-industry-4-0/
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=13&cad=rja&uact=8&ved=2ahUKEwiGidSA6trnAhVQY8AKHTpSDUIQFjAMegQICBAB&url=https%3A%2F%2Fwww.mdpi.com%2F2076-3387%2F9%2F3%2F71%2Fpdf&usg=AOvVaw3cx44OOMfNzJ_BJlCG8Gfj
[3] Database Administration: The Complete Guide to Practices and Procedures By Craig Mullins 2002
[4] https://www.vxchnge.com/blog/the-5-best-benefits-of-edge-computing
[5] https://www.zdnet.com/article/why-autonomous-vehicles-will-rely-on-edge-computing-and-not-the-cloud/
[6] https://developers.google.com/web/fundamentals/performance/why-performance-matters https://www.thinkwithgoogle.com/intl/en-154/insights-inspiration/research-data/need-mobile-speed-how-mobile-latency-impacts-publisher-revenue/
https://www.machmetrics.com/speed-blog/how-does-page-load-time-affect-your-site-revenue
https://datadome.co/bot-management-protection/website-performance-how-to-increase-your-business-by-blocking-bots/
[7] https://developers.google.com/web/fundamentals/performance/why-performance-matters
https://www.thinkwithgoogle.com/intl/en-154/insights-inspiration/research-data/need-mobile-speed-how-mobile-latency-impacts-publisher-revenue/
https://www.machmetrics.com/speed-blog/how-does-page-load-time-affect-your-site-revenue
https://datadome.co/bot-management-protection/website-performance-how-to-increase-your-business-by-blocking-bots/
[8] https://drum.lib.umd.edu/handle/1903/1233
https://www.tandfonline.com/doi/abs/10.1080/01449290500196963

Add an Edge to Your Project Today

Why Edge Computing is More Relevant in 2021 Than Ever

by Vivien | Jan 14, 2021 | Edge Computing, Insights, IoT

The world has been forced to digitize more quickly and to a greater extent in 2020 and 2021. COVID has created the need to remodel how work, socializing, production, entertainment, and supply chains function. Despite decades of digitization efforts, with the pandemic upon us, digitization challenges have become transparent. Many companies and countries realize now, they have fallen behind. And those that have not yet digitized were hit hardest by the pandemic. [1] With people leaning heavily on online digital solutions, internet infrastructure is at its capacity limit. [2] Accordingly, users are seeing broadband speeds drop by as much as half. [3] In Europe, governments even requested to reduce the quality of Netflix, Amazon Prime, Youtube and other streaming services to improve network speed. [4]

These challenges demonstrate the growing need for an alternative to cloud computing. Cloud computing is an inherently centralized computing paradigm. Edge Computing is a decentralized topology that is based on keeping data local, at the ‘edge’ of the network, as close to the source as possible. Edge Computing is ideal for applications that are data-intensive, have high latency-requirements, or need to work offline, independant from a cloud connection. Using data on the edge, directly on or near the source of the data, not only increases the efficiency and speed of data use, but it reduces unecessary network burden and data traffic waste.

Coronavirus accelerates the need to digitize

It was clear even before the outbreak that internet infrastructure was struggling to keep up with growing data volumes. However, the pandemic has made broadband limitations more apparent to everyday users.

Projections estimate that by 2025 there will be 20 million IoT devices [5] and 1.7MB of data created per second per person. It is slow, expensive, and wasteful to send all of this data to the cloud for storage and processing. This practice overburdens bandwidth and data center infrastructure. It makes projects expensive and unsustainable. Working with the data, locally, on the edge, where it was produced and is used, is more efficient than sending everything to the cloud and back. It brings reduced latency, reduced cloud usage and costs, independence from a network connection, more secure data and heightened data privacy – and even reduces CO2. Indeed, prior to the pandemic, edge computing was on the strategic roadmap for over 50% of mobility decision makers. [6]

As the world begins to recover from the coronavirus pandemic, digitization efforts will no doubt increase. We will see intelligent systems implemented across industries and value chains, accelerating innovation and alongside: data volumes and subsequent strain on network bandwidth. Edge computing is a key technology to ensure that this digitalization is both scalable and sustainable.

Edge Computing takes the ‘edge’ off bandwidth strain

What is Edge Computing?

With edge computing, data is stored and used on devices at the “edge” of the network – away from centralized cloud servers. Computing on the edge means that data is stored and used locally, on the device, e.g. a smart phone or IoT device. Edge computing delivers faster decision making, local and offline data processing, as well as reduced data transfer to the cloud (e.g. filtered, computed, extra- or interpolated data), which saves both bandwidth and cloud storage costs.

The Edge complements the Cloud

Although some might set cloud and edge in competition, the reality is that edge computing and cloud computing are both useful and relevant technologies. Both have different strengths and ideal use cases. Together they can provide the best of both worlds: decentralized local storage and processing, making efficient use of hardware on the edge and central storing and processing of some data, enabling additional centralized insights, data backups (redundancy), and remote access. To combine the best of both worlds, relevant and useful data must be synchronized between the edge and cloud in a smart and efficient way.

Edge computing is an ideal technology to reduce the strain on data centers, so those functions that need cloud connection have adequate bandwidth; while those use cases that benefit from reduced latency and offline functionality are optimized on the edge.

The Edge: interface between the Physical and the Digital World

Edge devices handle the interface between the physical world and the cloud, enabling a whole set of new use cases. “Data-driven experiences are rich, immersive and immediate. But they’re also delay-intolerant data hogs”. [8] And therefore need to happen locally, on the edge. We may see edge computing enabling new forms of remote engagement [9], particularly in a post-corona environment.

Edge devices can be anything from a thermostat or small sensor to a fridge or mobile phone or car – and they are part of our direct physical world and use data from their local environment to enable new use cases. Think self-stocking fridges, self-driving cars, drone-delivered pizzas. In the same way, Edge Computing is the key to the first real world search engine. I am waiting for it every day: “Hey Google, where are my keys?” Within a location like a house, the concepts and technologies to enable such a real-world search engine are all clear and available – it is just a matter of time and ongoing digitization. The basis will need to be a fast and sustainable edge infrastructure.

Sustainability on the Edge

Centralized data centers consume a lot of energy, produce a lot of carbon emissions and cause significant electronic waste. [10] While data centers are seeing a positive trend towards using green data centers, an even more sustainable approach is to cut unnecessary cloud traffic, central computation and storage as much as possible by shifting computation to the edge. Edge Computing strategies that harness the power of already deployed available hardware (like e.g. smartphones, machines, desktops, gateways) make the solution even more sustainable.

Intelligent Edge: AI and Edge advance hand in hand

The growth of Artificial Intelligence (AI) and the Edge will go hand in hand. As more and more data is generated at the edge of the network, there will be a greater demand for intelligent data processing and structured optimization to reduce raw data loads going to the cloud. [11] Edge AI will have the power to work with data on local devices, keeping data streams more useful and usable. In the near future, Machine Learning applications will have the ability to learn and create unique, localized, decentralized insights on the edge – based on local inputs.

“With Edge AI, personalization features that we want from the app can be achieved on device. Transferring data over networks and into cloud-based servers allows for latency. At each endpoint, there are security risks involved in the data transfer”. [12] Which is part of the reason why the Edge AI Software market is forecasted to reach 1.12 trillion dollars volume by 2023. The development of AI accelerators, which improve model inferencing on the edge, namely from NVIDIA, Intel and Google are helping to make AI on the edge more viable. [13] A fast edge database is a necessary base technology to enable more AI on the edge.

Edge Computing – an answer to Data Privacy concerns and a need for Resilience

As data collection grows in both breadth and depth, there is a stronger need for data privacy and security. Edge computing is one way to tackle this challenge: keeping data where it is produced, locally, makes data ownership clear and data less likely to be attacked and compromised. If compromised, the data compromised is clearly defined, making notification and subsequent actions manageable. ObjectBox, in its core and as an edge technology, is designed to keep data private, on those devices it was created on, and only share select data as needed.

The more our private and working lives as well as the larger economy depend on digitalization, the more important it is that systems, underlying computing paradigms as well as networks have strong resilience and security. In computer networking, resilience is the ability to “provide and maintain an acceptable level of service in the face of faults and challenges to normal operation.” [14]

ing initEdge Computing shifts computer workloads – the collection, processing, and storage of data – from central locations (like the cloud) to the edge of the networks to many individual devices such as cell phones. Accordingly, any strain is distributed to many devices. Therefore, the risk of a total breakdown is reduced: If one device does not work anymore, the rest is still working. Depending on the setup, the individual devices could even compensate for devices that have a problem.

The same applies to security risks: Even if data from one device is compromised, all other data sets are still safe; the loss is thus very limited and clear. Overall, as a complement to the cloud, edge computing provides improved strength and security in local networks around the world. These local infrastructures can relieve the pressure on the existing complex dependencies, and in turn make the wider system more resilient and flexible. With Edge Computing crisis response can therefore in all likelihood be faster, better informed, and more effective. [15]

Why Corona-Tracking-Apps need to work on the edge

There was initially quite some debate about taking a centralized versus decentralized approach to Corona-Tracking-Apps. [16] Many people were worried about their data. Edge Computing – storing most parts of the data locally, on the user’s device – is a great way to avoid unnecessary data sharing and keep data ownership clear. At the same time, data is by and large much more secure and less likely to be attacked and hacked, as the data to be gained is very reduced. An intelligent syncing mechanism like ObjectBox Sync ensures that the data which needs to be shared, is shared in a selective, transparent and secure way.

The next few years will see big cultural changes in both our personal and professional lives – a portion of those changes will be driven by increased digitalization. Edge computing is an important paradigm to ensure these changes are sustainable, scalable, and secure. Ultimately, we have the chance to rise from this crisis with new insights, new innovation, and a more sustainable future.

Discover Edge Use Cases

1. https://www.netzoekonom.de/2020/04/11/die-oekonomie-nach-corona-digitalisierung-und-automatisierung-in-hoechstgeschwindigkeit/
2. https://www.cnet.com/news/coronavirus-has-made-peak-internet-usage-into-the-new-normal/
3. https://www.nytimes.com/2020/03/26/business/coronavirus-internet-traffic-speed.html
4. https://www.theverge.com/2020/3/27/21195358/streaming-netflix-disney-hbo-now-youtube-twitch-amazon-prime-video-coronavirus-broadband-network
5. https://www.gartner.com/imagesrv/books/iot/iotEbook_digital.pdf
6. https://www.forbes.com/sites/forrester/2019/12/02/predictions-2020-edge-computing-makes-the-leap/#1aba50104201
7. https://www.gartner.com/smarterwithgartner/what-edge-computing-means-for-infrastructure-and-operations-leaders/
8. https://www.iotworldtoday.com/2020/03/19/ai-at-the-edge-still-mostly-consumer-not-enterprise-market/
9. https://www.accenture.com/us-en/insights/high-tech/edge-processing-remote-viewership
10. https://link.springer.com/article/10.1007/s12053-019-09833-8
11. https://www.forbes.com/sites/cognitiveworld/2020/04/16/edge-ai-is-the-future-intel-and-udacity-are-teaming-up-to-train-developers/#232c8fab68f2
12. https://www.forbes.com/sites/cognitiveworld/2020/04/16/edge-ai-is-the-future-intel-and-udacity-are-teaming-up-to-train-developers/#232c8fab68f2
13. https://www.forbes.com/sites/janakirammsv/2019/07/15/how-ai-accelerators-are-changing-the-face-of-edge-computing/#2c1304ce674f
14. https://en.wikipedia.org/wiki/Resilience_(network)
15. https://www.coindesk.com/how-edge-computing-can-make-us-more-resilient-in-a-crisis
16. https://venturebeat.com/2020/04/13/what-privacy-preserving-coronavirus-tracing-apps-need-to-succeed/

Digital Healthcare – Market, projections, and trends

by Vivien | Dec 6, 2020 | Edge Computing, Insights, IoT

Note to Readers (2024 Update)
This white paper, originally published in 2020, explores key projections, market dynamics, and trends in digital healthcare. Since its publication, the field has evolved significantly, and the accompanying detailed report has been updated to reflect the state of digital healthcare in 2024. For the most current insights and analysis, please refer to the updated report (also linked at the end of this blog).

If you work in the healthcare industry, you are likely familiar with some uses of IoT devices. According to Gartner (2020), 79% of healthcare providers are already successfully employing IoT solutions.[1] However, this is just the beginning. While before COVID-19, the growth of digital health adoption had stalled [2], the market is picking up speed again. Indeed, Q3 2020 was a record year for investments in healthcare companies [3] and the market expects rising investments in healthtech for next years [4]. Today, underutilized data plays a major role in healthtech innovation [17] and the growing importance of healthcare data for future offerings is evident [5]. Take a look how analyts from Gartner to Accenture and Forrester expect the market to grow:

The digital healthcare market 2020 and beyond

digital-healthcare-market-trends-2020-edge-iot

Analysts expect Artificial Intelligence in healthcare to reach $6.6 billion by 2021 (with a 40% CAGR). [6]
The Internet of Medical Things (IoMT) market is expected to cross $136 billion by 2021. [11]
Analysts expect the healthcare wearable market to have a market volume of $27 billion by 2023 (with a 27.9% CAGR). [7]
The IoT industry is projected to be worth $6.2 trillion by 2025 and around 30% of that market (or about $167 billion) will come from healthcare. [8]
Analysts expect the global Medical Health Apps market to grow to $236 billion by 2026, reflecting a shift towards value based care. [9]
The projected global digital health market is estimated to reach $510.4 billion by 2026 (with a 29% CAGR). [10]

The Healthcare industry has been struggling with shrinking payments and cost optimizations for years. [18] Fueled by the need to adopt in light of the COVID pandemic, digital technologies bring extensive changes quickly to this struggling industry now. Data is moving to the center of this changing ecosystem and harbors both risks and opportunities in a new dimension. [21] The basic architecture and infrastructure to have the data reliably, securely and quickly available where they are needed will be decisive for the success or failure of digital healthcare solutions. [17] [21]

We recommend keeping an eye on the following five trends

The 5 biggest digital healthcare trends to watch

Artificial Intelligence (AI)

Accenture estimates that AI applications can help save up to $150 billion annually for the US healthcare economy by 2026. [6] Therefore, it is no wonder that the healthcare sector is expected to be among the top five industries investing in AI in the next couple of years. [19] The top three greatest near-term value AI applications in healthcare are: 1. robot-assisted surgery ($40 billion), 2. virtual nursing assistants ($20 billion), and 3. administrative workflow assistance ($18 billion).

Big Data / Analytics

The goal of big data analytic solutions is to improve the quality of patient care and the overall healthcare ecosystem. The global healthcare Big Data Analytics market is predicted to reach $39 billion by 2025. [12] The main areas of growth are medical data generation in the form of Electronic Health Records (EHR), biometric data, sensors data.

internet-of-medical-things-digital-healthtech

Internet of Medical Things (IoMT)

IoMT is expected to grow to $508.8 billion by 2027. [13] According to Gartner, 79% of healthcare providers are already using IoT in their processes. [27] During COVID, IoMT devices have been used to increase safety and efficiency in healthcare, i.e. providing and automating clinical assistance and treatment to the infected patient, to lessen the burden of specialists. Future applications, like augmented reality glasses that assist during surgery, are leading to a focus more on IoMT-centric investments. [14]

Telehealth / Telemedicine

Telecommunications technology enables doctors to diagnose and treat patients remotely. Consumer adoption of telehealth has skyrocketed in 2020 and McKinsey believes that up to $250 billion of current US healthcare spend could potentially be virtualized. [25] Also, many patients view telehealth offerings more favorable and – having made good experiences – are planning to continue using telehealth in the future. [26] Not astonishingly, telemedicine stocks also grow rapidly. [14]

Edge Computing

Edge computing is a technological megashift happening in computing. [23] Instead of pushing data to the cloud to be computed, processing is done locally, on ‘the edge’. [15] Edge Computing is one of the key technologies to make healthcare more connected, secure, and efficient. [22] Indeed, the digital healthcare ecosystem of the future depends on an infrastructure layer that makes health data accessible when needed where needed (data liquidity). [21] Accordingly, IDC expects the worldwide edge computing market to reach $250.6 billion in 2024 with a (12.5% CAGR) [24] with healthcare identified as one of the leading industries that will adopt edge computing. [16]

The healthcare market is in the middle of a fast digital transformation process. Drivers such as COVID, growing IoT adoption in healthcare, and underlying social mega-trends are pushing digital healthcare growth to new heights. Therefore, the digital healthcare industry faces many challenges, both technical and regulatory. At the same time the healthcare market is offered a wealth of opportunities.

Read the full report

References

[1] https://www.computerworld.com/article/3529427/how-iot-is-becoming-the-pulse-of-healthcare.html / https://www.gartner.com/en/documents/3970072
[2] https://www.accenture.com/us-en/insights/health/leaders-make-recent-digital-health-gains-last
[3] https://sifted.eu/articles/europes-healthtech-industry-2020/
[4] https://www.mobihealthnews.com/news/emea/health-tech-investments-will-continue-rise-2020-according-silicon-valley-bank
[5] https://news.crunchbase.com/news/for-health-tech-startups-data-is-their-lifeline-now-more-than-ever/
[6] https://www.accenture.com/us-en/insight-artificial-intelligence-healthcare%C2%A0
[7] https://www.grandviewresearch.com/industry-analysis/wearable-medical-devices-market
[8] https://www.marketsandmarkets.com/PressReleases/iot-healthcare.asp
[9] https://www.grandviewresearch.com/press-release/global-mhealth-app-market
[10] https://www.globenewswire.com/news-release/2020/05/23/2037920/0/en/Global-Digital-Health-Market-was-Valued-at-USD-111-4-billion-in-2019-and-is-Expected-to-Reach-USD-510-4-billion-by-2025-Observing-a-CAGR-of-29-0-during-2020-2025-VynZ-Research.html
[11] https://www2.stardust-testing.com/en/the-digital-transformation-trends-and-challenges-in-healthcare
[12] https://www.prnewswire.com/news-releases/healthcare-analytics-market-size-to-reach-usd-40-781-billion-by-2025–cagr-of-23-55—valuates-reports-301041851.html#:~:text=Healthcare%20Big%20Data%20Analytics%20Market,13.6%25%20during%202019%2D2025
[13] https://www.globenewswire.com/news-release/2020/11/25/2133473/0/en/Global-Digital-Health-Market-Report-2020-Market-is-Expected-to-Witness-a-37-1-Spike-in-Growth-in-2021-and-will-Continue-to-Grow-and-Reach-US-508-8-Billion-by-2027.html
[14] https://www.nasdaq.com/articles/iomt-meets-new-healthcare-needs%3A-3-medtech-trends-to-watch-2020-11-27
[15] https://go.forrester.com/blogs/predictions-2021-technology-diversity-drives-iot-growth/
[16] https://www.prnewswire.com/news-releases/state-of-the-edge-forecasts-edge-computing-infrastructure-marketworth-700-billion-by-2028-300969120.html
[17] https://news.crunchbase.com/news/for-health-tech-startups-data-is-their-lifeline-now-more-than-ever/
[18] https://www.gartner.com/en/newsroom/press-releases/2020-05-21-gartner-says-50-percent-of-us-healthcare-providers-will-invest-in-rpa-in-the-next-three-years
[19] https://www.idc.com/getdoc.jsp?containerId=prUS46794720
[20] https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/the-great-acceleration-in-healthcare-six-trends-to-heed
[21] https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/the-next-wave-of-healthcare-innovation-the-evolution-of-ecosystems
[22] https://www.cbinsights.com/research/internet-of-medical-things-5g-edge-computing-changing-healthcare/
[23] https://siliconangle.com/2020/12/08/future-state-edge-computing/
[24] https://www.idc.com/getdoc.jsp?containerId=prUS46878020
[25] https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/telehealth-a-quarter-trillion-dollar-post-covid-19-reality
[26] https://go.forrester.com/blogs/will-virtual-care-stand-the-test-of-time-if-youre-asking-the-question-its-time-to-catch-up/
[27] https://www.computerworld.com/article/3529427/how-iot-is-becoming-the-pulse-of-healthcare.html

What are Time Series Database Use Cases?

by Alyssa | Oct 20, 2020 | Edge Computing, Insights, IoT, Time Series

What do self-driving cars, smart homes, autonomous stock/crypto trading algorithms, or energy sensor systems have in common? These applications are all based on a form of data that measures how things change over time. It’s called time-series data and it plays a very important role in our lives today.

Accordingly, time-series databases also became a hot topic.

What is a time-series database?

A time-series database (TSDB) can be defined simply as a database optimized for storing and using time-stamped or time-series data. You don’t need to use a TSDB to work with time-series data. Any relational or NoSQL database or a key-value-store will do, e.g. MongoDB or redis. However, when dealing with time-series data (e.g. temperature, air pressure or car velocity data), a TSDB makes your life as a developer a hell of a lot easier.

Indeed, the two main reasons why TSDBs is the fastest-growing category of databases, are usability and scalability. A purpose-built time-series database typically includes common functions of time-series data analysis, which is convenient when working with time-series data. Because time-series data typically continually produces new data entries, data grows pretty quickly, and with high-frequency data or many time-series data sources, data ingestion quickly becomes a challenge. Time-series databases are optimized to scale well for time-series data with time being a common denominator and outperform any other database without specific time-series optimizations. This is why more and more people are adopting time-series databases and using them for a variety of use cases.

What are time-series database use cases?

Monitoring sensor data

One of the use cases is the monitoring of sensor data for safety measurements, predictive maintenance, or assistance functions. E.g. a car stores and uses all kinds of sensor data like tyre pressure, surrounding temperature and humidity for driver assistance and maintenance support. An aircraft monitors gravity and aerodynamic principles to reassure pilots that everything is alright – or to alert them that something has gone wrong. In fact, a Boeing creates on average half a terabyte of data per flight, most of which is time-series data. [1]

Tracking assets

Tracking assets is ideal for a time-series database as you constantly want to monitor where assets are, e.g. the cars of a fleet or any goods you might be stocking or shipping. These applications typically include unique vehicle or asset IDs, GPS coordinates, and additional metadata per timestamp. Apart from keeping track of the assets in realtime, you also can use the data for logistics and optimize e.g. your stocking and delivery processes.

Analyzing and predicting shopping behavior

Or, many e-commerce systems store all information of an item from product inventory, logistics data and any available environmental data to transaction amount, all items of the shopping cart purchased, to payment data, order information etc. In this case, a TSDB will be used to collect these large amounts of data and analyze them quickly to determine e.g. what to recommend to customers to buy next or optimize the inventory or predict future shopping behavior.

What are the most popular time series databases?

Well, here is our list of popular / established time series databases to use in 2020 to get you started:

InfluxDB: an open-source time series database, written in Go and optimized for high-availability storage and retrieval of time series data for operations monitoring, application metrics, IoT sensor data, and real-time analytics
KairosDB: a fast distributed scalable time series database written on top of Cassandra.
Kdb+: is a column-based relational time series database with a focus on applications in the financial sector.
Objectbox TS: superfast object persistence with time-series data on the edge. Collect, store, and query time-series data on the edge and sync selective data to / from a central location on-premise or in the cloud as needed.
TimescaleDB: an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL and packaged as a PostgreSQL extension with full SQL support.

For an overview of time-series databases currently available for productive use, see DB Engines. The database of databases is also a good resource if you are deeply interested in the database landscape; it is more extensive, but it includes any DB available independent of the level of support or if it is still maintained, also hobby projects.

What do you do when you have more than just time-series data?

Typically, a time-series database is not well suited to model non-time-based data. Therefore, many companies choose to implement two databases. This increases overhead, disk space, and is especially impractical when you deal with edge devices.

Time Series + Object-Oriented Data Persistence

Storing and processing both time series data and objects, developers can collect complex datasets and combine them with time-series data. Combining these data types gives a more complete understanding and context to the data – not just what happens over time, but also other factors that affect the results.

The best option is a robust object-oriented database solution that lets you model your data as it reflects the factual use case / the real world in objects and on-top is optimized for time series data. You can model your world in objects and combine this with the power of time-series data to identify patterns in your data. If this is indeed a database optimized for restricted devices and Edge Computing, you can even use this data in real-time and on the device. By combining time series data with more complex data types, an object time-series edge database can empower new use cases on the edge based on a fast and easy all-in-one data persistence solution.

Still have questions? Feel free to contact us here!

—–

[1] Time Series Management Systems: A Survey Søren Kejser Jensen, Torben Bach Pedersen, Senior Member, IEEE, Christian Thomsen

« Older Entries

Next Entries »