What is Data Synchronization + How to Keep Data in Sync

What is Data Synchronization + How to Keep Data in Sync

What is Data Sync / Data Synchronization in app development?

Data Synchronization (Sync) is the process of establishing consistency and consolidation of data between different devices and fundamental to most IT solutions, especially in IoT and Mobile. Data Sync entails the continuous harmonization of data over time. It typically is a highly complex, non-trivial technical process. Even corporates struggle with its implementation and had to roll back synchronization solutions due to technical challenges. 

The question Data Sync answers is

phone-data-sync-with-machine-payment-automatic-data

How do you keep data sets from two (or more) data stores / databases – separated by space and time – mirrored with one another as closely as possible, in the most efficient way?

In solving this technical challenge one must deal with asynchrony, conflicts, slow bandwidth, flaky networks, third-party applications, and file systems that have different semantics.

Data Sync versus Data Replication in Databases

sync-data-better-than-replication

Data replication is the process of storing the same data in several locations to prevent data loss and improve data availability and accessibility. Typically, data replication means that all data is fully mirrored / backed up / replicated on another instance (device/server). This way, all data is stored at least twice. Replication typically works in one direction only (unidirectional); there is no additional logic to it and no possibility of conflicts.

In contrast, synchronization typically relates to a subset of the data (selection) and works in two directions (bi-directional), adding a layer of complexity, because now conflicts can arise. Of course, if you select all data for synchronisation into one direction, it will yield the same result as replication. However, replication cannot replace synchronization.

Why do you need to keep data in sync?

Think about it – if clocks were not in sync, everyone would live on a different time. While I can see an upside to this, it would result in many inefficiencies as you could not rely on schedules. When business data is not in sync (up-to-date everywhere), it harms the efficiency of the organization due to:

  • Isolated data silos
  • Conflicting data / information states
  • Duplicate data / double effort
  • Outdated information states / incorrect data

In the end, the members of such an organization would not be able to communicate and collaborate efficiently with each other. They would instead be spending a lot of time on unnecessary work and “conflict resolution”. On top, management would miss an accurate overview and data-driven insights to prioritize and steer the company. The underlying mechanism that keeps data up-to-date across devices is a technical process called data synchronization (Sync). And while we expect these processes to “just work”, someone needs to implement and maintain them, which is a non-trivial task.

Growing data masses and shifts in data privacy requirements call for sensible usage of network bandwidth and the cloud. Edge computing with selective data synchronization is an effective way to manage which data is sent to the cloud, and which data stays on the device. Keeping data on the edge and synchronizing selective data sets effectively, reduces the data volume that is transferred via the network and stored in the cloud. Accordingly, this means lower mobile networking and cloud costs. On top, it also enables higher data security and data privacy, because it makes it easy to store personal and private data with the user. When data stays with the user, data ownership is clear too.

Unidirectional Data Replication

replication-data-sync-database

Bidirectional Data Synchronization

how-to-sync-data-what-is-data-sync

Out-of-the-box Sync magic: Syncing is hard

Almost every mobile app or IoT application needs to sync data, so every developer is aware of the basic concept and challenges. This is why many experienced developers appreciate out-of-the-box solutions. While JSON / REST offers a great concept to transfer data, there is more to data Sync than what it looks like at a glance. Of course, the complexity of Sync varies widely depending on the use case. For example, the amount of data, data changes, synchronous / asynchronous sync, and number of devices (connections), and what kind of client-server or peer-to-peer setup is needed, all affect the complexity.

iceburg-building-data-synchronization

What looks easy in practice hides a complex bit of coding and opens a can of worms for testing. For an application to work seamlessly across devices – independent of the network, which can be offline, flaky, or only occasionally connected – an app developer must anticipate and handle a host of local and network failures to ensure data consistency. Moreover, for devices with restricted memory, battery and/or CPU resources (i.e. Mobile and IoT devices), resource sensitivity is also essential. Data storage and synchronization solutions must be both effective / efficient, and sustainable.

How to Keep Data in Sync Without the Headache?

Thankfully, there are out-of-the-box data synchronization solutions available on the market, which solve data syncing for developers. They fall broadly into two categories: cloud-dependent data synchronization, and independent, “edge” data synchronization. Cloud-based solutions, like Firebase, require a connection to the internet to function; data is synchronized to the back-end server to be stored. Edge solutions, like ObjectBox, can also operate offline; data is stored in an efficient on-device database, synchronization occurs only once the device goes online. There are only a few solutions to offer true edge synchronization, i.e. “peer to peer” data sync, directly between any two devices. Below, we summarize the most popular market offerings for data synchronization (offline and cloud based):

cloud-to-edge-data-sync-options-matrix-alternatives

Firebase

Firebase is a Backend as a Service (BaaS) offering from Google (acquired). Google offers it as a cloud hosted solution for mobile developers. 

Parse

Parse is a BaaS offering that Facebook acquired and shut down. Facebook open sourced the code. The GitHub repository is not officially maintained. You can host Parse yourself or use a Parse hosting service.

Mongo Realm

Realm was acquired by MongoDB in 2019; the Mongo Realm Sync solution is now in Alpha and available hosted with MongoDB.

Couchbase

Couchbase is a Cloud DB, Edge DB and Sync offering that requires the use of Couchbase servers.

ObjectBox

ObjectBox is a DB for any device, from restricted edge devices to servers, and offers an out-of-the-box Sync solution. ObjectBox enables self-hosting on-premise / in the cloud, as well as Offline Sync.

Data Sync, Edge Computing, and the Future of Data

Edge computing means storing and processing data close to the source of the data, i.e. on edge devices. It is a mega-shift happening in computing. Accordingly, the market is growing rapidly with projections estimating continuing growth with a 34% CAGR for the next five years. The move from the cloud to the edge is strongly driven by new use cases and growing data masses Edge data persistence and Data Sync, especially “Offline Sync”, are the key technologies needed for Edge Computing. Using edge data persistence, data can be stored and processed on the edge. This means the application always works, independent from a network connection, offline. Faster response times can be guaranteedWith Offline Sync, data can be synchronized between several edge devices, e.g. IoT gateways, smartphones, machines locally. Using Data Sync, selected data can be synchronized with  a central server, on premise or in the cloud. By exchanging less data with the cloud or a central instance, data synchronization reduces the burden on the network. This brings down mobile network and cloud costs, and reduces the amount of energy used: a win-win-win solution.

Digital Healthcare – a look at the market, projections, and trends with in-depth white paper

Digital Healthcare – a look at the market, projections, and trends with in-depth white paper

If you work in the healthcare industry, you are likely familiar with some uses of IoT devices. According to Gartner (2020), 79% of healthcare providers are already successfully employing IoT solutions.[1] However, this is just the beginning. While before COVID-19, the growth of digital health adoption had stalled [2], the market is picking up speed again. Indeed, Q3 2020 was a record year for investments in healthcare companies [3] and the market expects rising investments in healthtech for next years [4]. Today, underutilized data plays a major role in healthtech innovation [17] and the growing importance of healthcare data for future offerings is evident [5]. Take a look how analyts from Gartner to Accenture and Forrester expect the market to grow:

The digital healthcare market 2020 and beyond

digital-healthcare-market-trends-2020-edge-iot
  • Analysts expect Artificial Intelligence in healthcare to reach $6.6 billion by 2021 (with a 40% CAGR). [6]
  • The Internet of Medical Things (IoMT) market is expected to cross $136 billion by 2021. [11
  • Analysts expect the healthcare wearable market to have a market volume of $27 billion by 2023 (with a 27.9% CAGR). [7]
  • The IoT industry is projected to be worth $6.2 trillion by 2025 and around 30% of that market (or about $167 billion) will come from healthcare. [8]
  • Analysts expect the global Medical Health Apps market to grow to $236 billion by 2026, reflecting a shift towards value based care. [9]
  • The projected global digital health market is estimated to reach $510.4 billion by 2026 (with a 29% CAGR). [10]

The Healthcare industry has been struggling with shrinking payments and cost optimizations for years. [18] Fueled by the need to adopt in light of the COVID pandemic, digital technologies bring extensive changes quickly to this struggling industry now. Data is moving to the center of this changing ecosystem and harbors both risks and opportunities in a new dimension. [21] The basic architecture and infrastructure to have the data reliably, securely and quickly available where they are needed will be decisive for the success or failure of digital healthcare solutions. [17] [21]

We recommend keeping an eye on the following five trends

The 5 biggest digital healthcare trends to watch

AI-health-growth-market-tech

Artificial Intelligence (AI)  

Accenture estimates that AI applications can help save up to $150 billion annually for the US healthcare economy by 2026. [6] Therefore, it is no wonder that the healthcare sector is expected to be among the top five industries investing in AI in the next couple of years. [19] The top three greatest near-term value AI applications in healthcare are: 1. robot-assisted surgery ($40 billion), 2. virtual nursing assistants ($20 billion), and 3. administrative workflow assistance ($18 billion). 

big-data-health-analytics

Big Data / Analytics

The goal of big data analytic solutions is to improve the quality of patient care and the overall healthcare ecosystem. The global healthcare Big Data Analytics market is predicted to reach $39 billion by 2025. [12] The main areas of growth are medical data generation in the form of Electronic Health Records (EHR), biometric data, sensors data. 

internet-of-medical-things-digital-healthtech

Internet of Medical Things (IoMT)

IoMT is expected to grow to $508.8 billion by 2027. [13] According to Gartner, 79% of healthcare providers are already using IoT in their processes. [27] During COVID, IoMT devices have been used to increase safety and efficiency in healthcare, i.e. providing and automating clinical assistance and treatment to the infected patient, to lessen the burden of specialists. Future applications, like augmented reality glasses that assist during surgery, are leading to a focus more on IoMT-centric investments. [14]

telemedicine-virtual-healthcare-online

Telehealth / Telemedicine

Telecommunications technology enables doctors to diagnose and treat patients remotely. Consumer adoption of telehealth has skyrocketed in 2020 and McKinsey believes that up to $250 billion of current US healthcare spend could potentially be virtualized. [25] Also, many patients view telehealth offerings more favorable and – having made good experiences – are planning to continue using telehealth in the future. [26] Not astonishingly, telemedicine stocks also grow rapidly. [14]

edge-computing-hospital-clinic-offline

Edge Computing

Edge computing is a technological megashift happening in computing. [23] Instead of pushing data to the cloud to be computed, processing is done locally, on ‘the edge’. [15] Edge Computing is one of the key technologies to make healthcare more connected, secure, and efficient. [22]  Indeed, the digital healthcare ecosystem of the future depends on an infrastructure layer that makes health data accessible when needed where needed (data liquidity). [21] Accordingly, IDC expects the worldwide edge computing market to reach $250.6 billion in 2024 with a (12.5% CAGR) [24with healthcare identified as one of the leading industries that will adopt edge computing. [16

In-depth market overview with a look at the major market challenges and in-depth use cases

The healthcare market is in the middle of a fast digital transformation process. Drivers such as COVID,  growing IoT adoption in healthcare, and underlying social mega-trends are pushing digital healthcare growth to new heights. Therefore, the digital healthcare industry faces many challenges, both technical and regulatory. At the same time the healthcare market is offered a wealth of opportunities.

“While the challenges are numerous, leaders who seize the mindset that “disruptive change provides an opportunity to separate yourself from the pack” will build organizations meaningfully stronger than the ones they ran going into the crisis.” [20]

Interested in getting an in-depth fresh assessment of the digital healthcare landscape, including market drivers, biggest challenges and detailed use cases? 

Access your copy of the Digital Healthcare in 2020: Digitalization, IoT and Edge Computing white paper by singing up below:

By submitting this form, you agree to ObjectBox’ privacy policies.

Digital Healthcare whitepaper, table of contents

The Digital Healthcare Market 2020 and Beyond, a High Speed Revolution

Drivers of Growth and Change in Digital Healthcare

– COVID
– Growing Medical IoT Device Adoption
– The Explosion of Health Data
– Technological innovations: Edge Computing, AI, and VR
– Underlying Social Megatrends

The Challenges of Digital Healthcare / Healthtech

– Data Volume and Availability
– Reliability, Privacy, and Data Security
– IT Costs

Why Edge Computing is Integral to Data-driven Healthcare Ecosystems

– A quick look at Cloud and Edge Computing
– How Edge Computing contributes to resilient and secure healthcare data systems
– Transforming Healthcare with Edge Computing
– Use Case: Secure and Up to Date Digital Record Keeping in Doctors Offices
– Use Case: Connected Ambulances – Real Time Data from Home to Hospital

 

Digital Healthcare: Key Take-Aways

References

[1] https://www.computerworld.com/article/3529427/how-iot-is-becoming-the-pulse-of-healthcare.html / https://www.gartner.com/en/documents/3970072
[2] https://www.accenture.com/us-en/insights/health/leaders-make-recent-digital-health-gains-last
[3] https://sifted.eu/articles/europes-healthtech-industry-2020/
[4] https://www.mobihealthnews.com/news/emea/health-tech-investments-will-continue-rise-2020-according-silicon-valley-bank
[5] https://news.crunchbase.com/news/for-health-tech-startups-data-is-their-lifeline-now-more-than-ever/
[6] https://www.accenture.com/us-en/insight-artificial-intelligence-healthcare%C2%A0
[7] https://www.grandviewresearch.com/industry-analysis/wearable-medical-devices-market
[8] https://www.marketsandmarkets.com/PressReleases/iot-healthcare.asp
[9] https://www.grandviewresearch.com/press-release/global-mhealth-app-market
[10] https://www.globenewswire.com/news-release/2020/05/23/2037920/0/en/Global-Digital-Health-Market-was-Valued-at-USD-111-4-billion-in-2019-and-is-Expected-to-Reach-USD-510-4-billion-by-2025-Observing-a-CAGR-of-29-0-during-2020-2025-VynZ-Research.html
[11] https://www2.stardust-testing.com/en/the-digital-transformation-trends-and-challenges-in-healthcare
[12] https://www.prnewswire.com/news-releases/healthcare-analytics-market-size-to-reach-usd-40-781-billion-by-2025–cagr-of-23-55—valuates-reports-301041851.html#:~:text=Healthcare%20Big%20Data%20Analytics%20Market,13.6%25%20during%202019%2D2025 
[13] https://www.globenewswire.com/news-release/2020/11/25/2133473/0/en/Global-Digital-Health-Market-Report-2020-Market-is-Expected-to-Witness-a-37-1-Spike-in-Growth-in-2021-and-will-Continue-to-Grow-and-Reach-US-508-8-Billion-by-2027.html
[14] https://www.nasdaq.com/articles/iomt-meets-new-healthcare-needs%3A-3-medtech-trends-to-watch-2020-11-27
[15] https://go.forrester.com/blogs/predictions-2021-technology-diversity-drives-iot-growth/
[16] https://www.prnewswire.com/news-releases/state-of-the-edge-forecasts-edge-computing-infrastructure-marketworth-700-billion-by-2028-300969120.html
[17] https://news.crunchbase.com/news/for-health-tech-startups-data-is-their-lifeline-now-more-than-ever/ 
[18] https://www.gartner.com/en/newsroom/press-releases/2020-05-21-gartner-says-50-percent-of-us-healthcare-providers-will-invest-in-rpa-in-the-next-three-years
[19] https://www.idc.com/getdoc.jsp?containerId=prUS46794720 
[20] https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/the-great-acceleration-in-healthcare-six-trends-to-heed 
[21] https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/the-next-wave-of-healthcare-innovation-the-evolution-of-ecosystems 
[22] https://www.cbinsights.com/research/internet-of-medical-things-5g-edge-computing-changing-healthcare/
[23] https://siliconangle.com/2020/12/08/future-state-edge-computing/
[24] https://www.idc.com/getdoc.jsp?containerId=prUS46878020
[25] https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/telehealth-a-quarter-trillion-dollar-post-covid-19-reality
[26] https://go.forrester.com/blogs/will-virtual-care-stand-the-test-of-time-if-youre-asking-the-question-its-time-to-catch-up/
[27] https://www.computerworld.com/article/3529427/how-iot-is-becoming-the-pulse-of-healthcare.html

 

ObjectBox Recognized as a Sustainable Profitable Tech Solution by the Solar Impulse Foundation

ObjectBox Recognized as a Sustainable Profitable Tech Solution by the Solar Impulse Foundation

ObjectBox is proud to be officially recognized as a sustainable and efficient solution by the Solar Impulse Foundation. Although we have self-identified as a #sustainabletech company since our induction, we’re proud to be recognized as an “efficient, clean and profitable solutions with a positive impact on environment and quality of life,” after taking part in an in-depth technical and business evaluation with the team at the Solar Impulse Foundation.

Empowering tech innovation

This label recognizes that ObjectBox empowers innovation with a highly efficient and sustainable technology. The Solar Impulse Efficient Label identifies sustainable tech solutions from around the world to help companies choose their tech stack responsibly.  

solar-impulse-foundation-label-sustainable-software-for-the-edge

UN Sustainable Development Goals

All Solar Impulse awardees contribute to one or several of the UN Sustainable Development Goals; ObjectBox received the globally recognized label for supporting three of the Solar Impulse focused initiatives: 

  • Affordable and Clean Energy: ObjectBox
  • Clean Water and Sanitation
  • Industry, Innovation and Infrastructure : ObjectBox
  • Sustainable Cities and Communities: ObjectBox
  • Responsible Consumption and Production

How is ObjectBox sustainable?

objectbox-local-data-sustainable

ObjectBox enables scalable and sustainable digitalization with a high performance edge database solution and synchronization solution. The ObjectBox database empowers local data storage, while ObjectBox Sync reduces unnecessary data traffic. ObjectBox is therefore ideally suited for efficient, useful, and sustainable Edge Computing. 

Comparing the transmission of the same data sets, ObjectBox saves 20-60% on transmission data volume. By combining delta syncing with efficient compression based on standard and proprietary edge compression methods to keep data small, ObjectBox can reduce device energy consumption and thus CO2 emissions for data transmissions.

As our digital world grows, we all need to do what we can to structure these digital environments in an efficient and sustainable way. ObjectBox helps reduce digital waste. Digital waste unnecessarily burdens bandwidth infrastructure and fills cloud servers, forcing the expansion of cloud farms and in turn, contributing to the pollution of the environment. Therefore, we are excited to be part of the 1000solutions program.

Dr. Vivien Dollinger

CEO and Co-founder, ObjectBox

What does it mean to get a Solar Impulse Label? 

The Solar Impulse Label: a label focused on both the environment and profitability

The first label to assess the economic profitability of products or processes that protect the environment. The Solar Impulse Efficient Solution Label is attributed following a strict selection process performed by external independent experts. By ensuring high standards of sustainability and profitability, this internationally recognized label is considered as a credible marker of quality for solution seekers in business and governments, facilitating their sourcing of solutions to reach environmental commitments.

About the Solar Impulse Foundation

The Solar Impulse Foundation aims to identify clean, efficient and profitable solutions in order to accelerate their implementation and the transition to a sustainable economy. Thanks to the awarding of a label with high standards of sustainability and profitability, the Foundation can support political and economic decision-makers in their efforts to achieve their environmental targets and encourage them to adopt more ambitious energy regulations, necessary for implementation at large-scale of these solutions on the market. A way to take the success of the first round-the-world solar flight further.

white-leaf

Interesting in finding out how ObjectBox can make your edge computing project more sustainable?

What are Time Series Database Use Cases?

What are Time Series Database Use Cases?

What do self-driving cars, smart homes, autonomous stock/crypto trading algorithms, or energy sensor systems have in common? These applications are all based on a form of data that measures how things change over time. It’s called time-series data and it plays a very important role in our lives today.

Accordingly, time-series databases also became a hot topic.

time series database use cases

What is a time-series database?

A time-series database (TSDB) can be defined simply as a database optimized for storing and using time-stamped or time-series data. You don’t need to use a TSDB to work with time-series data. Any relational or NoSQL database or a key-value-store will do, e.g. MongoDB or redis. However, when dealing with time-series data (e.g. temperature, air pressure or car velocity data), a TSDB makes your life as a developer a hell of a lot easier.

Indeed, the two main reasons why TSDBs is the fastest-growing category of databases, are usability and scalability. A purpose-built time-series database typically includes common functions of time-series data analysis, which is convenient when working with time-series data. Because time-series data typically continually produces new data entries, data grows pretty quickly, and with high-frequency data or many time-series data sources, data ingestion quickly becomes a challenge. Time-series databases are optimized to scale well for time-series data with time being a common denominator and outperform any other database without specific time-series optimizations. This is why more and more people are adopting time-series databases and using them for a variety of use cases.

What are time-series database use cases?

Monitoring Use Case time series

Monitoring sensor data 

One of the use cases is the monitoring of sensor data for safety measurements, predictive maintenance, or assistance functions. E.g. a car stores and uses all kinds of sensor data like tyre pressure, surrounding temperature and humidity for driver assistance and maintenance support. An aircraft monitors gravity and aerodynamic principles to reassure pilots that everything is alright – or to alert them that something has gone wrong. In fact, a Boeing creates on average half a terabyte of data per flight, most of which is time-series data.  [1]

Logistics Use Case time series database

Tracking assets

Tracking assets is ideal for a time-series database as you constantly want to monitor where assets are, e.g. the cars of a fleet or any goods you might be stocking or shipping. These applications typically include unique vehicle or asset IDs, GPS coordinates, and additional metadata per timestamp. Apart from keeping track of the assets in realtime, you also can use the data for logistics and optimize e.g. your stocking and delivery processes.

edge time series ecommerce

Analyzing and predicting shopping behavior

Or, many e-commerce systems store all information of an item from product inventory, logistics data and any available environmental data to transaction amount, all items of the shopping cart purchased, to payment data, order information etc. In this case, a TSDB will be used to collect these large amounts of data and analyze them quickly to determine e.g. what to recommend to customers to buy next or optimize the inventory or predict future shopping behavior.

What are the most popular time series databases?

Well, here is our list of popular / established time series databases to use in 2020 to get you started:

  • InfluxDB: an open-source time series database, written in Go and optimized for high-availability storage and retrieval of time series data for operations monitoring, application metrics, IoT sensor data, and real-time analytics
  • KairosDB: a fast distributed scalable time series database written on top of Cassandra. 
  • Kdb+:  is a column-based relational time series database with a focus on applications in the financial sector.
  • Objectbox TS: superfast object persistence with time-series data on the edge. Collect, store, and query time-series data on the edge and sync selective data to / from a central location on-premise or in the cloud as needed.
  • TimescaleDB: an open-source database designed to make SQL scalable for time-series data. It is engineered up from PostgreSQL and packaged as a PostgreSQL extension with full SQL support.

For an overview of time-series databases currently available for productive use, see DB Engines. The database of databases is also a good resource if you are deeply interested in the database landscape; it is more extensive, but it includes any DB available independent of the level of support or if it is still maintained, also hobby projects. 

Time Series Database Use Cases

What do you do when you have more than just time-series data?

Typically, a time-series database is not well suited to model non-time-based data. Therefore, many companies choose to implement two databases. This increases overhead, disk space, and is especially impractical when you deal with edge devices. 

Time Series + Object-Oriented Data Persistence

Storing and processing both time series data and objects, developers can collect complex datasets and combine them with time-series data. Combining these data types gives a more complete understanding and context to the data – not just what happens over time, but also other factors that affect the results. 

The best option is a robust object-oriented database solution that lets you model your data as it reflects the factual use case / the real world in objects and on-top is optimized for time series data. You can model your world in objects and combine this with the power of time-series data to identify patterns in your data. If this is indeed a database optimized for restricted devices and Edge Computing, you can even use this data in real-time and on the device. By combining time series data with more complex data types, an object time-series edge database can empower new use cases on the edge based on a fast and easy all-in-one data persistence solution. 

Still have questions? Feel free to contact us here!

—–

[1] Time Series Management Systems: A Survey Søren Kejser Jensen, Torben Bach Pedersen, Senior Member, IEEE, Christian Thomsen

What Drives Edge Computing?

What Drives Edge Computing?

Data is exploding in every respect: in data volume, data velocity, and data variety (the 3 v’s). One driver of this phenomenon is the growing number of Mobile and IoT devices and thus, data sources. Making this data useful is one of the driving forces behind the adoption of Edge Computing. New use cases don’t only rely on using this data, but also upon the usability and speed of usability of this ever growing data. There are several practical challenges with this growing data volume that drive the adoption of Edge Computing:

New Use Cases Drive Edge Computing

what-drives-edge-computing

Bandwidth Limitations

The existing network infrastructure cannot support sending all the data to the cloud. Particularly in urban areas there is a concentration of devices and data overburdens existing infrastructure. While 5G promises some relief, it is no hailbringer. First of all, if you want to implement your IoT project now, 5G is not yet available and many questions about 5G remain, e.g. pricing. But moreover, as the number of devices and data is growing ever faster, it is already clear that data volumes will outpace what 5G can support. Edge Computing will be an important technology alongside 5G to enable IoT.

Fast Data Requirements  

Response time requirements are growing at the same time as data volumes are increasing. Sending data to the cloud for computation and storage means applications’ response times have a higher latency and depend on the network, which cannot guarantee response rates. Edge computing guarentees significantly faster data. Use cases that need speedy response times or guaranteed responses cannot rely on cloud computing. For example, in driver assistance, where every millisecond counts or in factory floors, where downtimes are too costly.

Sustainability

Sending data to the cloud and storing it there is inefficient and therefore costly – not only in plain €, but with regards to CO2 emissions too. The distance the data needs to travel needs hardware, connectivity and electric power. Therefore, sending data unnecessarily back and forth is wasteful beaviour and burdens the environment unnecessarily. With growing data volumes, that burden is growing. In fact, analysts predict  that cloud computing data centers will consume as much as 21% of the total global energy by 2030. [1]

To scale your prototype, you need to move to the edge

At the start of IoT projects, quick prototyping, testing and piloting on early iterations of an application’s functionalities, can effectively be done in the cloud. However, in productive environments when applications scale it is often hard or impossible to keep cloud costs at scale, making the business not viable. Then it is time to move to the edge.

At the same time, decreasing hardware costs and hardware sizes are enabling more complex local computing, meaning there is less need for additional cloud usage. E.g. increasingly AI and ML is done at the edge, including model training.

data-volume-edge-computing-solution-iot-mobile

Data accessibility and Smart Syncing

Today’s successful businesses require a smarter approach to data management and integration. Data synchronization increases operational efficiencies, saving time and resources by eliminating redundant data transfer. With data synchronization, only predefined, useful parts of a data set are sent to a central instance. This means that while large volumes of data can be collected and analyzed locally, not all of this data is sent to and saved in the cloud. This reduces the impact on bandwidth, utilizes the local hardware resources for fast guaranteed response times, and keeps project cloud costs low – ultimately creating a more sustainable and efficient model of data architecture, enabling long term project scalability. 

ObjectBox’ current database technology is enabling companies to persist and use data on edge devices, faster than any alternative on the market. It enables networks of edge devices working without a central instance, enabling even more new use cases.

How Building Green IoT Solutions on the Edge Can Help Save Energy and CO2

How Building Green IoT Solutions on the Edge Can Help Save Energy and CO2

The internet of things (IoT) has a huge potential to reduce carbon emissions, as it enables new ways of operating, living, and working [1] that are more efficient and sustainable. However, IoT’s huge and growing electricity demands are a challenge. This demand is due primarily to the transmission and storage of data in cloud data centers. [2] While data center efficiency and the use of green energy will reduce the CO2 emissions needed for this practice, it is not addressing the problem directly. [3

iot-data-cloud-energy-waste

With ObjectBox, we address this unseen and fast-growing CO2 source at the root: ObjectBox empowers edge computing, reducing the volume of data transmitted to central data storage, while at the same time, heightening data transmission and storage efficiency. [4] We’ve talked before about how edge computing is necessary for a sustainable future, below we dive into the numbers a bit deeper. TLRD: ObjectBox enables companies to cut the power consumption of their IoT applications, and thus their emissions, by 50 – 90%. For 2025, the potential impact of ObjectBox is a carbon emission reduction of 594 million metric tons (see calculations below).

How ObjectBox’ Technology Reduces Overall Data Transmission

 ObjectBox reduces data transmission in two ways: 1. ObjectBox reduces the need for data transmission, 2. ObjectBox makes data transmission more efficient. ObjectBox’ database solution allows companies to build products that store and process data on edge devices and work with that data offline (as well as online). This

Green IoT Solution

not only improves performance and customer experience, it also reduces the overall volume of data that is being sent to the cloud, and thus the energy needed to transfer the data as well as store it in the cloud. ObjectBox’ Synchronization solution makes it easy for companies to transmit only the data that needs to be transmitted through 1) selective two-way syncing and 2) differential delta syncing. Synchronizing select data reduces the energy required for unnecessarily transmitting all data to the cloud.

We have demonstrated in exemplary case studies that ObjectBox can reduce total data transmissions by 70-90%, depending on the case. There will, however, typically be value in transmitting some parts of data to a central data center (cloud); ObjectBox Sync combines efficient compression based on standard and proprietary edge compression methods to keep this data small. ObjectBox also has very little overhead. Comparing the transmission of the same data sets, ObjectBox saves 40-60% on transmission data volume through the delta syncing and compression, and thus saves equivalent CO2 emissions for data transmissions. Additional studies support these results, and have shown that moving from a centralized to a distributed data structure, saves between 32 and 93% of transmission data. [5

sync-sustainable-data-save-energy

Calculations: How Does ObjectBox Save CO2?

Physically using a device consumes little energy directly; it is the wireless cloud infrastructure in the backend (data center storage and data transmission) that is responsible for the high carbon footprint of mobile phones [6] and IoT devices. Estimates say that IoT devices will produce around 2,8 ZB of data in 2020 (or 2,823,000,000,000  GB), globally. [7] Only a small portion of that data actually gets stored and used; we chose to use a conservative estimate of 5% [8] (141,150,000,000 GB) and of that portion, 90% is transferred to the cloud [9] (127,035,000,000 GB). Transferring 1 GB of data to the cloud and storing it there costs between 3 and 7 kWh. [10] Assuming an average of 5 kWh this means a 127,035,000,000 GB multiplied by 5kWh, resulting in a total energy expenditure of 635,175,000,000 kWh. Depending on the energy generation used, CO2 emissions vary. We are using a global average of 0,475 kgCO2 / 1 kwH. [11] In total this means that there will be 301,708,125,000 KG of CO2, or roughly 301 million metric tons of CO2 produced to transfer data to the cloud and store it there in 2020. 

Projections for 2025 have data volumes as high as 79.4 ZB. [12] Following the same calculations as above, IoT devices would be responsible for 8 billion metric tons of CO2 in 2025.* We estimate that using ObjectBox can cut CO2 caused by data transmission and data centers by 50-90%, by keeping the majority of data on the device, and transmitting data efficiently. It will take time for ObjectBox to enter the market, so assuming a 10% market saturation by 2025 and an average energy reduction of 70%, using ObjectBox could cut projected CO2 emissions by 594 million metric tons in 2025.

ObjectBox is on a mission to reduce digital waste which unnecessarily burdens bandwidth infrastructure and fills cloud servers, forcing the expansion of cloud farms and in turn, contributing to the pollution of the environment. As our digital world grows, we all need to give some thought to how we should structure our digital environments to optimize and support useful, beneficial solutions, while also keeping them efficient and sustainable. 

*Of course, in that time, the technologies will all be more efficient and thus use less electricity while at the same time CO2 emissions / kWh will have dropped too. Thus, we are aware that this projection is an oversimplification of a highly complex and constantly changing system.

[1] https://www.theclimategroup.org/sites/default/files/archive/files/Smart2020Report.pdf
[2] https://www.iea.org/reports/tracking-buildings/data-centres-and-data-transmission-networks
[3]“Data centres… have eaten into any progress we made to achieving Ireland’s 40% carbon emissions reduction target.” from https://www.climatechangenews.com/2017/12/11/tsunami-data-consume-one-fifth-global-electricity-2025/
[4] https://medium.com/stanford-magazine/carbon-and-the-cloud-d6f481b79dfe
[5] https://www.researchgate.net/publication/323867714_The_carbon_footprint_of_distributed_cloud_storage
[6] https://www.resilience.org/stories/2020-01-07/the-invisible-and-growing-ecological-footprint-of-digital-technology/
[7] https://www.idc.com/getdoc.jsp?containerId=prUS45213219, https://priceonomics.com/the-iot-data-explosion-how-big-is-the-iot-data/, https://www.gartner.com/en/newsroom/press-releases/2018-11-07-gartner-identifies-top-10-strategic-iot-technologies-and-trends, https://www.iotjournaal.nl/wp-content/uploads/2017/02/white-paper-c11-738085.pdf, ObjectBox research
[8] Forrester (https://internetofthingsagenda.techtarget.com/blog/IoT-Agenda/Preventing-IoT-data-waste-with-the-intelligent-edge), Harvard BR (https://hbr.org/2017/05/whats-your-data-strategy), IBM (http://www.redbooks.ibm.com/redbooks/pdfs/sg248435.pdf), McKinsey (https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/the-internet-of-things-the-value-of-digitizing-the-physical-world)
[9] https://www.gartner.com/smarterwithgartner/what-edge-computing-means-for-infrastructure-and-operations-leaders/
[10] According to the American Council for an Energy-Efficient Economy: 5,12 kWh of electricity / GB of transferred data. According to a Carnegie Mellon University study: 7 kWh / GB. The American Council for an Energy-Efficient Economy concluded: 3.1 kWh / GB.
[11] https://www.iea.org/reports/global-energy-co2-status-report-2019/emissions
[12] https://www.idc.com/getdoc.jsp?containerId=prUS45213219