Embedded database

Last updated

An embedded database system is a database management system (DBMS) which is tightly integrated with an application software; it is embedded in the application. It is a broad technology category that includes: [1]

Contents

The term embedded database can be confusing because only a tiny subset of embedded database products are used in real-time embedded systems such as telecommunications switches and consumer electronics. [2] (See mobile database for small-footprint databases that could be used on embedded devices.)

Implementations

Major embedded database products include, in alphabetical order:

Storage engine comparison

Advantage Database Server

Sybase's Advantage Database Server (ADS) is an embedded database management system. It provides both Indexed Sequential Access Method (ISAM) and relational data access and is compatible with multiple platforms including Windows, Linux, and Netware. It is available as a royalty-free local file-server database or a full client-server version. ADS is highly scalable, with no administration, and has support for a variety of IDEs including .NET Framework (.NET), Object Pascal (Delphi), Visual FoxPro (FoxPro), PHP, Visual Basic (VB), Visual Objects (VO), Vulcan, Clipper, Perl, Java, xHarbour, etc.

Apache Derby

Derby is an embeddable SQL engine written entirely in Java. Fully transactional and multi-user, Derby is a mature engine and freely available under the Apache license and is actively maintained. Derby project page. It is also distributed as part of Oracle's Java SE Development Kit (JDK) under the name of Java DB.

Empress Embedded Database

Empress Software, Inc., developer of the Empress Embedded Database, is a privately held company founded in 1979. Empress Embedded Database is a full-function, relational database that has been embedded into applications by organizations small to large, with deployment environments including medical systems, network routers, nuclear power plant monitors, satellite management systems, and other embedded system applications that require reliability and power. [3] Empress is an ACID compliant, SQL database engine with C, C++, Java, JDBC, ODBC, SQL, ADO.NET and kernel level APIs. Applications developed using these APIs may be run in standalone and/or server modes. Empress Embedded Database runs on Linux, Unix, Microsoft Windows and real-time operating systems.

Extensible Storage Engine

ESE is an ISAM data storage technology from Microsoft, a core of Microsoft Exchange Server and Active Directory. Its purpose is to allow applications to store and retrieve data via indexed and sequential access. Windows Mail and Desktop Search in the Windows Vista operating system also make use of ESE to store indexes and property information respectively.

eXtremeDB

McObject LLC launched eXtremeDB as the first in-memory embedded database designed from scratch for real-time embedded systems. The initial product was soon joined by eXtremeDB High Availability (HA) for fault tolerant applications. The product family now includes 64-bit and transaction logging editions, and the hybrid eXtremeDB Fusion, which combines in-memory and on-disk data storage. In 2008, McObject introduced eXtremeDB Kernel Mode, the first embedded DBMS designed to run in an operating system kernel. [4] Today, eXtremeDB is used in millions of real-time and embedded systems worldwide. McObject also offers Perst, an open source, object-oriented embedded database for Java, Java ME, .NET, .NET Compact Framework and Silverlight.

Firebird Embedded

Firebird Embedded is a relational database engine. As an open-source fork of InterBase, it is ACID compliant, supports triggers and stored procedures, and is available on Linux, OSX and Windows systems. It has the same features as the classic and superserver version of Firebird; two or more threads (and applications) can access the same database at the same time starting with Firebird 2.5. Therefore, Firebird Embedded acts as a local server for one threaded client accessing its databases (that means it works properly for ASP.NET web applications, because there, each user has its own thread, which means two users could access the same database at the same time, but they would not be in the same thread, because ASP.NET opens a new thread for each user). It exports the standard Firebird API entry points. The main advantage of Firebird Embedded databases is, that unlike SQLite or Access databases, they can be plugged into a full Firebird server without any modifications at all also is multiplatform (runs on Linux, OS X with full ASP.NET Mono support)
Firebird is not truly embedded since it cannot be statically linked

H2

Written in Java Open source database engine. Embedded and Server mode, Clustering support, can run inside the Google App Engine. Supports encrypted database files (AES or XTEA). The development of H2 was started in May 2004, but it was first published on December 14, 2005. H2 is dual licensed and available under a modified version of the MPL 1.1 (Mozilla Public License) or under the (unmodified) EPL 1.0 (Eclipse Public License).

HailDB, formerly Embedded InnoDB

HailDB is a standalone, embeddable form of the InnoDB Storage Engine. Given that HailDB is based on the same code base as the InnoDB Storage Engine, it contains many of the same features, including high-performance and scalability, multiversion concurrency control (MVCC), row-level locking, deadlock detection, fault tolerance and automatic crash recovery. Because the embedded engine is completely independent from MySQL, it lacks server components such as networking, object-level permissions, etc. By eliminating the MySQL server overhead, InnoDB has a small footprint and is well-suited for embedding in applications which require high-performance and concurrency. As with most embedded database systems, HailDB is designed to be accessed primarily with an ISAM-like C API rather than SQL (though an extremely rudimentary SQL variant is supported). [5]

The project is no longer maintained as of 2015. [6]

HSQLDB

HSQLDB is an opensource relational database management system with a BSD-like license that runs in the same Java Virtual Machine as the embedded application. HSQLDB supports a variety of in-memory and disk-based table modes, Unicode, and SQL:2016.

InfinityDB

InfinityDB Embedded Java DBMS is a sorted hierarchical key/value store. It now has an Encrypted edition and a Client/Server edition. The multi-core speed is patent-applied-for. InfinityDB is secure, transactional, compressing, and robust, in a single file for instant installation and zero administration. APIs include the simple fast 'ItemSpace', a ConcurrentNavigableMap view, and JSON. A RemoteItemSpace can transparently redirect the embedded APIs to other db instances. Client/Server includes a light-weight Servlet server, web admin and database browsing, and REST for python.

Informix Dynamic Server

Informix Dynamic Server (IDS) is characterized as an enterprise class embeddable database server, combining embeddable features such as low footprint, programmable and autonomic capabilities with enterprise class database features such as high availability and flexible replication features. [7] IDS is used in deeply embedded scenarios such as IP telephony call-processing systems, point of sale applications and financial transaction processing systems.

InterBase

InterBase is an IoT Award-winning cross-platform, Unicode enabled SQL database platform able to be embedded within turn-key applications. Out of the box SMP support and on disk AES strength 256bit encryption, SQL 92 & ACID compliance and support for Windows, Macintosh, Linux, Solaris, iOS and Android platforms. Ideal for both small-to-medium and large enterprises supporting hundreds of users and mobile application development. InterBase Light is a free version that can be used on any mobile device and is ideal for mobile applications. Enterprises can switch to a paid version as requirements for change management and security increase. InterBase has high adoption in defense, airspace, oil and gas, and manufacturing industries.

LevelDB

LevelDB is an ordered key/value store created by Google as a lightweight implementation of the Bigtable storage design. As a library (which is the only way to use LevelDB), its native API is C++. It also includes official C wrappers for most functionality. Third-party API wrappers exist for Python, PHP, Go (pure Go LevelDB implementation exists but is in progress still), Node.js and Objective C. Google distributes LevelDB under the New BSD License.

LMDB

Lightning Memory-Mapped Database (LMDB) is a memory-mapped key-value database for the OpenLDAP Project. It is written in C and the API is modeled after the Berkeley DB API, though much simplified. The library is extremely compact, compiling down to under 40KB of x86 object code, being usually faster than similar libraries like Berkeley DB, LevelDB, etc. The library implements B+trees with multiversion concurrency control (MVCC), single-level store, Copy on write and provides full ACID transactions with no deadlocks. The library is optimized for high read concurrency; readers need no locks at all. Readers don't block writers and writers don't block readers, so read performance scales perfectly linearly across arbitrarily many threads and CPUs. Third-party wrappers exist for C++, Erlang and Python. LMDB is distributed by the OpenLDAP Project under the OpenLDAP Public License. As of 2013 the OpenLDAP Project is deprecating the use of Berkeley DB, in favor of LMDB.

Mimer SQL

An embedded zero maintenance version of the proprietary Mimer SQL relational database server is available. It has a small footprint due to its modular design, full support for the SQL standard, and with ports to Windows, Linux, Automotive Grade Linux, Android, QNX, INTEGRITY, among others.

MonetDB/e

MonetDB/e is the embedded version of the open source MonetDB SQL column store engine. Available for C, C++, Java (JDBC) and Python. MonetDB License, based on MPL 2.0. The predecessor MonetDBLite (for R, Python and Java) is no longer maintained. It's replaced by MonetDB/e.

MySQL Embedded Server Library

The Embedded MySQL Server Library provides most of the features of regular MySQL as a linkable library that can be run in the context of a client process. After initialization, clients can use the same C API calls as when talking to a separate MySQL server but with less communication overhead and with no need for a separate database process.

NexusDB

NexusDB is the commercial successor to the FlashFiler database which is now open source. They can both be embedded in Delphi applications to create stand-alone executables with full database functionality.

Oracle Berkeley DB

As the name implies, Oracle's embedded database is actually Berkeley DB, which Oracle acquired from Sleepycat Software. It was originally developed at the University of California. [8] Berkeley DB is a fast, open-source embedded database and is used in several well-known open-source products, including the Linux and BSD Unix operating systems, Apache Web server, OpenOffice productivity suite. Nonetheless, over recent years many well-known projects switched to using LMDB, because it outperform Berkeley DB in key scenarios on the ground of "less is more" design, as well due to the license changing. [9]

RocksDB

RocksDB, created at Facebook, began as a fork of LevelDB. [10] It focuses on performance, especially on SSDs. It adds many features, including transactions, [11] backups, [12] snapshots, [13] bloom filters, [14] column families, [15] expiry, [16] custom merge operators, [17] more tunable compaction, [18] statistics collection, [19] and geospatial indexing. [20] It is used as a storage engine inside of several other databases, including ArangoDB, [21] Ceph, [22] CockroachDB, [23] MongoRocks, [24] MyRocks, [25] Rocksandra, [26] TiKV. [27] [28] and YugabyteDB. [29]

solidDB

solid DB is a hybrid on-disk/in-memory, relational database and is often used as an embedded system database in telecommunications equipment, network software, and similar systems. In-memory database technology is used to achieve throughput of tens of thousands of transactions per second with response times measured in microseconds. High availability option maintains two copies of the data synchronized at all times. In case of system failure, applications can recover access to solid DB in less than a second without loss of data.

SQLite

SQLite is a software library that implements a self-contained, server-less, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code, chiefly C, for SQLite is in the public domain. It includes both a native C library and a simple command line client for its database. It's included in several operating systems; among them are Android, FreeBSD, iOS, OS X and Windows 10. [30] It's also used by Chromium web browser and derivatives. [31]

SQL Server Compact

SQL Server Compact is an embedded database by Microsoft with wide variety of features like multi-process connections, T-SQL, ADO.NET Sync Services to sync with any back-end database, Merge Replication with SQL Server, Programming API: LINQ to SQL, LINQ to Entities, ADO.NET. The product runs on both Desktop and Mobile Windows platforms. It has been in the market for long time, used by many enterprises in production software (Case Studies). The product went through multiple re-brandings and was known with multiple names like: SQL CE, SQL Server CE, SQL Server Mobile, SQL Mobile.

See also

Related Research Articles

Berkeley DB (BDB) is an embedded database software library for key/value data, historically significant in open source software. Berkeley DB is written in C with API bindings for many other programming languages. BDB stores arbitrary key/data pairs as byte arrays, and supports multiple data items for a single key. Berkeley DB is not a relational database, although it has database features including database transactions, multiversion concurrency control and write-ahead logging. BDB runs on a wide variety of operating systems including most Unix-like and Windows systems, and real-time operating systems.

<span class="mw-page-title-main">IBM Db2</span> Relational model database server

Db2 is a family of data management products, including database servers, developed by IBM. It initially supported the relational model, but was extended to support object–relational features and non-relational structures like JSON and XML. The brand name was originally styled as DB2 until 2017, when it changed to its present form.

<span class="mw-page-title-main">SQLite</span> Serverless relational database management system (RDBMS)

SQLite is a database engine written in the C programming language. It is not a standalone app; rather, it is a library that software developers embed in their apps. As such, it belongs to the family of embedded databases. It is the most widely deployed database engine, as it is used by several of the top web browsers, operating systems, mobile phones, and other embedded systems.

<span class="mw-page-title-main">Firebird (database server)</span> Relational database management system

Firebird is an open-source SQL relational database management system that supports Linux, Microsoft Windows, macOS and other Unix platforms. The database forked from Borland's open source edition of InterBase in 2000 but the code has been largely rewritten since Firebird 1.5.

The following tables compare general and technical information for a number of relational database management systems. Please see the individual products' articles for further information. Unless otherwise specified in footnotes, comparisons are based on the stable versions without any add-ons, extensions or external programs.

In computing, a solution stack or software stack is a set of software subsystems or components needed to create a complete platform such that no additional software is needed to support applications. Applications are said to "run on" or "run on top of" the resulting platform.

Apache Derby is a relational database management system (RDBMS) developed by the Apache Software Foundation that can be embedded in Java programs and used for online transaction processing. It has a 3.5 MB disk-space footprint.

<span class="mw-page-title-main">H2 (database)</span>

H2 is a relational database management system written in Java. It can be embedded in Java applications or run in client-server mode.

In relational databases, the information schema is an ANSI-standard set of read-only views that provide information about all of the tables, views, columns, and procedures in a database. It can be used as a source of the information that some databases make available through non-standard commands, such as:

 => SELECT count(table_name) FROM information_schema.tables;  count   -------  99    => SELECT column_name, data_type, column_default, is_nullable  FROM information_schema.columns WHERE table_name='alpha';  column_name | data_type | column_default | is_nullable   -------------+-----------+----------------+-------------  foo | integer | | YES  bar | character | | YES    => SELECT * FROM information_schema.information_schema_catalog_name;  catalog_name   --------------  johnd  

Microsoft SQL Server Compact is a discontinued relational database produced by Microsoft for applications that run on mobile devices and desktops. Prior to the introduction of the desktop platform, it was known as SQL Server for Windows CE and SQL Server Mobile Edition.

Web2py is an open-source web application framework written in the Python programming language. Web2py allows web developers to program dynamic web content using Python. Web2py is designed to help reduce tedious web development tasks, such as developing web forms from scratch, although a web developer may build a form from scratch if required.

LevelDB is an open-source on-disk key-value store written by Google fellows Jeffrey Dean and Sanjay Ghemawat. Inspired by Bigtable, LevelDB source code is hosted on GitHub under the New BSD License and has been ported to a variety of Unix-based systems, macOS, Windows, and Android.

eXtremeDB is a high-performance, low-latency, ACID-compliant embedded database management system using an in-memory database system (IMDS) architecture and designed to be linked into C/C++ based programs. It works on Windows, Linux, and other real-time and embedded operating systems.

<span class="mw-page-title-main">Actian Zen</span>

Actian Zen is an ACID-compliant, zero-DBA, embedded, nano-footprint, multi-model, Multi-Platform database management system (DBMS) developed originally by Pervasive Software, which was acquired by Actian Corporation in 2013.

Lightning Memory-Mapped Database (LMDB) is an embedded transactional database in the form of a key-value store. LMDB is written in C with API bindings for several programming languages. LMDB stores arbitrary key/data pairs as byte arrays, has a range-based search capability, supports multiple data items for a single key and has a special mode for appending records (MDB_APPEND) without checking for consistency. LMDB is not a relational database, it is strictly a key-value store like Berkeley DB and DBM.

The following outline is provided as an overview of and topical guide to MySQL:

<span class="mw-page-title-main">RocksDB</span> Embedded key-value database

RocksDB is a high performance embedded database for key-value data. It is a fork of Google's LevelDB optimized to exploit multi-core processors (CPUs), and make efficient use of fast storage, such as solid-state drives (SSD), for input/output (I/O) bound workloads. It is based on a log-structured merge-tree data structure. It is written in C++ and provides official language bindings for C++, C, and Java. Many third-party language bindings exist. RocksDB is free and open-source software, released originally under a BSD 3-clause license. However, in July 2017 the project was migrated to a dual license of both Apache 2.0 and GPLv2 license. This change helped its adoption in Apache Software Foundation's projects after blacklist of the previous BSD+Patents license clause.

<span class="mw-page-title-main">DBeaver</span> Multi-platform database administration software

DBeaver is a SQL client software application and a database administration tool. For relational databases it uses the JDBC application programming interface (API) to interact with databases via a JDBC driver. For other databases (NoSQL) it uses proprietary database drivers. It provides an editor that supports code completion and syntax highlighting. It provides a plug-in architecture that allows users to modify much of the application's behavior to provide database-specific functionality or features that are database-independent. This is a desktop application written in Java and based on Eclipse platform.

References

  1. "What is a Database Model". Lucidchart. Retrieved 2022-11-06.
  2. Graves, Steve. "COTS Databases For Embedded Systems" Archived 2007-11-14 at the Wayback Machine , Embedded Computing Design magazine, January, 2007. Retrieved on August 13, 2008.
  3. Mullins, Craig. "Empress Offers an Effective Embedded Database Solution", 2005. Retrieved on 2008-12-09
  4. Gorine, Andrei and Krivolapov, Alexander. "Kernel Mode Databases: A DBMS Technology For High-Performance Applications", Dr. Dobb's Journal, April, 2008. Retrieved on August 13, 2008.
  5. HailDB homepage
  6. Shutting down HailDB, By Stewart Smith on August 19, 2015, HailDB
  7. "Embedding Informix Dynamic Server", Retrieved on August 30, 2009.
  8. See Berkeley DB
  9. Niccolai, James. "Update: Oracle Buys Sleepycat open-source database vendor" Archived 2008-06-13 at the Wayback Machine , "InfoWorld", 2006-02-14. Retrieved on June 12, 2008.
  10. "RocksDB Basics". GitHub . Retrieved 2018-07-19.
  11. "RocksDB transactions". GitHub. Retrieved 2016-04-04.
  12. "How to backup RocksDB?". GitHub . Retrieved 2017-07-19.
  13. "Checkpoints". GitHub . Retrieved 2017-07-19.
  14. "RocksDB bloom filters". GitHub. Retrieved 2016-04-04.
  15. "Column families in RocksDB". GitHub. Retrieved 2016-04-04.
  16. "RocksDB TTL support". GitHub. Retrieved 2016-04-04.
  17. "RocksDB merge operator". GitHub. Retrieved 2016-04-04.
  18. "Universal compaction". GitHub. Retrieved 2016-04-04.
  19. "RocksDB perf context and IO stats context". GitHub. Retrieved 2016-04-04.
  20. "Spatial indexing in RocksDB". rocksdb.org. Retrieved 2018-07-19.
  21. "Comparing new RocksDB and MMFiles storage engines" . Retrieved 2018-07-19.
  22. "Storage Devices — Ceph Documentation" . Retrieved 2018-07-19.
  23. "Storage Layer - CockroachDB" . Retrieved 2018-07-19.
  24. "mongodb-partners/mongo-rocks: MongoDB storage integration layer for the Rocks storage engine". GitHub . Retrieved 2018-07-19.
  25. "MyRocks - A RocksDB storage engine with MySQL" . Retrieved 2018-07-19.
  26. "Open-sourcing a 10x reduction in Apache Cassandra tail latency". 5 March 2018. Retrieved 2018-07-19.
  27. "RocksDB in TiKV - PingCAP". 15 September 2017. Retrieved 2018-07-19.
  28. "A Glimpse into the World of Embedded Database Feat. RocksDB". 21 November 2019.
  29. Bautin, Mikhail (2019-02-20). "How We Built a High Performance Document Store on RocksDB?". The Distributed SQL Blog. Retrieved 2022-01-09.
  30. Answer, Usman (29 October 2015). "Shipping a New Mindset with SQLite in Windows 10". Microsoft. Archived from the original on 2016-01-31. Retrieved 6 March 2016.
  31. "SQLite abstraction layer". chromium.googlesource.com. Retrieved 2023-09-27.