DBOS

Last updated

DBOS is a Database-Oriented Operating System designed to simplify and improve the scalability, security and resilience of large-scale distributed applications. [1] [2] It started in 2020 as a joint open source project with MIT, Stanford and Carnegie Mellon University, after a brainstorm between Michael Stonebraker and Matei Zaharia on how to scale and improve scheduling and performance of millions of Apache Spark tasks. [2]

Contents

The basic idea is to run a multi-node multi-core, transactional, highly-available distributed database, such as VoltDB, as the only application for a microkernel, and then to implement scheduling, messaging, file systems and other operating system services on top of the database.

The architectural philosophy is described by this quote from the abstract of their initial preprint:

All operating system state should be represented uniformly as database tables, and operations on this state should be made via queries from otherwise stateless tasks. This design makes it easy to scale and evolve the OS without whole-system refactoring, inspect and debug system state, upgrade components without downtime, manage decisions using machine learning, and implement sophisticated security features. [3]

Stonebraker claims a variety of security benefits, from a "smaller, less porous attack surface", to the ability to log and analyze how the system state changes in real-time due to the transactional nature of the OS. [1] Recovery from a severe bug or an attack can be as simple as rolling back the database to a previous state. And since the database is already distributed, the complexity of orchestration systems like Kubernetes can be avoided.

A prototype was built with competitive performance to existing systems. [4]

DBOS Cloud

In March of 2024, DBOS Cloud became the first commercial service from DBOS Inc. It provides transactional Functions as a Service (FaaS), and is positioned as a competitor to serverless computing architectures like AWS Lambda. DBOS Cloud is currently based on FoundationDB, a fast ACID NoSQL database, running on the Firecracker microVM service from AWS. It provides built-in support for features like multinode scaling and a "time-traveler" debugger that can help track down elusive heisenbugs and works in Visual Studio Code. Another feature is reliable execution, allowing a program to continue running even if the operating system needs to be restarted, and ensuring that no work is repeated. [5]

Firecracker runs on stripped down Linux microkernel via a stripped down KVM hypervisor, so parts of the Linux kernel are still under the covers, but work is ongoing to eliminate them. [6]

DBOS CLoud has been tested running across 1,000 cores running applications. The first API provided is for TypeScript, via the open-source DBOS Transact framework. [6] It provides a runtime with built-in reliable message delivery and idempotency. [7]

Holger Mueller of Constellation Research wondered how well DBOS the company can scale. “Will a small team at DBOS be able to run an OS, database, observability, workflow and cyber stack as good as the combination of the best of breed vendors?” [8]

See also

PICK OS, another implementation of an operating system based on a DB.

Related Research Articles

<span class="mw-page-title-main">PostgreSQL</span> Free and open-source object relational database management system

PostgreSQL, also known as Postgres, is a free and open-source relational database management system (RDBMS) emphasizing extensibility and SQL compliance. PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures. It is supported on all major operating systems, including Linux, FreeBSD, OpenBSD, macOS, and Windows, and handles a range of workloads from single machines to data warehouses or web services with many concurrent users.

<span class="mw-page-title-main">Ingres (database)</span> Database software

Ingres Database is a proprietary SQL relational database management system intended to support large commercial and government applications.

<span class="mw-page-title-main">IBM Db2</span> Relational model database server

Db2 is a family of data management products, including database servers, developed by IBM. It initially supported the relational model, but was extended to support object–relational features and non-relational structures like JSON and XML. The brand name was originally styled as DB2 until 2017, when it changed to its present form.

A database transaction symbolizes a unit of work, performed within a database management system against a database, that is treated in a coherent and reliable way independent of other transactions. A transaction generally represents any change in a database. Transactions in a database environment have two main purposes:

  1. To provide reliable units of work that allow correct recovery from failures and keep a database consistent even in cases of system failure. For example: when execution prematurely and unexpectedly stops in which case many operations upon a database remain uncompleted, with unclear status.
  2. To provide isolation between programs accessing a database concurrently. If this isolation is not provided, the programs' outcomes are possibly erroneous.
<span class="mw-page-title-main">FOSDEM</span> Annual event in Brussels centered on free and open source software development

Free and Open source Software Developers' European Meeting (FOSDEM) is a non-commercial, volunteer-organized European event centered on free and open-source software development. It is aimed at developers and anyone interested in the free and open-source software movement. It aims to enable developers to meet and to promote the awareness and use of free and open-source software.

<span class="mw-page-title-main">Workplace OS</span> Defunct 1990s operating system

Workplace OS is IBM's ultimate operating system prototype of the 1990s. It is the product of an exploratory research program in 1991 which yielded a design called the Grand Unifying Theory of Systems (GUTS), proposing to unify the world's systems as generalized personalities cohabitating concurrently upon a universally sophisticated platform of object-oriented frameworks upon one microkernel. Developed in collaboration with Taligent and its Pink operating system imported from Apple via the AIM alliance, the ambitious Workplace OS was intended to improve software portability and maintenance costs by aggressively recruiting all operating system vendors to convert their products into Workplace OS personalities. In 1995, IBM reported that "Nearly 20 corporations, universities, and research institutes worldwide have licensed the microkernel, laying the foundation for a completely open microkernel standard." At the core of IBM's new unified strategic direction for the entire company, the project was intended also as a bellwether toward PowerPC hardware platforms, to compete with the Wintel duopoly.

K42 is a discontinued open-source research operating system (OS) for cache-coherent 64-bit multiprocessor systems. It was developed primarily at IBM Thomas J. Watson Research Center in collaboration with the University of Toronto and University of New Mexico. The main focus of this OS is to address performance and scalability issues of system software on large-scale, shared memory, non-uniform memory access (NUMA) multiprocessing computers.

In computing, a solution stack or software stack is a set of software subsystems or components needed to create a complete platform such that no additional software is needed to support applications. Applications are said to "run on" or "run on top of" the resulting platform.

<span class="mw-page-title-main">Amazon Elastic Compute Cloud</span> Cloud computing platform

Amazon Elastic Compute Cloud (EC2) is a part of Amazon.com's cloud-computing platform, Amazon Web Services (AWS), that allows users to rent virtual computers on which to run their own computer applications. EC2 encourages scalable deployment of applications by providing a web service through which a user can boot an Amazon Machine Image (AMI) to configure a virtual machine, which Amazon calls an "instance", containing any software desired. A user can create, launch, and terminate server-instances as needed, paying by the second for active servers – hence the term "elastic". EC2 provides users with control over the geographical location of instances that allows for latency optimization and high levels of redundancy. In November 2010, Amazon switched its own retail website platform to EC2 and AWS.

<span class="mw-page-title-main">Michael Stonebraker</span> American computer scientist (born 1943)

Michael Ralph Stonebraker is a computer scientist specializing in database systems. Through a series of academic prototypes and commercial startups, Stonebraker's research and products are central to many relational databases. He is also the founder of many database companies, including Ingres Corporation, Illustra, Paradigm4, StreamBase Systems, Tamr, Vertica and VoltDB, and served as chief technical officer of Informix. For his contributions to database research, Stonebraker received the 2014 Turing Award, often described as "the Nobel Prize for computing."

<span class="mw-page-title-main">Vertica</span> Software company

Vertica is an analytic database management software company. Vertica was founded in 2005 by the database researcher Michael Stonebraker with Andrew Palmer as the founding CEO. Ralph Breslauer and Christopher P. Lynch served as CEOs later on.

Redis is a source-available, in-memory storage, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Because it holds all data in memory and because of its design, Redis offers low-latency reads and writes, making it particularly suitable for use cases that require a cache. Redis is the most popular NoSQL database, and one of the most popular databases overall. Redis is used in companies like Twitter, Airbnb, Tinder, Yahoo, Adobe, Hulu, Amazon and OpenAI.

Amazon Relational Database Service is a distributed relational database service by Amazon Web Services (AWS). It is a web service running "in the cloud" designed to simplify the setup, operation, and scaling of a relational database for use in applications. Administration processes like patching the database software, backing up databases and enabling point-in-time recovery are managed automatically. Scaling storage and compute resources can be performed by a single API call to the AWS control plane on-demand. AWS does not offer an SSH connection to the underlying virtual machine as part of the managed service.

<span class="mw-page-title-main">Juju (software)</span> Open source service orchestration management tool

Juju is a free and open source application modeling tool developed by Canonical Ltd. Juju is an application management system. It was built to reduce the operation overhead of software by facilitating, deploying, configuring, scaling, integrating, and performing operational tasks on public and private cloud services along with bare-metal servers and local container-based deployments.

<span class="mw-page-title-main">OpenShift</span> Cloud computing software

OpenShift is a family of containerization software products developed by Red Hat. Its flagship product is the OpenShift Container Platform — a hybrid cloud platform as a service built around Linux containers orchestrated and managed by Kubernetes on a foundation of Red Hat Enterprise Linux. The family's other products provide this platform through different environments: OKD serves as the community-driven upstream, Several deployment methods are available including self-managed, cloud native under ROSA, ARO and RHOIC on AWS, Azure, and IBM Cloud respectively, OpenShift Online as software as a service, and OpenShift Dedicated as a managed service.

NewSQL is a class of relational database management systems that seek to provide the scalability of NoSQL systems for online transaction processing (OLTP) workloads while maintaining the ACID guarantees of a traditional database system.

<span class="mw-page-title-main">Apache Mesos</span> Software to manage computer clusters

Apache Mesos is an open-source project to manage computer clusters. It was developed at the University of California, Berkeley.

<span class="mw-page-title-main">Genode</span> Free and open-source software operating system

Genode is a free and open-source software operating system (OS) framework consisting of a microkernel abstraction layer and a set of user space components. The framework is notable as one of the few open-source operating systems not derived from a proprietary OS, such as Unix. The characteristic design philosophy is that a small trusted computing base is of primary concern in a security-oriented OS.

<span class="mw-page-title-main">Valkey</span> Freely available in-memory key–value database

Valkey is an open-source in-memory storage, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Because it holds all data in memory and because of its design, Valkey offers low-latency reads and writes, making it particularly suitable for use cases that require a cache. Valkey is the successor to Redis, the most popular NoSQL database, and one of the most popular databases overall. Valkey or its predecessor Redis are used in companies like Twitter, Airbnb, Tinder, Yahoo, Adobe, Hulu, Amazon and OpenAI.

References

  1. 1 2 Werner, John. "Put The OS In The Database: Performance, Cybersecurity And Endurance In The Cloud". Forbes. Retrieved 2023-12-27.
  2. 1 2 Clark, Lindsay. "Postgres pioneer promises to upend the database once more". www.theregister.com. Retrieved 2023-12-27.
  3. ↑ Cafarella, Michael; DeWitt, David; Gadepally, Vijay; Kepner, Jeremy; Kozyrakis, Christos; Kraska, Tim; Stonebraker, Michael; Zaharia, Matei (2020-07-21), DBOS: A Proposal for a Data-Centric Operating System, arXiv: 2007.11112
  4. ↑ Skiadopoulos, Athinagoras; Li, Qian; Kraft, Peter; Kaffes, Kostis; Hong, Daniel; Mathew, Shana; Bestor, David; Cafarella, Michael; Gadepally, Vijay; Graefe, Goetz; Kepner, Jeremy; Kozyrakis, Christos; Kraska, Tim; Stonebraker, Michael; Suresh, Lalith (2021-09-01). "DBOS: a DBMS-oriented operating system". Proceedings of the VLDB Endowment. 15 (1): 21–30. doi:10.14778/3485450.3485454. ISSN   2150-8097. S2CID   245827586.
  5. ↑ Wayne Williams (2024-03-17). "'What if the operating system is the problem': Linux was never created for the cloud — so engineers developed DBOS, a new operating system that is part OS, part database". TechRadar. Retrieved 2024-04-22.
  6. 1 2 Morgan, Timothy Prickett (2024-03-12). "The Cloud Outgrows Linux, And Sparks A New Operating System". The Next Platform. Retrieved 2024-04-22.
  7. ↑ dbos-inc/dbos-transact, DBOS, Inc., 2024-04-22, retrieved 2024-04-22
  8. ↑ Ghoshal, Anirban (2024-03-12). "DBOS Cloud overturns database-on-OS conventions for speed". InfoWorld. Retrieved 2024-04-22.

Official website

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.