DBOS

Last updated

DBOS is a Database-Oriented Operating System designed to simplify and improve the scalability, security and resilience of large-scale distributed applications. [1] [2] It started in 2020 as a joint open source project with MIT, Stanford and Carnegie Mellon University, after a brainstorm between Michael Stonebraker and Matei Zaharia on how to scale and improve scheduling and performance of millions of Apache Spark tasks. [2]

Contents

The basic idea is to run a multi-node multi-core, transactional, highly-available distributed database, such as VoltDB, as the only application for a microkernel, and then to implement scheduling, messaging, file systems and other operating system services on top of the database.

The architectural philosophy is described by this quote from the abstract of their initial preprint:

All operating system state should be represented uniformly as database tables, and operations on this state should be made via queries from otherwise stateless tasks. This design makes it easy to scale and evolve the OS without whole-system refactoring, inspect and debug system state, upgrade components without downtime, manage decisions using machine learning, and implement sophisticated security features. [3]

Stonebraker claims a variety of security benefits, from a "smaller, less porous attack surface", to the ability to log and analyze how the system state changes in real-time due to the transactional nature of the OS. [1] Recovery from a severe bug or an attack can be as simple as rolling back the database to a previous state. And since the database is already distributed, the complexity of orchestration systems like Kubernetes can be avoided.

A prototype was built with competitive performance to existing systems. [4]

DBOS Cloud

In March of 2024, DBOS Cloud became the first commercial service from DBOS Inc. It provides transactional Functions as a Service (FaaS), and is positioned as a competitor to serverless computing architectures like AWS Lambda. DBOS Cloud is currently based on FoundationDB, a fast ACID NoSQL database, running on the Firecracker microVM service from AWS. It provides built-in support for features like multinode scaling and a "time-traveler" debugger that can help track down elusive heisenbugs and works in Visual Studio Code. Another feature is reliable execution, allowing a program to continue running even if the operating system needs to be restarted, and ensuring that no work is repeated. [5]

Firecracker runs on stripped down Linux microkernel via a stripped down KVM hypervisor, so parts of the Linux kernel are still under the covers, but work is ongoing to eliminate them. [6]

DBOS Cloud has been tested running across 1,000 cores running applications. The first API provided is for TypeScript, via the open-source DBOS Transact framework. [6] It provides a runtime with built-in reliable message delivery and idempotency. [7]

Holger Mueller of Constellation Research wondered how well DBOS the company can scale. “Will a small team at DBOS be able to run an OS, database, observability, workflow and cyber stack as good as the combination of the best of breed vendors?” [8]

See also

Related Research Articles

<span class="mw-page-title-main">PostgreSQL</span> Free and open-source object relational database management system

PostgreSQL also known as Postgres, is a free and open-source relational database management system (RDBMS) emphasizing extensibility and SQL compliance. PostgreSQL features transactions with atomicity, consistency, isolation, durability (ACID) properties, automatically updatable views, materialized views, triggers, foreign keys, and stored procedures. It is supported on all major operating systems, including Windows, Linux, macOS, FreeBSD, and OpenBSD, and handles a range of workloads from single machines to data warehouses, data lakes, or web services with many concurrent users.

<span class="mw-page-title-main">IBM Db2</span> Relational model database server

Db2 is a family of data management products, including database servers, developed by IBM. It initially supported the relational model, but was extended to support object–relational features and non-relational structures like JSON and XML. The brand name was originally styled as DB2 until 2017, when it changed to its present form.

A database transaction symbolizes a unit of work, performed within a database management system against a database, that is treated in a coherent and reliable way independent of other transactions. A transaction generally represents any change in a database. Transactions in a database environment have two main purposes:

  1. To provide reliable units of work that allow correct recovery from failures and keep a database consistent even in cases of system failure. For example: when execution prematurely and unexpectedly stops in which case many operations upon a database remain uncompleted, with unclear status.
  2. To provide isolation between programs accessing a database concurrently. If this isolation is not provided, the programs' outcomes are possibly erroneous.

MkLinux is an open-source software computer operating system begun by the Open Software Foundation Research Institute and Apple Computer in February 1996, to port Linux to the PowerPC platform, and Macintosh computers. The name refers to the Linux kernel being adapted to run as a server hosted on the Mach microkernel, version 3.0.

<span class="mw-page-title-main">FOSDEM</span> Annual event in Brussels centered on free and open source software development

Free and Open source Software Developers' European Meeting (FOSDEM) is a non-commercial, volunteer-organized European event centered on free and open-source software development. It is aimed at developers and anyone interested in the free and open-source software movement. It aims to enable developers to meet and to promote the awareness and use of free and open-source software.

<span class="mw-page-title-main">Workplace OS</span> Defunct 1990s operating system

Workplace OS is IBM's ultimate operating system prototype of the 1990s. It is the product of an exploratory research program in 1991 which yielded a design called the Grand Unifying Theory of Systems (GUTS), proposing to unify the world's systems as generalized "personalities" cohabitating concurrently upon a universally sophisticated platform of object-oriented frameworks upon one microkernel. Using personalities, a single machine would be able to run applications from multiple conventional operating systems like Unix or OS/2.

K42 is a discontinued open-source research operating system (OS) for cache-coherent 64-bit multiprocessor systems. It was developed primarily at IBM Thomas J. Watson Research Center in collaboration with the University of Toronto and University of New Mexico. The main focus of this OS is to address performance and scalability issues of system software on large-scale, shared memory, non-uniform memory access (NUMA) multiprocessing computers.

A hypervisor, also known as a virtual machine monitor (VMM) or virtualizer, is a type of computer software, firmware or hardware that creates and runs virtual machines. A computer on which a hypervisor runs one or more virtual machines is called a host machine, and each virtual machine is called a guest machine. The hypervisor presents the guest operating systems with a virtual operating platform and manages the execution of the guest operating systems. Unlike an emulator, the guest executes most instructions on the native hardware. Multiple instances of a variety of operating systems may share the virtualized hardware resources: for example, Linux, Windows, and macOS instances can all run on a single physical x86 machine. This contrasts with operating-system–level virtualization, where all instances must share a single kernel, though the guest operating systems can differ in user space, such as different Linux distributions with the same kernel.

In computing, a solution stack or software stack is a set of software subsystems or components needed to create a complete platform such that no additional software is needed to support applications. Applications are said to "run on" or "run on top of" the resulting platform.

<span class="mw-page-title-main">Amazon Elastic Compute Cloud</span> Cloud computing platform

Amazon Elastic Compute Cloud (EC2) is a part of Amazon's cloud-computing platform, Amazon Web Services (AWS), that allows users to rent virtual computers on which to run their own computer applications. EC2 encourages scalable deployment of applications by providing a web service through which a user can boot an Amazon Machine Image (AMI) to configure a virtual machine, which Amazon calls an "instance", containing any software desired. A user can create, launch, and terminate server-instances as needed, paying by the second for active servers – hence the term "elastic". EC2 provides users with control over the geographical location of instances that allows for latency optimization and high levels of redundancy. In November 2010, Amazon switched its own retail website platform to EC2 and AWS.

<span class="mw-page-title-main">Michael Stonebraker</span> American computer scientist (born 1943)

Michael Ralph Stonebraker is an American computer scientist specializing in database systems. Through a series of academic prototypes and commercial startups, Stonebraker's research and products are central to many relational databases. He is also the founder of many database companies, including Ingres Corporation, Illustra, Paradigm4, StreamBase Systems, Tamr, Vertica and VoltDB, and served as chief technical officer of Informix. For his contributions to database research, Stonebraker received the 2014 Turing Award, often described as "the Nobel Prize for computing."

<span class="mw-page-title-main">Vertica</span> Software company

Vertica is an analytic database management software company. Vertica was founded in 2005 by the database researcher Michael Stonebraker with Andrew Palmer as the founding CEO. Ralph Breslauer and Christopher P. Lynch served as CEOs later on.

Redis is a source-available, in-memory storage, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Because it holds all data in memory and because of its design, Redis offers low-latency reads and writes, making it particularly suitable for use cases that require a cache. Redis is the most popular NoSQL database, and one of the most popular databases overall. Redis is used in companies like Twitter, Airbnb, Tinder, Yahoo, Adobe, Hulu, Amazon and OpenAI.

<span class="mw-page-title-main">OpenShift</span> Cloud computing software

OpenShift is a family of containerization software products developed by Red Hat. Its flagship product is the OpenShift Container Platform — a hybrid cloud platform as a service built around Linux containers orchestrated and managed by Kubernetes on a foundation of Red Hat Enterprise Linux. The family's other products provide this platform through different environments: OKD serves as the community-driven upstream, Several deployment methods are available including self-managed, cloud native under ROSA, ARO and RHOIC on AWS, Azure, and IBM Cloud respectively, OpenShift Online as software as a service, and OpenShift Dedicated as a managed service.

NewSQL is a class of relational database management systems that seek to provide the scalability of NoSQL systems for online transaction processing (OLTP) workloads while maintaining the ACID guarantees of a traditional database system.

FoundationDB is a free and open-source multi-model distributed NoSQL database developed by Apple Inc. with a shared-nothing architecture. The product was designed around a "core" database, with additional features supplied in "layers." The core database exposes an ordered key–value store with transactions. The transactions are able to read or write multiple keys stored on any machine in the cluster while fully supporting ACID properties. Transactions are used to implement a variety of data models via layers.

<span class="mw-page-title-main">Apache Mesos</span> Software to manage computer clusters

Apache Mesos is an open-source project to manage computer clusters. It was developed at the University of California, Berkeley.

<span class="mw-page-title-main">Anbox</span> Software to run Android apps on Linux

Anbox is a free and open-source compatibility layer that allows Android applications to run on Linux distributions by using containerization techniques. Originally introduced by Canonical, Anbox executes Android applications in a lightweight system container, isolated from the host system for security and efficiency.

<span class="mw-page-title-main">Genode</span> Free and open-source software operating system

Genode is a free and open-source software operating system (OS) framework consisting of a microkernel abstraction layer and a set of user space components. The framework is notable as one of the few open-source operating systems not derived from a proprietary OS, such as Unix. The characteristic design philosophy is that a small trusted computing base is of primary concern in a security-oriented OS.

<span class="mw-page-title-main">Valkey</span> Freely available in-memory key–value database

Valkey is an open-source in-memory storage, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Because it holds all data in memory and because of its design, Valkey offers low-latency reads and writes, making it particularly suitable for use cases that require a cache. Valkey is the successor to Redis, the most popular NoSQL database, and one of the most popular databases overall. Valkey or its predecessor Redis are used in companies like Twitter, Airbnb, Tinder, Yahoo, Adobe, Hulu, Amazon and OpenAI.

References

  1. 1 2 Werner, John. "Put The OS In The Database: Performance, Cybersecurity And Endurance In The Cloud". Forbes. Retrieved 2023-12-27.
  2. 1 2 Clark, Lindsay. "Postgres pioneer promises to upend the database once more". www.theregister.com. Retrieved 2023-12-27.
  3. ↑ Cafarella, Michael; DeWitt, David; Gadepally, Vijay; Kepner, Jeremy; Kozyrakis, Christos; Kraska, Tim; Stonebraker, Michael; Zaharia, Matei (2020-07-21), DBOS: A Proposal for a Data-Centric Operating System, arXiv: 2007.11112
  4. ↑ Skiadopoulos, Athinagoras; Li, Qian; Kraft, Peter; Kaffes, Kostis; Hong, Daniel; Mathew, Shana; Bestor, David; Cafarella, Michael; Gadepally, Vijay; Graefe, Goetz; Kepner, Jeremy; Kozyrakis, Christos; Kraska, Tim; Stonebraker, Michael; Suresh, Lalith (2021-09-01). "DBOS: a DBMS-oriented operating system". Proceedings of the VLDB Endowment. 15 (1): 21–30. doi:10.14778/3485450.3485454. hdl: 1721.1/143731 . ISSN   2150-8097. S2CID   245827586.
  5. ↑ Wayne Williams (2024-03-17). "'What if the operating system is the problem': Linux was never created for the cloud — so engineers developed DBOS, a new operating system that is part OS, part database". TechRadar. Retrieved 2024-04-22.
  6. 1 2 Morgan, Timothy Prickett (2024-03-12). "The Cloud Outgrows Linux, And Sparks A New Operating System". The Next Platform. Retrieved 2024-04-22.
  7. ↑ dbos-inc/dbos-transact, DBOS, Inc., 2024-04-22, retrieved 2024-04-22
  8. ↑ Ghoshal, Anirban (2024-03-12). "DBOS Cloud overturns database-on-OS conventions for speed". InfoWorld. Retrieved 2024-04-22.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.