Data build tool

Last updated
dbt
Developer(s) dbt-Labs
Initial releaseDecember 3, 2021;2 years ago (2021-12-03)
Stable release
1.8.4 / July 18, 2024;3 months ago (2024-07-18) [1]
Repository
Written in Python
Operating system Microsoft Windows, macOS, Linux
Available in Python
Type Data analytics, data management
License Apache License 2.0
Website docs.getdbt.com

Data build tool (dbt) is an open-source command line tool that helps analysts and engineers transform data in their warehouse more effectively. [2]

Contents

History

It started at RJMetrics in 2016 as a solution to add basic transformation capabilities to Stitch (acquired by Talend in 2018). [3] The earliest versions of dbt allowed analysts to contribute to the data transformation process following the best practices of software engineering. [4]

From the beginning, dbt was open source. [5] In 2018, the dbt Labs team (then called Fishtown Analytics) released a commercial product on top of dbt Core. [6]

Funding

In April 2020, dbt Labs announced its Series A led by Andreessen Horowitz. [7] In November, dbt Labs announced its Series B led by Andreessen Horowitz and Sequoia. [8] And in June 2021, dbt Labs raised its Series C led by Altimeter, Sequoia, and Andreessen Horowitz. [9] In February 2022, the company raised $222 million for its Series D, at a $4.2 billion valuation [10]

Overview

Dbt enables analytics engineers to transform data in their warehouses by writing select statements, and turns these select statements into tables and views. Dbt does the transformation (T) in extract, load, transform (ELT) processes – it does not extract or load data, but is designed to be performant at transforming data already inside of a warehouse. Dbt has the goal of allowing analysts to work more like software engineers, in line with the dbt viewpoint. [11]

Dbt uses YAML files to declare properties. seed is a type of reference table used in dbt for static or infrequently changed data, like for example country codes or lookup tables), which are CSV based and typically stored in a seeds folder.

Related Research Articles

<span class="mw-page-title-main">Marc Andreessen</span> American entrepreneur, investor, and software engineer (born 1971)

Marc Lowell Andreessen is an American businessman and former software engineer. He is the co-author of Mosaic, the first widely used web browser with a graphical user interface; co-founder of Netscape; and co-founder and general partner of Silicon Valley venture capital firm Andreessen Horowitz. He co-founded and later sold the software company Opsware to Hewlett-Packard; he also co-founded Ning, a company that provides a platform for social networking websites. He is an inductee in the World Wide Web Hall of Fame. Andreessen's net worth is estimated at $1.7 billion.

Andreessen Horowitz is a private American venture capital firm, founded in 2009 by Marc Andreessen and Ben Horowitz. The company is headquartered in Menlo Park, California. As of April 2023, Andreessen Horowitz ranks first on the list of venture capital firms by assets under management, with $42 billion as of May 2024.

SnapLogic is a commercial software company that provides integration platform as a service (iPaaS) tools for connecting cloud data sources, SaaS applications, and on-premises business software applications. Founded in 2006, the company is headquartered in San Mateo, California. SnapLogic is headed by Gaurav Dhillon, ex-CEO and co-founder of Informatica, and is venture-backed by Andreessen Horowitz, Ignition Partners, Floodgate Fund, Brian McClendon, and Naval Ravikant.

<span class="mw-page-title-main">Peter J. Levine</span> American venture capitalist

Peter J. Levine is an American software executive and venture capitalist.

Apptio, Inc. is a Bellevue, Washington-based company founded in 2007 that develops technology business management (TBM) software as a service (SaaS) applications. Apptio enterprise apps are designed to assess and communicate the cost of IT services for planning, budgeting and forecasting purposes; Apptio's services offer tools for CIOs to manage technology departments' storage, applications, energy usage, cybersecurity, and reporting obligations; manage the costs of public cloud, migration to public cloud and SaaS portfolios; and adopt and scale Agile across the enterprise.

<span class="mw-page-title-main">DigitalOcean</span> American cloud infrastructure provider

DigitalOcean Holdings, Inc. is an American multinational technology company and cloud service provider. The company is headquartered in New York City, New York, US, with 15 globally distributed data centers. DigitalOcean provides developers, startups, and SMBs with cloud infrastructure-as-a-service platforms.

Valar Ventures is a US-based venture capital fund founded by Andrew McCormack, James Fitzgerald and Peter Thiel in 2010. Historically, the majority of the firm's investments have been in technology startups based outside of Silicon Valley, including in Europe, the UK, the US and Canada. Valar Ventures originally spun out of Thiel Capital, Peter Thiel's global parent company based in San Francisco, and is now headquartered near Madison Square in Manhattan. The firm's namesake is the Valar of J. R. R. Tolkien's legendarium, who are god-like immortal spirits that chose to enter the mortal world to prepare it for their living creations.

Mixpanel is an event analytics service company that tracks user interactions with web and mobile applications.

RJMetrics is an American software company headquartered in Philadelphia, Pennsylvania. The company offers big data analytics to small and midsize businesses.

<span class="mw-page-title-main">Databricks</span> American software company

Databricks, Inc. is a global data, analytics, and artificial intelligence company founded by the original creators of Apache Spark.

GitLab Inc. is an open-core company that operates GitLab, a DevOps software package that can develop, secure, and operate software. GitLab includes a distributed version control based on Git, including features such as access control, bug tracking, software feature requests, task management, and wikis for every project, as well as snippets.

Imply Data, Inc. is an American software company. It develops and provides commercial support for the open-source Apache Druid, a real-time database designed to power analytics applications.

<span class="mw-page-title-main">Okta, Inc.</span> American information technology company

Okta, Inc. is an American identity and access management company based in San Francisco. It provides cloud software that helps companies manage and secure user authentication into applications, and for developers to build identity controls into applications, website, web services, and devices. It was founded in 2009 and had its initial public offering in 2017, reaching a valuation of over $6 billion.

<span class="mw-page-title-main">Netlify</span> American cloud computing company

Netlify is a remote-first cloud computing company that offers a development platform that includes build, deploy, and serverless backend services for web applications and dynamic websites.

<span class="mw-page-title-main">Applied Intuition</span> Software company for autonomous vehicles

Applied Intuition is an American software company specializing in products for creating autonomous vehicles (AVs) and advanced driver-assistance systems (ADAS). Founded in 2017 and based in Mountain View, California, Applied Intuition develops tools for the development, testing, and deployment of AV and ADAS technologies in various industries, including automotive, trucking, defense, agriculture, construction, and mining. The company leverages artificial intelligence (AI) technology to provide automakers and other entities with solutions to simulate driving scenarios in the development of autonomous systems.

Samsara Inc. is an American IoT company headquartered in San Francisco, California that provides software and insights for physical operations. The company has customers across North America and Europe. Samsara developed a connected operations cloud platform that provides insights to physical operations organizations in the transportation, construction, energy, utilities, public sector and retail industries, and supports the safety and efficiency of those operations.

<span class="mw-page-title-main">Deel (company)</span> Private San Francisco-based payroll and compliance provider

Deel is an Israeli-American payroll and outsourcing company based in San Francisco, California. The company provides hiring, payments, and outsourcing services for companies hiring international employees and contractors.

<span class="mw-page-title-main">Wiz (company)</span> Cloud computing security startup

Wiz, Inc. is an American cloud security startup headquartered in New York City. The company was founded in January 2020 by Assaf Rappaport, Yinon Costica, Roy Reznik, and Ami Luttwak, all of whom previously founded Adallom. Rappaport is CEO, Costica is VP of Product, Reznik is VP of Engineering, and Luttwak is CTO. The company's platform analyzes computing infrastructure hosted in Amazon Web Services, Microsoft Azure, Google Cloud Platform, Oracle Cloud Infrastructure, and Kubernetes for combinations of risk factors that could allow malicious actors to gain control of cloud resources and/or exfiltrate valuable data.

Loom, Inc. is a technology company that provides video communication software for work. Its technology includes screen and camera recording, video editing, transcription, and the ability to share the recorded video link with others.

References

  1. "Release dbt-core v1.8.4 · dbt-labs/dbt-core". GitHub. Retrieved 1 Aug 2024.
  2. Atwal, Harvinder (9 December 2019). Practical DataOps: Delivering Agile Data Science at Scale. Apress. p. 223. ISBN   978-1-4842-5104-1.
  3. "Stitch is joining Talend". Stitch Data. 2018-11-07. Archived from the original on 2021-11-07. Retrieved 2021-11-07.
  4. "Goodbye RJMetrics, Hello Fishtown Analytics". dbt Blog. 2016-08-01. Archived from the original on 2021-11-07. Retrieved 2021-11-07.
  5. Cai, Kenrick. "Dbt Labs In Talks To Raise At $6 Billion Valuation, Six Months After Becoming A Unicorn". Forbes. Retrieved 2023-04-01.
  6. "Sinter Release Notes, August 2018: pull request builder, fine-grained GitHub permissions, and more". 2018-07-31. Archived from the original on 2021-11-07. Retrieved 2021-11-07.
  7. "Fishtown Analytics raises $12.9M Series A for its open-source analytics engineering tool". TechCrunch. 2020-04-22. Archived from the original on 2021-11-07. Retrieved 2021-11-07.
  8. "Fishtown Analytics raises $29.5M Series B for its data engineering platform". TechCrunch. 2020-11-11. Archived from the original on 2021-11-07. Retrieved 2021-11-07.
  9. "Of the Community, By the Community, For the Community". dbt Blog. 2021-06-30. Archived from the original on 2021-11-07. Retrieved 2021-11-07.
  10. Cai, Kenrick (24 Feb 2022). "VENTURE CAPITAL Dbt Labs Raises At $4.2 Billion Valuation, $2 Billion Less Than First Planned". Forbes. Forbes. Archived from the original on 11 May 2022. Retrieved 11 May 2022. The Philadelphia-based data analytics startup revealed Thursday that it had settled on a $4.2 billion valuation as part of a $222 million Series D funding round
  11. "dbt viewpoint". Archived from the original on 2021-11-07. Retrieved 2021-11-07.