TATP Benchmark

Last updated

In transaction processing, the Telecommunication Application Transaction Processing Benchmark (TATP) is a benchmark designed to measure the performance of in-memory database transaction systems.

Contents

Benchmark

As database and microprocessor architectures change, so must the benchmarks that are employed to measure the combined performance of these critical components. While other industry standard throughput workloads already exist, none was designed specifically to exercise the relationship between in-memory database software and the memory processing subsystem in which it transacts. The TATP is a new open source workload designed specifically for high-throughout applications, well suited for in-memory database performance analysis and system comparison.

The TATP benchmark simulates a typical home location register (HLR) database used by a mobile carrier. The HLR is an application mobile network operators use to store all relevant information about valid subscribers, including the mobile phone number, the services to which they have subscribed, access privileges, and the current location of the subscriber's handset. Every call to and from a mobile phone involves lookups against the HLRs of both parties, making it is a perfect example of a demanding, high-throughput environment where the workloads are pertinent to all applications requiring extreme speed: telecommunications, financial services, gaming, event processing and alerting, reservation systems, software as a service (SaaS), and so on.

The benchmark generates a flooding load on a database server. This means that the load is generated up to the maximum throughput point that the server can sustain. The load is generated by issuing pre-defined transactions run against a specified target database. The target database schema is made to resemble a typical HLR database in a mobile phone network. The algorithm of what is known as the TATP Benchmark was originally published in a Master's Thesis. [1] The benchmark was modeled after a real test program that was used by a telecom equipment manufacturer to evaluate the applicability of various relational database systems to service control programming in mobile networks. Another derivative of the original test is the Network Database Benchmark. [2] TATP executes seven pre-defined transactions that insert, update, delete and query the data in the database. The TATP results show the Mean Qualified Throughput (MQTh) of the target database system, and the response time distributions per transaction type for all seven types of transactions.

The TATP benchmark has been used in industry [3] and research. [4] [5] [6] [7] [8] [9]

Related Research Articles

In software quality assurance, performance testing is in general a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.

Scalability is the property of a system to handle a growing amount of work. One definition for software systems specifies that this may be done by adding resources to the system.

The thermal design power (TDP), sometimes called thermal design point, is the maximum amount of heat generated by a computer chip or component that the cooling system in a computer is designed to dissipate under any workload.

MySQL Cluster is a technology providing shared-nothing clustering and auto-sharding for the MySQL database management system. It is designed to provide high availability and high throughput with low latency, while allowing for near linear scalability. MySQL Cluster is implemented through the NDB or NDBCLUSTER storage engine for MySQL.

<span class="mw-page-title-main">Benchmark (computing)</span> Comparing the relative performance of computers by running the same program on all of them

In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it.

Online transaction processing (OLTP) is a type of database system used in transaction-oriented applications, such as many operational systems. "Online" refers to that such systems are expected to respond to user requests and process them in real-time. The term is contrasted with online analytical processing (OLAP) which instead focuses on data analysis.

Replication in computing involves sharing information so as to ensure consistency between redundant resources, such as software or hardware components, to improve reliability, fault-tolerance, or accessibility.

A column-oriented DBMS or columnar DBMS is a database management system (DBMS) that stores data tables by column rather than by row. Benefits include more efficient access to data when only querying a subset of columns, and more options for data compression. However, they are typically less efficient for inserting new data.

SAP IQ is a column-based, petabyte scale, relational database software system used for business intelligence, data warehousing, and data marts. Produced by Sybase Inc., now an SAP company, its primary function is to analyze large amounts of data in a low-cost, highly available environment. SAP IQ is often credited with pioneering the commercialization of column-store technology.

In computing, computer performance is the amount of useful work accomplished by a computer system. Outside of specific contexts, computer performance is estimated in terms of accuracy, efficiency and speed of executing computer program instructions. When it comes to high computer performance, one or more of the following factors might be involved:

Transaction Processing over XML (TPoX) is a computing benchmark for XML database systems. As a benchmark, TPoX is used for the performance testing of database management systems that are capable of storing, searching, modifying and retrieving XML data. The goal of TPoX is to allow database designers, developers and users to evaluate the performance of XML database features, such as the XML query languages XQuery and SQL/XML, XML storage, XML indexing, XML Schema support, XML updates, transaction processing and logging, and concurrency control. TPoX includes XML update tests based on the XQuery Update Facility.

Web server benchmarking is the process of estimating a web server performance in order to find if the server can serve sufficiently high workload.

<span class="mw-page-title-main">Actian NoSQL Object Database</span>

Actian NoSQL Database is an object database software product initially developed by Versant Corporation and currently owned by Actian.

H-Store is an experimental database management system (DBMS). It was designed for online transaction processing applications. H-Store was developed by a team at Brown University, Carnegie Mellon University, the Massachusetts Institute of Technology, and Yale University in 2007 by researchers Michael Stonebraker, Sam Madden, Andy Pavlo and Daniel Abadi.

Transactional Synchronization Extensions (TSX), also called Transactional Synchronization Extensions New Instructions (TSX-NI), is an extension to the x86 instruction set architecture (ISA) that adds hardware transactional memory support, speeding up execution of multi-threaded software through lock elision. According to different benchmarks, TSX/TSX-NI can provide around 40% faster applications execution in specific workloads, and 4–5 times more database transactions per second (TPS).

<span class="mw-page-title-main">Oracle NoSQL Database</span>

Oracle NoSQL Database is a NoSQL-type distributed key-value database from Oracle Corporation. It provides transactional semantics for data manipulation, horizontal scalability, and simple administration and monitoring.

<span class="mw-page-title-main">Log-structured merge-tree</span> Data structure

In computer science, the log-structured merge-tree is a data structure with performance characteristics that make it attractive for providing indexed access to files with high insert volume, such as transactional log data. LSM trees, like other search trees, maintain key-value pairs. LSM trees maintain data in two or more separate structures, each of which is optimized for its respective underlying storage medium; data is synchronized between the two structures efficiently, in batches.

Hybrid transaction/analytical processing (HTAP) is a term created by Gartner Inc., an information technology research and advisory company, in its early 2014 research report Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation. As defined by Gartner:

Hybrid transaction/analytical processing (HTAP) is an emerging application architecture that "breaks the wall" between transaction processing and analytics. It enables more informed and "in business real time" decision making.

An AI accelerator is a class of specialized hardware accelerator or computer system designed to accelerate artificial intelligence and machine learning applications, including artificial neural networks and machine vision. Typical applications include algorithms for robotics, Internet of Things, and other data-intensive or sensor-driven tasks. They are often manycore designs and generally focus on low-precision arithmetic, novel dataflow architectures or in-memory computing capability. As of 2018, a typical AI integrated circuit chip contains billions of MOSFET transistors. A number of vendor-specific terms exist for devices in this category, and it is an emerging technology without a dominant design.

HammerDB is an open source database benchmarking application developed by Steve Shaw. HammerDB supports databases such as Oracle, SQL Server, Db2, MySQL and MariaDB. HammerDB is written in TCL and C, and is licensed under the GPL v3.

References

  1. ↑ Toni Strandell: "Open Source Database Systems: Systems study, Performance and Scalability". Master's Thesis, University of Helsinki, Department of Computer Science, May 2003, 54 p, at: http://ethesis.helsinki.fi/julkaisut/mat/tieto/pg/strandell/
  2. ↑ "Network Database Benchmark", an open-source project, at: http://sourceforge.net/projects/ndbb/
  3. ↑ Intel and IBM Collaborate to Double In-Memory Database Performance, Intel 2009 http://communities.intel.com/docs/DOC-2985
  4. ↑ Ippokratis Pandis, Ryan Johnson, Nikos Hardavellas, Anastasia Ailamaki: Data-Oriented Transaction Execution. PVLDB, 3(1), 2010.
  5. ↑ Ru Fang, Hui-I Hsiao, Bin He, C. Mohan , Yun Wang: A Novel Design of Database Logging System using Storage Class Memory. Proc. International Conference on Data Engineering (ICDE 2011), Hannover, Germany April 11–16, 2011.
  6. ↑ Kishore Kumar Pusukuri, Rajiv Gupta, Laxmi N. Bhuyan: No More Backstabbing... A Faithful Scheduling Policy for Multithreaded Programs. Proc. Parallel Architectures and Compilation Techniques (PACT 2011), Galveston Island, Texas, USA, October 10–14, 2011.
  7. ↑ Ryan Johnson, Ippokratis Pandis, Radu Stoica, Manos Athanassoulis, Anastasia Ailamaki: Scalability of write-ahead logging on multicore and multisocket hardware. VLDB Journal 21(2), 2011: 239-263.
  8. ↑ Per-Åke Larson, Spyros Blanas, Cristian Diaconu, Craig Freedman, Jignesh M. Patel, Mike Zwilling: High-Performance Concurrency Control Mechanisms for Main-Memory Databases. Proc. VLDB 2012 Conference, Istanbul, Turkey, Aug. 28-30, 2012, pp. 298-309.
  9. ↑ Kevin P. Gaffney, Martin Prammer, Larry Brasfield, D. Richard Hipp, Dan Kennedy, and Jignesh M. Patel. 2022. SQLite: past, present, and future. Proc. VLDB Endow. 15, 12 (August 2022), 3535–3547. https://doi.org/10.14778/3554821.3554842
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.