Real-time testing

Last updated

Real-time testing is the process of testing real-time computer systems.

Contents

Software testing is performed to detect and help correct bugs (errors) in computer software. Testing involves ensuring not only that the software is error-free but that it provides the required functionality to the user. Static and conventional methods of testing can detect bugs, but such techniques may not ensure correct results in real time software systems.
Real-time software systems have strict timing constraints and have a deterministic behavior. These systems have to schedule their tasks such that the timing constraints imposed on them are met. Conventional static way of analysis is not adequate to deal with such timing constraints, hence additional real-time testing is important. [1]

Strategy

Test case design for real time testing can be proposed in four steps [2]

In the very first step, each task is tested individually with conventional static testing. This testing is performed only to discover the errors in logic or syntax of the program. Order of the events doesn't matter as task testing doesn't deal with timing constraints and time properties of events.
Using the system models designed with the help of automated testing tools, it is possible to simulate behavior of real time system and impact of concurrent external events on its behavior.
Once the testing with the individual task is done, then task is supposed to be error free in coding and behavioral area. Time-related constraints are tested with intertask testing. To reveal the errors in communication, asynchronous tasks are tested with variable data rates and different payloads.
In this testing, software and hardware are integrated and full range of system tests are conducted to discover errors, if any, during software and hardware interfacing.

Tools for real time testing

As testing of real time systems is becoming more important, there are some tools designed for such testing.

MSC

Message Sequence Charts is an internationally accepted standard for capturing requirements. [3] MSC provides a graphical 2-D language often required for collecting requirements through some interaction scenarios.

SDL

Specification and Description Language is a standard used for design and analysis. SDL [4] supports the specification of complex software systems and has been extensively applied across a broad array of domains from telecommunications, automation, through to general software development

TTCN

Testing and Test Control Notation is the only internationally standard testing language. TTCN3 [5] provides a broader applicability, as compared to earlier versions of TTCN, which were primarily focused on OSI protocols only.

These three standards together are used for testing of real time applications. It is necessary that requirements be satisfied with these models and test cases generated must capture the functional and real time information needed to test systems. Also, the changes in the requirements of design and new information about the real time properties of systems should be fed into models so that its impact can be found out.

To accurately capture the real time properties of a given test system and to ensure that requirements and models are used to generate realistic and enforceable timing information, it is essential that the language itself (TTCN-3) has a well understood and semantically sound model of time.

TTCN-3

Background

TTCN-3 is the only currently available, internationally standardized testing language. Prior to TTCN3, its earlier versions were having limited functionality and limited scope over OSI protocol. But, TTCN3 is an advanced version and has broader applicability.
Characteristics of TTCN3 are:

  • ability to specify dynamic concurrent testing
  • operations for message-based and procedure based communication
  • the ability to specify data and signature templates with powerful matching mechanisms
  • type and value parameterization
  • the assignment and handling of test verdicts
  • test suite parameterization and test case selection mechanisms

The reason for using TTCN3 for real time testing is because of its timers. These timers are defined in function test suites. There are no any global kind timers used in TTCN3. These timers can be started, stopped and checked using simple functions like timer.start, timer.stop, and timer.read.

Snapshot Semantics is a technique in TTCN3 (also in TTCN2), which deals with the message passed during communication by system to system or implementation under test. When a series of responses are received by system under test, then snapshot is taken and they are evaluated in order of their arrival. So, each time around a set of attributes, a snapshot is taken and only those events are evaluated which are present in snapshot.

But this technique is not efficient as some events and their attribute information might get lost while the snapshot is taken. Some events might get recorded on processing queue, but not on snapshot. Such events can never get processed. Also, if the test executer equipment is not fast enough, then it can not communicate properly with the system under test. So, faults might get generated during such test evaluation.

Linux tools [6] [7]

See also

Related Research Articles

<span class="mw-page-title-main">Simple DirectMedia Layer</span> Free software multimedia library

Simple DirectMedia Layer (SDL) is a cross-platform software development library designed to provide a hardware abstraction layer for computer multimedia hardware components. Software developers can use it to write high-performance computer games and other multimedia applications that can run on many operating systems such as Android, iOS, Linux, macOS, and Windows.

<span class="mw-page-title-main">Embedded system</span> Computer system with a dedicated function

An embedded system is a computer system—a combination of a computer processor, computer memory, and input/output peripheral devices—that has a dedicated function within a larger mechanical or electronic system. It is embedded as part of a complete device often including electrical or electronic hardware and mechanical parts. Because an embedded system typically controls physical operations of the machine that it is embedded within, it often has real-time computing constraints. Embedded systems control many devices in common use today. In 2009, it was estimated that ninety-eight percent of all microprocessors manufactured were used in embedded systems.

RTLinux is a hard realtime real-time operating system (RTOS) microkernel that runs the entire Linux operating system as a fully preemptive process. The hard real-time property makes it possible to control robots, data acquisition systems, manufacturing plants, and other time-sensitive instruments and machines from RTLinux applications. The design was patented. Despite the similar name, it is not related to the Real-Time Linux project of the Linux Foundation.

<span class="mw-page-title-main">RTAI</span> Real-time extension for the Linux kernel

Real-time application interface (RTAI) is a real-time extension for the Linux kernel, which lets users write applications with strict timing constraints for Linux. Like Linux itself the RTAI software is a community effort. RTAI provides deterministic response to interrupts, POSIX-compliant and native RTAI real-time tasks. RTAI supports several architectures, including IA-32, x86-64, PowerPC, ARM, and MIPS.

<span class="mw-page-title-main">Watchdog timer</span> Electronic timer used to detect and recover from computer malfunctions

A watchdog timer is an electronic or software timer that is used to detect and recover from computer malfunctions. Watchdog timers are widely used in computers to facilitate automatic correction of temporary hardware faults, and to prevent errant or malevolent software from disrupting system operation.

The worst-case execution time (WCET) of a computational task is the maximum length of time the task could take to execute on a specific hardware platform.

LEON is a radiation-tolerant 32-bit central processing unit (CPU) microprocessor core that implements the SPARC V8 instruction set architecture (ISA) developed by Sun Microsystems. It was originally designed by the European Space Research and Technology Centre (ESTEC), part of the European Space Agency (ESA), without any involvement by Sun. Later versions have been designed by Gaisler Research, under a variety of owners. It is described in synthesizable VHSIC Hardware Description Language (VHDL). LEON has a dual license model: An GNU Lesser General Public License (LGPL) and GNU General Public License (GPL) free and open-source software (FOSS) license that can be used without licensing fee, or a proprietary license that can be purchased for integration in a proprietary product. The core is configurable through VHDL generics, and is used in system on a chip (SOC) designs both in research and commercial settings.

Specification and Description Language (SDL) is a specification language targeted at the unambiguous specification and description of the behaviour of reactive and distributed systems.

openSUSE Community-supported Linux distribution

openSUSE is a free and open source RPM-based Linux distribution developed by the openSUSE project.

<span class="mw-page-title-main">SUSE Linux Enterprise</span> Linux distribution

SUSE Linux Enterprise is a Linux-based operating system developed by SUSE. It is available in two editions, suffixed with Server (SLES) for servers and mainframes, and Desktop (SLED) for workstations and desktop computers. Its major versions are released at an interval of 3–4 years, while minor versions are released about every 12 months. SUSE Linux Enterprise products receive more intense testing than the upstream openSUSE community product, with the intention that only mature, stable versions of the included components will make it through to the released enterprise product.

<span class="mw-page-title-main">SUSE</span> Open-source software company

SUSE is a German-based multinational open-source software company that develops and sells Linux products to business customers. Founded in 1992, it was the first company to market Linux for enterprise. It is the developer of SUSE Linux Enterprise and the primary sponsor of the community-supported openSUSE Linux distribution project. While the openSUSE "Tumbleweed" variation is an upstream distribution for both the "Leap" variation and SUSE Linux Enterprise distribution, its branded "Leap" variation is part of a direct upgrade path to the enterprise version, which effectively makes openSUSE Leap a non-commercial version of its enterprise product.

SUSE Linux is a computer operating system developed by SUSE. It is built on top of the free and open source Linux kernel and is distributed with system and application software from other open source projects. SUSE Linux is of German origin, its name being an acronym of "Software und System-Entwicklung", and it was mainly developed in Europe. The first version appeared in early 1994, making SUSE one of the oldest existing commercial distributions. It is known for its YaST configuration tool.

In the context of computer programming, instrumentation refers to the measure of a product's performance, in order to diagnose errors and to write trace information. Instrumentation can be of two types: source instrumentation and binary instrumentation.

Nano-RK is a wireless sensor networking real-time operating system (RTOS) from Carnegie Mellon University, designed to run on microcontrollers for use in sensor networks. Nano-RK supports a fixed-priority fully preemptive scheduler with fine-grained timing primitives to support real-time task sets. "Nano" implies that the RTOS is small, using 2 KB of random-access memory (RAM) and using 18 KB of flash memory, while RK is short for resource kernel. A resource kernel provides reservations on how often system resources can be used. For example, a task might only be allowed to execute 10 ms every 150 ms, or a node might only be allowed to transmit 10 network packets per minute. These reservations form a virtual energy budget to ensure a node meets its designed battery lifetime and to prevent a failed node from generating excessive network traffic. Nano-RK is open-source software, is written in C and runs on the Atmel-based FireFly sensor networking platform, the MicaZ motes, and the MSP430 processor.

TNSDL stands for TeleNokia Specification and Description Language. TNSDL is based on the ITU-T SDL-88 language. It is used exclusively at Nokia Networks, primarily for developing applications for telephone exchanges.

<span class="mw-page-title-main">OpenComRTOS</span> Real-time operating system

OpenComRTOS is a commercial network-centric, formally developed real-time operating system (RTOS), aimed mainly at the embedded system market.

High performance computing applications run on massively parallel supercomputers consist of concurrent programs designed using multi-threaded, multi-process models. The applications may consist of various constructs with varying degree of parallelism. Although high performance concurrent programs use similar design patterns, models and principles as that of sequential programs, unlike sequential programs, they typically demonstrate non-deterministic behavior. The probability of bugs increases with the number of interactions between the various parallel constructs. Race conditions, data races, deadlocks, missed signals and live lock are common error types.

Linux on IBM Z or Linux on zSystems is the collective term for the Linux operating system compiled to run on IBM mainframes, especially IBM Z / IBM zSystems and IBM LinuxONE servers. Similar terms which imply the same meaning are Linux/390, Linux/390x, etc. The three Linux distributions certified for usage on the IBM Z hardware platform are Red Hat Enterprise Linux, SUSE Linux Enterprise Server, and Ubuntu.

PragmaDev Studio is a modeling and testing software tool introduced by PragmaDev in 2002 dedicated to the specification of communicating systems. It was initially called Real Time Developer Studio or RTDS. Its primary objective was to support SDL-RT modeling technology. Since V5.0 launched on October 7, 2015 RTDS is called PragmaDev Studio, and it is organized in four independent modules: Specifier, Developer, Tester and Tracer. V5.1 launched on November 29, 2016 introduces a freemium licensing model.

Time-triggered architecture, also known as a time-triggered system, is a computer system that executes one or more sets of tasks according to a pre-determined and set task schedule. Implementation of a TT system will typically involve use of a single interrupt that is linked to the periodic overflow of a timer. This interrupt may drive a task scheduler. The scheduler will‍—‌in turn‍—‌release the system tasks at predetermined points in time.

References

  1. ↑ Tsai, J.J.P.; Fang, K.-Y.; Bi, Y.-D. (1990). "On real-time software testing and debugging". Proceedings., Fourteenth Annual International Computer Software and Applications Conference. pp. 512–518. doi:10.1109/CMPSAC.1990.139423. ISBN   978-0-8186-2054-6.
  2. ↑ Pressman, Roger S. (2005). Software engineering : a practitioner's approach (6th ed.). Boston, Mass.: McGraw-Hill. ISBN   0-07-285318-2. OCLC   53848343.
  3. ↑ TTCN-3 Test Case Generation from Message Sequence Charts (PDF).
  4. ↑ "SDL Forum Society - Home page". sdl-forum.org. Retrieved 2021-03-08.
  5. ↑ "Home". TTCN-3. Retrieved 2021-03-08.
  6. ↑ Dahlke, Patrick (2018-02-06). "Realtime Linux". Medium. Retrieved 2021-03-10.
  7. ↑ "Realtime Testing Best Practices - eLinux.org". elinux.org. Retrieved 2021-03-10.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.