Endevor

Last updated

Endevor is a source code management and release management tool for mainframe computers running z/OS . [1] It is part of a family of administration tools by CA Technologies (formerly Computer Associates, and now part of Broadcom), which is used to maintain software applications and track their versions as well as automate lifecycle activities like builds and deployments. [2]

Contents

The word ENDEVOR is an acronym which originally stood for Environment for Developers and Operations but is now the formal product name for CA's flagship mainframe Application Lifecycle Management source control product. It also competes against another CA source code management tool, Panvalet.

Source Control

Endevor provides control of source and related code objects as individual elements. The reference "element" is used to define the smallest object for which Endevor controls. Standard source control functions are supported for element control including ADD, UPDATE, DELETE, MOVE, GENERATE, TRANSFER, SIGNIN and SIGNOUT.

Elements are edited under an associated lifecycle of Environments and Stages with changes typically being made from a development entry stage down the map from other environments like QA or PROD.  Changes can also be introduced up the map in higher environments to facilitate things like emergency fixes.  The lifecycle is also used to enable parallel development, either by creating multiple static paths to production with a merge point at some point in the map.  Dynamically added “branches” can also be created using Sandboxes and in an upcoming feature, using dynamic environments.

Endevor Element Functions

Endevor functions can be accessed interactively using 3270 ISPF panels or in batch mode using JCL and the Endevor API. Many functions executed in interactive mode are completed in batch mode. Newer interfaces exist now as well, including a Zowe conformant REST API and CLI plugin, an Eclipse Interface, a VS Code interface and an interface for Git. All Endevor element functions are described using a proprietary Software Control Language.

Add

The ADD instruction can be used to define a new element to an Endevor stage or add a previously registered element to the first stage in a define lifecycle. The ADD function invokes a generate processor which then executes all actions required to register or update the element metadata and process outputs. Generally, the result of issuing an ADD function is that the element will be registered to the target Endevor stage, or the element registration will be updated in the target stage and the appropriate generate processor will be invoked.

Generally, result of invoking a generate processor is that the source code is copied to the appropriate source library and, in the case of programs, the source is compiled and linked and the load modules are copied to the target stage load libraries.

Move

The move function moves an element registration from the target to the source stage and also, by invoking a move processor, moves the element output objects from the target to source libraries.

An element can only moved along a pre-defined lifecycle.

Delete

Deletes the target element registration and deletes the associated element source and load modules.

Transfer

Moves an element from any source stage to any target stage. The TRANSFER function is used to bypass pre-defined lifecycles.

Generate

Generate outputs. In the case of programs this includes the program LOAD module/s, DBRM and source. A GENERATE is generally executed immediately following an ADD or UPDATE.

Source Control vs Output Library Control

Endevor separates the control of source from the objects used as input and the objects created as output when an action is performed. In most cases, an element is stored internally by Endevor as the code base with subsequent changes stored as deltas. As such, control of source happens internally to Endevor and source control actions are separate from changes to objects in the output libraries which includes load modules and copies of processed code.

Endevor controlled libraries are secured with a mainframe security product such as RACF or ACF. The Endevor application executes system actions using two dedicated system accounts for which have the access to write to Endevor controlled libraries. As a general rule general mainframe users are prohibited from modifying Endevor controlled libraries. This ensures that the only way to generate and promote code is via Endevor and provides an audit trail of all changes.

Output libraries controlled by Endevor are used to store the following types of objects:

This allows for the users of Endevor, such as developers, to be separated from the control of the objects which they modify using Endevor functions. As Endevor provides an interface for creating, modifying, moving, deleting and transferring elements via pre-defined lifecycles there is no need for any end user to have alter or update access to libraries controlled by Endevor. This ensures that an audit trail is maintained for all actions and that the resulting objects controlled by Endevor can be trusted.

Release Management

Endevor supports release management, as defined by ITIL in the form of package control. Two options are available for package security - native Endevor security or Endevor External Security Interface (ESI).

Package Control

Endevor supports release management in the form of package control. A package is a container for Endevor SCL and associated control information for code release. At a minimum a package has a name, an execution window defined the time range for when the package can be executed, notes about the package, various flags and at least one value SCL statement.

A key attribute of package control is that security approver groups can be linked to each package. Each security group specifies a list of users authorised to act for that group with a set quorum per group.

Any Endevor stage which is flagged as being packaged controlled requires all actions to be executed using a package. This allows for defined systems development lifecycle stages to have set approvers and controlled releases.

As the Endevor product does not have a scheduling component a third party tool such as IBM Tivoli Workload Scheduler or CA7 must be used to execute Endevor packages according to release schedule.

Security

Endevor Native Security

Endevor native security is a built in security option which allows Endevor Administrators to define approver groups per Endevor Environment, approver group relationships per Endevor Environment and security tables per Endevor Environment and for specific Endevor functions.

Each approver group can have up to 16 userids. The first userid in an approver group is generally always set to be the Endevor internal system userid "NDVR" which grants the Endevor (batch) system the authority to execute package actions. This is due to the security model within Endevor for which requires explicit security access. This means that if the internal Endevor system userid is not linked into one of the approver groups when a package is cast then the Endevor (batch) system can not execute commands against that package.

Each approver in an approver group can be defined as being a mandatory or optional approver by specifying a flag in the approver group definition. The default configuration for Endevor approver groups is that no one specific approver in an approver group is mandatory.

Each approver group has a set quorum for which can be set to 0 to 16. Generally, an approver group will have a quorum of 1, meaning that at least one of the users listed in the approver group needs to approve the package in order for the package to progress to the APPROVED status. An approver group quorum of zero is used for approver groups where the users either need to be informed of package actions or users require the ability to DENY a package but are not explicitly required to approve. Where an approver group with a quorum of zero is attached to a package the approval state is automatically set to APPROVED for that group.

The quorum of zero is normally used for a scenario where a specific approver group requires the ability to deny a package in a situation where approval is automatic.

There is no limit to how many approver groups can be linked to a single package. Which approver groups are linked to a package is based on the approver group rules. As a general rule, when package control is used for an Endevor stage then every Endevor system will have at least one approver group for that stage and the approver group would have a quorum of zero. This means for every system referenced in the package there will be at least one approval required.

Approver groups can be dynamically altered by Endevor exits at cast time, for example, to change the quorum, link an additional approver group, or to add or remove users from an approver group. These changes are only applicable to the package being cast and are not permanent changes to approver group configuration.

Endevor also has a function to allow approvers to be interactively dynamically added to an approver group after a package is successfully cast.

Endevor Security Tables

Security tables are standard mainframe compiled assembler reference binary lookup tables. The table source is standard declared name pair mainframe table compiled (ASMA90) and linked (IEWL) assembler lookup (CONSDEF, TYPE=, TABLE=, GROUP=, USERID=) which is stored in a load library and referenced as a binary object by the Endevor security system as defined via the C1DEFLTS configuration module. Generally, there is one security table per Endevor environment, however a single security table can be referenced by multiple environments. These tables are where the access for Endevor functions is defined when using native security.

Endevor System Access Table

Access to Endevor Environments is defined by a single security table per Endevor environment. This security table defines the access control lists defined by groups for which users are assigned to and lists the stages for which users can interact with. Access can then be granted to one or more users based on the user's mainframe userid as a static value or using $ as a wildcard for 'any character'.

A user can be defined to multiple groups with least restrict access rules being applied.

Each Endevor environment has a granular breakdown of functions specific to that environment. This controls the ability for users to execute specific SCM functions such as ADD, UPDATE, DELETE, MOVE.

Endevor security tables are compiled and linked on the mainframe using the standard assembler compiler ASMA90 and linked using IEWL. A security table can be defined per Endevor environment to provide granular control for element actions down to per user if required.

Exits

Endevor administrators can modify Endevor functions and capture information using exits. There are several exit types, each attached to a specific Endevor function. Exits are generally written as COBOL or Assembler programs but theoretically can be any language that can be compiled and linked on z/OS. An exit will specify where in the exit tree that the code will be executed and what will be affected.

As an example, an exit could be written to trigger "before package cast" to link an additional approver group to the package being cast. As an example, an exit could fail the generate action where the CCID is invalid (a CCID is used to associate a change number or other ID with a related element action).

An Endevor exit program has access to most of the information relevant to the action being performed, for example name of the package, package action, package contents, etc.

Exits are generally used to enforce system policy and enhance Endevor functionality. In particular, exits are used to deny end users from executing actions which would subvert the integrity of the system development life cycle process.

An exit is generally used to determine when and to whom to send emails to, for example to notify Endevor approvers that a package is waiting to be reviewed or that a package has been reset.

Processors

Primary Endevor element functions, including GENERATE DELETE and MOVE, execute programs written in an Endevor specific language similar to JCL to perform SCL functions such as moving source code and load modules.

Every generate, move and delete action is executed by calling a processor. Every type has at least one processor defined for which lists the name of the processor that performs the action required. The default system processor executes a standard copy, move or delete action.

Processor code looks like and executes similar to JCL. Processors can use both Endevor symbols and Endevor variables. Endevor systems are defined by a specific symbol table which is essentially a list of name pair values.

Where a processor is not defined to a type / processor group / function, being generate delete or move, the default processor is invoked. The default process moves the element BASE, DELTA and SOURCE OUTPUT LIBRARY objects from the source libraries to the target libraries as defined by the element TYPE.

Element Types

Endevor maintains configuration control for each element registered in the system. Every element is distinguished by the element name, system, subsystem and type. The type definition determines how the element is stored and how subsequent changes, known as deltas, are handled.

Example of an Endevor TYPE definition for COBOL objects. In this example the type is COBOL in the TEST environment.

DISPLAY ----------------------  TYPE DEFINITION  ------------------------------ COMMAND ===>
CURRENT ENV:  TEST       STAGE ID:  T   SYSTEM:  FINANCE    TYPE:  COBOL NEXT    ENV:  PROD       STAGE ID:  P   SYSTEM:  FINANCE    TYPE:  COBOL
DESCRIPTION:       COBOL II UPDATED:           02JAN03 08:42 BY NDVADM              -----------------  ELEMENT OPTIONS  ------------------- FWD/REV/IMG DELTA: F (F/R/I)        COMPRESS BASE/ENCRYPT NAME:     Y (Y/N) DFLT PROC GRP:     COBOL    REGRESSION PCT:    75     REGR SEV:     C (I/W/C/E) SOURCE LENGTH:     80     COMPARE FROM:  1          COMPARE TO:     72 AUTO CONSOL:       Y (Y/N) LANGUAGE:     COBOL        PV/LB LANG:   COB CONSOL AT LVL:     95        HFS RECFM:  NL   (COMP/CR/CRLF/CRNL/F/LF/NL/V) LVLS TO CONSOL:    40      DATA FORMAT:  T            FILE EXT:              -------------  COMPONENT LIST OPTIONS  ---------------- FWD/REV DELTA:     F (F/R)  AUTO CONSOL: Y (Y/N)      CONSOL AT LVL:      96                                                       LVLS TO CONSOL:     50              --------------------   LIBRARIES  ---------------------   BASE/IMAGE LIBRARY:     NDVR.&C1ST..BASE.SORCLIB   DELTA LIBRARY:          NDVr.&C1ST..DELTA.COBOL   INCLUDE LIBRARY:   SOURCE O/P LIBRARY:     NDVR.&C1ST..SORCLIB     EXPAND INCLUDES:      N  (Y/N)

In the example shown the code related objects in the BASE library, DELTA library and SOURCE Output library are handled by the Endevor system not by the processor. The system uses these values to determine the source and target locations.

Types controlled by Endevor generally include COBOL, Assembler, JCL, REXX, ISPF Panels, ISPF Skeletons and parms. In this manner Endevor can be configured to handle most mainframe files.

Extensions

Endevor has several programs for which extend the primary Endevor functions.

ProgramPurpose
Quick-EditStandalone program to shortcut the process of element checkout, retrieve and checkin
Parallel Development ManagerA tool allowing developers to integrate parallel changes from other developers into their own changes
ACMAutomated configuration control which automatically tracks Endevor element component information and dependencies
Footprint SynchronizationAllows binaries to be traced back to the source code that created them by embedding footprint data in the binary for enhanced auditability and governance.
Integrations for Enterprise DevOpsProvides purpose built integrations between CA Endevor and popular DevOps tools, including the leading Enterprise Git repositories, allowing next generation developers to collaborate with traditional developers by synchronizing Git repos with CA Endevor inventory locations.
Eclipse PluginAllows direct connection to Endevor from an Eclipse-based IDE to work with elements and packages
VS Code ExtensionAllows direct connection to Endevor from a VS Code-based IDE to work with elements.
Zowe CLI PluginAllows direct connection to Endevor using the Zowe CLI framework, facilitating command line interaction with Endevor from any platform and integration via shell scripting in DevOps tools.

Environment Separation

Endevor provides multiple methods for installing and accessing distinct separate instances of Endevor for which are installed on the same LPAR. As a general rule, one version of Endevor is installed and used per LPAR.

Competing Products

The same functionality for source control and release management functions are provided by several other products.

Integration with other tools

Endevor integrates with IDz (IBM Developer for zOS) and other Eclipse based IDEs. Developers can code in the IDz IDE and seamlessly interface with Endevor to manage their changes. A basic VS Code extension for Endevor called Explorer for Endevor is also available.  The Zowe CLI plugin is also designed to allow integration to common DevOps tools by allowing Endevor operations to be scripted.

Related Research Articles

Computer programming is the process of performing a particular computation, usually by designing and building an executable computer program. Programming involves tasks such as analysis, generating algorithms, profiling algorithms' accuracy and resource consumption, and the implementation of algorithms. The source code of a program is written in one or more languages that are intelligible to programmers, rather than machine code, which is directly executed by the central processing unit. The purpose of programming is to find a sequence of instructions that will automate the performance of a task on a computer, often for solving a given problem. Proficient programming thus usually requires expertise in several different subjects, including knowledge of the application domain, specialized algorithms, and formal logic.

<span class="mw-page-title-main">COBOL</span> Programming language with English-like syntax

COBOL is a compiled English-like computer programming language designed for business use. It is an imperative, procedural and, since 2002, object-oriented language. COBOL is primarily used in business, finance, and administrative systems for companies and governments. COBOL is still widely used in applications deployed on mainframe computers, such as large-scale batch and transaction processing jobs. However, due to its declining popularity and the retirement of experienced COBOL programmers, programs are being migrated to new platforms, rewritten in modern languages or replaced with software packages. Most programming in COBOL is now purely to maintain existing applications; however, many large financial institutions were still developing new systems in COBOL as late as 2006.

PL/I is a procedural, imperative computer programming language developed and published by IBM. It is designed for scientific, engineering, business and system programming. It has been used by academic, commercial and industrial organizations since it was introduced in the 1960s, and is still used.

VME is a mainframe operating system developed by the UK company International Computers Limited. Originally developed in the 1970s to drive ICL's then new 2900 Series mainframes, the operating system is now known as OpenVME incorporating a Unix subsystem, and runs on ICL Series 39 and Trimetra mainframe computers, as well as industry-standard x64 servers.

The Burroughs Large Systems Group produced a family of large 48-bit mainframes using stack machine instruction sets with dense syllables. The first machine in the family was the B5000 in 1961. It was optimized for compiling ALGOL 60 programs extremely well, using single-pass compilers. It evolved into the B5500. Subsequent major redesigns include the B6500/B6700 line and its successors, as well as the separate B8500 line.

CICS IBM mainframe transaction monitor

IBM CICS is a family of mixed-language application servers that provide online transaction management and connectivity for applications on IBM mainframe systems under z/OS and z/VSE.

The MCP is the operating system of the Burroughs small, medium and large systems, including the Unisys Clearpath/MCP systems.

Maven is a build automation tool used primarily for Java projects. Maven can also be used to build and manage projects written in C#, Ruby, Scala, and other languages. The Maven project is hosted by the Apache Software Foundation, where it was formerly part of the Jakarta Project.

CA Harvest Software Change Manager is a software tool for the configuration management of source code and other software development assets.

In computer programming, an entry point is the place in a program where the execution of a program begins, and where the program has access to command line arguments.

Software deployment is all of the activities that make a software system available for use.

LINC is a fourth-generation programming language, used mostly on Unisys computer systems.

<span class="mw-page-title-main">System Support Program</span> Operating system for IBM minicomputers

System Support Program (SSP) was the operating system of the IBM System/34 and System/36 minicomputers. SSP was a command-based operating system released in 1977.

An artifact is one of many kinds of tangible by-products produced during the development of software. Some artifacts help describe the function, architecture, and design of software. Other artifacts are concerned with the process of development itself—such as project plans, business cases, and risk assessments.

A user exit is a subroutine invoked by a software package for a predefined event in the execution of the package. In some cases the exit is specified by the installation when configuring the package while in other cases the users of the package can substitute their own subroutines in place of the default ones provided by the package vendor to provide customized functionality. In some cases security controls restrict exits to authorized users, e.g., EXCP appendages in MVS.

Knowledge Discovery Metamodel (KDM) is a publicly available specification from the Object Management Group (OMG). KDM is a common intermediate representation for existing software systems and their operating environments, that defines common metadata required for deep semantic integration of Application Lifecycle Management tools. KDM was designed as the OMG's foundation for software modernization, IT portfolio management and software assurance. KDM uses OMG's Meta-Object Facility to define an XMI interchange format between tools that work with existing software as well as an abstract interface (API) for the next-generation assurance and modernization tools. KDM standardizes existing approaches to knowledge discovery in software engineering artifacts, also known as software mining.

System Modification Program/Extended (SMP/E), the proprietary version of System Modification Program (SMP), "is a tool designed to manage the installation of software products on [a] z/OS system and to track the modifications" to those products.

OS 2200 is the operating system for the Unisys ClearPath Dorado family of mainframe systems. The operating system kernel of OS 2200 is a lineal descendant of Exec 8 for the UNIVAC 1108. Documentation and other information on current and past Unisys systems can be found on the Unisys public support website.

Speakeasy (computational environment) Computer software environment with own programming language

Speakeasy was a numerical computing interactive environment also featuring an interpreted programming language. It was initially developed for internal use at the Physics Division of Argonne National Laboratory by the theoretical physicist Stanley Cohen. He eventually founded Speakeasy Computing Corporation to make the program available commercially.

<span class="mw-page-title-main">Definitive Media Library</span>

A Definitive Media Library is a secure Information Technology repository in which an organisation's definitive, authorised versions of software media are stored and protected. Before an organisation releases any new or changed application software into its operational environment, any such software should be fully tested and quality assured. The Definitive Media Library provides the storage area for software objects ready for deployment and should only contain master copies of controlled software media configuration items (CIs) that have passed appropriate quality assurance checks, typically including both procured and bespoke application and gold build source code and executables. In the context of the ITIL best practice framework, the term Definitive Media Library supersedes the term definitive software library referred to prior to version ITIL v3.

References

  1. "Product Sheet - CA Endevor® Software Change Manager Release 15.1" (PDF). ca Technologies. Retrieved 15 May 2013.
  2. PCmag.com