QTI

Last updated

The IMS Question and Test Interoperability specification (QTI) defines a standard format for the representation of assessment content and results, supporting the exchange of this material between authoring and delivery systems, repositories and other learning management systems. It allows assessment materials to be authored and delivered on multiple systems interchangeably. It is, therefore, designed to facilitate interoperability between systems. [1]

Contents

The specification consists of a data model that defines the structure of questions, assessments and results from questions and assessments together with an XML data binding that essentially defines a language for interchanging questions and other assessment material. The XML binding is used for exchanging questions between different authoring tools and by publishers. The assessment and results parts of the specification are less widely used.

As can be seen below, the standard is mainly implemented by commercial products, few open source assessment systems support it and the most popular open source learning management systems (Moodle does not support it and Canvas only supports the old 1.2 version) limit support. The fact that you have to register to download the specs begs the openness of the standard.

Background

QTI was produced by the IMS Global Learning Consortium (IMS GLC), which is an industry and academic consortium that develops specifications for interoperable learning technology. QTI was inspired by the need for interoperability in question design, and to avoid people losing or having to re-type questions when technology changes. Developing and validating good questions can be time consuming, and it's desirable to be able to create them in a platform and technology neutral format. IMS has less than 800 members and is not the voice for the entire industry. You cannot use the QTI name in anything other than an RFP according to their website. [2]

QTI version 1.0 was materially based on a proprietary Questions Markup Language (QML) language defined by QuestionMark, but the language has evolved over the years and can now describe almost any reasonable question that one might want to describe. (QML is still in use by Questionmark).

Version 2.0 was finalized in 2005 and addressed the item (that is, the individual question) level of the specification only. A draft version of Version 2.1, which covered the structure of tests and results, was also released in 2005. But because Version 2.0 did not address test-level issues and was not compatible with Version 1, and because 2.1 was still under development, adoption of Version 2 was retarded. This was compounded in 2009 when IMS GLC withdrew the Version 2.1 draft and advised the user community that the only version "fully endorsed" by IMS GLC was 1.2.1, in effect also deprecating Version 2.0. Despite this, after several more drafts, 2.1 was finalized and released in 2012.

The current version is 2.2, which was finalized in 2015, and has subsequently had two minor revisions, 2.2.1 and 2.2.2, the latest of which was in November 2017. Version 2.2 updated and improved integration with W3C standards such as HTML5, SSML, PLS, CSS, ARIA, and MathML, and otherwise made relatively small changes to the Version 2.1 core specification.

Version 2.x is a significant improvement on Version 1, defining a new underlying interaction model. It is also notable for its significantly greater degree of integration with other specifications (some of which did not exist during the production of v1): the specification addresses the relationship with IMS Content Packaging v1.2, IEEE Learning Object Metadata, IMS Learning Design, IMS Simple Sequencing and other standards such as XHTML. It also provides guidance on representing context-specific usage data and information to support the migration of content from earlier versions of the specification.

Version 3 is now available. IMS is now called 1EdTech.

Certification

IMS offers certification of compliance to QTI standards, as noted in the table below. However, it is only offered to members of the consortium, which costs US$1,000 to US$7,500 per year. [3] There is also a cost to certify your software in addition to the Membership cost. This effectively leaves open source projects without the ability to be certified.

Timeline

DateVersionComments
March 19990.5Internal to IMS
February 20001.0 public draft
May 20001.0 final release
August 20001.01
March 20011.1
January 20021.2
March 20031.2.1 addendum
September 20032.0 charterInitiation of working group
January 20052.0 final release
January 20062.1 public draft
July 20062.1 public draft version 2
March 20092.1 draft withdrawn, 2.0 deprecated
September 20122.1 final release
September 20152.2 final release
November 20172.2.2 revision of 2.2
May 20223.0 final release

Applications with IMS QTI support

NameQTI versionType of toolComment License Certified
ATutor 1.2, 2.1 LCMS QTI 1.2; QTI 2.1 export remains experimental [4] ATutor is no longer maintained -greggray Nov 5, 2021 GPL No
Canvas by Instructure 2.1LMS2.1, Import and export of QTI files, Support of authoring, item banking, and content delivery. Since September 2023, New Quizzes supports exporting QTI 1.2. [5] AGPL Yes [6]
CSGrader not statedDigital Education tool for creating programing assignments for schools, universities, and bootcampsDoes not support QTI Proprietary No
Chamilo 1.2, 2.0LMS/LCMSexport QTI 1.2 & 2.0, import QTI 2.0 (1.2 disabled by default but available) (supports SCORM 1.2) GPL No
Classera 2.2authoring Proprietary No
Cognero 1.2, 2.1 online assessment system providing online authoring, delivery, sharing, and reportingImport and export QTI 1.2, export QTI 2.1 [7] Proprietary Yes [6]
Desire2Learn 1.2LMScan export IMS QTI 1.2, an online tool provides QTI 1.2 import Proprietary No
FastTest 1.2, 1.2 (Pearson), 2.1Single ecosystem for item authoring, form assembly, Angoff studies, online delivery, reporting, psychometric analysis, and computerized adaptive testing with item response theory Supports both imports and exports Proprietary No
GETMARKED 2.1, 1.2Browser-based AI tool that automatically convert any docx or pdf quiz or exam into QTI. Does not require pdf or word document to conform to any specific format.Supports exports to QTI 2.1, QTI 1.2, Blackboard Pool, Google Classroom and Moodle XML. Proprietary No
GradeMaker 2.1, 2.2High stakes exam authoring tool for national Awarding Bodies and Higher Education with comprehensive workflow, versioning and auditing.Supports exports to QTI file and publishing direct to test players Proprietary No
ILIAS not statedLMSsupports SCORM 1.2 and SCORM 2004 GPL No
IMC - more than learning 1.2LMSSupports both imports and exports Proprietary No
Inspera Assessment 2.1, 2.2Digital e-Assessment tool supporting authoring, delivering, marking and reporting, as well as providing learning analytics, computerized adaptive testing with item response theory.Supports exporting and importing tests and items in QTI 2.1 and 2.2. The tests and items authored in Inspera's system are stored and conducted using QTI 2.2. Proprietary No
itslearning 2.1LMSSupports QTI import Proprietary Yes [6]
Learnosity 2.1Suite of APIs, including assessment item rendering, assessment delivery, authoring and reporting toolssupports both imports and exports Proprietary No
Lectora not statedauthoring toolsupports AICC, SCORM 1.2 and SCORM 2004 Proprietary No
OpenOLAT 2.1LMS / LCMSImplementation based on QTI 2.1, import/export QTI 2.1 supported, 12 different question types. Apache No
Online Learning And Training 1.2LCMSQTI 2.1 compliance can be achieved with ONYX as plugin Apache No
ONYX Testsuite 1.2, 2.1assessment suite, consists of authoring, delivery and reporting toolbased on QTI 2.1, import/export QTI 1.2, import Blackboard/WebCT/OLAT Proprietary Yes [8]
ONYX WebEditor 1.2, 2.1online authoring tool (incl. converter and item-bank)based on QTI 2.1, import/export QTI 1.2, import Blackboard/WebCT/OLAT Proprietary Yes [8]
QST Quiz/Survey/Test 1/2/3Complete Assessment Solution - Online and mobile, very scalable and secure. Question Bank, GradeBook, Free. Convert questions to different formats. Authoring, Delivery, Marking.Import QTI 1/2/3 files, Export QTI 2.2, Import/Export Moodle XML. From a simple mobile quiz to full proctored desktop testing to detailed analysis. GPL No
qtimigration 1.0, 2.0converts QTI v1 to QTI v2 BSD No
QTI-SDK 2.0, 2.1, 2.2Software Development Kit aiming at providing tools to manipulate and deliver QTI assessments.An IMS QTI Software Development Kit for PHP 5.5 and higher supporting a wide range of features described by the IMS QTI specification family. GPL No
QTIWorks (link is dead)2.1system for managing, verifying and delivering assessmentsDevelopment on system has ended BSD No
QuestBase not statedauthoring tool and delivery systemfree online tool that can import IMS QTI Proprietary No
QuestionMark Perception not statedauthoring tool and delivery systemcan export IMS QTI, an online tool provides QTI 1.2 import Proprietary No
Question Writer 2.0 Publisher Edition1.2authoring toolexports as QTI 1.2 and SCORM 1.2 Proprietary No
Question Writer 3.5 Professional1.2authoring toolexports as QTI 1.2 and SCORM 1.2. Also specific QTI export for Pearson VUE. Proprietary No
Rogō 1.2 e-Assessment Imports/exports QTI 1.2 format questions GPL No
Sakai 1.2LMSimport and export QTI 1.2 [9] [10] ECL No
SharePoint LMS 1.2LMScan export IMS QTI 1.2, an online tool provides QTI 1.2 import Proprietary No
TAO [11] 2.1computer-based assessment platformOnly package to achieve all four areas of compliance within a single platform, Content (Validity of the packages produced/ingested), Authoring (Ability to author and export valid QTI Items & Tests), Delivery (Ability to deliver QTI Items & Tests), Item / Test Bank (Ability to store, import, export QTI Items & Tests) GPL Yes [8]
Trifork - QTI Assessment Delivery Engine 2.2 & 3.0Delivery SystemQTI 2.2 & 3.0 Proprietary No
Trifork - QTI Editor 2.2 & 3.0Authoring toolQTI 2.2 & 3.0 Proprietary No

See also

Related Research Articles

A learning object is "a collection of content items, practice items, and assessment items that are combined based on a single learning objective". The term is credited to Wayne Hodgins, and dates from a working group in 1994 bearing the name. The concept encompassed by 'Learning Objects' is known by numerous other terms, including: content objects, chunks, educational objects, information objects, intelligent objects, knowledge bits, knowledge objects, learning components, media objects, reusable curriculum components, nuggets, reusable information objects, reusable learning objects, testable reusable units of cognition, training components, and units of learning.

Sharable Content Object Reference Model (SCORM) is a collection of standards and specifications for web-based electronic educational technology. It defines communications between client side content and a host system, which is commonly supported by a learning management system. SCORM also defines how content may be packaged into a transferable ZIP file called "Package Interchange Format."

X3D is a set of royalty-free ISO/IEC standards for declaratively representing 3D computer graphics. X3D includes multiple graphics file formats, programming-language API definitions, and run-time specifications for both delivery and integration of interactive network-capable 3D data. X3D version 4.0 has been approved by Web3D Consortium, and is under final review by ISO/IEC as a revised International Standard (IS).

An open file format is a file format for storing digital data, defined by an openly published specification usually maintained by a standards organization, and which can be used and implemented by anyone. An open file format is licensed with an open license. For example, an open format can be implemented by both proprietary and free and open-source software, using the typical software licenses used by each. In contrast to open file formats, closed file formats are considered trade secrets.

The Internet Calendaring and Scheduling Core Object Specification (iCalendar) is a media type which allows users to store and exchange calendaring and scheduling information such as events, to-dos, journal entries, and free/busy information, and together with its associated standards has been a cornerstone of the standardization and interoperability of digital calendars across different vendors. Files formatted according to the specification usually have an extension of .ics. With supporting software, such as an email reader or calendar application, recipients of an iCalendar data file can respond to the sender easily or counter-propose another meeting date/time. The file format is specified in a proposed Internet standard for calendar data exchange. The standard and file type are sometimes referred to as "iCal", which was the name of the Apple Inc. calendar program until 2012, which provides one of the implementations of the standard.

COLLADA is an interchange file format for interactive 3D applications. It is managed by the nonprofit technology consortium, the Khronos Group, and has been adopted by ISO as a publicly available specification, ISO/PAS 17506.

The Schools Interoperability Framework, Systems Interoperability Framework (UK), or SIF, is a data-sharing open specification for academic institutions from kindergarten through workforce. This specification is being used primarily in the United States, Canada, the UK, Australia, and New Zealand; however, it is increasingly being implemented in India, and elsewhere.

<span class="mw-page-title-main">Aviation Industry Computer-Based Training Committee</span>

The Aviation Industry Computer-Based Training Committee (AICC) was an international association of technology-based training professionals that existed from 1988 to 2014. The AICC developed guidelines for aviation industry in the development, delivery, and evaluation of CBT, WBT, and related training technologies.

<span class="mw-page-title-main">Learning object metadata</span> Data model

Learning Object Metadata is a data model, usually encoded in XML, used to describe a learning object and similar digital resources used to support learning. The purpose of learning object metadata is to support the reusability of learning objects, to aid discoverability, and to facilitate their interoperability, usually in the context of online learning management systems (LMS).

The Clinical Data Interchange Standards Consortium (CDISC) is a standards developing organization (SDO) dealing with medical research data linked with healthcare,made to enable information system interoperability and to improve medical research and related areas of healthcare. The standards support medical research from protocol through analysis and reporting of results and have been shown to decrease resources needed by 60% overall and 70–90% in the start-up stages when they are implemented at the beginning of the research process. Since December 2016, CDISC standards are mandatory for submission to US FDA.

IMS Learning Design was a specification for a metalanguage which enables the modelling of learning processes. The specification was maintained by IMS Global Learning Consortium.

IMS Vocabulary Definition Exchange is a mark-up language or grammar for controlled vocabularies developed by IMS Global as an open specification, with the Final Specification being approved in February 2004.

Workflow Management Coalition (WfMC) was a consortium formed to define standards for the interoperability of workflow management systems. The coalition was disbanded in 2019 and no longer exists.

<span class="mw-page-title-main">OLAT</span>

OLAT is an acronym for Online Learning And Training. It is a web application – a Learning Management System that supports different kinds of online learning, teaching, and tutoring with few educational restrictions. OLAT is free software and is open-source. Its development started in 1999 at the University of Zürich and OLAT won the MeDiDa-Prix prize in 2000. With version 3.0, the system was rebuilt and is now available as a component-oriented application developed in the Java programming language. OLAT has support for various E-learning standards such as IMS, and SCORM. With version 4.0, many add-ons have been introduced to the system, which makes it very easy to extend the LMS functionality. With version 5.0, new features like Wiki, Calendar, AJAX Beta Mode, and Fulltext Search have been implemented. Version 6.0 comprised a new and improved layout based on a usability evaluation. A subsequent version provides full scalability, meaning OLAT can be run on a cluster of servers. OLAT 7.0 adds new features, a course wizard plus the implementation of important standards like REST API, IMS Basic LTI and IMS QTI 2.1. In autumn 2011 some of the core community contributors left the OLAT community because of strategic differences and started an alternative fork which is named OpenOLAT.

ILIAS is an open-source web-based learning management system (LMS). It supports learning content management and tools for collaboration, communication, evaluation and assessment. The software is published under the GNU General Public License and can be run on any server that supports PHP and MySQL.

metacoon is an open-source platform that can be used for online learning, cooperative work and for the creation of learning material. The system offers the functionality of a Learning Management System. The developer team also offers tools for the creation of learning and knowledge content(e.g. QTIquestions and tests or SCORM ). The software is published under the GNU General Public License. The offline authoring tools are freeware.

<span class="mw-page-title-main">Mozilla Open Badges</span>

Image files that contain verifiable information about learning achievements, Open Badges are based on a group of specifications and open technical standards originally developed by the Mozilla Foundation with funding from the MacArthur Foundation. The Open Badges standard describes a method for packaging information about accomplishments, embedding it into portable image files as a digital badge, and establishing an infrastructure for badge validation. The standard was originally maintained by the Badge Alliance Standard Working Group, but transitioned officially to the IMS Global Learning Consortium as of January 1, 2017.

QML is a user interface markup language, used with Qt Quick.

<span class="mw-page-title-main">OpenOLAT</span> Web based learning management system

OpenOLAT is a web-based learning management system for teaching, education, assessment and communication. The name OpenOLAT stands for Open Online Learning And Training, highlighting its open source and online nature. OpenOLAT is open source software and is being developed by frentix GmbH starting in 2011. OpenOLAT is based on the LMS OLAT developed by the University of Zurich.

Learning Tools Interoperability (LTI) is an education technology specification developed by 1EdTech. It specifies a method for a learning system to invoke and to communicate with external systems. In the current version of the specification, v1.3, this is done using OAuth2, OpenID Connect, and JSON Web Tokens. For example, a Learning Management System (LMS) may use LTI to host course content and tools provided by external, third-party systems on a web site, without requiring a learner to log in separately on the external systems, with information about the learner and the learning context shared by the LMS with the external systems.

References

  1. Effective Practice with e-Assessment guide, p.44
  2. "IMS GLOBAL LEARNING CONSORTIUM, INC. SPECIFICATION DOCUMENT LICENSE | IMS Global Learning Consortium".
  3. Alliance membership
  4. ATutor Question and Test Interoperability Support in ATutor Wiki.
  5. Export QTI announcement in New Quizzes
  6. 1 2 3 "Product Certifications | IMS Global".
  7. "Cognero Features". cognero.com. Madeira Station LLC. 2013. Archived from the original on 2014-04-07. Retrieved 2013-08-09. [...] imports QTI 1.2 and exports QTI 1.2 and 2.1 [...]
  8. 1 2 3 "IMS GLC: CC/LTI Conformance Directory". imsglobal.org. IMS Global. Retrieved 2014-06-08.
  9. Sakai Import-Export Features.
  10. Tests and Quizzes (aka. SAMigo).
  11. "Assessment Software | Open Source Assessment Platform | TAO Testing".