Mixed Reality Toolkit

Last updated
Mixed Reality Toolkit (MRTK)
Other namesHoloToolkit (HTK)
Developer(s) Microsoft
Initial release2017;6 years ago (2017)
Stable release
v2.8.3 / November 7, 2022;9 months ago (2022-11-07)
Repository github.com/microsoft/MixedRealityToolkit-Unity
Written in C#
License MIT License
Website aka.ms/mrtkdocs
As ofMarch 2023

Mixed Reality Toolkit (MRTK) is an open-source software development kit (SDK) developed by Microsoft in 2016 for the development of mixed reality (MR) and augmented reality (AR) software applications. [1] [2] It consists of a collection of components and features designed to enhance the mixed reality user and developer experiences. [3] The toolkit was originally developed by Microsoft for the release of the HoloLens 1 augmented reality headset, although it supports a variety of platforms. [2]

Contents

History

Initial release

Known as the "HoloToolkit" (HTK) prior to its initial public release, the software development kit was created to aid in the development of the new Microsoft HoloLens augmented reality device. Mixed Reality Toolkit was released to the public as an open-source SDK in December 2017, one year after the release of Microsoft's HoloLens. The software has had multiple successful generations, including MRTK, MRTK2, and the soon-to-be-released MRTK3.

HoloLens Augmented Reality Headset HoloLens 2.jpeg
HoloLens Augmented Reality Headset

Layoffs

On January 18, 2023, Microsoft announced that it would be laying off approximately 10,000 employees. [4] Days later, these layoffs were found to include the entire team behind the development of MRTK, along with significant portions of the HoloLens hardware team. [5] Currently, the future of MRTK is uncertain, although it is expected that the extended reality (XR) community will continue to maintain the software due to it being released as open-source under the MIT License.

Supported platforms

Mixed Reality Toolkit is considered to be a platform-agnostic tool as it features a wide variety of supported platforms. [1] This high level of compatibility allows developers to quickly and easily build mixed reality applications tailored to the specific platform they are working on. [3] The following is a list of supported platforms:

Supported PlatformMinimum VersionDevices
OpenXR Unity 2020.3.8+ HoloLens 2, Windows Mixed Reality headsets, Meta Quest, SteamVR devices
Windows XR PluginN/AHoloLens, HoloLens 2, Windows Mixed Reality headsets
Oculus XR Plugin Unity 2019.3+Meta Quest, Meta Quest 2
ARCore XR PluginN/AAndroid devices (via AR Foundation)
ARKit XR PluginN/AiOS devices (via AR Foundation)
OpenVR N/AWindows Mixed Reality headsets, HTC Vive, Oculus Rift
Ultraleap Hand TrackingN/AUltraleap Leap Motion controller
Unreal Engine N/AHoloLens2, Windows mixed Reality, SteamVR

Components and features

Mixed Reality Toolkit is equipped with a range of components and features that allow developers to create immersive mixed reality applications. These provide developers with a broad range of tools and technologies, including low-level APIs and high-level development frameworks. [3] These components and features are designed to be used in combination with additional software libraries to create engaging and interactive mixed reality experiences.

MRTK primary feature areas
NameDescription
Input SystemProvides developers with the ability to read and handle device-level input events from a wide range of sources
Hand Tracking (HoloLens 2)Enables users to control their HoloLens 2 device with their hands
Eye Tracking (HoloLens 2)Interprets user eye movements to provide accurate interactions
ProfilesConfigurable profiles used to set up and manage the toolkit in each scene
Hand Tracking (Ultraleap)Enables users to control their device by interfacing with the Leap Motion Data Provider
UI ControlsPre-made UI components used to quickly create user interfaces
SolversComponents to solve complex problems in mixed reality, such as object placement and 3D manipulation
Multi-Scene ManagerSystem for managing multiple scenes in mixed reality, including scene loading, unloading, and transition effects
Spatial AwarenessDetects physical environment elements to enhance user experience and interactions
Diagnostic ToolInterface which provides critical runtime verification data to simplify system debugging and analysis
MRTK Standard Shader Shader system designed to implement high quality, yet performant, visuals on mixed reality devices
Speech & DictationUtilizes speech recognition and natural language processing to interpret user keywords and commands
Boundary SystemDefines the boundaries of a user's area for increased safety and awareness
In-Editor SimulationInterface designed to test a range of MRTK interactions within the editor without creating a new build
Experimental FeaturesFeatures currently in development or unfinished, which may not have been thoroughly tested

MRTK contains UX building block components, which include buttons, object manipulators, system keyboards, object collections, and more. [3] These tools are provided to developers with the intention of simplifying the development process and encouraging innovation in mixed reality.

See also

Related Research Articles

In computing, cross-platform software is computer software that is designed to work in several computing platforms. Some cross-platform software requires a separate build for each platform, but some can be directly run on any platform without special preparation, being written in an interpreted language or compiled to portable bytecode for which the interpreters or run-time packages are common or standard components of all supported platforms.

A software development kit (SDK) is a collection of software development tools in one installable package. They facilitate the creation of applications by having a compiler, debugger and sometimes a software framework. They are normally specific to a hardware platform and operating system combination. To create applications with advanced functionalities such as advertisements, push notifications, etc; most application software developers use specific software development kits.

<span class="mw-page-title-main">Eclipse (software)</span> Software development environment

Eclipse is an integrated development environment (IDE) used in computer programming. It contains a base workspace and an extensible plug-in system for customizing the environment. It is the second-most-popular IDE for Java development, and, until 2016, was the most popular. Eclipse is written mostly in Java and its primary use is for developing Java applications, but it may also be used to develop applications in other programming languages via plug-ins, including Ada, ABAP, C, C++, C#, Clojure, COBOL, D, Erlang, Fortran, Groovy, Haskell, JavaScript, Julia, Lasso, Lua, NATURAL, Perl, PHP, Prolog, Python, R, Ruby, Rust, Scala, and Scheme. It can also be used to develop documents with LaTeX and packages for the software Mathematica. Development environments include the Eclipse Java development tools (JDT) for Java and Scala, Eclipse CDT for C/C++, and Eclipse PDT for PHP, among others.

<span class="mw-page-title-main">Don Box</span>

Don Box is a former Microsoft Technical Fellow.

Microsoft Windows SDK, and its predecessors Platform SDK, and .NET Framework SDK, are software development kits (SDKs) from Microsoft that contain documentation, header files, libraries, samples and tools required to develop applications for Microsoft Windows and .NET Framework. Platform SDK specializes in developing applications for Windows 2000, XP and Windows Server 2003. .NET Framework SDK is dedicated to developing applications for .NET Framework 1.1 and .NET Framework 2.0. Windows SDK is the successor of the two and supports developing applications for Windows XP and later, as well as .NET Framework 3.0 and later.

Mobile app development is the act or process by which a mobile app is developed for one or more mobile devices, which can include personal digital assistants (PDA), enterprise digital assistants (EDA), or mobile phones. Such software applications are specifically designed to run on mobile devices, taking numerous hardware constraints into consideration. Common constraints include CPU architecture and speeds, available memory (RAM), limited data storage capacities, and considerable variation in displays and input methods. These applications can be pre-installed on phones during manufacturing or delivered as web applications, using server-side or client-side processing to provide an "application-like" experience within a web browser.

<span class="mw-page-title-main">Unity (game engine)</span> Cross-platform video game and simulation engine

Unity is a cross-platform game engine developed by Unity Technologies, first announced and released in June 2005 at Apple Worldwide Developers Conference as a Mac OS X game engine. The engine has since been gradually extended to support a variety of desktop, mobile, console and virtual reality platforms. It is particularly popular for iOS and Android mobile game development, is considered easy to use for beginner developers, and is popular for indie game development.

<span class="mw-page-title-main">Adobe AIR</span> Cross-platform runtime system for building rich web applications

Adobe AIR is a cross-platform runtime system currently developed by Harman International, in collaboration with Adobe Inc., for building desktop applications and mobile applications, programmed using Adobe Animate, ActionScript, and optionally Apache Flex. It was originally released in 2008. The runtime supports installable applications on Windows, macOS, and mobile operating systems, including Android, iOS, and BlackBerry Tablet OS.

Open Design Alliance is a nonprofit organization creating software development kits (SDKs) for engineering applications. ODA offers interoperability tools for CAD, BIM, and Mechanical industries including .dwg, .dxf, .dgn, Autodesk Revit, Autodesk Navisworks, and .ifc files and additional tools for visualization, web development, 3D PDF publishing and modeling.

<span class="mw-page-title-main">ARToolKit</span>

ARToolKit is an open-source computer tracking library for creation of strong augmented reality applications that overlay virtual imagery on the real world. Currently, it is maintained as an open-source project hosted on GitHub.

Titanium SDK is an open-source framework that allows the creation of native mobile applications on platforms iOS and Android from a single JavaScript codebase. Titanium SDK is presently developed by non-profit software foundation TiDev, Inc.

Metaio GmbH was a privately held augmented reality (AR) company that was acquired by Apple Inc. in May of 2015 for an undisclosed amount. Headquartered in Munich, Germany, with subsidiaries in San Francisco, California, New York City, New York and Dallas, Texas, Metaio provided a software development kit (SDK) for programming PC, web, mobile application and custom offline augmented reality applications. Additionally, Metaio was the creator of Junaio, a free mobile AR browser available for Android and iOS devices.

<span class="mw-page-title-main">Windows Mixed Reality</span> Mixed reality platform

Windows Mixed Reality is a platform introduced as part of the Windows 10 and 11 operating system, which provides augmented reality and virtual reality experiences with compatible head-mounted displays.

<span class="mw-page-title-main">Microsoft HoloLens</span> Mixed reality smartglasses

Microsoft HoloLens is an augmented reality (AR)/mixed reality (MR) headset developed and manufactured by Microsoft. HoloLens runs the Windows Mixed Reality platform under the Windows 10 operating system. Some of the positional tracking technology used in HoloLens can trace its lineage to the Microsoft Kinect, an accessory for Microsoft's Xbox 360 and Xbox One game consoles that was introduced in 2010.

WebXR Device API is a Web application programming interface (API) that describes support for accessing augmented reality and virtual reality devices, such as the HTC Vive, Oculus Rift, Oculus Quest, Google Cardboard, HoloLens, Magic Leap or Open Source Virtual Reality (OSVR), in a web browser. The WebXR Device API and related APIs are standards defined by W3C groups, the Immersive Web Community Group and Immersive Web Working Group. While the Community Group works on the proposals in the incubation period, the Working Group defines the final web specifications to be implemented by the browsers.

OpenVR is a software development kit (SDK) and application programming interface (API) developed by Valve for supporting the SteamVR and other virtual reality headset devices. The SteamVR platform uses it as the default application programming interface and runtime. It serves as the interface between the virtual reality hardware and software and is implemented by SteamVR.

OpenXR is an open-source, royalty-free standard for access to virtual reality and augmented reality platforms and devices. It is developed by a working group managed by the Khronos Group consortium. OpenXR was announced by the Khronos Group on February 27, 2017, during GDC 2017. A provisional version of the standard was released on March 18, 2019, to enable developers and implementers to provide feedback on it. On July 29, 2019, OpenXR 1.0 was released to the public by Khronos Group at SIGGRAPH 2019.

Flutter is an open-source UI software development kit created by Google. It is used to develop cross platform applications from a single codebase for any web browser, Fuchsia, Android, iOS, Linux, macOS, and Windows. First described in 2015, Flutter was released in May 2017.

<span class="mw-page-title-main">HoloLens 2</span> 2019 mixed reality smartglasses

Microsoft HoloLens 2 is a mixed reality head-mounted display developed and manufactured by Microsoft. It is the successor to the original Microsoft HoloLens. The first variant of the device, The HoloLens 2 enterprise edition, debuted on February 24, 2019. This was followed by a developer edition that was announced on May 2, 2019. The HoloLens 2 was subsequently released in limited numbers on November 7, 2019.

<span class="mw-page-title-main">Windows App SDK</span> Software development kit from Microsoft

Windows App SDK is a software development kit (SDK) from Microsoft that provides a unified set of APIs and components that can be used to develop desktop applications for both Windows 11 and Windows 10 version 1809 and later. The purpose of this project is to offer a decoupled implementation of capabilities which were previously tightly-coupled to the UWP app model. Windows App SDK allows native Win32 (USER32/GDI32) or .NET (WPF/WinForms) developers alike a path forward to enhance their apps with modern features.

References

  1. 1 2 What is the Mixed Reality Toolkit, Microsoft, 2023-03-12, retrieved 2023-03-12
  2. 1 2 Team, XR Today (2022-06-30). "What is MRTK-Unity? - Mixed Reality ToolKit". XR Today. Retrieved 2023-03-12.
  3. 1 2 3 4 polar-kev. "MRTK2-Unity Developer Documentation - MRTK 2". learn.microsoft.com. Retrieved 2023-03-12.
  4. ↑ "Subject: Focusing on our short- and long-term opportunity". The Official Microsoft Blog. 2023-01-18. Retrieved 2023-03-12.
  5. ↑ Viswanathan, Pradeep. "Microsoft has laid off entire Mixed Reality Toolkit (MRTK) development team | BigTechWire" . Retrieved 2023-03-12.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.