Studierfenster

Last updated

StudierFenster
Developer(s) University Hospital Essen, Graz University of Technology, Medical University of Graz
Initial release2018;6 years ago (2018)
Written in C, C++, Python, JavaScript, HTML
Operating system Cross-platform
(Windows, Mac OS X, Linux)
Available inEnglish
Type Image processing, scientific visualization, medical imaging, volume rendering, Interactive visualization
License GPL, CC BY-SA
Website studierfenster.icg.tugraz.at
Brain Tumor Segmentation under Studierfenster. Studierfenster Brain Tumor Segmentation.png
Brain Tumor Segmentation under Studierfenster.
Aortic Dissection Inpainting under Studierfenster. Aortic Dissection Inpainting.png
Aortic Dissection Inpainting under Studierfenster.

Studierfenster or StudierFenster (SF) [1] [2] [3] is a free, non-commercial open science client/server-based medical imaging processing online framework. It offers capabilities, like viewing medical data (computed tomography (CT), magnetic resonance imaging (MRI), etc.) in two- and three-dimensional space directly in the standard web browsers, like Google Chrome, Mozilla Firefox, Safari, and Microsoft Edge. Other functionalities are the calculation of medical metrics (dice score [4] and Hausdorff distance [5] ), manual slice-by-slice outlining of structures in medical images (segmentation [6] [7] ), manual placing of (anatomical) landmarks in medical image data, viewing medical data in virtual reality, a facial reconstruction and registration of medical data for augmented reality, [8] one click showcases for COVID-19 and veterinary scans, and a Radiomics module.

Contents

Other features of Studierfenster are the automatic cranial implant design with a neural network, [9] [10] the inpainting of aortic dissections [11] with a generative adversarial network, [12] [13] an automatic aortic landmark detection with deep learning [14] in computed tomography angiography scans, and a GrowCut algorithm implementation for image segmentation.

Studierfenster is currently hosted on a server at the Graz University of Technology [15] in Austria, and expanded jointly with the Institute for Artificial Intelligence in Medicine (IKIM) [16] in Essen, Germany.

History

Studierfenster was initiated within two bachelor theses during the summer bachelor program of the Institute of Computer Graphics and Vision (ICG) at Graz University of Technology, Austria, in cooperation with the Medical University of Graz, Austria, in 2018/2019. [17] [18]

The name Studierfenster (or StudierFenster) is German and can be translated to 'StudyWindow', whereby window refers here to a browser window. The word Studierfenster is an adaption from the word Studierstube ('study room'), which was an augmented reality project at the Vienna University of Technology in Austria. [19] [20]

Architecture

The Studierfenster Architecture. Studierfenster Architecture.svg
The Studierfenster Architecture.

Studierfenster is set up as a distributed application via a client–server model. The client side (front-end) consists of HTML and JavaScript with WebGL to enable 2D and 3D visualization, rendered on the client.

The server side (back-end) handles client requests via C, C++ and Python. [21] It interfaces to common open source libraries and software tools like the Insight Toolkit, [22] the Visualization Toolkit (VTK), [23] the X Toolkit (XTK) [24] and Slice:Drop. [25] The server communication is handled by AJAX requests [26] were needed.

Studierfenster employs a Flask server.

Features

Dicom browser

The Studierfenster DICOM Browser Studierfenster DICOM Browser.png
The Studierfenster DICOM Browser

This allows client-side parsing a local folder with DICOM (Digital Imaging and Communications in Medicine) [27] [28] files. Afterwards, the whole folder can be converted to compressed .nrrd (nearly raw raster data) files and downloaded as a single .zip file.

Nrrd is a library and file format for the representation and processing of n-dimensional raster data. It is intended to support scientific visualization and (medical) image processing applications. [29] With the "Dicom Browser" of Studierfenster, it is possible to select specific Studies or Series, and only convert these.

File converter

The file converter converts a medical volume file (e.g. a non-compressed .nrrd file) to a compressed/binary .nrrd file. After the conversion, the compressed .nrrd file can be downloaded and used with the "Medical 3D Viewer" for 2D and 3D visualization, and further image processing.

Metrics module

The Studierfenster Metrics Module Studierfenster Metrics Module.png
The Studierfenster Metrics Module

This can calculate the Dice similarity coefficient and Hausdorff distance between two segmentation masks (in .nrrd format) in a standard web browser.

The resulting table has seven columns: the file names for both files used in the calculation, the calculated Dice similarity coefficient, the calculated Hausdorff distance, the calculated directed HD for both directions, and the information if image spacing was used in the calculation. The table can be sorted, is searchable, and can be exported as a simple copy, an Excel spreadsheet, a comma-separated values file or as a portable document format.

The Metrics Module has been used to compare manual anatomical segmentations of brain tumors [30]

VR viewer

Virtual Reality under Studierfenster. Studierfenster Virtual Reality.png
Virtual Reality under Studierfenster.

The VR Viewer (or Medical VR Viewer) enables viewing (medical) data in Virtual Reality (VR) with devices like the Google Cardboard or the HTC Vive (via the WebVR App). [31] For viewing the data in VR, it needs to be converted to the VTI (.vti) format, which can be done with open-source, multi-platform data analysis and visualization application ParaView [32]

Critics

Studierfenster is not a certified medical product; it can only be used for educational, research, and informational purposes.

Related Research Articles

<span class="mw-page-title-main">Picture archiving and communication system</span> Medical imaging technology

A picture archiving and communication system (PACS) is a medical imaging technology which provides economical storage and convenient access to images from multiple modalities. Electronic images and reports are transmitted digitally via PACS; this eliminates the need to manually file, retrieve, or transport film jackets, the folders used to store and protect X-ray film. The universal format for PACS image storage and transfer is DICOM. Non-image data, such as scanned documents, may be incorporated using consumer industry standard formats like PDF, once encapsulated in DICOM. A PACS consists of four major components: The imaging modalities such as X-ray plain film (PF), computed tomography (CT) and magnetic resonance imaging (MRI), a secured network for the transmission of patient information, workstations for interpreting and reviewing images, and archives for the storage and retrieval of images and reports. Combined with available and emerging web technology, PACS has the ability to deliver timely and efficient access to images, interpretations, and related data. PACS reduces the physical and time barriers associated with traditional film-based image retrieval, distribution, and display.

Digital Imaging and Communications in Medicine (DICOM) is a technical standard for the digital storage and transmission of medical images and related information. It includes a file format definition, which specifies the structure of a DICOM file, as well as a network communication protocol that uses TCP/IP to communicate between systems. The primary purpose of the standard is to facilitate communication between the software and hardware entities involved in medical imaging, especially those that are created by different manufacturers. Entities that utilize DICOM files include components of picture archiving and communication systems (PACS), such as imaging machines (modalities), radiological information systems (RIS), scanners, printers, computing servers, and networking hardware.

<span class="mw-page-title-main">Volume rendering</span> Representing a 3D-modeled object or dataset as a 2D projection

In scientific visualization and computer graphics, volume rendering is a set of techniques used to display a 2D projection of a 3D discretely sampled data set, typically a 3D scalar field.

<span class="mw-page-title-main">VTK</span> Open-source software system for 3D computer graphics, image processing and visualization

The Visualization Toolkit (VTK) is an open-source software system for 3D computer graphics, image processing and scientific visualization.

<span class="mw-page-title-main">ImageJ</span> Java-based image processing program

ImageJ is a Java-based image processing program developed at the National Institutes of Health and the Laboratory for Optical and Computational Instrumentation. Its first version, ImageJ 1.x, is developed in the public domain, while ImageJ2 and the related projects SciJava, ImgLib2, and SCIFIO are licensed with a permissive BSD-2 license. ImageJ was designed with an open architecture that provides extensibility via Java plugins and recordable macros. Custom acquisition, analysis and processing plugins can be developed using ImageJ's built-in editor and a Java compiler. User-written plugins make it possible to solve many image processing and analysis problems, from three-dimensional live-cell imaging to radiological image processing, multiple imaging system data comparisons to automated hematology systems. ImageJ's plugin architecture and built-in development environment has made it a popular platform for teaching image processing.

<span class="mw-page-title-main">Mimics</span>

Materialise Mimics is an image processing software for 3D design and modeling, developed by Materialise NV, a Belgian company specialized in additive manufacturing software and technology for medical, dental and additive manufacturing industries. Materialise Mimics is used to create 3D surface models from stacks of 2D image data. These 3D models can then be used for a variety of engineering applications. Mimics is an acronym for Materialise Interactive Medical Image Control System. It is developed in an ISO environment with CE and FDA 510k premarket clearance. Materialise Mimics is commercially available as part of the Materialise Mimics Innovation Suite, which also contains Materialise3-matic, a design and meshing software for anatomical data. The current version is 24.0(released in 2021), and it supports Windows 10, Windows 7, Vista and XP in x64.

<span class="mw-page-title-main">Immersion (virtual reality)</span> Perception of being physically present in a non-physical world

Immersion into virtual reality (VR) is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.

<span class="mw-page-title-main">ITK-SNAP</span> Medical imaging software

ITK-SNAP is an interactive software application that allows users to navigate three-dimensional medical images, manually delineate anatomical regions of interest, and perform automatic image segmentation. The software was designed with the audience of clinical and basic science researchers in mind, and emphasis has been placed on having a user-friendly interface and maintaining a limited feature set to prevent feature creep. ITK-SNAP is most frequently used to work with magnetic resonance imaging (MRI), cone-beam computed tomography (CBCT) and computed tomography (CT) data sets.

<span class="mw-page-title-main">Object detection</span> Computer technology related to computer vision and image processing

Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class in digital images and videos. Well-researched domains of object detection include face detection and pedestrian detection. Object detection has applications in many areas of computer vision, including image retrieval and video surveillance.

<span class="mw-page-title-main">3D Slicer</span> Image analysis and scientific visualization software

3D Slicer (Slicer) is a free and open source software package for image analysis and scientific visualization. Slicer is used in a variety of medical applications, including autism, multiple sclerosis, systemic lupus erythematosus, prostate cancer, lung cancer, breast cancer, schizophrenia, orthopedic biomechanics, COPD, cardiovascular disease and neurosurgery.

Web-based simulation (WBS) is the invocation of computer simulation services over the World Wide Web, specifically through a web browser. Increasingly, the web is being looked upon as an environment for providing modeling and simulation applications, and as such, is an emerging area of investigation within the simulation community.

<span class="mw-page-title-main">Voreen</span> Volume visualization library and development platform

Voreen is an open-source volume visualization library and development platform. Through the use of GPU-based volume rendering techniques it allows high frame rates on standard graphics hardware to support interactive volume exploration.

<span class="mw-page-title-main">MeVisLab</span>

MeVisLab is a cross-platform application framework for medical image processing and scientific visualization. It includes advanced algorithms for image registration, segmentation, and quantitative morphological and functional image analysis. An IDE for graphical programming and rapid user interface prototyping is available.

Medical image computing (MIC) is an interdisciplinary field at the intersection of computer science, information engineering, electrical engineering, physics, mathematics and medicine. This field develops computational and mathematical methods for solving problems pertaining to medical images and their use for biomedical research and clinical care.

<span class="mw-page-title-main">Amira (software)</span> Software platform for 3D and 4D data visualization

Amira is a software platform for visualization, processing, and analysis of 3D and 4D data. It is being actively developed by Thermo Fisher Scientific in collaboration with the Zuse Institute Berlin (ZIB), and commercially distributed by Thermo Fisher Scientific — together with its sister software Avizo.

Orthanc is a standalone DICOM server. It is designed to improve the DICOM flows in hospitals and to support research about the automated analysis of medical images. Orthanc lets its users focus on the content of the DICOM files, hiding the complexity of the DICOM format and of the DICOM protocol. It is licensed under the GPLv3.

Visual computing is a generic term for all computer science disciplines dealing with images and 3D models, such as computer graphics, image processing, visualization, computer vision, virtual and augmented reality and video processing. Visual computing also includes aspects of pattern recognition, human computer interaction, machine learning and digital libraries. The core challenges are the acquisition, processing, analysis and rendering of visual information. Application areas include industrial quality control, medical image processing and visualization, surveying, robotics, multimedia systems, virtual heritage, special effects in movies and television, and computer games.

OpenVINO toolkit is a free toolkit facilitating the optimization of a deep learning model from a framework and deployment using an inference engine onto Intel hardware. The toolkit has two versions: OpenVINO toolkit, which is supported by open source community and Intel Distribution of OpenVINO toolkit, which is supported by Intel. OpenVINO was developed by Intel. The toolkit is cross-platform and free for use under Apache License version 2.0. The toolkit enables a write-once, deploy-anywhere approach to deep learning deployments on Intel platforms, including CPU, integrated GPU, Intel Movidius VPU, and FPGAs.

References

  1. ↑ "Studierfenster" . Retrieved April 23, 2020.
  2. ↑ Egger, Jan; Wild, Daniel; Weber, Maximilian; Ramirez Bedoya, Christopher; Karner, Florian; Prutsch, Alexander; Schmied, Michael; Dionysio, Christina; Krobath, Dominik; Jin, Yuan; Gsaxner, Christina; Li, Jianning; Pepe, Antonio (2022). "Studierfenster: an Open Science Cloud-Based Medical Imaging Analysis Platform". Journal of Digital Imaging. Cham: Springer International Publishing. 35 (2): 340–355. doi:10.1007/s10278-021-00574-8. PMC   8782222 . PMID   35064372.
  3. ↑ Weber, Maximilian (October 17, 2019). "A Client/Server based Online Environment for the Calculation of Medical Segmentation Scores". 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Vol. 2019. pp. 3463–3467. doi:10.1109/EMBC.2019.8856481. ISBN   978-1-5386-1311-5. PMID   31946624. S2CID   199373900.
  4. ↑ Dice, Lee R. (1945). "Measures of the Amount of Ecologic Association Between Species". Ecology. 26 (3): 297–302. Bibcode:1945Ecol...26..297D. doi:10.2307/1932409. JSTOR   1932409. S2CID   53335638.
  5. ↑ Rockafellar, R. Tyrrell; Wets, Roger J-B (2005). Variational Analysis. Springer-Verlag. p. 117. ISBN   3-540-62772-3.
  6. ↑ Linda G. Shapiro and George C. Stockman (2001): “Computer Vision”, pp 279–325, New Jersey, Prentice-Hall, ISBN   0-13-030796-3
  7. ↑ Barghout, Lauren, and Lawrence W. Lee. "Perceptual information processing system." Paravue Inc. U.S. Patent Application 10/618,543, filed July 11, 2003.
  8. ↑ Gsaxner, Christina; Pepe, Antonio; Wallner, Jürgen; Schmalstieg, Dieter; Egger, Jan (2019). "Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery". In Shen, Dinggang; Liu, Tianming; Peters, Terry M.; Staib, Lawrence H.; Essert, Caroline; Zhou, Sean; Yap, Pew-Thian; Khan, Ali (eds.). Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Lecture Notes in Computer Science. Vol. 11768. Cham: Springer International Publishing. pp. 236–244. doi:10.1007/978-3-030-32254-0_27. ISBN   978-3-030-32254-0. S2CID   202620139.
  9. ↑ Li, Jianning (January 2020). Deep Learning for Cranial Defect Reconstruction. Master Thesis, Institute of Computer Graphics and Vision, Graz University of Technology, Austria, pp. 1-68, January 2020. (Master's Thesis).
  10. ↑ Li, Jianning; Pepe, Antonio; Gsaxner, Christina; Egger, Jan (2021). "An online platform for automatic skull defect restoration and cranial implant design". In Linte, Cristian A.; Siewerdsen, Jeffrey H. (eds.). Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling. Vol. 11598. p. 59. arXiv: 2006.00980 . Bibcode:2021SPIE11598E..1QL. doi:10.1117/12.2580719. ISBN   9781510640252. S2CID   219177447.{{cite book}}: |journal= ignored (help)
  11. ↑ Pepe, Antonio; Li, Jianning; Rolf-Pissarczyk, Malte; Gsaxner, Christina; Chen, Xiaojun; Holzapfel, Gerhard A.; Egger, Jan (2020). "Detection, Segmentation, Simulation and Visualization of Aortic Dissections: A Review". Medical Image Analysis. 65: 101773. doi:10.1016/j.media.2020.101773. PMID   32738647. S2CID   220924735.
  12. ↑ Prutsch, Alexander. "Design and Development of a Web-based Tool for Inpainting ofDissected Aortae in Angiography Images" (PDF). Retrieved April 25, 2020.
  13. ↑ Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua (2014). Generative Adversarial Networks (PDF). Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014). pp. 2672–2680.
  14. ↑ Schmidhuber, J. (2015). "Deep Learning in Neural Networks: An Overview". Neural Networks. 61: 85–117. arXiv: 1404.7828 . doi:10.1016/j.neunet.2014.09.003. PMID   25462637. S2CID   11715509.
  15. ↑ "Graz University of Technology" . Retrieved April 28, 2020.
  16. ↑ "Artificial Intelligence in Medicine" . Retrieved February 9, 2022.
  17. ↑ Weber, Maximilian (December 13, 2018). A Client/Server based Online Environment for the calculation of Segmentation Scores (Bachelor Thesis). Austria: Institute of Computer Graphics and Vision, Graz University of Technology. pp. 1–40.
  18. ↑ Wild, Daniel; Weber, Maximilian; Egger, Jan (2019). "Client/Server Based Online Environment for Manual Segmentation of Medical Images". arXiv: 1904.08610 [cs.CV].
  19. ↑ "Studierstube" (PDF). Retrieved April 26, 2020.
  20. ↑ Szalavári, Zsolt; Schmalstieg, Dieter; Fuhrmann, Anton; Gervautz, Michael (1998). "?Studierstube?: An environment for collaboration in augmented reality". Virtual Reality. Lecture Notes in Computer Science. Springer International Publishing. 3: 37–48. doi:10.1007/BF01409796. S2CID   1122975.
  21. ↑ "Python" . Retrieved April 29, 2020.
  22. ↑ "The Insight Toolkit" . Retrieved April 27, 2020.
  23. ↑ "VTK - The Visualization Toolkit" . Retrieved April 27, 2020.
  24. ↑ "The X Toolkit: WebGL™ for Scientific Visualization". GitHub . April 25, 2020. Retrieved April 27, 2020.
  25. ↑ "Slice:Drop" . Retrieved April 27, 2020.
  26. ↑ "Ajax - Web developer guides". MDN Web Docs . Archived from the original on February 28, 2018. Retrieved February 27, 2018.
  27. ↑ "1 Scope and Field of Application". dicom.nema.org.
  28. ↑ DICOM brochure, nema.org.
  29. ↑ Aja-Fernández, Santiago; de Luis Garcia, Rodrigo; Tao, Dacheng; Li, Xuelong (2009). Tensors in Image Processing and Computer Vision. Advances in Computer Vision and Pattern Recognition. Springer Science & Business Media. ISBN   9781848822993.
  30. ↑ Bhandari, Abhishta; Koppen, Jarrad; Agzarian, Marc (2020). "Convolutional neural networks for brain tumour segmentation". Insights into Imaging. Lecture Notes in Computer Science. Springer Open. 11:77 (1): 77. doi: 10.1186/s13244-020-00869-4 . PMC   7280397 . PMID   32514649.
  31. ↑ Egger, Jan (March 12, 2017). "HTC Vive MeVisLab integration via OpenVR for medical applications". PLOS ONE. 12 (3): e0173972. arXiv: 1703.07575 . Bibcode:2017PLoSO..1273972E. doi: 10.1371/journal.pone.0173972 . PMC   5360258 . PMID   28323840.
  32. ↑ "ParaView" . Retrieved May 24, 2020.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.