Michael Gschwind

Last updated
Michael Gschwind
Michael Gschwind.jpg
Michael Gschwind
Born
Vienna, Austria
NationalityUSA
Alma mater Technische Universität Wien

Michael Karl Gschwind is an American computer scientist at Meta Platforms in Menlo Park, California. He is recognized for his seminal contributions to the design and exploitation of general-purpose programmable accelerators, as an early advocate of sustainability in computer design and as a prolific inventor. [1]

Contents

Accelerators

Gschwind led hardware and software architecture for the first general-purpose programmable accelerator Accelerators and is widely recognized for his contributionsHeterogeneous computing as architect of the Cell Broadband Engine processor used in the Sony PlayStation 3, [2] [3] and RoadRunner, the first supercomputer to reach sustained Petaflop operation. As Chief Architect for IBM System Architecture, he led the integration of Nvidia GPUs and IBM CPUs to create the Summit and Sierra supercomputers.

Gschwind was an early advocate for accelerator virtualization [4] [5] and as IBM System Chief Architect led I/O and accelerator virtualization. [6]

Gschwind has had a critical influence on the development of accelerator programming models with the development of APIs and best practices for accelerator programming, [7] [8] [9] [10] [11] application studies for a diverse range of HPC [12] and non-HPC applications. [13] and as co-editor of books [14] and journals [15] on practice and experience of programming accelerator-based systems.

AI Acceleration

Gschwind was an early advocate of AI Hardware Acceleration with GPUs and programmable accelerators. As IBM's Chief Engineer for AI, he led the development of IBM's first AI products and initiated the PowerAI project which brought to market AI-optimized hardware (codenamed "Minsky"), and the first prebuilt hardware-optimized AI frameworks. These frameworks were delivered as the firstfreely installable, binary package-managed AI software stacks paving the path for adoption. [16]

At Facebook, Gschwind demonstrated accelerated Large Language Models (LLMs) for Facebook's First Generation ASIC accelerators and for GPUs, leading the first LLLM production deployments at scale for embedding serving for content analysis and platform safety, and for numerous user surfaces such as Facebook Assistant, and FB Marketplace starting in 2020. [17] Gschwind led the development of and is one of the architects of Multiray, an accelerator-based platform for serving foundation models and the first production system to serve Large Language Models at scale in the industry, serving over 800 billion queries per day in 2022. [18] [19]

Gschwind led the company-wide adoption of ASIC [20] and Facebook's subsequent "strategic pivot" to GPU Inference, deploying GPU Inference at scale, a move highlighted by FB CEO Mark Zuckerburg in his earnings call. Among the first recommendation models deployed with GPU Inference was a Reels video recommendation model which delivered a 30% user surge within 2 weeks of deployment, as reported by FB CEO Mark Zuckerburg in his Q1 2022 earnings call, [21] and a subsequent $3B to $10B growth for REeels year-over-year. [22]

Gschwind also led AI Accelerator Enablement for PyTorch with a particular focus on LLM acceleration, leading the development of Accelerated Transformers [23] (formerly "Better Transformer" [24] ) and partnered with companies such as HuggingFace to drive industry-wide LLM Acceleration [25] to establish PyTorch 2.0 as the standard ecosystem for Large Language Models and Generative AI. [26] [27] [28] [29]

Gschwind subsequently led expanding LLM acceleration to on-device AI models with ExecuTorch, the PyTorch ecosystem solution for on-device AI, making on-device generative AI feasible for the first time. [30] ExecuTorch LLM acceleration (across multiple surfaces including NPUs, MPS, and Qualcomm accelerators) delivered significant speedups making it practical to deploy Llama3 unmodified on servers and on-device (demonstrated on iOS, Android, and Raspberry Pi 5) at launch with developers reporting up to 5x-10x speedups over prior on-device AI solutions. [31] [32]

Gschwind's multiple contributions to AI software stacks and frameworks, AI accelerators, mobile/embedded on-device AI and low-precision numeric representations in torchchat, [33] [34] representing a seminal milestone as the industry's first integrated softwarestack for servers and on-device AI with support for a broad set of server and embedded/mobile accelerators.

Gschwind is a pioneer and advocate of Sustainable AI. [35]

Supercomputer Design

Gschwind was a chief architect for hardware design and software architecture for several supercomputers, including three top-ranked supercomputer systems Roadrunner (June 2008 – November 2009), Sequoia (June 2012 – November 2012), and Summit (June 2018 – June 2020).

Roadrunner was a supercomputer built by IBM for the Los Alamos National Laboratory in New Mexico, USA. The US$100-million Roadrunner was designed for a peak performance of 1.7 petaflops. It achieved 1.026 petaflops on May 25, 2008, to become the world's first TOP500 LINPACK sustained 1.0 petaflops system. [36] [37] It was also the fourth-most energy-efficient supercomputer in the world on the Supermicro Green500 list, with an operational rate of 444.94 megaflops per watt of power used.

Sequoia was a petascale Blue Gene/Q supercomputer constructed by IBM for the National Nuclear Security Administration as part of the Advanced Simulation and Computing Program (ASC). It was delivered to the Lawrence Livermore National Laboratory (LLNL) in 2011 and was fully deployed in June 2012. [38] Sequoia was dismantled in 2020, its last position on the top500.org list was #22 in the November 2019 list.

Summit is a supercomputer developed by IBM for use at Oak Ridge Leadership Computing Facility (OLCF), a facility at the Oak Ridge National Laboratory. It held the number 1 position from November 2018 to June 2020. [39] [40] Its current LINPACK benchmark is clocked at 148.6 petaFLOPS. [41]

Many-Core Processor Design

Gschwind was an early advocate of many-core processor design to overcome the power and performance limitations of single-processor designs. Gschwind co-authored an analysis of the limitations of frequency scaling which arguably led to an industry-wide transition to many-core designs. [42] Gschwind was a lead architect for several many-core designs, including the first commercial many-core processor Cell with 9 cores, BlueGene/Q with 18 cores, and several enterprise and mainframe processors (POWER7/POWER8/POWER9 with up to 24 cores; z10-z15 with up to 12 cores).

As chip chief architect and chief microarchitect, Gschwind was critical to the reboot of the POWER architecture after the POWER6 high-frequency high-power dead-end, leading revival of the POWER5-style out-of-order design with POWER7, serving as unit lead and chief microarchitect for the instruction fetch, decode and branch prediction unit (also including logical instruction execution), and as acting lead for most other units at one point during the design. In subsequent generations of the POWER architecture, integration of the VMX SIMD design and FPU into VSX, little-endian support in POWER8 laying the foundation for little-endian PowerLinux (used in the Google POWER prototype and for GPU integration for the Minsky PowerAI system), and integration of NVLink for optimized GPU/CPU integration; and native support for Linux-style hardware-managed radix page-tables in POWER9, used in the world-leading Summit and Sierra Power+Nvidia supercomputers; and the introduction of PC-relative addressing and prefix instructions to transcend the limitations of the 32-bit instruction encodings of RISC architectures in POWER 10.

As architecture lead/manager and cross-platform chief architect, Gschwind also led the reboot of system z mainframe, with introduction of compiled code efficiency (with a particular view to C, C++ and Java) in IBM z10, out-of-order execution, PCIe-based I/O in z196 and z114, support for transactional memory in IBM zEC12, introduction of hardware multithreading and z/Vector SIMD architecture [43] (including shared software infrastructure with Power's VSX) in IBM z13; and the sunsetting of ESA390 for operating systems [44] substantially reducing verification and design complexity and improving time-to-market in IBM z14.

System Reliability

Gschwind coined the term "reliability wall" for obstacles to sustained operation of large-scale systems. He has made major contributions to system-level reliability modeling and improvements, with a particular view to enabling sustained supercomputing system operation. As chief architect of BlueGene/Q, he led system-level reliability and processor design in addition to being the chief ISA architect and QPU vector floating point unit design lead. [45] [46]

Gschwind led the first processor and chip-level architectural vulnerability modeling and selective hardening to achieve target MTBF, first implemented in BlueGene/Q using stacked DICE latches for critical state-holding latches. [47] To increase system reliability while avoiding the performance and power cost associated with ECC-based designs, Gschwind proposed and led the design of register files and minor buses protected with parity with state recovery. In accordance with this approach, error detection is implemented in datapaths which may occur in parallel with initiating compute operations, with a recovery operation when a soft error is detected in parallel with the operation. Recovery then proceeds from good-state maintained in alternate copies of the register file commonly used to scale the number of register file read portsa and reduce wiring delay from register file reads to execution units. [48]

Compiler Technologies

Gschwind has made seminal contributions to compiler technology, with a particular emphasis on pioneering contributions to just-in-time compilation, dynamic optimization, binary translation and compilers in supercomputing.

Just-in-time-Compilation

Gschwid was an early proponent of just-in-time compilation and has been a driving force in the field. He has proposed critical improvements for the implementation of JIT compilation based systems, with a particular view to dynamic optimization, binary translation and virtual machine implementation. Gschwind's contributions includes implementation of precise exceptions with deferred state materialization, [49] high-performance computing optimization such as software pipelining at JIT translation time, [50] [51] hardware/software co-design for binary emulation and dynamic optimization. [52] [53] [54] [55] Gschwind's seminal contributions to Virtual Machine design and implementation are reflected by being the most-cited author in the `Virtual Machines' textbook by Smith and Nair. [56]

Compilation for Accelerators and Accelerator-based Supercomputers

Gschwind is credited with seminal contributions for compiling general-purpose programmable accelerators and GPUs, supporting the launch of the nascent discipline as keynote speaker at the frst General-Purpose Programmable GPU workshop (GPGPU). His contributions include code partitioning, code optimization, code partitioning and APIs for accelerators. [57] [58] [59] [60]

His innovations include compiler/hardware co-design for integrated register files to resolve phase ordering issues in auto-vectorization between unit assignment and vectorization decisions to simplify the cost model, an innovation adopted by general-purpose programmable accelerators, including the Cell SPU and GPUseneral-purpose CPU designs, starting with Gschwind's pioneering work for SIMD CPU accelerators.

More recently, his contributions to HPC compilation have included pioneering work in enabling high-performance execution of AI workloads. [61] [62] [63]

System and Compiler APIs

Gschwind led the development of the ELFv2 Power execution environment, which has been broadly adopted for Power execution environments. Advantageously, the new environment updates the APIs and ABIs for object-oriented environments. Departing from traditional Power architecture big-endian data conventions, the ELFv2 ABI and APIs were first launched to support a new little-endian version of Linux on Power. This has since been adopted for all Linux versions on Power servers and to support GPU acceleration with Nvidia GPUs, e.g., in the Minsky AI-optimized servers and the Summit and Sierra supercomputers. [64] [65] [66]

SIMD Parallel Vector Architecture

Gschwind is a pioneer of SIMD parallel vector architecture to increase the number of operations which can be performed per cycle. To enable efficient compilation, Gschwind proposed the implementation of merged scalar and vector execution units, eliminating the cost of copies between scalar and vectorized code, and simplifying compiler architecture by resolving phase ordering problems in compilers.

The Cell's accelerator cores (Synergistic Processor Unit SPU) contain a single 128 element register file with 128 bit per register. Registers may hold either scalar or a vector of multiple values. [67] The simplified cost model leads to significantly improved vectorization success, improving overall program performance and efficiency. [68]

The vector-scalar approach was also adopted by the IBM Power VSX (Vector Scalar Extension) SIMD instructions, [69] BlueGene/Q vector instructions [70] [71] and System/z mainframe vector instruction set, [72] [73] the design of all three IBM vector-scalar architectures having been led by Gschwind as Chief Architect for IBM System Architecture.

Service, Education, Diversity, Inclusion and Digital Inclusion

Gschwind is a strong believer in the power of education and its power to help overcome the effects of all types of discrimination and colonialism. He has served as faculty member at [Princeton] and [TU Wien] to advance education. To overcome the effects of colonialism and bridge the digital divide, Gschwind has volunteered in Senegal to contribute to the expansion and improvement of Senegal's education and research network, snRER.

Background

Gschwind was born in Vienna and obtained his doctorate degree in Computer Engineering at the Technische Universität Wien in 1996. He joined the IBM Thomas J. Watson Research Center in Yorktown Heights, NY and also held positions IBM Systems product group and at its corporate headquarters in Armonk, NY. At Huawei, Gschwind served Vice President of Artificial Intelligence and Accelerated Systems at Huawei. Gschwind is currently a software engineer at Meta Platforms where he has been responsible for AI Acceleration and AI infrastructure.[ citation needed ]

Related Research Articles

<span class="mw-page-title-main">Single instruction, multiple data</span> Type of parallel processing

Single instruction, multiple data (SIMD) is a type of parallel processing in Flynn's taxonomy. SIMD can be internal and it can be directly accessible through an instruction set architecture (ISA), but it should not be confused with an ISA. SIMD describes computers with multiple processing elements that perform the same operation on multiple data points simultaneously.

AltiVec is a single-precision floating point and integer SIMD instruction set designed and owned by Apple, IBM, and Freescale Semiconductor — the AIM alliance. It is implemented on versions of the PowerPC processor architecture, including Motorola's G4, IBM's G5 and POWER6 processors, and P.A. Semi's PWRficient PA6T. AltiVec is a trademark owned solely by Freescale, so the system is also referred to as Velocity Engine by Apple and VMX by IBM and P.A. Semi.

Floating point operations per second is a measure of computer performance in computing, useful in fields of scientific computations that require floating-point calculations.

<span class="mw-page-title-main">Coprocessor</span> Type of computer processor

A coprocessor is a computer processor used to supplement the functions of the primary processor. Operations performed by the coprocessor may be floating-point arithmetic, graphics, signal processing, string processing, cryptography or I/O interfacing with peripheral devices. By offloading processor-intensive tasks from the main processor, coprocessors can accelerate system performance. Coprocessors allow a line of computers to be customized, so that customers who do not need the extra performance do not need to pay for it.

<span class="mw-page-title-main">Graphics processing unit</span> Specialized electronic circuit; graphics accelerator

A graphics processing unit (GPU) is a specialized electronic circuit initially designed for digital image processing and to accelerate computer graphics, being present either as a discrete video card or embedded on motherboards, mobile phones, personal computers, workstations, and game consoles. After their initial design, GPUs were found to be useful for non-graphic calculations involving embarrassingly parallel problems due to their parallel structure. Other non-graphical uses include the training of neural networks and cryptocurrency mining.

Cell is a 64-bit multi-core microprocessor microarchitecture that combines a general-purpose PowerPC core of modest performance with streamlined coprocessing elements which greatly accelerate multimedia and vector processing applications, as well as many other forms of dedicated computation.

<span class="mw-page-title-main">POWER7</span> 2010 family of multi-core microprocessors by IBM

POWER7 is a family of superscalar multi-core microprocessors based on the Power ISA 2.06 instruction set architecture released in 2010 that succeeded the POWER6 and POWER6+. POWER7 was developed by IBM at several sites including IBM's Rochester, MN; Austin, TX; Essex Junction, VT; T. J. Watson Research Center, NY; Bromont, QC and IBM Deutschland Research & Development GmbH, Böblingen, Germany laboratories. IBM announced servers based on POWER7 on 8 February 2010.

In computer science, stream processing is a programming paradigm which views streams, or sequences of events in time, as the central input and output objects of computation. Stream processing encompasses dataflow programming, reactive programming, and distributed data processing. Stream processing systems aim to expose parallel processing for data streams and rely on streaming algorithms for efficient implementation. The software stack for these systems includes components such as programming models and query languages, for expressing computation; stream management systems, for distribution and scheduling; and hardware components for acceleration including floating-point units, graphics processing units, and field-programmable gate arrays.

Software development for the Cell microprocessor involves a mixture of conventional development practices for the PowerPC-compatible PPU core, and novel software development challenges with regard to the functionally reduced SPU coprocessors.

<span class="mw-page-title-main">Peter Hofstee</span> Dutch physicist and computer scientist (born 1962)

Harm Peter Hofstee is a Dutch physicist and computer scientist who currently is a distinguished research staff member at IBM Austin, USA, and a part-time professor in Big Data Systems at Delft University of Technology, Netherlands.

Manycore processors are special kinds of multi-core processors designed for a high degree of parallel processing, containing numerous simpler, independent processor cores. Manycore processors are used extensively in embedded computers and high-performance computing.

<span class="mw-page-title-main">POWER9</span> 2017 family of multi-core microprocessors by IBM

POWER9 is a family of superscalar, multithreading, multi-core microprocessors produced by IBM, based on the Power ISA. It was announced in August 2016. The POWER9-based processors are being manufactured using a 14 nm FinFET process, in 12- and 24-core versions, for scale out and scale up applications, and possibly other variations, since the POWER9 architecture is open for licensing and modification by the OpenPOWER Foundation members.

Heterogeneous computing refers to systems that use more than one kind of processor or core. These systems gain performance or energy efficiency not just by adding the same type of processors, but by adding dissimilar coprocessors, usually incorporating specialized processing capabilities to handle particular tasks.

<span class="mw-page-title-main">Volta (microarchitecture)</span> GPU microarchitecture by Nvidia

Volta is the codename, but not the trademark, for a GPU microarchitecture developed by Nvidia, succeeding Pascal. It was first announced on a roadmap in March 2013, although the first product was not announced until May 2017. The architecture is named after 18th–19th century Italian chemist and physicist Alessandro Volta. It was Nvidia's first chip to feature Tensor Cores, specially designed cores that have superior deep learning performance over regular CUDA cores. The architecture is produced with TSMC's 12 nm FinFET process. The Ampere microarchitecture is the successor to Volta.

A vision processing unit (VPU) is an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks.

An AI accelerator, deep learning processor or neural processing unit (NPU) is a class of specialized hardware accelerator or computer system designed to accelerate artificial intelligence and machine learning applications, including artificial neural networks and computer vision. Typical applications include algorithms for robotics, Internet of Things, and other data-intensive or sensor-driven tasks. They are often manycore designs and generally focus on low-precision arithmetic, novel dataflow architectures or in-memory computing capability. As of 2024, a typical AI integrated circuit chip contains tens of billions of MOSFETs.

<span class="mw-page-title-main">AMD Instinct</span> Brand of data center GPUs by AMD

AMD Instinct is AMD's brand of data center GPUs. It replaced AMD's FirePro S brand in 2016. Compared to the Radeon brand of mainstream consumer/gamer products, the Instinct product line is intended to accelerate deep learning, artificial neural network, and high-performance computing/GPGPU applications.

Valentina Salapura is a researcher and expert in high-performance computing (HPC) and computer architecture. She has contributed to designing and developing advanced computing systems, focusing on scalable architectures, parallel processing, and energy-efficient computing. Her work has influenced both academic research and industry practices.

PyTorch is a machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is one of the most popular deep learning frameworks, alongside others such as TensorFlow and PaddlePaddle, offering free and open-source software released under the modified BSD license. Although the Python interface is more polished and the primary focus of development, PyTorch also has a C++ interface.

<span class="mw-page-title-main">DeepSpeed</span> Microsoft open source library

DeepSpeed is an open source deep learning optimization library for PyTorch.

References

  1. ↑ "Michael Karl Gschwind". www.ppubs.uspto.gov.
  2. ↑ David Becker (December 3, 2004). "PlayStation 3 chip goes easy on developers". CNET . Retrieved January 13, 2019.
  3. ↑ Scarpino, M. (2008). Programming the cell processor: for games, graphics, and computation. Pearson Education.
  4. ↑ https://on-demand.gputechconf.com/gtc/2017/presentation/S7320-tim-kaldewey-optimizing-efficiency-of-deep-learning-workloads-through-gpu-virtualization.pdf, https://on-demand.gputechconf.com/gtc/2017/presentation/S7320-tim-kaldewey-optimizing-efficiency-of-deep-learning-workloads-through-gpu-virtualization.pdf
  5. ↑ Optimizing the efficiency of deep learning through accelerator virtualization, https://ieeexplore.ieee.org/document/8030299
  6. ↑ I/O Vrtualization and System Acceleration in Power9, https://old.hotchips.org/wp-content/uploads/hc_archives/hc27/HC27.24-Monday-Epub/HC27.24.30-HP-Cloud-Comm-Epub/HC27.24.340-IO-Virtualization-POWER8-Gschwind-IBM.pdf
  7. ↑ Gschwind, Michael (2007-06-01). "The Cell Broadband Engine: Exploiting Multiple Levels of Parallelism in a Chip Multiprocessor". International Journal of Parallel Programming. 35 (3): 233–262. doi:10.1007/s10766-007-0035-4. ISSN   1573-7640.
  8. ↑ "ntegrated execution: A programming model for accelerators" . Retrieved 2024-09-04.
  9. ↑ Chip Multiprocessing and the Cell Broadband Engine, https://computingfrontiers.org/2006/cf06-gschwind.pdf
  10. ↑ CBE Programming Handbook
  11. ↑ CBE Programming Tutorial, https://public.dhe.ibm.com/software/dw/cell/CBE_Programming_Tutorial_v3.1.pdf
  12. ↑ Shi, Guochun; Kindratenko, Volodymyr; Pratas, Frederico; Trancoso, Pedro; Gschwind, Michael (2010). "Application acceleration with the cell broadband engine". Computing in Science and Engineering. 12 (1): 76–81. Bibcode:2010CSE....12a..76S. doi:10.1109/MCSE.2010.4. ISSN   1521-9615.
  13. ↑ Cell GC: using the cell synergistic processor as a garbage collection coprocessor, ACM Virtual Execution Environments, https://dominoweb.draco.res.ibm.com/reports/rc24520.pdf
  14. ↑ M. Gschwind, F. Gustavson, J. Prins (eds), High Performance Computing with the Cell Broadband Engine Scientific Programming 2009, https://www.semanticscholar.org/paper/High-Performance-Computing-with-the-Cell-Broadband-Gschwind-Gustavson/c6775765100eb3b9eb7b7bc003a8eba1ca90667f
  15. ↑ M. Gschwind, M. Perrone (Eds), Topical Issue On Hybrid Systems IBM Journal of Research and Development 53(5):1-2 September 2009, DOI:10.1147/JRD.2009.5429079
  16. ↑ "PowerAI: A Co-Optimized Software Stack for AI on Power" . Retrieved 2024-09-04.
  17. ↑ "From Ingestion to Deployment for Large Language Models | GTC Digital September 2022 | NVIDIA On-Demand". NVIDIA. Retrieved 2024-09-04.
  18. ↑ "MultiRay: Optimizing efficiency for large-scale AI models". ai.meta.com. Retrieved 2023-10-28.
  19. ↑ MultiRay: An Accelerated Embedding Service for Content Understanding, https://static.sched.com/hosted_files/pytorch2023/60/PyTorch_Conf_2023-Multiray.pdf
  20. ↑ First-Generation Inference Accelerator Deployment at Facebook, https://arxiv.org/pdf/2107.04140.pdf
  21. ↑ "Mark Zuckerberg says AI boosts monetization by 30% on Instagram, 40% on Facebook". Yahoo Finance. 2023-04-27. Retrieved 2024-09-04.
  22. ↑ Gairola, Ananya. "From $3B to $10B: Meta's AI-Driven Reels Skyrocketed Revenue Growth Beyond Expectations - Meta Platforms (NASDAQ:META)". Benzinga. Retrieved 2024-09-04.
  23. ↑ "PyTorch". www.pytorch.org. Retrieved 2023-10-28.
  24. ↑ "A BetterTransformer for Fast Transformer Inference". pytorch.org. Retrieved 2023-10-28.
  25. ↑ Belkada, Younes (2022-11-21). "BetterTransformer, Out of the Box Performance for Hugging Face Transformers". PyTorch. Retrieved 2024-09-04.
  26. ↑ "PyTorch 2.0: Our next generation release that is faster, more Pythonic and Dynamic as ever". PyTorch. Retrieved 2024-09-04.
  27. ↑ "Accelerated Generative Diffusion Models with PyTorch 2". PyTorch. Retrieved 2024-09-04.
  28. ↑ "Accelerating Large Language Models with Accelerated Transformers". PyTorch. Retrieved 2024-09-04.
  29. ↑ PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation, https://pytorch.org/assets/pytorch2-2.pdf
  30. ↑ "ExecuTorch Alpha: Taking LLMs and AI to the Edge with Our Community and Partners". PyTorch. Retrieved 2024-09-04.
  31. ↑ "Layla v4.6.0 has been published!". Layla. 2024-04-26. Retrieved 2024-09-04.
  32. ↑ "⚡️Blazing fast LLama2-7B-Chat on 8GB RAM Android device via Executorch". r/LocalLLaMA. 2024-05-15. Retrieved 2024-09-04.
  33. ↑ "Introducing torchchat: Accelerating Local LLM Inference on Laptop, Desktop and Mobile". PyTorch. Retrieved 2024-09-04.
  34. ↑ pytorch/torchchat, pytorch, 2024-09-04, retrieved 2024-09-04
  35. ↑ Sustainable AI: Environmental Implications, Challenges and Opportunities, https://arxiv.org/pdf/2111.00364.pdf
  36. ↑ Gaudin, Sharon (2008-06-09). "IBM's Roadrunner smashes 4-minute mile of supercomputing". Computerworld. Archived from the original on 2008-12-24. Retrieved 2008-06-10.
  37. ↑ Fildes, Jonathan (2008-06-09). "Supercomputer sets petaflop pace". BBC News. Retrieved 2008-06-09.
  38. ↑ NNSA awards IBM contract to build next generation supercomputer, February 3, 2009
  39. ↑ Lohr, Steve (8 June 2018). "Move Over, China: U.S. Is Again Home to World's Speediest Supercomputer". The New York Times. Retrieved 19 July 2018.
  40. ↑ "Top 500 List - November 2022". TOP500. November 2022. Retrieved 13 April 2022.
  41. ↑ "November 2022 | TOP500 Supercomputer Sites". TOP500. Retrieved 13 April 2022.
  42. ↑ "Optimizing pipelines for power and performance" . Retrieved 2024-09-04.
  43. ↑ Schwarz, E. M.; Krishnamurthy, R. B.; Parris, C. J.; Bradbury, J. D.; Nnebe, I. M.; Gschwind, M. (2015-07-01). "The SIMD accelerator for business analytics on the IBM z13". IBM J. Res. Dev. 59 (4–5): 2:1–2:16. doi:10.1147/JRD.2015.2426576. ISSN   0018-8646.
  44. ↑ Common boot sequence for control utility able to be initialized in multiple architectures, US Patent 9,588,774, https://patents.google.com/patent/US9588774B2
  45. ↑ "Michael Gschwind - ICS 2012 BlueGeneQ keynote presentation" . Retrieved 2024-09-04.
  46. ↑ US9081501B2,Asaad, Sameh; Bellofatto, Ralph E.& Blocksome, Michael A.et al.,"Multi-petascale highly efficient parallel supercomputer",issued 2015-07-14
  47. ↑ Gschwind, Michael; Salapura, Valentina; Trammell, Catherine; McKee, Sally A. (2011). "SoftBeam: Precise tracking of transient faults and vulnerability analysis at processor design time". 2011 IEEE 29th International Conference on Computer Design (ICCD). pp. 404–410. doi:10.1109/ICCD.2011.6081430. ISBN   978-1-4577-1954-7 . Retrieved 2024-09-04.
  48. ↑ US7512772B2,Gschwind, Michael Karl&Philhower, Robert,"Soft error handling in microprocessors",issued 2009-03-31
  49. ↑ "Efficient instruction scheduling with precise exceptions" . Retrieved 2024-09-04.
  50. ↑ "Optimizations and oracle parallelism with dynamic translation" . Retrieved 2024-09-04.
  51. ↑ "Dynamic and Transparent Binary Translation" . Retrieved 2024-09-04.
  52. ↑ "Dynamic binary translation and optimization" . Retrieved 2024-09-04.
  53. ↑ Altman, E.R.; Ebcioglu, K.; Gschwind, M.; Sathaye, S. (2001). "Advances and future challenges in binary translation and optimization". Proceedings of the IEEE. 89 (11): 1710–1722. doi:10.1109/5.964447 . Retrieved 2024-09-04.
  54. ↑ Binary translation and architecture convergence issues for IBM System/390, https://www.researchgate.net/profile/Michael-Gschwind/publication/221235791_Binary_translation_and_architecture_convergence_issues_for_IBM_system390/links/0046352f27d9de5653000000/Binary-translation-and-architecture-convergence-issues-for-IBM-system-390.pdf
  55. ↑ Advances and future challenges in binary translation and optimization, Proceedings of the IEEE, https://ieeexplore.ieee.org/document/964447
  56. ↑ Smith, Nair, Virtual Machines: Versatile Platforms for Systems and Processes, https://www.amazon.com/Virtual-Machines-Versatile-Platforms-Architecture/dp/1558609105
  57. ↑ Eichenberger, Alexandre E.; O'Brien, Kathryn; O'Brien, Kevin; Wu, Peng; Chen, Tong; Oden, Peter H.; Prener, Daniel A.; Shepherd, Janice C.; So, Byoungro; Sura, Zehra; Wang, Amy; Zhang, Tao; Zhao, Peng; Gschwind, Michael (2005-09-17). "Optimizing Compiler for the CELL Processor". 14th International Conference on Parallel Architectures and Compilation Techniques (PACT'05). PACT '05. USA: IEEE Computer Society. pp. 161–172. doi:10.1109/PACT.2005.33. ISBN   978-0-7695-2429-0.
  58. ↑ "An Open Source Environment for Cell Broadband Engine System Software" . Retrieved 2024-09-04.
  59. ↑ Chip Multiprocessing and the Cell Broadband Engine, https://www.computingfrontiers.org/2006/cf06-gschwind.pdf
  60. ↑ Gschwind, Michael (2007-06-01). "The Cell Broadband Engine: Exploiting Multiple Levels of Parallelism in a Chip Multiprocessor". International Journal of Parallel Programming. 35 (3): 233–262. doi:10.1007/s10766-007-0035-4. ISSN   1573-7640.
  61. ↑ "First-Generation Inference Accelerator Deployment at Facebook". research.facebook.com. Retrieved 2024-09-04.
  62. ↑ PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation, https://pytorch.org/assets/pytorch2-2.pdf
  63. ↑ "ExecuTorch Alpha: Taking LLMs and AI to the Edge with Our Community and Partners". PyTorch. Retrieved 2024-09-04.
  64. ↑ OpenPOWER Reengineering a server ecosystem for large-scale data centers, https://old.hotchips.org/wp-content/uploads/hc_archives/hc26/HC26-12-day2-epub/HC26.12-7-Dense-Servers-epub/HC26.12.730-%20OpenPower-Gschwind-IBM.pdf
  65. ↑ Power Architecture 64-Bit ELF V2 ABI Specification, https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16July2015_pub.pdf
  66. ↑ "Reengineering a server ecosystem for enhanced portability and performance" . Retrieved 2024-09-04.
  67. ↑ Gschwind, Michael; Hofstee, H. Peter; Flachs, Brian; Hopkins, Martin; Watanabe, Yukio; Yamazaki, Takeshi (2006). "Synergistic Processing in Cell's Multicore Architecture". IEEE Micro. 26 (2): 10–24. doi:10.1109/MM.2006.41 . Retrieved 2024-09-04.
  68. ↑ Eichenberger, Alexandre E.; O'Brien, Kathryn; O'Brien, Kevin; Wu, Peng; Chen, Tong; Oden, Peter H.; Prener, Daniel A.; Shepherd, Janice C.; So, Byoungro; Sura, Zehra; Wang, Amy; Zhang, Tao; Zhao, Peng; Gschwind, Michael (2005-09-17). "Optimizing Compiler for the CELL Processor". 14th International Conference on Parallel Architectures and Compilation Techniques (PACT'05). PACT '05. USA: IEEE Computer Society. pp. 161–172. doi:10.1109/PACT.2005.33. ISBN   978-0-7695-2429-0.
  69. ↑ Gschwind, M. (2016). "Workload acceleration with the IBM POWER vector-scalar architecture". IBM Journal of Research and Development. 60 (2–3): 14:1–14:18. doi:10.1147/JRD.2016.2527418 . Retrieved 2024-09-04.
  70. ↑ Haring, Ruud; Ohmacht, Martin; Fox, Thomas; Gschwind, Michael; Satterfield, David; Sugavanam, Krishnan; Coteus, Paul; Heidelberger, Philip; Blumrich, Matthias; Wisniewski, Robert; Gara, Alan; Chiu, George; Boyle, Peter; Chist, Norman; Kim, Changhoan (2012). "The IBM Blue Gene/Q Compute Chip". IEEE Micro. 32 (2): 48–60. doi:10.1109/MM.2011.108 . Retrieved 2024-09-04.
  71. ↑ Morgan, Timothy Prickett (22 November 2010). "IBM uncloaks 20 petaflops BlueGene/Q super". The Register.
  72. ↑ Schwarz, E. M.; Krishnamurthy, R. B.; Parris, C. J.; Bradbury, J. D.; Nnebe, I. M.; Gschwind, M. (2015-07-01). "The SIMD accelerator for business analytics on the IBM z13". IBM J. Res. Dev. 59 (4–5): 2:1–2:16. doi:10.1147/JRD.2015.2426576. ISSN   0018-8646.
  73. ↑ SIMD Processing on IBM z14, z13 and z13s, https://www.ibm.com/downloads/cas/WVPALM0N
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.