Bill Dally

Last updated
Bill Dally
William Dally, PCAST Member (cropped).jpg
Dally in 2021
Born
William James Dally

(1960-08-17) August 17, 1960 (age 63)
Alma mater
Awards
Scientific career
Institutions
Thesis A VLSI Architecture for Concurrent Data Structures  (1986)
Doctoral advisor Charles Seitz [1]
Doctoral students

William James Dally (born August 17, 1960) is an American computer scientist and educator. [1] [2] He is the chief scientist and senior vice president at Nvidia and was previously a professor of Electrical Engineering and Computer Science at Stanford University and MIT. Since 2021, he has been a member of the President's Council of Advisors on Science and Technology (PCAST). [3] [4]

Contents

Microelectronics

He developed a number of techniques used in modern interconnection networks including routing-based deadlock avoidance, wormhole routing, link-level retry, virtual channels, global adaptive routing, and high-radix routers.[ jargon ] He has developed efficient mechanisms for communication, synchronization, and naming in parallel computers including message-driven computing and fast capability-based addressing. He has developed a number of stream processors starting in 1995 including Imagine, for graphics, signal, and image processing, and Merrimac, for scientific computing.[ citation needed ]

He has published over 200 papers as well as the textbooks Digital Systems Engineering with John Poulton, and Principles and Practices of Interconnection Networks with Brian Towles. He was inventor or co-inventor on over 70 granted patents.

An author quoted him saying: "Locality is efficiency, Efficiency is power, Power is performance, Performance is king". [5]

Career

Bell Labs

Dally has received a Bachelor's degree in electrical engineering from Virginia Tech. [6] While working for Bell Telephone Laboratories he contributed to the design of the Bellmac 32, an early 32-bit microprocessor,[ citation needed ] and earned an Master's degree in electrical engineering from Stanford University in 1981. He then went to the California Institute of Technology (Caltech) from 1983 to 1986, [6] graduating with a Ph.D. degree in computer science in 1986. At Caltech he designed the MOSSIM simulation engine and an integrated circuit for routing. While at Caltech, he was part of the founding group of Stac Electronics in 1983. [7]

MIT

From 1986 to 1997 he taught at MIT where he and his group built the J–Machine and the M–Machine, [8] parallel machines emphasizing low overhead synchronization and communication. During his MIT times he claims to have collaborated on developing design of Cray T3D and Cray T3E supercomputers. He became the Willard R. and Inez Kerr Bell Professor in the Stanford University School of Engineering and chairman of the computer science department at Stanford. He served as chairman for twelve years before moving on to Nvidia. [9]

Dally's corporate involvements include various collaborations at Cray Research since 1989. He did Internet router work at Avici Systems starting in 1997, was chief technical officer at Velio Communications from 1999 until its 2003 acquisition by LSI Logic, founder and chairman of Stream Processors, Inc until it folded. [7]

Nvidia and IEEE fellow

Dally was elected a Fellow of the Association for Computing Machinery in 2002, and a Fellow of the IEEE, also in 2002. In 2003 he became a consultant for NVIDIA for the first time and helped to develop GeForce 8800 GPUs series. [10] He received the ACM/SIGARCH Maurice Wilkes Award in 2000, the Seymour Cray Computer Science and Engineering Award in 2004, and the IEEE Computer Society Charles Babbage Award in 2006. In 2007 he was elected to the American Academy of Arts and Sciences.

In January 2009 he was appointed chief scientist of Nvidia. [11] He worked full-time at Nvidia, while supervising about 12 of his graduate students at Stanford. [12] He is currently chief scientist and SVP of Nvidia Research. [13]

Among many contributions to technology at Nvidia, Dally also kick-started optical interconnects for GPU [14] and computing systems [15] using micro ring modulators utilizing multiple wavelengths. [16] [17] These systems can lead to the adoption of very high bandwidth, low energy per bit optical interconnects [18] in GPUs [19] and also lead to circuit switched GPU datacenters with significant boost to AI computing efficiency.

In 2009, he was elected to the National Academy of Engineering for contributions to the design of high-performance interconnect networks and parallel computer architectures.

He received the 2010 ACM/IEEE Eckert–Mauchly Award for "outstanding contributions to the architecture of interconnection networks and parallel computers." [20]

Personal life

Dally is married and has two children. He had a flight mishap in 1992 when he was flying his Cessna 210 from Hanscom Field to Farmingdale in bad weather conditions experienced an oil leak. He was forced to make a crash landing in the Long Island Sound and was retrieved by a rescue sailboat. [21]

Works

Related Research Articles

<span class="mw-page-title-main">Supercomputer</span> Type of extremely powerful computer

A supercomputer is a type of computer with a high level of performance as compared to a general-purpose computer. The performance of a supercomputer is commonly measured in floating-point operations per second (FLOPS) instead of million instructions per second (MIPS). Since 2017, supercomputers have existed which can perform over 1017 FLOPS (a hundred quadrillion FLOPS, 100 petaFLOPS or 100 PFLOPS). For comparison, a desktop computer has performance in the range of hundreds of gigaFLOPS (1011) to tens of teraFLOPS (1013). Since November 2017, all of the world's fastest 500 supercomputers run on Linux-based operating systems. Additional research is being conducted in the United States, the European Union, Taiwan, Japan, and China to build faster, more powerful and technologically superior exascale supercomputers.

Floating point operations per second is a measure of computer performance in computing, useful in fields of scientific computations that require floating-point calculations.

Cray Inc., a subsidiary of Hewlett Packard Enterprise, is an American supercomputer manufacturer headquartered in Seattle, Washington. It also manufactures systems for data storage and analytics. Several Cray supercomputer systems are listed in the TOP500, which ranks the most powerful supercomputers in the world.

<span class="mw-page-title-main">David Bader (computer scientist)</span> American computer scientist

David A. Bader is a Distinguished Professor and Director of the Institute for Data Science at the New Jersey Institute of Technology. Previously, he served as the Chair of the Georgia Institute of Technology School of Computational Science & Engineering, where he was also a founding professor, and the executive director of High-Performance Computing at the Georgia Tech College of Computing. In 2007, he was named the first director of the Sony Toshiba IBM Center of Competence for the Cell Processor at Georgia Tech.

<span class="mw-page-title-main">President's Council of Advisors on Science and Technology</span> White House advisory board

The President's Council of Advisors on Science and Technology (PCAST) is a council, chartered in each administration with a broad mandate to advise the president of the United States on science and technology. The current PCAST was established by Executive Order 13226 on September 30, 2001, by George W. Bush, was re-chartered by Barack Obama's April 21, 2010, Executive Order 13539, by Donald Trump's October 22, 2019, Executive Order 13895, and by Joe Biden's February 1, 2021, Executive Order 14007.

Wen-mei Hwu is the Walter J. Sanders III-AMD Endowed Chair professor in Electrical and Computer Engineering in the Coordinated Science Laboratory at the University of Illinois at Urbana-Champaign. His research is on compiler design, computer architecture, computer microarchitecture, and parallel processing. He is a principal investigator for the petascale Blue Waters supercomputer, is co-director of the Universal Parallel Computing Research Center (UPCRC), and is principal investigator for the first NVIDIA CUDA Center of Excellence at UIUC. At the Illinois Coordinated Science Lab, Hwu leads the IMPACT Research Group and is director of the OpenIMPACT project – which has delivered new compiler and computer architecture technologies to the computer industry since 1987. From 1997 to 1999, Hwu served as the chairman of the Computer Engineering Program at Illinois. Since 2009, Hwu has served as chief technology officer at MulticoreWare Inc., leading the development of compiler tools for heterogeneous platforms. The OpenCL compilers developed by his team at MulticoreWare are based on the LLVM framework and have been deployed by leading semiconductor companies. In 2020, Hwu retired after serving 33 years in University of Illinois at Urbana-Champaign. Currently, Hwu is a Senior Distinguished Research Scientist at Nvidia Research and Emeritus Professor at University of Illinois at Urbana-Champaign.

The National Center for Computational Sciences (NCCS) is a United States Department of Energy (DOE) Leadership Computing Facility that houses the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science User Facility charged with helping researchers solve challenging scientific problems of global interest with a combination of leading high-performance computing (HPC) resources and international expertise in scientific computing.

<span class="mw-page-title-main">Jaguar (supercomputer)</span> Cray supercomputer at Oak Ridge National Laboratory

Jaguar or OLCF-2 was a petascale supercomputer built by Cray at Oak Ridge National Laboratory (ORNL) in Oak Ridge, Tennessee. The massively parallel Jaguar had a peak performance of just over 1,750 teraFLOPS. It had 224,256 x86-based AMD Opteron processor cores, and operated with a version of Linux called the Cray Linux Environment. Jaguar was a Cray XT5 system, a development from the Cray XT4 supercomputer.

<span class="mw-page-title-main">Tianhe-1</span> Supercomputer

Tianhe-I, Tianhe-1, or TH-1 is a supercomputer capable of an Rmax of 2.5 peta FLOPS. Located at the National Supercomputing Center of Tianjin, China, it was the fastest computer in the world from October 2010 to June 2011 and was one of the few petascale supercomputers in the world.

The Seymour Cray Computer Engineering Award, also known as the Seymour Cray Award, is an award given by the IEEE Computer Society, to recognize significant and innovative contributions in the field of high-performance computing. The award honors scientists who exhibit the creativity demonstrated by Seymour Cray, founder of Cray Research, Inc., and an early pioneer of supercomputing. Cray was an American electrical engineer and supercomputer architect who designed a series of computers that were the fastest in the world for decades, and founded Cray Research which built many of these machines. Called "the father of supercomputing," Cray has been credited with creating the supercomputer industry. He played a key role in the invention and design of the UNIVAC 1103, a landmark high-speed computer and the first computer available for commercial use.

<span class="mw-page-title-main">Supercomputer architecture</span> Design of high-performance computers

Approaches to supercomputer architecture have taken dramatic turns since the earliest systems were introduced in the 1960s. Early supercomputer architectures pioneered by Seymour Cray relied on compact innovative designs and local parallelism to achieve superior computational peak performance. However, in time the demand for increased computational power ushered in the age of massively parallel systems.

<span class="mw-page-title-main">Titan (supercomputer)</span> American supercomputer

Titan or OLCF-3 was a supercomputer built by Cray at Oak Ridge National Laboratory for use in a variety of science projects. Titan was an upgrade of Jaguar, a previous supercomputer at Oak Ridge, that uses graphics processing units (GPUs) in addition to conventional central processing units (CPUs). Titan was the first such hybrid to perform over 10 petaFLOPS. The upgrade began in October 2011, commenced stability testing in October 2012 and it became available to researchers in early 2013. The initial cost of the upgrade was US$60 million, funded primarily by the United States Department of Energy.

Steve Scott is a computer architect who currently serves as Corporate Vice President at Microsoft. Scott was previously a Senior Vice President and Chief Technology Officer at Cray Inc., Principal Engineer at Google and the chief technology officer for Nvidia's Tesla business unit. Scott was employed by Cray Research, Inc., Silicon Graphics, Inc., and Cray, Inc. from 1992 to 2011.

<span class="mw-page-title-main">Supercomputing in Pakistan</span> Overview of supercomputing in Pakistan

The high performance supercomputing program started in mid-to-late 1980s in Pakistan. Supercomputing is a recent area of Computer science in which Pakistan has made progress, driven in part by the growth of the information technology age in the country. Developing on the ingenious supercomputer program started in 1980s when the deployment of the Cray supercomputers was initially denied.

XK7 is a supercomputing platform, produced by Cray, launched on October 29, 2012. XK7 is the second platform from Cray to use a combination of central processing units ("CPUs") and graphical processing units ("GPUs") for computing; the hybrid architecture requires a different approach to programming to that of CPU-only supercomputers. Laboratories that host XK7 machines host workshops to train researchers in the new programming languages needed for XK7 machines. The platform is used in Titan, the world's second fastest supercomputer in the November 2013 list as ranked by the TOP500 organization. Other customers include the Swiss National Supercomputing Centre which has a 272 node machine and Blue Waters has a machine that has Cray XE6 and XK7 nodes that performs at approximately 1 petaFLOPS (1015 floating-point operations per second).

<span class="mw-page-title-main">NVLink</span> High speed chip interconnect

NVLink is a wire-based serial multi-lane near-range communications link developed by Nvidia. Unlike PCI Express, a device can consist of multiple NVLinks, and devices use mesh networking to communicate instead of a central hub. The protocol was first announced in March 2014 and uses a proprietary high-speed signaling interconnect (NVHS).

<span class="mw-page-title-main">Cray XC40</span> Supercomputer manufactured by Cray

The Cray XC40 is a massively parallel multiprocessor supercomputer manufactured by Cray. It consists of Intel Haswell Xeon processors, with optional Nvidia Tesla or Intel Xeon Phi accelerators, connected together by Cray's proprietary "Aries" interconnect, stored in air-cooled or liquid-cooled cabinets. The XC series supercomputers are available with the Cray DataWarp applications I/O accelerator technology.

An AI accelerator, deep learning processor, or neural processing unit (NPU) is a class of specialized hardware accelerator or computer system designed to accelerate artificial intelligence and machine learning applications, including artificial neural networks and machine vision. Typical applications include algorithms for robotics, Internet of Things, and other data-intensive or sensor-driven tasks. They are often manycore designs and generally focus on low-precision arithmetic, novel dataflow architectures or in-memory computing capability. As of 2024, a typical AI integrated circuit chip contains tens of billions of MOSFETs.

<span class="mw-page-title-main">Frontier (supercomputer)</span> American supercomputer

Hewlett Packard Enterprise Frontier, or OLCF-5, is the world's first exascale supercomputer. It is hosted at the Oak Ridge Leadership Computing Facility (OLCF) in Tennessee, United States and became operational in 2022. As of December 2023, Frontier is the world's fastest supercomputer. It is based on the Cray EX and is the successor to Summit (OLCF-4). Frontier achieved an Rmax of 1.102 exaFLOPS, which is 1.102 quintillion floating-point operations per second, using AMD CPUs and GPUs.

<span class="mw-page-title-main">Keren Bergman</span> American electrical engineer and professor

Keren Bergman is an American electrical engineer who is the Charles Batchelor Professor at Columbia University. She also serves as the director of the Lightwave Research Laboratory, a silicon photonics research group at Columbia University. Her research focuses on nano-photonics and particularly optical interconnects for low power, high bandwidth computing applications.

References

  1. 1 2 Bill Dally at the Mathematics Genealogy Project
  2. ↑ Date information sourced from Library of Congress Authorities data, via corresponding WorldCat Identities  linked authority file (LAF) .
  3. ↑ "President Biden Announces Members of President's Council of Advisors on Science and Technology" (Press release). Washington: The White House. 2021-09-22.
  4. ↑ "William Dally, PhD". PCAST . The White House. Retrieved 2023-06-11.
  5. ↑ Johnson, Matt (2011). An Analysis of Linux Scalability to Many Cores. p. 4. Locality is efficiency, Efficiency is power, Power is performance, Performance is king
  6. 1 2 "William Dally". Research. Nvidia . Retrieved 2023-06-11.
  7. 1 2 William Dally (November 4, 2011). "From Science to Technology, From Research to Product" (PDF). Slides from Norway Science Week. Stanford Engineering . Retrieved March 7, 2017.
  8. ↑ "Practical AI #15: Artificial intelligence at NVIDIA with Chief Scientist Bill Dally". Changelog. 8 October 2018. Retrieved 2019-04-25. I was on the faculty at MIT for 11 years, where I built a research group that built a number of pioneering supercomputers,
  9. ↑ "Bill Dally". Stanford HAI. Retrieved 2023-06-11.
  10. ↑ "Practical AI #15: Artificial intelligence at NVIDIA with Chief Scientist Bill Dally". Changelog. 8 October 2018. Retrieved 2019-04-25.
  11. ↑ "Nvidia Names Stanford's Bill Dally Chief Scientist, VP Of Research". Press release. January 28, 2009. Archived from the original on February 3, 2009. Retrieved March 7, 2017.
  12. ↑ Ashlee Vance (April 8, 2009). "Hello, Dally: Nvidia Scientist Breaks Silence, Criticizes Intel". The New York Times. Retrieved March 10, 2017.
  13. ↑ MarketScreener. "William Dally - Biography". www.marketscreener.com. Retrieved 2023-06-11.
  14. ↑ Dally, Bill (2023-08-27). "Hardware for Deep Learning". 2023 IEEE Hot Chips 35 Symposium (HCS). IEEE: 1–58. doi:10.1109/hcs59251.2023.10254716. ISBN   979-8-3503-3907-9. S2CID   263180552.
  15. ↑ HOTI 2023 - Day 1: Session 2 - Keynote by Bill Dally (NVIDIA): Accelerator Clusters , retrieved 2024-03-09
  16. ↑ "Accelerator Clusters: the New Supercomputer | HotI30 (2023)". 2023-07-12. Retrieved 2024-03-09.
  17. ↑ Manipatruni, Sasikanth; Chen, Long; Lipson, Michal (2010-07-23). "Ultra high bandwidth WDM using silicon microring modulators". Optics Express. 18 (16): 16858–16867. Bibcode:2010OExpr..1816858M. doi: 10.1364/oe.18.016858 . ISSN   1094-4087. PMID   20721078.
  18. ↑ Manipatruni, Sasikanth; Lipson, Michal; Young, Ian A. (March 2013). "Device Scaling Considerations for Nanophotonic CMOS Global Interconnects". IEEE Journal of Selected Topics in Quantum Electronics. 19 (2): 8200109. arXiv: 1207.6819 . Bibcode:2013IJSTQ..1900109M. doi:10.1109/JSTQE.2013.2239262. ISSN   1558-4542. S2CID   6589733.
  19. ↑ Chen, Long; Preston, Kyle; Manipatruni, Sasikanth; Lipson, Michal (2009). "Integrated GHZ silicon photonic interconnect with micrometer-scale modulators and detectors". Optics Express. 17 (17): 15248–15256. arXiv: 0907.0022 . Bibcode:2009OExpr..1715248C. doi:10.1364/oe.17.015248. PMID   19688003 . Retrieved 2024-03-09.
  20. ↑ "ACM Award Citation". Association for Computing Machinery. Archived from the original on 2 April 2012. Retrieved 25 October 2010.
  21. ↑ Ball, Charles H. (September 30, 1992). "Dally Has Harrowing Flight Mishap". MIT News.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.