Ilya Sutskever

Last updated

Ilya Sutskever
FRS
איליה סוצקבר‎
Илья Суцкевер
Born
Илья́ Ефи́мович Суцке́вер
Ilya Efimovich Sutskever

(1986-12-08) 8 December 1986 (age 37)
Gorky, Russian SFSR, Soviet Union [1] [2]
CitizenshipCanadian, Israeli [3]
Alma mater Open University of Israel
University of Toronto (BS, MS, PhD)
Known for AlexNet
Co-founding OpenAI
Founding SSI Inc.
Scientific career
Fields Machine learning
Neural networks
Artificial intelligence
Deep learning [4]
Institutions University of Toronto
Google Brain
OpenAI
Thesis Training Recurrent Neural Networks  (2013)
Doctoral advisor Geoffrey Hinton [5] [6]
Website www.cs.toronto.edu/~ilya/ OOjs UI icon edit-ltr-progressive.svg

Ilya Sutskever FRS (born 8 December 1986) is a Canadian-Israeli-Russian computer scientist who specializes in machine learning. [4]

Contents

Sutskever has made several major contributions to the field of deep learning. [7] [8] [9] He is notably the co-inventor, with Alex Krizhevsky and Geoffrey Hinton, of AlexNet, a convolutional neural network. [10]

Sutskever co-founded and is a former chief scientist at OpenAI. [11] In 2023, he was one of the members of OpenAI's board that ousted Sam Altman from his position as CEO; Altman returned a week later, and Sutskever stepped down from the board. In June 2024, Sutskever co-founded the company Safe Superintelligence with Daniel Gross and Daniel Levy. [12] [13]

Early life and education

Sutskever was born into a Jewish family [14] in Nizhny Novgorod, Russia (then Gorky, Soviet Union). At the age of 5, he made aliyah with his family and lived in Jerusalem, Israel, [15] [16] until he was 16, when his family moved to Canada. [17] Sutskever attended the Open University of Israel from 2000 to 2002. [18] After moving to Canada, he attended the University of Toronto in Ontario. [18]

Sutskever received a Bachelor of Science in mathematics from the University of Toronto in 2005, [18] [19] [2] [20] a Master of Science in computer science in 2007, [19] [21] and a Doctor of Philosophy in computer science in 2013. [6] [22] [23] His doctoral supervisor was Geoffrey Hinton. [5]

In 2012, Sutskever built AlexNet in collaboration with Hinton and Alex Krizhevsky. To support AlexNet's computing demands, he bought many GTX 580 GPUs online. [24]

Career and research

Sutskever (second from right) at the Stanford Graduate School of Business in 2014 Democratizing Deep Learning with Nervana and Google Brain (15105407149).jpg
Sutskever (second from right) at the Stanford Graduate School of Business in 2014

In 2012, Sutskever spent about two months as a postdoc with Andrew Ng at Stanford University. He then returned to the University of Toronto and joined Hinton's new research company DNNResearch, a spinoff of Hinton's research group. In 2013, Google acquired DNNResearch and hired Sutskever as a research scientist at Google Brain. [25]

At Google Brain, Sutskever worked with Oriol Vinyals and Quoc Viet Le to create the sequence-to-sequence learning algorithm, [26] and worked on TensorFlow. [27] He is also one of the AlphaGo paper's many co-authors. [28]

At the end of 2015, Sutskever left Google to become cofounder and chief scientist of the newly founded organization OpenAI. [29] [30] [31]

Sutskever is considered to have played a key role in the development of ChatGPT. [32] [33] In 2023, he announced that he would co-lead OpenAI's new "Superalignment" project, which is trying to solve the alignment of superintelligences within four years. He wrote that even if superintelligence seems far off, it could happen this decade. [34]

Sutskever was formerly one of the six board members of the nonprofit entity that controls OpenAI. [35] On November 17, 2023, the board fired Sam Altman, saying that "he was not consistently candid in his communications with the board". [36] The Information speculated that the decision was partly driven by conflict over the extent to which the company should commit to AI safety. [37] In an all-hands company meeting shortly after the board meeting, Sutskever said that firing Altman was "the board doing its duty", [38] but the next week, he expressed regret at having participated in Altman's ouster. [39] Altman's firing and Brockman's resignation led three senior researchers to resign from OpenAI. [40] After that, Sutskever stepped down from the OpenAI board. [41] After that, he was absent from OpenAI's office. Some sources suggested he was leading the team remotely, while others said he no longer had access to the team's work. [42]

In May 2024, Sutskever announced his departure from OpenAI to focus on a new project that was "very personally meaningful" to him. His decision followed a turbulent period at OpenAI marked by leadership crises and internal debates about the direction of AI development and alignment protocols. Jan Leike, the other leader of the superalignment project, announced his departure hours later, citing an erosion of safety and trust in OpenAI's leadership. [43]

In June 2024, Sutskever announced Safe Superintelligence Inc., a new company he founded with Daniel Gross and Daniel Levy with offices in Palo Alto and Tel Aviv. [44] In contrast to OpenAI, which releases revenue-generating products, Sutskever said the new company's "first product will be the safe superintelligence, and it will not do anything else up until then". [13] In September 2024, the company announced that it had raised $1 billion from venture capital firms including Andreessen Horowitz, Sequoia Capital, DST Global, and SV Angel. [45]

In an October 2024 interview after winning the Nobel Prize in Physics, Geoffrey Hinton expressed support for Sutskever's decision to fire Altman, emphasizing concerns about AI safety. [46] [47]

Awards and honors

Related Research Articles

<span class="mw-page-title-main">Geoffrey Hinton</span> British computer scientist (born 1947)

Geoffrey Everest Hinton is a British-Canadian computer scientist, cognitive scientist, cognitive psychologist, known for his work on artificial neural networks which earned him the title as the "Godfather of AI".

Artificial general intelligence (AGI) is a type of artificial intelligence (AI) that matches or surpasses human cognitive capabilities across a wide range of cognitive tasks. This contrasts with narrow AI, which is limited to specific tasks. Artificial superintelligence (ASI), on the other hand, refers to AGI that greatly exceeds human cognitive capabilities. AGI is considered one of the definitions of strong AI.

A superintelligence is a hypothetical agent that possesses intelligence surpassing that of the brightest and most gifted human minds. "Superintelligence" may also refer to a property of problem-solving systems whether or not these high-level intellectual competencies are embodied in agents that act in the world. A superintelligence may or may not be created by an intelligence explosion and associated with a technological singularity.

<span class="mw-page-title-main">Activation function</span> Artificial neural network node function

The activation function of a node in an artificial neural network is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear. Modern activation functions include the smooth version of the ReLU, the GELU, which was used in the 2018 BERT model, the logistic (sigmoid) function used in the 2012 speech recognition model developed by Hinton et al, the ReLU used in the 2012 AlexNet computer vision model and in the 2015 ResNet model.

<span class="mw-page-title-main">Sam Altman</span> American entrepreneur and investor (born 1985)

Samuel Harris Altman is an American entrepreneur and investor best known as the chief executive officer of OpenAI since 2019. He is also the Chairman of clean energy companies Oklo Inc. and Helion Energy. Altman is considered to be one of the leading figures of the AI boom. He dropped out of Stanford University after two years and founded Loopt, a mobile social networking service, raising more than $30 million in venture capital. In 2011, Altman joined Y Combinator, a startup accelerator, and was its president from 2014 to 2019.

<span class="mw-page-title-main">Yann LeCun</span> French computer scientist (born 1960)

Yann André LeCun is a French-American computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice President, Chief AI Scientist at Meta.

Google Brain was a deep learning artificial intelligence research team that served as the sole AI branch of Google before being incorporated under the newer umbrella of Google AI, a research division at Google dedicated to artificial intelligence. Formed in 2011, it combined open-ended machine learning research with information systems and large-scale computing resources. It created tools such as TensorFlow, which allow neural networks to be used by the public, and multiple internal AI research projects, and aimed to create research opportunities in machine learning and natural language processing. It was merged into former Google sister company DeepMind to form Google DeepMind in April 2023.

Multimodal learning is a type of deep learning that integrates and processes multiple types of data, referred to as modalities, such as text, audio, images, or video. This integration allows for a more holistic understanding of complex data, improving model performance in tasks like visual question answering, cross-modal retrieval, text-to-image generation, aesthetic ranking, and image captioning.

<span class="mw-page-title-main">Yoshua Bengio</span> Canadian computer scientist

Yoshua Bengio is a Canadian computer scientist, most noted for his work on artificial neural networks and deep learning. He is a professor at the Department of Computer Science and Operations Research at the Université de Montréal and scientific director of the Montreal Institute for Learning Algorithms (MILA).

OpenAI is an American artificial intelligence (AI) research organization founded in December 2015 and headquartered in San Francisco, California. Its mission is to develop "safe and beneficial" artificial general intelligence (AGI), which it defines as "highly autonomous systems that outperform humans at most economically valuable work". As a leading organization in the ongoing AI boom, OpenAI is known for the GPT family of large language models, the DALL-E series of text-to-image models, and a text-to-video model named Sora. Its release of ChatGPT in November 2022 has been credited with catalyzing widespread interest in generative AI.

Wojciech Zaremba is a Polish computer scientist, a founding team member of OpenAI (2016–present), where he leads both the Codex research and language teams. The teams actively work on AI that writes computer code and creating successors to GPT-3 respectively.

<span class="mw-page-title-main">AlexNet</span> An influential convolutional neural network published in 2012

AlexNet is the name of a convolutional neural network (CNN) architecture, designed by Alex Krizhevsky in collaboration with Ilya Sutskever and Geoffrey Hinton, who was Krizhevsky's Ph.D. advisor at the University of Toronto. It had 60 million parameters and 650,000 neurons.

Alex Krizhevsky is a Ukrainian-born Canadian computer scientist most noted for his work on artificial neural networks and deep learning. In 2012, Krizhevsky, Ilya Sutskever and their PhD advisor Geoffrey Hinton, at the University of Toronto, developed a powerful visual-recognition network AlexNet using only two GeForce NVIDIA GPU cards. This revolutionized research in neural networks. Previously neural networks were trained on CPUs. The transition to GPUs opened the way to the development of advanced AI models. AlexNet won the ImageNet challenge in 2012. Krizhevsky and Sutskever sold their startup, DNN Research Inc., to Google, shortly after winning the contest. Krizhevsky left Google in September 2017 after losing interest in the work, to work at the company Dessa in support of new deep-learning techniques. Many of his numerous papers on machine learning and computer vision are frequently cited by other researchers. He is also the main author of the CIFAR-10 and CIFAR-100 datasets.

Generative Pre-trained Transformer 3 (GPT-3) is a large language model released by OpenAI in 2020.

<span class="mw-page-title-main">GPT-2</span> 2019 text-generating language model

Generative Pre-trained Transformer 2 (GPT-2) is a large language model by OpenAI and the second in their foundational series of GPT models. GPT-2 was pre-trained on a dataset of 8 million web pages. It was partially released in February 2019, followed by full release of the 1.5-billion-parameter model on November 5, 2019.

<span class="mw-page-title-main">Generative pre-trained transformer</span> Type of large language model

A generative pre-trained transformer (GPT) is a type of large language model (LLM) and a prominent framework for generative artificial intelligence. It is an artificial neural network that is used in natural language processing by machines. It is based on the transformer deep learning architecture, pre-trained on large data sets of unlabeled text, and able to generate novel human-like content. As of 2023, most LLMs had these characteristics and are sometimes referred to broadly as GPTs.

<span class="mw-page-title-main">Greg Brockman</span> American entrepreneur, investor and software developer (born 1987)

Greg Brockman is an American entrepreneur, investor and software developer who is a co-founder and currently the president of OpenAI. He began his career at Stripe in 2010, upon leaving MIT, and became their CTO in 2013. He left Stripe in 2015 to co-found OpenAI, where he also assumed the role of CTO.

<span class="mw-page-title-main">Removal of Sam Altman from OpenAI</span> 2023 business action

On November 17, 2023, OpenAI's board of directors ousted co-founder and chief executive Sam Altman after the board had no confidence in his leadership. The removal was caused by concerns about his handling of artificial intelligence safety, and allegations of abusive behavior. Altman was reinstated on November 22 after pressure from employees and investors.

<span class="mw-page-title-main">Attention Is All You Need</span> 2017 research paper by Google

"Attention Is All You Need" is a 2017 landmark research paper in machine learning authored by eight scientists working at Google. The paper introduced a new deep learning architecture known as the transformer, based on the attention mechanism proposed in 2014 by Bahdanau et al. It is considered a foundational paper in modern artificial intelligence, as the transformer approach has become the main architecture of large language models like those based on GPT. At the time, the focus of the research was on improving Seq2seq techniques for machine translation, but the authors go further in the paper, foreseeing the technique's potential for other tasks like question answering and what is now known as multimodal Generative AI.

Safe Superintelligence Inc. or SSI Inc. is an American artificial intelligence company founded by Ilya Sutskever, Daniel Gross and Daniel Levy. The company's mission is to focus on safely developing a superintelligence, an agent capable of surpassing human intelligence.

References

  1. ↑ "Heard It Through the AI | University of Toronto Magazine". University of Toronto Magazine. 28 September 2022. Archived from the original on 29 May 2023. Retrieved 9 October 2022.
  2. 1 2 "Season 1 Ep. 22 Ilya Sutskever". The Robot Brains Podcast. 21 September 2021. Archived from the original on 1 May 2023. Retrieved 14 August 2022– via YouTube.
  3. ↑ "OpenAI's Israeli co-founder departs as US gears up regulation". Jewish News Syndicate. 17 May 2024.
  4. 1 2 Ilya Sutskever publications indexed by Google Scholar OOjs UI icon edit-ltr-progressive.svg
  5. 1 2 Ilya Sutskever at the Mathematics Genealogy Project OOjs UI icon edit-ltr-progressive.svg
  6. 1 2 Sutskever, Ilya (2013). Training Recurrent Neural Networks. utoronto.ca (PhD thesis). University of Toronto. hdl:1807/36012. OCLC   889910425. ProQuest   1501655550. Archived from the original on 26 March 2023. Retrieved 24 March 2023.
  7. ↑ Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E (2012). "ImageNet Classification with Deep Convolutional Neural Networks". Advances in Neural Information Processing Systems. 25. Curran Associates, Inc.
  8. ↑ Srivastava, Nitish; Hinton, Geoffrey; Krizhevsky, Alex; Sutskever, Ilya; Salakhutdinov, Ruslan (2014). "Dropout: A Simple Way to Prevent Neural Networks from Overfitting". Journal of Machine Learning Research. 15 (56): 1929–1958. ISSN   1533-7928.
  9. ↑ Sutskever, Ilya; Vinyals, Oriol; Le, Quoc V. (8 December 2014). "Sequence to sequence learning with neural networks". Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2. NIPS'14. Cambridge, MA, USA: MIT Press: 3104–3112. arXiv: 1409.3215 .
  10. ↑ Alex Krizhevsky; Ilya Sutskever; Geoffrey E. Hinton (24 May 2017). "ImageNet classification with deep convolutional neural networks". Communications of the ACM . 60 (6): 84–90. doi:10.1145/3065386. ISSN   0001-0782. Wikidata   Q59445836.
  11. ↑ Metz, Cade (19 April 2018). "A.I. Researchers Are Making More Than $1 Million, Even at a Nonprofit". The New York Times . Archived from the original on 1 March 2023. Retrieved 22 October 2018.
  12. ↑ "Safe Superintelligence Inc". SSI. 19 June 2024. Retrieved 19 June 2024.
  13. 1 2 Vance, Ashlee (19 June 2024). "Ilya Sutskever Has a New Plan for Safe Superintelligence". Bloomberg Businessweek . Retrieved 19 June 2024.
  14. ↑ Titcomb, James (16 May 2024). "How ChatGPT's Russian-born genius sealed his fate after a shocking coup" . The Daily Telegraph . Retrieved 29 September 2024.
  15. ↑ מן, יובל (18 November 2022). ""הבינה המלאכותית מהמדע הבדיוני תהפוך למציאות"". Ynet (in Hebrew). Archived from the original on 18 April 2023. Retrieved 18 April 2023.
  16. ↑ Wrobel, Sharon (20 June 2024). "OpenAI co-founder creates a new AI startup with a research lab in Tel Aviv" . The Times of Israel . Retrieved 29 September 2024.
  17. ↑ Ansari, Tasmia (7 March 2023). "The Brain That Supercharged ChatGPT, ImageNet and TF". Analytics India Magazine. Archived from the original on 14 March 2023. Retrieved 14 March 2023.
  18. 1 2 3 "Neural networking". The Varsity. 25 October 2010. Archived from the original on 8 June 2020. Retrieved 14 August 2022.
  19. 1 2 Johnston, Jessica Leigh (8 December 2010). "A Neural Network for a New Millennium". University of Toronto Magazine. Archived from the original on 14 April 2023. Retrieved 14 August 2022.
  20. ↑ Ilya Sutskever on LinkedIn OOjs UI icon edit-ltr-progressive.svg
  21. ↑ Sutskever, Ilya (2007). Nonlinear multilayered sequence models. utoronto.ca (MSc thesis). University of Toronto. hdl:1807/119676. OCLC   234120052. Archived from the original on 24 March 2023. Retrieved 24 March 2023.
  22. ↑ "RAM Workshop". thespermwhale.com. Archived from the original on 5 December 2022. Retrieved 14 August 2022.
  23. ↑ "Episode 85: A Conversation with Ilya Sutskever". Voices in AI. Gigaom. Archived from the original on 22 March 2023. Retrieved 14 August 2022.
  24. ↑ "Exclusive: Ilya Sutskever, OpenAI's chief scientist, on his hopes and fears for the future of AI". MIT Technology Review. Archived from the original on 1 November 2023. Retrieved 1 November 2023.
  25. ↑ McMillan, Robert (13 March 2013). "Google Hires Brains that Helped Supercharge Machine Learning". wired.com. Archived from the original on 16 June 2018. Retrieved 9 November 2019.
  26. 1 2 Anon (2022). "Ilya Sutskever". royalsociety.org. London: Royal Society. Archived from the original on 15 March 2023. Retrieved 12 May 2022.
  27. ↑ Martín Abadi; Ashish Agarwal; Paul Barham; et al. (16 March 2016). TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (PDF). arXiv: 1603.04467 . Wikidata   Q29040034.
  28. ↑ David Silver; Aja Huang; Chris J. Maddison; et al. (27 January 2016). "Mastering the game of Go with deep neural networks and tree search". Nature . 529 (7587): 484–489. doi:10.1038/NATURE16961. ISSN   1476-4687. PMID   26819042. Wikidata   Q28005460.
  29. ↑ "OpenAI Blog". 12 December 2015. Archived from the original on 6 October 2021. Retrieved 17 July 2016.
  30. ↑ www.cs.toronto.edu/~ilya/ OOjs UI icon edit-ltr-progressive.svg
  31. ↑ Metz, Cade (27 April 2016). "Inside OpenAI, Elon Musk's Wild Plan to Set Artificial Intelligence Free". wire.com. Archived from the original on 27 April 2016. Retrieved 9 November 2019.
  32. ↑ Ansari, Tasmia (7 March 2023). "The Brain That Supercharged ChatGPT, ImageNet and TF". AIM. Retrieved 17 May 2024.
  33. ↑ Heaven, Will Douglas (2 May 2023). "Geoffrey Hinton tells us why he's now scared of the tech he helped build". MIT Technology Review. Retrieved 17 May 2024.
  34. ↑ "Introducing Superalignment". openai.com. Archived from the original on 14 July 2023. Retrieved 14 July 2023.
  35. ↑ "Our structure". Archived from the original on 29 July 2023. Retrieved 20 November 2023. OpenAI is governed by the board of the OpenAI Nonprofit, composed of OpenAI Global, LLC employees Greg Brockman (Chairman & President), Ilya Sutskever (Chief Scientist), and Sam Altman (CEO), and non-employees Adam D'Angelo, Tasha McCauley, Helen Toner.
  36. ↑ Metz, Cade (17 November 2023). "OpenAI's Board Pushes Out Sam Altman, Its High-Profile C.E.O." The New York Times.
  37. ↑ "Before OpenAI Ousted Altman, Employees Disagreed Over AI 'Safety'". The Information. Archived from the original on 18 November 2023. Retrieved 20 November 2023.
  38. ↑ Edwards, Benj (18 November 2023). "Details emerge of surprise board coup that ousted CEO Sam Altman at OpenAI". Ars Technica. Archived from the original on 19 November 2023. Retrieved 20 November 2023.
  39. ↑ Rosenberg, Scott (20 November 2023). "OpenAI's Sutskever says he regrets board's firing of Altman". Axios. Retrieved 20 November 2023.
  40. ↑ Thompson, Polly. "3 senior OpenAI researchers resign in the wake of Sam Altman's shock dismissal as CEO, report says". Business Insider. Archived from the original on 18 November 2023. Retrieved 18 November 2023.
  41. ↑ "OpenAI announces return of Sam Altman as CEO". 22 November 2023.
  42. ↑ Kahn, Jeremy (21 May 2024). "OpenAI promised 20% of its computing power to combat the most dangerous kind of AI—but never delivered, sources say". Fortune. Retrieved 22 May 2024.
  43. ↑ Samuel, Sigal (17 May 2024). ""I lost trust": Why the OpenAI team in charge of safeguarding humanity imploded". Vox. Retrieved 18 May 2024.
  44. ↑ Tabassum, Juveria (19 June 2024). "Former OpenAI chief scientist to start a new AI company". Reuters.
  45. ↑ Kai, Kenrick; Hu, Krystal; Tong, Anna (4 September 2024). "OpenAI co-founder Sutskever's new safety-focused AI startup SSI raises $1 billion". reuters.com. Retrieved 4 September 2024.
  46. ↑ Hetzner, Christiaan. "New Nobel Prize winner, AI godfather Geoffrey Hinton, says he's proud his student fired OpenAI boss Sam Altman". Fortune. Retrieved 12 October 2024.
  47. ↑ Tan, Kwan Wei Kevin. "The Nobel Prize-winning 'godfather of AI' says he's glad one of his students had a hand in ousting Sam Altman from OpenAI". Business Insider. Retrieved 12 October 2024.
  48. ↑ "35 Innovators Under 35: Ilya Sutskever". technologyreview.com. Archived from the original on 4 May 2019. Retrieved 17 July 2016.
  49. ↑ Martin, Scott (15 September 2018). "Reinforcement Learning 'Really Works' for AI Against Pro Gamers, OpenAI Trailblazer Says". Nvidia Blog. Nvidia. Archived from the original on 28 March 2023. Retrieved 28 March 2023.
  50. ↑ "The man who revolutionized computer vision, machine translation, games and robotics · AI Frontiers Conference". aifrontiers.com. Archived from the original on 16 June 2023. Retrieved 16 June 2023.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.