AI Safety Institute

Last updated

An AI Safety Institute (AISI), in general, is a state-backed institute aiming to evaluate and ensure the safety of the most advanced artificial intelligence (AI) models, also called frontier AI models. [1]

Contents

AI safety gained prominence in 2023, notably with public declarations about potential existential risks from AI. During the AI Safety Summit in November 2023, the United Kingdom (UK) and the United States (US) both created their own AISI. During the AI Seoul Summit in May 2024, international leaders agreed to form a network of AI Safety Institutes, comprising institutes from the UK, the US, Japan, France, Germany, Italy, Singapore, South Korea, Australia, Canada and the European Union. [2]

Timeline

In 2023, Rishi Sunak, the Prime Minister of the United Kingdom, expressed his intention to "make the U.K. not just the intellectual home but the geographical home of global AI safety regulation" and unveiled plans for an AI Safety Summit. [3] He emphasized the need for independent safety evaluations, stating that AI companies cannot "mark their own homework". [4] During the summit in November 2023, the UK AISI was officially established as an evolution of the Frontier AI Taskforce, [5] and the US AISI as part of the NIST. Japan followed by launching an AI safety institute in February 2024. [6]

Politico reported in April 2024 that many AI companies had not shared pre-deployment access to their most advanced AI models for evaluation. Meta's president of global affairs Nick Clegg said that many AI companies were waiting for the UK and the US AI Safety Institutes to work out common evaluation rules and procedures. [7] An agreement was indeed concluded between the UK and the US in April 2024 to collaborate on at least one joint safety test. [8] Initially established in London, the UK AI Safety Institute announced in May 2024 that it would open an office in San Francisco, where many AI companies are located. This is part of a plan to "set new, international standards on AI safety", according to UK's technology minister Michele Donelan. [9] [10]

At the AI Seoul Summit in May 2024, the European Union and other countries agreed to create their own AI safety institutes, forming an international network. [2]

United Kingdom

The United Kingdom founded in April 2023 a safety organisation called Frontier AI Taskforce, with an initial budget of £100 million. [11] In November 2023, it evolved into the UK AISI, and continued to be led by Ian Hogarth. The AISI is part of the United Kingdom's Department for Science, Innovation and Technology. [5]

The United Kingdom's AI strategy aims to balance safety and innovation. Unlike the European Union which adopted the AI Act, the UK is reluctant to legislate early, considering that it may lower the sector's growth, and that laws might be rendered obsolete by technological progress. [6]

In May 2024, the institute open-sourced an AI safety tool called "Inspect", which evaluates AI model capabilities such as reasoning and their degree of autonomy. [12]

United States

The US AISI was founded in November 2023 as part of the NIST. This happened the day after the signature of the Executive Order on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. [13] In February 2024, Joe Biden's former economic policy adviser Elizabeth Kelly was appointed to lead it. [14]

In February 2024, the US government created the US AI Safety Institute Consortium (AISIC), regrouping more than 200 organizations such as Google, Anthropic or Microsoft. [15]

In March 2024, a budget of $10 million was allocated. [16] Observers noted that this investment is relatively small, especially considering the presence of many big AI companies in the US. The NIST itself, which hosts the AISI, is also known for its chronic lack of funding. [17] [6] Biden administration's request for additional funding was met with further budget cuts from congressional appropriators. [18] [17]

See also

Related Research Articles

The ethics of artificial intelligence covers a broad range of topics within the field that are considered to have particular ethical stakes. This includes algorithmic biases, fairness, automated decision-making, accountability, privacy, and regulation. It also covers various emerging or potential future challenges such as machine ethics, lethal autonomous weapon systems, arms race dynamics, AI safety and alignment, technological unemployment, AI-enabled misinformation, how to treat certain AI systems if they have a moral status, artificial superintelligence and existential risks.

Ian Hogarth is an investor and entrepreneur. He co-founded Songkick in 2007 and Plural Platform in 2021. Hogarth is the current Chair of the UK Government's AI Foundation Model Taskforce, which conducts artificial intelligence safety research.

Existential risk from artificial intelligence refers to the idea that substantial progress in artificial general intelligence (AGI) could lead to human extinction or an irreversible global catastrophe.

Regulation of artificial intelligence is the development of public sector policies and laws for promoting and regulating artificial intelligence (AI). It is part of the broader regulation of algorithms. The regulatory and policy landscape for AI is an emerging issue in jurisdictions worldwide, including for international organizations without direct enforcement power like the IEEE or the OECD.

<span class="mw-page-title-main">Adam H. Russell</span> American anthropologist

Adam H. Russell is an American anthropologist who serves as Chief Vision Officer of the U.S. AI Safety Institute. He previously served as the acting deputy director of the Advanced Research Projects Agency for Health.

<span class="mw-page-title-main">Premiership of Rishi Sunak</span> Period of Government of the United Kingdom from 2022 to 2024

Rishi Sunak's tenure as Prime Minister of the United Kingdom began on 25 October 2022 when he accepted an invitation from King Charles III to form a government, succeeding Liz Truss, and ended on 5 July 2024 upon his resignation. He is the first British Asian and the first Hindu to hold the office. Sunak's premiership was dominated by the Russian invasion of Ukraine, the Israel-Hamas war, the cost-of-living crisis, and the Rwanda asylum plan. As prime minister, Sunak also served simultaneously as First Lord of the Treasury, Minister for the Civil Service, and Minister for the Union.

Rishi Sunak is a British politician who served as the Prime Minister of the United Kingdom from 2022 to 2024. He has been Leader of the Conservative Party since October 2022; after the 2024 general election in July 2024, he became Leader of the Opposition. The first British Indian to hold those offices, Sunak has been Member of Parliament (MP) for Richmond and Northallerton, previously Richmond (Yorks), since 2015. He is the most recent Conservative Party prime minister.

AI safety is an interdisciplinary field focused on preventing accidents, misuse, or other harmful consequences arising from artificial intelligence (AI) systems. It encompasses machine ethics and AI alignment, which aim to ensure AI systems are moral and beneficial, as well as monitoring AI systems for risks and enhancing their reliability. The field is particularly concerned with existential risks posed by advanced AI models.

Dan Hendrycks is an American machine learning researcher. He serves as the director of the Center for AI Safety.

On May 30, 2023, hundreds of artificial intelligence experts and other notable figures signed the following short Statement on AI Risk:

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Paul Christiano is an American researcher in the field of artificial intelligence (AI), with a specific focus on AI alignment, which is the subfield of AI safety research that aims to steer AI systems toward human interests. He serves as the Head of Safety for the U.S. Artificial Intelligence Safety Institute inside NIST. He formerly led the language model alignment team at OpenAI and became founder and head of the non-profit Alignment Research Center (ARC), which works on theoretical AI alignment and evaluations of machine learning models. In 2023, Christiano was named as one of the TIME 100 Most Influential People in AI.

<span class="mw-page-title-main">Elham Tabassi</span>

Elham Tabassi is an engineer and government leader. She was listed on the inaugural TIME100 Most Influential People in AI. Tabassi led the creation of the United States Artificial Intelligence Risk Management Framework, adopted by both industry and government. Tabassi was selected to serve on the National Artificial Intelligence (AI) Research Resource Task Force. Tabassi began her career in government at the National Institute of Standards and Technology, pioneering various machine learning and computer vision projects with applications in biometrics evaluation and standards, included in over twenty five publications. Her research has been deployed by the FBI and Department of Homeland Security.

<span class="mw-page-title-main">AI Safety Summit</span> 2023 global summit on AI safety

The AI Safety Summit was an international conference discussing the safety and regulation of artificial intelligence. It was held at Bletchley Park, Milton Keynes, United Kingdom, on 1–2 November 2023. It was the first ever global summit on artificial intelligence, and is planned to become a recurring event.

Executive Order 14110, titled Executive Order on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence is the 126th executive order signed by U.S. President Joe Biden. Signed on October 30, 2023, the order defines the administration's policy goals regarding artificial intelligence (AI), and orders executive agencies to take actions pursuant to these goals. The order is considered to be the most comprehensive piece of governance by the United States regarding AI.

Discussions on regulation of artificial intelligence in the United States have included topics such as the timeliness of regulating AI, the nature of the federal regulatory framework to govern and promote AI, including what agency should lead, the regulatory and governing powers of that agency, and how to update regulations in the face of rapidly changing technology, as well as the roles of state governments and courts.

Jade Leung is the Chief Technology Officer of the United Kingdom's AI Safety Institute, where she designs and oversees safety evaluations for frontier AI models.

PauseAI is a global political movement founded in the Netherlands with the stated aim of achieving global coordination to stop the development of artificial intelligence systems more powerful than GPT-4, at least until it is known how to build them safely, and keep them under democratic control. The movement was established in Utrecht in May 2023 by software entrepreneur Joep Meindertsma.

The Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, or SB 1047, is a 2024 California bill intended to "mitigate the risk of catastrophic harms from AI models so advanced that they are not yet known to exist". Specifically, the bill would apply to models which cost more than $100 million to train and were trained using a quantity of computing power greater than 1026 integer or floating-point operations. SB 1047 would apply to all AI companies doing business in California—the location of the company does not matter. The bill creates protections for whistleblowers and requires developers to perform risk assessments of their models prior to release, under the supervision of the Government Operations Agency. It would also establish CalCompute, a University of California public cloud computing cluster for startups, researchers and community groups.

Connor Leahy is a German-American artificial intelligence researcher and entrepreneur known for cofounding EleutherAI and being CEO of AI safety research company Conjecture. He has warned of the existential risk from artificial general intelligence, and has called for regulation such as "a moratorium on frontier AI runs" implemented through a cap on compute.

References

  1. ↑ "Safety institutes to form 'international network' to boost AI research and tests". The Independent. 2024-05-21. Retrieved 2024-07-06.
  2. 1 2 Desmarais, Anna (2024-05-22). "World leaders agree to launch network of AI safety institutes". euronews. Retrieved 2024-06-15.
  3. ↑ Browne, Ryan (2023-06-12). "British Prime Minister Rishi Sunak pitches UK as home of A.I. safety regulation as London bids to be next Silicon Valley". CNBC. Retrieved 2024-06-21.
  4. ↑ "Rishi Sunak: AI firms cannot 'mark their own homework'". BBC. 2023-11-01. Retrieved 2024-06-21.
  5. 1 2 "Introducing the AI Safety Institute". GOV.UK. November 2023. Retrieved 2024-06-15.
  6. 1 2 3 Henshall, Will (April 1, 2024). "U.S., U.K. Announce Partnership to Safety Test AI Models". TIME. Retrieved 2024-07-06.
  7. ↑ "Rishi Sunak promised to make AI safe. Big Tech's not playing ball". Politico. 2024-04-26. Retrieved 2024-06-15.
  8. ↑ David, Emilia (2024-04-02). "US and UK will work together to test AI models for safety threats". The Verge. Retrieved 2024-06-21.
  9. ↑ Coulter, Martin (20 May 2024). "Britain's AI safety institute to open US office". Reuters.
  10. ↑ Browne, Ryan (2024-05-20). "Britain expands AI Safety Institute to San Francisco amid scrutiny over regulatory shortcomings". CNBC. Retrieved 2024-06-15.
  11. ↑ "Initial £100 million for expert taskforce to help UK build and adopt next generation of safe AI". GOV.UK. Retrieved 2024-07-06.
  12. ↑ Wodecki, Ben (May 15, 2024). "AI Safety Institute Launches AI Model Safety Testing Tool Platform". AI Business.
  13. ↑ Henshall, Will (2023-11-01). "Why Biden's AI Executive Order Only Goes So Far". TIME. Retrieved 2024-07-07.
  14. ↑ Henshall, Will (2024-02-07). "Biden Economic Adviser Elizabeth Kelly Picked to Lead AI Safety Testing Body". TIME. Retrieved 2024-07-06.
  15. ↑ Shepardson, David (February 8, 2024). "US says leading AI companies join safety consortium to address risks". Reuters.
  16. ↑ "Majority Leader Schumer Announces First-Of-Its-Kind Funding To Establish A U.S. Artificial Intelligence Safety Institute; Funding Is A Down Payment On Balancing Safety With AI Innovation And Will Aid Development Standards, Tools, And Tests To Ensure AI Systems Operate Safely". www.democrats.senate.gov. 2024-03-07. Retrieved 2024-07-06.
  17. 1 2 Zakrzewski, Cat (2024-03-08). "This agency is tasked with keeping AI safe. Its offices are crumbling". Washington Post. ISSN   0190-8286 . Retrieved 2024-07-06.
  18. ↑ "NIST would 'have to consider' workforce reductions if appropriations cut goes through". FedScoop. 2024-05-24. Retrieved 2024-07-06.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.