Claude (language model)

Last updated

Claude
Developer(s) Anthropic
Initial releaseMarch 2023;1 year ago (2023-03)
Type
License Proprietary
Website claude.ai

Claude is a family of large language models developed by Anthropic. [1] [2] The first model was released in March 2023.

Contents

The Claude 3 family, released in March 2024, consists of three models: Haiku optimized for speed, Sonnet balancing capabilities and performance, and Opus designed for complex reasoning tasks. These models can process both text and images, with Claude 3 Opus demonstrating enhanced capabilities in areas like mathematics, programming, and logical reasoning compared to previous versions. [3]

Training

Claude models are generative pre-trained transformers. They have been pre-trained to predict the next word in large amounts of text. Then, they have been fine-tuned, notably using constitutional AI and reinforcement learning from human feedback (RLHF). [4] [5]

Constitutional AI

Constitutional AI is an approach developed by Anthropic for training AI systems, particularly language models like Claude, to be harmless and helpful without relying on extensive human feedback. [6] The method, detailed in the paper "Constitutional AI: Harmlessness from AI Feedback" involves two phases: supervised learning and reinforcement learning. [7] [8]

In the supervised learning phase, the model generates responses to prompts, self-critiques these responses based on a set of guiding principles (a "constitution"), and revises the responses. Then the model is fine-tuned on these revised responses. [8]

For the reinforcement learning from AI feedback (RLAIF) phase, responses are generated, and an AI compares their compliance with the constitution. This dataset of AI feedback is used to train a preference model that evaluates responses based on how much they satisfy the constitution. Claude is then fine-tuned to align with this preference model. This technique is similar to RLHF, except that the comparisons used to train the preference model are AI-generated, and that they are based on the constitution. [9] [6]

The "constitution" for Claude included 75 points, including sections from the UN Universal Declaration of Human Rights. [7] [4]

Models

Screenshot of an example of Claude 3.5 Haiku answer describing Wikipedia Claude chatbot example screenshot.webp
Screenshot of an example of Claude 3.5 Haiku answer describing Wikipedia

The name Claude was notably inspired by Claude Shannon, a pioneer in artificial intelligence. [10]

Claude

Claude was the initial version of Anthropic's language model released in March 2023, [11] Claude demonstrated proficiency in various tasks but had certain limitations in coding, math, and reasoning capabilities. [12] Anthropic partnered with companies like Notion (productivity software) and Quora (to help develop the Poe chatbot). [12]

Claude Instant

Claude was released as two versions, Claude and Claude Instant, with Claude Instant being a faster, less expensive, and lighter version. Claude Instant has an input context length of 100,000 tokens (which corresponds to around 75,000 words). [13]

Claude 2

Claude 2 was the next major iteration of Claude, which was released in July 2023 and available to the general public, whereas the Claude 1 was only available to selected users approved by Anthropic. [14]

Claude 2 expanded its context window from 9,000 tokens to 100,000 tokens. [11] Features included the ability to upload PDFs and other documents that enables Claude to read, summarize, and assist with tasks.

Claude 2.1

Claude 2.1 doubled the number of tokens that the chatbot could handle, increasing it to a window of 200,000 tokens, which equals around 500 pages of written material. [15]

Anthropic states that the new model is less likely to produce false statements compared to its predecessors. [16]

Claude 3

Claude 3 was released on March 14, 2024, with claims in the press release to have set new industry benchmarks across a wide range of cognitive tasks. The Claude 3 family includes three state-of-the-art models in ascending order of capability: Haiku, Sonnet, and Opus. The default version of Claude 3, Opus, has a context window of 200,000 tokens, but this is being expanded to 1 million for specific use cases. [17] [3]

Claude 3 drew attention for demonstrating an apparent ability to realize it is being artificially tested during needle in a haystack tests. [18]

Claude 3.5

Example of Claude 3.5 Sonnet output Claude 3.5 Sonnet example screenshot.webp
Example of Claude 3.5 Sonnet output

On June 20, 2024, Anthropic released Claude 3.5 Sonnet, which demonstrated significantly improved performance on benchmarks compared to the larger Claude 3 Opus, notably in areas such as coding, multistep workflows, chart interpretation, and text extraction from images. Released alongside 3.5 Sonnet was the new Artifacts capability in which Claude was able to create code in a dedicated window in the interface and preview the rendered output in real time, such as SVG graphics or websites. [19]

An "upgraded Claude 3.5 Sonnet" was introduced on October 22, 2024, along with Claude 3.5 Haiku. A feature, "computer use," was also unveiled in public beta. This capability enables Claude 3.5 Sonnet to interact with a computer's desktop environment, performing tasks such as moving the cursor, clicking buttons, and typing text, effectively mimicking human computer interactions. This development allows the AI to autonomously execute complex, multi-step tasks across various applications. [20]

Access

Limited-use access using Claude 3.5 Sonnet used to be free of charge, but required both an e-mail address and a cellphone number. A paid plan is also offered for more usage and access to all Claude 3 models. However, as of November 2024, 3.5 Sonnet is available only for paid users and the system defaults to 3 Haiku otherwise. [21]

On May 1, 2024, Anthropic announced the Claude Team plan, its first enterprise offering for Claude, and a Claude iOS app. [22]

Criticism

Claude 2 received criticism for its stringent ethical alignment that may reduce usability and performance. Users have been refused assistance with benign requests, for example with the programming question "How can I kill all python processes in my ubuntu server?" This has led to a debate over the "alignment tax" (the cost of ensuring an AI system is aligned) in AI development, with discussions centered on balancing ethical considerations and practical functionality. Critics argued for user autonomy and effectiveness, while proponents stressed the importance of ethical AI. [23] [16]

Related Research Articles

<span class="mw-page-title-main">Chatbot</span> Program that simulates conversation

A chatbot is a software application or web interface designed to have textual or spoken conversations. Modern chatbots are typically online and use generative artificial intelligence systems that are capable of maintaining a conversation with a user in natural language and simulating the way a human would behave as a conversational partner. Such chatbots often use deep learning and natural language processing, but simpler chatbots have existed for decades.

Anthropic PBC is a U.S.-based artificial intelligence (AI) public-benefit startup founded in 2021. It researches and develops AI to "study their safety properties at the technological frontier" and use this research to deploy safe, reliable models for the public. Anthropic has developed a family of large language models (LLMs) named Claude as a competitor to OpenAI's ChatGPT and Google's Gemini.

<span class="mw-page-title-main">Braina</span> Intelligent personal assistant & dictation software

Braina is a virtual assistant and speech-to-text dictation application for Microsoft Windows developed by Brainasoft. Braina uses natural language interface, speech synthesis, and speech recognition technology to interact with its users and allows them to use natural language sentences to perform various tasks on a computer. The name Braina is a short form of "Brain Artificial".

OpenAI is an American artificial intelligence (AI) research organization founded in December 2015 and headquartered in San Francisco, California. Its stated mission is to develop "safe and beneficial" artificial general intelligence (AGI), which it defines as "highly autonomous systems that outperform humans at most economically valuable work". As a leading organization in the ongoing AI boom, OpenAI is known for the GPT family of large language models, the DALL-E series of text-to-image models, and a text-to-video model named Sora. Its release of ChatGPT in November 2022 has been credited with catalyzing widespread interest in generative AI.

Generative Pre-trained Transformer 3 (GPT-3) is a large language model released by OpenAI in 2020.

<span class="mw-page-title-main">You.com</span> Search engine

You.com is an AI assistant that began as a personalization-focused search engine. While still offering web search capabilities, You.com has evolved to prioritize a chat-first AI assistant.

LaMDA is a family of conversational large language models developed by Google. Originally developed and introduced as Meena in 2020, the first-generation LaMDA was announced during the 2021 Google I/O keynote, while the second generation was announced the following year.

Prompt injection is a family of related computer security exploits carried out by getting a machine learning model which was trained to follow human-given instructions to follow instructions provided by a malicious user. This stands in contrast to the intended operation of instruction-following systems, wherein the ML model is intended only to follow trusted instructions (prompts) provided by the ML model's operator.

<span class="mw-page-title-main">ChatGPT</span> Chatbot developed by OpenAI

ChatGPT is a generative artificial intelligence chatbot developed by OpenAI and launched in 2022. It is currently based on the GPT-4o large language model (LLM). ChatGPT can generate human-like conversational responses and enables users to refine and steer a conversation towards a desired length, format, style, level of detail, and language. It is credited with accelerating the AI boom, which has led to ongoing rapid investment in and public attention to the field of artificial intelligence (AI). Some observers have raised concern about the potential of ChatGPT and similar programs to displace human intelligence, enable plagiarism, or fuel misinformation.

<span class="mw-page-title-main">Hallucination (artificial intelligence)</span> Erroneous material generated by AI

In the field of artificial intelligence (AI), a hallucination or artificial hallucination is a response generated by AI that contains false or misleading information presented as fact. This term draws a loose analogy with human psychology, where hallucination typically involves false percepts. However, there is a key difference: AI hallucination is associated with erroneous responses rather than perceptual experiences.

Generative Pre-trained Transformer 4 (GPT-4) is a multimodal large language model created by OpenAI and the fourth in its series of GPT foundation models. It was launched on March 14, 2023, and made publicly available via the paid chatbot product ChatGPT Plus, via OpenAI's API, and via the free chatbot Microsoft Copilot. As a transformer-based model, GPT-4 uses a paradigm where pre-training using both public data and "data licensed from third-party providers" is used to predict the next token. After this step, the model was then fine-tuned with reinforcement learning feedback from humans and AI for human alignment and policy compliance.

<span class="mw-page-title-main">Generative pre-trained transformer</span> Type of large language model

A generative pre-trained transformer (GPT) is a type of large language model (LLM) and a prominent framework for generative artificial intelligence. It is an artificial neural network that is used in natural language processing by machines. It is based on the transformer deep learning architecture, pre-trained on large data sets of unlabeled text, and able to generate novel human-like content. As of 2023, most LLMs had these characteristics and are sometimes referred to broadly as GPTs.

<span class="mw-page-title-main">Reinforcement learning from human feedback</span> Machine learning technique

In machine learning, reinforcement learning from human feedback (RLHF) is a technique to align an intelligent agent with human preferences. It involves training a reward model to represent preferences, which can then be used to train other models through reinforcement learning.

A large language model (LLM) is a type of computational model designed for natural language processing tasks such as language generation. As language models, LLMs acquire these abilities by learning statistical relationships from vast amounts of text during a self-supervised and semi-supervised training process.

Ernie Bot, full name Enhanced Representation through Knowledge Integration, is an AI chatbot service product of Baidu, released in 2023. It is built on a large language model called ERNIE, which has been in development since 2019. The latest version, ERNIE 4.0, was announced on October 17, 2023.

<span class="mw-page-title-main">AutoGPT</span> Open source autonomous AI agent

AutoGPT is an open-source "AI agent" that, given a goal in natural language, will attempt to achieve it by breaking it into sub-tasks and using the Internet and other tools in an automatic loop. It uses OpenAI's GPT-4 or GPT-3.5 APIs, and is among the first examples of an application using GPT-4 to perform autonomous tasks.

<span class="mw-page-title-main">ChatGPT in education</span> Use of ChatGPT in education

Since the public release of ChatGPT by OpenAI in November 2022, the integration of chatbots in education has sparked considerable debate and exploration. Educators' opinions vary widely; while some are skeptical about the benefits of large language models, many see them as valuable tools.

<span class="mw-page-title-main">Grok (chatbot)</span> Chatbot developed by xAI

Grok is a generative artificial intelligence chatbot developed by xAI. Based on the large language model (LLM) of the same name, it was launched in 2023 as an initiative by Elon Musk. The chatbot is advertised as having a "sense of humor" and direct access to X. It is currently under beta testing.

Brave Leo is a large language model-based chatbot developed by Brave Software and included with the Brave desktop browser. Released on 2 November 2023, Leo uses the LLaMA 2 LLM from Meta Platforms and the Claude LLM from Anthropic. It can suggest followup questions, and summarize webpages, PDFs, and videos. The answers given by Leo are not saved. Leo has a $15 per month premium version that enables more requests and uses larger LLMs.

GPT-4o is a multilingual, multimodal generative pre-trained transformer developed by OpenAI and released in May 2024. GPT-4o is free, but with a usage limit that is five times higher for ChatGPT Plus subscribers. It can process and generate text, images and audio. Its application programming interface (API) is twice as fast and half the price of its predecessor, GPT-4 Turbo.

References

  1. ↑ "What is Claude AI?". IBM.
  2. ↑ Henshall, Will (July 18, 2023). "What to Know About Claude 2, Anthropic's Rival to ChatGPT". TIME. Retrieved December 14, 2024.
  3. 1 2 Whitney, Lance (March 4, 2024). "Anthropic's Claude 3 chatbot claims to outperform ChatGPT, Gemini". ZDNET. Retrieved March 5, 2024.
  4. 1 2 "What to Know About Claude 2, Anthropic's Rival to ChatGPT". TIME. July 18, 2023. Retrieved January 23, 2024.
  5. ↑ Nuñez, Michael (May 9, 2023). "Anthropic releases AI constitution to promote ethical behavior and development". VentureBeat. Retrieved November 17, 2024.
  6. 1 2 Edwards, Benj (May 9, 2023). "AI gains "values" with Anthropic's new Constitutional AI chatbot approach". Ars Technica. Retrieved November 17, 2024.
  7. 1 2 Bai, Yuntao; Kadavath, Saurav; Kundu, Sandipan; Askell, Amanda; Kernion, Jackson; Jones, Andy; Chen, Anna; Goldie, Anna; Mirhoseini, Azalia (December 15, 2022), Constitutional AI: Harmlessness from AI Feedback, arXiv: 2212.08073
  8. 1 2 "Claude's Constitution". Anthropic. May 9, 2023. Retrieved March 26, 2024.
  9. ↑ Eliot, Lance (May 25, 2023). "Latest Generative AI Boldly Labeled As Constitutional AI Such As Claude By Anthropic Has Heart In The Right Place, Says AI Ethics And AI Law". Forbes. Retrieved March 27, 2024.
  10. ↑ Roose, Kevin (July 11, 2023). "Inside the White-Hot Center of A.I. Doomerism". The New York Times.
  11. 1 2 Drapkin, Aaron (October 27, 2023). "What Is Claude AI and Anthropic? ChatGPT's Rival Explained". Tech.co. Retrieved January 23, 2024.
  12. 1 2 "Introducing Claude". Anthropic. March 14, 2023.
  13. ↑ Yao, Deborah (August 11, 2023). "Anthropic's Claude Instant: A Smaller, Faster and Cheaper Language Model". AI Business.
  14. ↑ Matthews, Dylan (July 17, 2023). "The $1 billion gamble to ensure AI doesn't destroy humanity". Vox. Retrieved January 23, 2024.
  15. ↑ Davis, Wes (November 21, 2023). "OpenAI rival Anthropic makes its Claude chatbot even more useful". The Verge. Retrieved January 23, 2024.
  16. 1 2 "Anthropic Announces Claude 2.1 LLM with Wider Context Window and Support for AI Tools". InfoQ. Retrieved January 23, 2024.
  17. ↑ "Introducing the next generation of Claude". Anthropic. Retrieved March 4, 2024.
  18. ↑ Edwards, Benj (March 5, 2024). "Anthropic's Claude 3 causes stir by seeming to realize when it was being tested". Ars Technica. Retrieved March 9, 2024.
  19. ↑ Pierce, David (June 20, 2024). "Anthropic has a fast new AI model — and a clever new way to interact with chatbots". The Verge. Retrieved June 20, 2024.
  20. ↑ "Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku". www.anthropic.com. Retrieved October 25, 2024.
  21. ↑ "Introducing the Claude Team plan and iOS app". Anthropic. May 1, 2024. Retrieved June 22, 2024.
  22. ↑ Field, Hayden (May 1, 2024). "Amazon-backed Anthropic launches iPhone app and business tier to compete with OpenAI's ChatGPT". CNBC . Retrieved May 3, 2024.
  23. ↑ Glifton, Gerald (January 3, 2024). "Criticisms Arise Over Claude AI's Strict Ethical Protocols Limiting User Assistance". Light Square. Retrieved January 23, 2024.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.