Language creation in artificial intelligence

Last updated

In artificial intelligence, researchers teach AI systems to develop their own ways of communicating by having them work together on tasks and use symbols as parts of a new language. These languages might grow out of human languages or be built completely from scratch. When AI is used for translating between languages, it can even create a new shared language to make the process easier. Natural Language Processing (NLP) helps these systems understand and generate human-like language, making it possible for AI to interact and communicate more naturally with people.

Contents

Evolution from English

In 2017 Facebook Artificial Intelligence Research (FAIR) trained chatbots on a corpus of English text conversations between humans playing a simple trading game involving balls, hats, and books. [1] When programmed to experiment with English and tasked with optimizing trades, the chatbots seemed to evolve a reworked version of English to better solve their task. In some cases the exchanges seemed nonsensical: [2] [3] [4]

Bob: "I can can I I everything else"
Alice: "Balls have zero to me to me to me to me to me to me to me to me to"

Facebook's Dhruv Batra said: "There was no reward to sticking to English language. Agents will drift off understandable language and invent codewords for themselves. Like if I say 'the' five times, you interpret that to mean I want five copies of this item." [4] It's often unclear exactly why a neural network decided to produce the output that it did. [2] Because the agents' evolved language was opaque to humans, Facebook modified the algorithm to explicitly provide an incentive to mimic humans. This modified algorithm is preferable in many contexts, even though it scores lower in effectiveness than the opaque algorithm, because clarity to humans is important in many use cases. [1]

In The Atlantic , Adrienne LaFrance analogized the wondrous and "terrifying" evolved chatbot language to cryptophasia, the phenomenon of some twins developing a language that only the two children can understand. [5]

Beginning of the AI language creation

In 2017 researchers at OpenAI demonstrated a multi-agent environment and learning methods that bring about emergence of a basic language ab initio without starting from a pre-existing language. The language consists of a stream of "ungrounded" (initially meaningless) abstract discrete symbols uttered by agents over time, which comes to evolve a defined vocabulary and syntactical constraints. One of the tokens might evolve to mean "blue-agent", another "red-landmark", and a third "goto", in which case an agent will say "goto red-landmark blue-agent" to ask the blue agent to go to the red landmark. In addition, when visible to one another, the agents could spontaneously learn nonverbal communication such as pointing, guiding, and pushing. The researchers speculated that the emergence of AI language might be analogous to the evolution of human communication. [2] [6] [7]

Similarly, a 2017 study from Abhishek Das (programmer) and colleagues demonstrated the emergence of language and communication in a visual question-answer context, showing that a pair of chatbots can invent a communication protocol that associates ungrounded tokens with colors and shapes. [5] [8]

This shows the language generation and how models were trained from scratch for the AI to understand and build off for human communication and understanding.[ citation needed ]

Interlingua

In 2016, Google deployed to Google Translate an AI designed to directly translate between any of 103 different natural languages, including pairs of languages that it had never before seen translated between. Researchers examined whether the machine learning algorithms were choosing to translate human-language sentences into a kind of "interlingua", and found that the AI was indeed encoding semantics within its structures. The researchers cited this as evidence that a new interlingua, evolved from the natural languages, exists within the network. [2] [9]

Current standpoint of language generation in AI

At the timeline of this page[ when? ], AI generation is at a slow pace. The development of Natural Language Processing (NLP) has changed the game of language generation which is currently being used throughout various generative AI chatbots such as ChatGPT, Bing AI, and Bard AI.[ citation needed ] The whole basis of language generation is through the training of computer models and algorithms which can learn from a large dataset of information. For example, there are mixed sentence models which tend to perform better as they take a larger sampling size of sentenced data rather than just words[10]. These models continuously develop over time through the integration of more data. This allows for better communication over time as more information is being learned from which the AI can feed.

The image on the right portrays how these models are implemented to communicate with users trying to learn about information and things around the world.

Live use of ChatGPT to learn about the Wales Tiananmen Square. ChatGPT Wales Tiananmen Square 2022-12-30 181035.png
Live use of ChatGPT to learn about the Wales Tiananmen Square.

Applications of generative AI

Generative AI for language use has been applicate to industries and markets across the world such as customer service, games, translation, and other technical tasks such as understanding large chunks of data. Focusing in customer service, AI chatbots such as ChatGPT and Bard AI utilize natural language processing (NLP) to work, understand, and communicate with users live to offer responses and opinions depending on the questions asked. They not only mimic human interaction but represent themselves as their own being which allows for one-on-one interaction with users by developing language and their own way of talking. In the field of gaming, non-playable characters (NPC's) are used to better the in game experience by providing insights from the bots and other characters that are implemented in many story-mode and first person shooter (FPS) games. In addition, when using for translation, these generative AI's are able to understand thousands of other languages and translate them to help the user understand information. This is helpful and leads to a larger appeal of an audience. These applications are evolving over time and portray the various uses of language through AI in industries, markets, and daily situations.

Challenges and limitations of AI language creation

Although AI seems to be evolving rapidly, it faces many technical challenges. For example, in many cases the language used by AI is very vague, and thus confusing for the user to understand. In addition, there is a "black-box problem"[11] [10] in which there is a lack of transparency and interpretability in the language of AI outputs. In addition, as premium versions of AI chatbots come forward, they can scrape data from the web, which may lead to biases in the information they present. AI models could accidentally form opinions based on the language (words and sentences) from which they are trained. This is undesirable for a neutral-minded AI.

It is intended to overcome these limitations and challenges in future, as the models learn more language through conversations and information they receive. This will strengthen language creation and aid in the conversational skills and understanding of the AI, which can then be implemented to an acceptable standard.

Ethical risks in AI language development

Many ethical risks arise from the challenges of AI language development and conversation, such as the misuse of these chatbots to create fake information or manipulate others. In addition, there is a strong privacy concern when using chatbots. Many are concerned with the AI saving and selling information. There are many guidelines from journals such as IEEE and the EU that mention the necessary measures "to ensure privacy preservation ... involving sensitive information". [11] That article calls for responsible AI use, especially for sensitive medical data, as explained within the article.

As these technologies advance, it is critical that ethical standards are met, in order to achieve privacy of information and to maintain a neutral standpoint in communicating with users. [10] [12] [13] [14]

Future of AI language creation

In conclusion, as AI technology continue to evolve, the goal is to develop refined systems in which there is a neutral, but informative standpoint from the AI. There are many types of upcoming deep learning and neural network models that will be used to dive deeper and develop multiple layers of checking which will be helpful for the NLP as it will ensure enhanced interactions with users. These integrations and stronger models will lead to a safer environment of communication to prevent biases, any irrational claims, and a better environment within games, customer service, VR/AR systems, and translation within thousands of languages. There is a future towards medical scribing and communication with doctors during live surgeries. The future is promising for generative AI language as it will continue to grow by being trained on millions of new words, sentences, and dialect day by day through the use of intricate computational models[14].

File:Deep Learning in Natural Language Processing.jpeg [ dead link ‍] (this image portrays the intricate modeling of NLP and how it ensures its accuracy during communication)

See also

Related Research Articles

Artificial intelligence (AI), in its broadest sense, is intelligence exhibited by machines, particularly computer systems. It is a field of research in computer science that develops and studies methods and software that enable machines to perceive their environment and use learning and intelligence to take actions that maximize their chances of achieving defined goals. Such machines may be called AIs.

Natural language processing (NLP) is a subfield of computer science and especially artificial intelligence. It is primarily concerned with providing computers with the ability to process data encoded in natural language and is thus closely related to information retrieval, knowledge representation and computational linguistics, a subfield of linguistics. Typically data is collected in text corpora, using either rule-based, statistical or neural-based approaches in machine learning and deep learning.

<span class="mw-page-title-main">Chatbot</span> Program that simulates conversation

A chatbot is a software application or web interface designed to have textual or spoken conversations. Modern chatbots are typically online and use generative artificial intelligence systems that are capable of maintaining a conversation with a user in natural language and simulating the way a human would behave as a conversational partner. Such chatbots often use deep learning and natural language processing, but simpler chatbots have existed for decades.

Natural language generation (NLG) is a software process that produces natural language output. A widely-cited survey of NLG methods describes NLG as "the subfield of artificial intelligence and computational linguistics that is concerned with the construction of computer systems that can produce understandable texts in English or other human languages from some underlying non-linguistic representation of information".

Artificial intelligence marketing (AIM) is a form of marketing that uses artificial intelligence concepts and models such as machine learning, Natural process Languages, and Bayesian Networks to achieve marketing goals. The main difference between AIM and traditional forms of marketing resides in the reasoning, which is performed by a computer algorithm rather than a human.

Artificial intelligence (AI) has been used in applications throughout industry and academia. In a manner analogous to electricity or computers, AI serves as a general-purpose technology. AI programes emulate perception and understanding, and are designed to adapt to new information and new situations. Machine learning has been used for various scientific and commercial purposes including language translation, image recognition, decision-making, credit scoring, and e-commerce.

<span class="mw-page-title-main">Virtual assistant</span> Software agent

A virtual assistant (VA) is a software agent that can perform a range of tasks or services for a user based on user input such as commands or questions, including verbal ones. Such technologies often incorporate chatbot capabilities to simulate human conversation, such as via online chat, to facilitate interaction with their users. The interaction may be via text, graphical interface, or voice - as some virtual assistants are able to interpret human speech and respond via synthesized voices.

<span class="mw-page-title-main">Deep learning</span> Branch of machine learning

Deep learning is a subset of machine learning that focuses on utilizing neural networks to perform tasks such as classification, regression, and representation learning. The field takes inspiration from biological neuroscience and is centered around stacking artificial neurons into layers and "training" them to process data. The adjective "deep" refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

DeepMind Technologies Limited, also known by its trade name Google DeepMind, is a British-American artificial intelligence research laboratory which serves as a subsidiary of Google. Founded in the UK in 2010, it was acquired by Google in 2014 and merged with Google AI's Google Brain division to become Google DeepMind in April 2023. The company is based in London, with research centres in Canada, France, Germany, and the United States.

This glossary of artificial intelligence is a list of definitions of terms and concepts relevant to the study of artificial intelligence (AI), its subdisciplines, and related fields. Related glossaries include Glossary of computer science, Glossary of robotics, and Glossary of machine vision.

In the field of artificial intelligence (AI), AI alignment aims to steer AI systems toward a person's or group's intended goals, preferences, and ethical principles. An AI system is considered aligned if it advances the intended objectives. A misaligned AI system pursues unintended objectives.

Explainable AI (XAI), often overlapping with interpretable AI, or explainable machine learning (XML), either refers to an artificial intelligence (AI) system over which it is possible for humans to retain intellectual oversight, or refers to the methods to achieve this. The main focus is usually on the reasoning behind the decisions or predictions made by the AI which are made more understandable and transparent. This has been brought up again as a topic of active research as users now need to know the safety and explain what automated decision making is in different applications. XAI counters the "black box" tendency of machine learning, where even the AI's designers cannot explain why it arrived at a specific decision.

<span class="mw-page-title-main">Semantic parsing</span>

Semantic parsing is the task of converting a natural language utterance to a logical form: a machine-understandable representation of its meaning. Semantic parsing can thus be understood as extracting the precise meaning of an utterance. Applications of semantic parsing include machine translation, question answering, ontology induction, automated reasoning, and code generation. The phrase was first used in the 1970s by Yorick Wilks as the basis for machine translation programs working with only semantic representations. Semantic parsing is one of the important tasks in computational linguistics and natural language processing.

A conversational user interface (CUI) is a user interface for computers that emulates a conversation with a real human. Historically, computers have relied on text-based user interfaces and graphical user interfaces (GUIs) to translate the user's desired action into commands the computer understands. While an effective mechanism of completing computing actions, there is a learning curve for the user associated with GUI. Instead, CUIs provide opportunity for the user to communicate with the computer in their natural language rather than in a syntax specific commands.

Synthetic media is a catch-all term for the artificial production, manipulation, and modification of data and media by automated means, especially through the use of artificial intelligence algorithms, such as for the purpose of misleading people or changing an original meaning. Synthetic media as a field has grown rapidly since the creation of generative adversarial networks, primarily through the rise of deepfakes as well as music synthesis, text generation, human image synthesis, speech synthesis, and more. Though experts use the term "synthetic media," individual methods such as deepfakes and text synthesis are sometimes not referred to as such by the media but instead by their respective terminology Significant attention arose towards the field of synthetic media starting in 2017 when Motherboard reported on the emergence of AI altered pornographic videos to insert the faces of famous actresses. Potential hazards of synthetic media include the spread of misinformation, further loss of trust in institutions such as media and government, the mass automation of creative and journalistic jobs and a retreat into AI-generated fantasy worlds. Synthetic media is an applied form of artificial imagination.

Generative Pre-trained Transformer 3 (GPT-3) is a large language model released by OpenAI in 2020.

Toloka, based in Amsterdam, is a crowdsourcing and generative AI services provider.

<span class="mw-page-title-main">Meta AI</span> Artificial intelligence division of Meta Platforms

Meta AI is a company owned by Meta that develops artificial intelligence and augmented and artificial reality technologies. Meta AI deems itself an academic research laboratory, focused on generating knowledge for the AI community, and should not be confused with Meta's Applied Machine Learning (AML) team, which focuses on the practical applications of its products.

<span class="mw-page-title-main">Generative pre-trained transformer</span> Type of large language model

A generative pre-trained transformer (GPT) is a type of large language model (LLM) and a prominent framework for generative artificial intelligence. It is an artificial neural network that is used in natural language processing by machines. It is based on the transformer deep learning architecture, pre-trained on large data sets of unlabeled text, and able to generate novel human-like content. As of 2023, most LLMs had these characteristics and are sometimes referred to broadly as GPTs.

References

  1. 1 2 "Chatbots learn how to negotiate and drive a hard bargain". New Scientist. 14 June 2017. Retrieved 24 January 2018.
  2. 1 2 3 4 Baraniuk, Chris (1 August 2017). "'Creepy Facebook AI' story sweeps media". BBC News. Retrieved 24 January 2018.
  3. ↑ "Facebook robots shut down after they talk to each other in language only they understand". The Independent. 31 July 2017. Retrieved 24 January 2018.
  4. 1 2 Field, Matthew (1 August 2017). "Facebook shuts down robots after they invent their own language". The Telegraph. Retrieved 24 January 2018.
  5. 1 2 LaFrance, Adrienne (20 June 2017). "What an AI's Non-Human Language Actually Looks Like". The Atlantic. Retrieved 24 January 2018.
  6. ↑ "It Begins: Bots Are Learning to Chat in Their Own Language". WIRED. 16 March 2017. Retrieved 24 January 2018.
  7. ↑ Mordatch, I., & Abbeel, P. (2017). Emergence of Grounded Compositional Language in Multi-Agent Populations. arXiv : 1703.04908.
  8. ↑ Das, A., Kottur, S., Moura, J. M., Lee, S., & Batra, D. (2017). Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning. arXiv : 1703.06585.
  9. ↑ Johnson, Melvin; Schuster, Mike; Le, Quoc V.; Krikun, Maxim; Wu, Yonghui; Chen, Zhifeng; Thorat, Nikhil; Viégas, Fernanda; Wattenberg, Martin; Corrado, Greg; Hughes, Macduff; Dean, Jeffrey (2017). "Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation". Transactions of the Association for Computational Linguistics. 5: 339–351. arXiv: 1611.04558 . doi:10.1162/tacl_a_00065.
  10. 1 2 khan, Bangul; Fatima, Hajira; Qureshi, Ayatullah; Kumar, Sanjay; Hanan, Abdul; Hussain, Jawad; Abdullah, Saad (2023-02-08). "Drawbacks of Artificial Intelligence and Their Potential Solutions in the Healthcare Sector". Biomedical Materials & Devices (New York, N.Y.). 1 (2): 731–738. doi:10.1007/s44174-023-00063-2. ISSN   2731-4812. PMC   9908503 . PMID   36785697.
  11. ↑ Martinelli, Fabio (28 September 2020). "Enhanced Privacy and Data Protection using Natural Language Processing and Artificial Intelligence". 2020 International Joint Conference on Neural Networks (IJCNN). pp. 1–8. doi:10.1109/IJCNN48605.2020.9206801. ISBN   978-1-7281-6926-2.{{cite book}}: |journal= ignored (help)
  12. ↑ Goodman, Joshua (2001-08-09). "A Bit of Progress in Language Modeling". arXiv: cs/0108005 .
  13. ↑ Martinelli, Fabio (28 September 2020). "Enhanced Privacy and Data Protection using Natural Language Processing and Artificial Intelligence". 2020 International Joint Conference on Neural Networks (IJCNN). pp. 1–8. doi:10.1109/IJCNN48605.2020.9206801. ISBN   978-1-7281-6926-2.{{cite book}}: |journal= ignored (help)
  14. ↑ Rita, Mathieu; Michel, Paul; Chaabouni, Rahma; Pietquin, Olivier; Dupoux, Emmanuel; Strub, Florian (2024-03-18). "Language Evolution with Deep Learning". arXiv: 2403.11958 [cs.CL].
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.