Legal informatics

Last updated

Legal informatics is an area within information science.

Contents

The American Library Association defines informatics as "the study of the structure and properties of information, as well as the application of technology to the organization, storage, retrieval, and dissemination of information." Legal informatics therefore, pertains to the application of informatics within the context of the legal environment and as such involves law-related organizations (e.g., law offices, courts, and law schools) and users of information and information technologies within these organizations. [1]

Policy issues

Policy issues in legal informatics arise from the use of informational technologies in the implementation of law, such as the use of subpoenas for information found in emails, search queries, and social networks. Policy approaches to legal informatics issues vary throughout the world. For example, European countries tend to require the destruction or anonymization of data so that it cannot be used for discovery. [2]

Technology

Cloud computing

The widespread introduction of cloud computing provides several benefits in delivering legal services. Legal service providers can use the Software as a Service model to earn a profit by charging customers a per-use or subscription fee. This model has several benefits over traditional bespoke services.

Software as a service also complicates the attorney-client relationship in a way that may have implications for attorney–client privilege. The traditional delivery model makes it easy to create delineations of when attorney-client privilege attaches and when it does not. But in more complex models of legal service delivery other actors or automated processes may moderate the relationship between a client and their attorney making it difficult to tell which communications should be legally privileged. [3]

Artificial intelligence

Artificial intelligence is employed in online dispute resolution platforms that use optimization algorithms and blind-bidding. [4] Artificial intelligence is also frequently employed in modeling the legal ontology, "an explicit, formal, and general specification of a conceptualization of properties of and relations between objects in a given domain". [5]

Artificial intelligence and law (AI and law) is a subfield of artificial intelligence (AI) mainly concerned with applications of AI to legal informatics problems and original research on those problems. It is also concerned to contribute in the other direction: to export tools and techniques developed in the context of legal problems to AI in general. For example, theories of legal decision making, especially models of argumentation, have contributed to knowledge representation and reasoning; models of social organization based on norms have contributed to multi-agent systems; reasoning with legal cases has contributed to case-based reasoning; and the need to store and retrieve large amounts of textual data has resulted in contributions to conceptual information retrieval and intelligent databases. [6] [7] [8]

History

Although Loevinger, [9] Allen [10] and Mehl [11] anticipated several of the ideas that would become important in AI and Law, the first serious proposal for applying AI techniques to law is usually taken to be Buchanan and Headrick. [12] Early work from this period includes Thorne McCarty's influential TAXMAN project [13] in the US and Ronald Stamper's LEGOL project [14] in the UK. Landmarks in the early 1980s include Carole Hafner's work on conceptual retrieval, [15] Anne Gardner's work on contract law, [16] Rissland's work on legal hypotheticals [17] and the work at Imperial College London on the representation of legislation by means of executable logic programs. [18]

Early meetings of scholars included a one-off meeting at Swansea, [19] the series of conferences organized by IDG in Florence [20] and the workshops organised by Charles Walter at the University of Houston in 1984 and 1985. [21] In 1987 a biennial conference, the International Conference on AI and Law (ICAIL), was instituted. [22] This conference began to be seen as the main venue for publishing and the developing ideas within AI and Law, [23] and it led to the foundation of the International Association for Artificial Intelligence and Law (IAAIL), to organize and convene subsequent ICAILs. This, in turn, led to the foundation of the Artificial Intelligence and Law Journal, first published in 1992. [24] In Europe, the annual JURIX conferences (organised by the Jurix Foundation for Legal Knowledge Based Systems), began in 1988. Initially intended to bring together the Dutch-speaking (i.e. Dutch and Flemish) researchers, JURIX quickly developed into an international, primarily European, conference and since 2002 has regularly been held outside the Dutch speaking countries. [25] Since 2007 the JURISIN workshops have been held in Japan under the auspices of the Japanese Society for Artificial Intelligence. [26]

The interoperable legal documents standardAkoma Ntoso allows machine-driven processes to operate on the syntactic and semantic components of digital parliamentary, judicial and legislative documents, thus facilitating the development of high-quality information resources and forming a basis for AI tools. Its goal is to substantially enhance the performance, accountability, quality and openness of parliamentary and legislative operations based on best practices and guidance through machine-assisted drafting and machine-assisted (legal) analysis. Embedded in the environment of the semantic web, it forms the basis for a heterogenous yet interoperable ecosystem, with which these tools can operate and communicate, as well as for future applications and use cases based on digital law or rule representation. [27]

In 2019, the city of Hangzhou, China established a pilot program artificial intelligence-based Internet Court to adjudicate disputes related to ecommerce and internet-related intellectual property claims. [28] :124 Parties appear before the court via videoconference and AI evaluates the evidence presented and applies relevant legal standards. [28] :124

Scope

Today, AI and law embrace a wide range of topics, including:

  • Formal models of legal reasoning
  • Computational models of argumentation and decision-making
  • Computational models of evidential reasoning
  • Legal reasoning in multi-agent systems
  • Executable models of legislation
  • Automatic legal text classification and summarization
  • Automated information extraction from legal databases and texts
  • Machine learning and data mining for e-discovery and other legal applications
  • Conceptual or model-based legal information retrieval
  • Lawbots to automate minor and repetitive legal tasks [29]
  • Risk assessment, pricing and timeline predictions of litigation using machine learning and artificial intelligence. [30]

Formal models of legal texts and legal reasoning have been used in AI and Law to clarify issues, to give a more precise understanding and to provide a basis for implementations. A variety of formalisms have been used, including propositional and predicate calculi; deontic, temporal and non-monotonic logics; and state transition diagrams. Prakken and Sartor [31] give a detailed and authoritative review of the use of logic and argumentation in AI and Law, together with a comprehensive set of references.

An important role of formal models is to remove ambiguity. In fact, legislation abounds with ambiguity: Because it is written in natural language there are no brackets and so the scope of connectives such as "and" and "or" can be unclear. "Unless" is also capable of several interpretations, and legal draftsman never write "if and only if", although this is often what they intend by "if". In perhaps the earliest use of logic to model law in AI and Law, Layman Allen advocated the use of propositional logic to resolve such syntactic ambiguities in a series of papers. [10]

In the late 1970s and throughout the 1980s a significant strand of work on AI and Law involved the production of executable models of legislation, originating with Thorne McCarty's TAXMAN [13] and Ronald Stamper's LEGOL. [14] TAXMAN was used to model the majority and minority arguments in a US Tax law case (Eisner v Macomber), and was implemented in the micro-PLANNER programming language. LEGOL was used to provide a formal model of the rules and regulations that govern an organization, and was implemented in a condition-action rule language of the kind used for expert systems.

The TAXMAN and LEGOL languages were executable, rule-based languages, which did not have an explicit logical interpretation. However, the formalisation of a large portion of the British Nationality Act by Sergot et al. [18] showed that the natural language of legal documents bears a close resemblance to the Horn clause subset of first order predicate calculus. Moreover, it identified the need to extend the use of Horn clauses by including negative conditions, to represent rules and exceptions. The resulting extended Horn clauses are executable as logic programs.

Later work on larger applications, such as that on Supplementary Benefits, [32] showed that logic programs need further extensions, to deal with such complications as multiple cross references, counterfactuals, deeming provisions, amendments, and highly technical concepts (such as contribution conditions). The use of hierarchical representations [33] was suggested to address the problem of cross reference; and so-called isomorphic [34] representations were suggested to address the problems of verification and frequent amendment. As the 1990s developed this strand of work became partially absorbed into the development of formalisations of domain conceptualisations, (so-called ontologies), which became popular in AI following the work of Gruber. [35] Early examples in AI and Law include Valente's functional ontology [36] and the frame based ontologies of Visser and van Kralingen. [37] Legal ontologies have since become the subject of regular workshops at AI and Law conferences and there are many examples ranging from generic top-level and core ontologies [38] to very specific models of particular pieces of legislation.

Since law comprises sets of norms, it is unsurprising that deontic logics have been tried as the formal basis for models of legislation. These, however, have not been widely adopted as the basis for expert systems, perhaps because expert systems are supposed to enforce the norms, whereas deontic logic becomes of real interest only when we need to consider violations of the norms. [39] In law directed obligations, [40] whereby an obligation is owed to another named individual are of particular interest, since violations of such obligations are often the basis of legal proceedings. There is also some interesting work combining deontic and action logics to explore normative positions. [41]

In the context of multi-agent systems, norms have been modelled using state transition diagrams. Often, especially in the context of electronic institutions, [42] the norms so described are regimented (i.e., cannot be violated), but in other systems violations are also handled, giving a more faithful reflection of real norms. For a good example of this approach see Modgil et al. [43]

Law often concerns issues about time, both relating to the content, such as time periods and deadlines, and those relating to the law itself, such as commencement. Some attempts have been made to model these temporal logics using both computational formalisms such as the Event Calculus [44] and temporal logics such as defeasible temporal logic. [45]

In any consideration of the use of logic to model law it needs to be borne in mind that law is inherently non-monotonic, as is shown by the rights of appeal enshrined in all legal systems, and the way in which interpretations of the law change over time. [46] [47] [48] Moreover, in the drafting of law exceptions abound, and, in the application of law, precedents are overturned as well as followed. In logic programming approaches, negation as failure is often used to handle non-monotonicity, [49] but specific non-monotonic logics such as defeasible logic [50] have also been used. Following the development of abstract argumentation, [51] however, these concerns are increasingly being addressed through argumentation in monotonic logic rather than through the use of non-monotonic logics.

Both academic and proprietary quantitative legal prediction models exist. One of the earliest examples of a working quantitative legal prediction model occurred in the form of the Supreme Court forecasting project. The Supreme Court forecasting model attempted to predict the results of all the cases on the 2002 term of the Supreme Court. The model predicted 75% of cases correctly compared to experts who only predicted 59.1% of cases. [52] Another example of an academic quantitative legal prediction models is a 2012 model that predicted the result of Federal Securities class action lawsuits. [53] Some academics and legal technology startups are attempting to create algorithmic models to predict case outcomes. [54] [55] Part of this overall effort involves improved case assessment for litigation funding. [56]

In order to better evaluate the quality of case outcome prediction systems, a proposal has been made to create a standardised dataset that would allow comparisons between systems. [57]

Within the practice issues conceptual area, progress continues to be made on both litigation and transaction focused technologies. In particular, technology including predictive coding has the potential to effect substantial efficiency gains in law practice. Though predictive coding has largely been applied in the litigation space, it is beginning to make inroads in transaction practice, where it is being used to improve document review in mergers and acquisitions. [58] Other advances, including XML coding in transaction contracts, and increasingly advanced document preparation systems demonstrate the importance of legal informatics in the transactional law space. [59] [60]

Current applications of AI in the legal field utilize machines to review documents, particularly when a high level of completeness and confidence in the quality of document analysis is depended upon, such as in instances of litigation and where due diligence play a role. [61] Predictive coding leverages small samples to cross-reference similar items, weed out less relevant documents so attorneys can focus on the truly important key documents, produces statistically validated results, equal to or surpassing the accuracy and, prominently, the rate of human review. [61]

Delivery of services

Advances in technology and legal informatics have led to new models for the delivery of legal services. Legal services have traditionally been a "bespoke" product created by a professional attorney on an individual basis for each client. [62] However, to work more efficiently, parts of these services will move sequentially from (1) bespoke to (2) standardized, (3) systematized, (4) packaged, and (5) commoditized. [62] Moving from one stage to the next will require embracing different technologies and knowledge systems. [62]

The spread of the Internet and development of legal technology and informatics are extending legal services to individuals and small-medium companies.

Corporate legal departments may use legal informatics for such purposes as to manage patent portfolios, [63] and for preparation, customization and management of documents. [64]

See also

Related Research Articles

Knowledge representation and reasoning is the field of artificial intelligence (AI) dedicated to representing information about the world in a form that a computer system can use to solve complex tasks such as diagnosing a medical condition or having a dialog in a natural language. Knowledge representation incorporates findings from psychology about how humans solve problems, and represent knowledge in order to design formalisms that will make complex systems easier to design and build. Knowledge representation and reasoning also incorporates findings from logic to automate various kinds of reasoning.

<span class="mw-page-title-main">Semantic Web</span> Extension of the Web to facilitate data exchange

The Semantic Web, sometimes known as Web 3.0, is an extension of the World Wide Web through standards set by the World Wide Web Consortium (W3C). The goal of the Semantic Web is to make Internet data machine-readable.

<span class="mw-page-title-main">Information science</span> Academic field concerned with collection and analysis of information

Information science is an academic field which is primarily concerned with analysis, collection, classification, manipulation, storage, retrieval, movement, dissemination, and protection of information. Practitioners within and outside the field study the application and the usage of knowledge in organizations in addition to the interaction between people, organizations, and any existing information systems with the aim of creating, replacing, improving, or understanding the information systems.

<span class="mw-page-title-main">Logic in computer science</span> Academic discipline

Logic in computer science covers the overlap between the field of logic and that of computer science. The topic can essentially be divided into three main areas:

<span class="mw-page-title-main">Robert Kowalski</span> British computer scientist (born 1941)

Robert Anthony Kowalski is an American-British logician and computer scientist, whose research is concerned with developing both human-oriented models of computing and computational models of human thinking. He has spent most of his career in the United Kingdom.

In philosophy of logic, defeasible reasoning is a kind of provisional reasoning that is rationally compelling, though not deductively valid. It usually occurs when a rule is given, but there may be specific exceptions to the rule, or subclasses that are subject to a different rule. Defeasibility is found in literatures that are concerned with argument and the process of argument, or heuristic reasoning.

<span class="mw-page-title-main">Deborah McGuinness</span>

Deborah Louise McGuinness is an American computer scientist and researcher at Rensselaer Polytechnic Institute (RPI). She is a professor of Computer, Cognitive and Web Sciences, Industrial and Systems Engineering, and an endowed chair in the Tetherless World Constellation, a multidisciplinary research institution within RPI that focuses on the study of theories, methods and applications of the World Wide Web. Her fields of expertise include interdisciplinary data integration, artificial intelligence, specifically in knowledge representation and reasoning, description logics, the semantic web, explanation, and trust.

Vasant G. Honavar is an Indian-American computer scientist, and artificial intelligence, machine learning, big data, data science, causal inference, knowledge representation, bioinformatics and health informatics researcher and professor.

The following outline is provided as an overview of and topical guide to artificial intelligence:

Norms can be considered from different perspectives in artificial intelligence to create computers and computer software that are capable of intelligent behaviour.

Machine interpretation of documents and services in Semantic Web environment is primarily enabled by (a) the capability to mark documents, document segments and services with semantic tags and (b) the ability to establish contextual relations between the tags with a domain model, which is formally represented as ontology. Human beings use natural languages to communicate an abstract view of the world. Natural language constructs are symbolic representations of human experience and are close to the conceptual model that Semantic Web technologies deal with. Thus, natural language constructs have been naturally used to represent the ontology elements. This makes it convenient to apply Semantic Web technologies in the domain of textual information. In contrast, multimedia documents are perceptual recording of human experience. An attempt to use a conceptual model to interpret the perceptual records gets severely impaired by the semantic gap that exists between the perceptual media features and the conceptual world. Notably, the concepts have their roots in perceptual experience of human beings and the apparent disconnect between the conceptual and the perceptual world is rather artificial. The key to semantic processing of multimedia data lies in harmonizing the seemingly isolated conceptual and the perceptual worlds. Representation of the Domain knowledge needs to be extended to enable perceptual modeling, over and above conceptual modeling that is supported. The perceptual model of a domain primarily comprises observable media properties of the concepts. Such perceptual models are useful for semantic interpretation of media documents, just as the conceptual models help in the semantic interpretation of textual documents.

Knowledge retrieval seeks to return information in a structured form, consistent with human cognitive processes as opposed to simple lists of data items. It draws on a range of fields including epistemology, cognitive psychology, cognitive neuroscience, logic and inference, machine learning and knowledge discovery, linguistics, and information technology.

John L. Pollock (1940–2009) was an American philosopher known for influential work in epistemology, philosophical logic, cognitive science, and artificial intelligence.

Legal information retrieval is the science of information retrieval applied to legal text, including legislation, case law, and scholarly works. Accurate legal information retrieval is important to provide access to the law to laymen and legal professionals. Its importance has increased because of the vast and quickly increasing amount of legal documents available through electronic means. Legal information retrieval is a part of the growing field of legal informatics.

In information technology a reasoning system is a software system that generates conclusions from available knowledge using logical techniques such as deduction and induction. Reasoning systems play an important role in the implementation of artificial intelligence and knowledge-based systems.

Trevor Bench-Capon is a British computer scientist and an Honorary Visiting Professor of computer science at the University of Liverpool, where he taught from 1987 until his retirement in 2012. He is the author of work on computer science and ontology and is one of the editors in chief of the Artificial Intelligence and Law Journal.

A legal expert system is a domain-specific expert system that uses artificial intelligence to emulate the decision-making abilities of a human expert in the field of law. Legal expert systems employ a rule base or knowledge base and an inference engine to accumulate, reference and produce expert knowledge on specific subjects within the legal domain.

Computational Law is the branch of legal informatics concerned with the automation of legal reasoning. What distinguishes Computational Law systems from other instances of legal technology is their autonomy, i.e. the ability to answer legal questions without additional input from human legal experts.

<span class="mw-page-title-main">Legal technology</span> Technology and software to provide legal services

Legal technology, also known as Legal Tech, refers to the use of technology and software to provide legal services and support the legal industry. Legal Tech companies are often startups founded with the purpose of disrupting the traditionally conservative legal market.

Rulelog is an expressive semantic rule-based knowledge representation and reasoning (KRR) language. It underlies knowledge representation languages used in systems such as Flora-2, SILK and others. It extends well-founded declarative logic programs with features for higher-order syntax, frame syntax, defeasibility, general quantified expressions both in the bodies of the rules and their heads, user-defined functions, and restraint bounded rationality.

References

  1. Erdelez, Sanda; O'Hare, Sheila (1997). "Legal Informatics: Application of Information Technology in Law". Annual Review of Information Science and Technology. 32: 367.
  2. Dolin, Ron A. "Search Query Privacy: The Problem of Anonymization". Hastings Science & Technology Law Journal. 2010: 137.
  3. Chris Johnson, Leveraging Technology to Deliver Legal Services, 23 HARV. J.L. & TECH.259, 279 (2009).
  4. David Allen Larson, "Brother, Can You Spare A Dime?" Technology Can Reduce Dispute Resolution Costs When Times Are Tough and Improve Outcomes, 11 Nev. L.J. 523, 550 (2011)
  5. Wyner, A. (2008). "An Ontology in OWL for Legal Case-Based Reasoning". Artificial Intelligence and Law. 16 (4): 361–387. CiteSeerX   10.1.1.64.1896 . doi:10.1007/s10506-008-9070-8. S2CID   14173766.
  6. Rissland, Edwina L.; Ashley, Kevin D.; Loui, R.P. (November 2003). "AI and Law: A fruitful synergy". Artificial Intelligence. 150 (1–2): 1–15. doi: 10.1016/s0004-3702(03)00122-x . ISSN   0004-3702.
  7. Ashley, Kevin D. (1992). "Case-based reasoning and its implications for legal expert systems". Artificial Intelligence and Law. 1 (2–3): 113–208. doi:10.1007/bf00114920. ISSN   0924-8463.
  8. Nguyen, Ha-Thanh; Phi, Manh-Kien; Ngo, Xuan-Bach; Tran, Vu; Nguyen, Le-Minh; Tu, Minh-Phuong (2022-12-27). "Attentive deep neural networks for legal document retrieval". Artificial Intelligence and Law. 32 (1): 57–86. arXiv: 2212.13899 . doi:10.1007/s10506-022-09341-8. ISSN   0924-8463.
  9. Loevinger, Lee. Jurimetrics--The Next Step Forward. Minn. L. Rev. 33 (1948): 455.
  10. 1 2 Allen, Layman E. Symbolic logic: A razor-edged tool for drafting and interpreting legal documents. Yale LJ 66 (1956): 833.
  11. Mehl, L.Automation in the Legal World: From the Machine Processing of Legal Information to the" Law Machine,. Mechanisation of Thought Processes (1958): 757-787.
  12. Buchanan, Bruce G., and Headrick, Thomas E. Some speculation about artificial intelligence and legal reasoning Archived 2019-05-12 at the Wayback Machine . Stanford Law Review (1970): 40-62.
  13. 1 2 McCarty, L. Thorne. Reflections on" Taxman: An Experiment in Artificial Intelligence and Legal Reasoning. Harvard Law Review (1977): 837-893.
  14. 1 2 Stamper, Ronald K. The LEGOL 1 prototype system and language. The Computer Journal 20.2 (1977): 102-108.
  15. Hafner, Carole D., (1981). Representing knowledge in an information retrieval system. in Oddy, R et al. (editors) (1981). Information Retrieval Research. London: Butterworths.
  16. Gardner, Anne The design of a legal analysis program. AAAI-83. 1983.
  17. Rissland, Edwina L. Examples in Legal Reasoning: Legal Hypotheticals. IJCAI. 1983.
  18. 1 2 Sergot, Marek J., et al. The British Nationality Act as a logic program. Communications of the ACM 29.5 (1986): 370–386.
  19. Niblett, Bryan, ed. Computer science and law. CUP Archive, 1980.
  20. e.g. Ciampi, Costantino, and Martino, Antonio. Artificial intelligence and legal information systems. Elsevier Science Inc., 1982.
  21. Walter, Charles. Computer power & legal language: the use of computational linguistics, artificial intelligence, & expert systems in the law. Greenwood Publishing Group Inc., 1988.
  22. List Archived December 17, 2014, at the Wayback Machine of past ICAIL conferences
  23. For a contemporary discussion of a selection of papers from the first thirteen conferences, see Bench-Capon, Trevor, et al. A history of AI and Law in 50 papers: 25 years of the international conference on AI and Law. Artificial Intelligence and Law 20.3 (2012): 215-319.
  24. List of AI and Law journal volumes
  25. List of Jurix conferences
  26. See the list of workshops on the Jurisin 2014 page
  27. Flatt, Amelie; Langner, Arne; Leps, Olof (2022). Model-Driven Development of Akoma Ntoso Application Profiles - A Conceptual Framework for Model-Based Generation of XML Subschemas (1st ed.). Heidelberg: Sprinter Nature. ISBN   978-3-031-14131-7.
  28. 1 2 Šimalčík, Matej (2023). "Rule by Law". In Kironska, Kristina; Turscanyi, Richard Q. (eds.). Contemporary China: a New Superpower?. Routledge. ISBN   978-1-03-239508-1.
  29. Jones, Rupert. "Rise of the lawbots". Counsel. Lexis Nexis. Retrieved 24 April 2019.
  30. Shelly, Jared (28 January 2019). "Here's How Artificial Intelligence Is Poised to Transform Insurance Underwriting for the Better". Risk & Insurance. Retrieved 24 April 2019.
  31. H. Prakken and G.Sartor, Law and logic: A review from an argumentation perspective, Artificial Intelligence. Available on-line June 2015.
  32. T.J.M. Bench-Capon, G.O. Robinson, T.W. Routen, M.J. Sergot, Logic programming for large scale applications in law: a formalisation of supplementary benefit legislation, in: Proceedings of the First International Conference on Artificial Intelligence and Law, ACM Press, New York, 1987, pp. 190–198.
  33. T. Routen, T.J.M. Bench-Capon, Hierarchical formalizations, International Journal of Man-Machine Studies 35 (1991) 69–93.
  34. T.J.M. Bench-Capon, F.P. Coenen, Isomorphism and legal knowledge based systems, Artificial Intelligence and Law 1 (1992) 65–86.
  35. Thomas R. Gruber: The Role of Common Ontology in Achieving Sharable, Reusable Knowledge Bases. Knowledge Representation 1991: 601-602
  36. Valente, A. 1995. Legal Knowledge Engineering; A Modelling Approach, IOS Press, Amsterdam.
  37. Robert W. van Kralingen, Pepijn R. S. Visser, Trevor J. M. Bench-Capon, H. Jaap van den Herik: A principled approach to developing legal knowledge systems. International Journal of Human-Computer Studies. 51(6): 1127-1154 (1999)
  38. Rinke Hoekstra, Joost Breuker, Marcello Di Bello, Alexander Boer: The LKIF Core Ontology of Basic Legal Concepts. Proceedings of the 2nd Workshop on Legal Ontologies and Artificial Intelligence Techniques. 2007: 43-63
  39. A.J. Jones, M.J. Sergot, On the characterisation of law and computer systems: the normative systems perspective, in: J.-J.Ch. Meyer, R. Wieringa (Eds.), Deontic Logic in Computer Science: Normative System Specification, Wiley, 1993, pp. 275–307
  40. H. Herrestad, C. Krogh, Obligations directed from bearers to counterparties, in: Proceedings of the Fifth International Conference on Artificial Intelligence and Law, ACM Press, New York, 1995, pp. 210–218.
  41. M.J. Sergot, A computational theory of normative positions, ACM Trans. Comput. Log. 2 (2001) 581–622.
  42. Marc Esteva, Juan A. Rodríguez-Aguilar, Josep Lluís Arcos, Carles Sierra, Pere Garcia: Institutionalizing Open Multi-Agent Systems. ICMAS 2000: 381–382
  43. Sanjay Modgil, Nir Oren, Noura Faci, Felipe Meneguzzi, Simon Miles and Michael Luck, Monitoring Compliance with E-Contracts and Norms, Artificial Intelligence and Law 23(2) (2015).
  44. R. Hernandez Marin, G. Sartor, Time and norms: a formalisation in the event-calculus [ dead link ], in: Proceedings of the Seventh International Conference on Artificial Intelligence and Law, ACM, New York, 1999, pp. 90–100.
  45. G. Governatori, A. Rotolo, G. Sartor, Temporalised normative positions in defeasible logic, in: Proceedings of the Tenth International Conference on Artificial Intelligence and Law, ACM Press, New York, 2005, pp. 25–34.
  46. Schauer, Frederick. "On the supposed defeasibility of legal rules." Current Legal Problems 51.1 (1998): 223.
  47. Prakken, Henry, and Giovanni Sartor. "The three faces of defeasibility in the law." Ratio Juris 17.1 (2004): 118-139.
  48. R. Loui: Paths to Defeasibility: Reply to Schauer on Hart Archived 2016-06-04 at the Wayback Machine . APA 12:2, 2013
  49. Robert A. Kowalski: The Treatment of Negation in Logic Programs for Representing Legislation. Proceedings of Second International Conference on Artificial Intelligence and Law. 1989: 11–15
  50. Benjamin Johnston, Guido Governatori: Induction of Defeasible Logic Theories in the Legal Domain. Proceedings of the Ninth International Conference on Artificial Intelligence and Law 2003:204–213
  51. Phan Minh Dung: On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games. Artificial Intelligence 77(2): 321–358 (1995)
  52. Theodore W. Ruger, Pauline T. Kim, Andrew D. Martin, & Kevin M. Quinn, "The Supreme Court Forecasting Project: Legal and Political Science Approaches to Predicting Supreme Court Decisionmaking" Columbia Law Review Volume 104 May 2004
  53. McShane, Blakeley B. (2012). "Predicting Securities Fraud Settlements and Amounts: A Hierarchical Bayesian Model of Federal Securities Class Action Lawsuits". Journal of Empirical Legal Studies. 9 (3): 482–510. doi:10.1111/j.1740-1461.2012.01260.x. S2CID   28041529.
  54. Alexander, Charlotte; Al Jadda, Khalifeh; Feizollahi, Mhomamed J.; Tucker, Anne M. (5 September 2018). "Using Text Analytics to Predict Litigation Outcomes". Georgia State University College of Law. SSRN   3230224.{{cite journal}}: Cite journal requires |journal= (help)
  55. Tromans, Richard (13 February 2018). "Litigation Prediction Battle Heats Up; BLM Links with LSE". Artificial Lawyer. Retrieved 19 April 2019.
  56. Tashea, Jason (September 2018). "Good Data, Bad Data: When it comes to predicting outcomes in litigation, algorithms are only as good as their underlying information". ABA Journal. Retrieved 19 April 2019.
  57. Steffek, Felix; Bell, Ludwig (24 May 2018). "Law and Autonomous Systems Series: Paving the Way for Legal Artificial Intelligence – A Common Dataset for Case Outcome Predictions". Oxford University. Retrieved 1 May 2019.
  58. "NC Lawyer by NC Bar Association - NC Bar Association's NC Lawyer". ncbar.org. Retrieved 26 December 2016.[ permanent dead link ]
  59. Darryl Mountain, XML E-Contracts: Documents that Describe Themselves, 11(3) INT'L J.L. & TECH. 274 (2003)
  60. Hunziker, R (Feb 2017). "New Invention Disclosure Standard – Why Would I Care?" (PDF). IPO Law Journal. Intellectual Property Owners Association.
  61. 1 2 Donahue L, A Primer on Using Artificial Intelligence in the Legal Profession https://jolt.law.harvard.edu/digest/a-primer-on-using-artificial-intelligence-in-the-legal-profession.
  62. 1 2 3 Richard Susskind, From Bespoke to Commodity, LEGAL TECH. J., 2006, at 4, 4–7. "Legal Technology Journal". Archived from the original on 2011-01-29. Retrieved 2011-01-27.
  63. Boninoa, Dario; Ciaramella, Alberto; Corno, Fulvio (March 2010). "Review of the state-of-the-art in patent information and forthcoming evolutions in intelligent patent informatics". World Patent Information. 32 (1): 30–38. doi:10.1016/j.wpi.2009.05.008.
  64. Snyder, Paul (1 April 1986). "Stop Playing Hide and Seek With Your Documents". ABA Journal. 72 (3): 54–56. JSTOR   20758698.