Recently I’ve been evaluating the ability of LLMs to perform simple reasoning, using the Miss Manners benchmark. This article ranks the LLMs on this benchmark and summarises the results.
Continue reading “Miss Manners LLM Benchmark”
I repeated my experiment with the Miss Manners benchmark using Google Gemini. The results were inferior to ChatGPT.
Continue reading “Miss Manners with Gemini”
Introduction
The conceptual leap from managing contract text and data, to understanding the real-time rights and obligations of the various parties to a contract is a major one! To get even a partial view of the rights and obligations of contractual parties requires creating a computable representation of the logic and workflow inherent/implicit in the contact, as well as tapping into a digital representation of real-world contract events.
Continue reading “Contract Digitisation Literature Review”
The maturity model for contracts and document generation looks like this:
- Copy/paste a MS Word document and use find-and-replace to update the contract details (aka deal-points).
- Manage templates as separate artefacts, and use a software platform to automatically find-and-replace the template variables with their values, with values supplied by contract managers via web-forms or read via API from external systems
- Introduce conditional logic and calculations into the templates: which necessitates a type-system for the variables
- To tame the combinatorial explosion in the number of templates, modularize the templates using a set of included micro-templates, often managed as a clause library, dynamically assembling templates based on incoming data
This is part two of a series of articles on Accord Project TemplateMark. In this article I will show how TemplateMark can be statically compiled to TypeScript. I recommend you read Getting Started with TemplateMark if you have not yet done so.
Continue reading “TemplateMark Compilation”
Anthropic recently released Claude their LLM trained to be “helpful, honest, and harmless”. Much has been written about Anthropic’s laudable approach, including their philosophy of “constitutional AI“. In this post we take a look at how Claude works in practice, and the enormous challenges posed by using natural language as a general purpose interface.
Continue reading “Ethical LLM Whac-A-Mole”
The latest advances in artificial intelligence (particularly large language models) continue to reverberate. Even for an “old school” AI person like myself (who cut his teeth with Prolog) it is clear that there has been a step change in our ability to create computer systems that can interact with humans using natural language. GPT-4 et al are exhibiting early signs of “common sense” and have encoded useful conceptual representations of the world. The debate rages on as to whether this is “intelligence”, but to an engineer like me, it sure seems useful!
Continue reading “The 8 Billion Person Question”
If you work in technology you will be aware that OpenAI ChatGPT has taken off like a 🚀, and the press is filled with people making rash prognostications, from gloomsters and doomsters to hypesters, and everything between.
Continue reading “Does ChatGPT Understand?”
This weekend I started re-reading one of my favourite technical books, “Data Processing” by K.N. Dodd. It’s a slim volume, published in 1969 by The English Universities Press. I picked up my beautifully musty copy in a second-hand bookshop in March 1995. It is a profoundly practical book (at least in 1969!), which educates the reader on the “state-of-the-art” of The Modern Computer, as well as elaborating on business applications:
Continue reading “The Modern Computer”
Innovation That Matters 