I repeated my experiment with the Miss Manners benchmark using Clause Sonnet (Clause 3.5 Pro). The results were the best I’ve seen so far.
Continue reading “Miss Manners with Claude Sonnet”
I repeated my experiment with the Miss Manners benchmark using Clause Opus (Clause Pro). The results were better than GPT4, but inferior to Mistral (Large).
Continue reading “Miss Manners with Claude Opus”
Recently I’ve been evaluating the ability of LLMs to perform simple reasoning, using the Miss Manners benchmark. This article ranks the LLMs on this benchmark and summarises the results.
Continue reading “Miss Manners LLM Benchmark”
I repeated my experiment with the Miss Manners benchmark using Google Gemini. The results were inferior to ChatGPT.
Continue reading “Miss Manners with Gemini”
I repeated my experiment with the Miss Manners benchmark using Mistral.ai (Large). The results were impressive!
Continue reading “Miss Manners with Mistral (Large)”
“Miss Manners” is organizing a dinner party and needs to devise a seating arrangement for her guests. She has a large circular table and will be inviting 16 guests: 8 males and 8 females. Miss Manners is an aging lady of a bygone era and isn’t aware that gender is not binary. She would like to ensure that guests are not seated next to someone of the same gender, and that guests seated next to each other share at least one hobby.
In this article I will examine how ChatGTP fares with this venerable optimisation (or product rules) benchmark and present conclusions.
Continue reading “Miss Manners with ChatGPT”
Innovation That Matters 