wordle game

Looks at writing a wordle-like game in Groovy that incorporates different generations of string similarity technologies including string metrics, phonetic algorithms, deep learning models, and LLMs. Related blog post is here:

https://github.com/paulk-asert/groovy-string-similarity

Transcript

1. Building a wordle-like game with string metrics, phonetics, and semantic similarity using machine learning and deep neural networks Dr Paul King, VP Apache Groovy & Distinguished Engineer, Object Computing Apache Groovy: Repo: Related blog Slides: Twitter/X | Mastodon | Bluesky: https://groovy.apache.org/ https://groovy-lang.org/ https://github.com/paulk-asert/groovy-string-similarity https://groovy.apache.org/blog/groovy-text-similarity https://speakerdeck.com/paulk/wordle-game @ApacheGroovy | @ApacheGroovy@fosstodon.org | @groovy.apache.org
2. Credits
3. History of text similarity algorithms Simple String Metrics: • Edit Distance Algorithms: Levenshtein, Jaro-Winkler distance; useful for identifying typographical errors or minor variations • Character/Token-based Methods: Jaccard similarity and N-gram similarity; shared characters or sequences of characters/words Phonetic Algorithms: • Soundex, Metaphone, and Double Metaphone; aim to find similar-sounding words even with different spellings Statistical and Lexical Methods (Pre-Deep Learning): • TF-IDF (Term Frequency-Inverse Document Frequency): Assigns weights to words based on their frequency, allowing for vector representation and cosine similarity calculation between documents • Lexical Databases: Use resources like WordNet to measure semantic similarity through relationships (synonyms, hypernyms) between words Deep Learning Engines (Word Embeddings and Contextualized Embeddings): • Static Word Embeddings: Word2Vec and GloVe represent words as dense vectors in a continuous space, where semantically similar words are located close to each other, allowing the capture of semantic relationships beyond simple string matching • Contextualized Embeddings: ELMo and BERT use word embeddings that vary based on the word's context within a sentence, significantly improving the capture of nuanced meaning Large Language Models (LLMs): • Transformer Architecture: BERT, GPT, and their successors; capture long-range dependencies and complex semantic relationships with unprecedented accuracy • Generative Capabilities and Semantic Understanding: LLMs can generate human-like text, answer complex questions, and perform zero-shot and few-shot learning, demonstrating a deep understanding of context and meaning that far surpasses previous methods
4. History of text similarity algorithms Simple String Metrics: • Edit Distance Algorithms: Levenshtein, Jaro-Winkler distance; useful for identifying typographical errors or minor variations • Character/Token-based Methods: Jaccard similarity and N-gram similarity; shared characters or sequences of characters/words Phonetic Algorithms: • Soundex, Metaphone, and Double Metaphone; aim to find similar-sounding words even with different spellings Statistical and Lexical Methods (Pre-Deep Learning): • TF-IDF (Term Frequency-Inverse Document Frequency): Assigns weights to words based on their frequency, allowing for vector representation and cosine similarity calculation between documents • Lexical Databases: Use resources like WordNet to measure semantic similarity through relationships (synonyms, hypernyms) between words Deep Learning Engines (Word Embeddings and Contextualized Embeddings): • Static Word Embeddings: Word2Vec and GloVe represent words as dense vectors in a continuous space, where semantically similar words are located close to each other, allowing the capture of semantic relationships beyond simple string matching • Contextualized Embeddings: ELMo and BERT use word embeddings that vary based on the word's context within a sentence, significantly improving the capture of nuanced meaning Large Language Models (LLMs): • Transformer Architecture: BERT, GPT, and their successors; capture long-range dependencies and complex semantic relationships with unprecedented accuracy • Generative Capabilities and Semantic Understanding: LLMs can generate human-like text, answer complex questions, and perform zero-shot and few-shot learning, demonstrating a deep understanding of context and meaning that far surpasses previous methods
5. Agenda 1. String metrics • Libraries, common algorithms 2. Phonetic algorithms • Libraries, common algorithms 3. Deep learning • DeepLearning4J, DeepJavaLibrary 4. LLMs • Ollama
6. Agenda 1. String metrics • Libraries, common algorithms 2. Phonetic algorithms • Libraries, common algorithms 3. Deep learning • DeepLearning4J, DeepJavaLibrary 4. LLMs • Ollama A tiny bit of theory A look at available libraries A look at how easy to code on the JVM (in Groovy)
7. The Game Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): aftershock LongestCommonSubsequence 0 Levenshtein Distance: 10, Insert: 0, Delete: 3, Substitute: 7 Jaccard 0% JaroWinkler PREFIX 0% / SUFFIX 0% Phonetic Metaphone=AFTRXK 47% / Soundex=A136 0% Meaning AnglE 45% / Use 21% / ConceptNet 2% / GloVe -4% / fastText 19%
8. What is Groovy? • A flexible and extensible Java-like language for the JVM • Java-like feel and syntax, but with added productivity features • Developed since 2003 because Java (at the time) was * not extensible enough * not succinct enough for scripting * missing cool features from Smalltalk, Python, Ruby, …
9. Concise Code … private static boolean isInteger(String s) { return s.matches("-?\\d+"); } private static int partitionPoint(List<String> list, Predicate<String> predicate) { for (int i = 0; i < list.size(); i++) { if (!predicate.test(list.get(i))) { return i; } } return list.size(); } } // 10 times slower import java.util.Arrays; import java.util.Comparator; import java.util.List; import java.util.Map; import java.util.function.Predicate; import java.util.stream.Collectors; public class SortNumericStrings { private static final Map<String, Integer> WORD_NUMBERS = Map.of( "ZERO", 0, "ONE", 1, "TWO", 2, /* ...add more... */ "TEN", 10 ); public static void main(String[] args) { List<String> nums = Arrays.asList("222", "ZERO", "4", "33", "PI", "TWO"); nums.sort(Comparator.comparing(SortNumericStrings::isInteger)); int pp = partitionPoint(nums, s -> !isInteger(s)); nums.subList(0, pp).sort(Comparator.reverseOrder()); nums.subList(pp, nums.size()).sort(Comparator.comparingInt(Integer::parseInt)); if (!nums.equals(List.of("ZERO", "TWO", "PI", "4", "33", "222"))) { throw new AssertionError("Sort mismatch: " + nums); } List<Number> result = nums.stream().map(s -> { if (isInteger(s)) { return Integer.parseInt(s); } else if ("PI".equals(s)) { return java.lang.Math.PI; } else if ("E".equals(s)) { return java.lang.Math.E; } else if (WORD_NUMBERS.containsKey(s)) { return WORD_NUMBERS.get(s); } else { throw new IllegalArgumentException("Unknown literal: " + s); } }).collect(Collectors.toList()); List<Number> expected = List.of(0, 2, Math.PI, 4, 33, 222); if (!result.equals(expected)) { throw new AssertionError("Transformation mismatch: " + result); } } … var nums = ['222', 'ZERO', '4', '33', 'PI', 'TWO'] assert nums.toSorted() == ['222', '33', '4', 'PI', 'TWO', 'ZERO'] nums.sort(String::isInteger) var pp = nums.partitionPoint{ !it.integer } nums.sort(0..<pp, Comparator.reverseOrder()) nums.sort(pp..-1, String::toInteger) assert nums == ['ZERO', 'TWO', 'PI', '4', '33', '222'] assert nums.collect { switch(it) { case { it.integer } -> it.toInteger() case ['PI', 'E'] -> Math."$it" default -> BigInteger."$it" } } == [0, 2, 3.141592653589793, 4, 33, 222]
10. Extension methods • 2000 extension methods to 150+ JDK classes enhance JDK functionality • 350 new methods added in Groovy 5 Better
11. AST Transformations // imports not shown public class Book { private String $to$string; private int $hash$code; private final List<String> authors; private final String title; private final Date publicationDate; private static final java.util.Comparator this$TitleComparator; private static final java.util.Comparator this$PublicationDateComparator; public Book(List<String> authors, String title, Date publicationDate) { if (authors == null) { this.authors = null; } else { if (authors instanceof Cloneable) { List<String> authorsCopy = (List<String>) ((ArrayList<?>) authors).clone(); this.authors = (List<String>) (authorsCopy instanceof SortedSet ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof SortedMap ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof Set ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof Map ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof List ? DefaultGroovyMethods.asImmutable(authorsCopy) : DefaultGroovyMethods.asImmutable(authorsCopy)); } else { this.authors = (List<String>) (authors instanceof SortedSet ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof SortedMap ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof Set ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof Map ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof List ? DefaultGroovyMethods.asImmutable(authors) : DefaultGroovyMethods.asImmutable(authors)); } } this.title = title; if (publicationDate == null) { this.publicationDate = null; } else { this.publicationDate = (Date) publicationDate.clone(); } } public Book(Map args) { if ( args == null) { args = new HashMap(); } ImmutableASTTransformation.checkPropNames(this, args); if (args.containsKey("authors")) { if ( args.get("authors") == null) { this .authors = null; } else { if (args.get("authors") instanceof Cloneable) { List<String> authorsCopy = (List<String>) ((ArrayList<?>) args.get("authors")).clone(); this.authors = (List<String>) (authorsCopy instanceof SortedSet ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof SortedMap ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof Set ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof Map ? DefaultGroovyMethods.asImmutable(authorsCopy) : authorsCopy instanceof List ? DefaultGroovyMethods.asImmutable(authorsCopy) : DefaultGroovyMethods.asImmutable(authorsCopy)); } else { List<String> authors = (List<String>) args.get("authors"); this.authors = (List<String>) (authors instanceof SortedSet ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof SortedMap ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof Set ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof Map ? DefaultGroovyMethods.asImmutable(authors) : authors instanceof List ? DefaultGroovyMethods.asImmutable(authors) : DefaultGroovyMethods.asImmutable(authors)); } } } else { this .authors = null; } if (args.containsKey("title")) {this .title = (String) args.get("title"); } else { this .title = null;} if (args.containsKey("publicationDate")) { if (args.get("publicationDate") == null) { this.publicationDate = null; } else { this.publicationDate = (Date) ((Date) args.get("publicationDate")).clone(); } } else {this.publicationDate = null; } } … public Book() { this (new HashMap()); } public int compareTo(Book other) { if (this == other) { return 0; } Integer value = 0 value = this .title <=> other .title if ( value != 0) { return value } value = this .publicationDate <=> other .publicationDate if ( value != 0) { return value } return 0 } public static Comparator comparatorByTitle() { return this$TitleComparator; } public static Comparator comparatorByPublicationDate() { return this$PublicationDateComparator; } public String toString() { StringBuilder _result = new StringBuilder(); boolean $toStringFirst= true; _result.append("Book("); if ($toStringFirst) { $toStringFirst = false; } else { _result.append(", "); } _result.append(InvokerHelper.toString(this.getAuthors())); if ($toStringFirst) { $toStringFirst = false; } else { _result.append(", "); } _result.append(InvokerHelper.toString(this.getTitle())); if ($toStringFirst) { $toStringFirst = false; } else { _result.append(", "); } _result.append(InvokerHelper.toString(this.getPublicationDate())); _result.append(")"); if ($to$string == null) { $to$string = _result.toString(); } return $to$string; } public int hashCode() { if ( $hash$code == 0) { int _result = HashCodeHelper.initHash(); if (!(this.getAuthors().equals(this))) { _result = HashCodeHelper.updateHash(_result, this.getAuthors()); } if (!(this.getTitle().equals(this))) { _result = HashCodeHelper.updateHash(_result, this.getTitle()); } if (!(this.getPublicationDate().equals(this))) { _result = HashCodeHelper.updateHash(_result, this.getPublicationDate()); } $hash$code = (int) _result; } return $hash$code; } public boolean canEqual(Object other) { return other instanceof Book; } … public boolean equals(Object other) { if ( other == null) { return false; } if (this == other) { return true; } if (!( other instanceof Book)) { return false; } Book otherTyped = (Book) other; if (!(otherTyped.canEqual( this ))) { return false; } if (!(this.getAuthors() == otherTyped.getAuthors())) { return false; } if (!(this.getTitle().equals(otherTyped.getTitle()))) { return false; } if (!(this.getPublicationDate().equals(otherTyped.getPublicationDate()))) { return false; } return true; } public final Book copyWith(Map map) { if (map == null || map.size() == 0) { return this; } Boolean dirty = false; HashMap construct = new HashMap(); if (map.containsKey("authors")) { Object newValue = map.get("authors"); Object oldValue = this.getAuthors(); if (newValue != oldValue) { oldValue = newValue; dirty = true; } construct.put("authors", oldValue); } else { construct.put("authors", this.getAuthors()); } if (map.containsKey("title")) { Object newValue = map.get("title"); Object oldValue = this.getTitle(); if (newValue != oldValue) { oldValue = newValue; dirty = true; } construct.put("title", oldValue); } else { construct.put("title", this.getTitle()); } if (map.containsKey("publicationDate")) { Object newValue = map.get("publicationDate"); Object oldValue = this.getPublicationDate(); if (newValue != oldValue) { oldValue = newValue; dirty = true; } construct.put("publicationDate", oldValue); } else { construct.put("publicationDate", this.getPublicationDate()); } return dirty == true ? new Book(construct) : this; } public void writeExternal(ObjectOutput out) throws IOException { out.writeObject(authors); out.writeObject(title); out.writeObject(publicationDate); } public void readExternal(ObjectInput oin) throws IOException, ClassNotFoundException { authors = (List) oin.readObject(); title = (String) oin.readObject(); publicationDate = (Date) oin.readObject(); } … static { this$TitleComparator = new Book$TitleComparator(); this$PublicationDateComparator = new Book$PublicationDateComparator(); } public String getAuthors(int index) { return authors.get(index); } public List<String> getAuthors() { return authors; } public final String getTitle() { return title; } public final Date getPublicationDate() { if (publicationDate == null) { return publicationDate; } else { return (Date) publicationDate.clone(); } } public int compare(java.lang.Object param0, java.lang.Object param1) { return -1; } private static class Book$TitleComparator extends AbstractComparator<Book> { public Book$TitleComparator() { } public int compare(Book arg0, Book arg1) { if (arg0 == arg1) { return 0; } if (arg0 != null && arg1 == null) { return -1; } if (arg0 == null && arg1 != null) { return 1; } return arg0.title <=> arg1.title; } public int compare(java.lang.Object param0, java.lang.Object param1) { return -1; } } private static class Book$PublicationDateComparator extends AbstractComparator<Book> { public Book$PublicationDateComparator() { } public int compare(Book arg0, Book arg1) { if ( arg0 == arg1 ) { return 0; } if ( arg0 != null && arg1 == null) { return -1; } if ( arg0 == null && arg1 != null) { return 1; } return arg0 .publicationDate <=> arg1 .publicationDate; } public int compare(java.lang.Object param0, java.lang.Object param1) { return -1; } } } @Immutable(copyWith = true) @Sortable(excludes = 'authors') @AutoExternalize class Book { @IndexedProperty List<String> authors String title Date publicationDate } 10 lines of Groovy or 600 lines of Java
12. Domain Specific Languages assert XII + IX == XXI assert [LVII + LVII, V * III, IV..(V+I)] == [ CXIV, XV, IV..VI] assert switch(L) { case L -> '50 exactly' case XLV..<L -> 'just below 50' case ~/LI{1,3}/ -> 'just above 50' default -> 'not close to 50' } == '50 exactly’ assert [X, IX, IV, V, VI].sort() == [iv, v, vi, ix, x] assert MMMCMXCIX + I == MMMM
13. Nice Repl (groovysh)
14. Agenda 1. String metrics • Libraries o info.debatty:java-string-similarity:2.0.0 o org.apache.commons:commons-text:1.14.0 • Algorithms o Levenshtein o Jaccard o Hamming o LongestCommonSubsequence o JaroWinkler 2. Phonetic algorithms 3. Deep learning 4. LLMs
15. String metrics: Levenshtein • The minimum number of "edits" (inserts, deletes, or substitutions) required to convert from one word to another K I T T E N A G R O U N D S I T T I N G G R O U N D S (2 subs + 1 add) (1 del + 1 add) D = 3 D = 2 • Damerau–Levenshtein distance is a variant that allows transposition of two adjacent letters to count as a single edit
16. String metrics: Levenshtein • The minimum number of "edits" (inserts, deletes, or substitutions) required to convert from one word to another • Sim = (N – Dist) / N K I T T E N A G R O U N D S I T T I N G G R O U N D S (2 subs + 1 add) (1 del + 1 add) D = 3 D = 2 S = (7 – 3) / 7 = 0.57 S = (7 – 2) / 7 = 0.71
17. String metrics: Levenshtein • Damerau–Levenshtein is used by the Groovy compiler 'foo'.toUpper() No signature of method: java.lang.String.toUpper() is applicable for argument types: () values: [] Possible solutions: toUpperCase(), toURI(), toURL(), toURI(), toURL(), toSet() 'foo'.touppercase() No signature of method: java.lang.String.touppercase() is applicable for argument types: () values: [] Possible solutions: toUpperCase(), toUpperCase(java.util.Locale), toLowerCase(), toLowerCase(java.util.Locale)
18. String metrics: Jaccard • Divide the number of common elements (the intersection) by the total number of unique elements in both sets (the union) K I T T E N A G R O U N D S I T T I N G G R O U N D S (inter=ITN, union=ITNKSG) (inter=GROUND, union=GROUNDSA) S = 3/6 = 0.5 S = 6 / 8 = 0.75
19. String metrics: Jaccard • Divide the number of common elements (the intersection) by the total number of unique elements in both sets (the union) K I T T E N A G R O U N D S I T T I N G G R O U N D S (inter=ITN, union=ITNKSG) (inter=GROUND, union=GROUNDSA) S = 3/6 = 0.5 S = 6 / 8 = 0.75 • Can also be k-length letter sequences • Can also be words in a sentence, sentences in a document
20. String metrics: LongestCommonSubsequence • length of longest common subsequence • A subsequence is obtained by deleting zero or more elements without changing the order of the remaining elements K I T T E N A G R O U N D S I T T I N G G R O U N D S D = 4 D = 6
21. String metrics: Jaro-Winkler • Edit distance measure • Gives a “prefix boost” to letters at the start of a word • SHIPPING and SHIPMENT more likely to be related than SHIPMENT and BASEMENT
22. String metrics: our own WordleLike metric • Each correct letter (Green) scores 2 point • Each letter in the wrong spot (Yellow) scores 1 point • Incorrect letters (Gray) scores 0 points • For words of differing size, slide smallest word along largest and select smallest distance (max similarity) with padding on either end scoring 0
23. String metrics: similarity algorithms var simAlgs = [ NormalizedLevenshtein: new NormalizedLevenshtein()::similarity, 'Jaccard (debatty k=1)': new Jaccard(1)::similarity, 'Jaccard (debatty k=2)': new Jaccard(2)::similarity, 'Jaccard (debatty k=3)': new Jaccard()::similarity, 'Jaccard (commons text k=1)': new JaccardSimilarity()::apply, 'JaroWinkler (debatty)': new JaroWinkler()::similarity, 'JaroWinkler (commons text)': new JaroWinklerSimilarity()::apply, RatcliffObershelp: new RatcliffObershelp()::similarity, SorensenDice: new SorensenDice()::similarity, Cosine: new Cosine()::similarity, WordleLike: this::wordleSimilaritySliding ]
24. String metrics: similarity algorithms var pairs = [ ['cat', 'hat'], ['cat', 'kitten'], ['cat', 'dog'], ['bear', 'bare'], ['bear', 'bean'], ['pair', 'pear'], ['there', 'their'], ['sort', 'sought'], ['cow', 'bull'], ['cow', 'cowbell'], ['winners', 'grinners'], ['knows', 'nose'], ['ground', 'aground'], ['grounds', 'aground'], ['peeler', 'repeal'], ['hippo', 'hippopotamus'], ['superstar', 'supersonic'], ['partnership', 'leadership'], ['elton john', 'john elton'], ['elton john', 'nhoj notle'], ['my name is Yoda', 'Yoda my name is'], ['the cat sat on the mat', 'the fox jumped over the dog'], ['poodles are cute', 'dachshunds are delightful'] ]
25. String metrics: similarity algorithms results pairs.each { wordPair -> var results = simAlgs.collectValues { alg -> alg(*wordPair) } // display results ... } there VS their JaroWinkler (commons text) 0.91 ██████████████████ JaroWinkler (debatty) 0.91 ██████████████████ Jaccard (debatty k=1) 0.80 ████████████████ Jaccard (commons text k=1) 0.80 ████████████████ RatcliffObershelp 0.80 ████████████████ WordleLike 0.70 ██████████████ NormalizedLevenshtein 0.60 ████████████ Cosine 0.33 ██████ Jaccard (debatty k=2) 0.33 ██████ SorensenDice 0.33 ██████ Jaccard (debatty k=3) 0.20 ████
26. String metrics: distance algorithms var distAlgs = [ NormalizedLevenshtein: new NormalizedLevenshtein()::distance, 'WeightedLevenshtein (t is near r)': new WeightedLevenshtein({ char c1, char c2 -> c1 == 't' && c2 == 'r' ? 0.5 : 1.0 })::distance, Damerau: new Damerau()::distance, OptimalStringAlignment: new OptimalStringAlignment()::distance, MetricLCS: new MetricLCS()::distance, 'NGram(2)': new NGram(2)::distance, 'NGram(4)': new NGram(4)::distance, QGram: new QGram(2)::distance, CosineDistance: new CosineDistance()::apply, HammingDistance: new HammingDistance()::apply, JaccardDistance: new JaccardDistance()::apply, JaroWinklerDistance: new JaroWinklerDistance()::apply, LevenshteinDetailedDistance: { a, b -> LevenshteinDetailedDistance.defaultInstance.apply(a, b).toString() }, LevenshteinDistance: LevenshteinDistance.defaultInstance::apply, 'LongestCommonSubsequenceDistance (commons text)': new LongestCommonSubsequenceDistance()::apply, 'LongestCommonSubsequence (commons text)': new org.apache.commons.text.similarity.LongestCommonSubsequence()::apply, ]
27. String metrics: similarity algorithms var phrases = [ 'The sky is blue', 'The blue sky', 'The blue cat', 'The sea is blue', 'Blue skies following me', 'My ferrari is red', 'Apples are red', 'I read a book', 'The wind blew', 'Numbers are odd or even', 'Red noses', 'Read knows', 'Hippopotamus', ]
28. String metrics: distance algorithms results • Query: The blue car NormalizedLevenshtein: The blue cat (0.08), The blue sky (0.25), The wind blew (0.62) WeightedLevenshtein (t is near r): The blue cat (0.50), The blue sky (3.00), The wind blew (8.00) Damerau: The blue cat (1.00), The blue sky (3.00), The wind blew (8.00) OptimalStringAlignment: The blue cat (1.00), The blue sky (3.00), The wind blew (8.00) LongestCommonSubsequence (debatty): The blue cat (2.00), The blue sky (6.00), The sky is bl ue (11.00) MetricLCS: The blue cat (0.08), The blue sky (0.25), The wind blew (0.46) NGram(2): The blue cat (0.04), The blue sky (0.21), The wind blew (0.58) NGram(4): The blue cat (0.02), The blue sky (0.13), The wind blew (0.50) QGram: The blue cat (2.00), The blue sky (6.00), The sky is blue (11.00) CosineDistance: The blue sky (0.33), The blue cat (0.33), The sky is blue (0.42) HammingDistance: The blue cat (1), The blue sky (3), Hippopotamus (12) JaccardDistance: The blue cat (0.18), The sea is blue (0.33), The blue sky (0.46) JaroWinklerDistance: The blue cat (0.03), The blue sky (0.10), The wind blew (0.32) LevenshteinDistance: The blue cat (1), The blue sky (3), The wind blew (8) LongestCommonSubsequenceDistance (commons text): The blue cat (2), The blue sky (6), The s ky is blue (11)
29. String metrics: distance algorithms results • Query: Red roses NormalizedLevenshtein: Red noses (0.11), Read knows (0.50), Apples are red (0.71) WeightedLevenshtein (t is near r): Red noses (1.00), Read knows (5.00), The blue sky (9.00) Damerau: Red noses (1.00), Read knows (5.00), The blue sky (9.00) OptimalStringAlignment: Red noses (1.00), Read knows (5.00), The blue sky (9.00) MetricLCS: Red noses (0.11), Read knows (0.40), The blue sky (0.67) NGram(2): Red noses (0.11), Read knows (0.55), Apples are red (0.75) NGram(4): Red noses (0.11), Read knows (0.53), Apples are red (0.82) QGram: Red noses (4.00), Read knows (13.00), Apples are red (15.00) CosineDistance: Red noses (0.50), The sky is blue (1.00), The blue sky (1.00) HammingDistance: Red noses (1), The sky is blue (-), The blue sky (-) JaccardDistance: Red noses (0.25), Read knows (0.45), Apples are red (0.55) JaroWinklerDistance: Red noses (0.04), Read knows (0.20), The sea is blue (0.37) LevenshteinDistance: Red noses (1), Read knows (5), The blue sky (9) LongestCommonSubsequenceDistance (commons text): Red noses (2), Read knows (7), The blue sk y (13)
30. String metrics: similarity algorithms discussion For Jaccard = 1, we know that we have correctly guessed all the letters! For Jaccard = 0, we can rule out all letters from our guess! bear VS bare Jaccard (debatty k=1) 1.00 ████████████████████ cow VS bull Jaccard (debatty k=1) 0.00 ▏
31. String metrics: similarity algorithms discussion Consider comparing a social media account @elton_john with email john.elton@gmail.com elton john VS john elton Jaccard (debatty k=1) 1.00 ████████████████████ Jaccard (debatty k=2) 0.80 ████████████████ Jaccard (debatty k=3) 0.45 █████████ elton john VS nhoj notle Jaccard (debatty k=1) 1.00 ████████████████████ Jaccard (debatty k=2) 0.00 ▏ Jaccard (debatty k=3) 0.00 ▏
32. String metrics: similarity algorithms discussion superstar VS supersonic Jaccard (debatty k=1) 0.45 █████████ JaroWinkler (debatty) 0.90 ██████████████████ partnership VS leadership Jaccard (debatty k=1) 0.64 ████████████ JaroWinkler (debatty) 0.73 ██████████████
33. String metrics: similarity algorithms discussion superstar VS supersonic Jaccard (debatty k=1) 0.45 █████████ JaroWinkler (debatty) 0.90 ██████████████████ partnership VS leadership Jaccard (debatty k=1) 0.64 ████████████ JaroWinkler (debatty) 0.73 ██████████████ Rev. JaroWinkler (debatty) 0.90 ██████████████████
34. String metrics: distance algorithms discussion • For bat vs tab, the Jaccard similarity of 100% tells us that the guess contains exactly all the (distinct) letters of the hidden word. The LCS value of 1 tells us that no two letters appear in the correct order. The only way this is possible is if the letters appear in reverse order. • For back vs buck, the LCS of 3 combined with the Jaccard of 3/5 tells us that the three shared letters also appear in the same order in both the guess and hidden words. bat vs tab: LongestCommonSubsequence (1), Jaccard (3/3) back vs buck: LongestCommonSubsequence (3) Jaccard (3/5)
35. String metrics: decisions Settle on the following string metrics • LongestCommonSubsequence and Levenshtein distances • Jaccard, JaroWinkler (fwd/rev) similarity measures Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): aftershock LongestCommonSubsequence 0 Levenshtein Distance: 10, Insert: 0, Delete: 3, Substitute: 7 Jaccard 0% JaroWinkler PREFIX 0% / SUFFIX 0% Phonetic Metaphone=AFTRXK 47% / Soundex=A136 0% Meaning AnglE 45% / Use 21% / ConceptNet 2% / GloVe -4% / fastText 19%
36. Agenda 1. String metrics 2. Phonetic algorithms • Libraries o org.openrefine:main:3.9.3 o commons-codec:commons-codec:1.19.0 • Algorithms o Soundex and variants o Metaphone and variants o Miscellaneous 3. Deep learning 4. LLMs
37. Phonetic algorithms: Soundex The correct value can be found as follows: • Retain the first letter of the name and drop all other occurrences of a, e, i, o, u, y, h, w. • Replace consonants with digits as follows (after the first letter): ▪ b, f, p, v → 1 ▪ c, g, j, k, q, s, x, z → 2 ▪ d, t → 3 ▪ l → 4 ▪ m, n → 5 ▪ r → 6 • If two or more letters with the same number are adjacent in the original name (before step 1), only retain the first letter; also two letters with the same number separated by 'h', 'w' or 'y' are coded as a single number, whereas such letters separated by a vowel are coded twice. This rule also applies to the first letter. • If there are too few letters in the word to assign three numbers, append zeros until there are three numbers. If there are four or more numbers, retain only the first three. Source: https://en.wikipedia.org/wiki/Soundex
38. Phonetic algorithms: soundex var result = { a, b -> ["$a|$b", a == b] } var soundex = [ Soundex: { a, b -> new Soundex().with{result(soundex(a), soundex(b)) }}, RefinedSoundex: { a, b -> new RefinedSoundex().with{ result(encode(a), encode(b)) }}, DaitchMokotoffSoundex: { a, b -> new DaitchMokotoffSoundex().with{ result(encode(a), encode(b)) }}, ] def runAndDisplay(algs) { var results = [pairs, algs].combinations().collect { pair, namedAlg -> namedAlg.value(pair) } display(algs, pairs, results) println() } runAndDisplay(soundex)
39. Phonetic algorithms: soundex Pair Soundex RefinedSoundex DaitchMokotoffSoundex cat|hat C300|H300 C306|H06 430000|530000 bear|bare B600|B600 B109|B1090 790000|790000 pair|pare P600|P600 P109|P1090 790000|790000 there|their T600|T600 T6090|T609 390000|390000 sort|sought S630|S230 S3096|S30406 493000|453000 cow|bull C000|B400 C30|B107 470000|780000 winning|grinning W552|G655 W08084|G4908084 766500|596650 knows|nose K520|N200 K3803|N8030 567400|640000 ground|aground G653|A265 G49086|A049086 596300|059630 peeler|repeal P460|R140 P10709|R90107 789000|978000 hippo|hippopotamus H100|H113 H010|H0101060803 570000|577364
40. Phonetic algorithms: metaphone Pair Metaphone Metaphone(8) DblMetaphone(8) Metaphone3 cat|hat KT|HT KT|HT KT|HT KT|HT bear|bare BR|BR BR|BR PR|PR PR|PR pair|pare PR|PR PR|PR PR|PR PR|PR there|their 0R|0R 0R|0R 0R|0R 0R|0R sort|sought SRT|ST SRT|ST SRT|SKT SRT|ST cow|bull K|BL K|BL K|PL K|PL winning|grinning WNNK|KRNN WNNK|KRNNK ANNK|KRNNK ANNK|KRNNK knows|nose NS|NS NS|NS NS|NS NS|NS ground|aground KRNT|AKRN KRNT|AKRNT KRNT|AKRNT KRNT|AKRNT peeler|repeal PLR|RPL PLR|RPL PLR|RPL PLR|RPL hippo|hippopotamus HP|HPPT HP|HPPTMS HP|HPPTMS HP|HPPTMS
41. Phonetic algorithms: other Pair Nysiis Caverphone2 cat|hat CAT|HAT KT11111111|AT11111111 bear|bare BAR|BAR PA11111111|PA11111111 pair|pare PAR|PAR PA11111111|PA11111111 there|their TAR|TAR TA11111111|TA11111111 sort|sought SAD|SAGT ST11111111|ST11111111 cow|bull C|BAL KA11111111|PA11111111 winning|grinning WANANG|GRANAN WNNK111111|KRNNK11111 knows|nose N|NAS KNS1111111|NS11111111 ground|aground GRAD|AGRAD KRNT111111|AKRNT11111 peeler|repeal PALAR|RAPAL PLA1111111|RPA1111111 hippo|hippopotamus HAP|HAPAPA APA1111111|APPTMS1111
42. Phonetic algorithms: game metrics Pair SoundexDiff Metaphone5LCS Metaphone5Lev cat|hat 75% 50% 50% bear|bare 100% 100% 100% pair|pare 100% 100% 100% there|their 100% 100% 100% sort|sought 75% 67% 67% cow|bull 50% 0% 0% winning|grinning 25% 60% 60% knows|nose 25% 100% 100% ground|aground 0% 80% 80% peeler|repeal 25% 67% 33% hippo|hippopotamus 50% 40% 40%
43. Phonetics: decisions Settle on the following phonetic measures • Soundex SoundexDiff • Metaphone(5) (Metaphone5LCS + Metaphone5Lev) / 2 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): aftershock LongestCommonSubsequence 0 Levenshtein Distance: 10, Insert: 0, Delete: 3, Substitute: 7 Jaccard 0% JaroWinkler PREFIX 0% / SUFFIX 0% Phonetic Metaphone=AFTRXK 47% / Soundex=A136 0% Meaning AnglE 45% / Use 21% / ConceptNet 2% / GloVe -4% / fastText 19%
44. Agenda 1. String metrics 2. Phonetic algorithms 3. Deep learning • DeepLearning4J: Word2Vec (fasttext, GloVe, ConceptNet) • DeepJavaLibrary: Pytorch (AnglE), Tensorflow (Use) 4. LLMs
45. DeepLearning4J Architecture https://medium.com/@abhinav.neo/deeplearning4j-comprehensive-overview-a6ab2d3b3cdc
46. DeepLearning4J: fasttext var modelName = 'fasttext-wiki-news-subwords-300.bin’ var path = Paths.get(ConceptNet.classLoader.getResource(modelName).toURI()).toFile() Word2Vec model = WordVectorSerializer.readWord2VecModel(path) String[] words = ['bull', 'calf', 'bovine', 'cattle', 'livestock', 'horse'] var similarity = words .collectEntries { [it, model.similarity('cow', it)] } .sort { -it.value } .collectValues('%4.2f'::formatted) var nearest = model.wordsNearest('cow', 4) println "fastText similarity to cow: $similarity" println "Nearest words in vocab: $nearest" fastText similarity to cow: [bovine:0.72, cattle:0.70, calf:0.67, bull:0.67, livestock:0.61, horse:0.60] Nearest words in vocab: [cows, goat, pig, bovine]
47. DeepLearning4J: GloVe var modelName = 'glove-wiki-gigaword-300.bin' var path = Paths.get(ConceptNet.classLoader.getResource(modelName).toURI()).toFile() Word2Vec model = WordVectorSerializer.readWord2VecModel(path) String[] words = ['bull', 'calf', 'bovine', 'cattle', 'livestock', 'horse'] var similarity = words .collectEntries { [it, model.similarity('cow', it)] } .sort { -it.value } .collectValues('%4.2f'::formatted) var nearest = model.wordsNearest('cow', 4) println "fastText similarity to cow: $similarity" println "Nearest words in vocab: $nearest" GloVe similarity to cow: [bovine:0.67, cattle:0.62, livestock:0.47, calf:0.44, horse:0.42, bull:0.38] Nearest words in vocab: [cows, mad, bovine, cattle]
48. Dimensionality reduction
49. DeepLearning4J: GloVe
50. DeepLearning4J: ConceptNet var modelName = 'conceptnet-numberbatch-17-06-300.bin' var path = Paths.get(ConceptNet.classLoader.getResource(modelName).toURI()).toFile() Word2Vec model = WordVectorSerializer.readWord2VecModel(path) String[] words = ['bull', 'calf', 'bovine', 'cattle', 'livestock', 'horse'] var similarity = words .collectEntries { ["/c/en/$it", model.similarity('/c/en/cow', "/c/en/$it")] } .sort { -it.value } .collectValues('%4.2f'::formatted) var nearest = model.wordsNearest(‘/c/en/cow', 4) println "fastText similarity to cow: $similarity" println "Nearest words in vocab: $nearest" ConceptNet similarity to /c/en/cow: [/c/en/bovine:0.77, /c/en/cattle:0.77, /c/en/livestock:0.63, /c/en/bull:0.54, /c/en/calf:0.53, /c/en/horse:0.50] Nearest words in vocab: [/c/ast/vaca, /c/be/карова, /c/ur/یاگ, /c/gv/booa]
51. DeepLearning4J: ConceptNet Algorithm conceptnet /c/fr/vache █████████ /c/de/kuh █████████ /c/en/cow /c/en/bovine ███████ /c/fr/bovin ███████ /c/en/bull █████ /c/fr/taureau █████████ /c/en/cow █████ /c/en/bull /c/fr/vache █████ /c/de/kuh █████ /c/fr/bovin █████ /c/de/kuh █████ /c/en/cow █████ /c/en/calf /c/fr/vache █████ /c/en/bovine █████ /c/fr/bovin █████ /c/en/cow █████████ /c/fr/vache █████████ /c/de/kuh /c/fr/bovin ███████ /c/en/bovine ███████ /c/en/calf █████ /c/en/cat ████████ /c/de/katze ████████ /c/en/kitten /c/en/bull ██ /c/en/cow █ /c/de/kuh █ /c/de/katze █████████ /c/en/kitten ████████ /c/en/cat /c/en/bull ██ /c/en/cow ██ /c/fr/taureau █ /c/en/cat █████████ /c/en/kitten ████████ /c/de/katze /c/en/bull ██ /c/de/kuh ██ /c/fr/taureau ██ /c/fr/bovin █████████ /c/en/cow ███████ /c/en/bovine /c/de/kuh ███████ /c/fr/vache ███████ /c/en/calf █████ /c/en/bovine █████████ /c/fr/vache ███████ /c/fr/bovin /c/de/kuh ███████ /c/en/cow ███████ /c/fr/taureau █████ /c/en/cow █████████ /c/de/kuh █████████ /c/fr/vache /c/fr/bovin ███████ /c/en/bovine ███████ /c/fr/taureau █████ /c/en/bull █████████ /c/fr/bovin █████ /c/fr/taureau /c/fr/vache █████ /c/en/cow █████ /c/de/kuh █████
52. DeepJavaLibrary • Engine-Agnostic Architecture • MXNet, PyTorch, TensorFlow, ONNX RuntimeXGBoost, LightGBM • Core API & Modules • Loading and managing deep learning engines • Working with n-dimensional arrays and memory • Translating between Java objects and NDArrays • Setting up inference pipelines • Building neural network blocks • Collecting metrics during training or evaluation • Pre-Trained Models (Zoo) • GluonCV, HuggingFace, TorchHub, Keras
53. DeepJavaLibrary: Pytorch (AnglE) var samplePhrases = [ 'bull', 'bovine', 'kitten', 'hay', 'The sky is blue', 'The sea is blue', 'The grass is green', 'One two three', 'Bulls consume hay', 'Bovines convert grass to milk', 'Dogs play in the grass', 'Bulls trample grass', 'Dachshunds are delightful', 'I like cats and dogs'] var queries = [ 'cow', 'cat', 'dog', 'grass', 'Cows eat grass', 'Poodles are cute', 'The water is turquoise’] var modelName = 'UAE-Large-V1.zip' var path = Paths.get(DjlPytorchAngle.classLoader.getResource(modelName).toURI()) var criteria = Criteria.builder() .setTypes(String, float[]) .optModelPath(path) .optTranslatorFactory(new DeferredTranslatorFactory()) .optProgress(new ProgressBar()) .build()
54. DeepJavaLibrary: Pytorch (AnglE) var model = criteria.loadModel() var predictor = model.newPredictor() var sampleEmbeddings = samplePhrases.collect(predictor::predict) queries.each { query -> println "\n $query" var queryEmbedding = predictor.predict(query) sampleEmbeddings .collect { cosineSimilarity(it, queryEmbedding) } .withIndex() .sort { -it.v1 } .take(5) .each { printf '%s (%4.2f)%n', samplePhrases[it.v2], it.v1 } }
55. DeepJavaLibrary: Pytorch (AnglE) cow bovine (0.72) bull (0.57) Bulls consume hay (0.46) hay (0.45) kitten (0.44) cat kitten (0.75) I like cats and dogs (0.39) bull (0.35) hay (0.31) bovine (0.26) dog kitten (0.54) Dogs play in the grass (0.45) bull (0.39) I like cats and dogs (0.37) hay (0.35) Poodles are cute Dachshunds are delightful (0.54) I like cats and dogs (0.42) Dogs play in the grass (0.27) Bulls consume hay (0.19) bovine (0.16) The water is turquoise The sea is blue (0.56) The grass is green (0.39) The sky is blue (0.38) kitten (0.17) One two three (0.17) grass The grass is green (0.61) Bulls trample grass (0.56) Dogs play in the grass (0.52) hay (0.51) Bulls consume hay (0.47) Cows eat grass Bovines convert grass to milk (0.60) Bulls trample grass (0.58) Dogs play in the grass (0.56) Bulls consume hay (0.53) bovine (0.44)
56. DeepJavaLibrary: Tensorflow (Use) var criteria = Criteria.builder() .optApplication(Application.NLP.TEXT_EMBEDDING) .setTypes(String[], float[][]) .optModelUrls(modelUrl) .optTranslator(new UseTranslator()) .optEngine("TensorFlow") .optProgress(new ProgressBar()) .build()
57. Deep learning: decisions Settle on the following semantic measures • All metrics! Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): aftershock LongestCommonSubsequence 0 Levenshtein Distance: 10, Insert: 0, Delete: 3, Substitute: 7 Jaccard 0% JaroWinkler PREFIX 0% / SUFFIX 0% Phonetic Metaphone=AFTRXK 47% / Soundex=A136 0% Meaning AnglE 45% / Use 21% / ConceptNet 2% / GloVe -4% / fastText 19%
58. Ending the Game Hopefully: Hints (discussed later): If more than 30 turns: Congratulations, you guessed correctly! You seem to be having trouble, here are one or more clues: XXX, YYY, ZZZ Sorry, you took too many turns! The hidden word was XXX
59. Playing the Game: Round 1, Turn 1 Let’s start with aftershock. It’s a long word with distinct letters and common vowels and consonants. • We can actually rule out all the letters A, F, T, E, R, S, H, O, C, and K. The game automatically does this for us if we ever receive a Jaccard score of 0%, or for a Jaccard score of 100%, it keeps those letters and discards all others. We’ll see that the "Possible letters" line changes. • Because we deleted 3 letters, we know that the hidden word has 7 letters. • Even though no letter is correct, the Metaphone score isn’t 0, so we need to be on the lookout for other consonants that transform into the same groups. E.g. Q and G can transform to K, D can transform to T Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): aftershock LongestCommonSubsequence 0 Levenshtein Distance: 10, Insert: 0, Delete: 3, Substitute: 7 Jaccard 0% JaroWinkler PREFIX 0% / SUFFIX 0% Phonetic Metaphone=AFTRXK 47% / Soundex=A136 0% Meaning AnglE 45% / Use 21% / ConceptNet 2% / GloVe -4% / fastText 19%
60. Playing the Game: Round 1, Turn 2 Possible letters: b d g i j l m n p q u v w x y z Guess the hidden word (turn 2): fruit LongestCommonSubsequence 2 Levenshtein Distance: 6, Insert: 2, Delete: 0, Substitute: 4 Jaccard 22% JaroWinkler PREFIX 56% / SUFFIX 45% Phonetic Metaphone=FRT 39% / Soundex=F630 0% Meaning AnglE 64% / Use 41% / ConceptNet 37% / GloVe 31% / fastText 44% In terms of vowels, unless it’s a word like 'rhythm', U and I are our likely candidates. Let’s burn a turn to confirm that hunch. We’ll pick a word containing those two vowels plus a mixture of consonants from aftershock - we don’t want information from other consonants to blur what we might learn about the vowels. • Since LCS is 2, both U and I are in the answer in that order, but there could be duplicates. • Jaccard of 22% is 2 / 9. We know that F, R, and T aren’t in the hidden word, so the 7-letter hidden word has 6 distinct letters, i.e. it has one duplicate. • The semantic meaning scores jumped up just a little, so the hidden word has some relationship to fruit.
61. Playing the Game: Round 1, Turn 2 A common prefix is ing and all those letters are still possible. Some possibilities are jumping, dumping, guiding, bugging, bumping and mugging. But, we also know there is exactly one duplicate letter, so we could try judging, pulling, budding, buzzing, bulging, piquing, pumping, mulling, numbing, and pudding (among others). Since we know there is some semantic relationship with fruit, two of these stand out. Budding is something that a fruit tree would need to do to later produce fruit. Pudding is a kind of food. Possible letters: b d g i j l m n p q u v w x y z Guess the hidden word (turn 2): fruit LongestCommonSubsequence 2 Levenshtein Distance: 6, Insert: 2, Delete: 0, Substitute: 4 Jaccard 22% JaroWinkler PREFIX 56% / SUFFIX 45% Phonetic Metaphone=FRT 39% / Soundex=F630 0% Meaning AnglE 64% / Use 41% / ConceptNet 37% / GloVe 31% / fastText 44%
62. Playing the Game: Round 1, Turn 3 Possible letters: b d g i j l m n p q u v w x y z Guess the hidden word (turn 3): budding LongestCommonSubsequence 6 Levenshtein Distance: 1, Insert: 0, Delete: 0, Substitute: 1 Jaccard 71% JaroWinkler PREFIX 90% / SUFFIX 96% Phonetic Metaphone=BTNK 79% / Soundex=B352 75% Meaning AnglE 52% / Use 35% / ConceptNet 2% / GloVe 4% / fastText 25% It’s a 50/50 guess between budding and pudding. Let’s try the first. • We have 6 letters right in a row and 5 of the 6 distinct letters. Also, Metaphone and Soundex scores are high, and JaroWinkler says the front part of our guess is close and the back half is very close. Our other guess of pudding sounds right.
63. Playing the Game: Round 1, Turn 4 Possible letters: b d g i j l m n p q u v w x y z Guess the hidden word (turn 4): pudding LongestCommonSubsequence 7 Levenshtein Distance: 0, Insert: 0, Delete: 0, Substitute: 0 Jaccard 100% JaroWinkler PREFIX 100% / SUFFIX 100% Phonetic Metaphone=PTNK 100% / Soundex=P352 100% Meaning AnglE 100% / Use 100% / ConceptNet 100% / GloVe 100% / fastText 100% Congratulations, you guessed correctly! • Let’s try it!
64. Playing the Game: Round 2, Turn 1 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): bail LongestCommonSubsequence 1 Levenshtein Distance: 7, Insert: 4, Delete: 0, Substitute: 3 Jaccard 22% JaroWinkler PREFIX 42% / SUFFIX 46% Phonetic Metaphone=BL 38% / Soundex=B400 25% Meaning AnglE 46% / Use 40% / ConceptNet 0% / GloVe 0% / fastText 31% • Since LCS is 1, the letters shared with the hidden word are in the reverse order. • There were 4 inserts and 0 deletes which means the hidden word has 8 letters. • Jaccard of 22% is 2 / 9. Therefore, there are 2 letters in the hidden word that are in bail and five that are not. There are 7 unique letters in the hidden word. It has one duplicate.
65. Playing the Game: Round 2, Turn 2 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 2): leg LongestCommonSubsequence 2 Levenshtein Distance: 6, Insert: 5, Delete: 0, Substitute: 1 Jaccard 25% JaroWinkler PREFIX 47% / SUFFIX 0% Phonetic Metaphone=LK 38% / Soundex=L200 0% Meaning AnglE 50% / Use 18% / ConceptNet 11% / GloVe 13% / fastText 37% • Jaccard of 2 / 8 tells us two of the letters in 'leg' appear in the hidden word. • LCS of 2 tells us that they appear in the same order as in the hidden word. • JaroWinkler has a high prefix score of 47%, but a suffix score of 0%. This suggests that the two correct letters are near the beginning of the word. • Metaphone has picked up some similarity with the encoding LK, suggesting the hidden word has some group of consonants encoded to either an 'L' or 'K'.
66. Playing the Game: Round 2, Turn 3 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 3): languish LongestCommonSubsequence 2 Levenshtein Distance: 8, Insert: 0, Delete: 0, Substitute: 8 Jaccard 15% JaroWinkler PREFIX 50% / SUFFIX 50% Phonetic Metaphone=LNKX 34% / Soundex=L522 0% Meaning AnglE 46% / Use 12% / ConceptNet -11% / GloVe -4% / fastText 25% • Let’s try a word with 'L' and 'G' near the start: • 8 substitutions means none of the letters are in the same spot as 'languish'.
67. Playing the Game: Round 2, Turn 4 • Let’s try a word with 'L' and 'E' near the start, bringing at most two letters from languish: • Jaccard tells us we have 4 distinct letters shared with the hidden word and yet we have a LCS of 5. The duplicate 'E' must be correct and the order of all correct letters must match the hidden word. • Only 4 substitutions means 8-4=4 letters are in the correct position. • JaroWinkler slightly favours the prefix over the suffix suggesting that the incorrect letters are probably closer to the end. • The phonetic metrics have increased. For example, 'languish' encodes to LNKX and scored only 34% whereas election, which encodes to ELKXN, scores 50%. Both metrics strongly suggest the hidden word starts with E. Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 4): election LongestCommonSubsequence 5 Levenshtein Distance: 4, Insert: 0, Delete: 0, Substitute: 4 Jaccard 40% JaroWinkler PREFIX 83% / SUFFIX 75% Phonetic Metaphone=ELKXN 50% / Soundex=E423 75% Meaning AnglE 47% / Use 13% / ConceptNet -5% / GloVe -7% / fastText 26%
68. Playing the Game: Round 2, Turn 5 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 5): elevator LongestCommonSubsequence 8 Levenshtein Distance: 0, Insert: 0, Delete: 0, Substitute: 0 Jaccard 100% JaroWinkler PREFIX 100% / SUFFIX 100% Phonetic Metaphone=ELFTR 100% / Soundex=E413 100% Meaning AnglE 100% / Use 100% / ConceptNet 100% / GloVe 100% / fastText 100% Congratulations, you guessed correctly! • From the LCS of 2 with leg, either 'L','E' is in the hidden word or 'E','G'. Trying 'L','E':
69. Playing the Game: Round 3, Turn 1 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 1): aftershock LongestCommonSubsequence 3 Levenshtein Distance: 8, Insert: 0, Delete: 4, Substitute: 4 Jaccard 50% JaroWinkler PREFIX 61% / SUFFIX 49% Phonetic Metaphone=AFTRXK 33% / Soundex=A136 25% Meaning AnglE 44% / Use 11% / ConceptNet -7% / GloVe 1% / fastText 15% • we deleted 4 letters, so the hidden word has 6 letters • Jaccard of 50% is either 5/10 or 6/12. If the latter, we’d have all the letters, so there can’t be 2 additional letters in the hidden word, so it’s 5/10. That means we need to pick 5 letters from aftershock, duplicate one of them, and we’ll have all the letters • phonetic clues suggest it probably doesn’t start with A
70. Playing the Game: Round 3, Turn 2 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 2): coarse LongestCommonSubsequence 3 Levenshtein Distance: 4, Insert: 0, Delete: 0, Substitute: 4 Jaccard 57% JaroWinkler PREFIX 67% / SUFFIX 67% Phonetic Metaphone=KRS 74% / Soundex=C620 75% Meaning AnglE 51% / Use 12% / ConceptNet 5% / GloVe 23% / fastText 26% • In aftershock, F, H, and K, are probably least common. Let’s pick a 6-letter word from the remaining 7 letters that abides by our LCS clue. We know this can’t be right because we aren’t duplicating a letter yet, but we just want to narrow down the possibilities. • We now have 4 of the 5 distinct letters (we should discard 2) • Phonetics indicates we are close but not very close yet, from the Metaphone value of KRS we should drop one and keep two.
71. Playing the Game: Round 3, Turn 3 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 3): roasts LongestCommonSubsequence 3 Levenshtein Distance: 6, Insert: 0, Delete: 0, Substitute: 6 Jaccard 67% JaroWinkler PREFIX 56% / SUFFIX 56% Phonetic Metaphone=RSTS 61% / Soundex=R232 25% Meaning AnglE 54% / Use 25% / ConceptNet 18% / GloVe 18% / fastText 31% • Let’s assume C and E are wrong and bring in the other common letter, T. We need to find a word that matches the LCS conditions from previous guesses, and we’ll duplicate one letter, S. • Phonetics dropped, so maybe S wasn’t the correct letter to bring in, we want the K (from letter C) and R from the previous guess. • Also, the semantic meaning has bumped up to warm (from cold for previous guesses). Maybe the hidden word is related to roasts.
72. Playing the Game: Round 3, Turn 4 Possible letters: a b c d e f g h i j k l m n o p q r s t u v w x y z Guess the hidden word (turn 4): carrot LongestCommonSubsequence 6 Levenshtein Distance: 0, Insert: 0, Delete: 0, Substitute: 0 Jaccard 100% JaroWinkler PREFIX 100% / SUFFIX 100% Phonetic Metaphone=KRT 100% / Soundex=C630 100% Meaning AnglE 100% / Use 100% / ConceptNet 100% / GloVe 100% / fastText 100% Congratulations, you guessed correctly! • Let’s try to word starting with C, related to roasts.
73. Cheat/hint mode: Part 1 Guess the hidden word (turn 3): hint: what is a 6 letter word a little related to roasts and sounding a little like coarse Hints would have been for Game 3. After round 8: root_vegetable, daucus After round 16: diced, cauliflower, cucumber After round 24: celery, onion, sticks, zucchini
74. Agenda 1. String metrics 2. Phonetic algorithms 3. Deep learning 4. LLMs • Ollama
75. We activated a secondary cheat/hint mode • We used Ollama to have a local LLM engine o Tried Mistal, Llama3, Qwen3 models o The latter proved most useful
76. Cheat/hint mode: Part 2 if (guess.startsWith('hint:')) { println new OllamaAPI().generate( 'qwen3', guess - 'hint:', null).response count++ continue }
77. Cheat/hint mode: Part 2 Guess the hidden word (turn 3): hint: what is a seven letter word, with 6 unique letters, containing no letters from the word aftershock, and slightly related to fruit The seven-letter word you're looking for is likely **"budding"** or **"pudding"**, both of which meet the criteria: 1. **Seven letters**: Both words are exactly seven letters long. 2. **Six unique letters**: - **Budding**: Letters are B, U, D, I, N, G (six unique letters). - **Pudding**: Letters are P, U, D, I, N, G (six unique letters). 3. **No letters from "aftershock"**: - "Budding" and "Pudding" share no letters with "aftershock" (A, F, T, E, R, S, H, O, C, K). 4. **Slightly related to fruit**: - **Budding** refers to the growth of buds on plants, which is directly tied to fruit production. - **Pudding** is a dessert, but it might be considered a stretch for a "slightly related" connection to fruit (e.g., fruit-based puddings). **Final Answer**: The word is likely **"budding"** (more directly related to fruit growth) or **"pudding"** (as a dessert with potential fruit ties). Both are valid based on the criteria. Guess the hidden word (turn 4): pudding
78. Cheat/hint mode: Part 2 Guess the hidden word (turn 4): hint: what is a 6 letter word a little related to roasts and sounding a little like coarse The 6-letter word you're looking for is "course". Relation to roasts: In a meal context, a "course" refers to a sequence of dishes (e.g., appetizer, main course, dessert), which can include roasted items. Sound like "coarse": "Course" and "coarse" are homophones (same pronunciation). This fits the clue perfectly! Guess the hidden word (turn 5): carrot
79. Building a wordle-like game with string metrics, phonetics, and semantic similarity using machine learning and deep neural networks Dr Paul King, VP Apache Groovy & Distinguished Engineer, Object Computing Apache Groovy: Repo: Related blog Slides: Twitter/X | Mastodon | Bluesky: https://groovy.apache.org/ https://groovy-lang.org/ https://github.com/paulk-asert/groovy-string-similarity https://groovy.apache.org/blog/groovy-text-similarity https://speakerdeck.com/paulk/wordle-game @ApacheGroovy | @ApacheGroovy@fosstodon.org | @groovy.apache.org Questions?