giles
5945b51cfd
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 41s
editDist as an O(m*n) row-based Levenshtein DP (naive recursion is exponential and times out under load); fuzzyTerms/fuzzyDocs/fuzzyRankTfIdf expand a term to indexed terms within a max edit distance. 166/166. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
13 lines
1.4 KiB
Plaintext
13 lines
1.4 KiB
Plaintext
;; search fuzzy matching — Haskell source fragment. Depends on index + rank.
|
|
;; Levenshtein edit distance (O(m*n) row-based DP — the naive recursive version is
|
|
;; exponential and far too slow under load) expands a query term to all indexed
|
|
;; terms within a max distance, then unions / ranks their docs.
|
|
;; editDist :: String -> String -> Int
|
|
;; fuzzyTerms :: Int -> String -> Index -> [Term] (sorted)
|
|
;; fuzzyDocs :: Int -> String -> Index -> [DocId] (sorted union)
|
|
;; fuzzyRankTfIdf :: Int -> String -> Index -> [DocId]
|
|
|
|
(define
|
|
search/fuzzy-src
|
|
"edMin3 a b c = min a (min b c)\nedCost x y = if x == y then 0 else 1\nedUpto i n = if i > n then [] else i : edUpto (i + 1) n\nedLast [x] = x\nedLast (x:xs) = edLast xs\nedNrow x [] prev left = []\nedNrow x (y:ys) prev left = let v = edMin3 (head (tail prev) + 1) (left + 1) (head prev + edCost x y) in v : edNrow x ys (tail prev) v\nedRow x ys prev = let f = head prev + 1 in f : edNrow x ys prev f\nedRows [] ys prev = prev\nedRows (x:xs) ys prev = edRows xs ys (edRow x ys prev)\neditDist xs ys = edLast (edRows xs ys (edUpto 0 (length ys)))\nqWithinDist maxd term t = editDist term t <= maxd\nfuzzyTerms maxd term idx = filter (qWithinDist maxd term) (allTerms idx)\nfuzzyDocs maxd term idx = foldl (candStep idx) [] (fuzzyTerms maxd term idx)\nfuzzyRankTfIdf maxd term idx = rankTfIdf (fuzzyTerms maxd term idx) idx\n")
|