Skip to content

Offline, privacy-first grammar checker. Fast, open-source, Rust-powered

feat(#1897): transposition costs one instead of two (#1899)

Grant Lemonsd56bd72b3c42
diff --git a/harper-core/src/spell/fst_dictionary.rs b/harper-core/src/spell/fst_dictionary.rs
index a558deb4b..8f5eb010e 100644
--- a/harper-core/src/spell/fst_dictionary.rs
+++ b/harper-core/src/spell/fst_dictionary.rs
@@ -23,7 +23,7 @@ pub struct FstDictionary {
}
const EXPECTED_DISTANCE: u8 = 3;
-const TRANSPOSITION_COST_ONE: bool = false;
+const TRANSPOSITION_COST_ONE: bool = true;
lazy_static! {
static ref DICT: Arc<FstDictionary> = Arc::new((*MutableDictionary::curated()).clone().into());
@@ -223,6 +223,26 @@ mod tests {
use super::FstDictionary;
+ #[test]
+ fn damerau_transposition_costs_one() {
+ let lev_automata =
+ levenshtein_automata::LevenshteinAutomatonBuilder::new(1, true).build_dfa("woof");
+ assert_eq!(
+ lev_automata.eval("wofo"),
+ levenshtein_automata::Distance::Exact(1)
+ );
+ }
+
+ #[test]
+ fn damerau_transposition_costs_two() {
+ let lev_automata =
+ levenshtein_automata::LevenshteinAutomatonBuilder::new(1, false).build_dfa("woof");
+ assert_eq!(
+ lev_automata.eval("wofo"),
+ levenshtein_automata::Distance::AtLeast(2)
+ );
+ }
+
#[test]
fn fst_map_contains_all_in_full_dict() {
let dict = FstDictionary::curated();
diff --git a/harper-core/tests/text/linters/Alice's Adventures in Wonderland.snap.yml b/harper-core/tests/text/linters/Alice's Adventures in Wonderland.snap.yml
index d357de83c..a69e4e598 100644
--- a/harper-core/tests/text/linters/Alice's Adventures in Wonderland.snap.yml
+++ b/harper-core/tests/text/linters/Alice's Adventures in Wonderland.snap.yml
@@ -118,8 +118,8 @@ Message: |
69 | curtseying as you’re falling through the air! Do you think you could manage it?)
Suggest:
- Replace with: “curtsy”
+ - Replace with: “courtesy”
- Replace with: “curse”
- - Replace with: “cursed”
diff --git a/harper-core/tests/text/linters/Computer science.snap.yml b/harper-core/tests/text/linters/Computer science.snap.yml
index 33c1b8734..1e1e6fca7 100644
--- a/harper-core/tests/text/linters/Computer science.snap.yml
+++ b/harper-core/tests/text/linters/Computer science.snap.yml
@@ -975,8 +975,8 @@ Message: |
| ^~~ Did you mean to spell `HCI` this way?
Suggest:
- Replace with: “Hi”
+ - Replace with: “Chi”
- Replace with: “CI”
- - Replace with: “MCI”
@@ -987,8 +987,8 @@ Message: |
349 | that focus on the relationship between emotions, social behavior and brain
Suggest:
- Replace with: “Hi”
+ - Replace with: “Chi”
- Replace with: “CI”
- - Replace with: “MCI”
@@ -1210,7 +1210,7 @@ Message: |
Suggest:
- Replace with: “got”
- Replace with: “goo”
- - Replace with: “gore”
+ - Replace with: “goths”
diff --git a/harper-core/tests/text/linters/Difficult sentences.snap.yml b/harper-core/tests/text/linters/Difficult sentences.snap.yml
index 28dd66340..434d88956 100644
--- a/harper-core/tests/text/linters/Difficult sentences.snap.yml
+++ b/harper-core/tests/text/linters/Difficult sentences.snap.yml
@@ -111,8 +111,8 @@ Message: |
249 | The bullet is about five centimetres in.
| ^~~~~~~~~~~ Did you mean to spell `centimetres` this way?
Suggest:
- - Replace with: “centimeter's”
- Replace with: “centimeters”
+ - Replace with: “centimeter's”
- Replace with: “centimeter”
diff --git a/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml b/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml
index 05ee42a58..628e8efcd 100644
--- a/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml
+++ b/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml
@@ -149,7 +149,7 @@ Message: |
Suggest:
- Replace with: “Nissan”
- Replace with: “Nisan”
- - Replace with: “Norman”
+ - Replace with: “Nampa”
@@ -330,9 +330,9 @@ Message: |
141 | Eugene Charniak points out in Statistical techniques for natural language
| ^~~~~~~~ Did you mean to spell `Charniak` this way?
Suggest:
+ - Replace with: “Carina”
- Replace with: “Carnap”
- Replace with: “Chadian”
- - Replace with: “Chadwick”
diff --git a/harper-core/tests/text/linters/Spell.US.snap.yml b/harper-core/tests/text/linters/Spell.US.snap.yml
index 226e97a78..1234c982a 100644
--- a/harper-core/tests/text/linters/Spell.US.snap.yml
+++ b/harper-core/tests/text/linters/Spell.US.snap.yml
@@ -13,9 +13,9 @@ Message: |
10 | - Centre.
| ^~~~~~ Did you mean to spell `Centre` this way?
Suggest:
+ - Replace with: “Center”
- Replace with: “Entire”
- Replace with: “Entry”
- - Replace with: “Genre”
@@ -132,9 +132,11 @@ Suggest:
Lint: Spelling (63 priority)
Message: |
22 | - Theatre.
- | ^~~~~~~ Did you mean `Sheathe`?
+ | ^~~~~~~ Did you mean to spell `Theatre` this way?
Suggest:
- - Replace with: “Sheathe”
+ - Replace with: “Theater”
+ - Replace with: “Heater”
+ - Replace with: “Cheater”
diff --git a/harper-core/tests/text/linters/Spell.snap.yml b/harper-core/tests/text/linters/Spell.snap.yml
index 4007e125c..5f6d3a269 100644
--- a/harper-core/tests/text/linters/Spell.snap.yml
+++ b/harper-core/tests/text/linters/Spell.snap.yml
@@ -80,8 +80,8 @@ Message: |
| ^~~~~~ Did you mean to spell `centre` this way?
Suggest:
- Replace with: “center”
+ - Replace with: “central”
- Replace with: “censure”
- - Replace with: “cent”
@@ -101,9 +101,9 @@ Message: |
13 | At the centre of the theatre I dropped a litre of coke.
| ^~~~~ Did you mean to spell `litre` this way?
Suggest:
- - Replace with: “liter”
- Replace with: “lire”
- Replace with: “lite”
+ - Replace with: “liter”
diff --git a/harper-core/tests/text/linters/The Great Gatsby.snap.yml b/harper-core/tests/text/linters/The Great Gatsby.snap.yml
index d5707b265..d578d65df 100644
--- a/harper-core/tests/text/linters/The Great Gatsby.snap.yml
+++ b/harper-core/tests/text/linters/The Great Gatsby.snap.yml
@@ -181,8 +181,8 @@ Message: |
62 | seemed like the ragged edge of the universe—so I decided to go East and learn
Suggest:
- Replace with: “center”
+ - Replace with: “central”
- Replace with: “censure”
- - Replace with: “cent”
@@ -973,7 +973,7 @@ Message: |
Suggest:
- Replace with: “crepe”
- Replace with: “crape”
- - Replace with: “cope”
+ - Replace with: “crime”
@@ -1308,7 +1308,7 @@ Message: |
1122 | Daisy! Dai———”
| ^~~ Did you mean to spell `Dai` this way?
Suggest:
- - Replace with: “Day”
+ - Replace with: “Dab”
- Replace with: “Dad”
- Replace with: “Dais”
@@ -1570,8 +1570,8 @@ Message: |
1204 | triumph, glide on through the sea-change of faces and voices and color under the
Suggest:
- Replace with: “center”
+ - Replace with: “central”
- Replace with: “censure”
- - Replace with: “cent”
@@ -2252,9 +2252,9 @@ Message: |
1838 | was nephew to Von Hindenburg and second cousin to the devil. Reach me a rose,
| ^~~ Did you mean to spell `Von` this way?
Suggest:
+ - Replace with: “Van”
- Replace with: “Vol”
- Replace with: “Yon”
- - Replace with: “Con”
@@ -2410,6 +2410,7 @@ Message: |
Suggest:
- Replace with: “Schroeder's”
- Replace with: “Schroeder”
+ - Replace with: “Threaders”
@@ -2969,8 +2970,8 @@ Message: |
| ^~~~~~~~ Did you mean to spell `Faustina` this way?
1894 | least once and the Baedeker girls and young Brewer, who had his nose shot off in
Suggest:
- - Replace with: “Faustino”
- Replace with: “Faustian”
+ - Replace with: “Faustino”
@@ -3093,9 +3094,9 @@ Message: |
| ^~~~ Did you mean to spell `Bois` this way?
1982 | de Boulogne.
Suggest:
- - Replace with: “Boss”
+ - Replace with: “Bogs”
- Replace with: “Boris”
- - Replace with: “Bis”
+ - Replace with: “Bios”
@@ -3241,7 +3242,7 @@ Message: |
Suggest:
- Replace with: “somber”
- Replace with: “hombre”
- - Replace with: “some”
+ - Replace with: “sober”
@@ -3412,9 +3413,9 @@ Message: |
2142 | “Sure he went.” Mr. Wolfshiem’s nose flashed at me indignantly. “He turned
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
@@ -3902,7 +3903,7 @@ Message: |
Suggest:
- Replace with: “blared”
- Replace with: “bleated”
- - Replace with: “beard”
+ - Replace with: “blurred”
@@ -4341,12 +4342,10 @@ Suggest:
Lint: Spelling (63 priority)
Message: |
3031 | Jay Gatsby who borrowed a rowboat, pulled out to the Tuolomee, and informed Cody
- | ^~~~~~~~ Did you mean to spell `Tuolomee` this way?
+ | ^~~~~~~~ Did you mean `Toulouse`?
3032 | that a wind might catch him and break him up in half an hour.
Suggest:
- - Replace with: “Togolese”
- - Replace with: “Tolkien”
- - Replace with: “Toltec”
+ - Replace with: “Toulouse”
@@ -4422,12 +4421,10 @@ Suggest:
Lint: Spelling (63 priority)
Message: |
3085 | and a yachting cap. And when the Tuolomee left for the West Indies and the
- | ^~~~~~~~ Did you mean to spell `Tuolomee` this way?
+ | ^~~~~~~~ Did you mean `Toulouse`?
3086 | Barbary Coast Gatsby left too.
Suggest:
- - Replace with: “Togolese”
- - Replace with: “Tolkien”
- - Replace with: “Toltec”
+ - Replace with: “Toulouse”
@@ -5017,8 +5014,8 @@ Message: |
| ^~~~~~ Did you mean to spell `centre` this way?
Suggest:
- Replace with: “center”
+ - Replace with: “central”
- Replace with: “censure”
- - Replace with: “cent”
@@ -5599,9 +5596,9 @@ Message: |
| ^~~ Did you mean to spell `Asa` this way?
4084 | brought him around at the last minute and asked if we had room for him.”
Suggest:
- - Replace with: “Ass”
+ - Replace with: “As”
+ - Replace with: “Asap”
- Replace with: “Ada”
- - Replace with: “Ala”
@@ -6021,9 +6018,9 @@ Message: |
4479 | “Oh, my Ga-od! Oh, my Ga-od! Oh, Ga-od! Oh, my Ga-od!”
| ^~ Did you mean to spell `od` this way?
Suggest:
+ - Replace with: “ob”
- Replace with: “odd”
- Replace with: “ode”
- - Replace with: “of”
@@ -6032,9 +6029,9 @@ Message: |
4479 | “Oh, my Ga-od! Oh, my Ga-od! Oh, Ga-od! Oh, my Ga-od!”
| ^~ Did you mean to spell `od` this way?
Suggest:
+ - Replace with: “ob”
- Replace with: “odd”
- Replace with: “ode”
- - Replace with: “of”
@@ -6043,9 +6040,9 @@ Message: |
4479 | “Oh, my Ga-od! Oh, my Ga-od! Oh, Ga-od! Oh, my Ga-od!”
| ^~ Did you mean to spell `od` this way?
Suggest:
+ - Replace with: “ob”
- Replace with: “odd”
- Replace with: “ode”
- - Replace with: “of”
@@ -6054,9 +6051,9 @@ Message: |
4479 | “Oh, my Ga-od! Oh, my Ga-od! Oh, Ga-od! Oh, my Ga-od!”
| ^~ Did you mean to spell `od` this way?
Suggest:
+ - Replace with: “ob”
- Replace with: “odd”
- Replace with: “ode”
- - Replace with: “of”
@@ -6119,8 +6116,8 @@ Message: |
| ^~~~~~ Did you mean to spell `stopus` this way?
Suggest:
- Replace with: “stop's”
+ - Replace with: “stoups”
- Replace with: “stops”
- - Replace with: “stomp's”
@@ -6300,9 +6297,9 @@ Message: |
4636 | faces of “Wolfshiem’s people,” behind him in the dark shrubbery.
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
@@ -6412,9 +6409,9 @@ Message: |
| ^~~~~ Did you mean to spell `Beale` this way?
4851 | hundred pairs of golden and silver slippers shuffled the shining dust. At the
Suggest:
- - Replace with: “Bale”
- - Replace with: “Beadle”
+ - Replace with: “Belle”
- Replace with: “Berle”
+ - Replace with: “Beadle”
@@ -6859,9 +6856,9 @@ Message: |
5208 | The chauffeur—he was one of Wolfshiem’s protégés—heard the shots—afterward he
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
@@ -6962,9 +6959,9 @@ Message: |
5279 | Meyer Wolfshiem’s name wasn’t in the phone book. The butler gave me his office
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
@@ -7002,9 +6999,9 @@ Message: |
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
5310 | I began to have a feeling of defiance, of scornful solidarity between Gatsby and
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
@@ -7300,9 +7297,9 @@ Message: |
5472 | “Nobody’s in,” she said. “Mr. Wolfshiem’s gone to Chicago.”
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
@@ -7331,21 +7328,19 @@ Message: |
5481 | At this moment a voice, unmistakably Wolfshiem’s, called “Stella!” from the
| ^~~~~~~~~~~ Did you mean to spell `Wolfshiem’s` this way?
Suggest:
+ - Replace with: “Waldheim's”
- Replace with: “Welshmen's”
- Replace with: “Wolfe's”
- - Replace with: “Wolsey's”
Lint: Spelling (63 priority)
Message: |
5500 | She vanished. In a moment Meyer Wolfsheim stood solemnly in the doorway, holding
- | ^~~~~~~~~ Did you mean to spell `Wolfsheim` this way?
+ | ^~~~~~~~~ Did you mean `Florsheim`?
5501 | out both hands. He drew me into his office, remarking in a reverent voice that
Suggest:
- - Replace with: “Waldheim”
- - Replace with: “Wolfe's”
- - Replace with: “Wilhelm”
+ - Replace with: “Florsheim”