Skip to content

Offline, privacy-first grammar checker. Fast, open-source, Rust-powered

chore: new words, tag fixes, add phrasal verbs (#1219)

Andrew Dunbar7101716d0033
diff --git a/harper-core/dictionary.dict b/harper-core/dictionary.dict
index ffed99001..f6060cbd1 100644
--- a/harper-core/dictionary.dict
+++ b/harper-core/dictionary.dict
@@ -1,4 +1,4 @@
-52900
+53000
# Start of original dictionary import
# combined with dialect spelling dictionary import.
@@ -7761,6 +7761,7 @@ PM/14SMDG # prime minister; private message
PMS/1245M
PNG/2M # Papua New Guinea; image format
PO/15
+POS/1M # part-of-speech
POV/1SM # point of view
POW/1MS # prisoner of war
PP/12
@@ -19849,6 +19850,7 @@ curacao/1
curacy/1SM
curare/1M
curate/~14DSMGV
+curation/1MS
curative/~51MS
curator/~1KMS
curatorial/5
@@ -21200,7 +21202,7 @@ dirty/~54DRSTGP
dis/~418M
disable/~45DSGL
disablement/1M
-disambiguate/~4N
+disambiguate/~4NSDG
disappointing/~54Y
disarming/54Y
disassembly/1
@@ -29548,7 +29550,7 @@ inguinal/5
inhabit/~4DG
inhabitable/5U
inhabitant/~15SM
-inhabited/~5U
+inhabited/~5UP
inhalant/51SM
inhalation/~1MS
inhalator/1MS
@@ -33870,7 +33872,7 @@ ml/~
mm/~1
mnemonic/~51MS
mnemonically/j
-mo/~51CKHS
+mo/~51KHS # removed `C` de- prefix should only be for negation; breaks heuristics otherwise
moan/14MDRSZG
moaner/1M
moat/~14MDS
@@ -34093,6 +34095,7 @@ monopolizer/1M
monopoly/~1SM
monorail/~1MS
monorepo/1SM
+monospace/5
monosyllabic/51
monosyllable/1MS
monotheism/1M
@@ -52191,6 +52194,7 @@ ATF/1M # automatic transmission fluid
Akismet/2M
Alexa/2M # voice assistant
Alexandre/2SM
+Alibaba/2M
AliExpress/2M
AlphaFold/2M # Google DeepMind project
Ambien/1SM # insomnia treatment
@@ -52198,6 +52202,7 @@ Anker/SM
Archytas/SM # !! please check and comment !! we also have archytas
Automattic/SM # WordPress company
Automattician/SM
+B2B/5 # business-to-business
BCD/1M # binary coded decimal
BCPL/1M # old programming language
BLM/1M
@@ -52440,7 +52445,8 @@ TSMC/2M # Taiwanese Semiconductor Manufacturing Company
TSR/1MS # terminate and stay resident
TUI/12SM # text user interface
Tandy/2M # surname, given name, old computer company
-TechCrunch/SM
+TechCrunch/M
+Tencent/2M
ThinkPad/2SM # pc brand
TikTok/SM
Todo # FIXME! elsewhere in Harper we refer to TODO and to-do
@@ -52450,7 +52456,7 @@ Trocla/SM
TrueNAS/SM
Tumblr/2M
Typst/SM
-UDP/12SM
+UDP/12SM # internet protocol
ULA/1MS # uncommitted logic array
URI/1SM # uniform resource identifier
VAG/2M # car company
@@ -52535,6 +52541,7 @@ sioyek/SM # !! please check and comment !!
stacktrace/SM1 # dictionaries prefer: stack trace
standup/SM # dictionaries prefer: stand-up
strat/1MS # gamer slang strategy
+tagset/1MS
vid/1MS # video
whitespace/~1SM # dictionaries prefer: white space
wikilink/~1SM # !! please check and comment !! elsewhere we have Wikilink
@@ -52566,10 +52573,10 @@ barn find/1MS
beck and call/1
bell-pull/1MS
bench-warmer/1MS
+blow-up/51MS
blue-blooded/5
blue-collar/5
bona fide/j5
-boot up/4
break out/4
breaks out/4
Bretton Woods/2M
@@ -52577,6 +52584,7 @@ built-in/1MS5
built-up/5
car bomb/1MS
car-bomb/4
+cash-strapped/5
CD-ROM/1MS
centre-left/5!@_
centre-right/5!@_
@@ -52630,6 +52638,7 @@ first-person/5
first time/1MS
first-time/5
flash point/1MS
+forward-thinking/5
four-speed/1MS
front-line/5
front man/1M
@@ -52692,6 +52701,7 @@ jury-rig/4S
jury-rigged/4
jury-rigging/4
kick start/4SDG
+kick-starter/1MS
knock-on effect/1MS
kung fu/1M
labor-intensive/5
@@ -52768,6 +52778,8 @@ OS X/2M # before macOS
out-and-out/5
out of date
Pan Am/1M # old airline
+part of speech/1M
+parts speech/9
Pax Americana/2M
peer review/1MSDG
peer-reviewed/5
@@ -52777,6 +52789,7 @@ petty bourgeois/1M
petty bourgeoisie/1M
pick-me-up/1MS
plan B/M
+play-by-play/51MS
post-truth/14M
pre-COVID/5
pro-democracy/5
@@ -52841,6 +52854,7 @@ sussed out/4
susses out/4
sussing out/4
ta-da
+take-home/51MS
tax-exempt/5
tech-savvy/5
teeny-tiny/5
@@ -52892,6 +52906,7 @@ well-used/5
Wi-Fi/1M
wishy-washy/5
working-class/15MS
+world-class/5
would-be/5
write off/4
write-off/1MS
@@ -52903,7 +52918,13 @@ year-over-year/5
# Remember not to add any affixes since only the verb part at the start undergoes inflection.
back up/4
+black out/4
+blast off/4
+blow up/4
+boot up/4
break up/4
+buy back/4
+buy out/4
carry out/4
check in/4
check out/4
@@ -52944,10 +52965,25 @@ sign out/4
sign up/4
slow down/4
splash down/4
-start out/4
+stand off/4
+stand out/4
start up/4
take down/4
+take off/4
+take out/4
+throw back/4
+tie back/4
+toss up/4
+try out/4
+tune up/4
+turn about/4
+turn around/4
+turn off/4
turn out/4
+walk out/4
+wash out/4
+white out/4
+wipe out/4
work around/4
work out/4
diff --git a/harper-core/tests/text/linters/Computer science.snap.yml b/harper-core/tests/text/linters/Computer science.snap.yml
index bb3c57090..c3d5b47ba 100644
--- a/harper-core/tests/text/linters/Computer science.snap.yml
+++ b/harper-core/tests/text/linters/Computer science.snap.yml
@@ -177,9 +177,10 @@ Lint: Spelling (63 priority)
Message: |
82 | During the 1940s, with the development of new and more powerful computing
83 | machines such as the Atanasoff–Berry computer and ENIAC, the term computer came
- | ^~~~~~~~~ Did you mean `Standoff`?
+ | ^~~~~~~~~ Did you mean to spell `Atanasoff` this way?
Suggest:
- Replace with: “Standoff”
+ - Replace with: “Stand off”
@@ -1019,9 +1020,9 @@ Message: |
394 | Johnson and Frederick P. Brooks Jr., members of the Machine Organization
| ^~ Did you mean to spell `P.` this way?
Suggest:
- - Replace with: “Pt”
- - Replace with: “Pa”
- - Replace with: “Pf”
+ - Replace with: “Pp”
+ - Replace with: “Pg”
+ - Replace with: “Pi”
diff --git a/harper-core/tests/text/linters/Difficult sentences.snap.yml b/harper-core/tests/text/linters/Difficult sentences.snap.yml
index 99d7ba20e..9164a8ea2 100644
--- a/harper-core/tests/text/linters/Difficult sentences.snap.yml
+++ b/harper-core/tests/text/linters/Difficult sentences.snap.yml
@@ -279,7 +279,7 @@ Message: |
443 | With their reputation on the line, they decided to fire their PR team.
| ^~ Did you mean to spell `PR` this way?
Suggest:
- - Replace with: “Pre”
+ - Replace with: “Pro”
- Replace with: “P”
- Replace with: “Pa”
diff --git a/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml b/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml
index d0257eb94..57b515246 100644
--- a/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml
+++ b/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml
@@ -11,18 +11,6 @@ Message: |
-Lint: Spelling (63 priority)
-Message: |
- 8 | In corpus linguistics, part-of-speech tagging (POS tagging or PoS tagging or
- | ^~~ Did you mean to spell `POS` this way?
- 9 | POST), also called grammatical tagging is the process of marking up a word in a
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
Lint: Spelling (63 priority)
Message: |
8 | In corpus linguistics, part-of-speech tagging (POS tagging or PoS tagging or
@@ -35,29 +23,6 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 15 | Once performed by hand, POS tagging is now done in the context of computational
- | ^~~ Did you mean to spell `POS` this way?
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
-Lint: Spelling (63 priority)
-Message: |
- 17 | parts of speech, by a set of descriptive tags. POS-tagging algorithms fall into
- | ^~~ Did you mean to spell `POS` this way?
- 18 | two distinctive groups: rule-based and stochastic. E. Brill's tagger, one of the
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
Lint: Spelling (63 priority)
Message: |
18 | two distinctive groups: rule-based and stochastic. E. Brill's tagger, one of the
@@ -81,18 +46,6 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 18 | two distinctive groups: rule-based and stochastic. E. Brill's tagger, one of the
- 19 | first and most widely used English POS-taggers, employs rule-based algorithms.
- | ^~~ Did you mean to spell `POS` this way?
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
Lint: Readability (127 priority)
Message: |
33 | as the more common plural noun. Grammatical context is one way to determine
@@ -143,18 +96,6 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 49 | tags. For example, NN for singular common nouns, NNS for plural common nouns, NP
- 50 | for singular proper nouns (see the POS tags used in the Brown Corpus). Other
- | ^~~ Did you mean to spell `POS` this way?
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
Lint: Spelling (63 priority)
Message: |
55 | 150 separate parts of speech for English. Work on stochastic methods for tagging
@@ -212,17 +153,6 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 63 | The most popular "tag set" for POS tagging for American English is probably the
- | ^~~ Did you mean to spell `POS` this way?
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
Lint: Spelling (63 priority)
Message: |
63 | The most popular "tag set" for POS tagging for American English is probably the
@@ -235,26 +165,12 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 69 | POS tagging work has been done in a variety of languages, and the set of POS
- | ^~~ Did you mean to spell `POS` this way?
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
-Lint: Spelling (63 priority)
+Lint: WordChoice (63 priority)
Message: |
- 69 | POS tagging work has been done in a variety of languages, and the set of POS
- | ^~~ Did you mean to spell `POS` this way?
- 70 | tags used varies greatly with language. Tags usually are designed to include
+ 73 | cross-language differences. The tag sets for heavily inflected languages such as
+ | ^~~~~~~~ Did you mean the closed compound noun “tagsets”?
Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
+ - Replace with: “tagsets”
@@ -561,18 +477,6 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 186 | "unsupervised" tagging. Unsupervised tagging techniques use an untagged corpus
- 187 | for their training data and produce the tagset by induction. That is, they
- | ^~~~~~ Did you mean to spell `tagset` this way?
-Suggest:
- - Replace with: “tablet”
- - Replace with: “tagged”
- - Replace with: “target”
-
-
-
Lint: Spelling (63 priority)
Message: |
200 | Some current major algorithms for part-of-speech tagging include the Viterbi
@@ -606,18 +510,6 @@ Suggest:
-Lint: Spelling (63 priority)
-Message: |
- 207 | Many machine learning methods have also been applied to the problem of POS
- | ^~~ Did you mean to spell `POS` this way?
- 208 | tagging. Methods such as SVM, maximum entropy classifier, perceptron, and
-Suggest:
- - Replace with: “PBS”
- - Replace with: “PMS”
- - Replace with: “POV”
-
-
-
Lint: Spelling (63 priority)
Message: |
208 | tagging. Methods such as SVM, maximum entropy classifier, perceptron, and
diff --git a/harper-core/tests/text/linters/The Great Gatsby.snap.yml b/harper-core/tests/text/linters/The Great Gatsby.snap.yml
index fa9ce9352..3d17ea0c1 100644
--- a/harper-core/tests/text/linters/The Great Gatsby.snap.yml
+++ b/harper-core/tests/text/linters/The Great Gatsby.snap.yml
@@ -1000,8 +1000,8 @@ Message: |
| ^~ Did you mean to spell `B.` this way?
Suggest:
- Replace with: “Bu”
- - Replace with: “Be”
- Replace with: “Bf”
+ - Replace with: “Bk”
@@ -1274,8 +1274,8 @@ Message: |
| ^~ Did you mean to spell `B.` this way?
Suggest:
- Replace with: “Bu”
- - Replace with: “Be”
- Replace with: “Bf”
+ - Replace with: “Bk”
@@ -2297,9 +2297,9 @@ Message: |
1858 | farther out on the Island came the Cheadles and the O. R. P. Schraeders, and the
| ^~ Did you mean to spell `P.` this way?
Suggest:
- - Replace with: “Pt”
- - Replace with: “Pa”
- - Replace with: “Pf”
+ - Replace with: “Pp”
+ - Replace with: “Pg”
+ - Replace with: “Pi”
@@ -2374,8 +2374,8 @@ Message: |
| ^~ Did you mean to spell `B.` this way?
Suggest:
- Replace with: “Bu”
- - Replace with: “Be”
- Replace with: “Bf”
+ - Replace with: “Bk”
@@ -2591,8 +2591,8 @@ Message: |
| ^~ Did you mean to spell `B.` this way?
Suggest:
- Replace with: “Bu”
- - Replace with: “Be”
- Replace with: “Bf”
+ - Replace with: “Bk”
@@ -2934,9 +2934,9 @@ Message: |
1896 | Fitz-Peters and Mr. P. Jewett, once head of the American Legion, and Miss
| ^~ Did you mean to spell `P.` this way?
Suggest:
- - Replace with: “Pt”
- - Replace with: “Pa”
- - Replace with: “Pf”
+ - Replace with: “Pp”
+ - Replace with: “Pg”
+ - Replace with: “Pi”
@@ -6770,8 +6770,8 @@ Message: |
| ^~ Did you mean to spell `B.` this way?
Suggest:
- Replace with: “Bu”
- - Replace with: “Be”
- Replace with: “Bf”
+ - Replace with: “Bk”
diff --git a/harper-core/tests/text/linters/this and that.snap.yml b/harper-core/tests/text/linters/this and that.snap.yml
index c43b18521..e69de29bb 100644
--- a/harper-core/tests/text/linters/this and that.snap.yml
+++ b/harper-core/tests/text/linters/this and that.snap.yml
@@ -1,9 +0,0 @@
-Lint: Spelling (63 priority)
-Message: |
- 2 | As such, disambiguating them is necessary.
- | ^~~~~~~~~~~~~~ Did you mean `disambiguation`?
-Suggest:
- - Replace with: “disambiguation”
-
-
-
diff --git a/harper-core/tests/text/tagged/Part-of-speech tagging.md b/harper-core/tests/text/tagged/Part-of-speech tagging.md
index 1ba6e5ea3..28757b0aa 100644
--- a/harper-core/tests/text/tagged/Part-of-speech tagging.md
+++ b/harper-core/tests/text/tagged/Part-of-speech tagging.md
@@ -11,7 +11,7 @@
>
#
> In corpus linguistics , part - of - speech tagging ( POS tagging or PoS tagging or
-# NPrSg/V/J/P NSg NSg . NSg/V/J . P . NSg/V NSg/V . ? NSg/V NPrSg/C ? NSg/V NPrSg/C
+# NPrSg/V/J/P NSg NSg . NSg/V/J . P . NSg/V NSg/V . NSg NSg/V NPrSg/C NSg NSg/V NPrSg/C
> POST ) , also called grammatical tagging is the process of marking up a word in a
# NPrSg/V/P . . W? V/J J NSg/V VL D NSg P NSg/V NSg/V/J/P D/P NSg P D/P
> text ( corpus ) as corresponding to a particular part of speech , based on both its
@@ -25,15 +25,15 @@
>
#
> Once performed by hand , POS tagging is now done in the context of computational
-# NSg/C V/J NSg/J/P NSg/V . ? NSg/V VL NPrSg/V/J/C NSg/V/J P D NSg P J
+# NSg/C V/J NSg/J/P NSg/V . NSg NSg/V VL NPrSg/V/J/C NSg/V/J P D NSg P J
> linguistics , using algorithms which associate discrete terms , as well as hidden
# NSg . V NPl I/C NSg/V/J J NPl/V . NSg/R NSg/V/J NSg/R V/J
> parts of speech , by a set of descriptive tags . POS - tagging algorithms fall into
-# NPl/V P NSg/V . P D/P NPrSg/J P NSg/J NPl/V . ? . NSg/V NPl NSg/V P
+# NPl/V P NSg/V . P D/P NPrSg/J P NSg/J NPl/V . NSg . NSg/V NPl NSg/V P
> two distinctive groups : rule - based and stochastic . E. Brill's tagger , one of the
# NSg NSg/J NPl/V . NSg/V . V/J V/C J . ? ? NSg . NSg/I/V/J P D
> first and most widely used English POS - taggers , employs rule - based algorithms .
-# NSg/J V/C NSg/I/J J/R V/J NPrSg/V/J ? . NPl . NPl/V NSg/V . V/J NPl .
+# NSg/J V/C NSg/I/J J/R V/J NPrSg/V/J NSg . NPl . NPl/V NSg/V . V/J NPl .
>
#
> Principle
@@ -95,7 +95,7 @@
> tags . For example , NN for singular common nouns , NNS for plural common nouns , NP
# NPl/V . C/P NSg/V . ? C/P NSg/J NSg/V/J NPl/V . ? C/P NSg/J NSg/V/J NPl/V . NPrSg
> for singular proper nouns ( see the POS tags used in the Brown Corpus ) . Other
-# C/P NSg/J NSg/J NPl/V . NSg/V D ? NPl/V V/J P D NPrSg/J NSg . . NSg/V/J
+# C/P NSg/J NSg/J NPl/V . NSg/V D NSg NPl/V V/J P D NPrSg/J NSg . . NSg/V/J
> tagging systems use a smaller number of tags and ignore fine differences or
# NSg/V NPl NSg/V D/P J NSg/V/J P NPl/V V/C V NSg/V/J NSg/V NPrSg/C
> model them as features somewhat independent from part - of - speech .
@@ -121,7 +121,7 @@
>
#
> The most popular " tag set " for POS tagging for American English is probably the
-# D NSg/I/J NSg/J . NSg/V NPrSg/V/J . C/P ? NSg/V C/P NPrSg/J NPrSg/V/J VL R D
+# D NSg/I/J NSg/J . NSg/V NPrSg/V/J . C/P NSg NSg/V C/P NPrSg/J NPrSg/V/J VL R D
> Penn tag set , developed in the Penn Treebank project . It is largely similar to
# NPr NSg/V NPrSg/V/J . V/J P D NPr ? NSg/V . NPrSg/ISg VL J/R NSg/J P
> the earlier Brown Corpus and LOB Corpus tag sets , though much smaller . In
@@ -133,7 +133,7 @@
>
#
> POS tagging work has been done in a variety of languages , and the set of POS
-# ? NSg/V NSg/V V NSg/V NSg/V/J P D/P NSg P NPl/V . V/C D NPrSg/J P ?
+# NSg NSg/V NSg/V V NSg/V NSg/V/J P D/P NSg P NPl/V . V/C D NPrSg/J P NSg
> tags used varies greatly with language . Tags usually are designed to include
# NPl/V V/J NPl/V J/R P NSg/V . NPl/V J/R V V/J P NSg/V
> overt morphological distinctions , although this leads to inconsistencies such as
@@ -369,7 +369,7 @@
> " unsupervised " tagging . Unsupervised tagging techniques use an untagged corpus
# . V/J . NSg/V . V/J NSg/V NPl NSg/V D/P ? NSg
> for their training data and produce the tagset by induction . That is , they
-# C/P D NSg NSg V/C NSg/V D ? NSg/J/P NSg . N/I/C/D VL . IPl
+# C/P D NSg NSg V/C NSg/V D NSg NSg/J/P NSg . N/I/C/D VL . IPl
> observe patterns in word use , and derive part - of - speech categories themselves .
# NSg/V NPl/V NPrSg/V/J/P NSg/V NSg/V . V/C NSg/V NSg/V/J . P . NSg/V NPl I .
> For example , statistics readily reveal that " the " , " a " , and " an " occur in
@@ -409,7 +409,7 @@
>
#
> Many machine learning methods have also been applied to the problem of POS
-# N/I/J/D NSg/V V NPl/V NSg/VX W? NSg/V V/J P D NSg/J P ?
+# N/I/J/D NSg/V V NPl/V NSg/VX W? NSg/V V/J P D NSg/J P NSg
> tagging . Methods such as SVM , maximum entropy classifier , perceptron , and
# NSg/V . NPl/V NSg/I NSg/R ? . NSg/J NSg NSg . N . V/C
> nearest - neighbor have all been tried , and most can achieve accuracy above
diff --git a/harper-core/tests/text/tagged/this and that.md b/harper-core/tests/text/tagged/this and that.md
index 6083cf415..d94f92c6d 100644
--- a/harper-core/tests/text/tagged/this and that.md
+++ b/harper-core/tests/text/tagged/this and that.md
@@ -1,7 +1,7 @@
> " This " and " that " are common and fulfill multiple purposes in everyday English .
# . I/D . V/C . N/I/C/D . V NSg/V/J V/C V NSg/J NPl/V NPrSg/V/J/P NSg/J NPrSg/V/J .
> As such , disambiguating them is necessary .
-# NSg/R NSg/I . ? N/I VL NSg/J .
+# NSg/R NSg/I . V N/I VL NSg/J .
>
#
> This document contains various sentences that use " this " , " that " , " these " , and