Offline, privacy-first grammar checker. Fast, open-source, Rust-powered
chore: new words, tag fixes, add phrasal verbs (#1219)
Andrew Dunbar7101716d0033
diff --git a/harper-core/dictionary.dict b/harper-core/dictionary.dictindex ffed99001..f6060cbd1 100644--- a/harper-core/dictionary.dict+++ b/harper-core/dictionary.dict@@ -1,4 +1,4 @@-52900+53000# Start of original dictionary import# combined with dialect spelling dictionary import.@@ -7761,6 +7761,7 @@ PM/14SMDG # prime minister; private messagePMS/1245MPNG/2M # Papua New Guinea; image formatPO/15+POS/1M # part-of-speechPOV/1SM # point of viewPOW/1MS # prisoner of warPP/12@@ -19849,6 +19850,7 @@ curacao/1curacy/1SMcurare/1Mcurate/~14DSMGV+curation/1MScurative/~51MScurator/~1KMScuratorial/5@@ -21200,7 +21202,7 @@ dirty/~54DRSTGPdis/~418Mdisable/~45DSGLdisablement/1M-disambiguate/~4N+disambiguate/~4NSDGdisappointing/~54Ydisarming/54Ydisassembly/1@@ -29548,7 +29550,7 @@ inguinal/5inhabit/~4DGinhabitable/5Uinhabitant/~15SM-inhabited/~5U+inhabited/~5UPinhalant/51SMinhalation/~1MSinhalator/1MS@@ -33870,7 +33872,7 @@ ml/~mm/~1mnemonic/~51MSmnemonically/j-mo/~51CKHS+mo/~51KHS # removed `C` de- prefix should only be for negation; breaks heuristics otherwisemoan/14MDRSZGmoaner/1Mmoat/~14MDS@@ -34093,6 +34095,7 @@ monopolizer/1Mmonopoly/~1SMmonorail/~1MSmonorepo/1SM+monospace/5monosyllabic/51monosyllable/1MSmonotheism/1M@@ -52191,6 +52194,7 @@ ATF/1M # automatic transmission fluidAkismet/2MAlexa/2M # voice assistantAlexandre/2SM+Alibaba/2MAliExpress/2MAlphaFold/2M # Google DeepMind projectAmbien/1SM # insomnia treatment@@ -52198,6 +52202,7 @@ Anker/SMArchytas/SM # !! please check and comment !! we also have archytasAutomattic/SM # WordPress companyAutomattician/SM+B2B/5 # business-to-businessBCD/1M # binary coded decimalBCPL/1M # old programming languageBLM/1M@@ -52440,7 +52445,8 @@ TSMC/2M # Taiwanese Semiconductor Manufacturing CompanyTSR/1MS # terminate and stay residentTUI/12SM # text user interfaceTandy/2M # surname, given name, old computer company-TechCrunch/SM+TechCrunch/M+Tencent/2MThinkPad/2SM # pc brandTikTok/SMTodo # FIXME! elsewhere in Harper we refer to TODO and to-do@@ -52450,7 +52456,7 @@ Trocla/SMTrueNAS/SMTumblr/2MTypst/SM-UDP/12SM+UDP/12SM # internet protocolULA/1MS # uncommitted logic arrayURI/1SM # uniform resource identifierVAG/2M # car company@@ -52535,6 +52541,7 @@ sioyek/SM # !! please check and comment !!stacktrace/SM1 # dictionaries prefer: stack tracestandup/SM # dictionaries prefer: stand-upstrat/1MS # gamer slang strategy+tagset/1MSvid/1MS # videowhitespace/~1SM # dictionaries prefer: white spacewikilink/~1SM # !! please check and comment !! elsewhere we have Wikilink@@ -52566,10 +52573,10 @@ barn find/1MSbeck and call/1bell-pull/1MSbench-warmer/1MS+blow-up/51MSblue-blooded/5blue-collar/5bona fide/j5-boot up/4break out/4breaks out/4Bretton Woods/2M@@ -52577,6 +52584,7 @@ built-in/1MS5built-up/5car bomb/1MScar-bomb/4+cash-strapped/5CD-ROM/1MScentre-left/5!@_centre-right/5!@_@@ -52630,6 +52638,7 @@ first-person/5first time/1MSfirst-time/5flash point/1MS+forward-thinking/5four-speed/1MSfront-line/5front man/1M@@ -52692,6 +52701,7 @@ jury-rig/4Sjury-rigged/4jury-rigging/4kick start/4SDG+kick-starter/1MSknock-on effect/1MSkung fu/1Mlabor-intensive/5@@ -52768,6 +52778,8 @@ OS X/2M # before macOSout-and-out/5out of datePan Am/1M # old airline+part of speech/1M+parts speech/9Pax Americana/2Mpeer review/1MSDGpeer-reviewed/5@@ -52777,6 +52789,7 @@ petty bourgeois/1Mpetty bourgeoisie/1Mpick-me-up/1MSplan B/M+play-by-play/51MSpost-truth/14Mpre-COVID/5pro-democracy/5@@ -52841,6 +52854,7 @@ sussed out/4susses out/4sussing out/4ta-da+take-home/51MStax-exempt/5tech-savvy/5teeny-tiny/5@@ -52892,6 +52906,7 @@ well-used/5Wi-Fi/1Mwishy-washy/5working-class/15MS+world-class/5would-be/5write off/4write-off/1MS@@ -52903,7 +52918,13 @@ year-over-year/5# Remember not to add any affixes since only the verb part at the start undergoes inflection.back up/4+black out/4+blast off/4+blow up/4+boot up/4break up/4+buy back/4+buy out/4carry out/4check in/4check out/4@@ -52944,10 +52965,25 @@ sign out/4sign up/4slow down/4splash down/4-start out/4+stand off/4+stand out/4start up/4take down/4+take off/4+take out/4+throw back/4+tie back/4+toss up/4+try out/4+tune up/4+turn about/4+turn around/4+turn off/4turn out/4+walk out/4+wash out/4+white out/4+wipe out/4work around/4work out/4diff --git a/harper-core/tests/text/linters/Computer science.snap.yml b/harper-core/tests/text/linters/Computer science.snap.ymlindex bb3c57090..c3d5b47ba 100644--- a/harper-core/tests/text/linters/Computer science.snap.yml+++ b/harper-core/tests/text/linters/Computer science.snap.yml@@ -177,9 +177,10 @@ Lint: Spelling (63 priority)Message: |82 | During the 1940s, with the development of new and more powerful computing83 | machines such as the Atanasoff–Berry computer and ENIAC, the term computer came- | ^~~~~~~~~ Did you mean `Standoff`?+ | ^~~~~~~~~ Did you mean to spell `Atanasoff` this way?Suggest:- Replace with: “Standoff”+ - Replace with: “Stand off”@@ -1019,9 +1020,9 @@ Message: |394 | Johnson and Frederick P. Brooks Jr., members of the Machine Organization| ^~ Did you mean to spell `P.` this way?Suggest:- - Replace with: “Pt”- - Replace with: “Pa”- - Replace with: “Pf”+ - Replace with: “Pp”+ - Replace with: “Pg”+ - Replace with: “Pi”diff --git a/harper-core/tests/text/linters/Difficult sentences.snap.yml b/harper-core/tests/text/linters/Difficult sentences.snap.ymlindex 99d7ba20e..9164a8ea2 100644--- a/harper-core/tests/text/linters/Difficult sentences.snap.yml+++ b/harper-core/tests/text/linters/Difficult sentences.snap.yml@@ -279,7 +279,7 @@ Message: |443 | With their reputation on the line, they decided to fire their PR team.| ^~ Did you mean to spell `PR` this way?Suggest:- - Replace with: “Pre”+ - Replace with: “Pro”- Replace with: “P”- Replace with: “Pa”diff --git a/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml b/harper-core/tests/text/linters/Part-of-speech tagging.snap.ymlindex d0257eb94..57b515246 100644--- a/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml+++ b/harper-core/tests/text/linters/Part-of-speech tagging.snap.yml@@ -11,18 +11,6 @@ Message: |-Lint: Spelling (63 priority)-Message: |- 8 | In corpus linguistics, part-of-speech tagging (POS tagging or PoS tagging or- | ^~~ Did you mean to spell `POS` this way?- 9 | POST), also called grammatical tagging is the process of marking up a word in a-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”---Lint: Spelling (63 priority)Message: |8 | In corpus linguistics, part-of-speech tagging (POS tagging or PoS tagging or@@ -35,29 +23,6 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 15 | Once performed by hand, POS tagging is now done in the context of computational- | ^~~ Did you mean to spell `POS` this way?-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”----Lint: Spelling (63 priority)-Message: |- 17 | parts of speech, by a set of descriptive tags. POS-tagging algorithms fall into- | ^~~ Did you mean to spell `POS` this way?- 18 | two distinctive groups: rule-based and stochastic. E. Brill's tagger, one of the-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”---Lint: Spelling (63 priority)Message: |18 | two distinctive groups: rule-based and stochastic. E. Brill's tagger, one of the@@ -81,18 +46,6 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 18 | two distinctive groups: rule-based and stochastic. E. Brill's tagger, one of the- 19 | first and most widely used English POS-taggers, employs rule-based algorithms.- | ^~~ Did you mean to spell `POS` this way?-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”---Lint: Readability (127 priority)Message: |33 | as the more common plural noun. Grammatical context is one way to determine@@ -143,18 +96,6 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 49 | tags. For example, NN for singular common nouns, NNS for plural common nouns, NP- 50 | for singular proper nouns (see the POS tags used in the Brown Corpus). Other- | ^~~ Did you mean to spell `POS` this way?-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”---Lint: Spelling (63 priority)Message: |55 | 150 separate parts of speech for English. Work on stochastic methods for tagging@@ -212,17 +153,6 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 63 | The most popular "tag set" for POS tagging for American English is probably the- | ^~~ Did you mean to spell `POS` this way?-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”---Lint: Spelling (63 priority)Message: |63 | The most popular "tag set" for POS tagging for American English is probably the@@ -235,26 +165,12 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 69 | POS tagging work has been done in a variety of languages, and the set of POS- | ^~~ Did you mean to spell `POS` this way?-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”----Lint: Spelling (63 priority)+Lint: WordChoice (63 priority)Message: |- 69 | POS tagging work has been done in a variety of languages, and the set of POS- | ^~~ Did you mean to spell `POS` this way?- 70 | tags used varies greatly with language. Tags usually are designed to include+ 73 | cross-language differences. The tag sets for heavily inflected languages such as+ | ^~~~~~~~ Did you mean the closed compound noun “tagsets”?Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”+ - Replace with: “tagsets”@@ -561,18 +477,6 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 186 | "unsupervised" tagging. Unsupervised tagging techniques use an untagged corpus- 187 | for their training data and produce the tagset by induction. That is, they- | ^~~~~~ Did you mean to spell `tagset` this way?-Suggest:- - Replace with: “tablet”- - Replace with: “tagged”- - Replace with: “target”---Lint: Spelling (63 priority)Message: |200 | Some current major algorithms for part-of-speech tagging include the Viterbi@@ -606,18 +510,6 @@ Suggest:-Lint: Spelling (63 priority)-Message: |- 207 | Many machine learning methods have also been applied to the problem of POS- | ^~~ Did you mean to spell `POS` this way?- 208 | tagging. Methods such as SVM, maximum entropy classifier, perceptron, and-Suggest:- - Replace with: “PBS”- - Replace with: “PMS”- - Replace with: “POV”---Lint: Spelling (63 priority)Message: |208 | tagging. Methods such as SVM, maximum entropy classifier, perceptron, anddiff --git a/harper-core/tests/text/linters/The Great Gatsby.snap.yml b/harper-core/tests/text/linters/The Great Gatsby.snap.ymlindex fa9ce9352..3d17ea0c1 100644--- a/harper-core/tests/text/linters/The Great Gatsby.snap.yml+++ b/harper-core/tests/text/linters/The Great Gatsby.snap.yml@@ -1000,8 +1000,8 @@ Message: || ^~ Did you mean to spell `B.` this way?Suggest:- Replace with: “Bu”- - Replace with: “Be”- Replace with: “Bf”+ - Replace with: “Bk”@@ -1274,8 +1274,8 @@ Message: || ^~ Did you mean to spell `B.` this way?Suggest:- Replace with: “Bu”- - Replace with: “Be”- Replace with: “Bf”+ - Replace with: “Bk”@@ -2297,9 +2297,9 @@ Message: |1858 | farther out on the Island came the Cheadles and the O. R. P. Schraeders, and the| ^~ Did you mean to spell `P.` this way?Suggest:- - Replace with: “Pt”- - Replace with: “Pa”- - Replace with: “Pf”+ - Replace with: “Pp”+ - Replace with: “Pg”+ - Replace with: “Pi”@@ -2374,8 +2374,8 @@ Message: || ^~ Did you mean to spell `B.` this way?Suggest:- Replace with: “Bu”- - Replace with: “Be”- Replace with: “Bf”+ - Replace with: “Bk”@@ -2591,8 +2591,8 @@ Message: || ^~ Did you mean to spell `B.` this way?Suggest:- Replace with: “Bu”- - Replace with: “Be”- Replace with: “Bf”+ - Replace with: “Bk”@@ -2934,9 +2934,9 @@ Message: |1896 | Fitz-Peters and Mr. P. Jewett, once head of the American Legion, and Miss| ^~ Did you mean to spell `P.` this way?Suggest:- - Replace with: “Pt”- - Replace with: “Pa”- - Replace with: “Pf”+ - Replace with: “Pp”+ - Replace with: “Pg”+ - Replace with: “Pi”@@ -6770,8 +6770,8 @@ Message: || ^~ Did you mean to spell `B.` this way?Suggest:- Replace with: “Bu”- - Replace with: “Be”- Replace with: “Bf”+ - Replace with: “Bk”diff --git a/harper-core/tests/text/linters/this and that.snap.yml b/harper-core/tests/text/linters/this and that.snap.ymlindex c43b18521..e69de29bb 100644--- a/harper-core/tests/text/linters/this and that.snap.yml+++ b/harper-core/tests/text/linters/this and that.snap.yml@@ -1,9 +0,0 @@-Lint: Spelling (63 priority)-Message: |- 2 | As such, disambiguating them is necessary.- | ^~~~~~~~~~~~~~ Did you mean `disambiguation`?-Suggest:- - Replace with: “disambiguation”---diff --git a/harper-core/tests/text/tagged/Part-of-speech tagging.md b/harper-core/tests/text/tagged/Part-of-speech tagging.mdindex 1ba6e5ea3..28757b0aa 100644--- a/harper-core/tests/text/tagged/Part-of-speech tagging.md+++ b/harper-core/tests/text/tagged/Part-of-speech tagging.md@@ -11,7 +11,7 @@>#> In corpus linguistics , part - of - speech tagging ( POS tagging or PoS tagging or-# NPrSg/V/J/P NSg NSg . NSg/V/J . P . NSg/V NSg/V . ? NSg/V NPrSg/C ? NSg/V NPrSg/C+# NPrSg/V/J/P NSg NSg . NSg/V/J . P . NSg/V NSg/V . NSg NSg/V NPrSg/C NSg NSg/V NPrSg/C> POST ) , also called grammatical tagging is the process of marking up a word in a# NPrSg/V/P . . W? V/J J NSg/V VL D NSg P NSg/V NSg/V/J/P D/P NSg P D/P> text ( corpus ) as corresponding to a particular part of speech , based on both its@@ -25,15 +25,15 @@>#> Once performed by hand , POS tagging is now done in the context of computational-# NSg/C V/J NSg/J/P NSg/V . ? NSg/V VL NPrSg/V/J/C NSg/V/J P D NSg P J+# NSg/C V/J NSg/J/P NSg/V . NSg NSg/V VL NPrSg/V/J/C NSg/V/J P D NSg P J> linguistics , using algorithms which associate discrete terms , as well as hidden# NSg . V NPl I/C NSg/V/J J NPl/V . NSg/R NSg/V/J NSg/R V/J> parts of speech , by a set of descriptive tags . POS - tagging algorithms fall into-# NPl/V P NSg/V . P D/P NPrSg/J P NSg/J NPl/V . ? . NSg/V NPl NSg/V P+# NPl/V P NSg/V . P D/P NPrSg/J P NSg/J NPl/V . NSg . NSg/V NPl NSg/V P> two distinctive groups : rule - based and stochastic . E. Brill's tagger , one of the# NSg NSg/J NPl/V . NSg/V . V/J V/C J . ? ? NSg . NSg/I/V/J P D> first and most widely used English POS - taggers , employs rule - based algorithms .-# NSg/J V/C NSg/I/J J/R V/J NPrSg/V/J ? . NPl . NPl/V NSg/V . V/J NPl .+# NSg/J V/C NSg/I/J J/R V/J NPrSg/V/J NSg . NPl . NPl/V NSg/V . V/J NPl .>#> Principle@@ -95,7 +95,7 @@> tags . For example , NN for singular common nouns , NNS for plural common nouns , NP# NPl/V . C/P NSg/V . ? C/P NSg/J NSg/V/J NPl/V . ? C/P NSg/J NSg/V/J NPl/V . NPrSg> for singular proper nouns ( see the POS tags used in the Brown Corpus ) . Other-# C/P NSg/J NSg/J NPl/V . NSg/V D ? NPl/V V/J P D NPrSg/J NSg . . NSg/V/J+# C/P NSg/J NSg/J NPl/V . NSg/V D NSg NPl/V V/J P D NPrSg/J NSg . . NSg/V/J> tagging systems use a smaller number of tags and ignore fine differences or# NSg/V NPl NSg/V D/P J NSg/V/J P NPl/V V/C V NSg/V/J NSg/V NPrSg/C> model them as features somewhat independent from part - of - speech .@@ -121,7 +121,7 @@>#> The most popular " tag set " for POS tagging for American English is probably the-# D NSg/I/J NSg/J . NSg/V NPrSg/V/J . C/P ? NSg/V C/P NPrSg/J NPrSg/V/J VL R D+# D NSg/I/J NSg/J . NSg/V NPrSg/V/J . C/P NSg NSg/V C/P NPrSg/J NPrSg/V/J VL R D> Penn tag set , developed in the Penn Treebank project . It is largely similar to# NPr NSg/V NPrSg/V/J . V/J P D NPr ? NSg/V . NPrSg/ISg VL J/R NSg/J P> the earlier Brown Corpus and LOB Corpus tag sets , though much smaller . In@@ -133,7 +133,7 @@>#> POS tagging work has been done in a variety of languages , and the set of POS-# ? NSg/V NSg/V V NSg/V NSg/V/J P D/P NSg P NPl/V . V/C D NPrSg/J P ?+# NSg NSg/V NSg/V V NSg/V NSg/V/J P D/P NSg P NPl/V . V/C D NPrSg/J P NSg> tags used varies greatly with language . Tags usually are designed to include# NPl/V V/J NPl/V J/R P NSg/V . NPl/V J/R V V/J P NSg/V> overt morphological distinctions , although this leads to inconsistencies such as@@ -369,7 +369,7 @@> " unsupervised " tagging . Unsupervised tagging techniques use an untagged corpus# . V/J . NSg/V . V/J NSg/V NPl NSg/V D/P ? NSg> for their training data and produce the tagset by induction . That is , they-# C/P D NSg NSg V/C NSg/V D ? NSg/J/P NSg . N/I/C/D VL . IPl+# C/P D NSg NSg V/C NSg/V D NSg NSg/J/P NSg . N/I/C/D VL . IPl> observe patterns in word use , and derive part - of - speech categories themselves .# NSg/V NPl/V NPrSg/V/J/P NSg/V NSg/V . V/C NSg/V NSg/V/J . P . NSg/V NPl I .> For example , statistics readily reveal that " the " , " a " , and " an " occur in@@ -409,7 +409,7 @@>#> Many machine learning methods have also been applied to the problem of POS-# N/I/J/D NSg/V V NPl/V NSg/VX W? NSg/V V/J P D NSg/J P ?+# N/I/J/D NSg/V V NPl/V NSg/VX W? NSg/V V/J P D NSg/J P NSg> tagging . Methods such as SVM , maximum entropy classifier , perceptron , and# NSg/V . NPl/V NSg/I NSg/R ? . NSg/J NSg NSg . N . V/C> nearest - neighbor have all been tried , and most can achieve accuracy abovediff --git a/harper-core/tests/text/tagged/this and that.md b/harper-core/tests/text/tagged/this and that.mdindex 6083cf415..d94f92c6d 100644--- a/harper-core/tests/text/tagged/this and that.md+++ b/harper-core/tests/text/tagged/this and that.md@@ -1,7 +1,7 @@> " This " and " that " are common and fulfill multiple purposes in everyday English .# . I/D . V/C . N/I/C/D . V NSg/V/J V/C V NSg/J NPl/V NPrSg/V/J/P NSg/J NPrSg/V/J .> As such , disambiguating them is necessary .-# NSg/R NSg/I . ? N/I VL NSg/J .+# NSg/R NSg/I . V N/I VL NSg/J .>#> This document contains various sentences that use " this " , " that " , " these " , and