# Mass proofreading with LanguageTool
**URL:** https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730
**Category:** The Lab
**Created:** [October 21, 2019, 7:31am UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730 "2019-10-21T07:31:42Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@int\_ua](https://discourse.cataclysmdda.org/u/int_ua)
#### Post date: [October 21, 2019, 7:31am UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730/1 "2019-10-21T07:31:42Z")
I’ve experimented a bit with using [LanguageTool](https://github.com/languagetool-org/languagetool) (FOSS proofreading) CLI version to detect errors in items descriptions. So far I like it, got rid of most false positives by disabling unnecessary rules. Tested on items/tools.json and tool\_armor.json. About a dozen errors just in these two files were found, mostly wrong articles.
I’ve found [tools/json\_tools/util.py](https://github.com/CleverRaven/Cataclysm-DDA/blob/7787bc041afa43883ee41bc38e70988c5d5e2d08/tools/json_tools/util.py) that may help getting all the texts to check them.
But first I’d like some feedback on the general idea. **Should I** (or we if someone wants to help, see below) **proceed and make some translators work a bit more?**
Currently I just dumped all texts in one stream and feed it to LT, no corrections in-place so in current state changes need to be applied manually.
Proof of concept code:
```
import argparse
import json
PARSER = argparse.ArgumentParser(description='')
PARSER.add_argument(
'-f', '--filename', required=True, help='')
ARGS = PARSER.parse_args()
with open(ARGS.filename) as filehandler:
data = json.load(filehandler)
for item in data:
line = item.get('description', None)
if isinstance(line, dict):
line = line.get('description')
print(line, end='\n\n')
```
~~python3 ~/cdda/texts.py -f in.json | java -jar languagetool-commandline.jar -l en --json -d EN\_QUOTES,ENGLISH\_WORD\_REPEAT\_BEGINNING\_RULE,CD\_NN,DASH\_RULE,USE\_TO\_VERB,THE\_WORSE\_OF,PUNCTUATION\_PARAGRAPH\_END --disablecategories REDUNDANCY,TYPOGRAPHY,STYLE - | python3 -m json.tool \> ~/tmpfs/out.json~~
```
#!/bin/bash
for file in `find "$1" -name "*.json"`; do
echo $file
(python3 ~/cdda/texts.py -f $file | java -jar languagetool-commandline.jar -l en --json -d EN_QUOTES,ENGLISH_WORD_REPEAT_BEGINNING_RULE,CD_NN,DASH_RULE,USE_TO_VERB,THE_WORSE_OF,PUNCTUATION_PARAGRAPH_END,MASS_AGREEMENT,UNIT_SPACE,EN_DIACRITICS_REPLACE,WORD_CONTAINS_UNDERSCORE --disablecategories REDUNDANCY,TYPOGRAPHY,STYLE - | python3 -m json.tool > /tmpfs/out.json) && (
(echo "grepping"; grep -q "\"matches\": \[\]" /tmpfs/out.json ) && (echo "continuing"; continue) ||\
(
echo "matches found"
$EDITOR $file &
$EDITOR /tmpfs/out.json
)
)
done
```
---
### Author:  [@Zhilkin](https://discourse.cataclysmdda.org/u/Zhilkin)
#### Post date: [February 2, 2020, 10:28pm UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730/2 "2020-02-02T22:28:35Z")
Sure, make pull requests with suggested changes to fix typos and errors.
---
### Author:  [@kevin.granade](https://discourse.cataclysmdda.org/u/kevin.granade)
#### Post date: [February 2, 2020, 11:18pm UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730/3 "2020-02-02T23:18:38Z")
We don’t want to regress translations right before a release, but we can merge them right afterwards.
---
### Author:  [@int\_ua](https://discourse.cataclysmdda.org/u/int_ua)
#### Post date: [February 17, 2020, 2:04pm UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730/4 "2020-02-17T14:04:02Z")
>
---
### Author:  [@int\_ua](https://discourse.cataclysmdda.org/u/int_ua)
#### Post date: [February 17, 2020, 2:12pm UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730/5 "2020-02-17T14:12:24Z")
Many suggestions were left out, tried fixing only the ones I was sure about.
Most numerous I haven’t checked yet:
- adding space between measurement and units in ammo (“9 mm” vs “9mm”)
- adding diacritics in foreign words
- adding commas after “Typically”, “Still”, etc.
- making invisible-in-game descriptions all start with upper case just to appease LT
---
### Author:  [@int\_ua](https://discourse.cataclysmdda.org/u/int_ua)
#### Post date: [August 11, 2020, 7:56am UTC](https://discourse.cataclysmdda.org/t/mass-proofreading-with-languagetool/21730/6 "2020-08-11T07:56:51Z")
For anyone interested: this PR [https://github.com/CleverRaven/Cataclysm-DDA/pull/42585](https://github.com/CleverRaven/Cataclysm-DDA/pull/42585) was done by pasting `python3 table.py -f csv description` output into LT Desktop GUI with some mass auto-replacements like removing blank lines. It took a considerable amount of time and several GiB of RAM to process all descriptions at once.