From 198830588caa5047ff2cee4382b8f2a9bd391bff Mon Sep 17 00:00:00 2001 From: Jens Tirsvad Nielsen Date: Wed, 21 Oct 2026 10:18:30 +0800 Subject: [PATCH 1/3] Judge each line of a comment block, then the block The trial of the English-comments check found that a Danish comment line next to an English one was missed: the joined block counted as English. A block of several lines is now judged line by line first (at least four words and a probability of 0.7), and as a whole when no line is flagged. Found and fixed while preparing MIL-013 task 8; the guide, the wiki and ADR-0012 say so. Refs #200 Refs #204 Task: MIL-013#4 Task: MIL-013#10 --- .../adr/adr-0012-english-comment-detection.md | 1 + docs/wiki/EnglishComments.da.md | 7 ++- docs/wiki/EnglishComments.md | 5 +- docs/wiki/EnglishComments.ms.md | 7 ++- guides/english-comments.md | 5 +- tests/test_english_comments.py | 18 +++++- tools/english_comments.py | 59 ++++++++++++------- 7 files changed, 74 insertions(+), 28 deletions(-) diff --git a/docs/adr/adr-0012-english-comment-detection.md b/docs/adr/adr-0012-english-comment-detection.md index dca9c09..86094ba 100644 --- a/docs/adr/adr-0012-english-comment-detection.md +++ b/docs/adr/adr-0012-english-comment-detection.md @@ -52,6 +52,7 @@ A comment is judged **not English** when `langid` 1.1.6, restricted to `en da no - drop every token that is not a word: a token with a code character (`/ \ { } [ ] " ` = < > | $ @ # % & + ^ ~ *`), an underscore, a digit or `::` (a path, a URL, a JSON key, an identifier); a token that starts with `-` (an option), contains `.` (a file name), is written in capitals (an acronym or a constant) or is camelCase; a leading label such as `Usage:` or `Note:`; and the words of the allowed-words file. Split what is left on `-`; - keep a token only if it is letters, with at most one inner apostrophe (letters include `æ ø å ä ö ü` and the other letters of Latin scripts); +- a block of several consecutive comment lines is judged **line by line first**, each line only with at least four words and a probability of at least 0.7, so an English line cannot hide a Danish one next to it (the first version judged the joined block and missed exactly that case in the trial of [MIL-013] task 8); when no line is flagged, the joined block is judged with the values above, so a sentence cut over short lines is not missed. A single comment line is judged once, with the values above; - a comment with fewer than three words left is **not judged** (it passes), and so is a comment that holds only identifiers, URLs or numbers. On the corpus this gives 0 false positives in 300 English comments, Danish 44/45 (97.8 %), Malay 41/45 (91.1 %), and 48/48 for the other languages, meeting criterion 1 of [MIL-013]. The threshold, the minimum of three words and the candidate languages are inputs of the workflow with these values as defaults; the rates are a test that fails when the code changes them (task 7). diff --git a/docs/wiki/EnglishComments.da.md b/docs/wiki/EnglishComments.da.md index 547546b..8f55632 100644 --- a/docs/wiki/EnglishComments.da.md +++ b/docs/wiki/EnglishComments.da.md @@ -1,4 +1,4 @@ - + > Dette er en oversættelse af [`EnglishComments`](EnglishComments). Den engelske side er autoritativ, og ændringer her bliver overskrevet, næste gang wikien publiceres. @@ -52,7 +52,10 @@ kommentar. Kommentarerne og docstrings i filer på de sprog, kodetælleren kender ([kodemålinger](CodeMetrics)): Python, shell, PowerShell, C, C++, C#, Java, JavaScript, TypeScript, Go, Rust og SQL. Strenge og kode læses ikke. Kommentarlinjer -efter hinanden bedømmes som én blok. +efter hinanden danner en blok. Hver linje i en blok bedømmes for sig (mindst fire ord og en +sandsynlighed på mindst 0,7), så en engelsk linje ikke kan skjule en dansk ved siden af; når ingen linje +markeres, bedømmes blokken som helhed (mindst `min-words` ord og `threshold`), så en sætning delt over korte +linjer ikke overses. Før en blok bedømmes, fjernes alt, der ikke er et ord: stier, URL'er, tilvalg (`--output`), filnavne, identifikatorer (`snake_case`, `camelCase`), tal, diff --git a/docs/wiki/EnglishComments.md b/docs/wiki/EnglishComments.md index f0b16c4..5cdcd8a 100644 --- a/docs/wiki/EnglishComments.md +++ b/docs/wiki/EnglishComments.md @@ -51,7 +51,10 @@ the check still fails the run, but no comment is written. The comments and docstrings of the files of the languages in the code counter ([code metrics](CodeMetrics)): Python, shell, PowerShell, C, C++, C#, Java, JavaScript, TypeScript, Go, Rust and SQL. Strings and code are not read. -Consecutive comment lines are judged as one block. +Consecutive comment lines form a block. Each line of a block is judged on its own +(at least four words and a probability of at least 0.7), so an English line cannot +hide a Danish one next to it; when no line is flagged, the block is judged as a whole +(at least `min-words` words and `threshold`), so a sentence cut over short lines is not missed. Before a block is judged, everything that is not a word is removed: paths, URLs, options (`--output`), file names, identifiers (`snake_case`, `camelCase`), diff --git a/docs/wiki/EnglishComments.ms.md b/docs/wiki/EnglishComments.ms.md index 5ce80ca..94fea87 100644 --- a/docs/wiki/EnglishComments.ms.md +++ b/docs/wiki/EnglishComments.ms.md @@ -1,4 +1,4 @@ - + > Ini ialah terjemahan [`EnglishComments`](EnglishComments). Halaman bahasa Inggeris ialah teks yang sah, dan perubahan di sini akan ditimpa pada penerbitan wiki yang seterusnya. @@ -52,7 +52,10 @@ semakan masih menggagalkan larian, tetapi tiada komen ditulis. Komen dan docstring dalam fail bahasa yang dikenali oleh pengira kod ([metrik kod](CodeMetrics)): Python, shell, PowerShell, C, C++, C#, Java, JavaScript, TypeScript, Go, Rust dan SQL. String dan kod tidak dibaca. Baris komen -yang berturutan dinilai sebagai satu blok. +yang berturutan membentuk satu blok. Setiap baris dalam blok dinilai secara berasingan (sekurang-kurangnya +empat perkataan dan kebarangkalian sekurang-kurangnya 0.7), jadi baris bahasa Inggeris tidak dapat menyembunyikan +baris bahasa Denmark di sebelahnya; apabila tiada baris ditandakan, blok dinilai secara keseluruhan +(sekurang-kurangnya `min-words` perkataan dan `threshold`), supaya ayat yang dipecahkan kepada baris pendek tidak terlepas. Sebelum sesuatu blok dinilai, segala yang bukan perkataan dibuang: laluan, URL, pilihan (`--output`), nama fail, pengecam (`snake_case`, `camelCase`), nombor, diff --git a/guides/english-comments.md b/guides/english-comments.md index 5779cff..4a7aa43 100644 --- a/guides/english-comments.md +++ b/guides/english-comments.md @@ -45,7 +45,10 @@ the check still fails the run, but no comment is written. The comments and docstrings of the files of the languages in the code counter ([code metrics](code-metrics.md)): Python, shell, PowerShell, C, C++, C#, Java, JavaScript, TypeScript, Go, Rust and SQL. Strings and code are not read. -Consecutive comment lines are judged as one block. +Consecutive comment lines form a block. Each line of a block is judged on its own +(at least four words and a probability of at least 0.7), so an English line cannot +hide a Danish one next to it; when no line is flagged, the block is judged as a whole +(at least `min-words` words and `threshold`), so a sentence cut over short lines is not missed. Before a block is judged, everything that is not a word is removed: paths, URLs, options (`--output`), file names, identifiers (`snake_case`, `camelCase`), diff --git a/tests/test_english_comments.py b/tests/test_english_comments.py index 6129183..15bfe9a 100644 --- a/tests/test_english_comments.py +++ b/tests/test_english_comments.py @@ -114,9 +114,9 @@ class TestExtract: def test_a_file_of_an_unknown_language_has_no_comments(self): assert ec.extract("notes.md", "# Heading\n") == [] - def test_consecutive_lines_are_one_block(self): + def test_consecutive_lines_are_one_group(self): lines = [(1, "a"), (2, "b"), (4, "c"), (5, "d"), (5, "e")] - assert ec.blocks(lines) == [(1, 2, "a b"), (4, 5, "c d e")] + assert ec.groups(lines) == [[(1, "a"), (2, "b")], [(4, "c"), (5, "d"), (5, "e")]] class TestCheckText: @@ -134,6 +134,20 @@ class TestCheckText: assert [f.line for f in ec.check_text("a.py", text, {3})] == [3] assert ec.check_text("a.py", text, {2}) == [] + def test_a_danish_line_next_to_an_english_line_is_still_found(self): + english = "# This function adds the totals of the basket together and returns the sum\n" + findings = ec.check_text("a.py", english + self.DANISH) + assert [(f.line, f.end_line, f.language) for f in findings] == [(2, 2, "da")] + + def test_a_sentence_cut_over_short_lines_is_judged_as_a_whole(self): + text = "# Denne funktion\n# beregner totalen\n# for kurven\n# og gemmer resultatet\n" + [finding] = ec.check_text("a.py", text) + assert (finding.line, finding.end_line, finding.language) == (1, 4, "da") + + def test_only_a_flagged_line_is_reported_when_the_block_is_mixed(self): + text = self.DANISH + "# and this line is plain English text about the totals\n" + assert [f.line for f in ec.check_text("a.py", text, {2})] == [] + def test_the_marker_skips_the_block(self): text = self.DANISH.rstrip() + " # english-comments: ignore\n" assert ec.check_text("a.py", text) == [] diff --git a/tools/english_comments.py b/tools/english_comments.py index 4877cf0..3df0d61 100644 --- a/tools/english_comments.py +++ b/tools/english_comments.py @@ -1,11 +1,12 @@ """Find comments in source code that are not written in English (MIL-013, ADR-0012). Run by .gitea/workflows/english-comments.yml. It reads the comments and docstrings -of the source files of the languages of ADR-0010, joins the lines of one comment -block, and judges the block with ``langid`` (restricted to a few candidate -languages) after removing everything that is not a word: paths, options, -identifiers, numbers and the words of the allowed-words file. A block with fewer -than ``--min-words`` words left is not judged. +of the source files of the languages of ADR-0010 and judges each comment line +with ``langid`` (restricted to a few candidate languages) after removing +everything that is not a word: paths, options, identifiers, numbers and the words +of the allowed-words file. When no line of a block of consecutive comment lines +is flagged, the block is judged as a whole, so a sentence cut over short lines is +not missed. A text with fewer than ``--min-words`` words left is not judged. Usage: python tools/english_comments.py check [--directory DIR] [--scope changed|all] @@ -38,6 +39,9 @@ from pathlib import Path DEFAULT_THRESHOLD = 0.6 DEFAULT_MIN_WORDS = 3 +# a single line of a longer block is judged more strictly than a whole block (ADR-0012) +LINE_THRESHOLD = 0.7 +LINE_MIN_WORDS = 4 MAX_FILE_BYTES = 1_000_000 CANDIDATES = ["en", "da", "no", "sv", "de", "fr", "ms", "id", "nl"] LANGUAGE_NAMES = { @@ -149,16 +153,15 @@ def extract(path: str, text: str) -> list[tuple[int, str]]: return sorted((n, t) for n, t in found if t and not _PRAGMA.match(t)) -def blocks(lines: list[tuple[int, str]]) -> list[tuple[int, int, str]]: - """Join comment lines on consecutive line numbers: (first line, last line, text).""" - joined: list[tuple[int, int, str]] = [] +def groups(lines: list[tuple[int, str]]) -> list[list[tuple[int, str]]]: + """Group comment lines on consecutive line numbers: one group is one comment block.""" + found: list[list[tuple[int, str]]] = [] for number, text in lines: - if joined and joined[-1][1] in (number, number - 1): - first, _, previous = joined[-1] - joined[-1] = (first, number, f"{previous} {text}") + if found and found[-1][-1][0] in (number, number - 1): + found[-1].append((number, text)) else: - joined.append((number, number, text)) - return joined + found.append([(number, text)]) + return found def words(text: str, allowed: frozenset[str] = frozenset()) -> list[str]: @@ -169,7 +172,7 @@ def words(text: str, allowed: frozenset[str] = frozenset()) -> list[str]: tokens = tokens[1:] # a label such as "Usage:" or "Note:" is not prose for token in tokens: if len(token) > 1 and token[0] == "." and token[1].isalpha(): - continue # a hidden file such as .clang-format + continue # a dotfile, for example one of a linter token = token.strip("().,;:!?") # a token with code characters (quotes, braces, `=`, `/`, `_`, digits) is not a word if not token or _NOT_WORD_CHARS.search(token) or "::" in token: @@ -225,14 +228,30 @@ def check_text( ) -> list[Finding]: """Findings of one file; with ``lines`` only the blocks that touch those line numbers.""" findings: list[Finding] = [] - for first, last, body in blocks(extract(path, text)): + for group in groups(extract(path, text)): + body = " ".join(line for _, line in group) if IGNORE_MARKER.search(body): continue - if lines is not None and not any(first <= n <= last for n in lines): - continue - verdict = judge(body, allowed, threshold, min_words) - if verdict: - findings.append(Finding(path, first, last, verdict[0], round(verdict[1], 2), body)) + found: list[Finding] = [] + if len(group) > 1: + # each line on its own, so an English line cannot hide a Danish one next to it + found = [ + Finding(path, number, number, verdict[0], round(verdict[1], 2), line) + for number, line in group + if ( + verdict := judge( + line, allowed, max(threshold, LINE_THRESHOLD), max(min_words, LINE_MIN_WORDS) + ) + ) + ] + if not found and (verdict := judge(body, allowed, threshold, min_words)): + # the block as a whole, so a sentence cut over short lines is not missed + found = [Finding(path, group[0][0], group[-1][0], verdict[0], round(verdict[1], 2), body)] + findings += [ + f + for f in found + if lines is None or any(f.line <= n <= f.end_line for n in lines) + ] return findings -- 2.54.0 From 3d46c4b60aa86981163a12327fb28dbb24ceadfd Mon Sep 17 00:00:00 2001 From: Jens Tirsvad Nielsen Date: Wed, 21 Oct 2026 10:26:13 +0800 Subject: [PATCH 2/3] Tell the author in a called workflow: ask for the pull request, not the event The trial on the instance showed that gitea.event_name is workflow_call inside a called workflow, so the step that comments on the pull request was skipped. The pull request number, its author and the base ref are available there, and the step now runs when the number is set. Refs #201 Refs #204 Task: MIL-013#5 --- .gitea/workflows/english-comments.yml | 3 ++- tests/test_english_comments_report.py | 6 ++++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/.gitea/workflows/english-comments.yml b/.gitea/workflows/english-comments.yml index 4402174..754fa04 100644 --- a/.gitea/workflows/english-comments.yml +++ b/.gitea/workflows/english-comments.yml @@ -100,7 +100,8 @@ jobs: --output "$FINDINGS_FILE" - name: Tell the author - if: ${{ always() && gitea.event_name == 'pull_request' }} + # in a called workflow gitea.event_name is workflow_call, so ask for the pull request itself + if: ${{ always() && gitea.event.pull_request.number != '' }} continue-on-error: true env: GITEA_API_URL: ${{ gitea.api_url }} diff --git a/tests/test_english_comments_report.py b/tests/test_english_comments_report.py index 6f21d4a..d11cd88 100644 --- a/tests/test_english_comments_report.py +++ b/tests/test_english_comments_report.py @@ -262,6 +262,12 @@ class TestWorkflow: assert tell["continue-on-error"] is True and "always()" in tell["if"] assert "steps.check.outcome == 'failure'" in steps["Fail the run when a comment is not in English"]["if"] + def test_the_author_step_asks_for_the_pull_request_not_for_the_event_name(self, document): + # in a called workflow gitea.event_name is workflow_call (seen on the instance, MIL-013 task 8) + tell = next(s for s in document["jobs"]["english-comments"]["steps"] if s.get("name") == "Tell the author") + assert "event_name" not in tell["if"] + assert "gitea.event.pull_request.number != ''" in tell["if"] + def test_inputs_reach_the_scripts_as_environment_variables_only(self, document): for step in document["jobs"]["english-comments"]["steps"]: assert "inputs." not in step.get("run", "") -- 2.54.0 From 39bcaa5bacaf85c87c0b19d73ef97106655359f3 Mon Sep 17 00:00:00 2001 From: Jens Tirsvad Nielsen Date: Wed, 21 Oct 2026 10:29:20 +0800 Subject: [PATCH 3/3] Link the run by its id in the comment on the pull request The trial showed that the web address of a run on the instance holds the id of the run, not its number, so the link in the comment pointed to another run. Refs #201 Refs #204 Task: MIL-013#5 --- .gitea/workflows/english-comments.yml | 2 +- tests/test_english_comments_report.py | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/.gitea/workflows/english-comments.yml b/.gitea/workflows/english-comments.yml index 754fa04..e0ae657 100644 --- a/.gitea/workflows/english-comments.yml +++ b/.gitea/workflows/english-comments.yml @@ -108,7 +108,7 @@ jobs: SOURCE_REPOSITORY: ${{ gitea.repository }} PR_NUMBER: ${{ gitea.event.pull_request.number }} PR_AUTHOR: ${{ gitea.event.pull_request.user.login }} - RUN_URL: ${{ gitea.server_url }}/${{ gitea.repository }}/actions/runs/${{ gitea.run_number }} + RUN_URL: ${{ gitea.server_url }}/${{ gitea.repository }}/actions/runs/${{ gitea.run_id }} REPORT_TOKEN: ${{ secrets.REPORT_TOKEN || gitea.token }} run: python3 shared-scripts/tools/english_comments_report.py diff --git a/tests/test_english_comments_report.py b/tests/test_english_comments_report.py index d11cd88..8fd40b0 100644 --- a/tests/test_english_comments_report.py +++ b/tests/test_english_comments_report.py @@ -268,6 +268,11 @@ class TestWorkflow: assert "event_name" not in tell["if"] assert "gitea.event.pull_request.number != ''" in tell["if"] + def test_the_link_to_the_run_uses_the_run_id(self, document): + # the URL of a run on the instance holds its id, not its number (MIL-013 task 8) + tell = next(s for s in document["jobs"]["english-comments"]["steps"] if s.get("name") == "Tell the author") + assert tell["env"]["RUN_URL"].endswith("/actions/runs/${{ gitea.run_id }}") + def test_inputs_reach_the_scripts_as_environment_variables_only(self, document): for step in document["jobs"]["english-comments"]["steps"]: assert "inputs." not in step.get("run", "") -- 2.54.0