Add a coverage gate to validate-workflows and stop tracking .coverage
TirSystem/github-action: Publish wiki / publish-wiki (push) Skipped
TirSystem/github-action: Sync GitHub mirror metadata / sync-metadata (push) Skipped
Validate workflows / validate (pull_request) Successful in 22s

Task: MIL-003#3
This commit is contained in:
2026-10-11 12:05:47 +08:00
parent 80fb0cfa82
commit 075fae2e45
10 changed files with 55 additions and 16 deletions
BIN
View File
Binary file not shown.
+12
View File
@@ -0,0 +1,12 @@
# Coverage of the scripts in tools/ by the offline tests. validate-workflows.yml
# runs `coverage run -m pytest -q tests` and `coverage report`, which fails
# below fail_under. Measured at 97% on 2026-10-11; the gate leaves room for a
# few lines and fails when a tool grows without tests.
[run]
branch = True
source = tools
[report]
fail_under = 95
show_missing = True
skip_empty = True
+5 -2
View File
@@ -20,5 +20,8 @@ jobs:
python3 -m venv .venv python3 -m venv .venv
.venv/bin/python -m pip install --quiet -r tests/requirements.txt .venv/bin/python -m pip install --quiet -r tests/requirements.txt
- name: Validate workflow files and run the offline tests - name: Validate workflow files and run the offline tests under coverage
run: .venv/bin/python -m pytest -q tests run: .venv/bin/python -m coverage run -m pytest -q tests
- name: Check the coverage of the tools
run: .venv/bin/python -m coverage report
+3
View File
@@ -14,3 +14,6 @@
.venv/ .venv/
__pycache__/ __pycache__/
.pytest_cache/ .pytest_cache/
# Local coverage data (python -m coverage run)
.coverage
+4 -3
View File
@@ -255,7 +255,7 @@ Trigger -> Workflow: Start workflow
activate Workflow activate Workflow
Workflow -> Workflow: Check out the repository Workflow -> Workflow: Check out the repository
Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt
Workflow -> Workflow: Run pytest on tests/ (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API Workflow -> Workflow: Run pytest on tests/ under coverage (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API
alt a test fails alt a test fails
Workflow --> Trigger: Workflow fails (the pull request is not ready) Workflow --> Trigger: Workflow fails (the pull request is not ready)
@@ -298,10 +298,11 @@ The workflow files and the scripts in `tools/` are validated by offline tests th
python3 -m venv .venv python3 -m venv .venv
source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1 source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install -r tests/requirements.txt python -m pip install -r tests/requirements.txt
python -m pytest -q tests python -m coverage run -m pytest -q tests
python -m coverage report
``` ```
The same checks run on every pull request and push to `main` through [`validate-workflows.yml`](.gitea/workflows/validate-workflows.yml). They parse every workflow as YAML, check that none embeds Python, compile the scripts in `tools/`, require explicit job permissions, and test credential parsing, push mirror address parsing and the sync and publish flows against a fake API. The same checks run on every pull request and push to `main` through [`validate-workflows.yml`](.gitea/workflows/validate-workflows.yml). They parse every workflow as YAML, check that none embeds Python, compile the scripts in `tools/`, require explicit job permissions, and test credential parsing, push mirror address parsing and the sync and publish flows against a fake API. The run fails when the line and branch coverage of `tools/` falls below 95% (`.coveragerc`).
After a change to the sync workflow, also trigger **Sync GitHub mirror metadata** manually in Gitea and compare the description and topics on the GitHub mirror. After a change to the sync workflow, also trigger **Sync GitHub mirror metadata** manually in Gitea and compare the description and topics on the GitHub mirror.
+1
View File
@@ -1,2 +1,3 @@
pytest>=8,<10 pytest>=8,<10
PyYAML>=6,<7 PyYAML>=6,<7
coverage>=7,<8
+16
View File
@@ -73,3 +73,19 @@ def test_no_workflow_copy_outside_the_workflow_directories():
if path.name == SYNC_WORKFLOW.name if path.name == SYNC_WORKFLOW.name
] ]
assert not stray, f"duplicate workflow copies: {stray}" assert not stray, f"duplicate workflow copies: {stray}"
def test_validation_runs_the_tests_under_a_coverage_gate():
"""validate-workflows.yml measures tools/ and fails below the .coveragerc limit."""
text = (REPO_ROOT / ".gitea" / "workflows" / "validate-workflows.yml").read_text(
encoding="utf-8"
)
assert "-m coverage run -m pytest -q tests" in text
assert "-m coverage report" in text
config = (REPO_ROOT / ".coveragerc").read_text(encoding="utf-8")
assert "source = tools" in config and "branch = True" in config
assert any(
line.replace(" ", "") == "fail_under=95" for line in config.splitlines()
), "the gate is 95 percent"
requirements = (REPO_ROOT / "tests" / "requirements.txt").read_text(encoding="utf-8")
assert "coverage" in requirements
+5 -4
View File
@@ -1,4 +1,4 @@
<!-- translation of Home.md, source sha256 8d1f5ed6df38 --> <!-- translation of Home.md, source sha256 510b243ec5f2 -->
> Dette er en oversættelse af [`Home`](Home). Den engelske side er autoritativ, og ændringer her bliver overskrevet, næste gang wikien publiceres. > Dette er en oversættelse af [`Home`](Home). Den engelske side er autoritativ, og ændringer her bliver overskrevet, næste gang wikien publiceres.
@@ -252,7 +252,7 @@ Trigger -> Workflow: Start workflow
activate Workflow activate Workflow
Workflow -> Workflow: Check out the repository Workflow -> Workflow: Check out the repository
Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt
Workflow -> Workflow: Run pytest on tests/ (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API Workflow -> Workflow: Run pytest on tests/ under coverage (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API
alt a test fails alt a test fails
Workflow --> Trigger: Workflow fails (the pull request is not ready) Workflow --> Trigger: Workflow fails (the pull request is not ready)
@@ -299,9 +299,10 @@ Workflow-filerne og scripts i `tools/` valideres af offline-tests, som hverken k
python3 -m venv .venv python3 -m venv .venv
source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1 source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install -r tests/requirements.txt python -m pip install -r tests/requirements.txt
python -m pytest -q tests python -m coverage run -m pytest -q tests
python -m coverage report
``` ```
De samme kontroller kører ved hver pull request og hvert push til `main` gennem [`validate-workflows.yml`](https://git.tirsystem.com/TirSystem/github-action/src/branch/main/.gitea/workflows/validate-workflows.yml). De parser alle workflows som YAML, kontrollerer at ingen indlejrer Python, kompilerer scripts i `tools/`, kræver eksplicitte jobtilladelser og tester parsing af credentials, parsing af push-spejlets adresse og synkroniserings- og publiceringsforløbene mod et falsk API. De samme kontroller kører ved hver pull request og hvert push til `main` gennem [`validate-workflows.yml`](https://git.tirsystem.com/TirSystem/github-action/src/branch/main/.gitea/workflows/validate-workflows.yml). De parser alle workflows som YAML, kontrollerer at ingen indlejrer Python, kompilerer scripts i `tools/`, kræver eksplicitte jobtilladelser og tester parsing af credentials, parsing af push-spejlets adresse og synkroniserings- og publiceringsforløbene mod et falsk API. Kørslen fejler, når linje- og forgreningsdækningen af `tools/` er under 95 % (`.coveragerc`).
Efter en ændring af synkroniseringsworkflowet skal du også starte **Sync GitHub mirror metadata** manuelt i Gitea og sammenligne beskrivelsen og emnerne på GitHub-spejlet. Efter en ændring af synkroniseringsworkflowet skal du også starte **Sync GitHub mirror metadata** manuelt i Gitea og sammenligne beskrivelsen og emnerne på GitHub-spejlet.
+4 -3
View File
@@ -252,7 +252,7 @@ Trigger -> Workflow: Start workflow
activate Workflow activate Workflow
Workflow -> Workflow: Check out the repository Workflow -> Workflow: Check out the repository
Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt
Workflow -> Workflow: Run pytest on tests/ (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API Workflow -> Workflow: Run pytest on tests/ under coverage (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API
alt a test fails alt a test fails
Workflow --> Trigger: Workflow fails (the pull request is not ready) Workflow --> Trigger: Workflow fails (the pull request is not ready)
@@ -299,9 +299,10 @@ The workflow files and the scripts in `tools/` are validated by offline tests th
python3 -m venv .venv python3 -m venv .venv
source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1 source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install -r tests/requirements.txt python -m pip install -r tests/requirements.txt
python -m pytest -q tests python -m coverage run -m pytest -q tests
python -m coverage report
``` ```
The same checks run on every pull request and push to `main` through [`validate-workflows.yml`](https://git.tirsystem.com/TirSystem/github-action/src/branch/main/.gitea/workflows/validate-workflows.yml). They parse every workflow as YAML, check that none embeds Python, compile the scripts in `tools/`, require explicit job permissions, and test credential parsing, push mirror address parsing and the sync and publish flows against a fake API. The same checks run on every pull request and push to `main` through [`validate-workflows.yml`](https://git.tirsystem.com/TirSystem/github-action/src/branch/main/.gitea/workflows/validate-workflows.yml). They parse every workflow as YAML, check that none embeds Python, compile the scripts in `tools/`, require explicit job permissions, and test credential parsing, push mirror address parsing and the sync and publish flows against a fake API. The run fails when the line and branch coverage of `tools/` falls below 95% (`.coveragerc`).
After a change to the sync workflow, also trigger **Sync GitHub mirror metadata** manually in Gitea and compare the description and topics on the GitHub mirror. After a change to the sync workflow, also trigger **Sync GitHub mirror metadata** manually in Gitea and compare the description and topics on the GitHub mirror.
+5 -4
View File
@@ -1,4 +1,4 @@
<!-- translation of Home.md, source sha256 8d1f5ed6df38 --> <!-- translation of Home.md, source sha256 510b243ec5f2 -->
> Ini ialah terjemahan [`Home`](Home). Halaman bahasa Inggeris ialah teks yang sah, dan perubahan di sini akan ditimpa pada penerbitan wiki yang seterusnya. > Ini ialah terjemahan [`Home`](Home). Halaman bahasa Inggeris ialah teks yang sah, dan perubahan di sini akan ditimpa pada penerbitan wiki yang seterusnya.
@@ -252,7 +252,7 @@ Trigger -> Workflow: Start workflow
activate Workflow activate Workflow
Workflow -> Workflow: Check out the repository Workflow -> Workflow: Check out the repository
Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt Workflow -> Workflow: Create a virtual environment and\ninstall tests/requirements.txt
Workflow -> Workflow: Run pytest on tests/ (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API Workflow -> Workflow: Run pytest on tests/ under coverage (offline, no secrets):\nworkflows parse as YAML, no embedded Python,\nscripts compile, wiki pages and translations are current,\nsync, publish and release flows against a fake API
alt a test fails alt a test fails
Workflow --> Trigger: Workflow fails (the pull request is not ready) Workflow --> Trigger: Workflow fails (the pull request is not ready)
@@ -299,9 +299,10 @@ Fail aliran kerja dan skrip dalam `tools/` disahkan oleh ujian luar talian yang
python3 -m venv .venv python3 -m venv .venv
source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1 source .venv/bin/activate # Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install -r tests/requirements.txt python -m pip install -r tests/requirements.txt
python -m pytest -q tests python -m coverage run -m pytest -q tests
python -m coverage report
``` ```
Semakan yang sama dijalankan pada setiap pull request dan setiap push ke `main` melalui [`validate-workflows.yml`](https://git.tirsystem.com/TirSystem/github-action/src/branch/main/.gitea/workflows/validate-workflows.yml). Semakan itu menghurai setiap aliran kerja sebagai YAML, memastikan tiada yang menanam Python, mengkompil skrip dalam `tools/`, memerlukan kebenaran kerja yang jelas, dan menguji penghuraian kelayakan (credentials), penghuraian alamat push mirror serta aliran penyegerakan dan penerbitan terhadap API palsu. Semakan yang sama dijalankan pada setiap pull request dan setiap push ke `main` melalui [`validate-workflows.yml`](https://git.tirsystem.com/TirSystem/github-action/src/branch/main/.gitea/workflows/validate-workflows.yml). Semakan itu menghurai setiap aliran kerja sebagai YAML, memastikan tiada yang menanam Python, mengkompil skrip dalam `tools/`, memerlukan kebenaran kerja yang jelas, dan menguji penghuraian kelayakan (credentials), penghuraian alamat push mirror serta aliran penyegerakan dan penerbitan terhadap API palsu. Larian gagal apabila liputan baris dan cabang `tools/` di bawah 95% (`.coveragerc`).
Selepas perubahan pada aliran kerja penyegerakan, cetuskan juga **Sync GitHub mirror metadata** secara manual dalam Gitea dan bandingkan penerangan dan topik pada cermin GitHub. Selepas perubahan pada aliran kerja penyegerakan, cetuskan juga **Sync GitHub mirror metadata** secara manual dalam Gitea dan bandingkan penerangan dan topik pada cermin GitHub.