MIL-001: repository setup and tooling #35

Merged
Tirsvad merged 7 commits from mil-001-repository-setup into main 2026-10-28 07:11:09 +01:00
Showing only changes of commit 0aad94bff6 - Show all commits
+107 -1
View File
@@ -1,2 +1,108 @@
# 045-scrape-movies # 🚀 scrape-movies
Day 45 of Udemy's "100 Days of Code: The Complete Python Pro Bootcamp": web scraping with Requests and BeautifulSoup. It parses a local HTML page, scrapes the Hacker News front page and writes Empire's 100 greatest movies to `movies.txt`. It is for fellow coursists and anyone looking for a small, tested scraping example.
## 📚 Table of Contents
- [Overview](#-overview)
- [Requirements](#-requirements)
- [Setup](#-setup)
- [Run](#-run)
- [Tests](#-tests)
- [License](#-license)
- [Links](#-links)
- [Appendix: Project documents](#-appendix-project-documents)
## 🧭 Overview
| Part | What it is for |
| --- | --- |
| `src/scrape_movies/` | The code in clean-architecture layers (`domain`, `application`, `infrastructure`, `presentation`) plus `constants.py`; the lecture steps `soup_basics`, `hacker_news` and the movies scraper follow milestone by milestone |
| `tests/` | pytest tests that run offline against saved pages in `tests/fixtures/` |
| `docs/` | Project documents (business case, plan, milestones, reviews) and the wiki in English, Danish and Bahasa Malaysia |
| `Doxyfile` | Generates the API pages from the Doxygen comments in the source |
The only runtime dependencies are `requests` and `beautifulsoup4`.
## 📋 Requirements
- Python 3.13 or newer (the `venv` module comes with it).
- `requests` and `beautifulsoup4` (installed by the setup below).
- Optional: [Doxygen](https://www.doxygen.nl/) to build the API pages.
## 🛠️ Setup
Create a local virtual environment named `.venv`, update `pip`, then install the project with its test tools.
Windows (PowerShell):
```powershell
python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
```
Linux and macOS:
```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
```
Leave out `[dev]` to install only the runtime dependencies. Personal tokens belong in a git-ignored `.env`; the project never reads it.
## ▶️ Run
The lecture steps are added milestone by milestone; each gets a section with its command, its output and a screenshot here.
```bash
python -m scrape_movies # available from the movies milestone
```
## 🧪 Tests
```bash
python -m pytest
```
The tests use saved pages and never touch the network. Lint and types: `ruff check .`, `ruff format --check .` and `mypy src tests`. API pages: `doxygen Doxyfile`.
## 📄 License
See [LICENSE](./LICENSE).
## 🔗 Links
- [Repository][repository]
- [Documentation][documentation]
- [Issue tracker][issues]
## 📎 Appendix: Project documents
| Document | What it tells you |
| --- | --- |
| [AGENTS.md][agents] | The rules AI agents follow in this repository |
| [Business Case][BC-001] | Why the project exists, its scope and success criteria |
| [Stakeholder Analysis][SA-001] | Who is involved; the stakeholder IDs used for owners and reviewers |
| [Project Plan][PP-001] | The gateways and their schedule |
| [Milestones][milestones] | One document per gateway: Go/No-Go criteria and tasks |
| [Artifact registry][registry] | Where each artifact type lives, the next version, the PO language and domain |
| [Framework README][framework-readme] | The framework this project uses |
| [Framework changelog][framework-changelog] | What changed in each framework release |
---
[repository]: https://git.tirsystem.com/Tirsvad-Udemy-100-days-of-code/045-scrape-movies
[documentation]: https://git.tirsystem.com/Tirsvad-Udemy-100-days-of-code/045-scrape-movies/wiki
[issues]: https://git.tirsystem.com/Tirsvad-Udemy-100-days-of-code/045-scrape-movies/issues
[agents]: ./AGENTS.md
[BC-001]: ./docs/business-case.md
[SA-001]: ./docs/stakeholder-analysis.md
[PP-001]: ./docs/project-plan.md
[milestones]: ./docs/milestones/
[registry]: ./docs/artifact-registry.md
[framework-readme]: ./framework/README.md
[framework-changelog]: ./framework/CHANGELOG.md