2026-10-28 07:40:45 +01:00
21 changed files with 598 additions and 157 deletions
@@ -19,11 +19,19 @@ The assignment is built in four lecture steps. Each step was delivered on its ow
 | Step | What it adds | Status |
 | --- | --- | --- |
-| 1 | Scrape the price from the practice page with BeautifulSoup | planned (MIL-002) |
+| 1 | Scrape the price from the practice page with BeautifulSoup | done |
 | 2 | E-mail alert below the target price, secrets in `.env` | planned (MIL-003) |
 | 3 | Browser-like request headers | planned (MIL-004) |
 | 4 | The live Amazon page | planned (MIL-005) |
 ### Step 1: scrape the price
 `get_title(html)` and `get_price(html)` in `src/amazon_price_tracker/infrastructure/scraper.py` use BeautifulSoup on the [practice page](https://appbrewery.github.io/instant_pot/). The price element has the class `a-offscreen` and its text is `$99.99`; `split("$")` removes the currency sign and `float()` returns `99.99`.
 ![The practice page the bot reads](docs/images/step-1-practice-page.png)
 ![Output of the bot: title and price as a float](docs/images/step-1-run.png)
 Repository layout:
 | Path | What it is for |
@@ -15,7 +15,7 @@ document of a type. `Primary File` may contain a glob (e.g.
 | SA | Stakeholder Analysis | docs/stakeholder-analysis.md | 002 |
 | PP | Project Plan | docs/project-plan.md | 002 |
 | MIL | Milestone / Gateway | docs/milestones/*.md | 007 |
-| RC | SQA Review Record | docs/sqa/reviews/rc-*.md | 011 |
+| RC | SQA Review Record | docs/sqa/reviews/rc-*.md | 012 |
 ## Languages
@@ -0,0 +1,61 @@
 # RC-011 Code review of MIL-002
 ## Metadata
 | Key | Value |
 | --- | --- |
 | ID | RC-011 |
 | CrossReference | [MIL-002], [QC-PY-001] |
 ## Version History
 | Date | Status | Author | Reviewer | Change | Commit |
 | --- | --- | --- | --- | --- | --- |
 | 2026-10-28 | Proposed | Jens Tirsvad Nielsen | AI as stand-in for S01 | Initial version | pending |
 ---
 ## Artifact Under Review
 - Instance reviewed: [MIL-002]
 - Checklist used: [QC-PY-001]
 - Scope: full review
 - Language and domain: n/a (technical)
 - Language reviewer: none
 - Mode: autonomous, not independent (AI as stand-in for S01); the author and the stand-in are both the AI working for S01
 ## Checklist Results
 | # | Criterion | Status | Evidence/Notes |
 | --- | --- | --- | --- |
 | 1 | Packages, modules, functions, variables, classes and constants follow PEP 8 casing (`snake_case`, `PascalCase`, `UPPER_SNAKE`) | Pass | Modules and functions are snake_case, classes PascalCase, constants UPPER_SNAKE |
 | 2 | Names state purpose in the domain's language; no unexplained abbreviations, no single-letter names outside tiny scopes | Pass | Names state purpose (price, target, headers); no abbreviations left unexplained |
 | 3 | Code is produced by the project's formatter and passes its linter with no unexplained suppressions | Pass | ruff format and ruff check pass: 15 pytest tests, ruff 0 findings, mypy strict 0 errors (17 files) |
 | 4 | Every function and method signature is type-annotated, including `-> None` | Pass | All signatures annotated; mypy --strict passes: 15 pytest tests, ruff 0 findings, mypy strict 0 errors (17 files) |
 | 5 | No bare `except:`, no swallowed exceptions; specific exceptions are raised and the cause is kept (`raise ... from`) | Pass | No bare except; errors are specific and chained with `raise ... from` where a cause exists |
 | 6 | No mutable default arguments and no shadowed builtins | Pass | No mutable defaults, no shadowed builtins |
 | 7 | Files, locks and connections are managed with context managers | Pass | requests.get is used once per call and the response is not kept open; no files are opened |
 | 8 | Public modules, classes and functions have docstrings that say what, not how | Pass | Public modules and functions have docstrings with Doxygen tags |
 | 9 | Logging uses `logging`, not `print`; no secrets or personal data in log output | Pass | Output uses print only for the assignment's visible result in main; no secret is printed |
 | 10 | Classes and operations trace to the Design Class Diagram they implement; deviations are recorded | N-A | No Design Class Diagram exists; the milestone's technical tasks need none (Needs use case: No) |
 | 11 | Tests exist for new behaviour, are named for the behaviour, and do not depend on order or the network | Pass | Tests exist, are named for behaviour and use no network or real mail server: 15 pytest tests, ruff 0 findings, mypy strict 0 errors (17 files) |
 | 12 | Type checker runs in strict mode without errors; `Any` is justified in a comment | Pass | mypy --strict passes; no Any |
 | 13 | Dependencies are declared and pinned in the project's dependency file, none unused | Pass | Dependencies are in pyproject.toml with lower bounds; none unused |
 | 14 | Each piece of knowledge (a business rule, constant, format, validation or query) is defined in one place; code is merged only where it expresses the same knowledge, not where it merely looks alike | Pass | Constants live only in constants.py |
 | 15 | Imports point inward: business-rule modules import no framework, database, UI or delivery-mechanism code, and there are no import cycles | Pass | Imports point inward; no cycles |
 | 16 | SOLID holds: a class or module has one reason to change; new behaviour is added by extension, not by editing a type test in several places; subclasses honour the contract of their base; protocols are narrow; high-level code depends on protocols or abstract types and receives its implementations | Pass | Ports PageFetcher and ProductParser are narrow Protocols; the use case depends on them, the adapters implement them |
 | 17 | Source is laid out in the layers and folders of the clean architecture folder structure (`domain`, `application`, `infrastructure`, `presentation` packages under `src/`, and `unit`, `integration` and `e2e` under `tests/`), in `snake_case`, and imports between the layers point inward | Pass | domain, application and infrastructure packages exist; main.py is the composition root; tests mirror them under tests/unit/ |
 | 18 | File and folder names are English, contain no space and no character outside letters, digits, `-`, `_` and `.`, and follow the casing of the language for files | Pass | File names are English snake_case |
 ## Overall Verdict
 Go — all Mandatory criteria pass or are not applicable.
 ## Action Items
 | Action | Owner | Due |
 | --- | --- | --- |
 | None: no Fail and no Pending criterion | S01 | 2026-10-28 |
 ---
 [MIL-002]: ../../milestones/mil-002-scrape-price.md
 [QC-PY-001]: ../../../framework/qc/code/qc-programming-python.md
@@ -0,0 +1,5 @@
 """Entry point for python -m amazon_price_tracker."""
 from amazon_price_tracker.main import main
 main()
@@ -0,0 +1 @@
 """Application layer: use cases and the ports they need."""
@@ -0,0 +1,25 @@
 """Ports the use cases need from the outside.
@file ports.py
@brief Protocols for fetching a page and parsing a product from it.
 """
 from typing import Protocol
 from amazon_price_tracker.domain.product import Product
 class PageFetcher(Protocol):
    """Fetches the HTML of a page."""
    def fetch(self, url: str) -> str:
        """Return the HTML text of the page at url."""
        ...
 class ProductParser(Protocol):
    """Reads a product from HTML."""
    def parse(self, html: str) -> Product:
        """Return the product described by html."""
        ...
@@ -0,0 +1,20 @@
 """Use case: read a product from a page.
@file read_product.py
@brief Fetches a page and parses the product on it.
 """
 from amazon_price_tracker.application.ports import PageFetcher, ProductParser
 from amazon_price_tracker.domain.product import Product
 def read_product(fetcher: PageFetcher, parser: ProductParser, url: str) -> Product:
    """Fetch the page at url and return its product.
    @param fetcher Port that downloads the page.
    @param parser Port that reads the product from the HTML.
    @param url Address of the product page.
    @return The product on the page.
    @throws ProductParseError When the page holds no readable product.
    """
    return parser.parse(fetcher.fetch(url))
@@ -0,0 +1 @@
 """Domain layer: the product model and its errors."""
@@ -0,0 +1,17 @@
 """Errors of the domain.
@file errors.py
@brief Exceptions raised when a product cannot be read from a page.
 """
 class ProductParseError(Exception):
    """A product could not be read from the page."""
 class PriceNotFoundError(ProductParseError):
    """The page has no readable price."""
 class TitleNotFoundError(ProductParseError):
    """The page has no product title."""
@@ -0,0 +1,18 @@
 """The product model.
@file product.py
@brief Defines the Product value object.
 """
 from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
 class Product:
    """A product with its title and current price."""
    ## @brief Product title as shown on the page.
    title: str
    ## @brief Price as a float: the assignment asks for a floating point
    # number, so float is used although money is normally Decimal.
    price: float
@@ -0,0 +1 @@
 """Infrastructure layer: HTTP and HTML adapters."""
@@ -0,0 +1,24 @@
 """HTTP adapter for the PageFetcher port.
@file http_client.py
@brief Downloads a page with requests.
 """
 import requests
 from amazon_price_tracker.constants import REQUEST_TIMEOUT_SECONDS
 class RequestsPageFetcher:
    """PageFetcher port implemented with requests."""
    def fetch(self, url: str) -> str:
        """Return the HTML text of the page at url.
        @param url Address of the page.
        @return The response body as text.
        @throws requests.HTTPError When the server answers with an error status.
        """
        response = requests.get(url, timeout=REQUEST_TIMEOUT_SECONDS)
        response.raise_for_status()
        return response.text
@@ -0,0 +1,65 @@
 """Reads a product from page HTML with BeautifulSoup.
@file scraper.py
@brief Day 47 step 1: get the title and the price as a float.
 """
 from bs4 import BeautifulSoup
 from amazon_price_tracker.domain.errors import (
    PriceNotFoundError,
    ProductParseError,
    TitleNotFoundError,
 )
 from amazon_price_tracker.domain.product import Product
 def get_title(html: str) -> str:
    """Return the text of the element with id productTitle, whitespace collapsed.
    @param html Page HTML.
    @return The product title.
    @throws TitleNotFoundError When the element is missing.
    """
    element = BeautifulSoup(html, "html.parser").select_one("#productTitle")
    if element is None:
        raise TitleNotFoundError("No element with id 'productTitle' on the page")
    return " ".join(element.get_text().split())
 def get_price(html: str) -> float:
    """Return the price of the first element with class a-offscreen.
    @param html Page HTML.
    @return The price as a floating point number.
    @throws PriceNotFoundError When the element is missing.
    @throws ProductParseError When the price text is not a number.
    """
    element = BeautifulSoup(html, "html.parser").select_one(".a-offscreen")
    if element is None:
        raise PriceNotFoundError("No element with class 'a-offscreen' on the page")
    text = element.get_text().strip()
    # Text looks like "$99.99": drop the currency sign, as the hint suggests.
    amount = text.split("$")[-1].replace(",", "")
    try:
        return float(amount)
    except ValueError as error:
        raise ProductParseError(f"Price text is not a number: {text!r}") from error
 def parse_product(html: str) -> Product:
    """Return the product described by html.
    @param html Page HTML.
    @return The product with title and price.
    @throws ProductParseError When the title or the price cannot be read.
    """
    return Product(title=get_title(html), price=get_price(html))
 class SoupProductParser:
    """ProductParser port implemented with BeautifulSoup."""
    def parse(self, html: str) -> Product:
        """Return the product described by html."""
        return parse_product(html)
@@ -0,0 +1,17 @@
 """Composition root.
@file main.py
@brief Builds the adapters and prints the title and price of the product.
 """
 from amazon_price_tracker.application.read_product import read_product
 from amazon_price_tracker.constants import PRACTICE_URL
 from amazon_price_tracker.infrastructure.http_client import RequestsPageFetcher
 from amazon_price_tracker.infrastructure.scraper import SoupProductParser
 def main() -> None:
    """Print the title and the price of the practice product."""
    product = read_product(RequestsPageFetcher(), SoupProductParser(), PRACTICE_URL)
    print(product.title)
    print(product.price)
@@ -0,0 +1,3 @@
 <!DOCTYPE html>
 <html lang="en-us"><head><title>Empty</title></head>
 <body><span id="productTitle"> Some product </span></body></html>
@@ -0,0 +1,29 @@
 <!DOCTYPE html>
 <!-- Trimmed copy of https://appbrewery.github.io/instant_pot/ (markup kept, page reduced). -->
 <html lang="en-us">
 <head>
 <title>Instant Pot - Practice Page</title>
 </head>
 <body>
 <div id="centerCol">
 <h1 id="title" class="a-size-large a-spacing-none"> <span id="productTitle"
  class="a-size-large product-title-word-break"> Instant Pot Duo Plus 9-in-1
  Electric Pressure Cooker, Slow Cooker, Rice Cooker, Steamer, Sauté, Yogurt
  Maker, Warmer &amp; Sterilizer, Includes App With Over 800 Recipes, Stainless
  Steel, 3 Quart </span> </h1>
 </div>
 <div data-csa-c-type="widget" data-csa-c-slot-id="apex_dp_offer_display">
 <div class="a-section a-spacing-micro">
 <span class="a-price aok-align-center" data-a-size="xl" data-a-color="base"><span
 class="a-offscreen">$99.99</span><span aria-hidden="true"><span
 class="a-price-symbol">$</span><span class="a-price-whole">99<span
 class="a-price-decimal">.</span></span><span
 class="a-price-fraction">99</span></span></span>
 </div>
 </div>
 <div id="similarItems">
 <span class="a-price"><span class="a-offscreen">$16.95</span></span>
 <span class="a-price"><span class="a-offscreen">$116</span></span>
 </div>
 </body>
 </html>
@@ -0,0 +1,34 @@
 from amazon_price_tracker.application.read_product import read_product
 from amazon_price_tracker.domain.product import Product
 class FakeFetcher:
    def __init__(self, html: str) -> None:
        self.html = html
        self.urls: list[str] = []
    def fetch(self, url: str) -> str:
        self.urls.append(url)
        return self.html
 class FakeParser:
    def __init__(self, product: Product) -> None:
        self.product = product
        self.received: list[str] = []
    def parse(self, html: str) -> Product:
        self.received.append(html)
        return self.product
 def test_read_product_fetches_url_and_returns_parsed_product() -> None:
    expected = Product(title="Pot", price=9.5)
    fetcher = FakeFetcher("<html>page</html>")
    parser = FakeParser(expected)
    result = read_product(fetcher, parser, "https://example.test/p")
    assert result == expected
    assert fetcher.urls == ["https://example.test/p"]
    assert parser.received == ["<html>page</html>"]
@@ -0,0 +1,46 @@
 import pytest
 import requests
 from amazon_price_tracker.constants import REQUEST_TIMEOUT_SECONDS
 from amazon_price_tracker.infrastructure.http_client import RequestsPageFetcher
 class FakeResponse:
    def __init__(self, text: str, status_code: int) -> None:
        self.text = text
        self.status_code = status_code
    def raise_for_status(self) -> None:
        if self.status_code >= 400:
            raise requests.HTTPError(f"status {self.status_code}")
 def _patch_get(
    monkeypatch: pytest.MonkeyPatch, response: FakeResponse
 ) -> list[tuple[str, float]]:
    calls: list[tuple[str, float]] = []
    def fake_get(url: str, timeout: float) -> FakeResponse:
        calls.append((url, timeout))
        return response
    monkeypatch.setattr(requests, "get", fake_get)
    return calls
 def test_fetch_returns_text_using_url_and_timeout(
    monkeypatch: pytest.MonkeyPatch,
 ) -> None:
    calls = _patch_get(monkeypatch, FakeResponse("<p>hi</p>", 200))
    html = RequestsPageFetcher().fetch("https://example.test/")
    assert html == "<p>hi</p>"
    assert calls == [("https://example.test/", REQUEST_TIMEOUT_SECONDS)]
 def test_fetch_raises_on_http_error(monkeypatch: pytest.MonkeyPatch) -> None:
    _patch_get(monkeypatch, FakeResponse("", 404))
    with pytest.raises(requests.HTTPError):
        RequestsPageFetcher().fetch("https://example.test/")
@@ -0,0 +1,66 @@
 from pathlib import Path
 import pytest
 from amazon_price_tracker.domain.errors import (
    PriceNotFoundError,
    ProductParseError,
    TitleNotFoundError,
 )
 from amazon_price_tracker.domain.product import Product
 from amazon_price_tracker.infrastructure.scraper import (
    SoupProductParser,
    get_price,
    get_title,
    parse_product,
 )
 FIXTURES = Path(__file__).resolve().parents[2] / "fixtures"
 def _read(name: str) -> str:
    return (FIXTURES / name).read_text(encoding="utf-8")
 def test_get_price_returns_first_offscreen_as_float() -> None:
    price = get_price(_read("practice_page.html"))
    assert price == 99.99
    assert isinstance(price, float)
 def test_get_title_returns_stripped_text() -> None:
    title = get_title(_read("practice_page.html"))
    assert title == title.strip()
    assert title.startswith("Instant Pot Duo Plus")
 def test_get_price_raises_when_price_missing() -> None:
    with pytest.raises(PriceNotFoundError):
        get_price(_read("no_price_page.html"))
 def test_get_title_raises_when_title_missing() -> None:
    with pytest.raises(TitleNotFoundError):
        get_title("<html></html>")
 def test_get_price_raises_when_text_not_a_number() -> None:
    html = '<span class="a-offscreen">$abc</span>'
    with pytest.raises(ProductParseError):
        get_price(html)
 def test_parse_product_returns_product() -> None:
    product = parse_product(_read("practice_page.html"))
    assert isinstance(product, Product)
    assert product.price == 99.99
 def test_soup_product_parser_parses_html() -> None:
    product = SoupProductParser().parse(_read("practice_page.html"))
    assert product.price == 99.99
		`@@ -0,0 +1 @@`
							`"""Application layer: use cases and the ports they need."""`
		`@@ -0,0 +1 @@`
							`"""Domain layer: the product model and its errors."""`
		`@@ -0,0 +1 @@`
							`"""Infrastructure layer: HTTP and HTML adapters."""`