MIL-002: parsing HTML and making soup #36
@@ -56,12 +56,54 @@ Leave out `[dev]` to install only the runtime dependencies. Personal tokens belo
|
||||
|
||||
## ▶️ Run
|
||||
|
||||
The lecture steps are added milestone by milestone; each gets a section with its command, its output and a screenshot here.
|
||||
Every lecture step is a command of the form `python -m scrape_movies <lecture> <step>`; without arguments the help is printed. Each step below shows the command, what it teaches and a screenshot of its output.
|
||||
|
||||
### Lecture: Parsing HTML and making soup
|
||||
|
||||
The sample page is `tests/fixtures/website.html`; the code is `src/scrape_movies/infrastructure/soup_basics.py`.
|
||||
|
||||
**1. Create a soup and read the title** (`BeautifulSoup(html, "html.parser")`, `soup.title.string`)
|
||||
|
||||
```bash
|
||||
python -m scrape_movies # available from the movies milestone
|
||||
python -m scrape_movies soup title
|
||||
```
|
||||
|
||||

|
||||
|
||||
**2. Prettify** (`soup.prettify()` indents one tag per line)
|
||||
|
||||
```bash
|
||||
python -m scrape_movies soup prettify
|
||||
```
|
||||
|
||||

|
||||
|
||||
**3. Find all tags** (`find_all("a")`, then `get_text()` and `get("href")` in a loop)
|
||||
|
||||
```bash
|
||||
python -m scrape_movies soup find-all
|
||||
```
|
||||
|
||||

|
||||
|
||||
**4. Find by id and by class** (`find(id=...)`, `find_all("h3", class_=...)`; `class_` because `class` is a Python keyword)
|
||||
|
||||
```bash
|
||||
python -m scrape_movies soup find
|
||||
```
|
||||
|
||||

|
||||
|
||||
**5. CSS selectors** (`select_one("p a")` and `select(".heading")`)
|
||||
|
||||
```bash
|
||||
python -m scrape_movies soup select
|
||||
```
|
||||
|
||||

|
||||
|
||||
The screenshots are made by `python tools/make_screenshots.py`, which renders the command output as an image; no desktop is captured.
|
||||
|
||||
## 🧪 Tests
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user