Tirsvad-Udemy-100-days-of-code/045-scrape-movies
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Page: Gains
Pages
Gains da Gains ms Gains Hacker News da Hacker News ms Hacker News Home da Home ms Home Parsing HTML da Parsing HTML ms Parsing HTML Scraping Ethics da Scraping Ethics ms Scraping Ethics Setup da Setup ms Setup Top 100 Movies da Top 100 Movies ms Top 100 Movies
Clone
1
Gains
Tirsvad edited this page 2026-10-28 07:40:34 +01:00
Table of Contents

English · Dansk · Bahasa Melayu

What we gained

Lecture What we can do now
Parsing HTML Make a soup, read its structure and select elements by tag, id, class or CSS selector
Hacker News Fetch a live page, read titles, links and scores, convert text to numbers, find the maximum
Scraping ethics Check robots.txt, respect copyright and logins, keep requests rare
Top 100 movies Scrape a ranked list, reverse it and write it to a file
Setup Work in a virtual environment with pyproject.toml and offline tests

Beyond the lectures

  • Parsing is separate from fetching. The parsing functions take text, so the tests use saved pages and never need the network.
  • Layers keep rules apart. The ranking rule has no knowledge of BeautifulSoup or Requests.
  • Screenshots are made by a script (tools/make_screenshots.py), so they can be regenerated after a change.

English · Dansk · Bahasa Melayu

Dansk

Bahasa Melayu

scrape-movies · Day 45 of 100 Days of Code · Repository · English · Dansk · Bahasa Melayu