Skip to main content

Table sources

The Wikipedia pages below already store their data in a clean HTML table, which makes them easier to scrape with rvest::html_table() or pandas.read_html() (without any text parsing).

%%{init: {'theme': 'default', 'themeVariables': { 'fontFamily': 'monospace', "fontSize":"16px", "darkMode":false}}}%%

flowchart TB
    wiki(["<strong>Wikipedia page</strong><br/>HTML table"]) -->|"i. Request"| scrape["<code>httr2</code>/<code>rvest</code><br/>or <code>requests</code>/<code>pandas</code>"]
    scrape -->|"ii. Save raw"| csv[("<code>data/</code><br><code>*.csv</code>")]
    csv -->|"iii. Clean"| clean["<code>dplyr</code> or <code>pandas</code> clean + reshape"]
    clean -->|"iv. Save clean"| cleancsv[("<code>data/</code><br><code>*_clean.csv</code>")]

    style wiki fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
    style scrape fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style csv fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
    style clean fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style cleancsv fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px

Scraped topics