library(httr2)
library(rvest)
library(readr)
html <- request("https://en.wikipedia.org/wiki/Comparison_of_online_dating_services") |>
req_user_agent("wikimill-project/0.1") |>
req_perform() |>
resp_body_html()
tables <- html_table(html)
dating <- Filter(\(t) any(grepl("Registered users", names(t))), tables)[[1]]
names(dating) <- c(
"name", "description", "owner", "registered_users", "free",
"all_features_free", "hetero", "poly", "free_messaging", "std_notification"
)
write_csv(dating, "data/dating_services.csv")
head(dating)Scrape: dating services
The Comparison of online dating services table mixes a registered-user count with a reporting year in one free-text cell, and the “Free” and “All features free” columns hold full sentences rather than a plain Yes/No. We save the raw columns as-is here; the parsing happens in clean_dating.qmd.
In R, httr2 fetches the page, rvest::html_table() parses every table on it, and Filter() keeps the one with “Registered users” in its header.
In Python, requests fetches the page, then pandas.read_html() finds every table and match= picks out the same one.
import io
import pandas as pd
import requests
dating_html = requests.get(
"https://en.wikipedia.org/wiki/Comparison_of_online_dating_services",
headers={"User-Agent": "wikimill-project/0.1"}
).text
tables = pd.read_html(io.StringIO(dating_html), match="Registered users")
dating = tables[0]
dating.columns = [
"name", "description", "owner", "registered_users", "free",
"all_features_free", "hetero", "poly", "free_messaging", "std_notification"
]
dating.to_csv("data/dating_services.csv", index=False)
dating.head()Both versions take about the same code and keep the same table:
rvest::html_table()returns every table on the page, andFilter()keeps the one whose header mentions “Registered users”.pandas.read_html()does the same filtering in one step withmatch=.
The raw text stays as-is in both CSVs, so the cleaning step parses the same strings either way.
The code above is shown for reference only. Run Rscript src/scrape_dating.R or python src/scrape_dating.py from the project root to refresh data/dating_services.csv.