Skip to main content

Scrape: dating services

The Comparison of online dating services table mixes a registered-user count with a reporting year in one free-text cell, and the “Free” and “All features free” columns hold full sentences rather than a plain Yes/No. We save the raw columns as-is here; the parsing happens in clean_dating.qmd.

In R, httr2 fetches the page, rvest::html_table() parses every table on it, and Filter() keeps the one with “Registered users” in its header.

library(httr2)
library(rvest)
library(readr)

html <- request("https://en.wikipedia.org/wiki/Comparison_of_online_dating_services") |>
  req_user_agent("wikimill-project/0.1") |>
  req_perform() |>
  resp_body_html()

tables <- html_table(html)
dating <- Filter(\(t) any(grepl("Registered users", names(t))), tables)[[1]]
names(dating) <- c(
  "name", "description", "owner", "registered_users", "free",
  "all_features_free", "hetero", "poly", "free_messaging", "std_notification"
)
write_csv(dating, "data/dating_services.csv")
head(dating)

In Python, requests fetches the page, then pandas.read_html() finds every table and match= picks out the same one.

import io
import pandas as pd
import requests

dating_html = requests.get(
    "https://en.wikipedia.org/wiki/Comparison_of_online_dating_services",
    headers={"User-Agent": "wikimill-project/0.1"}
).text
tables = pd.read_html(io.StringIO(dating_html), match="Registered users")
dating = tables[0]
dating.columns = [
    "name", "description", "owner", "registered_users", "free",
    "all_features_free", "hetero", "poly", "free_messaging", "std_notification"
]
dating.to_csv("data/dating_services.csv", index=False)
dating.head()
NoteWhich language should I use?

Both versions take about the same code and keep the same table:

  • rvest::html_table() returns every table on the page, and Filter() keeps the one whose header mentions “Registered users”.

  • pandas.read_html() does the same filtering in one step with match=.

The raw text stays as-is in both CSVs, so the cleaning step parses the same strings either way.

The code above is shown for reference only. Run Rscript src/scrape_dating.R or python src/scrape_dating.py from the project root to refresh data/dating_services.csv.