Skip to main content

Clean: dating services

In the CSV from scrape_dating.qmd, the registered-user count and the reporting year are both embedded in one string, like "30,000,000 as of 2006[2]". The user count always has thousands separators, so matching a comma-grouped number pulls it out first; whatever’s left over is where the year lives. The “Free” and “All features free” columns are full sentences; we keep the first word (Yes/No/Maybe) as a clean category and leave the original text in place.

%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': 'monospace', "fontSize":"16px", "darkMode":false}}}%%

flowchart TB

    raw[("<code>data/dating_services.csv</code>")] -->|"i. Extract"| users["User count<br/>from <code>registered_users</code>"]
    users -->|"ii. Extract"| year["Year from<br/>the leftover text"]
    year -->|"iii. Categorize"| free["Yes/No/Maybe from<br/><code>free</code> columns"]
    free -->|"iv. Filter"| filt["Drop missing counts<br/>and duplicate names"]
    filt -->|"v. Save"| clean[("<code>data/dating_services_clean.csv</code>")]

    style raw fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
    style clean fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
    style users fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style year fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style free fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style filt fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5

library(dplyr)
library(readr)
library(stringr)

dating_clean <- read_csv("data/dating_services.csv") |>
  mutate(
    users_raw = str_extract(registered_users, "\\d{1,3}(?:,\\d{3})+"),
    registered_users_n = as.numeric(str_remove_all(users_raw, ",")),
    users_year = as.integer(str_extract(str_remove(registered_users, fixed(users_raw)), "\\d{4}")),
    free_clean = str_extract(free, regex("^(Yes|No|Maybe)", ignore_case = TRUE)),
    all_features_free_clean = str_extract(all_features_free, regex("^(Yes|No|Maybe)", ignore_case = TRUE))
  ) |>
  select(-users_raw) |>
  filter(!is.na(registered_users_n)) |>
  distinct(name, .keep_all = TRUE)
write_csv(dating_clean, "data/dating_services_clean.csv")
dating_clean
## # A tibble: 26 × 14
##    name  description owner registered_users free  all_features_free hetero poly 
##    <chr> <chr>       <chr> <chr>            <chr> <chr>             <chr>  <chr>
##  1 Adul… Members ca… <NA>  30,000,000 as o… No    "No, free users … Yes    Yes  
##  2 Anas… Connects W… <NA>  4,000,000 as of… No    "No"              Yes    ?    
##  3 Ashl… Members ar… <NA>  60,000,000 as o… Yes   "No; Free users … Yes    Yes  
##  4 Badoo Available … Bumb… 500,000,000 as … Yes   "No; Charge for … Yes    ?    
##  5 Beau… Dating web… <NA>  800,000 as of 2… No    "No; free profil… Yes    ?    
##  6 Bumb… Dating app… Bumb… 22,000,000 as o… Yes   "No"              Yes    ?    
##  7 Chyr… Dating app… Gree… 1,000,000 downl… Yes   "No; Free users … Yes    No   
##  8 Chri… Dating sit… Spar… 12,000,000 as o… No    "No"              Yes    ?    
##  9 Date… Dedicated … <NA>  179,000 as of 2… No; … "Yes"             ?      Part…
## 10 Duol… Open-sourc… Inde… as of 2026 15,0… Yes   "No; optional \"… Yes    Unkn…
## # ℹ 16 more rows
## # ℹ 6 more variables: free_messaging <chr>, std_notification <chr>,
## #   registered_users_n <dbl>, users_year <int>, free_clean <chr>,
## #   all_features_free_clean <chr>
import pandas as pd

dating = pd.read_csv("data/dating_services.csv")
users_raw = dating["registered_users"].str.extract(r"(\d{1,3}(?:,\d{3})+)")[0]
dating["registered_users_n"] = pd.to_numeric(users_raw.str.replace(",", ""))
leftover = pd.Series(
    [r.replace(u, "", 1) if pd.notna(u) else None for r, u in zip(dating["registered_users"], users_raw)],
    index=dating.index, dtype="object"
)
dating["users_year"] = pd.to_numeric(leftover.str.extract(r"(\d{4})")[0]).astype("Int64")
dating["free_clean"] = dating["free"].str.extract(r"^(Yes|No|Maybe)", flags=2)[0]
dating["all_features_free_clean"] = dating["all_features_free"].str.extract(r"^(Yes|No|Maybe)", flags=2)[0]

dating_clean = (
    dating.dropna(subset=["registered_users_n"])
    .drop_duplicates(subset="name")
)
dating_clean.to_csv("data/dating_services_clean.csv", index=False)
dating_clean.head()
                 name  ... all_features_free_clean
1  Adult FriendFinder  ...                      No
2       AnastasiaDate  ...                      No
3      Ashley Madison  ...                      No
4               Badoo  ...                      No
5     BeautifulPeople  ...                      No

[5 rows x 14 columns]
Tipstringr has a slight edge

Finding the year means removing each row’s own user count from its text first. str_remove() is vectorized over the pattern, so one call handles every row. The .str methods in pandas take a single pattern for the whole column, so the Python version loops over the rows with zip().

See viz_dating.qmd for the chart built on this.