Skip to main content

Clean: minimum wage

The minimum wage CSV from scrape_minimum_wage.qmd has more columns than the chart needs. The code below keeps the country and nominal annual wage columns and drops rows with no wage on record.

%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': 'monospace', "fontSize":"16px", "darkMode":false}}}%%

flowchart TB

    raw[("<code>data/minimum_wage.csv</code>")] -->|"i. Select"| sel["Country and<br/>nominal annual wage"]
    sel -->|"ii. Rename"| ren["<code>country</code>, <code>annual_wage_usd</code>"]
    ren -->|"iii. Filter"| filt["Drop rows<br/>with no wage"]
    filt -->|"iv. Save"| clean[("<code>data/minimum_wage_clean.csv</code>")]

    style raw fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
    style clean fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
    style sel fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style ren fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
    style filt fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5

library(dplyr)
library(readr)

wage_clean <- read_csv("data/minimum_wage.csv") |>
  select(matches("Country"), matches("Annual_Nominal")) |>
  rename(country = 1, annual_wage_usd = 2) |>
  mutate(annual_wage_usd = as.double(annual_wage_usd)) |>
  filter(!is.na(annual_wage_usd))
write_csv(wage_clean, "data/minimum_wage_clean.csv")
wage_clean
## # A tibble: 157 × 2
##    country             annual_wage_usd
##    <chr>                         <dbl>
##  1 Afghanistan                     858
##  2 Albania                        4637
##  3 Algeria                        2132
##  4 Andorra                       18253
##  5 Angola                          663
##  6 Antigua and Barbuda            6317
##  7 Armenia                        1787
##  8 Australia                     35810
##  9 Azerbaijan                     2435
## 10 The Bahamas                   10920
## # ℹ 147 more rows
import pandas as pd

wage = pd.read_csv("data/minimum_wage.csv")
wage_clean = wage.filter(regex="Country|Annual_Nominal").iloc[:, :2]
wage_clean.columns = ["country", "annual_wage_usd"]
wage_clean["annual_wage_usd"] = pd.to_numeric(wage_clean["annual_wage_usd"], errors="coerce")
wage_clean = wage_clean.dropna(subset=["annual_wage_usd"])
wage_clean.to_csv("data/minimum_wage_clean.csv", index=False)
wage_clean.head()
       country  annual_wage_usd
0  Afghanistan            858.0
1      Albania           4637.0
2      Algeria           2132.0
3      Andorra          18253.0
4       Angola            663.0
NoteEither works

matches() and filter(regex=) both pick columns by pattern. matches() ignores case by default, while filter(regex=) doesn’t, which doesn’t matter for these column names.

See viz_minimum_wage.qmd for the chart built on this.