%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': 'monospace', "fontSize":"16px", "darkMode":false}}}%%
flowchart TB
raw[("<code>data/defunct_malls.csv</code>")] -->|"i. Order"| status["<code>status</code> factor<br/>Demolished, Abandoned, Defunct"]
status -->|"ii. Sort"| arr["Most specific<br/>status first"]
arr -->|"iii. Deduplicate"| dist["One row per <code>title</code>"]
dist -->|"iv. Save"| clean[("<code>data/defunct_malls_clean.csv</code>")]
style raw fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
style clean fill:#FFFFFF,color:#000000,stroke:#333,stroke-width:1px
style status fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
style arr fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
style dist fill:#4CBB9D,color:#FFFFFF,rx:5,ry:5
Clean: malls
A mall in the “Demolished” or “Abandoned” subcategory from scrape_malls.qmd is also a member of the parent “Defunct” category, so each title can appear more than once. The code below keeps the most specific status for each mall and counts how many malls fall into each one.
library(dplyr)
library(readr)
malls_clean <- read_csv("data/defunct_malls.csv") |>
mutate(status = factor(status, levels = c("Demolished", "Abandoned", "Defunct"))) |>
arrange(status) |>
distinct(title, .keep_all = TRUE)
write_csv(malls_clean, "data/defunct_malls_clean.csv")
malls_clean |> count(status, sort = TRUE)# A tibble: 3 × 2
status n
<fct> <int>
1 Demolished 416
2 Defunct 278
3 Abandoned 25
import pandas as pd
malls = pd.read_csv("data/defunct_malls.csv")
malls["status"] = pd.Categorical(
malls["status"], categories=["Demolished", "Abandoned", "Defunct"], ordered=True
)
malls_clean = (
malls.sort_values("status", kind="stable")
.drop_duplicates(subset="title")
)
malls_clean.to_csv("data/defunct_malls_clean.csv", index=False)
malls_clean["status"].value_counts()status
Demolished 416
Defunct 278
Abandoned 25
Name: count, dtype: int64
NoteEither works
An ordered factor in R and an ordered Categorical in pandas both sort the statuses by specificity. arrange() is always stable; in pandas, sort_values() needs kind="stable" so ties keep their original order before drop_duplicates().
See viz_malls.qmd for the chart built on this.