Methodology

Raw pageviews have two problems as a measure of cultural interest. First, total Wikipedia traffic has drifted since 2015, so a flat view count can hide a real rise or fall in interest. Second, topics live on very different scales: a famous athlete draws far more views than a discontinued toy. The index fixes both problems by normalizing twice.

How is the index calculated?

Each article goes through the three steps shown below.

%%{init: {'theme': 'base', 'themeVariables': { 'fontFamily': 'monospace', "darkMode":true}}}%%
flowchart LR
    V[/"article views"/]
    T[/"total project views"/]
    P["views per million"]
    S["28 day rolling median"]
    B{{"baseline median<br>first 12 months"}}
    X["nostalgia_index"]
    R("revival events<br>raw index > 3x trailing 90 day median")
    V --> P
    T --> P
    P --> S
    S --> X
    B --> X
    P -.-> R

style V fill:#ffbe00,color:#1e1c21,rx:5,ry:5
style T fill:#ffbe00,color:#1e1c21,rx:5,ry:5
style P fill:#ffbe00,color:#1e1c21,rx:5,ry:5
style S fill:#ffbe00,color:#1e1c21,rx:5,ry:5
style B fill:#ffbe00,color:#1e1c21,rx:5,ry:5
style X fill:#f4640d,color:#1e1c21,rx:5,ry:5
style R fill:#ffbe00,color:#1e1c21,rx:5,ry:5

views_per_million = article_views / total_project_views x 1e6
smoothed          = rolling 28 day median
nostalgia_index   = smoothed / baseline_median x 100

The first step divides an article’s daily views by total project views that day (English Wikipedia by default), which removes drift in overall traffic. The second step smooths the series with a rolling median, which is less sensitive to a single viral day than a mean. The third step divides by the baseline median and multiplies by 100, so every topic starts on the same scale.

The baseline is the median of the smoothed series over its first 12 months. An index of 100 means a topic draws the same share of Wikipedia traffic as it did during its baseline period; an index of 200 means it draws twice that share.

What counts as a revival?

A revival event is a day where the unsmoothed index exceeds 3x its trailing 90 day median. The unsmoothed index matters here, because the rolling median would flatten the very spikes we’re trying to detect.

All three thresholds are arguments to NostalgiaIndex(): window (28 days), baseline_months (12), and spike_factor (3.0).

Caveats

Keep a few limitations in mind when reading the index. The Wikimedia pageviews data starts on 2015-07-01, so nothing earlier is available. Only agent=user traffic is used, which excludes spiders. Redirects are resolved through the MediaWiki API, so a misspelled title like Tamagochi is counted as Tamagotchi.

Warning

Topics without an article in 2015 have no true baseline. Their index is relative to their own first year of data, not to a shared starting point.

Worked example

The example below runs the index on the sample data in fixtures/, so it doesn’t call the API. summary() returns one row per topic with its current index, peak value and date, change versus baseline, and revival count.

import pandas as pd
from nostalgia_index import NostalgiaIndex

views = pd.read_csv("fixtures/views.csv", parse_dates=["date"])
totals = pd.read_csv("fixtures/totals.csv", parse_dates=["date"])
ni = NostalgiaIndex()
idx = ni.compute(views, totals)
ni.summary(idx).round(1)

/tmp/ipykernel_36375/1864511853.py:8: UserWarning: obj.round has no effect with datetime, timedelta, or period dtypes. Use obj.dt.round(...) instead.
  ni.summary(idx).round(1)

	article	current	peak	peak_date	pct_vs_baseline	revivals
0	Furby	82.0	107.0	2015-11-21	-18.0	1
1	Tamagotchi	182.8	189.4	2019-12-23	82.8	0

revivals() lists each revival event with its unsmoothed index, the trailing median it was compared against, and the resulting multiple.

ni.revivals(idx).head()

	date	article	raw_index	trailing_median	multiple
0	2017-12-17	Furby	632.203184	94.416844	6.695873

Recap

The nostalgia index divides pageviews by total Wikipedia traffic, smooths them with a rolling median, and scales each topic to 100 at its baseline. Revival events flag days where the unsmoothed index jumps above three times its recent median. Read results for topics created after 2015 with care, since their baseline is their first year rather than 2015.