renv::init()Lab 1: Environments as Code
This lab shows how to create isolated, reproducible package environments with renv (R) and venv (Python), which every lab that follows depends on. The first lab includes two Quarto documents:
_labs/lab01/
├── eda-basic.qmd
└── model-basic-py.qmdProject-oriented workflows
Key takeaways:
Each project should have an isolated package environment that preserves the names and versions.
The project’s dependencies/requirements should be captured in a way that makes it exportable/transferable to another machine.1
Each project should have a virtual environment in a standalone directory that documents everything the project needs (usually in a GitHub or GitLab repo).2
Virtual environments
As do4ds notes, venv and renv “keep user-level package caches of the actual packages and use symlinks, so only one copy of the package is installed.”
renv
First, read the documentation on project level management with renv and using renv with Quarto.
I’ve summarized the steps below:
Create a new R project-local environment
install.packages("renv") renv::init()Install packages into the renv-managed environment
install.packages(c("tidyverse", "shiny", "ggplot2"))Snapshot the state of the environment and package versions in the lock file (
renv.lock)renv::snapshot()To deactivate the environment, restart R or close the session
renvprojects are automatically reactivated the time a project is opened. Restore the environment on a new system with:renv::restore()
To demonstrate how to use renv, I’ve copied the eda-basic.qmd file into it’s own ‘project’ folder (renv-eda-basic):
_labs/lab01/renv-eda-basic/
└── eda-basic.qmdThen I ran renv::init():
Initiating renv in a project root directory will create the .Rprofile and renv.lock files and a renv/ folder:
_labs/lab01/renv-eda-basic/
├── .Rprofile
├── eda-basic.qmd
├── renv/
├── renv-eda-basic.Rproj
└── renv.lockrenv::init() will also create renv-eda-basic.Rproj, which helps enforce project-oriented workflows (but these are specific to RStudio).
To install the packages, we can use install.packages() or pak::pak() for more flexible installations:
install.packages("pak")
pak::pak(c("tidyverse", "shiny", "ggplot2"))Create a snapshot (i.e., the current versions of R and any installed packages, along with the R repositories) with renv::snapshot():
renv::snapshot()The dependencies are saved in the renv.lock file, which should be committed to version control.
To make sure the dependencies in the renv.lock file have been installed (i.e., after cloning), run:
renv::restore()The diagram below maps how packages flow between CRAN, the renv cache, the project library, and the lockfile:
%%{init: {'theme': 'base', 'look': 'classic', 'themeVariables': {'fontFamily': 'monospace'}}}%%
flowchart TD
%% External sources
CRAN("CRAN/<br>GitHub/...")
%% Local components
SysLib("system<br>library")
ProjLib("project<br>library")
RenvCache[("renv<br>cache")]
Lockfile[("lockfile")]
%% Subgraph for "your computer" vs "the internet"
subgraph Internet ["<strong>Internet</strong>"]
CRAN
end
subgraph Computer ["<strong>Computer</strong>"]
SysLib
ProjLib
RenvCache
Lockfile
end
%% Connections with function labels
SysLib -->|"<strong>init()<strong>"| ProjLib
CRAN -->|"<strong>install()</strong><br><strong>update()</strong>"| RenvCache
RenvCache -->|"renv"| ProjLib
ProjLib <-->|"<strong>status()</strong>"| Lockfile
ProjLib -->|"<strong>snapshot()</strong>"| Lockfile
Lockfile -->|"<strong>restore()</strong>"| ProjLib
%% styling
%% classDef default fill:#F2E6C2,stroke:#2A6F77,color:#1B2A41
style CRAN fill:#5B8C5A,stroke:#000000,stroke-width:1px,color:#ffffff
style SysLib fill:#2A6F77,stroke:#000000,stroke-width:1px,color:#ffffff
style RenvCache fill:#D2562B,stroke:#000000,stroke-width:1px,color:#ffffff
style ProjLib fill:#2A6F77,stroke:#000000,stroke-width:1px,color:#ffffff
style Lockfile fill:#485466,stroke:#000000,stroke-width:1px,color:#ffffff
style Internet fill:#fbf7ec,stroke:#5B8C5A,color:#1B2A41
style Computer fill:#fbf7ec,stroke:#2A6F77,color:#1B2A41
venv
Read the documentation on Python virtual environments and using venv with Quarto.
I’ve summarized the steps below:
Create new Python 3 virtual environment in the
.venvdirectorypython3 -m venv .venvActivate the environment
source .venv/bin/activateUse
pipto install packages into your environmentpython3 -m pip install jupyter matplotlib pandasDeactivate an environment
deactivateSave environment
python3 -m pip freeze > requirements.txtRestoring environment
python3 -m pip install -r requirements.txt
Use the Terminal to navigate to the venv-model-basic-py folder and create the Python virtual environment:
python3 -m venv .venvThis will create a .venv/ folder inside venv-model-basic-py.
_labs/lab01/venv-model-basic-py/
├── .venv
│ ├── bin
│ ├── include
│ ├── lib
│ └── pyvenv.cfg
└── model-basic-py.qmdTo activate the virtual environment, run the following in the Terminal:
source .venv/bin/activateThis should change the Terminal prompt from:
Bash@MacBook venv-model-basic-py$ to:
(.venv) Bash@MacBook venv-model-basic-py$Now we can install required Python packages (we include jupyter for interactive outputs)
pip install palmerpenguins pandas numpy scikit-learn jupyterTo store the installed packages, save them to a requirements.txt file with pip freeze:
python3 -m pip freeze > requirements.txtThe diagram below maps how packages flow between PyPI, the pip cache, the virtual environment, and requirements.txt:
%%{init: {'theme': 'base', 'look': 'classic', 'themeVariables': {'fontFamily': 'monospace'}}}%%
flowchart TD
%% External sources
PyPI("PyPI/<br>GitHub/...")
%% Local components
SysPython("system<br>Python")
VirtEnv("virtual<br>environment")
PipCache[("pip<br>cache")]
Requirements[("requirements.txt")]
%% Subgraph for "your computer" vs "the internet"
subgraph Internet ["<strong>Internet</strong>"]
PyPI
end
subgraph Computer ["<strong>Computer</strong>"]
SysPython
VirtEnv
PipCache
Requirements
end
%% Connections with function labels
SysPython -->|"<strong>python -m venv</strong>"| VirtEnv
PyPI -->|"<strong>pip install</strong><br><strong>pip upgrade</strong>"| PipCache
PipCache -->|"pip"| VirtEnv
VirtEnv <--->|"<strong>pip list</strong><br><strong>pip show</strong>"| Requirements
VirtEnv -->|"<strong>pip freeze</strong>"| Requirements
Requirements -->|"<strong>pip install -r</strong>"| VirtEnv
%% Additional workflow arrows
VirtEnv -.->|"<strong>source activate</strong><br><strong>deactivate</strong>"| VirtEnv
%% Styling
style PyPI fill:#5B8C5A,stroke:#000000,stroke-width:1px,color:#ffffff
style SysPython fill:#5B8C5A,stroke:#000000,stroke-width:1px,color:#ffffff
style VirtEnv fill:#2A6F77,stroke:#000000,stroke-width:1px,color:#ffffff
style PipCache fill:#D2562B,stroke:#000000,stroke-width:1px,color:#ffffff
style Requirements fill:#485466,stroke:#000000,stroke-width:1px,color:#ffffff
style Internet fill:#fbf7ec,stroke:#5B8C5A,color:#1B2A41
style Computer fill:#fbf7ec,stroke:#2A6F77,color:#1B2A41
Rendering with Python
To render, use the following in the Terminal:
quarto preview model-basic-py.qmd --no-browser --no-watch-inputsTo tell Quarto to use the virtual environment, we can place the following in the model-basic-py.qmd file:
execute:
python: .venv/bin/pythonOr we can set the Python interpreter with the Terminal:3
quarto use python .venv/bin/python3If you’re working inside RStudio or Posit Workbench, consider letting reticulate manage the Python environment:
library(reticulate)
use_virtualenv("myenv", required = TRUE)
virtualenv_install(
envname = "myenv",
packages = c("palmerpenguins", "pandas", "numpy",
"scikit-learn", "jupyter"))This ensures RStudio uses a known-good Python environment.
Now, if we open the model-basic-py.qmd file in RStudio and click the Render button in the toolbar we should see the rendered output.
Recap
Each project gets its own isolated, transferable package environment. In R,
renv::init()creates the project library andrenv.lock,renv::snapshot()records the package versions, andrenv::restore()reinstalls them on another machine.In Python,
python3 -m venv .venvcreates the environment andsource .venv/bin/activateturns it on.pip freeze > requirements.txtcaptures the installed packages, andpip install -r requirements.txtrestores them.Quarto has to be told which Python interpreter to use. We can add
python: .venv/bin/pythonunderexecute:in the document’s YAML, runquarto use python .venv/bin/python3in the Terminal, or letreticulatemanage the environment inside RStudio.
Next, Modeling in Python walks through the code in model-basic-py.qmd.