Lab 1: Environments as Code

The first lab includes two Quarto documents:

_labs/lab1/
├── eda-basic.qmd
└── model-basic-py.qmd

Project-oriented workflows

Key takeaways:

1. Each project should have an isolated package environment that preserves the names and versions.

2. The project’s dependencies/requirements should be captured in a way that makes it exportable/transferable to another machine.¹

3. Each project should have a virtual environment in a standalone directory that documents everything the project needs (usually in a GitHub or GitLab repo).²

Virtual environments

venv and renv, “keep user-level package caches of the actual packages and use symlinks, so only one copy of the package is installed.”

renv

First, read the documentation on project level management with renv and using renv with Quarto.

I’ve summarized the steps below:

# create a new R project-local environment
install.packages("renv")
renv::init()

# install packages into the renv-managed environment
install.packages(c("tidyverse", "shiny", "ggplot2"))

# snapshot the state of the environment and package versions in 
# the lock file (renv.lock)
renv::snapshot()

# to deactivate the environment, restart R or close the session
# renv projects are automatically reactivated the time a project
# is opened

# restore the environment on a new system with
renv::restore()

To demonstrate how to use renv, I’ve copied the eda-basic.qmd file into it’s own ‘project’ folder (renv-eda-basic):

_labs/lab1/renv-eda-basic/
└── eda-basic.qmd

Then I ran renv::init():

renv::init()

Initiating renv in a project root directory will create the .Rprofile and renv.lock files and a renv/ folder:

_labs/lab1/renv-eda-basic/
├── .Rprofile
├── eda-basic.qmd
├── renv/
├── renv-eda-basic.Rproj
└── renv.lock

renv::init() will also create renv-eda-basic.Rproj, which helps enforce project-oriented workflows (but these are specific to RStudio).

To install the packages, we can use install.packages() or pak::pak() for more flexible installations:

install.packages("pak")
pak::pak(c("tidyverse", "shiny", "ggplot2"))

NoteNote

renv::install() will install cached packages if they are available if not, it will install the latest version from CRAN.

Create a snapshot (i.e., the current versions of R and any installed packages, along with the R repositories) with renv::snapshot():

renv::snapshot()

The dependencies are saved in the renv.lock file, which should be committed to version control.

To make sure the dependencies in the renv.lock file have been installed (i.e., after cloning), run:

renv::restore()

%%{init: {'theme': 'neutral', 'look': 'handDrawn', 'themeVariables': { 'fontFamily': 'monospace', "fontSize":"16px"}}}%%

flowchart TD
    %% External sources
    CRAN["CRAN/<br/>GitHub/..."]
    
    %% Local components
    SysLib["system<br/>library"]
    ProjLib["project<br/>library"]
    RenvCache["renv<br/>cache"]
    Lockfile["lockfile"]
    
    %% Subgraph for "your computer" vs "the internet"
    subgraph Internet ["<strong>Internet</strong>"]
        CRAN
    end
    
    subgraph Computer ["<strong>Computer</strong>"]
        SysLib
        ProjLib
        RenvCache
        Lockfile
    end
    
    %% Connections with function labels
    SysLib -->|"<strong>init()<strong>"| ProjLib
    
    CRAN -->|"<strong>install()</strong><br/><strong>update()</strong>"| RenvCache
    RenvCache -->|"renv"| ProjLib
    
    ProjLib <-->|"<strong>status()</strong>"| Lockfile
    ProjLib -->|"<strong>snapshot()</strong>"| Lockfile
    Lockfile -->|"<strong>restore()</strong>"| ProjLib
    
    %% Styling
    style Internet fill:#d8e4ff
    style Computer fill:#31e981
    
    

renv

venv

Read the documentation on Python virtual environments and using venv with Quarto.

I’ve summarized the steps below:

# create new Python 3 virtual environment in the '.venv' directory 
python3 -m venv .venv
# activate the environment
source .venv/bin/activate
# use pip to install packages into your environment
python3 -m pip install jupyter matplotlib pandas
# deactivate an environment
deactivate
# save environment
python3 -m pip freeze > requirements.txt
# restoring environment
python3 -m pip install -r requirements.txt

Use the Terminal to navigate to the venv-model-basic-py folder and create the Python virtual environment:

python3 -m venv .venv

This will create a .venv/ folder inside venv-model-basic-py.

_labs/lab1/venv-model-basic-py/
├── .venv
│   ├── bin
│   ├── include
│   ├── lib
│   └── pyvenv.cfg
└── model-basic-py.qmd

To activate the virtual environment, run the following in the Terminal:

source .venv/bin/activate

This should change the Terminal prompt from:

Bash@MacBook venv-model-basic-py$ 

to:

(.venv) Bash@MacBook venv-model-basic-py$

Now we can install required Python packages (we include jupyter for interactive outputs)

pip install palmerpenguins pandas numpy scikit-learn jupyter

To store the installed packages

%%{init: {'theme': 'neutral', 'look': 'handDrawn', 'themeVariables': { 'fontFamily': 'monospace', "fontSize":"16px"}}}%%

flowchart TD
    %% External sources
    PyPI["PyPI/<br/>GitHub/..."]
    
    %% Local components
    SysPython["system<br/>Python"]
    VirtEnv["virtual<br/>environment"]
    PipCache["pip<br/>cache"]
    Requirements["requirements.txt"]
    
    %% Subgraph for "your computer" vs "the internet"
    subgraph Internet ["<strong>Internet</strong>"]
        PyPI
    end
    
    subgraph Computer ["<strong>Computer</strong>"]
        SysPython
        VirtEnv
        PipCache
        Requirements
    end
    
    %% Connections with function labels
    SysPython -->|"<strong>python -m venv</strong>"| VirtEnv
    
    PyPI -->|"<strong>pip install</strong><br/><strong>pip upgrade</strong>"| PipCache
    PipCache -->|"pip"| VirtEnv
    
    VirtEnv <-->|"<strong>pip list</strong><br/><strong>pip show</strong>"| Requirements
    VirtEnv -->|"<strong>pip freeze</strong>"| Requirements
    Requirements -->|"<strong>pip install -r</strong>"| VirtEnv
    
    %% Additional workflow arrows
    VirtEnv -.->|"<strong>source activate</strong><br/><strong>deactivate</strong>"| VirtEnv
    
    %% Styling
    style Internet fill:#d8e4ff
    style Computer fill:#31e981

    classDef component fill:#fff
    classDef workflow fill:#fff2e8
    

    class SysPython,VirtEnv,PipCache,Requirements,PyPI component
    
    classDef component fill:#fff
    class SysLib,ProjLib,RenvCache,Lockfile,CRAN component

    

venv

Rendering with Python

To render, use the following in the Terminal:

quarto preview model-basic-py.qmd --no-browser --no-watch-inputs

To tell Quarto to use the virtual environment, we can place the following in the model-basic-py.qmd file:

execute:
  python: .venv/bin/python

Or we can set the Python interpreter with the Terminal:³

quarto use python .venv/bin/python3

If you’re working inside RStudio or Posit Workbench, consider letting reticulate manage the Python environment:

library(reticulate)
use_virtualenv("myenv", required = TRUE)
virtualenv_install(
  envname = "myenv", 
  packages = c("palmerpenguins", "pandas", "numpy", 
               "scikit-learn", "jupyter"))

This ensures RStudio uses a known-good Python environment.

Now, if we open the model-basic-py.qmd file in RStudio and click the Render button in the toolbar we should see the rendered output.

---

1. In R, this is renv. In Python, this is venv.

2. Read more in What They Forgot to Teach You About R: Chapter 3, Project-oriented workflow

3. On Windows try .venv\Scripts\python.exe