Skip to main content

Install Apps

Published

2026-11-06

WarningCaution

This section is being revised. Thank you for your patience.

Finding your applications

Installing an application and being able to run it are two different problems. A package manager like yum puts binaries somewhere the shell already knows to look, but anything I install by hand usually lands somewhere the shell doesn’t check. That’s when a command I know is installed comes back as command not found.

The PATH variable

PATH is an environment variable holding a colon separated list of directories. When I type a command, the shell walks that list from left to right and runs the first matching executable it finds. It never searches the whole filesystem.

Print it to see what the shell is actually checking:

echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin

To find out which copy wins, use which:

which python3
/usr/bin/python3

If a command isn’t found, the executable is almost always on the box, just not in one of those directories. Adding it is a matter of appending the directory to PATH in a startup file so the change survives a new login:

echo 'export PATH="/opt/python/3.12.1/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Putting the new directory first means it takes priority over anything with the same name further down the list.

%%{init: {'theme': 'base', 'look': 'classic', 'themeVariables': {'fontFamily': 'monospace'}}}%%

graph TD
    Cmd(["shell command<br/><strong>python3</strong>"]) --> Check1("Check<br/><strong>/usr/local/bin</strong>")
    Check1 -->|"not found"| Check2("Check<br/><strong>/usr/bin</strong>")
    Check2 -->|"found!"| Run("Run<br/><strong>/usr/bin/python3</strong>")
    Check1 -->|"found!"| Run2("Run<br/><strong>/usr/local/bin/python3</strong><br/>(first match wins)")

    style Cmd fill:#d2562b,stroke:#fff,stroke-width:2px,color:#fff
    style Check1 fill:#e8a33d,stroke:#000,stroke-width:2px,color:#000
    style Check2 fill:#e8a33d,stroke:#000,stroke-width:2px,color:#000
    style Run fill:#5b8c5a,stroke:#fff,stroke-width:2px,color:#fff
    style Run2 fill:#5b8c5a,stroke:#fff,stroke-width:2px,color:#fff

How PATH Search Works

Editing ~/.bashrc only changes PATH for that one account. A service running under systemd doesn’t read it at all, and neither does a different user who logs in later. If an application has to be visible to everyone (and to services), a symlink into /usr/local/bin is more reliable than asking every account to edit a startup file.

Where do applications live?

Linux spreads a single application across several directories by purpose, not by product. Knowing the convention is how I find a config file or a log without searching for it.

Table 1: Common application locations
Location What lives there
/bin, /usr/bin Core system executables, managed by the package manager
/usr/local/bin Executables installed outside the package manager, plus symlinks
/opt Self contained third party software, one directory per version
/etc Configuration files
/var/log Log files 

%%{init: {'theme': 'base', 'look': 'classic', 'themeVariables': {'fontFamily': 'monospace'}}}%%

graph TD
    Root(["/"])
    Bin("<strong>/bin</strong> or<br><strong>/usr/bin</strong>")
    Local("<strong>/usr/local/bin</strong>")
    Opt("<strong>/opt</strong>")
    OptPy("<strong>/opt/python/3.12.1</strong>")
    OptR("<strong>/opt/R/4.6.1</strong>")
    Etc("<strong>/etc</strong>")
    Log("<strong>/var/log</strong>")

    Root --"<em>OS managed</em>"--> Bin
    Root --"<em>symlinks here</em>"--> Local
    Root --"<em>versions isolated</em>"--> Opt
    Root --"<em>config files</em>"--> Etc
    Root --"<em>log files</em>"--> Log
    Opt --"<em>lib, bin, share</em>"--> OptPy
    Opt --"<em>lib, bin, share</em>"--> OptR

    style Root fill:#d2562b,stroke:#fff,stroke-width:2px,color:#fff,font-size:12px,font-family:monospace
    style Bin fill:#2a6f77,stroke:#fff,stroke-width:1px,color:#fff,font-size:12px,font-family:monospace
    style Local fill:#5b8c5a,stroke:#fff,stroke-width:1px,color:#fff,font-size:12px,font-family:monospace
    style Opt fill:#e8a33d,stroke:#000,stroke-width:2px,color:#000,font-size:12px,font-family:monospace
    style OptPy fill:#e8a33d,stroke:#000,stroke-width:1px,color:#000,font-size:12px,font-family:monospace
    style OptR fill:#e8a33d,stroke:#000,stroke-width:1px,color:#000,font-size:12px,font-family:monospace
    style Etc fill:#666,stroke:#999,stroke-width:1px,color:#fff,font-size:12px,font-family:monospace
    style Log fill:#666,stroke:#999,stroke-width:1px,color:#fff,font-size:12px,font-family:monospace

Application Filesystem Hierarchy

/opt is the important one for data science work. Installing there keeps a version fully contained in its own folder, so nothing overwrites anything the operating system depends on.

Language versions

EC2 instances don’t ship with current versions of R or Python, so this section installs both on the instance we’ve SSH’d into.

Python version

The Python already on the instance belongs to the operating system. Amazon Linux 2 uses it for yum and other internal tooling, so upgrading it or installing packages into it with sudo pip install risks breaking the box itself.

The safer pattern is to install your own Python into /opt, leave the system copy alone, and create virtual environments from the /opt copy (the steps that build it are in Building Python from source below):

/opt/python/3.12.1/bin/python3.12 -m venv ~/my-app/venv

Activate the virtual environment:

source ~/my-app/venv/bin/activate

Deactivate the virtual environment with:

deactivate

R version

R has no equivalent system dependency, so multiple versions can coexist in /opt/R without conflict. rig, the R Installation Manager, handles installing and switching between them.

rig isn’t in the Amazon Linux 2 repositories, so install it first. The portable Linux build is a tarball that unpacks into /usr/local, which puts rig in /usr/local/bin and on the PATH for every account:

curl -Ls https://github.com/r-lib/rig/releases/download/latest/rig-linux-x86_64-latest.tar.gz | \
  sudo tar xz -C /usr/local
rig --version

On a Graviton (arm64) instance, swap x86_64 for aarch64 in the URL. With rig installed, add R and make it the default:

sudo rig add 4.6.1

Set it as the default:

sudo rig default 4.6.1

Confirm the install; the asterisk marks the default version:

rig list
* 4.6.1

Your output may look different; newer rig versions print a table with a header row and more columns.

rig is one way to get R. On an Ubuntu machine (a local workstation, or an Ubuntu AMI instead of Amazon Linux), CRAN publishes its own apt repository, which tracks the current release (R 4.6.1 as of this writing). Add CRAN’s signing key and repository, then install r-base:

sudo apt update -qq

Install the tools add-apt-repository and key handling depend on:

sudo apt install --no-install-recommends software-properties-common dirmngr

Add CRAN’s signing key:

wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | \
  sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc

Add the CRAN repository for your Ubuntu release:

sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"

Install R:

sudo apt install --no-install-recommends r-base

$(lsb_release -cs) fills in the Ubuntu release codename. Pop!_OS is built on Ubuntu and reports the Ubuntu codename (noble on Pop!_OS 24.04), so the same commands work unchanged. CRAN currently builds for 26.04 (resolute), 24.04 (noble), and 22.04 (jammy).

Confirm the version:

R --version | head -1
R version 4.6.1 (2026-06-24) -- "Happy Hop"

This installs a single, system wide R under /usr/lib/R that apt upgrade keeps current. If you need several versions side by side, use rig instead; it works the same way on Ubuntu.

rig answers “which R is installed on this box?” It doesn’t pin the packages a project uses or the system libraries those packages compile against. rix covers that side. It writes a default.nix file describing a project’s R version, R packages, and system dependencies, and Nix builds that environment into /nix/store, isolated from both the system and anything rig installed.

Table 2: rig versus rix
Tool Scope Lives in
rig R versions for the whole instance /opt/R/<version>
rix + Nix R, packages, and system libraries for one project /nix/store, described by default.nix

Install Nix with the Determinate Systems installer, which the rix docs recommend. The extra configuration adds the rstats-on-nix binary cache, so R packages download prebuilt instead of compiling on the instance:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | \
  sh -s -- install --no-confirm --extra-conf "
trusted-users = root $USER
substituters = https://cache.nixos.org https://rstats-on-nix.cachix.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= rstats-on-nix.cachix.org-1:vdiiVgocg6WeJrODIqdprZRUrhi1JzhBnXv7aWI6+F0="

Open a new shell so nix-shell and nix-build are on your PATH. The R that rig installed can generate the environment file, so install rix into it and describe the project:

mkdir -p ~/my-r-app && cd ~/my-r-app
Rscript -e 'install.packages("rix", repos = "https://cloud.r-project.org")'
Rscript -e '
rix::rix(
  r_ver = "4.6.1",
  r_pkgs = c("dplyr", "shiny"),
  system_pkgs = NULL,
  git_pkgs = NULL,
  ide = "none",
  project_path = ".",
  overwrite = TRUE
)'

ide = "none" fits a server with no editor attached. rix() writes default.nix and an .Rprofile; the .Rprofile keeps R inside the Nix environment from reading the library paths of the rig installed R.

Build the environment, then enter it:

nix-build
nix-shell

Inside nix-shell, R and Rscript resolve to the Nix copy, not /opt/R:

which R
Rscript -e 'R.version.string'

which R should print a path under /nix/store. exit leaves the shell and puts the rig default back in front. Commit default.nix with the project and anyone running nix-build gets the same R, packages, and system libraries, which also sidesteps the -devel packages below for anything Nix provides.

System libraries

Both languages depend on system libraries that neither one installs for you. A package that compiles from source will fail with a missing header error until the matching development package is present:

sudo yum install -y libcurl-devel openssl-devel libxml2-devel

These are the three that account for most failed installs of curl, openssl, and xml2, and every R or Python package that depends on them.

%%{init: {'theme': 'base', 'look': 'classic', 'themeVariables': {'fontFamily': 'monospace'}}}%%

graph TD
    SysPy["System Python<br/><em>owned by OS</em>"]
    OptPy(["Your Python<br/><strong>/opt/python/3.12.1</strong><br/><em>isolated version</em>"])
    Venv("Virtual Env<br/><strong>~/my-app/venv</strong><br/><em>project packages</em>")

    SysPy -->|"leave untouched"| SysTools("yum, AWS CLI,<br/>OS internal tools")
    OptPy -->|"create venv from"| Venv
    Venv -->|"install packages to"| Packages("<strong>pip install<br/>my-packages</strong>")

    R["System R<br/><em>usually absent</em>"]
    OptR("Multiple R versions<br/><strong>/opt/R/4.6.1</strong><br/><strong>/opt/R/4.3.2</strong><br/><em>rig switches</em>")
    Libs("System libraries<br/><strong>libcurl-devel</strong><br/><strong>openssl-devel</strong><br/><strong>libxml2-devel</strong>")

    R -->|"alternative"| OptR
    OptR --> Libs
    Libs -->|"required for"| RPackages("R packages<br/>from source")

    style SysPy fill:#d2562b,stroke:#fff,stroke-width:2px,color:#fff
    style OptPy fill:#5b8c5a,stroke:#fff,stroke-width:2px,color:#fff
    style Venv fill:#5b8c5a,stroke:#fff,stroke-width:2px,color:#fff
    style Packages fill:#2a6f77,stroke:#fff,stroke-width:1px,color:#fff
    style R fill:#d2562b,stroke:#fff,stroke-width:2px,color:#fff
    style OptR fill:#5b8c5a,stroke:#fff,stroke-width:2px,color:#fff
    style Libs fill:#e8a33d,stroke:#000,stroke-width:2px,color:#000
    style RPackages fill:#2a6f77,stroke:#fff,stroke-width:1px,color:#fff
    style SysTools fill:#2a6f77,stroke:#fff,stroke-width:1px,color:#fff

Language Runtime Installation Pattern

Building Python from source

The packaged Python is sometimes too old for what you need to run. Amazon Linux 2 ships with 3.7:

python3 --version
Python 3.7.16

That’s the copy yum depends on, so upgrading it in place is off the table. The fix is the /opt pattern from above: build a second Python, install it into its own versioned directory, and leave the system one alone.

%%{init: {'theme': 'base', 'look': 'classic', 'themeVariables': {'fontFamily': 'monospace', "fontSize":"12px"}}}%%

graph TD
    Check{"<strong>python3 --version</strong><br/>3.11 or newer?"}
    Yes("Use it directly")
    Build("Build into<br/><strong>/opt/python/3.12.1</strong>")
    Venv("<strong>/opt/myapp/.venv</strong>")
    Sys["<strong>/usr/bin/python3</strong><br/><em>3.7, owned by yum</em>"]

    Check --"yes (AL2023, Ubuntu 22.04+)"--> Yes
    Check --"no (Amazon Linux 2)"--> Build
    Yes --> Venv
    Build --"<em>python3 -m venv</em>"--> Venv
    Sys --"<em>leave for</em>"--> Yum("yum, OS tooling")

    style Check fill:#e8a33d,stroke:#000,stroke-width:2px,color:#000
    style Yes fill:#5b8c5a,stroke:#fff,stroke-width:1px,color:#fff
    style Build fill:#2a6f77,stroke:#fff,stroke-width:1px,color:#fff
    style Venv fill:#5b8c5a,stroke:#fff,stroke-width:2px,color:#fff,font-family:monospace
    style Sys fill:#d2562b,stroke:#fff,stroke-width:2px,color:#fff
    style Yum fill:#666,stroke:#999,stroke-width:1px,color:#fff

Which Python Runs the App

Install the compiler and the development headers first, plus a few that Python needs for ssl, bz2, and ctypes support:

sudo yum groupinstall -y "Development Tools"
sudo yum install -y openssl-devel bzip2-devel libffi-devel zlib-devel git

Download and unpack the source:

cd /tmp
wget https://www.python.org/ftp/python/3.12.1/Python-3.12.1.tgz
tar xzf Python-3.12.1.tgz
cd Python-3.12.1

Then configure it to land in /opt, and build:

sudo ./configure --prefix=/opt/python/3.12.1 --enable-optimizations
sudo make altinstall

--prefix is what keeps the install self contained. altinstall is the flag that matters most: it installs python3.12 without also writing a python3 symlink that would shadow the system copy.

Confirm you got what you asked for:

/opt/python/3.12.1/bin/python3.12 --version
Python 3.12.1

--enable-optimizations, passed to ./configure above, is what triples the compile time. It builds Python twice: once to produce an instrumented interpreter, then runs it against Python’s own test suite to record which code paths get hit most, then rebuilds using that profile data to optimize those paths (profile guided optimization, or PGO). It also turns on link time optimization (LTO). On a t3.micro the whole thing takes 20 to 30 minutes with the CPU pegged.

‘Runtime speed’ here means how fast the finished interpreter executes Python code after it’s installed, not how long the build itself takes. PGO typically buys 10 to 20 percent on CPU bound benchmarks. A subprocess-heavy or I/O-bound app isn’t CPU bound, so that speedup is not something you’d notice in practice. Drop the flag (sudo ./configure --prefix=/opt/python/3.12.1) and the build finishes in a few minutes with a standard, unoptimized interpreter.

If make dies partway through with an out of memory error, see Instance Size for what that means and how to work around it on a small instance.

More on the flag itself: https://docs.python.org/3/using/configure.html#cmdoption-enable-optimizations

The interpreter you just built has no SSL support, so pip can’t reach PyPI over HTTPS at all. The cause traces back to the ./configure step: Amazon Linux 2’s default openssl-devel package is OpenSSL 1.0.2k, and Python 3.10 and newer require 1.1.1 or later to build the _ssl extension. Configure looked for a new enough OpenSSL, didn’t find one, and silently skipped the module rather than failing the build. make altinstall finished without complaint because none of the steps you ran so far touch SSL, so the problem doesn’t show up until pip tries to talk to the internet.

AL2 ships that newer OpenSSL under a separate package, openssl11, but its -devel package can’t be installed alongside the old one: both own the same header path, /usr/include/openssl/*.h, so yum refuses with a conflict rather than letting them collide. The fix is to swap the devel package out, build, then swap it back, since Python only needs the headers at build time, not at runtime:

sudo yum remove -y openssl-devel

sudo yum install -y openssl11-devel

cd /tmp/Python-3.12.1

sudo ./configure --prefix=/opt/python/3.12.1 --enable-optimizations

sudo make altinstall

Python’s configure script checks for openssl11 via pkg-config before falling back to the system default, which is what makes this work without a --with-openssl flag. You don’t need to redownload the source or delete the existing /opt/python/3.12.1 tree, altinstall overwrites in place.

Confirm the module actually built this time before retrying the venv:

/opt/python/3.12.1/bin/python3.12 -c 'import ssl; print(ssl.OPENSSL_VERSION)'

That should print an OpenSSL 1.1.1 version string, not a traceback. Once it does, put the original devel package back so other builds on the box aren’t left pointing at openssl11:

sudo yum remove -y openssl11-devel

sudo yum install -y openssl-devel

Recap

  1. PATH decides what the shell can find, and which tells you which copy is winning. A command not found error usually means a directory is missing from the list, not that the software is missing.

  2. Install language runtimes into /opt and symlink them into /usr/local/bin. That keeps the system Python untouched, makes the version visible to every account and service, and turns a version change into a one line edit.

  3. When the packaged interpreter is too old, build your own into /opt with make altinstall rather than touching the system copy. Installing a Real Application applies the same pattern to a full app.