# Compute Containers

> **TL;DR** – The containers you launch are meant for interactive development,
> not for heavy‑weight HPC jobs. They run on dedicated Kubernetes nodes (20 CPU
> / 250 GiB RAM) but each container is limited to **8 CPU**, **50 GiB RAM** and
> a modest storage quota.

---

## What a “Compute Container” is (in plain language)

- It is a **JupyterLab notebook server** that starts in a few seconds.
- The container gives you a ready‑to‑use Python environment (the Sciserver
  images) and direct access to the platform’s shared data volumes.
- Think of it as a **personal, cloud‑based workstation** – convenient for
  prototyping, data exploration, and light analyses.
- Containers are ephemeral by nature. Make sure to get comfortable using your
  personal storage and python environments to make the most out of containers.

---

## Resource limits you should know

| Resource | What the container gets |
|----------|-------------------------|
| **CPU** | **8 vCPU** (hard limit) |
| **Memory** | **50 GiB** (hard limit) |
| **Storage** | **≈4 GiB** per container (size‑limit) |

*You do **not** need to configure any of this – it is applied automatically
  when you create **“New Container”**.*

---

## Parallelization

> **Important:** Heavy CPU/GPU use or long‑running batch tasks should run in
> **Compute Jobs** or, preferably, on the
> [MPCDF High‑Performance Computing (HPC) cluster](https://docs.mpcdf.mpg.de/doc/computing/viper-user-guide.html),
> which has vastly more resources.

Parallelization in Python can speed up processing by using multiple cores, but
in SciServer all containers share finite CPU and memory with other users.
Intensive parallel work may slow down colleagues’ jobs. Moreover, you have
access to exponentially more CPUs and Memory using available alternatives.

If your workflow already needs parallelization, you’re likely ready to move to
HPC. Contact us and we’ll guide you on integrating your data and code into the
cluster.

---

## Why the storage limit matters

The container’s root filesystem is a **temporary layer** that disappears when
the session ends. If you install a lot of Python packages (e.g., via
`mamba install …`) the cache can fill the ~4 GiB quota and the installation will
fail with “No space left on device”.

**Best practice:** keep the package cache **outside** the container, on your
personal persistent storage.

---

## Quick checklist for new users

1. **Launch** a Compute container from the portal (choose the image you need).
1. **Set the cache location** (run the three commands above).
1. **Install** additional packages with `mamba install …`.
1. **Work** in JupyterLab as usual – your notebooks and data are saved in the
   mounted `/workspace/Storage/<username>/persistent/` directory, so they
   survive container restarts.
1. **When you’re done**, simply close the session; the container is terminated
   but your files and the mamba cache remain.

---

## Managing Python Environments

By default, software installed inside a container disappears when the container
stops. To keep your Python packages and libraries, you must store them in your
**Persistent User Storage**.

We recommend two workflows depending on your needs:

1. **Persistent Environments (Recommended):** Store the actual environment
   folder on your storage. This is fastest for daily work.
   - 📘 **[Guide: Setting up Persistent
     Environments](./best_practices/persistent_environments.md)**

2. **Reproducible Environments:** Export an `environment.yml` file. This is best
   for sharing code or ensuring exact version matches across projects.
   - See the
     [Conda Environment Guide](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html).

---

## Where to get help

Upstream docs: <https://www.sciserver.org/about/compute/>

For help, see the **[Support & Contact](./support)** page.