Skip to content

Latest commit

 

History

History
119 lines (91 loc) · 3.57 KB

INSTALLATION.md

File metadata and controls

119 lines (91 loc) · 3.57 KB

Installation Guide

Setup Options

You can run the code either:

  • Remotely using an AWS SageMaker environment.
  • Locally on your own machine.

The setup differs slightly due to AWS S3 access and GPU availability.

Setting Up a SageMaker Environment

Important: SageMaker instances incur costs while running. Make sure to stop your instance when you're done.

0. Open SageMaker Studio

  • Go to the AWS Console and search for “Amazon SageMaker AI”.
  • Open “SageMaker Studio”.
  • Click “Open Studio” to launch the environment.

1. Choose Your SageMaker Space

You can use different SageMaker spaces:

  • Code Editor (VSCode) -> Recommended for a full development workflow (terminal, Git, debugging)
  • JupyterLab

Choose based on your preference.

2. Configure Your Instance

Recommended settings:

  • Instance type: ml.g5.xlarge
    See Vantage for pricing and GPU details.
  • Volume size:
    • Measurement data: < 20 GB
    • Weather data: < 1 GB
    • Logs: potentially large
      → Recommended: 250 GB

    Note: Increasing storage after initial allocation may cause errors.

3. Clone the Repository

Use either:

  • GitHub integration in VSCode, or
  • Git CLI

4. Run the Setup Script

On first boot:

cd s4casting
bash setup.sh

This will:

  • Install python package manager (uv)
  • Install CUDA
  • Install common development packages / toolchain support

Takes a few minutes. Only needed once per instance.

5. Download data

You can download all the data with the following command:

bash data/get_data.sh

6. You are ready to go!

Setting Up a Local Environment

Note: These local setup instructions target WSL (Ubuntu) or native Linux. Use WSL on Windows.

1. Install and Configure uv

Uv is a python dependency manager, a replacement for the traditional requirements.txt file. Create the virtual environment and install dependencies:

pip install uv
uv venv
uv sync --python 3.12

2. Download the Data

From the home directory run:

./data/get_data.sh

3. Install CUDA (optional)

If you want to train locally with GPU acceleration, you need to install CUDA.

There are several ways to do this depending on your environment, for example using system packages:

# Add NVIDIA keyring and repo
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get install -y cuda-toolkit-12-4

Working Behind a Private Package Mirror

Some corporate environments require all package downloads to go through an internal registry mirror rather than public PyPI. The committed uv.lock always resolves from public PyPI (it is the basis for publicly reproducible builds and is kept up to date automatically); CI rejects lockfiles resolved against private registries.

If your environment requires a mirror, configure it as uv's default index globally on your machine (see the uv index documentation), then keep your locally-resolved lockfile out of git:

  • make lock-local — marks uv.lock as locally managed (git update-index --skip-worktree) and resolves it through your default index. Re-run it regularly to pick up refreshed dependency versions.
  • make lock-reset — restores the committed lockfile. Run it before pulling or switching branches whenever uv.lock changed upstream, then re-run make lock-local.