Massed Compute

Launch L40 VM

Create a new L40 instance with Ubuntu 24.04. Use the gpu_1x_l40 or gpu_1x_l40_spot SKU for optimal Lance performance.

curl -X POST https://vm.massedcompute.com/api/instances/launch \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "productName": "gpu_1x_l40",
    "imageId": 184,
    "regionName": "any",
    "instanceName": "lance-l40"
  }'

Connect via SSH

Wait for the instance to reach running status, then connect with your SSH key. Clear any stale host keys if reusing an IP.

ssh-keygen -R YOUR_VM_IP
ssh -i ~/.ssh/your-key ubuntu@YOUR_VM_IP

Verify GPU and Install Miniconda

Confirm the NVIDIA L40 is detected and install Miniconda for Python environment management.

nvidia-smi   # Verify L40 + driver ≥ 555

cd ~ && wget -q https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh \
  && bash miniconda.sh -b -p "$HOME/miniconda3" \
  && eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"

Clone Lance and Create Environment

Download Lance from ByteDance Research and create a Python 3.11 conda environment with required dependencies.

mkdir -p ~/work && cd ~/work
git clone https://github.com/bytedance/Lance.git && cd Lance
eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"

# Accept Anaconda Terms of Service
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r

conda create -n Lance python=3.11 -y && conda activate Lance
python --version   # Must print 3.11.x

Install PyTorch with CUDA Support

Install PyTorch 2.8.0 with CUDA 12.6 support and Lance dependencies from requirements.txt.

eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"
conda activate Lance
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 \
  --index-url https://download.pytorch.org/whl/cu126
pip install -r requirements.txt

Install Flash Attention

Install flash-attention 2.8.3 using a prebuilt wheel to avoid compilation requirements.

eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"
conda activate Lance
pip install --no-cache-dir --no-deps --force-reinstall \
  "https://huggingface.co/strangertoolshf/flash_attention_2_wheelhouse/resolve/main/wheelhouse-flash_attn-2.8.3/linux_x86_64/torch2.8/cu12/abiTRUE/cp311/flash_attn-2.8.3+cu12torch2.8cxx11abiTRUE-cp311-cp311-linux_x86_64.whl"

Download Model Weights

Download Lance-3B and Lance-3B_Video weights (~30GB total) from Hugging Face. Use a scoped read-only token to avoid rate limits.

eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"
conda activate Lance
mkdir -p downloads
python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="bytedance-research/Lance",
    local_dir="./downloads/",
    cache_dir="./downloads/cache",
    local_dir_use_symlinks=False,
    resume_download=True,
    allow_patterns=["*.json","*.safetensors","*.bin","*.py","*.md","*.txt","*.pth"],
)
PY

du -sh downloads/*   # Lance_3B ≈ 12GB, Lance_3B_Video ≈ 14GB

Launch Gradio Interface

Set optimization environment variables and start the Gradio web interface for interactive multimodal tasks.

export NVIDIA_TF32_OVERRIDE=1
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512
export CUDA_MODULE_LOADING=LAZY

eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"
cd ~/work/Lance && conda activate Lance
python lance_gradio.py --server-name 127.0.0.1 --server-port 7860

Open an SSH tunnel from your local machine: ssh -L 7860:localhost:7860 YOUR_VM_IP

Access the interface at http://localhost:7860. First generation takes 30-90 seconds for model loading.

Run CLI Smoke Test

Validate the installation with a noninteractive image understanding test to verify all components work correctly.

eval "$("$HOME/miniconda3/bin/conda" shell.bash hook)"
cd ~/work/Lance && conda activate Lance
export NVIDIA_TF32_OVERRIDE=1
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512
export CUDA_MODULE_LOADING=LAZY

bash inference_lance.sh \
  --TASK_NAME x2t_image \
  --MODEL_PATH downloads/Lance_3B \
  --SAVE_PATH_GEN results/qa_x2t_image \
  --VALIDATION_NUM_TIMESTEPS 1 \
  --USE_KVCACHE true

find results/qa_x2t_image -maxdepth 2 -type f -printf '%p %s bytes\n' | sort