Releases · skypilot-org/skypilot

We are excited to release SkyPilot 0.2.0, which receives a host of new features, with many enhancements and fixes.

Highlights

Managed Spot is made much more robust and easier to use.
- Try using sky spot launch on your existing yamls!
- We've seen users running 1000s of spot jobs in a recurring schedule.
TPU Pods are now supported.
- To use a TPU Pod, simply modify e.g., accelerators: tpu-v2-8 to accelerators: tpu-v2-32.
Benchmark: use sky bench to easily measure the performance and cost of different cloud resources for your task.
Provisioning is sped up by ~1 minute.
Catalog is updated to V3 with 100s of resource changes and 1000s of price changes.
- A100-80GB is now available on 3 clouds. Check out sky show-gpus -a for GPU prices.
- No action needed as this will be automatically downloaded.

CLI & Task interface

New Features

Add zone support in YAML #1014
Add shell completion support for CLI by #1162
Add --no-setup option to sky launch to allow for remounting of files without running setup commands again #1184
Add sky start --all to start all clusters #1065
Add glob support for sky storage delete #1117
Add --no-follow option to sky logs and sky spot logs (print logs so far and exit)

Enhancements

Show vCPUs in optimizer/benchmark messages #1076
Make entrypoint optional: for quick VM launching, no more sky launch <flags> '', simply do sky launch <flags> #1191
Make sky check automatically enable necessary GCP APIs (#1197, #1209); make it more robust for AWS checks (#1194)

Managed spot

New Features

sky spot launch now automatically translates file_mounts in a YAML to use cloud storage. #1081 #1215
- This means the same YAML for on-demand resources launched by sky launch can now be launched by sky spot launch.
Add --retry-until-up for sky spot launch; improve the responsiveness for sky spot cancel #1098
Expose a $SKYPILOT_RUN_ID environment variable shared by all recoveries of the same spot job (useful for identifying it in Weights & Biases) #1196
- See the last Note block in docs.

Enhancements

Distinguish spot controller names for different users #1101
- This may leak an old stopped controller if you have used spot launch with <= 0.1.2.
Add retry for spot cluster termination #1139
Enable purge for spot controller #1107
Show FAILED_CONTROLLER when controller exit abnormally #1143
Make get_job_timestamp fetching more robust #1148
Fail early when spot cluster name too long occurs on GCP #1183

Fixes

Fix the retry logic for spot cluster launching #1150
Fix non-persistent storage deletion for spot #974
Fix spot recovery without cloud specified #1077
Fix spot job duration #1104
Fix sky spot status -a for resources and region information #1135

TPU support

Support TPU Pod #1001

Provisioner

Enhancements

Improving provision speed by ~1 minute (#1092, #1103, #1108, #1111, #1126)
Add host VM - GPU compatibility checks for GCP #989

Fixes

Fix GCP VM leak issue #1102
Fix GCP A100 launch error #1166
Fix K80 gpunode by correcting GCP image version #1090

On-prem

Enhancements

Simplified on-prem deployment
- sky admin deploy now automatically installs skypilot, ray (and python3 and pip3) on the local cluster under admin user #1116
Add cluster config schema check #1044
Modify Sky Admin's Setup on Docs #1085
Align Python Versions #1086

Fixes

Fix Sky Status Logging #1041

Backend

Enhancements

Catalog is updated to V3 with 100s of resource changes and 1000s of price changes #1204
Canonicalize accelerator names in Resources #1075
Reduce the frequency of job status update and remove parallel query #1096
Increase thread limit and fix nofile limit #1128

Fixes

[Storage] Fix public bucket source check in SkyPilot Storage #1087
Fixes ray dashboard hanging problem (#1088) #1109
Fix placement group not scheduled issue (issue #1130) #1134

Misc. enhancements

New example: Stable Diffusion #1149
pip install skypilot now installs skypilot[aws] by default #1055
Improve error messages for cloud import errors #1156
Change ~/.ssh/config permissions #1174
Relative cluster yaml #1176
UX: remove DURATION, move HOURLY_PRICE in status table (-a) #1129

Thanks to all Contributors!

New contributors

@sumanthgenz made their first contribution in #1065
@ewzeng made their first contribution in #1174

Many thanks to all contributors who contributed to this release!

@Michaelvll, @concretevitamin, @infwinston, @michaelzhiluo, @WoosukKwon, @romilbhardwaj, @sumanthgenz, @ewzeng, @iojw, @franklsf95

Highlights

This is our first release for SkyPilot -- a framework for easily running machine learning workloads on any cloud through a unified interface. No knowledge of cloud offerings is required or expected – you simply define the workload and its resource requirements, and SkyPilot will automatically execute it on AWS, Google Cloud Platform or Microsoft Azure.

Key features