Skip to content

Commit

Permalink
[Docs] Clarify Storage mounting details (skypilot-org#1365)
Browse files Browse the repository at this point in the history
* fix incorrect statements

* fix incorrect statements

* fix incorrect statements
  • Loading branch information
romilbhardwaj authored and Sumanth committed Jan 15, 2023
1 parent db79241 commit ba1e031
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 6 deletions.
16 changes: 13 additions & 3 deletions docs/source/reference/storage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,8 @@ When specifying a storage object, you can specify either of two modes:
In effect, files are streamed from the backing source bucket as and when
they are accessed by applications. This mode also allows applications to
write to the mount path. All writes are replicated to remote bucket (and
any other VMs mounting the same bucket). Please note that this mode
uses a close-to-open consistency model, which means a file write is
committed to the backing store only after :code:`close()` is called on it.
any other VMs mounting the same bucket).


- :code:`mode: COPY`
This mode pre-fetches your files from remote storage and caches them on the
Expand Down Expand Up @@ -176,6 +175,17 @@ and storage mounting:
For mounts not using SkyPilot Storage (e.g., those using rsync) the symbolic links are directly copied, not their target data.
The targets must be separately mounted or else the symlinks may break.

.. note::
:code:`MOUNT` mode employs a close-to-open consistency model. This means calling
:code:`close()` on a file will upload the entire file to the backing object store.
Any subsequent reads, either using SkyPilot Storage or external utilities (such as
aws/gsutil cli) will see the latest data.

.. note::
:code:`MOUNT` mode does not support the full POSIX interface and some file
operations may fail. Most notably, random writes and append operations are
not supported.

.. note::
Storage only supports uploading directories (i.e., :code:`source` cannot be a file).
To upload a single file to a bucket, please put in a directory and specify the directory as the source.
Expand Down
6 changes: 3 additions & 3 deletions examples/perf/results.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ We copy a 1 GB file from S3 to EBS using `aws s3 cp` command.
* SkyPilot Storage offers higher **read** performance than EBS because it can directly
stream files from S3 to memory.
* SkyPilot Storage **write** performance is significantly higher than EBS because it
writes to memory and then asynchronously uploads to S3. SkyPilot Storage offers
only eventual consistency, so a write operation to SkyPilot Storage may not reflect
immediately on the S3 storage.
writes directly to S3 over network instead of disk.
* SkyPilot Storage offers close-to-open consistency, so it is guaranteed that when a file is closed,
subsequent opens will see the latest changes.
* These benchmarks are run on single large files. The performance of SkyPilot Storage
degrades when using many small files.

0 comments on commit ba1e031

Please sign in to comment.