Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Clarify Storage mounting details #1365

Merged
merged 3 commits into from
Nov 3, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions docs/source/reference/storage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,8 @@ When specifying a storage object, you can specify either of two modes:
In effect, files are streamed from the backing source bucket as and when
they are accessed by applications. This mode also allows applications to
write to the mount path. All writes are replicated to remote bucket (and
any other VMs mounting the same bucket). Please note that this mode
uses a close-to-open consistency model, which means a file write is
committed to the backing store only after :code:`close()` is called on it.
any other VMs mounting the same bucket).


- :code:`mode: COPY`
This mode pre-fetches your files from remote storage and caches them on the
Expand Down Expand Up @@ -176,6 +175,17 @@ and storage mounting:
For mounts not using SkyPilot Storage (e.g., those using rsync) the symbolic links are directly copied, not their target data.
The targets must be separately mounted or else the symlinks may break.

.. note::
:code:`MOUNT` mode employs a close-to-open consistency model. This means calling
:code:`close()` on a file will upload the entire file to the backing object store.
Any subsequent reads, either using SkyPilot Storage or external utilities (such as
aws/gsutil cli) will see the latest data.

.. note::
:code:`MOUNT` mode does not support the full POSIX interface and some file
operations may fail. Most notably, random writes and append operations are
not supported.

.. note::
Storage only supports uploading directories (i.e., :code:`source` cannot be a file).
To upload a single file to a bucket, please put in a directory and specify the directory as the source.
Expand Down
6 changes: 3 additions & 3 deletions examples/perf/results.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ We copy a 1 GB file from S3 to EBS using `aws s3 cp` command.
* SkyPilot Storage offers higher **read** performance than EBS because it can directly
stream files from S3 to memory.
* SkyPilot Storage **write** performance is significantly higher than EBS because it
writes to memory and then asynchronously uploads to S3. SkyPilot Storage offers
only eventual consistency, so a write operation to SkyPilot Storage may not reflect
immediately on the S3 storage.
writes directly to S3 over network instead of disk.
* SkyPilot Storage offers close-to-open consistency, so it is guaranteed that when a file is closed,
subsequent opens will see the latest changes.
* These benchmarks are run on single large files. The performance of SkyPilot Storage
degrades when using many small files.