Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vault operator migrate does not work with s3 storage source #7362

Closed
vroy opened this issue Aug 26, 2019 · 1 comment · Fixed by #7966
Closed

vault operator migrate does not work with s3 storage source #7362

vroy opened this issue Aug 26, 2019 · 1 comment · Fixed by #7966
Assignees
Labels
bug Used to indicate a potential bug core/migration ecosystem

Comments

@vroy
Copy link

vroy commented Aug 26, 2019

Describe the bug

When running vault operator migrate with a s3 source, no keys are actually copied.

storage_source "s3" {
  bucket = "<source-bucket>"
  region = "us-east-1"
}

storage_destination "postgresql" {
  connection_url = "postgres://postgres:dev@127.0.0.1/postgres?sslmode=disable"
}

To Reproduce

Steps to reproduce the behavior:

  1. Initialize Vault with S3 backend
  2. Run vault operator migrate from s3 source to another backend

Expected behavior

I expected the keys to be copied to the destination backend.

Environment:

  • Vault CLI Version (retrieve with vault version): 1.2.2
  • Server Operating System/Architecture: Mac and Linux

Additional context

This works with Vault 1.1.4.

I think I've narrowed it down to this pull request / code block: https://github.com/hashicorp/vault/pull/7157/files#diff-114a477785f71145d9fc9bbdce32378bR269-R272

Some quick testing with python/boto3 demonstrates that setting Prefix to "/" returns nothing:

import boto3

s3 = boto3.client("s3", region_name="us-east-1")

response = s3.list_objects_v2(
    Bucket="<my-vault-bucket>",
    Prefix="/",
    Delimiter="/",
)
print(response["KeyCount"]) #=> 0

response = s3.list_objects_v2(
    Bucket="<my-vault-bucket>",
    Prefix="",
    Delimiter="/",
)
print(response["KeyCount"]) #=> 3
@chrishoffman chrishoffman added bug Used to indicate a potential bug ecosystem labels Aug 27, 2019
@ilatvala
Copy link

ilatvala commented Dec 2, 2019

Confirm this same bug with vault 1.3 and CEPH S3 storage backend. Used vault 1.1.3 to migrate, since that works ok.

@kalafut kalafut self-assigned this Dec 3, 2019
kalafut pushed a commit that referenced this issue Dec 4, 2019
Also remove some incorrect skipping of the S3 test.

Fixes #7362
kalafut pushed a commit that referenced this issue Dec 4, 2019
Also remove some incorrect skipping of the S3 test.

Fixes #7362
briankassouf pushed a commit that referenced this issue Dec 18, 2019
Also remove some incorrect skipping of the S3 test.

Fixes #7362
briankassouf added a commit that referenced this issue Dec 18, 2019
* Output human duration in TTL warnings (#7901)

* Add enable_hostname_label option to telementry stanza (#7902)

* store secret key and value as an object to fix copy/show secret bug (#7926)

* Add accept header check for prometheus mime type (#7958)

* Add accept header check for prometheus mime type

* Fix small header filter bug. Add test

* Fix S3 configurable path handling (#7966)

Also remove some incorrect skipping of the S3 test.

Fixes #7362

* Ui/fix demoting status menu (#7997)

* fix bug where users couldn't click on update primary

* don't show status menu items when cluster isSecondary since those links don't work

* show the mode of replication in the status menu

* do not show server header in status menu when the contents are empty

* show Disaster Recovery instead of 'DR'

* do not show http metrics in status menu unless user is authenticated

* fix typo so icons in status menu show

* Transit: error when restoring to a name that looks like a path (#7998)

* Add test to verify #7663

* Validate name in transit key restore to not be a path

* overwrite bulma bug that crashes safari (#8023)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Used to indicate a potential bug core/migration ecosystem
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants