Use a role cache to avoid separate locking paths

[jefferai]

Due to the various locked/nonlocked paths we had a case where we weren't
always checking for secondary status before trying to upgrade. This
broadly simplifies things by using a cache to store the current role
values (avoiding a lot of storage hits) and updating the cache on any
write, delete, or invalidation.

[tyrannosaurus-becks]

These changes look good to me. The tests that have been added are passing. I do see a test failure in the aws credential plugin directory, but it appears unrelated:

--- FAIL: TestBackendAcc_LoginWithCallerIdentity (2.42s)
    backend_test.go:1564: bad: failed to create role: resp:&logical.Response{Secret:<nil>, Auth:<nil>, Data:map[string]interface {}{"error":"unable to resolve ARN \"arn:aws:iam::047646243813:user/tbex\" to internal ID: InvalidClientTokenId: The security token included in the request is invalid\n\tstatus code: 403, request id: 46f88dbd-92de-11e9-ba31-51c4e4fb1518"}, Redirect:"", Warnings:[]string(nil), WrapInfo:(*wrapping.ResponseWrapInfo)(nil), Headers:map[string][]string(nil)}
        err:<nil>

I haven't been able to verify yet if this failure also occurs on master because it's not part of automated tests.

[jefferai]

Yeah that seems unrelated.

[catsby]

TestBackendAcc_LoginWithCallerIdentity

This test passes for me:

[vault][aws-role-cache](4)$ make testacc TEST=./builtin/credential/aws TESTARGS="-run=TestBackendAcc_LoginWithCallerIdentity"
==> Checking that build is using go version >= 1.11...
==> Using go version 1.12.6...
VAULT_ACC=1 go test -tags='' ./builtin/credential/aws -v -run=TestBackendAcc_LoginWithCallerIdentity -timeout=45m
=== RUN   TestBackendAcc_LoginWithCallerIdentity
--- PASS: TestBackendAcc_LoginWithCallerIdentity (3.03s)
PASS
ok      github.com/hashicorp/vault/builtin/credential/aws       3.262s