Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: bound service account tokens #1460

Merged
merged 1 commit into from
Apr 20, 2018
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
239 changes: 239 additions & 0 deletions contributors/design-proposals/auth/bound-service-account-tokens.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
# Bound Service Account Tokens

Author: @mikedanese

# Objective

This document describes an API that would allow workloads running on Kubernetes
to request JSON Web Tokens that are audience, time and eventually key bound.

# Background

Kubernetes already provisions JWTs to workloads. This functionality is on by
default and thus widely deployed. The current workload JWT system has serious
issues:

1. Security: JWTs are not audience bound. Any recipient of a JWT can masquerade
as the presenter to anyone else.
1. Security: The current model of storing the service account token in a Secret
and delivering it to nodes results in a broad attack surface for the
Kubernetes control plane when powerful components are run - giving a service
account a permission means that any component that can see that service
account's secrets is at least as powerful as the component.
1. Security: JWTs are not time bound. A JWT compromised via 1 or 2, is valid
for as long as the service account exists. This may be mitigated with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-expiring tokens can be revoked by removing the associated Secret or ServiceAccount object.

I think we'd still want some form of this in order to be able to generate legacy-compatible tokens out of the API. What about this:

  • a non-expiring token must be bound to a deletable object name+uid (like a pod or secret)
  • non-expiring tokens can only be for the apiserver audience (since only the apiserver can be counted on to verify the bound object still exists)
  • the API server must verify the object+uid still exists before verifying the token

This gets the confidential and bulky portion of unexpiring service account tokens out of the API, and still lets them be revocable. It also lets us move to this new model for both legacy and expiring/rotating cases, which is desirable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not only desirable, required. I think we want to completely supplant the old model.

Copy link
Member Author

@mikedanese mikedanese Dec 13, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes OIDC signing key rotation difficult or impossible. Do we need to support non-expiring forever or just for some number of releases (in accordance to our GA deprecation policy)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes key OIDC signing key rotation difficult or impossible.

How so? A non-expiring token doesn't mean it can't be revoked or made invalid, it just doesn't have an inherent expiration time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It means you could never remove a public key form the JWKS_URI since it could potentially need to exist forever to verify a token.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd consider that working as intended. No inherent expiration, but an admin decision to revoke or rotate signing keys could invalidate it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is that it's not easy to maintain compatibility with the current client-go while also rotating SA signing keys. If we rotate signing keys every 4 days, how is a non-expiring token useful?

service account signing key rotation but is not supported by client-go and
not automated by the control plane and thus is not widely deployed.
1. Scalability: JWTs require a Kubernetes secret per service account.

# Proposal

Infrastructure to support on demand token requests will be implemented in the
core apiserver. Once this API exists, a client of the apiserver will request an
attenuated token for it's own use. The API will enforce required attenuations,
e.g. audience and time binding.

## Token attenuations

### Audience binding

Tokens issued from this API will be audience bound. Audience of requested tokens
will be bound by the `aud` claim. The `aud` claim is an array of strings
(usually URLs) that correspond to the intended audience of the token. A
recipient of a token is responsible for verifying that it identifies as one of
the values in the audience claim, and should otherwise reject the token. The
TokenReview API will support this validation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the tokenreview API only support the apiserver audience?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think so. I hope that it will support any audience so integrators can easily validate tokens.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what audience would the apiserver use when making a TokenReview request to an authentication webhook?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last question is still outstanding

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, missed this. I expect that the apiserver would populate audience with every name that it identifies itself as. Initially that would be ["https://kubernetes.default.svc"] but we could add configuration to override or add more identifiers to that list. This would be the same list of audiences that the apiserver would validate service account id tokens against in the service account id token authenticator.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that seems like the correct semantics to me. If you agree, I'll add that to this proposal

Copy link
Member

@liggitt liggitt Feb 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking through the audience implications on TokenReview for the following cases:

  • kubelets verifying presented bearer tokens against kube-apiserver's TokenReview
    • old kubelets won't specify an audience... default should mean "valid against kube-apiserver" as it does today
    • what audience(s) should new kubelets specify?
  • extension API servers verifying bearer tokens against kube-apiserver's TokenReview
    • old extension servers won't specify an audience... default should mean "valid against kube-apiserver" as it does today
    • what audience(s) should new extension API servers specify?
  • kube-apiserver verifying bearer tokens against the authentication webhook TokenReview endpoint
    • old webhooks won't be audience-aware and will just verify the token
    • what audience(s) should new webhooks expect/verify?

Also, the kube-apiserver doesn't have the plumbing needed to verify tokens for arbitrary audiences (which would be required for it to handle an Audiences field in TokenReview). The way kube-apiserver implements TokenReview today is to pass a constructed http request containing Authorization: Bearer <token> to the request authenticator:

https://github.com/kubernetes/kubernetes/blob/c17b418f89e056bea272ed46ca9f63f8b1e37589/pkg/registry/authentication/tokenreview/storage.go#L61-L69

I'm afraid adding arbitrary audience support would mean expanding the AuthenticateRequest, AuthenticateToken, and AuthenticatePassword interface signatures to include Audiences []string.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubelets verifying presented bearer tokens against kube-apiserver's TokenReview

  • old kubelets won't specify an audience... default should mean "valid against kube-apiserver" as it does today
  • what audience(s) should new kubelets specify?

When configured to have individual audiences, the identifiers that they request as SANs seems reasonable.

https://github.com/kubernetes/kubernetes/blob/9847c8ee0a2c71fc4e9e1665017f05de7c9f815e/pkg/kubelet/kubelet.go#L745-L746

extension API servers verifying bearer tokens against kube-apiserver's TokenReview

  • old extension servers won't specify an audience... default should mean "valid against kube-apiserver" as it does today
  • what audience(s) should new extension API servers specify?

Extension apiservers are rarely accessed directly today. In the cases where they are, a security conscious operator may want to configure a separate audience for the extension apiserver.

kube-apiserver verifying bearer tokens against the authentication webhook TokenReview endpoint

  • old webhooks won't be audience-aware and will just verify the token
  • what audience(s) should new webhooks expect/verify?

I would not expect an google access token with scope "cloudplatform" to be valid against vault. I would not expect a google oidc id token (where the audience is the client_id) to be valid against vault. Moving forward, I would expect TokenReview webhooks to become aware of the new field.

I'm afraid adding arbitrary audience support would mean expanding the AuthenticateRequest, AuthenticateToken, and AuthenticatePassword

It seems like it's this or some hacky side channel.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect TokenReview webhooks to become aware of the new field.

We could also not forward TokenReviews with non-apiserver audiences for some period of time.

Copy link
Member

@liggitt liggitt Apr 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think those make sense, though I'm unsure how we'd automatically transition to the "use individual audience" option over time. Would be unfortunate for that to be a off-by-default option that never gets used


### Time binding

Tokens issued from this API will be time bound. Time validity of these tokens
will be claimed in the following fields:

* `exp`: expiration time
* `nbf`: not before
* `iat`: issued at

A recipient of a token should verify that the token is valid at the time that
the token is presented, and should otherwise reject the token. The TokenReview
API will support this validation.

Cluster administrators will be able to configure the maximum validity duration
for expiring tokens. During the migration off of the old service account tokens,
clients of this API may request tokens that are valid for many years. These
tokens will be drop in replacements for the current service account tokens.
Copy link
Member Author

@mikedanese mikedanese Jan 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I've opted to require time binding on all JWTs issued by this API. It eliminates the need to support two styles of token and can still support everything we want to support.


### Object binding

Tokens issued from this API may be bound to a Kubernetes object in the same
namespace as the service account. The name, group, version, kind and uid of the
object will be embedded as claims in the issued token. A token bound to an
object will only be valid for as long as that object exists.

Only a subset of object kinds will support object binding. Initially the only
kinds that will be supported are:

* v1/Pod
* v1/Secret

The TokenRequest API will validate this binding.

## API Changes

### Add `tokenrequests.authentication.k8s.io`

We will add an imperative API (a la TokenReview) to the
`authentication.k8s.io` API group:

```golang
type TokenRequest struct {
Spec TokenRequestSpec
Status TokenRequestStatus
}

type TokenRequestSpec struct {
Copy link
Member

@liggitt liggitt Jan 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all the use cases I can think of for token requests benefit from including resource+name+uid of at least one object as a claim in the token:

  • a token requested by node n for service account sa should be bound to a pod with serviceAccountName=sa and nodeName=n (and stop working against the apiserver when that pod goes away)
  • a non-expiring token should be bound to some object in the namespace that indicates the existence of the token, and can be deleted in order to revoke the token (just like a service account token secret behaves today)

What if the TokenRequestSpec included an object reference (or list of object references) that the requester wanted the token bound to, and the TokenRequestStatus include the object reference (or list of references) the token was actually bound to.

Immediately, that would let the NodeRestriction plugin limit nodes to requesting tokens bound to pods scheduled to themselves and referencing the correct serviceaccount.

Longer term, it would let the API provision an object purpose-built for recording the existence of a service account token without containing any confidential info (e.g. a ServiceAccountToken object) in response to an otherwise unbound token request, and returning the reference to the requester.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be an ObjectReference or LocalObjectReference (would have to add a few fields to LocalObjectReference) or should I create a new type for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a section on object binding.

// Audiences are the intendend audiences of the token. A token issued
// for multiple audiences may be used to authenticate against any of
// the audiences listed. This implies a high degree of trust between
// the target audiences.
Audiences []string

// ValidityDuration is the requested duration of validity of the request. The
// token issuer may return a token with a different validity duration so a
// client needs to check the 'expiration' field in a response.
ValidityDuration metav1.Duration

// BoundObjectRef is a reference to an object that the token will be bound to.
// The token will only be valid for as long as the bound object exists.
BoundObjectRef *BoundObjectReference
}

type BoundObjectReference struct {
// Kind of the referent. Valid kinds are 'Pod' and 'Secret'.
Kind string
// API version of the referent.
APIVersion string

// Name of the referent.
Name string
// UID of the referent.
UID types.UID
}

type TokenRequestStatus struct {
// Token is the token data
Token string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can token ever be data that isn't unicode data? I.e. do we need to deal with binary tokens? Or will we hardcode and assume all tokens are safe for HTTP header values and for strings?

Copy link
Member

@liggitt liggitt Dec 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can token ever be data that isn't unicode data? I.e. do we need to deal with binary tokens?

I would not expect that. Every spec I've seen gives bearer tokens a really limited character set

b64token    = 1*( ALPHA / DIGIT / "-" / "." / "_" / "~" / "+" / "/" ) *"="
credentials = "Bearer" 1*SP b64token

TokenReviewSpec has Token string as well


// Expiration is the time of expiration of the returned token. Empty means the
// token does not expire.
Expiration metav1.Time
}

```

This API will be exposed as a subresource under a serviceacccount object. A
requestor for a token for a specific service account will `POST` a
`TokenRequest` to the `/token` subresource of that service account object.

### Modify `tokenreviews.authentication.k8s.io`

The TokenReview API will be extended to support passing an additional audience
field which the service account authenticator will validate.

```golang
type TokenReviewSpec struct {
// Token is the opaque bearer token.
Token string
// Audiences is the identifier that the client identifies as.
Audiences []string
}
```

### Example Flow

```
> POST /apis/v1/namespaces/default/serviceaccounts/default/token
> {
> "kind": "TokenRequest",
> "apiVersion": "authentication.k8s.io/v1",
> "spec": {
> "audience": [
> "https://kubernetes.default.svc"
> ],
> "validityDuration": "99999h",
> "boundObjectRef": {
> "kind": "Pod",
> "apiVersion": "v1",
> "name": "pod-foo-346acf"
> }
> }
> }
{
"kind": "TokenRequest",
"apiVersion": "authentication.k8s.io/v1",
"spec": {
"audience": [
"https://kubernetes.default.svc"
],
"validityDuration": "99999h",
"boundObjectRef": {
"kind": "Pod",
"apiVersion": "v1",
"name": "pod-foo-346acf"
}
},
"status": {
"token":
"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJz[payload omitted].EkN-[signature omitted]",
"expiration": "Jan 24 16:36:00 PST 3018"
}
}
```

The token payload will be:

```
{
"iss": "https://example.com/some/path",
"sub": "system:serviceaccount:default:default,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mikedanese , do you mean Subject here? Why is it removed from API definition now?😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a claim in the generated service account JWT. it is still present.

"aud": [
"https://kubernetes.default.svc"
],
"exp": 24412841114,
"iat": 1516841043,
"nbf": 1516841043,
"kubernetes.io": {
"serviceAccountUID": "c0c98eab-0168-11e8-92e5-42010af00002",
"boundObjectRef": {
"kind": "Pod",
"apiVersion": "v1",
"uid": "a4bb8aa4-0168-11e8-92e5-42010af00002",
"name": "pod-foo-346acf"
}
}
}
```

## Service Account Authenticator Modification

The service account token authenticator will be extended to support validation
of time and audience binding claims.

## ACLs for TokenRequest

The NodeAuthorizer will allow the kubelet to use its credentials to request a
service account token on behalf of pods running on that node. The
NodeRestriction admission controller will require that these tokens are pod
bound.

## Footnotes

* New apiserver flags:
* --service-account-issuer: Identifier of the issuer.
* --service-account-signing-key: Path to issuer private key used for signing.
* --service-account-api-audience: Identifier of the API. Used to validate
tokens authenticating to the Kubernetes API.
* The Kubernetes apiserver will identify itself as `kubernetes.default.svc`
which is the DNS name of the Kubernetes apiserver. When no audience is
requested, the audience is defaulted the audience is defaulted to an array
containing only this identifier.