Config yaml: where we're heading

[BrynCooke]

The tower branch currently has the following YAML format:

server:
  listen: 1.2.3.4:5
  cors:
    origins: [foo, bar, baz]
    methods: [foo, bar]

subgraphs:
  example:
    layers: 
    - kind: header_propagation # There can be multiple instances of a layer
      header_name: "foo"

plugins:
  opentelemetry: # There can be only one instance of a plugin of a particular name
    some_config: "foo"

The restriction around plugin instance is only in the config format. The plugin registry allows for any number of instances to be created, but should we allow this?

Some things to discuss:

• What does WASM/Deno config look like?
• How can we support multiple service implementations/configuration? e.g. Rest

[BrynCooke]

Proposed format with WASM plugin/layers:

subgraphs:
  example:
    layers: 
       - kind: wasm # There can be multiple instances of a layer. But users should be encouraged to use this layer only once if possible.
         files:
          - "layer_1.js"
          - "layer_2.js"         

plugins:
  wasm: # There can be only one instance of a plugin of a particular name
    files:
    - "plugin_1.js"
    - "plugin_2.js"

In this format we don't allow embedding of code in the config because:

1. Although embedding code in the config seems nice it will quickly become unwieldy.
2. Having a single plugin/layer that delegates to multiple WASM files prevents us from having to cross the Rust->WASM boundary many times.

There is no escaping that if the user has a WASM plugin and a WASM layer then the cost of crossing from Rust to WASM will be incurred twice. There may be other layers that are non-wasm between the plugin and layer so this is unavoidable without making the ordering of layer execution opaque to the user.

[BrynCooke]

Proposed format with custom service:

subgraphs:
  example:
    service:
      kind: rest
      mapping: "foo"

Basically the same as our current layer stuff.
Defaults to remote_http is no subgraph service is defined.

[BrynCooke]

Proposed format header manipulation layer:

subgraphs:
  example:
    layers: 
       - kind: headers # There can be multiple instances of a layer. But users should be encouraged to use this layer only once if possible.
         delete: "foo"
         forward: 
            from: "foo"
            to: "bar"
            default: "bif"
         insert:
            name: "foo"
            value: "bar"

This would change the demonstration propagate headers layer to accept multiple operations and would reduce but not eliminate the need to box as the user will only be using one layer.

[o0Ignition0o]

side thought: I suspect one would like to have one yml per plugin and a way to knit them together ? eg:

our_amazing_tracing_related_header_propagation_rules.yml
ensure_security_headers_arent_propagated.yml

etc etc

[abernix]

@EverlastingBugstopper ☝️ It'd be good for you to have your eyes on this YAML design probably. If both Rover and Rover have subgraphs sections, I could imagine we'd be well-served to think about their overlap (cognitively and functionally!) — I think you've given this some thought already!

[EverlastingBugstopper]

Yes! None of what's laid out here stomps on what I've been thinking on (though I have been thinking it could be useful to have more than one supergraph in a project for development purposes which is something to keep in mind.)

I also have the code for the config that Rover uses in the federation-rs repo, so it would be good to get all of the router stuff in there and just make sure we're doing everything the same (or at least similarly, maybe the router only operates on one supergraph at a time so wouldn't need the "multiple supergraphs" config that I think we want for Rover).

[BrynCooke]

On thing that is changing from our old config is routing_url is disappearing. Instead a layer is used to change the URL of the subgraph request.

This cleans up the confusion about where subgraph URL should be fetched from, the supergraph schema or the config. The new config format makes it clear that it is the supergraph schema. users can still change the target URL, but it's a transformation on the pipeline.

We could of course add support for routing_url, but it'd be a shame to not clear this up once and for all.

[garypen]

• I wonder if we need some kind of versioning information for:
  • Overall File
  • Individual Components

How should we document what the appropriate verbs and parameters for a layer/plugin are? Is that an interface that we should support in a layer (i.e. should there be a way to interrogate a layer and extract its interface?)? Or should it be done via documentation?

[bnjjj]

Maybe we could adopt the same strategy than docker-compose put a version in each files (if we have multiple files) and then by having the router version in files we know how to handle it and we could have a kind of retro-compatibility by doing this.

[BrynCooke]

Created #438 for versioning.

[BrynCooke]

Using JSON Schema would be an option to investigate. This can be generated automatically based on the plugins/layers that are present.
Not sure how this fits in with versioning though.