Add datadog span processor. #6092
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
CI performance tests
|
BrynCooke
force-pushed
the
bryn/datadog-span-processor
branch
from
56bacac
to
3a60672
Compare
Move the datadog agent processor out to separate files.
BrynCooke
force-pushed
the
bryn/datadog-span-processor
branch
from
3a60672
to
6b198cb
Compare
BrynCooke
commented
Oct 1, 2024
| @@ -239,7 +239,7 @@ async fn test_otlp_request_with_zipkin_trace_context_propagator_with_datadog( | |||
| .await?; | |||
| // ---------------------- | |||
| // Testing for a trace containing a tracestate m and psr with psr set to 1 for DD and an unsampled trace, so it should be sent to the otlp exporter with sampling priority set to 1 | |||
| // But it should send the trace to subgraph as the tracestate contains a psr set to 1 | |||
| // It should not send the trace to the subgraph as we didn't use the datadog propagator and therefore the trace will remain unsampled. | |||
There was a problem hiding this comment.
@bnjjj This is a key test change. As we're not using datadog propagation I don't think the subgraph should pick up the trace.
There was a problem hiding this comment.
Are we sure we should not take care of this tracestate ? Do we have any specs or docs to help us to choose ?
There was a problem hiding this comment.
BrynCooke
commented
Oct 1, 2024
| // The propagator gets first dibs on setting the trace state, so if it sets it, we don't override it unless we are not parent based. | ||
| match result.decision { | ||
| SamplingDecision::Drop | SamplingDecision::RecordOnly => { | ||
| result.decision = SamplingDecision::RecordOnly; |
There was a problem hiding this comment.
@bnjjj This is the only change to the logic
bnjjj
reviewed
Oct 1, 2024
| } | ||
|
|
||
| fn on_end(&self, mut span: SpanData) { | ||
| let new_trace_flags = span.span_context.trace_flags().with_sampled(true); |
There was a problem hiding this comment.
Should we check for either sampling priority to be set or measured to be set to set it to true ? I know for now we don't have that case but should we already try to handle this ?
| @@ -239,7 +239,7 @@ async fn test_otlp_request_with_zipkin_trace_context_propagator_with_datadog( | |||
| .await?; | |||
| // ---------------------- | |||
| // Testing for a trace containing a tracestate m and psr with psr set to 1 for DD and an unsampled trace, so it should be sent to the otlp exporter with sampling priority set to 1 | |||
| // But it should send the trace to subgraph as the tracestate contains a psr set to 1 | |||
| // It should not send the trace to the subgraph as we didn't use the datadog propagator and therefore the trace will remain unsampled. | |||
There was a problem hiding this comment.
Are we sure we should not take care of this tracestate ? Do we have any specs or docs to help us to choose ?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes the datadog agent sampling feature use Record rather than RecordAndSample for spans that would otherwise have been dropped.
This allows us to stay closer to the original telemetry logic for setting the sampled trace flag.
This complexity arrises because BatchSPanProcessors usually drop spans that are not sampled.
To get around this we introduce a wrapper that is active for otlp and datadog exporters if agent sampling is enabled that updates the spandata to say that it was sampled.
Checklist
Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.
Exceptions
Note any exceptions here
Notes
Footnotes
It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this. ↩
Configuration is an important part of many changes. Where applicable please try to document configuration examples. ↩
Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions. ↩