-
Notifications
You must be signed in to change notification settings - Fork 335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[issue #807] dlq topic producer options #809
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just had one in-line question.
pulsar/dlq_router.go
Outdated
|
||
// the origin code sets to LZ4 compression with no options | ||
// so the new design allows compression type to be overwritten but still set lz4 by default | ||
if r.policy.DLQProducerOptions.CompressionType == NoCompression { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are there any cases where a user would want to explicitly set to NoCompression?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The current implementation complies with the default LZ4 compression type in the previous implementation. So user cannot choose NoCompression. If NoCompression is required (I would prefer data has to be sent compressed as the original design), we might have to introduce another option to maintain backward compatibility.
DLQ and RLQ are created together, and the same data goes first to RLQ and then to DLQ,so the producer configuration of RLQ also needs to be modified? pulsar-client-go/pulsar/retry_router.go Lines 128 to 132 in 6a8e7f3
|
Added code to use the same ProducerOptions for both DLQ and RLQ.
|
May I ask which version this pull will be megred in? |
@wolfstudy @michaeljmarshall Can you help review this PR? @ne1llee is asking when to merge this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Fixes #807
Motivation
To customize producer options for DLQ topics.
Modifications
Add DLQProducerOptions to consumer's DLQ policy.
Verifying this change
(Please pick either of the following options)
The existing DLQ test is enhanced to cover customized producer options.
Does this pull request potentially affect one of the following parts:
A new producer options field,
DLQProuducerOptions
, is introduced for DLQ policy to govern the producer options.Documentation