Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for truly sorted API's where replication_key is not an incremental one #729

Closed
ilkkapeltola opened this issue Jun 17, 2022 · 0 comments · Fixed by #730
Closed

Support for truly sorted API's where replication_key is not an incremental one #729

ilkkapeltola opened this issue Jun 17, 2022 · 0 comments · Fixed by #730

Comments

@ilkkapeltola
Copy link
Contributor

ilkkapeltola commented Jun 17, 2022

My need comes from trying to build a tap for the UserFlow API.
Documentation: https://userflow.com/docs/api#pagination

In short, the API allows to order by a created timestamp, but does not allow filtering based on that.
The API supports pagination based on a uuid - which isn't incremental.

The current SDK behavior is that if I define the id as the replication_key, something like ff31e109-f873-41b6-9f1c-2d88b5bf6a2a will be tracked as the "latest record", which isn't true. It just happens to be the "highest" because of the 'ff...' in front.

So, what I would instead want to have is an API where is_sorted = True, but that would not check for the replication_key to be an increment, and instead trust that the last one seen is the most recent one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants