Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] History api #409

Closed
wants to merge 94 commits into from
Closed

[WIP] History api #409

wants to merge 94 commits into from

Conversation

Florimond
Copy link
Member

[WIP]

kelindar and others added 27 commits September 7, 2023 19:53
This PR adds support for MQTT Retain flag. In order to make retained messages play well with emitter's message history, we consider that the retained messages are just messages with a large TTL (defaults to `30 days` ). The `retain` TTL is configurable in the storage provider, so you can setup a longer or shorter time.

1. The retain flag will only work if the key has `store` permission
2. The subscriptions with `load` permission will default to `last=1` . This is done in order to be compliant with the MQTT spec as the retain message should always be sent to the subscriptions.

Now, you can still have `ttl` and `last` options and it should play well with the retained messages, sending everything.
* add max message size as configurable

* fix review comments - add limit structure and omitempty

* improve comments

* fix failing test cases
* support of shared subscriptions

* share as per spec

* fix short subs
* updated dependencies

* update deps

* keeping vendor for now

* tidy up
* fix large N subscriptions

* export Contains

* cleanup a bit
So that a status can be requested without passing the "changes" parameter and therefore without touching the Trie.
Signed-off-by: CodeLingo Bot <bot@codelingo.io>
So that when subscribing to presence messages without request a status, there isn't a status message with empty Who array sent anymore.
* add prometheus support

* refactored to manage own registry

* go mod tidy

* removed ProcessCollector as this is only supported on linux
* embed keygen form

* Added additional tests

* Updated generation responses, fixed router subscription

* added html template
kelindar and others added 26 commits September 7, 2023 19:53
* use slice index instead of copy of 2 value in range

* fix #305
* Updated dependencies and LWWSet

* mesh

* remove unsafe pointer arithmetic
* Added v3 license with shuffled salsa cipher

* add workflows

* edge just on push

* separate workflows

* fix test error

* master only
* Fixes presence notification when peer goes offline

* add peer name to event

* improved subscription encoding

* fix test

* fix test
* Refactored survey

* fix merge
* added durable CRDT

* Add cache for durable

* decrement conn
* Added support to block/unblock keys

* avoid unnecessary notifications
This PR adds an implementation of "Last Will & Testament" of MQTT to emitter. This handles 2 scenarios:

 1. When client is terminated but the broker is alive.
 2. When the broker is terminated.

In both cases, last will will be sent to the cluster and optionally message will be retained. This uses consistent hashing to determine which peer will send out the last will in case of the broker termination.

Moreover, I have done a massive refactoring of the service (long overdue). In this refactoring, all of the handlers and services have been made simpler with clear dependencies (see service/ folder). In addition, all tests for the handlers have been rewritten and they have 100% code coverage.
* upgrade deps

* upgrade to 1.16

* upgrade coveralls
CompareAndSwapUint32 instead of just an atomic load of the value,
and changing the value later, which was not producing the needed
behavior in terms of atomicity.

Co-authored-by: Roman Atachiants <roman.atachiants@gmail.com>
* replace inmemory storage

* update deps
AppVeyor is not being used anymore
[WIP] Add history api

The message history of a channel can be retrieved when subscribing.
This API allows requesting the history independently of the
subscription request.

The code essentially comes from the the subcription handler.

TODO:
- Write tests.
- Actually test to see if it even works, this is just a draft.

[WIP] Limiter

[WIP] New limiter technique
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.