Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a data_version to the db_write hook an implement optimistic locking #3358

Merged
merged 6 commits into from
Jan 2, 2020
Merged

Add a data_version to the db_write hook an implement optimistic locking #3358

merged 6 commits into from
Jan 2, 2020

Commits on Dec 27, 2019

  1. db: Consolidate access to the changes in a db 

    We were passing them in separately, while we could just retrieve them from the
db instance instead.
    @cdecker
    cdecker committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    c065503 View commit details
    Browse the repository at this point in the history

  2. db: Move db_migrate transaction up one level 

    We are about to do some more operations before committing, so moving this up
allows us to reuse the same transaction.
    @cdecker
    cdecker committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    d84f5a9 View commit details
    Browse the repository at this point in the history

  3. db: Add tracking of whether the current transaction is dirty

    @cdecker
    cdecker committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    c6c543c View commit details
    Browse the repository at this point in the history

  4. db: Add numeric data_version counter to count modifying transactions 

    This counter is incremented on each dirty transaction.
    @cdecker
    cdecker committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    73a5d5a View commit details
    Browse the repository at this point in the history

  5. db: Track the data_version in the database 

    This increments the `data_version` upon committing dirty transactions, reads
the last data_version upon startup, and tracks the number in memory in
parallel to the DB (see next commit for rationale).

Changelog-Changed: JSON-RPC: Added a `data_version` field to the `db_write` hook which returns a numeric transaction counter.
    @cdecker
    cdecker committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    1285d26 View commit details
    Browse the repository at this point in the history

  6. db: Turn the transaction counter into an optimistic lock 

    The optimistic lock prevents multiple instances of c-lightning making
concurrent modifications to the database. That would be unsafe as it messes up
the state in the DB. The optimistic lock is implemented by checking whether a
gated update on the previous value of the `data_version` actually results in
an update. If that's not the case the DB has been changed under our feet.

The lock provides linearizability of DB modifications: if a database is
changed under the feet of a running process that process will `abort()`, which
from a global point of view is as if it had crashed right after the last
successful commit. Any process that also changed the DB must've started
between the last successful commit and the unsuccessful one since otherwise
its counters would not have matched (which would also have aborted that
transaction). So this reduces all the possible timelines to an equivalent
where the first process died, and the second process recovered from the DB.

This is not that interesting for `sqlite3` where we are also protected via the
PID file, but when running on multiple hosts against the same DB, e.g., with
`postgres`, this protection becomes important.

Changelog-Added: DB: Optimistic logging prevents instances from running concurrently against the same database, providing linear consistency to changes.
    @cdecker
    cdecker committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    8076248 View commit details
    Browse the repository at this point in the history