Skip to content

Commit

Permalink
[wip] Sync halo2 lib 0.4.0 (#80)
Browse files Browse the repository at this point in the history
* feat: call synthesize in `MockProver` multiple times to behave same as real prover

* modify previous commit

* Expose mod `permutation` and re-export `permutation::keygen::Assembly` (privacy-scaling-explorations#149)

* feat: expose mod ule `permutation` and re-export `permutation::keygen::Assembly`

* feat: derive `lone` for `permutation::keygen::Assembly`

* feat: bump MSRV for `inferno`

* change: Migrate workspace to pasta_curves-0.5 (privacy-scaling-explorations#157)

* change: Migrate workspace to pasta_curves-0.5

This ports the majority of the workspace to the `pasta_curves-0.5.0`
leaving some tricky edge-cases that we need to handle carefully.

Resolves: privacy-scaling-explorations#132

* fix: Complete latest trait bounds to compile halo2proofs

* change: Migrate examples & benches to pasta 0.5

* change: Migrate halo2_gadgets to pasta-0.5

* change: Update gadgets outdated code with latest upstream

* fix: Sha3 gadget circuit

* fix: doc tests

* chore: Update merged main

* fix: Apply review suggestions

* fix previous commit

* Extend Circuit trait to take parameters in config (privacy-scaling-explorations#168)

* Extend Circuit trait to take parameters in config

The Circuit trait is extended with the following:
```
pub trait Circuit<F: Field> {
    /// [...]
    type Params: Default;

    fn params(&self) -> Self::Params {
        Self::Params::default()
    }

    fn configure_with_params(meta: &mut ConstraintSystem<F>, params: &Self::Params) -> Self::Config {
        Self::configure(meta)
    }

    fn configure(meta: &mut ConstraintSystem<F>) -> Self::Config;
}
```

This allows runtime parametrization of the circuit configuration.  The extension to the Circuit trait has been designed to minimize the breaking change: existing circuits only need to define the associated `type Params`.

Unfortunately "Associated type defaults" are unstable in Rust, otherwise this would be a non-breaking change.  See rust-lang/rust#29661

* Implement circuit params under feature flag

* Don't overwrite configure method

* Fix doc test

* Allow halo2 constraint names to have non static names (privacy-scaling-explorations#156)

* static ref to String type in Gates, Constraints, VirtualCell, Argument

* 'lookup'.to_string()

* return &str for gate name and constriant_name, also run fmt

* Update halo2_gadgets/Cargo.toml

Co-authored-by: Han <tinghan0110@gmail.com>

* upgrade rust-toochain

---------

Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: Han <tinghan0110@gmail.com>

* Improve halo2 query calls (privacy-scaling-explorations#154)

* return expression from cell

* add example

* selector

* recurse Expression to fill in index

* minimized changes from the original

* backword compatible meta.query_X & challange.expr()

* cargo fmt

* fixed lookup to pass all tests

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* update

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* add primitives.rs back

* remove example2

* backward compatible meta.query_X & Column.cur(), next(), prev(), at(usize)

* impl Debug & make side effects only when query.index.is_none()

* change impl Debug for Expression instead & revert test in plonk_api

* upgrade rust-toolchain

* Update halo2_proofs/src/plonk/circuit.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* Update halo2_proofs/src/plonk/circuit.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* ran clippy

* Update halo2_proofs/src/plonk/circuit.rs

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>
Co-authored-by: Han <tinghan0110@gmail.com>

* Implement Clone trait for Hash, Absorbing, and Sponge structs (privacy-scaling-explorations#171)

* fix: Fix serialization for VerifyingKey (privacy-scaling-explorations#178)

Now the value returned when the number of selectors is a multiple of 8
is correct.

Resolves: privacy-scaling-explorations#175

* Add more getters to expose internal fields

* add a constructor (privacy-scaling-explorations#164)

* add a constructor

* add more comment

* fix as review

* remove clone

* remove

* no need to use new variable

* change comment

* fix clippy

* rename to from_parts

* remove n declaration

* feat: send sync region (privacy-scaling-explorations#180)

* feat: send / sync region

* Update layout.rs

* update

* lol

* debug

* Update keygen.rs

* Update keygen.rs

* Update keygen.rs

* Update keygen.rs

* thread-safe-region feature flag

* cleanup

* patch dev-graph

* patch non-determinism in mapping creation

* reduce mem usage for vk and pk

* mock proving examples

* swap for hashmap for insertion speed

* reduce update overhead

* replace BTree with Vec

* add benchmarks

* make the benchmarks massive

* patch clippy

* simplify lifetimes

* patch benches

* Update halo2_proofs/src/plonk/permutation/keygen.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* Update halo2_proofs/examples/vector-mul.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* rm benches

* order once

* patch lints

---------

Co-authored-by: Han <tinghan0110@gmail.com>

* fix previous commit

* Fix `parallelize` workload imbalance (privacy-scaling-explorations#186)

* fix parallelize workload imbalance

* remove the need of unsafe

* Updates halo2_curves dependency to released package (privacy-scaling-explorations#190)

THe package release ressets the version from those inherited by the legacy
halo2curves repo's fork history.

The upstream diff is:
https://github.com/privacy-scaling-explorations/halo2curves/compare/9f5c50810bbefe779ee5cf1d852b2fe85dc35d5e..9a7f726fa74c8765bc7cdab11519cf285d169ecf

* fix: explicitly define mds diff type (privacy-scaling-explorations#196)

* fix: explicitly define mds diff type

* rm paren

* feat: expose `transcript_repr` of `VerifyingKey` and reduce the trait constraint (privacy-scaling-explorations#200)

* implement native shuffle argument and api

fix: remove nonsense comment

strictly check shuffle rows

address doc typos

move compression into product commitment

typo

add shuffle errors for `verify_at_rows_par`

dedup expression evaluation

cargo fmt

fix fields in sanity-checks feature

* feat: public cells to allow for implementations of custom `Layouter`  (privacy-scaling-explorations#192)

* feat: public cells

* Update mds.rs

* Update mds.rs

* Update single_pass.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* bump toolchain to resolve errors

* fix clippy errors for CI run

* rustfmt post clippy

* plz let it be the last lint

* patch clippy lints in gadgets

* clippy lints for sha256 bench

* patch halo2proof benches

* Update assigned.rs

* Update halo2_gadgets/src/poseidon/primitives/mds.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* Update halo2_gadgets/src/poseidon/primitives/mds.rs

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Han <tinghan0110@gmail.com>

* Synchronize with upstream (privacy-scaling-explorations#199)

* refactor: add default impl for `SyncDeps` for backward compatability

* feat: pick changes from zcash#728 and changes of flag `test-dev-graph`

* feat: pick changes from zcash#622

* feat: pick changes about mod `circuit` and mod `dev`

* feat: pick rest changes of `halo2_proofs`

* fix: when `--no-default-features`

* ci: sync from upstream, and deduplicate jobs when
push to `main`, and remove always failing job `codecov`.

* fix: make `commit_zk` runnable when `--no-default-features`

* chore: Update rust-toolchain to 1.66 for testing  (privacy-scaling-explorations#208)

* chore: Update rust-toolchain to 1.66 for testing

Note that tests will not compile due to the silent MSRV bump in
`blake2b_simd`.

Hence, we need to use `1.66` as toolchain.

Resolves: privacy-scaling-explorations#207

* change: UIpdate MSRVs in Cargo.toml

* fix: clippy (privacy-scaling-explorations#203)

* fix: clippy

* fmt

* fix: Final clippy complains & adjustments

---------

Co-authored-by: CPerezz <c.perezbaro@gmail.com>

* Implement Sum and Product for Expression (privacy-scaling-explorations#209)

* Make it Eq to make it easier for tests

* Implement Sum and Product for Expression

* Make it readable

* chore: update poseidon dependency

* fix: compiling bug with feautes=parallel_syn

* feat(MockProver): replace errors by asserts(privacy-scaling-explorations#150)

* boundary offset lost when resolving conflict

* disable multiphase prover

* Sync halo2 lib 0.4.0 merging (#81)

* Use thread pool for assign_regions (#57)

* feat: use rayon threadpool

* feat: add UT for many subregions

* refact: move common struct out to module level

* refact: reuse common configure code

* fix ci errors

---------

Co-authored-by: kunxian xia <xiakunxian130@gmail.com>

* Move `env_logger` dependency to dev-depdendencies (only for test). (#69)

* sync ff/group 0.13

* fix clippy

* fix clippy

* fmg

* [FEAT] Upgrading table16 for SHA256 (#73)

* upgrade sha256

* fix clippy

* Bus auto (#72)

* bus: expose global offset of regions

* bus-auto: add query_advice and query_fixed function in witness generation

* bus-auto: fix clippy

---------

Co-authored-by: Aurélien Nicolas <info@nau.re>

* fix-tob-scroll-21 (#59)

* fix-tob-scroll-21

* expose param field for re-randomization

* enable accessing for table16 (#75)

* chore: update poseidon link

* merge sha256 gadget changes

* Fix the CI errors (#78)

* cargo fmt

* fix clippy error

* Feat: switch to logup scheme for lookup argument  (#71)

* Multi-input mv-lookup. (#49)

* Add mv_lookup.rs

* mv_lookup::prover, mv_lookup::verifier

* Replace lookup with mv_lookup

* replace halo2 with mv lookup

Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>

* cleanups

Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>

* ConstraintSystem: setup lookup_tracker

Co-authored-by: Andrija <akinovak@gmail.com>

* mv_lookup::hybrid_prover

Co-authored-by: Andrija <akinovak@gmail.com>

* WIP

* mv_multi_lookup: enable lookup caching

Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

* Rename hybrid_lookup -> lookup

* Chunk lookups using user-provided minimum degree

Co-authored-by: Andrija <akinovak@gmail.com>

* mv_lookup bench

Co-authored-by: Andrija <akinovak@gmail.com>

* Introduce counter feature for FFTs and MSMs

Co-authored-by: Andrija <akinovak@gmail.com>

* Fix off-by-one errors in chunk_lookup

Co-authored-by: Andrija <akinovak@gmail.com>

* bench wip

* time evaluate_h

* KZG

* more efficient batch inversion

* extended lookup example

* Finalize mv lookup

Author: therealyingtong <yingtong.lai@gmail.com>

* Remove main/

* Fix according to the comments

* replace scan with parallel grand sum computation

* Revert Cargo.lock

* mv lookup Argument name

* parallel batch invert

---------

Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

* fmt

* fix unit test

* fix clippy errors

* add todo in mv_lookup's prover

* fmt and clippy

* fix clippy

* add detailed running time of steps in logup's prover

* fmt

* add more log hooks

* more running time logs

* use par invert

* use sorted-vector to store how many times a table element occurs in input

* par the process to get inputs_inv_sum

* use par

* fix par

* add feature to skip inv sums

* add new feature flag

* fix clippy error

---------

Co-authored-by: Sphere L <sph6r6.l1u@gmail.com>
Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

* fix some simple building errs

* upgrade pathfinder_simd to newer version as it can't compile on mac m1 pro

* resolve merge conflict

* fmt

* clippy

* more clippy fix

* more lint fix

* fmt

* minor syntax fix

* fix ipa multiopen test failure

* fix clippy warning

* fmt

* fix par scan of log_inv diff

* remove uncessary clone

---------

Co-authored-by: alannotnerd <alan1995wang@outlook.com>
Co-authored-by: kunxian xia <xiakunxian130@gmail.com>
Co-authored-by: Steven <asongala@163.com>
Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: zhenfei <zhenfei.zhang@hotmail.com>
Co-authored-by: Ho <noel.wei@gmail.com>
Co-authored-by: naure <naure@users.noreply.github.com>
Co-authored-by: Aurélien Nicolas <info@nau.re>
Co-authored-by: Sphere L <sph6r6.l1u@gmail.com>
Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

---------

Co-authored-by: han0110 <tinghan0110@gmail.com>
Co-authored-by: Velaciela <git.rover@outlook.com>
Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: Eduard S <eduardsanou@posteo.net>
Co-authored-by: CeciliaZ030 <45245961+CeciliaZ030@users.noreply.github.com>
Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>
Co-authored-by: Enrico Bottazzi <85900164+enricobottazzi@users.noreply.github.com>
Co-authored-by: Ethan-000 <s2026080@ed.ac.uk>
Co-authored-by: dante <45801863+alexander-camuto@users.noreply.github.com>
Co-authored-by: Mamy Ratsimbazafy <mamy_github@numforge.co>
Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
Co-authored-by: kilic <onurkilic1004@gmail.com>
Co-authored-by: Thor <7041313+thor314@users.noreply.github.com>
Co-authored-by: CPerezz <c.perezbaro@gmail.com>
Co-authored-by: chokermaxx <135603985+chokermaxx@users.noreply.github.com>
Co-authored-by: Zhang Zhuo <mycinbrin@gmail.com>
Co-authored-by: alannotnerd <alan1995wang@outlook.com>
Co-authored-by: kunxian xia <xiakunxian130@gmail.com>
Co-authored-by: Steven <asongala@163.com>
Co-authored-by: Ho <noel.wei@gmail.com>
Co-authored-by: naure <naure@users.noreply.github.com>
Co-authored-by: Aurélien Nicolas <info@nau.re>
Co-authored-by: Sphere L <sph6r6.l1u@gmail.com>
Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>
  • Loading branch information
1 parent 84003a2 commit 0008b86
Show file tree
Hide file tree
Showing 115 changed files with 5,234 additions and 1,826 deletions.
79 changes: 75 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,25 @@
name: CI checks

on: [pull_request, push]
on:
merge_group:
pull_request:
push:
branches:
- main

jobs:
test:
name: Test on ${{ matrix.os }}
name: Test on ${{ matrix.os }} with ${{ matrix.feature_set }} features
runs-on: ${{ matrix.os }}
strategy:
matrix:
feature_set: [basic, all]
os: [ubuntu-latest, windows-latest, macOS-latest]
include:
- feature_set: basic
features: batch,dev-graph,gadget-traces
- feature_set: all
features: batch,dev-graph,gadget-traces,multicore,test-dev-graph,thread-safe-region,sanity-checks,circuit-params

steps:
- uses: actions/checkout@v3
Expand All @@ -19,8 +30,67 @@ jobs:
uses: actions-rs/cargo@v1
with:
command: test
args: --verbose --release --all --all-features
if: ${{ false }}
args: --verbose --release --workspace --no-default-features --features "${{ matrix.features }}"

build:
name: Build target ${{ matrix.target }}
runs-on: ubuntu-latest
strategy:
matrix:
target:
- wasm32-unknown-unknown
- wasm32-wasi

steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
override: false
- name: Add target
run: rustup target add ${{ matrix.target }}
- name: cargo build
uses: actions-rs/cargo@v1
with:
command: build
args: --no-default-features --features batch,dev-graph,gadget-traces --target ${{ matrix.target }}

bitrot:
name: Bitrot check
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
override: false
# Build benchmarks to prevent bitrot
- name: Build benchmarks
uses: actions-rs/cargo@v1
with:
command: build
args: --benches --examples --all-features

doc-links:
name: Intra-doc links
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
override: false
- name: cargo fetch
uses: actions-rs/cargo@v1
with:
command: fetch

# Ensure intra-documentation links all resolve correctly
# Requires #![deny(intra_doc_link_resolution_failure)] in crates.
- name: Check intra-doc links
uses: actions-rs/cargo@v1
with:
command: doc
args: --all --document-private-items

fmt:
name: Rustfmt
Expand All @@ -36,3 +106,4 @@ jobs:
with:
command: fmt
args: --all -- --check

106 changes: 0 additions & 106 deletions .github/workflows/ci_main.yml

This file was deleted.

Loading

0 comments on commit 0008b86

Please sign in to comment.