Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: geospatial types support #4208

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
f3784ac
feat: geospatial types support
Oreilles Nov 23, 2023
e605279
chore: add postgis database to query engine test matrix
Oreilles Sep 8, 2023
0a750da
fix: mysql geometries should be serialized to wkb by quaint
Oreilles Sep 8, 2023
9844033
fix: failing inequality operator with PostGIS
Oreilles Sep 8, 2023
b267f61
fix: PostGIS tests need to happen with the public schema exposed
Oreilles Sep 8, 2023
2aaf40b
fix: add missing GeometryFiltering capability constraint to geometric…
Oreilles Sep 8, 2023
b9f411f
fix: srid introspection in postgres dialect
Oreilles Sep 9, 2023
f42f3ba
fix: handle geography default srid value inconsistency in tests
Oreilles Sep 9, 2023
c44ee7a
fix: remove unused variable
Oreilles Sep 9, 2023
c16fcd7
fix: geography srid introspection test in postgres dialect
Oreilles Sep 9, 2023
6f2db0f
fix: actually fix geography srid introspection test in postgres dialect
Oreilles Sep 9, 2023
a5097de
fix: simplify mysql srid introspection on mysql and fixes vitess tests
Oreilles Sep 9, 2023
98c913f
fix: add missing MySQL version constraint for test requiring support …
Oreilles Nov 12, 2023
e287c38
fix: ignore unsuccessful initialization of SpatiaLite in SQLite query…
Oreilles Nov 12, 2023
4e4308c
fix: replace intersection test due to MySQL 5.6 quirk
Oreilles Nov 13, 2023
16f004c
fix: formatting
Oreilles Nov 19, 2023
71dae2e
fix: make Spatialite geometry filters stricter and tighten geometry f…
Oreilles Nov 19, 2023
93f582c
fix: add SQLite test versions 3 and 3-spatialite, only run query engi…
Oreilles Nov 25, 2023
6cd70be
chore: add Spatialite tests to CI
Oreilles Nov 24, 2023
e6185be
fix: incorrect dependency order
Oreilles Nov 26, 2023
42e5914
fix: add missing Spatialite system table to the list
Oreilles Nov 26, 2023
e1da2f7
fix: null_geometry return value
Oreilles Nov 26, 2023
5a255ac
refactor: share SQLITE_SYSTEM_TABLES with sql-schema-connector
Oreilles Nov 28, 2023
1290ef7
refactor: execute InitSpatialMetaData in quaint
Oreilles Nov 28, 2023
bb7e33c
fix: Spatialite diff migration tests
Oreilles Nov 28, 2023
455b274
fix: formatting
Oreilles Nov 28, 2023
35a9f15
fix: revert Sqlite total queries to 9
Oreilles Nov 28, 2023
7b8348c
fix: install Spatialite with sudo
Oreilles Nov 28, 2023
31228b5
fix: add WGS84_ONLY option to `InitSpatialMetadata` and only call the…
Oreilles Nov 29, 2023
8a5288b
fix: formatting
Oreilles Nov 29, 2023
e0f834d
fix: skip geometric_comparison_filters test for MariaDB while we figu…
Oreilles Nov 30, 2023
f7f2892
refactor: return conversion error in GeometryValue::from_str
Oreilles Nov 30, 2023
f3fa6ef
Merge remote-tracking branch 'prisma/main' into feat/geometry
Oreilles Aug 4, 2024
858653f
refactor: simplify spatialite CI setup
Oreilles Aug 4, 2024
9feff00
Merge remote-tracking branch 'prisma/main' into feat/geometry
Oreilles Sep 25, 2024
f7f25c9
Fix toolchain, features and lint
Oreilles Sep 25, 2024
0a879b4
Add JSArgType::Geometry
Oreilles Sep 30, 2024
e8658e9
Fix: fmt
Oreilles Sep 30, 2024
f74d4de
Fix test_native_types_multifile
Oreilles Sep 30, 2024
33048e9
Update wasm toolchain
Oreilles Sep 30, 2024
56644f3
Fix PostGIS version test
Oreilles Sep 30, 2024
a3db88e
Add Geometry handling in quaint from_type_identifier
Oreilles Sep 30, 2024
a16d952
Use SRID 4326 in Sqlite introspection tests
Oreilles Sep 30, 2024
ff3e9aa
Update test unknown_type_mysql
Oreilles Sep 30, 2024
9f205ed
Add GeoJson handling in MongoDB into_bson
Oreilles Sep 30, 2024
7d6d960
Fix geometry_type_mysql
Oreilles Sep 30, 2024
2b35a94
Use SRID 4326 in spatialite describer tests
Oreilles Oct 1, 2024
fffdacc
Add cfd1 and libsql.js.wasm to spatialite tests exclude list
Oreilles Oct 1, 2024
0c912c0
Add cfd1 and libsql.js.wasm to other spatialite tests exclude list
Oreilles Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/test-query-engine-template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,10 @@ jobs:
- name: "Start ${{ inputs.name }} (${{ matrix.engine_protocol }})"
run: make start-${{ inputs.name }}

- name: Install Spatialite
if: ${{ inputs.name == 'spatialite' }}
run: sudo apt install -y libsqlite3-mod-spatialite && echo "SPATIALITE_PATH=mod_spatialite" >> $GITHUB_ENV

- run: export WORKSPACE_ROOT=$(pwd) && cargo nextest run -p query-engine-tests --partition hash:${{ matrix.partition }} --test-threads=1
if: ${{ inputs.single_threaded }}
env:
Expand Down
17 changes: 13 additions & 4 deletions .github/workflows/test-query-engine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ jobs:
version: "10"
- name: "postgres9"
version: "9"
- name: 'postgis16'
version: '16-postgis'
uses: ./.github/workflows/test-query-engine-template.yml
name: postgres ${{ matrix.database.version }}
with:
Expand Down Expand Up @@ -81,7 +83,6 @@ jobs:
matrix:
database:
- name: "cockroach_23_1"
connector: "cockroachdb"
version: "23.1"
- name: "cockroach_22_2"
version: "22.2"
Expand Down Expand Up @@ -137,11 +138,19 @@ jobs:
relation_load_strategy: '["query"]'

sqlite:
strategy:
fail-fast: false
matrix:
database:
- name: "sqlite"
version: "3"
- name: "spatialite"
version: "3-spatialite"
uses: ./.github/workflows/test-query-engine-template.yml
name: sqlite
name: sqlite ${{ matrix.database.version }}
with:
name: "sqlite"
version: 3
name: ${{ matrix.database.name }}
version: ${{ matrix.database.version }}
connector: "sqlite"
relation_load_strategy: '["query"]'

Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/test-schema-engine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ jobs:
url: "postgresql://postgres:prisma@localhost:5438"
- name: postgres16
url: "postgresql://postgres:prisma@localhost:5439"
- name: postgis16
url: 'postgresql://postgres:prisma@localhost:5440'
- name: cockroach_23_1
url: "postgresql://prisma@localhost:26260"
- name: cockroach_22_2
Expand All @@ -99,6 +101,8 @@ jobs:
url: "postgresql://prisma@localhost:26257"
- name: sqlite
url: sqlite
- name: spatialite
url: sqlite
- name: vitess_8_0
url: "mysql://root:prisma@localhost:33807/test"
shadow_database_url: "mysql://root:prisma@localhost:33808/shadow"
Expand All @@ -124,6 +128,10 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Install Spatialite
if: ${{ matrix.database.name == 'spatialite' }}
run: sudo apt install -y libsqlite3-mod-spatialite && echo "SPATIALITE_PATH=mod_spatialite" >> $GITHUB_ENV

- name: "Start ${{ matrix.database.name }}"
run: make start-${{ matrix.database.name }}

Expand Down
1 change: 1 addition & 0 deletions .test_database_urls/postgis_16
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export TEST_DATABASE_URL="postgresql://postgres:prisma@localhost:5440"
89 changes: 89 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,11 @@ start-sqlite:
dev-sqlite:
cp $(CONFIG_PATH)/sqlite $(CONFIG_FILE)

start-spatialite:

dev-spatialite:
cp $(CONFIG_PATH)/spatialite $(CONFIG_FILE)

dev-react-native:
cp $(CONFIG_PATH)/react-native $(CONFIG_FILE)

Expand Down Expand Up @@ -248,6 +253,12 @@ start-postgres16:
dev-postgres16: start-postgres16
cp $(CONFIG_PATH)/postgres16 $(CONFIG_FILE)

start-postgis16:
docker compose -f docker-compose.yml up -d --remove-orphans postgis16

dev-postgis16: start-postgis16
cp $(CONFIG_PATH)/postgis16 $(CONFIG_FILE)

start-cockroach_23_1:
docker compose -f docker-compose.yml up --wait -d --remove-orphans cockroach_23_1

Expand Down
13 changes: 13 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,19 @@ services:
networks:
- databases

postgis16:
image: postgis/postgis:16-3.4
restart: unless-stopped
command: postgres -c 'max_connections=1000'
environment:
POSTGRES_PASSWORD: "prisma"
POSTGRES_HOST_AUTH_METHOD: "md5"
POSTGRES_INITDB_ARGS: "--auth-host=md5"
ports:
- "5440:5432"
networks:
- databases

mysql-5-6:
image: mysql:5.6.50
command: mysqld --table_definition_cache=2000
Expand Down
4 changes: 4 additions & 0 deletions libs/prisma-value/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ pub enum PrismaValue {
Uuid(Uuid),
List(PrismaListValue),
Json(String),
GeoJson(String),
Geometry(String),

/// A collections of key-value pairs constituting an object.
#[serde(serialize_with = "serialize_object")]
Expand Down Expand Up @@ -330,6 +332,8 @@ impl fmt::Display for PrismaValue {
PrismaValue::Null => "null".fmt(f),
PrismaValue::Uuid(x) => x.fmt(f),
PrismaValue::Json(x) => x.fmt(f),
PrismaValue::GeoJson(x) => x.fmt(f),
PrismaValue::Geometry(x) => x.fmt(f),
PrismaValue::BigInt(x) => x.fmt(f),
PrismaValue::List(x) => {
let as_string = format!("{x:?}");
Expand Down
9 changes: 7 additions & 2 deletions libs/test-setup/src/postgres.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,14 @@ pub(crate) fn get_postgres_tags(database_url: &str) -> Result<BitFlags<Tags>, St
let fut = async {
let quaint = Quaint::new(database_url).await.map_err(|err| err.to_string())?;
let mut tags = Tags::Postgres.into();
let version = quaint.version().await.map_err(|err| err.to_string())?;

match version {
if let Ok(_postgis_version) = quaint.query_raw("SELECT PostGIS_version()", &[]).await {
tags |= Tags::PostGIS;
}

let postgres_version = quaint.version().await.map_err(|err| err.to_string())?;

match postgres_version {
None => Ok(tags),
Some(version) => {
eprintln!("version: {version:?}");
Expand Down
18 changes: 18 additions & 0 deletions libs/test-setup/src/sqlite.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
use enumflags2::BitFlags;
use once_cell::sync::Lazy;
use quaint::{prelude::Queryable, single::Quaint};

use crate::{runtime::run_with_thread_local_runtime as tok, Tags};

pub fn sqlite_test_url(db_name: &str) -> String {
std::env::var("SQLITE_TEST_URL").unwrap_or_else(|_| format!("file:{}", sqlite_test_file(db_name)))
Expand All @@ -25,3 +29,17 @@ fn sqlite_test_file(db_name: &str) -> String {

file_path.to_string_lossy().into_owned()
}

pub(crate) fn get_sqlite_tags() -> Result<BitFlags<Tags>, String> {
let fut = async {
let mut tags: BitFlags<Tags> = Tags::Sqlite.into();
// The SpatiaLite extension is loaded by quaint, assuming the SPATIALITE_PATH env variable is set
// If the extension can be loaded in a dummy database, it means it will also be available for the tests
let quaint = Quaint::new_in_memory().map_err(|err| err.to_string())?;
if let Ok(_has_spatialite) = quaint.query_raw("SELECT spatialite_version();", &[]).await {
tags |= Tags::Spatialite;
}
Ok(tags)
};
tok(fut)
}
2 changes: 2 additions & 0 deletions libs/test-setup/src/tags.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ tags![
CockroachDb222 = 1 << 20,
CockroachDb231 = 1 << 21,
Postgres16 = 1 << 22,
PostGIS = 1 << 23,
Spatialite = 1 << 24,
];

pub fn tags_from_comma_separated_list(input: &str) -> BitFlags<Tags> {
Expand Down
4 changes: 2 additions & 2 deletions libs/test-setup/src/test_api_args.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use crate::{logging, mssql, mysql, postgres, Capabilities, Tags};
use crate::{logging, mssql, mysql, postgres, sqlite, Capabilities, Tags};
use enumflags2::BitFlags;
use once_cell::sync::Lazy;
use quaint::single::Quaint;
Expand Down Expand Up @@ -41,7 +41,7 @@ static DB_UNDER_TEST: Lazy<Result<DbUnderTest, String>> = Lazy::new(|| {
match prefix {
"file" | "sqlite" => Ok(DbUnderTest {
database_url,
tags: Tags::Sqlite.into(),
tags: sqlite::get_sqlite_tags()?,
capabilities: Capabilities::CreateDatabase.into(),
provider: "sqlite",
shadow_database_url,
Expand Down
2 changes: 1 addition & 1 deletion libs/user-facing-errors/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ impl Error {

/// Construct a new UnknownError from a `PanicInfo` in a panic hook. `UnknownError`s created
/// with this constructor will have a proper, useful backtrace.
pub fn new_in_panic_hook(panic_info: &std::panic::PanicInfo<'_>) -> Self {
pub fn new_in_panic_hook(panic_info: &std::panic::PanicHookInfo<'_>) -> Self {
let message = panic_info
.payload()
.downcast_ref::<&str>()
Expand Down
1 change: 1 addition & 0 deletions libs/user-facing-errors/src/schema_engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ pub struct DestructiveMigrationDetected {
code = "P3002",
message = "The attempted migration was rolled back: {database_error}"
)]
#[allow(dead_code)]
struct MigrationRollback {
pub database_error: String,
}
Expand Down
1 change: 1 addition & 0 deletions prisma-fmt/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,7 @@ pub fn validate(validate_params: String) -> Result<(), String> {
}

/// Given a list of Prisma schema files (and their locations), returns the merged schema.
///
/// This is useful for `@prisma/client` generation, where the client needs a single - potentially large - schema,
/// while still allowing the user to split their schema copies into multiple files.
/// Internally, it uses `[validate]`.
Expand Down
Loading
Loading