forked from apache/datafusion-java
-
Notifications
You must be signed in to change notification settings - Fork 0
Permalink
Choose a base ref
{{ refName }}
default
Choose a head ref
{{ refName }}
default
Checking mergeability…
Don’t worry, you can still create the pull request.
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also or
learn more about diff comparisons.
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
Learn more about diff comparisons here.
base repository: devoopsman45/datafusion-java
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Could not load branches
Nothing to show
Loading
Could not load tags
Nothing to show
{{ refName }}
default
Loading
...
head repository: apache/datafusion-java
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Could not load branches
Nothing to show
Loading
Could not load tags
Nothing to show
{{ refName }}
default
Loading
- 4 commits
- 44 files changed
- 3 contributors
Commits on Jun 9, 2026
-
feat(session): add tableExists and deregisterTable to SessionContext (a…
…pache#102) ## Which issue does this PR close? - Closes apache#101 . ## Rationale for this change SessionContext can let a caller register tables but there is no way to Check whether a table name is already registered Remove a registered table from the session This makes it tough to write safer code while registering ## What changes are included in this PR? - `SessionContext.tableExists(String name)` — returns true if a table with that name is registered in the session - `SessionContext.deregisterTable(String name)` — removes a registered table; no-op if the name is not found Both are thin JNI wrappers over DataFusion's existing `SessionContext::table_exist` and `SessionContext::deregister_table` on the Rust side. ## Are these changes tested? Yes. `SessionContextTableRegistrationTest` covers: - tableExists returns false for an unregistered table - tableExists returns true after registerCsv - deregisterTable removes a registered table - deregisterTable is a no-op for an unregistered table - Both methods throw IllegalStateException on a closed context --> ## Are there any user-facing changes? Yes — two new public methods on `SessionContext`. Additive only, no breaking changes.
Configuration menu - View commit details
-
Copy full SHA for 60351ba - Browse repository at this point
Copy the full SHA 60351baView commit details
Commits on Jun 10, 2026
-
Configuration menu - View commit details
-
Copy full SHA for 051b00b - Browse repository at this point
Copy the full SHA 051b00bView commit details
Commits on Jun 13, 2026
-
Configuration menu - View commit details
-
Copy full SHA for ab57cb9 - Browse repository at this point
Copy the full SHA ab57cb9View commit details
Commits on Aug 6, 2026
-
chore(deps): bump DataFusion to 54.1.0 (apache#114)
## Which issue does this PR close? N/A — routine dependency bump; no tracking issue was filed. ## Rationale for this change Keeps the binding current with upstream DataFusion. 54.1.0 is the latest release line, and staying close to it keeps the next bump small and makes upstream fixes available to Java callers. ## What changes are included in this PR? Bumps `datafusion`, `datafusion-proto`, `datafusion-spark` and `datafusion-substrait` from 53.1.0 to 54.1.0. The `datafusion.version` Maven property moves in lock step, since it selects the upstream tag the `datafusion.proto` / `datafusion_common.proto` definitions are downloaded from — the generated Java protobuf classes must match what `datafusion-proto` 54.1.0 decodes. The pinned sha512 digests for both protos are updated accordingly; each was verified to match the copy vendored in the published `datafusion-proto` / `datafusion-proto-common` 54.1.0 crates, independently of the GitHub download. `arrow` (58) and `object_store` (0.13) are unchanged — 54.1.0 resolves to the same majors, so the `object_store` pin comment still holds. Adapting to the upstream API changes the bump requires: - `TableProvider`, `ExecutionPlan` and `ScalarUDFImpl` now take `Any` as a supertrait, so the manual `as_any` overrides are no longer trait members and are removed. - `MemoryPool` gained a `name()` method and a `Display` supertrait. `TrackingMemoryPool` implements both, following upstream's wrapper convention: name the wrapper, add the counters it exists to expose, and defer to the inner pool for the usage detail. - `CacheManagerConfig::table_files_statistics_cache` is renamed to `file_statistics_cache`, and the accompanying limit is now the on/off switch — `CacheManager::try_new` installs a default statistics cache whenever `file_statistics_cache_limit > 0`, even when the cache slot is `None`, and the default limit is non-zero. An explicit `fileStatisticsCache(false)` from the Java surface therefore has to zero the limit as well; otherwise upstream would install the very cache the caller asked us to skip. This is the one place the bump would have silently changed observable Java behavior. - The `cache_unit` module is gone; the default cache impls now live in `cache::file_statistics_cache` and `cache`. - `DataFusionError::AvroError` is gone — DataFusion 54 reads Avro through `arrow-avro` rather than `apache-avro`. Avro decode failures now arrive as `ArrowError::AvroError`, which the exception classifier already routed to `ExecutionException` alongside the `CsvError` / `JsonError` decoder variants, so the mapping stays coherent. The dead arm is dropped, along with the `avro` feature on `datafusion-jni-common` that existed only to gate it. One test fixture is also corrected. `SessionContextSubstraitTest` built plans whose base schema declared both columns `NULLABILITY_REQUIRED`, while the tests register a CSV — whose inferred schema is always nullable. DataFusion 54's Substrait consumer now validates that a field a plan declares non-nullable really is non-nullable in the table, and rejects the mismatch. That check is correct: a plan built around a "never null" assumption must not run against data that can contain nulls. The fixture is fixed to declare nullable columns; it only passed before because 53 did not check. ## Are these changes tested? Covered by the existing suites — this is a dependency bump, so the value is in the current tests continuing to pass against the new version rather than in new assertions. - `./mvnw test` — 349 tests, 0 failures. Run with the `substrait` Cargo feature enabled (`cargo build -p datafusion-jni --features substrait`) so the Substrait tests execute rather than skip. - `cargo test --workspace` — all passing. - `cargo clippy --workspace --all-targets -- -D warnings` — clean. - `cargo fmt --all -- --check` and `./mvnw -q spotless:check` — clean. - `cargo build --workspace --all-features` with `RUSTFLAGS="--cfg tokio_unstable"`, to cover the optional `substrait` and `runtime-metrics` features that the default build does not compile. ## Are there any user-facing changes? No API changes. Two behavioral notes, both inherited from upstream: - Avro decode failures now surface as `ExecutionException` rather than `IoException`, following the move to `arrow-avro`. The `IoException` javadoc is updated to match. - DataFusion 54 enables the file statistics and list files caches by default. Callers who never configured `CacheManagerOptions` pick up upstream's new defaults; an explicit `fileStatisticsCache(false)` continues to disable the cache, as described above.
Configuration menu - View commit details
-
Copy full SHA for 0d2c72b - Browse repository at this point
Copy the full SHA 0d2c72bView commit details
Loading
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff main...main