Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: devoopsman45/datafusion-java
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: apache/datafusion-java
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 4 commits
  • 44 files changed
  • 3 contributors

Commits on Jun 9, 2026

  1. feat(session): add tableExists and deregisterTable to SessionContext (a…

    …pache#102)
    
    ## Which issue does this PR close?
    
    - Closes apache#101 .
    
    ## Rationale for this change
    SessionContext can let a caller register tables but there is no way to
    
        Check whether a table name is already registered
        Remove a registered table from the session
    
    This makes it tough to write safer code while registering
    
    ## What changes are included in this PR?
    
    - `SessionContext.tableExists(String name)` — returns true if a table
        with that name is registered in the session
      - `SessionContext.deregisterTable(String name)` — removes a registered
        table; no-op if the name is not found
    
      Both are thin JNI wrappers over DataFusion's existing
      `SessionContext::table_exist` and `SessionContext::deregister_table`
      on the Rust side.
    
    
    ## Are these changes tested?
    
    Yes. `SessionContextTableRegistrationTest` covers:
      - tableExists returns false for an unregistered table
      - tableExists returns true after registerCsv
      - deregisterTable removes a registered table
      - deregisterTable is a no-op for an unregistered table
      - Both methods throw IllegalStateException on a closed context
    
    
    -->
    
    ## Are there any user-facing changes?
    
      Yes — two new public methods on `SessionContext`. Additive only, no
      breaking changes.
    devoopsman45 authored Jun 9, 2026
    Configuration menu
    Copy the full SHA
    60351ba View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2026

  1. Configuration menu
    Copy the full SHA
    051b00b View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2026

  1. Configuration menu
    Copy the full SHA
    ab57cb9 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2026

  1. chore(deps): bump DataFusion to 54.1.0 (apache#114)

    ## Which issue does this PR close?
    
    N/A — routine dependency bump; no tracking issue was filed.
    
    ## Rationale for this change
    
    Keeps the binding current with upstream DataFusion. 54.1.0 is the latest
    release
    line, and staying close to it keeps the next bump small and makes
    upstream fixes
    available to Java callers.
    
    ## What changes are included in this PR?
    
    Bumps `datafusion`, `datafusion-proto`, `datafusion-spark` and
    `datafusion-substrait` from 53.1.0 to 54.1.0. The `datafusion.version`
    Maven
    property moves in lock step, since it selects the upstream tag the
    `datafusion.proto` / `datafusion_common.proto` definitions are
    downloaded from —
    the generated Java protobuf classes must match what `datafusion-proto`
    54.1.0
    decodes. The pinned sha512 digests for both protos are updated
    accordingly; each
    was verified to match the copy vendored in the published
    `datafusion-proto`
    / `datafusion-proto-common` 54.1.0 crates, independently of the GitHub
    download.
    
    `arrow` (58) and `object_store` (0.13) are unchanged — 54.1.0 resolves
    to the
    same majors, so the `object_store` pin comment still holds.
    
    Adapting to the upstream API changes the bump requires:
    
    - `TableProvider`, `ExecutionPlan` and `ScalarUDFImpl` now take `Any` as
    a
    supertrait, so the manual `as_any` overrides are no longer trait members
    and
      are removed.
    - `MemoryPool` gained a `name()` method and a `Display` supertrait.
    `TrackingMemoryPool` implements both, following upstream's wrapper
    convention:
    name the wrapper, add the counters it exists to expose, and defer to the
    inner
      pool for the usage detail.
    - `CacheManagerConfig::table_files_statistics_cache` is renamed to
    `file_statistics_cache`, and the accompanying limit is now the on/off
    switch —
      `CacheManager::try_new` installs a default statistics cache whenever
    `file_statistics_cache_limit > 0`, even when the cache slot is `None`,
    and the
    default limit is non-zero. An explicit `fileStatisticsCache(false)` from
    the
    Java surface therefore has to zero the limit as well; otherwise upstream
    would
    install the very cache the caller asked us to skip. This is the one
    place the
      bump would have silently changed observable Java behavior.
    - The `cache_unit` module is gone; the default cache impls now live in
      `cache::file_statistics_cache` and `cache`.
    - `DataFusionError::AvroError` is gone — DataFusion 54 reads Avro
    through
    `arrow-avro` rather than `apache-avro`. Avro decode failures now arrive
    as
    `ArrowError::AvroError`, which the exception classifier already routed
    to
    `ExecutionException` alongside the `CsvError` / `JsonError` decoder
    variants,
    so the mapping stays coherent. The dead arm is dropped, along with the
    `avro`
      feature on `datafusion-jni-common` that existed only to gate it.
    
    One test fixture is also corrected. `SessionContextSubstraitTest` built
    plans
    whose base schema declared both columns `NULLABILITY_REQUIRED`, while
    the tests
    register a CSV — whose inferred schema is always nullable. DataFusion
    54's
    Substrait consumer now validates that a field a plan declares
    non-nullable
    really is non-nullable in the table, and rejects the mismatch. That
    check is
    correct: a plan built around a "never null" assumption must not run
    against data
    that can contain nulls. The fixture is fixed to declare nullable
    columns; it
    only passed before because 53 did not check.
    
    ## Are these changes tested?
    
    Covered by the existing suites — this is a dependency bump, so the value
    is in
    the current tests continuing to pass against the new version rather than
    in new
    assertions.
    
    - `./mvnw test` — 349 tests, 0 failures. Run with the `substrait` Cargo
    feature
      enabled (`cargo build -p datafusion-jni --features substrait`) so the
      Substrait tests execute rather than skip.
    - `cargo test --workspace` — all passing.
    - `cargo clippy --workspace --all-targets -- -D warnings` — clean.
    - `cargo fmt --all -- --check` and `./mvnw -q spotless:check` — clean.
    - `cargo build --workspace --all-features` with `RUSTFLAGS="--cfg
    tokio_unstable"`,
    to cover the optional `substrait` and `runtime-metrics` features that
    the
      default build does not compile.
    
    ## Are there any user-facing changes?
    
    No API changes. Two behavioral notes, both inherited from upstream:
    
    - Avro decode failures now surface as `ExecutionException` rather than
    `IoException`, following the move to `arrow-avro`. The `IoException`
    javadoc
      is updated to match.
    - DataFusion 54 enables the file statistics and list files caches by
    default.
    Callers who never configured `CacheManagerOptions` pick up upstream's
    new
    defaults; an explicit `fileStatisticsCache(false)` continues to disable
    the
      cache, as described above.
    andygrove authored Aug 6, 2026
    Configuration menu
    Copy the full SHA
    0d2c72b View commit details
    Browse the repository at this point in the history
Loading