Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 37 additions & 5 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,27 +17,59 @@

name: Build

# Source-affecting paths only. Docs-only edits skip this workflow; the
# Lint and RAT workflows still run on every change.
on:
push:
branches: [main, 'branch-*']
paths:
- 'core/**'
- 'examples/**'
- 'native/**'
- 'proto/**'
- 'pom.xml'
- '**/pom.xml'
- 'Makefile'
- 'mvnw'
- '.mvn/**'
- '.github/workflows/build.yml'
pull_request:
branches: [main, 'branch-*']
paths:
- 'core/**'
- 'examples/**'
- 'native/**'
- 'proto/**'
- 'pom.xml'
- '**/pom.xml'
- 'Makefile'
- 'mvnw'
- '.mvn/**'
- '.github/workflows/build.yml'

concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

permissions:
contents: read

jobs:
build:
name: make test
runs-on: ubuntu-latest
name: make test (JDK ${{ matrix.java }})
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
java: ['17', '21']
steps:
- uses: actions/checkout@v4

- name: Set up JDK 17
- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
java-version: ${{ matrix.java }}
cache: maven

- name: Set up Rust toolchain
Expand All @@ -55,5 +87,5 @@ jobs:
key: ${{ runner.os }}-cargo-${{ hashFiles('native/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-

- name: Build native and run JVM tests
- name: Build native and run tests
run: make test
89 changes: 89 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: Lint

on:
push:
branches: [main, 'branch-*']
pull_request:
branches: [main, 'branch-*']

concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

permissions:
contents: read

jobs:
fmt-check:
name: Format check
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
cache: maven

- name: Set up Rust toolchain
run: |
rustup update stable
rustup default stable
rustup component add rustfmt

- name: Check Java formatting
run: ./mvnw -q spotless:check

- name: Check Rust formatting
run: cd native && cargo fmt --all -- --check

clippy:
name: Clippy
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
cache: maven

- name: Set up Rust toolchain
run: |
rustup update stable
rustup default stable
rustup component add clippy

- name: Cache cargo
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
native/target
key: ${{ runner.os }}-clippy-${{ hashFiles('native/Cargo.lock') }}
restore-keys: ${{ runner.os }}-clippy-

- name: Run clippy
run: cd native && cargo clippy --all-targets -- -D warnings
45 changes: 45 additions & 0 deletions .github/workflows/pr_title_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: Check PR Title

concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

on:
pull_request:
types: [opened, edited, reopened]

permissions:
contents: read

jobs:
check-pr-title:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Check PR title
env:
PR_TITLE: ${{ github.event.pull_request.title }}
run: |
if ! echo $PR_TITLE | grep -Eq '^(\w+)(\(.+\))?: .+$'; then
echo "PR title does not follow conventional commit style."
echo "Please use a title in the format: type: message, or type(scope): message"
echo "Example: feat: Add support for sort-merge join"
exit 1
fi
50 changes: 50 additions & 0 deletions .github/workflows/rat.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: RAT License Check

# No `paths:` filter: license-header drift in docs or workflows
# matters too, so this gate runs on every change.
on:
push:
branches: [main, 'branch-*']
pull_request:
branches: [main, 'branch-*']

concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

permissions:
contents: read

jobs:
rat:
name: RAT License Check
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: '17'
cache: maven

- name: Run Apache RAT
run: ./mvnw -B -N apache-rat:check
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.

.PHONY: all native native-runtime-metrics jvm test clean tpch-data
.PHONY: all native native-runtime-metrics jvm test format clean tpch-data

all: native jvm

Expand All @@ -35,6 +35,12 @@ jvm:
test: native
./mvnw test

# Apply Java + Rust formatters in place. CI verifies the equivalent
# `:check` form inline in .github/workflows/lint.yml.
format:
./mvnw -q spotless:apply
cd native && cargo fmt --all

clean:
cd native && cargo clean
./mvnw clean
Expand Down
17 changes: 17 additions & 0 deletions dev/release/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1,18 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

PyGitHub
28 changes: 19 additions & 9 deletions docs/source/contributor-guide/code-style.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,28 +19,38 @@ under the License.

# Code style

To apply Java + Rust formatters in one go:

```sh
make format
```

CI verifies formatting, lint, and license headers on every PR via the
**Lint** and **RAT License Check** workflows.

## Java

Run the Spotless formatter before committing. CI fails the build if
formatting drifts:
Spotless is configured in the parent `pom.xml`. The CI `fmt-check` job
runs:

```sh
./mvnw spotless:apply
./mvnw spotless:check
```

## Rust

Run inside `native/`:
The CI `clippy` job runs (from `native/`):

```sh
cargo fmt
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
```

`-D warnings` turns clippy warnings into build failures, matching CI.
`-D warnings` escalates every clippy warning to a build failure.

## License headers

New source files need the Apache 2.0 license header. Apache RAT enforces
this during `verify` — `./mvnw verify` will fail if a tracked file is
missing the header.
Every tracked source file needs the Apache 2.0 license header. The
**RAT License Check** workflow runs `./mvnw -N apache-rat:check` on every
PR (including docs-only changes). Exclusions live in
`dev/release/rat_exclude_files.txt`.
6 changes: 5 additions & 1 deletion docs/source/contributor-guide/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ cd native && cargo build

The native library must be built before running JVM tests.

Before pushing, run `make format` to apply the Java + Rust formatters in
place. CI verifies formatting, clippy, and license headers on every PR.

The first build in a fresh checkout reaches out to
`raw.githubusercontent.com` to fetch the DataFusion `.proto` files used
to generate the `datafusion-proto` Java classes. Subsequent builds are
Expand Down Expand Up @@ -80,7 +83,8 @@ The repository is a multi-module Maven build:
cannot fall out of sync with the API.
- `native/` — Rust crate (JNI + Arrow C Data Interface).
- `proto/` — Protobuf definitions shared between Java and Rust.
- `Makefile` — top-level build orchestration (`make test`, `make tpch-data`).
- `Makefile` — top-level build orchestration (`make test`, `make format`,
`make tpch-data`).
- `mvnw`, `mvnw.cmd` — bundled Maven wrapper.
- `docs/` — Sphinx documentation source and build scripts.

Expand Down
Loading