Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 0 additions & 29 deletions .github/workflows/gradle-extraction-check.yml

This file was deleted.

29 changes: 0 additions & 29 deletions .github/workflows/solrj-test.yml

This file was deleted.

91 changes: 91 additions & 0 deletions .github/workflows/tests-core.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Solr Core Tests

# solr/core is most of the whole test suite, so it is split into four slices by
# Java package that run in parallel. The slices are balanced from measured CI
# timings: the cloud package alone is the largest unit in solr/core and sets
# the critical path, so its api and overseer sub-packages run in the search
# slice instead. The "other" slice is exclusion-based so that a new package
# can never silently fall outside every slice.

on:
pull_request:
branches:
- '*'
paths:
- '.github/workflows/tests-core.yml'
- 'gradle/libs.versions.toml'
- 'solr/core/**'
- 'solr/solrj-zookeeper/**'
# core tests are built on these; a change here can break them
Comment thread
janhoy marked this conversation as resolved.
Resolved
- 'solr/test-framework/**'
- 'solr/api/**'
- 'solr/solrj/**'
- 'solr/solrj-jetty/**'
- 'solr/solrj-streaming/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
name: Run Solr Core Tests (${{ matrix.slice }})

runs-on: ubuntu-latest
timeout-minutes: 25

strategy:
fail-fast: false
matrix:
slice: [cloud, search, handler, other]

env:
# Top-level packages under org.apache.solr for each named slice.
# The "other" slice runs everything not listed here.
SLICE_CLOUD: cloud

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we're going to balance by time, the slice names have lost their meaning. Let's use what use suggested of SLICE1, SLICE2, ...

SLICE_SEARCH: search uninverting spelling highlight analysis legacy index parser request cloud.api cloud.overseer
SLICE_HANDLER: handler response servlet jersey rest security api

Comment on lines +43 to +48

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking aloud: If I could re-organize our core module code, it'd be around: cli (soon to be separated!), cloud, server, node, core. And maybe others, but those are the big pillars in my mind. Ah well. I'm tempted to propose a big bang break-up for Solr 11... at least for the code to be in the right paths with the right packages... even if the surrounding modules aren't ready to be formed yet.

steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: ./.github/actions/prepare-for-build

- name: Test solr/core slice "${{ matrix.slice }}"
run: |
tasks=(:solr:core:test)
args=()
case "${{ matrix.slice }}" in
cloud)
pkgs="$SLICE_CLOUD"
args+=("-Ptests.exclude=org.apache.solr.cloud.api.*,org.apache.solr.cloud.overseer.*")
;;
search) pkgs="$SLICE_SEARCH" ;;
handler) pkgs="$SLICE_HANDLER" ;;
other)
# This slice has the most headroom, so it also runs solrj-zookeeper.
# It passes no --tests, so one invocation can drive both test tasks,
# and the excluded packages do not overlap solrj-zookeeper's tests.
pkgs=""
tasks+=(:solr:solrj-zookeeper:test)
excl=""
for p in $SLICE_CLOUD $SLICE_SEARCH $SLICE_HANDLER; do
excl="${excl:+$excl,}org.apache.solr.$p.*"
done
args+=("-Ptests.exclude=$excl")
;;
esac
for p in $pkgs; do
args+=(--tests "org.apache.solr.$p.*")
done
./gradlew "${tasks[@]}" -Ptests.jvms=2 "${args[@]}"

- name: Archive test output
if: ${{ failure() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-output-core-${{ matrix.slice }}
path: |
solr/core/build/test-results/**
solr/solrj-zookeeper/build/test-results/**
41 changes: 41 additions & 0 deletions .github/workflows/tests-extraction.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Extraction Module Tests

# The extraction module tests need Docker for the TikaServer tests of SolrCell,
# so they get a workflow of their own.

Comment thread
janhoy marked this conversation as resolved.
Resolved
on:
pull_request:
branches:
- '*'
paths:
- '.github/workflows/tests-extraction.yml'
- 'gradle/libs.versions.toml'
- 'solr/modules/extraction/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
name: Run Extraction Module Tests

runs-on: ubuntu-latest
timeout-minutes: 15

steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: ./.github/actions/prepare-for-build

- name: Run extraction module tests
run: ./gradlew --no-daemon :solr:modules:extraction:check

- name: Archive test output
if: ${{ failure() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-output-extraction
path: |
solr/modules/extraction/build/test-results/**
44 changes: 44 additions & 0 deletions .github/workflows/tests-misc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Misc Tests

# Small test suites that do not belong to any other tests-* workflow.

on:
pull_request:
branches:
- '*'
paths:
- '.github/workflows/tests-misc.yml'
- 'gradle/libs.versions.toml'
- 'solr/benchmark/**'
- 'solr/test-framework/**'
- 'solr/solr-ref-guide/modules/*/examples/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
name: Run Misc Tests

runs-on: ubuntu-latest
timeout-minutes: 15

steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: ./.github/actions/prepare-for-build

- name: Test the test-framework, benchmark and ref-guide example code
run: ./gradlew :solr:test-framework:test :solr:benchmark:test :solr:solr-ref-guide:test

- name: Archive test output
if: ${{ failure() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-output-misc
path: |
solr/test-framework/build/test-results/**
solr/benchmark/build/test-results/**
solr/solr-ref-guide/build/test-results/**
51 changes: 51 additions & 0 deletions .github/workflows/tests-modules-ops.yml

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the new name -- ops

Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Ops Modules Tests

on:
pull_request:
branches:
- '*'
paths:
- '.github/workflows/tests-modules-ops.yml'
- 'gradle/libs.versions.toml'
- 'solr/cross-dc-manager/**'
- 'solr/modules/cross-dc/**'
- 'solr/modules/gcs-repository/**'
- 'solr/modules/jwt-auth/**'
Comment thread
janhoy marked this conversation as resolved.
Resolved
- 'solr/modules/opentelemetry/**'
Comment thread
janhoy marked this conversation as resolved.
Resolved
- 'solr/modules/s3-repository/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
name: Run Ops Modules Tests

runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: ./.github/actions/prepare-for-build

- name: Test the ops and infrastructure modules
run: >
./gradlew
:solr:cross-dc-manager:test
:solr:modules:cross-dc:test
:solr:modules:gcs-repository:test
:solr:modules:jwt-auth:test
:solr:modules:opentelemetry:test
:solr:modules:s3-repository:test

- name: Archive test output
if: ${{ failure() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-output-modules-ops
path: |
solr/cross-dc-manager/build/test-results/**
solr/modules/*/build/test-results/**
54 changes: 54 additions & 0 deletions .github/workflows/tests-modules-search.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Search Modules Tests

on:
pull_request:
branches:
- '*'
paths:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps add the test-framework too, as everything we do sits on it!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes perhaps. Not sure if all these tests need to replicate the dependency structure between modules though. E.g. most modules/ depend on core, but I don't want them all to run for every tiny javadoc edit in core. We must not forget that we also have Continous Testing in Jenkins that will catch those corner case interactions where something in one (gradle) module unexpetedly breaks something in another. I think the right call for most PR testing is focused module tests.

- '.github/workflows/tests-modules-search.yml'
- 'gradle/libs.versions.toml'
- 'solr/modules/analysis-extras/**'
- 'solr/modules/clustering/**'
- 'solr/modules/cuvs/**'
- 'solr/modules/langid/**'
- 'solr/modules/language-models/**'
- 'solr/modules/ltr/**'
- 'solr/modules/scripting/**'
- 'solr/modules/sql/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
name: Run Search Modules Tests

runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: ./.github/actions/prepare-for-build

- name: Test the search-related modules
run: >
./gradlew
:solr:modules:analysis-extras:test
:solr:modules:clustering:test
:solr:modules:cuvs:test
:solr:modules:langid:test
:solr:modules:language-models:test
:solr:modules:ltr:test
:solr:modules:scripting:test
:solr:modules:sql:test

- name: Archive test output
if: ${{ failure() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-output-modules-search
path: |
solr/modules/*/build/test-results/**
44 changes: 44 additions & 0 deletions .github/workflows/tests-solrj.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: SolrJ Tests

on:
pull_request:
branches:
- '*'
paths:
- '.github/workflows/tests-solrj.yml'
- 'gradle/libs.versions.toml'
- 'solr/api/**'
- 'solr/solrj/**'
- 'solr/solrj-jetty/**'
- 'solr/solrj-streaming/**'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
name: Run SolrJ Tests

runs-on: ubuntu-latest
timeout-minutes: 20

steps:
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: ./.github/actions/prepare-for-build

- name: Test the SolrJ modules and the API module
run: ./gradlew :solr:api:test :solr:solrj:test :solr:solrj-jetty:test :solr:solrj-streaming:test -Ptests.jvms=3

- name: Archive test output
if: ${{ failure() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: test-output-solrj
path: |
solr/api/build/test-results/**
solr/solrj/build/test-results/**
solr/solrj-jetty/build/test-results/**
solr/solrj-streaming/build/test-results/**
Loading
Loading