DE-159650: Upgrade elasticsearch/elasticsearch from Elasticsearch v7.1.1 to Elasticsearch v9.3.0

[sumitkhopade1986]

Description: Upgrade elasticsearch/elasticsearch from v7.1.1 to v9.3.0 with PHP ≥8.2 requirement
Possible impact: Elasticsearch index operations, document operations, bulk indexing, search, cluster/snapshot/pipeline APIs, transport layer, CI pipeline

---

Elasticsearch v7 → v9.3 Migration

Major version upgrade of BrandEmbassy/Elastica from elasticsearch/elasticsearch ^7.1.1 to ^9.3.

Why

ES v9 dropped the legacy typed \Elasticsearch\Client facade and its type-based document APIs entirely. The existing Elastica layer built on ^7 could not communicate with an ES v9 cluster. This migration eliminates all v7 compatibility shims and moves to the officially supported elastic/elasticsearch ^9.3 SDK with full ES v9 REST API compliance.

---

Architecture: Dual-Client Approach

Client and Index now operate with a dual-client approach:

• Native v9 client (elastic/elasticsearch ^9.3): Used for all index-management operations — indices()->create(), indices()->putMapping(), indices()->delete(), stats, settings, reindex, snapshot, pipeline, cluster, node APIs
• Elastica HTTP transport: Retained for document operations — bulk index, search, get/delete documents

Connection was extended to hold and expose the native client instance. Client::putIndexMapping() was added as a dedicated testable bridge for mapping updates.

Native client calls in Index::delete(), Index::create(), Pipeline::deletePipeline(), and Reindex::run() are wrapped in try/catch blocks that convert ClientResponseException/ServerResponseException into Elastica's own ResponseException — preserving the existing exception contract for all callers.

OpenSearch Compatibility

By default the native client middleware injects X-Elastic-Product: Elasticsearch into every response so OpenSearch clusters (which do not send this header) pass the elasticsearch-php v9 product check. Set bypass_product_check: false in connection params to disable this when strict product verification is required.

---

What Changed (66 files)

Category	Files	Reason
Core ES9 SDK migration	~30 src files	ES9 removed AbstractEndpoint; all API calls rewritten using native v9 typed methods
PHP ≥8.2 upgrade	7	ES9 SDK requires PHP 8.x; minimum bumped from ^7.2|^8.0 to >=8.2
CI & tooling	4	ubuntu-20.04 runner retired, actions/cache@v2 blocked, php-cs-fixer 3.8.0 cannot parse PHP 8.x syntax
Test suite updates	~25	ES9 API changes: removed queries, renamed exceptions, range params, fielddata, percentile precision, sort behaviour

---

Dependency Updates (composer.json)

• elasticsearch/elasticsearch: ^7.1.1 → ^9.3
• php: ^7.2 || ^8.0 → >=8.2 (PHP 8.1 is EOL since Dec 2024)
• psr/log: ^1.0 || ^2.0 || ^3.0 → ^2.0 || ^3.0
• Added: "allow-plugins": { "php-http/discovery": true }

CI (continuous-integration.yaml)

• ES version: 7.15.2 → 9.3.0 (GitHub Actions service container)
• PHP matrix: drop 7.x/8.0/8.1, keep 8.2 + 8.3
• CS job: PHP 7.4 + php-cs-fixer 3.8.0 → PHP 8.2 + 3.94.2
• Updated deprecated actions: checkout@v2→@v4, cache@v2→@v4, ::set-output→$GITHUB_OUTPUT

.php-cs-fixer.dist.php

Rules disabled to prevent cosmetic churn on files not part of the ES9 migration:

global_namespace_import, trailing_comma_in_multiline, fully_qualified_strict_types, phpdoc_separation, new_with_parentheses, escape_implicit_backslashes, declare_strict_types, no_unneeded_control_parentheses, string_implicit_backslashes, phpdoc_no_alias_tag, yoda_style, curly_braces_position, braces_position, no_extra_blank_lines, blank_line_after_opening_tag, native_function_invocation, phpdoc_order, cast_spaces, native_constant_invocation, single_line_throw, declare_parentheses, class_attributes_separation, linebreak_after_opening_tag, declare_equal_normalize, single_quote, ordered_class_elements, blank_line_between_import_groups

---

Breaking Changes

• PHP 7.x, 8.0 and 8.1 no longer supported — requires PHP 8.2+
• Type-based document API removed (no _type in requests/responses)
• include_type_name parameter removed
• src/Elasticsearch/Endpoints/Update.php removed (AbstractEndpoint no longer exists in ES9 SDK)
• requestEndpoint() on Client, Index, and Pipeline now throws RuntimeException — ES9 SDK removed AbstractEndpoint; use direct client methods instead

---

Key Source Changes

File	Change
src/Connection.php	Added getClient() returning native ES9 ElasticsearchClient; OpenSearch product-check middleware (on by default)
src/Client.php	Rewritten for ES9 SDK; putIndexMapping() added; array|string type on $config
src/Index.php	create(), delete() use native client with exception wrapping
src/Pipeline.php	deletePipeline() uses native client with correct DELETE request context
src/Reindex.php	run() uses native client with correct POST request context and body
src/Bulk.php	Removed $apiVersion param from BulkResponse constructor; removed unused assignment
src/Transport/Http.php	Fixed double-slash URI join; native function prefixing

Key Test Changes

File	ES9 Change
tests/Exception/ResponseExceptionTest.php	mapper_parsing_exception → document_parsing_exception
tests/MappingTest.php	Removed boost mapping param (removed in ES9)
tests/Query/CommonTest.php	Common query removed — replaced with BoolQuery + should + minimum_should_match
tests/Query/RangeTest.php	from/to params → gte/lte
tests/Query/FunctionScoreTest.php	_id fielddata disabled — seed field changed to price
tests/ReindexTest.php	409 conflict behaviour; allowlist/whitelist regex
tests/ResponseFunctionalTest.php	Replaced removed API with index-not-found test
tests/ResultTest.php	Sort by seq field instead of _id
tests/Aggregation/GeoBoundsTest.php	assertEqualsWithDelta for floating-point coordinates
tests/Aggregation/PercentilesTest.php	Delta tolerance; keys returned as floats in ES9
tests/Multi/SearchTest.php	terminate_after moved to query param; assert on result count

---

Test Plan

• Unit tests — OK (565 tests, 2582 assertions) on PHP 8.2 and 8.3
• Functional tests — all passing against ES 9.3.0 on PHP 8.2 and 8.3
• PHPStan — 0 errors
• Coding style — 0 violations with php-cs-fixer 3.94.2 (497 files)
• Full integration test with platform-backend

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Reference/testing PR that consolidates the full migration from legacy elasticsearch/elasticsearch v7 endpoint objects to the native elasticsearch-php v9 client API, alongside the supporting infra/test updates needed to run against ES 9.x.

Changes:

• Upgrades runtime/infra to ES 9.x + PHP 8.1 and updates PHPUnit config.
• Replaces Elasticsearch\Endpoints\* usage across src/ with native v9 client calls.
• Updates (and in several cases skips/relaxes) functional tests to align with ES v9 response/behavior changes.

Reviewed changes

Copilot reviewed 59 out of 59 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/Transport/NullTransportTest.php	Skips NullTransport functional coverage
tests/Transport/HttpTest.php	Skips proxy/body reuse/compression tests
tests/Transport/GuzzleTest.php	Skips proxy/body reuse tests; guzzle check
tests/StatusTest.php	Broaden exception handling for ES v9
tests/SnapshotTest.php	Snapshot test tweaks for ES v9
tests/ResultTest.php	Sort expectations updated for ES v9
tests/ResultSetTest.php	Adjust invalid offset tests + skipping
tests/ResultSet/ProcessingBuilderTest.php	Threads ApiVersion through builders
tests/ResultSet/ChainProcessorTest.php	Threads ApiVersion through ResultSet
tests/ResultSet/BuilderTest.php	DefaultBuilder now requires ApiVersion
tests/ResponseFunctionalTest.php	Skips removed type-based mapping behavior
tests/ReindexTest.php	Handles v9 client exceptions in assertions
tests/Query/RangeTest.php	Uses gte/lte for range query
tests/Query/FunctionScoreTest.php	Random score field changes for ES v9
tests/Query/CommonTest.php	Skips removed “common” query
tests/Processor/AttachmentProcessorTest.php	Plugin presence check in setUp
tests/PipelineTest.php	Handles v9 error response structure
tests/Node/InfoTest.php	Skips ingest-attachment assertion if absent
tests/Multi/SearchTest.php	Error-handling + search options tweaks
tests/Multi/MultiBuilderTest.php	Adds ApiVersion argument threading
tests/MappingTest.php	Mapping payload updated for ES v9
tests/IndexTest.php	Replaces endpoint usage with native client
tests/Index/SettingsTest.php	Handles v9 exception formats; cleanup
tests/Exception/ResponseExceptionTest.php	Adapts expected error types for v9
tests/Exception/PartialShardFailureExceptionTest.php	Builder now requires ApiVersion
tests/DocumentTest.php	PHPUnit assertion API update
tests/ConnectionTest.php	Transport factory now requires logger args
tests/Cluster/SettingsTest.php	Cluster settings updated + v9 exceptions
tests/ClientFunctionalTest.php	Uses native client for endpoint-like tests
tests/BulkTest.php	Updates bulk update action creation for v9
tests/Bulk/Action/UpdateDocumentTest.php	Uses AbstractDocument::create w/ ApiVersion
tests/Bulk/Action/AbstractDocumentTest.php	AbstractDocument::create signature updates
tests/Base.php	Functional teardown + ingest pipeline via v9 client
tests/Aggregation/PercentilesTest.php	Looser assertions for percentile variability
tests/Aggregation/GeoBoundsTest.php	Delta assertions for float values
src/Task.php	Tasks API migrated to native client
src/Status.php	Alias/stats calls migrated to native client
src/Snapshot.php	Snapshot restore migrated to native client
src/Reindex.php	Reindex API migrated + param whitelist
src/Pipeline.php	Ingest pipeline APIs migrated to native client
src/Node/Stats.php	Node stats via native client
src/Node/Info.php	Node info via native client
src/Mapping.php	Mapping updates routed via Client helper
src/Index/Stats.php	Index stats via native client
src/Index/Recovery.php	Index recovery via native client
src/Index.php	Large-scale migration to native indices/docs APIs
src/Elasticsearch/Endpoints/Update.php	Removes legacy endpoint shim
src/ElasticSearchVersion.php	Adds VERSION_9 constant
src/Connection.php	Adds cached v9 client builder on Connection
src/Cluster/Health.php	Cluster health via native client
src/Cluster.php	Cluster state via native client
src/Client.php	Native client usage + logging tweaks + new helper
src/Bulk.php	Bulk response construction updated
src/ApiVersion.php	Adds API_VERSION_9 constant
phpunit.xml.dist	Updates schema + coverage config
phpstan-baseline.neon	Removes baseline entries resolved by refactor
docker/php/Dockerfile	PHP base image bumped to 8.1
docker/docker-compose.es.yml	ES cluster updated to 9.3.0 + settings
composer.json	Requires PHP 8.1 + elasticsearch v9.3

Comments suppressed due to low confidence (9)

src/Index.php:248

• $tags is extracted from $options but never used. This is dead code now that getDocument() bypasses Elastica\Client::request() (which is where tags are used for logging). Either remove CustomOptions::REQUEST_TAGS handling here, or re-route this call through Elastica\Client::request() so tags still have an effect.

        $tags = $options[CustomOptions::REQUEST_TAGS] ?? [];
        unset($options[CustomOptions::REQUEST_TAGS]);

        $params = \array_merge($options, [
            'index' => $this->getName(),
            'id' => $id,
        ]);

src/Index.php:500

• $tags is extracted from $options but never used in create(). This suggests request tagging/logging support was lost when moving away from requestEndpoint(). Either remove tag extraction or ensure tags still influence the request/logging path (e.g., by performing the request via Elastica\Client::request()).

        $tags = $options[CustomOptions::REQUEST_TAGS] ?? [];
        unset($options[CustomOptions::REQUEST_TAGS]);

        $allowedOptions = [
            'master_timeout',
            'timeout',

src/Client.php:698

• requestEndpoint() now always throws a RuntimeException. Even with a deprecation notice, this is a runtime-breaking behavior change for any consumers still calling requestEndpoint() (including those who may not be on ES v9 yet). Consider either removing the method in a major-version bump (so callers fail at compile/static analysis time), or keeping a compatibility implementation (e.g., accept a path/method/body shape) while emitting a deprecation warning, rather than throwing unconditionally.

    /**
     * Makes calls to the elasticsearch server with usage official client Endpoint.
     *
     * @deprecated This method is deprecated in v9. Use direct client methods instead.
     *
     * @param string[] $tags
     *
     * V9 NOTE: AbstractEndpoint class removed in elasticsearch-php v9.
     * This method is kept for backward compatibility but throws an exception.
     * Each endpoint should now use the client's direct methods.
     */
    public function requestEndpoint($endpoint, array $tags = []): Response
    {
        throw new \RuntimeException('requestEndpoint() is deprecated in Elasticsearch v9. AbstractEndpoint class no longer exists. Use direct client methods like $client->indices()->refresh() instead.');
    }

tests/SnapshotTest.php:79

• This assertion is duplicated back-to-back, which doesn't add coverage and makes future edits noisier. Remove the duplicate assertion (or replace it with the intended stronger check, e.g., count/contents validation).
  tests/ClientFunctionalTest.php:176
• This test deletes two documents but only verifies that document 1 is gone. If the delete operation regresses for document 2, this test would still pass. Add an assertion (or a second try/catch) that also verifies document 2 cannot be retrieved after deleteDocuments().
  tests/Transport/GuzzleTest.php:16
• PHPUnit lifecycle hook is case-sensitive: "setUpBeforeClass" (capital B) is the method PHPUnit calls. With the current method name "setUpbeforeClass", the guzzle dependency check/skip won't run, so this test class may execute even when guzzle isn't installed.
  tests/SnapshotTest.php:69
• Catching and swallowing a broad \Exception here can hide legitimate failures (permissions, repository misconfiguration, etc.) and make the test harder to debug. Prefer catching the specific "not found"/"snapshot missing" exception type that is expected when the snapshot doesn't exist, and fail the test for unexpected exceptions.
  src/Connection.php:284
• Connection::getClient() builds the host URL by concatenating the configured path verbatim. If Connection::getPath() returns a non-empty value without a leading "/" (e.g. "es"), the resulting host string becomes invalid ("http://host:9200es"). Normalize the path (ensure it is empty or begins with "/") before formatting the host string.

        $hosts = [];
        $scheme = $this->hasParam('ssl') && $this->getParam('ssl') ? 'https' : 'http';
        $host = $this->getHost();
        $port = $this->getPort();
        $path = $this->getPath();

        $hostString = \sprintf('%s://%s:%d%s', $scheme, $host, $port, $path);
        $hosts[] = $hostString;

src/Connection.php:303

• The native elasticsearch-php client builder here ignores several existing Connection settings (proxy, timeouts, compression/transport config, and other config options). This appears to be the reason transport/proxy/compression tests are now unconditionally skipped. Consider wiring these Connection options into ClientBuilder (or documenting/removing the unsupported options) so existing Connection APIs like setProxy()/setTimeout() continue to have effect.

        $builder = ClientBuilder::create()
            ->setHosts($hosts)
        ;

        if ($this->hasParam('username') && $this->hasParam('password')) {
            $builder->setBasicAuthentication(
                $this->getParam('username'),
                $this->getParam('password')
            );
        }

        if ($this->hasParam('api_key')) {
            $builder->setApiKey($this->getParam('api_key'));
        }

        $this->_client = $builder->build();

        return $this->_client;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 167 out of 167 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

Pull request overview

Copilot reviewed 66 out of 66 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.