Skip to content

docs(opensearch): fix the ES-to-OpenSearch migration guide - #826

Open
SuJinpei wants to merge 1 commit into
mainfrom
docs/opensearch-migration-fixes
Open

docs(opensearch): fix the ES-to-OpenSearch migration guide#826
SuJinpei wants to merge 1 commit into
mainfrom
docs/opensearch-migration-fixes

Conversation

@SuJinpei

@SuJinpei SuJinpei commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

What

Fixes How_to_Migrate_from_Elasticsearch_to_OpenSearch.md. The guide was reviewed, then both migration paths were run end to end on a real cluster — ES 7.10.2 (3 nodes) → snapshot to S3 → OpenSearch 2.19.6 → reindex → upgrade to OpenSearch 3.3.1, plus ECK-deployed Elasticsearch 8.17.5 → reindex from remote. Everything below is either a defect that breaks a reader following the guide, or an explanation the run showed to be wrong.

Errors that break a reader following the guide verbatim

Problem Effect
s3.client.default.endpoint: "<http://minio.example.com:9000>" in 3 samples Markdown autolink brackets leaked into the YAML value — the S3 client is configured with an invalid host
repository-s3 URLs hardcoded to specific versions The guide itself says the plugin version must match the node exactly; opensearch-plugin install refuses a mismatch and the node will not start
Init container image from an internal registry Not pullable by customers
Keystore steps written as a single-pod action The keystore is per-node; snapshots fail on the pods that were skipped
"indices": "*" while the note below advises excluding system indices The command contradicts its own guidance
Mappings extracted with sed Silently drops index settings — an index using a custom analyzer cannot be recreated at all
OpenSearch upgraded to 3.3.1, Dashboards to 3.3.0 Mismatched pair
Phase 1's cluster resource cannot start Two separate omissions, each fatal — see below
version.created given as 7102099 for ES 7.10.2 The real value is 7100299; the reader never sees the documented number

Why Phase 1 could not start

  • No spec.bootstrap.pluginsList. Every general.additionalConfig entry is rendered onto the bootstrap pod too, and s3.client.* are only valid settings once repository-s3 is installed, so the bootstrap pod dies with unknown setting [s3.client.default.region]. The other nodes are pinned to it via cluster.initial_master_nodes, so the cluster never forms. The prerequisites section already showed the bootstrap: block — Phase 1 did not.
  • No admin password. OpenSearch 2.12+ exits with No custom admin password found. security.config.adminCredentialsSecret does not supply it — that is only the credential the Operator uses to reach the cluster — and the bootstrap pod has no env field, so it must go through additionalConfig.

Explanations corrected

  • Reindex from Remote was presented as ES 8.x-only. It also works from ES 7.10 and can target OpenSearch 3.x directly, removing the entire two-hop (verified: 5/5 documents from ES 7.10.2 straight into 3.3.1). Added a Choosing a Method section describing the real decision — copying index files vs re-indexing documents — including that reindex carries no settings, mappings or aliases.
  • The ES 8.x Lucene claim is unverified. The guide said ES 8.x snapshots are unreadable because of "a newer Lucene version with incompatible metadata protocols". Measured, ES 8.17.5 is on Lucene 9.12.0 and OpenSearch 2.19.6 on 9.12.3 — the same Lucene minor. An actual ES 8 snapshot restore was not tested, so rather than substitute another explanation the text now states the behaviour without asserting a cause.
  • additionalConfig becomes environment variables on every pod, not entries in an opensearch.yml ConfigMap. Corrected everywhere it was described.

Added

  • A Troubleshooting section for a node stuck in CrashLoopBackOff after an additionalConfig change: values there are not validated by the Operator, a bad key is only rejected at node boot, and the rollout halts at the first failed node while the rest keep serving the previous config.
  • A warning that reindex.remote.whitelist was removed in OpenSearch 3.x in favour of reindex.remote.allowlist — a real failure observed on a 3.3.1 cluster:
    SettingsException[unknown setting [reindex.remote.whitelist] please check that any
    required plugins are installed, or check the breaking changes documentation for
    removed settings]
    
  • What a restore does not carry over (index templates, ingest pipelines, ILM policies, users and roles, Kibana saved objects), replica counts on a smaller target cluster, and taking a snapshot before the major upgrade.
  • Relative links to the sibling OpenSearch guides, following this directory's convention.

Structure

The ES 7.10 path is renumbered Phase 0 / Phase 1 / Phase 2. Previously the source-side snapshot steps sat in an unnumbered Procedure section between Prerequisites and Phase 1, and step numbering restarted three times.

Notes for reviewers

  • English only. docs/zh is regenerated from the English source by the pipeline.
  • The example versions (7.10.2, 2.19.3, 3.3.1) are unchanged, but are now explicitly labelled as examples to substitute, with the command to read the running version. Aligning literal version numbers across the whole OpenSearch KB set is a separate change.
  • The admin-password route deserves a second opinion. Passing it through additionalConfig stores it in plain text in the cluster resource — the warning says so and points at securityConfigSecret for anything long-lived — but if there is a better supported route for OpenSearch 2.12+ on Operator 2.8.x, that belongs here instead.

Corrections to How_to_Migrate_from_Elasticsearch_to_OpenSearch.md, found by
reviewing it and then running both migration paths end to end against a real
ES 7.10.2 / OpenSearch 2.19.6 / OpenSearch 3.3.1 / Elasticsearch 8.17.5
setup.

Errors that break a reader following the guide verbatim:

- The S3 endpoint was written as "<http://minio.example.com:9000>" in three
  samples. The angle brackets are markdown autolink syntax that leaked into
  the YAML value, so the S3 client is pointed at an invalid host.
- The repository-s3 download URLs hardcoded versions, while the guide itself
  states the plugin version must match the node version exactly. Replaced
  with URL patterns plus the command that reads the running version.
- The Elasticsearch init container used an image from an internal registry
  that customers cannot pull.
- The keystore steps read as a single-pod action. The keystore is per-node
  and reload_secure_settings only reloads what is already on each node.
- The snapshot request used "indices": "*" while the note below it advised
  excluding system indices.
- The reindex step extracted mappings with sed, silently dropping the index
  settings, so any index with a custom analyzer could not be recreated.
- OpenSearch was upgraded to 3.3.1 but OpenSearch Dashboards to 3.3.0.
- The Phase 1 cluster resource could not start: it omitted
  spec.bootstrap.pluginsList (additionalConfig is rendered onto the bootstrap
  pod too, so s3.client.* is an unknown setting there without the plugin) and
  supplied no admin password (OpenSearch 2.12+ refuses to start without one,
  and the Operator has no dedicated field for it).
- The verification step cited 7102099 as the version.created of an ES 7.10.2
  index; the real value is 7100299. The check no longer names a literal,
  since the number varies with the patch release.

Corrections to the explanations:

- Reindex from Remote was presented as an ES 8.x-only method. It also works
  from ES 7.10 and can target OpenSearch 3.x directly, removing the need for
  an intermediate 2.x cluster. Added a "Choosing a Method" section covering
  the actual trade-off, including that reindex carries no settings, mappings
  or aliases.
- The ES 8.x section blamed the snapshot incompatibility on a newer Lucene
  version. Measured, ES 8.17.5 and OpenSearch 2.19.6 are on the same Lucene
  minor, so that is not the cause; the text now states the behaviour without
  asserting a reason.
- additionalConfig entries become environment variables on every pod, not
  entries in an opensearch.yml ConfigMap. Corrected everywhere it was
  described, including the troubleshooting step that told readers to grep
  ConfigMaps.

Additions:

- A Troubleshooting section for a node stuck in CrashLoopBackOff after an
  additionalConfig change, including that a bad key is only rejected at node
  boot and that the rollout halts at the first failed node.
- A warning that reindex.remote.whitelist was removed in OpenSearch 3.x in
  favour of reindex.remote.allowlist, which is a real failure seen on 3.3.1.
- Notes on what a restore does not carry over (index templates, ingest
  pipelines, ILM policies, users and roles, Kibana saved objects), replica
  counts on a smaller target, and taking a snapshot before a major upgrade.
- Relative links to the sibling OpenSearch guides.

The ES 7.10 path is renumbered Phase 0 / 1 / 2; the source-side snapshot
steps previously sat in an unnumbered section between the phases.

docs/zh is left untouched; it is regenerated from the English source by the
pipeline.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@SuJinpei
SuJinpei force-pushed the docs/opensearch-migration-fixes branch from 4f55f43 to 818144e Compare August 7, 2026 09:58
@SuJinpei SuJinpei changed the title docs(opensearch): fix breaking errors in the ES-to-OpenSearch migration guide docs(opensearch): fix the ES-to-OpenSearch migration guide Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant