Fix injected annotation removal check

[jefchien]

Description of the issue

The operator's auto-annotation flow can delete explicit opt-out annotations and then re-enable instrumentation that had previously been explicitly disabled.

For each instrumentation type, the operator pairs an insert mutation (adds the managed annotation keys with the value set to "true" when missing) with a remove mutation (deletes the same keys). Both are built from the same key/value pairs

  - instrumentation.opentelemetry.io/inject-{java,python,dotnet,nodejs}: "true"
  - cloudwatch.aws.amazon.com/auto-annotate-{java,python,dotnet,nodejs}: "true"

but the remove mutation only matches the keys when determining whether they should be deleted.

So if a pod has both managed keys and sets them to "false" to opt out, the remove mutation still sees them as operator owned and will remove them. Once removed, the opt-out is gone and the keys get repopulated with "true" on the next update re-enabling injection.

Description of change

Updated the remove mutation so it only removes annotations the operator actually added by checking the values as well as the keys. It removes the managed annotation pair only when the current values still match the injected values, so an explicit "false" opt-out is preserved.

Testing

Added a test case to the existing unit tests to assert that a value set to "false" survives when the managed value is "true".

Built the image and installed it on an EKS cluster using the amazon-cloudwatch-observability helm chart with autoMonitor.monitorAllServices: true. Used a test Deployment with a pod template that set instrumentation.opentelemetry.io/inject-java: "false" and cloudwatch.aws.amazon.com/auto-annotate-java: "false".

Reproduced the issue using the current operator 3.7.0. On create, the webhook deleted both of the "false" opt-outs. After adding a Service that points to the workload and triggering an update, the managed keys for all 4 supported languages get added including the Java ones.

With the fixed operator, the java opt-outs are preserved on create and after the update (only the 3 other language managed annotations got added). Verified that removing the Service still resulted in all the operator's managed keys (the other 3 languages) getting removed and leaving the opt-outs.

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.