feat: added sidecar to turn logs into metrics and forward remaining logs MAPCO-11460

[CptSchnitz]

No description provided.

[syncush]

Summary & Overall Impressions

Great work on this PR! The architecture for the Fluent Bit log-processing sidecar is very clean and well thought out:

• Clean separation between stdout logging (human-readable or JSON) and internal syslog log forwarding.
• Safe helper functions using deepCopy to prevent mutating shared .Values.
• Comprehensive local test harness (test/local) with Docker Compose, otel-collector, and lgtm stacks for local verification.

Here are a few observations and recommendations for improvement:

---

1. Nginx Error Log Regex Parser (helm/config/fluent-bit.yaml)

In helm/config/fluent-bit.yaml:

regex: '^(?:(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) )?\[(?<level>\w+)\] (?<pid>\d+)#\d+:(?: \*\d+)? (?<message>.*?)(?:, client: (?<client>[^,]+), server: (?<server>[^,]*), request: "(?<request>[^"]*)".*)?$'

Observation:
The tail fields (, client: ..., server: ..., request: ...) are currently enclosed in a single contiguous optional non-capturing group.

In Nginx, errors that occur before a request line is negotiated (e.g. SSL handshake failures, early connection drops, TLS alert errors, client timeouts during connect) log client: and server: but omit request:. Because request: is missing in those lines, the entire optional group fails to match, and the client and server fields are not extracted into attributes (they remain trapped inside the lazy message field).

Suggested Fix:
Split each optional field into its own sequential optional group so client and server are extracted even when request is absent:

-    regex: '^(?:(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) )?\[(?<level>\w+)\] (?<pid>\d+)#\d+:(?: \*\d+)? (?<message>.*?)(?:, client: (?<client>[^,]+), server: (?<server>[^,]*), request: "(?<request>[^"]*)".*)?$'
+    regex: '^(?:(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) )?\[(?<level>\w+)\] (?<pid>\d+)#\d+:(?: \*\d+)? (?<message>.*?)(?:, client: (?<client>[^,]+))?(?:, server: (?<server>[^,]+))?(?:, request: "(?<request>[^"]*)")?.*$'

---

2. ConfigMap Multi-line Readability (helm/templates/fluentbit-configmap.yaml)

In helm/templates/fluentbit-configmap.yaml:

data:
  fluent-bit.yaml: {{ tpl (.Files.Get "config/fluent-bit.yaml") . | quote }}
  metadata.lua: {{ tpl (.Files.Get "config/metadata.lua") . | quote }}

Observation:
Using quote renders multi-line files as single escaped strings with \n. While valid Kubernetes YAML, it makes inspection via kubectl get configmap <name> -o yaml harder to read and debug.

Suggested Fix:
Using standard YAML block scalars (|) keeps the rendered files nicely formatted:

 data:
-  fluent-bit.yaml: {{ tpl (.Files.Get "config/fluent-bit.yaml") . | quote }}
+  fluent-bit.yaml: |
+    {{- tpl (.Files.Get "config/fluent-bit.yaml") . | nindent 4 }}
   # Chart-owned Lua; fluent-bit.lua below is the operator's, on its own key and mount.
-  metadata.lua: {{ tpl (.Files.Get "config/metadata.lua") . | quote }}
+  metadata.lua: |
+    {{- tpl (.Files.Get "config/metadata.lua") . | nindent 4 }}

---

3. JSON Decode Guard Note (helm/config/fluent-bit.yaml)

The grep filter guard:

    - name: grep
      match: nginx.access
      regex: "$Attributes['http.response.status_code'] ."

This is a great safeguard against sidecar crashes caused by log_to_metrics on missing fields. Just worth keeping in mind that any access log line that fails JSON parsing will be dropped rather than forwarded to Loki.

[syncush]

Plus you are missing MAPCO-# in your title, add the workflow

[github-actions]

🎫 Related Jira Issue: MAPCO-11460