From d3d2b567d71d1211d54f8a3e3f835a7be01810d5 Mon Sep 17 00:00:00 2001 From: Jovan Kostovski Date: Thu, 20 Aug 2026 10:12:42 +0200 Subject: [PATCH 1/9] add rfc for platform logs over otel Co-authored-by: Joris Baum Co-authored-by: Katharina Weisbarth Co-authored-by: Beyhan Veli Co-authored-by: Silvestre Zabala --- .../rfc-platform-logs-with-opentelemetry.md | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 toc/rfc/rfc-platform-logs-with-opentelemetry.md diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-platform-logs-with-opentelemetry.md new file mode 100644 index 000000000..c3b653876 --- /dev/null +++ b/toc/rfc/rfc-platform-logs-with-opentelemetry.md @@ -0,0 +1,92 @@ +# Meta +[meta]: #meta +- Name: Support Platform Logs Collection and Processing with OpenTelemetry +- Start Date: 2026-08-19 +- Author(s): @chombium, @jorbaum, @Katharin4, @beyhan, @silvestre +- Status: Draft +- Related RFCs : [rfc-0018-aggregate-metric-egress-with-opentelemetry-collector](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md) +- RFC Pull Request: + + +## Summary + +Members of the [App Runtime Platform WG](https://github.com/cloudfoundry/community/blob/main/toc/working-groups/app-runtime-platform.md) propose adding support for collection and processing of platform component logs with OpenTelemetry. The goal is to modernize the consumption of platform component logs in Cloud Foundry and add another more modern alternative to the Syslog Protocol. Operators would be able to choose how they want the platform component logs to be processed and forwarded. + +We believe that adding OpenTelemetry support for platform component logs will enable more reliable delivery and open possibilities for better processing of the telemetry data downstream inside and outside of Cloud Foundry. + +## Problem + +In Cloud Foundry the platform component logs are collected and processed with the [syslog-release](https://github.com/cloudfoundry/syslog-release) which provides a tool to tail log files from `/var/vcap/sys/log` called [blackbox](https://github.com/cloudfoundry/blackbox) and forwarding and processing rules for [RSyslog](https://www.rsyslog.com/). Rsyslog is provided with the stemcell. Rsyslog is highly specialized for parsing and processing text strings and JSON, but is hard to extend (depends on plugins delivered with the stemcell), its configuration is done with ReinerScript and some legacy formats. Rsyslog still has some use case, but it doesn't fit in the modern observability pipelines. + +On the other hand, OpenTelemetry has become the defacto standard in the Observability World and it has vibrant community. +The OpenTelemetry Collector offers huge advantages over Syslog: +- Can be easily extended with many community managed components like receivers, processors, extensions and exporters from the [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) repository. +- The configuration is standardized yaml. +- The information from all telemetry signals (logs, metrics, traces and baggage) can be easily correlated on the fly while it is still moving through the pipelines which enables faster problem discovery, root cause analysis and problem solving. + +## Proposal + +We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all oft the goodies mentioned above. The `syslog-release` has to be analyzed in its smallest detail to get to know of all its functionalities and find their proper replacements with OpenTelemetry. + +### Affected Working Groups +- Foundational Infrastructure +- Application Runtime Platform + +### Implementation + +Cloud Foundry supports platform log collection on two major type of operating systems Unix-like with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). It has to be ensured that the OpenTelemetry replacement works equally good for both of them. Both releases have to be analyzed to find the details how they function, so that that proper replacement can be built. + +#### Focus + +This implementation will focus on collection and processing of the platform component logs (BOSH job logs) stored in `/var/vcap/sys/log` directory. Supporting other paths where logs are stored is nice to have capability. + +The two basic functions of the `syslog-release` will be replaced as follows: +- blackbox -> [filelogreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) +- RSyslog parsing rules -> Otel Collector Pipeline which with the same functionality. Additional processors and receivers might be needed + +#### Phases + +##### Phase 1 - Discovery and Stakeholder Alignment (prerequisite) + +Before we start with the technical implementation we have to: +- Analyze the internals of both `syslog-release` and `windows-syslog-release` release, how they work and what functionality do they provide: + - BOSH jobs + - Configuration parameters + - Access permissions for log files. On Ubuntu Noble and higher the AppArmor rules are enforced. + - Setting of proper context data with attributes like deployment, az, id, instance etc.. All the data from [syslog-release-forwarding-setup.conf.erb](https://github.com/cloudfoundry/syslog-release/blob/main/jobs/syslog_forwarder/templates/syslog-release-forwarding-setup.conf.erb) has to be covered. +- The data in OpenTelemetry format will follow the [OpenTelemetry Semantic Conventions for Cloud Foundry](https://opentelemetry.io/docs/specs/semconv/resource/cloudfoundry/). SAP has internal conventions for BOSH which have to be published as well as they are crucial in order to have consistent naming scheme. + +##### Phase 2 - Implementation + +The implementation will based on the findings from the `Discovery Phase` and will follow the new and modern trends for processing logs with Open Telemetry. + +The three main topics in the implementation phase will be: +- `otel-collector-release` adjustments to support collection and delivery of platform logs: + - BOSH release: + - adjust the BOSH jobs, so that they support the same functionality that is provided with the `syslog-release` + - OpenTelemetry Collector: + - introduce new receivers, processors and extension + - introduce some standard pipeline(s) which process the data in the same way that the current `syslog-release` does + - add examples on how to do further processing of the data inside the OpenTelemetry Collector + +- Ops-file for activating collection and processing of platform component logs with OpenTelemetry + - The default platform logs forwarding mechanism will remain Syslog up until further notice + +#### Deliverables + +- Adjusted `otel-collector-release` which supports consumption and processing of platform component logs +- Ops file in [cf-deployment](https://github.com/cloudfoundry/cf-deployment) with which the operators can select the mechanism for forwarding platform logs + +## Impact and Consequences + +### Positive + +- Platform operators will be able to use modern technologies to process Cloud Foundry platform logs +- Easier integration with other Observability backends +- Ability to correlate platform component logs with platform component metrics and get better insights much faster + +### Negative + +- Usage of OpenTelemetry Semantic Convention is a must to enable good inter-operability with other systems, but they mean that operators have to adjust their alerts and dashboards in their observability backends, because of the changes of the names of the log metadata +- On the Diego Cells the application logs might affect the throughput and delivery of the platform logs. The operators should set an [application log rate limit](https://docs.cloudfoundry.org/loggregator/app-log-rate-limits.html) in order to limit the produced application logs which are accepted in Loggregator. + From 8683cf112fe0cd0241cc10ed4cf118215e3080c1 Mon Sep 17 00:00:00 2001 From: Joris Baum Date: Fri, 21 Aug 2026 14:32:58 +0200 Subject: [PATCH 2/9] Fix minor typos --- toc/rfc/rfc-platform-logs-with-opentelemetry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-platform-logs-with-opentelemetry.md index c3b653876..647e590f0 100644 --- a/toc/rfc/rfc-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-platform-logs-with-opentelemetry.md @@ -26,7 +26,7 @@ The OpenTelemetry Collector offers huge advantages over Syslog: ## Proposal -We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all oft the goodies mentioned above. The `syslog-release` has to be analyzed in its smallest detail to get to know of all its functionalities and find their proper replacements with OpenTelemetry. +We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all of the goodies mentioned above. The `syslog-release` has to be analyzed in its smallest detail to get to know of all its functionalities and find their proper replacements with OpenTelemetry. ### Affected Working Groups - Foundational Infrastructure @@ -58,7 +58,7 @@ Before we start with the technical implementation we have to: ##### Phase 2 - Implementation -The implementation will based on the findings from the `Discovery Phase` and will follow the new and modern trends for processing logs with Open Telemetry. +The implementation will be based on the findings from the `Discovery Phase` and will follow the new and modern trends for processing logs with Open Telemetry. The three main topics in the implementation phase will be: - `otel-collector-release` adjustments to support collection and delivery of platform logs: From b90a2bd2a5c37a8b62b7c812f5029d4ad36fbf74 Mon Sep 17 00:00:00 2001 From: Joris Baum Date: Fri, 21 Aug 2026 16:25:24 +0200 Subject: [PATCH 3/9] Fix a bunch of typos --- toc/rfc/rfc-platform-logs-with-opentelemetry.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-platform-logs-with-opentelemetry.md index 647e590f0..cfb3d45e7 100644 --- a/toc/rfc/rfc-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-platform-logs-with-opentelemetry.md @@ -26,7 +26,7 @@ The OpenTelemetry Collector offers huge advantages over Syslog: ## Proposal -We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all of the goodies mentioned above. The `syslog-release` has to be analyzed in its smallest detail to get to know of all its functionalities and find their proper replacements with OpenTelemetry. +We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all of the goodies mentioned above. The `syslog-release` has to be analyzed to get to know all its functionalities and find their proper replacements with OpenTelemetry. ### Affected Working Groups - Foundational Infrastructure @@ -34,7 +34,7 @@ We propose adding support to the [otel-collector-release](https://github.com/clo ### Implementation -Cloud Foundry supports platform log collection on two major type of operating systems Unix-like with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). It has to be ensured that the OpenTelemetry replacement works equally good for both of them. Both releases have to be analyzed to find the details how they function, so that that proper replacement can be built. +Cloud Foundry supports platform log collection on two major types of operating systems: Linux with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). It has to be ensured that the OpenTelemetry replacement works equally good for both of them. Both releases have to be analyzed to find the details how they function, so that that proper replacement can be built. #### Focus From 441c0d7770ab636ad0d859e0a67483e1c08d8899 Mon Sep 17 00:00:00 2001 From: Joris Baum Date: Fri, 21 Aug 2026 16:39:07 +0200 Subject: [PATCH 4/9] Add affected components --- toc/rfc/rfc-platform-logs-with-opentelemetry.md | 1 + 1 file changed, 1 insertion(+) diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-platform-logs-with-opentelemetry.md index cfb3d45e7..ddb65ac17 100644 --- a/toc/rfc/rfc-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-platform-logs-with-opentelemetry.md @@ -6,6 +6,7 @@ - Status: Draft - Related RFCs : [rfc-0018-aggregate-metric-egress-with-opentelemetry-collector](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md) - RFC Pull Request: +- Affected Component(s): otel-collector-release, cf-deployment ## Summary From 0c965be86d1d59befe9fc4a7845c4a4ee4e0ed28 Mon Sep 17 00:00:00 2001 From: Joris Baum Date: Fri, 21 Aug 2026 17:00:05 +0200 Subject: [PATCH 5/9] Fix spacing --- toc/rfc/rfc-platform-logs-with-opentelemetry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-platform-logs-with-opentelemetry.md index ddb65ac17..4d94edc8d 100644 --- a/toc/rfc/rfc-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-platform-logs-with-opentelemetry.md @@ -4,7 +4,7 @@ - Start Date: 2026-08-19 - Author(s): @chombium, @jorbaum, @Katharin4, @beyhan, @silvestre - Status: Draft -- Related RFCs : [rfc-0018-aggregate-metric-egress-with-opentelemetry-collector](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md) +- Related RFCs: [rfc-0018-aggregate-metric-egress-with-opentelemetry-collector](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md) - RFC Pull Request: - Affected Component(s): otel-collector-release, cf-deployment From bde34c73f4096bf7b27f4307ccff0ee9c2d56a3f Mon Sep 17 00:00:00 2001 From: Joris Baum Date: Mon, 24 Aug 2026 15:21:50 +0200 Subject: [PATCH 6/9] Questions, diagram, semantic convention clarification --- .../rfc-platform-logs-with-opentelemetry.md | 53 ++++++++++++++++--- 1 file changed, 47 insertions(+), 6 deletions(-) diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-platform-logs-with-opentelemetry.md index 4d94edc8d..5180337c9 100644 --- a/toc/rfc/rfc-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-platform-logs-with-opentelemetry.md @@ -17,9 +17,9 @@ We believe that adding OpenTelemetry support for platform component logs will en ## Problem -In Cloud Foundry the platform component logs are collected and processed with the [syslog-release](https://github.com/cloudfoundry/syslog-release) which provides a tool to tail log files from `/var/vcap/sys/log` called [blackbox](https://github.com/cloudfoundry/blackbox) and forwarding and processing rules for [RSyslog](https://www.rsyslog.com/). Rsyslog is provided with the stemcell. Rsyslog is highly specialized for parsing and processing text strings and JSON, but is hard to extend (depends on plugins delivered with the stemcell), its configuration is done with ReinerScript and some legacy formats. Rsyslog still has some use case, but it doesn't fit in the modern observability pipelines. +In Cloud Foundry the platform component logs are collected and processed with the [syslog-release](https://github.com/cloudfoundry/syslog-release) which provides a tool to tail log files from `/var/vcap/sys/log` called [blackbox](https://github.com/cloudfoundry/blackbox) and forwarding and processing rules for [RSyslog](https://www.rsyslog.com/). Rsyslog is provided with the stemcell. Rsyslog is highly specialized for parsing and processing text strings and JSON, but is hard to extend (depends on plugins delivered with the stemcell), its configuration is done with ReinerScript and some legacy formats. Rsyslog still has some use cases, but it doesn't fit in the modern observability pipelines. -On the other hand, OpenTelemetry has become the defacto standard in the Observability World and it has vibrant community. +On the other hand, OpenTelemetry has become the defacto standard in the Observability World and it has a vibrant community. The OpenTelemetry Collector offers huge advantages over Syslog: - Can be easily extended with many community managed components like receivers, processors, extensions and exporters from the [opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) repository. - The configuration is standardized yaml. @@ -29,13 +29,15 @@ The OpenTelemetry Collector offers huge advantages over Syslog: We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all of the goodies mentioned above. The `syslog-release` has to be analyzed to get to know all its functionalities and find their proper replacements with OpenTelemetry. +The `syslog-release` remains the default and stays in place; OpenTelemetry is offered as an opt-in alternative. Deprecating or removing the `syslog-release` is out of scope for this RFC and would be handled by a future RFC. + ### Affected Working Groups - Foundational Infrastructure - Application Runtime Platform ### Implementation -Cloud Foundry supports platform log collection on two major types of operating systems: Linux with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). It has to be ensured that the OpenTelemetry replacement works equally good for both of them. Both releases have to be analyzed to find the details how they function, so that that proper replacement can be built. +Cloud Foundry supports platform log collection on two major types of operating systems: Linux with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). It has to be ensured that the OpenTelemetry replacement works equally good for both of them. Both releases have to be analyzed to find the details how they function, so that proper replacement can be built. #### Focus @@ -43,7 +45,25 @@ This implementation will focus on collection and processing of the platform comp The two basic functions of the `syslog-release` will be replaced as follows: - blackbox -> [filelogreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) -- RSyslog parsing rules -> Otel Collector Pipeline which with the same functionality. Additional processors and receivers might be needed +- RSyslog parsing rules -> Otel Collector Pipeline with the same functionality. Additional processors and receivers might be needed + +The following diagram contrasts today's syslog-based flow with the proposed OpenTelemetry Collector pipeline: + +```mermaid +flowchart LR + subgraph today["Today: syslog-release"] + L1["/var/vcap/sys/log/*.log"] --> BB[blackbox] + BB --> RS[RSyslog
parsing rules] + RS --> SD[Syslog drain /
backend] + end + + subgraph proposed["Proposed: otel-collector-release"] + L2["/var/vcap/sys/log/*.log"] --> FLR[filelogreceiver] + FLR --> PROC[processors
parse + set resource attrs] + PROC --> EXP[exporters
OTLP / syslog / ...] + EXP --> BE[Observability
backend] + end +``` #### Phases @@ -55,7 +75,7 @@ Before we start with the technical implementation we have to: - Configuration parameters - Access permissions for log files. On Ubuntu Noble and higher the AppArmor rules are enforced. - Setting of proper context data with attributes like deployment, az, id, instance etc.. All the data from [syslog-release-forwarding-setup.conf.erb](https://github.com/cloudfoundry/syslog-release/blob/main/jobs/syslog_forwarder/templates/syslog-release-forwarding-setup.conf.erb) has to be covered. -- The data in OpenTelemetry format will follow the [OpenTelemetry Semantic Conventions for Cloud Foundry](https://opentelemetry.io/docs/specs/semconv/resource/cloudfoundry/). SAP has internal conventions for BOSH which have to be published as well as they are crucial in order to have consistent naming scheme. +- **Prerequisite — agree on a resource-attribute naming convention.** The data in OpenTelemetry format will follow the [OpenTelemetry Semantic Conventions for Cloud Foundry](https://opentelemetry.io/docs/specs/semconv/resource/cloudfoundry/). Those conventions cover `cloudfoundry.*` (app/org/space/process) attributes but do **not** define BOSH-level context (deployment, az, instance-group, instance-id, job). This RFC therefore has to *establish* a convention for those BOSH attributes rather than merely follow an existing one. SAP has internal conventions for BOSH which we propose to contribute and publish as the basis for this. Because attribute names are the public contract with every downstream backend, they must be settled before Phase 2 produces stable output. See [Open Questions](#open-questions). ##### Phase 2 - Implementation @@ -88,6 +108,27 @@ The three main topics in the implementation phase will be: ### Negative -- Usage of OpenTelemetry Semantic Convention is a must to enable good inter-operability with other systems, but they mean that operators have to adjust their alerts and dashboards in their observability backends, because of the changes of the names of the log metadata +- When opting in to use OpenTelemetry instead of syslog, operators have to adjust their alerts and dashboards in their observability backends, because the OpenTelemetry Semantic Conventions change the names of the log metadata. This is a one-time cost of adopting the new opt-in path, not a change forced on operators who stay on syslog. - On the Diego Cells the application logs might affect the throughput and delivery of the platform logs. The operators should set an [application log rate limit](https://docs.cloudfoundry.org/loggregator/app-log-rate-limits.html) in order to limit the produced application logs which are accepted in Loggregator. +## FAQ + +### Why extend the `otel-collector-release` instead of creating a new release? + +The `otel-collector-release` already deploys an OpenTelemetry Collector to CF VMs for metric egress (see [RFC-0018](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md)). Reusing it keeps platform logs and metrics in the same Collector, which is what enables the cross-signal correlation described above, and avoids operators having to deploy and maintain a second release with an overlapping component set. + +### Why not keep RSyslog and just add an OpenTelemetry exporter to it? + +That would preserve the parts of the current pipeline we most want to move away from: RSyslog is hard to extend, its configuration relies on RainerScript and legacy formats, and its plugins are tied to the stemcell. Bolting an exporter onto RSyslog carries that maintenance burden forward instead of moving processing into the standardized, community-supported OpenTelemetry Collector configuration. + +### Why OpenTelemetry over other log shippers (Fluent Bit, Vector, etc.)? + +OpenTelemetry has become the de facto standard in the observability space, has a large and active community, and is already established in Cloud Foundry via the `otel-collector-release` for metrics. Standardizing on a single collector for logs and metrics reduces the agent footprint and the number of distinct configuration formats operators have to learn. + +## Open Questions + +- **BOSH resource-attribute naming convention.** The OpenTelemetry Semantic Conventions for Cloud Foundry do not define BOSH-level attributes (deployment, az, instance-group, instance-id, job). This RFC proposes to establish one, seeded from SAP's internal BOSH conventions. Open: is this owned as a Cloud Foundry standard, who maintains it, and should it be pursued for upstream OpenTelemetry adoption? This must be settled before Phase 2. +- **Windows parity.** Can the OpenTelemetry Collector / `filelogreceiver` cover everything `windows-syslog-release` does, and with acceptable resource usage on Windows cells? To be answered in Phase 1. +- **Log-file access permissions.** On Ubuntu Noble and higher, AppArmor rules are enforced. What changes are needed for the Collector to read `/var/vcap/sys/log` under those rules? +- **Non-standard log paths.** Handling logs stored outside `/var/vcap/sys/log` is currently listed as nice-to-have — do any in-scope components require it? + From dd3e3b7004d6f047a0f2780e3a5bea6b9fe0143b Mon Sep 17 00:00:00 2001 From: Beyhan Veli Date: Thu, 10 Sep 2026 11:38:14 +0200 Subject: [PATCH 7/9] Add small fixes RFC and K8s --- ...draft-platform-logs-with-opentelemetry.md} | 46 ++++++++++++------- 1 file changed, 29 insertions(+), 17 deletions(-) rename toc/rfc/{rfc-platform-logs-with-opentelemetry.md => rfc-draft-platform-logs-with-opentelemetry.md} (76%) diff --git a/toc/rfc/rfc-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md similarity index 76% rename from toc/rfc/rfc-platform-logs-with-opentelemetry.md rename to toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md index 5180337c9..bb1f374bd 100644 --- a/toc/rfc/rfc-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md @@ -27,9 +27,9 @@ The OpenTelemetry Collector offers huge advantages over Syslog: ## Proposal -We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all of the goodies mentioned above. The `syslog-release` has to be analyzed to get to know all its functionalities and find their proper replacements with OpenTelemetry. +We propose adding support to the [otel-collector-release](https://github.com/cloudfoundry/otel-collector-release) for collection and processing of platform component logs in a way that is compatible with what the `syslog-release` currently offers and the use cases it covers. Having the platform component logs and metrics in the same place would enable usage for all of the goodies mentioned above. The `syslog-release` should be analyzed to understand its functionalities and identify how they can be supported by an OpenTelemetry-based alternative. -The `syslog-release` remains the default and stays in place; OpenTelemetry is offered as an opt-in alternative. Deprecating or removing the `syslog-release` is out of scope for this RFC and would be handled by a future RFC. +The `syslog-release` remains the default and stays in place; OpenTelemetry is offered as an opt-in alternative. Deprecating or removing the `syslog-release` is out of scope for this RFC and and may be addressed in a future RFC if needed. ### Affected Working Groups - Foundational Infrastructure @@ -37,13 +37,13 @@ The `syslog-release` remains the default and stays in place; OpenTelemetry is of ### Implementation -Cloud Foundry supports platform log collection on two major types of operating systems: Linux with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). It has to be ensured that the OpenTelemetry replacement works equally good for both of them. Both releases have to be analyzed to find the details how they function, so that proper replacement can be built. +Cloud Foundry supports platform log collection on two major types of operating systems: Linux with the `syslog-release` and Windows with the [windows-syslog-release](https://github.com/cloudfoundry/windows-syslog-release). To offer a well-integrated opt-in option, both releases should be analyzed so the OpenTelemetry-based approach can complement rather than disrupt existing setups. #### Focus This implementation will focus on collection and processing of the platform component logs (BOSH job logs) stored in `/var/vcap/sys/log` directory. Supporting other paths where logs are stored is nice to have capability. -The two basic functions of the `syslog-release` will be replaced as follows: +The two basic functions of the `syslog-release` will be supported by the OpenTelemetry-based approach as follows: - blackbox -> [filelogreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) - RSyslog parsing rules -> Otel Collector Pipeline with the same functionality. Additional processors and receivers might be needed @@ -67,31 +67,43 @@ flowchart LR #### Phases -##### Phase 1 - Discovery and Stakeholder Alignment (prerequisite) +##### Phase 1 - Discovery and Stakeholder Alignment -Before we start with the technical implementation we have to: -- Analyze the internals of both `syslog-release` and `windows-syslog-release` release, how they work and what functionality do they provide: - - BOSH jobs - - Configuration parameters - - Access permissions for log files. On Ubuntu Noble and higher the AppArmor rules are enforced. - - Setting of proper context data with attributes like deployment, az, id, instance etc.. All the data from [syslog-release-forwarding-setup.conf.erb](https://github.com/cloudfoundry/syslog-release/blob/main/jobs/syslog_forwarder/templates/syslog-release-forwarding-setup.conf.erb) has to be covered. -- **Prerequisite — agree on a resource-attribute naming convention.** The data in OpenTelemetry format will follow the [OpenTelemetry Semantic Conventions for Cloud Foundry](https://opentelemetry.io/docs/specs/semconv/resource/cloudfoundry/). Those conventions cover `cloudfoundry.*` (app/org/space/process) attributes but do **not** define BOSH-level context (deployment, az, instance-group, instance-id, job). This RFC therefore has to *establish* a convention for those BOSH attributes rather than merely follow an existing one. SAP has internal conventions for BOSH which we propose to contribute and publish as the basis for this. Because attribute names are the public contract with every downstream backend, they must be settled before Phase 2 produces stable output. See [Open Questions](#open-questions). +**Track 1 — Analyze existing releases** + +Analyze the internals of both `syslog-release` and `windows-syslog-release` to understand what functionality they provide: +- BOSH jobs +- Configuration parameters +- Access permissions for log files. On Ubuntu Noble and higher the AppArmor rules are enforced. +- Setting of proper context data with attributes like deployment, az, id, instance etc. All the data from [syslog-release-forwarding-setup.conf.erb](https://github.com/cloudfoundry/syslog-release/blob/main/jobs/syslog_forwarder/templates/syslog-release-forwarding-setup.conf.erb) has to be covered. + +**Track 2 — Agree on a resource-attribute naming convention** + +The OpenTelemetry format will follow the [OpenTelemetry Semantic Conventions for Cloud Foundry](https://opentelemetry.io/docs/specs/semconv/resource/cloudfoundry/). Those conventions cover `cloudfoundry.*` (app/org/space/process) attributes but do **not** define BOSH-level context (deployment, az, instance-group, instance-id, job). A convention for those BOSH attributes therefore needs to be established. SAP has internal conventions for BOSH which we propose to contribute and publish as the basis for this. Because attribute names are the public contract with every downstream backend, they must be settled and documented before Phase 2 produces stable output. See [Open Questions](#open-questions). ##### Phase 2 - Implementation The implementation will be based on the findings from the `Discovery Phase` and will follow the new and modern trends for processing logs with Open Telemetry. -The three main topics in the implementation phase will be: +The implementation phase covers two deployment targets: + +**BOSH-based CF (cf-deployment)** + - `otel-collector-release` adjustments to support collection and delivery of platform logs: - BOSH release: - - adjust the BOSH jobs, so that they support the same functionality that is provided with the `syslog-release` + - adjust the BOSH jobs, so that they support the same functionality that is provided with the `syslog-release` - OpenTelemetry Collector: - - introduce new receivers, processors and extension + - introduce new receivers, processors and extensions - introduce some standard pipeline(s) which process the data in the same way that the current `syslog-release` does - add examples on how to do further processing of the data inside the OpenTelemetry Collector - - Ops-file for activating collection and processing of platform component logs with OpenTelemetry - - The default platform logs forwarding mechanism will remain Syslog up until further notice + - The default platform logs forwarding mechanism will remain Syslog up until further notice + +**CF on Kubernetes** + +- Investigate how platform component logs are currently collected and forwarded in a CF on Kubernetes deployment +- Define the equivalent opt-in mechanism to enable OpenTelemetry-based log collection alongside the existing approach +- Provide the necessary configuration (e.g. Helm values, overlays) to activate the OpenTelemetry pipeline as an opt-in alternative #### Deliverables From 7f90865be143cb6aa4ab3e5093efa1c25be69ed9 Mon Sep 17 00:00:00 2001 From: Beyhan Veli Date: Fri, 11 Sep 2026 16:27:26 +0200 Subject: [PATCH 8/9] Update toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md Co-authored-by: Joris Baum --- toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md index bb1f374bd..0a34d3ca6 100644 --- a/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md @@ -41,7 +41,7 @@ Cloud Foundry supports platform log collection on two major types of operating s #### Focus -This implementation will focus on collection and processing of the platform component logs (BOSH job logs) stored in `/var/vcap/sys/log` directory. Supporting other paths where logs are stored is nice to have capability. +This implementation will focus on collection and processing of the platform component logs (BOSH job logs) stored in `/var/vcap/sys/log` directory. Supporting logs stored in other paths is a nice-to-have capability. The two basic functions of the `syslog-release` will be supported by the OpenTelemetry-based approach as follows: - blackbox -> [filelogreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) From bf87f4fcc6c713170227787962bfdcaf7d2d6aaf Mon Sep 17 00:00:00 2001 From: Beyhan Veli Date: Fri, 11 Sep 2026 16:28:37 +0200 Subject: [PATCH 9/9] Update start date --- toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md b/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md index 0a34d3ca6..0587770ff 100644 --- a/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md +++ b/toc/rfc/rfc-draft-platform-logs-with-opentelemetry.md @@ -1,7 +1,7 @@ # Meta [meta]: #meta - Name: Support Platform Logs Collection and Processing with OpenTelemetry -- Start Date: 2026-08-19 +- Start Date: 2026-09-10 - Author(s): @chombium, @jorbaum, @Katharin4, @beyhan, @silvestre - Status: Draft - Related RFCs: [rfc-0018-aggregate-metric-egress-with-opentelemetry-collector](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md)