Skip to content

Remove kolla entrypoint pattern from Placement, harden SecurityContext - #1176

Open
stuggi wants to merge 1 commit into
openstack-k8s-operators:mainfrom
stuggi:remove-kolla-placement
Open

Remove kolla entrypoint pattern from Placement, harden SecurityContext#1176
stuggi wants to merge 1 commit into
openstack-k8s-operators:mainfrom
stuggi:remove-kolla-placement

Conversation

@stuggi

@stuggi stuggi commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Eliminate kolla_start/kolla_set_configs and the config.json staging pattern for the placement service. Configs are now mounted directly to final paths via SubPath volume mounts, removing the need for root privilege escalation at container startup. Second operator in the remove-kolla effort, following keystone-operator.

Key changes:

  • Delete placement-api-config.json / placement-dbsync-config.json
  • Mount configs via SubPath to /etc/placement/, /etc/httpd/, /etc/my.cnf
  • Add emptyDir volumes for writable paths (/run/httpd, /tmp, /var/log/httpd) using lib-common volume.WritableDirVolume() and volume.WritableDirVolumeMount() helpers with named constants (RunHttpdVolumeName, TmpVolumeName, VarLogHttpdVolumeName); the pre-existing /var/log/placement "logs" emptyDir (used by the oslo.log tailing sidecar) is unrelated to kolla and unchanged
  • Standardize config-data Secret DefaultMode to 0440 (was 0640), matching the lib-common configSecretMode convention
  • Replace kolla_start with /usr/sbin/httpd -DFOREGROUND (api container) and placement-manage db sync (db-sync job)
  • Remove KOLLA_CONFIG_STRATEGY and KOLLA_BOOTSTRAP env vars
  • Use pod.RestrictiveSecurityContext and RestrictivePodSecurityContext from lib-common on all 3 containers (-log, -api, -db-sync), setting ReadOnlyRootFilesystem, dropping ALL capabilities, and enabling seccomp RuntimeDefault
  • Set AutomountServiceAccountToken=false on both workloads
  • Import PlacementUID from lib-common modules/serviceuser
  • Override TLS CertMount/KeyMount to /etc/pki/tls/ for Apache
  • Migrate placement's SCC from anyuid to nonroot-v2 -- additive change in this multi-controller operator: nova and cyborg keep their own anyuid markers/rbacRules untouched, config/rbac/role.yaml gains a new nonroot-v2 rule alongside the existing anyuid one
  • httpd.conf: User/Group placement, PidFile /run/httpd/httpd.pid, re-enable Include conf.d/*.conf (previously disabled due to a syntax error that no longer applies once kolla's generated ssl.conf is what's actually loaded), vhost ErrorLog to stderr
  • policy.yaml mounted conditionally on DefaultConfigOverwrite, matching prior (narrow) kolla behavior -- no scope expansion
  • Adopt lib-common volume helpers (volume.WritableDirVolume, volume.WritableDirVolumeMount) for all emptyDir volumes and mounts, using constants for run-httpd, var-log-httpd, and tmp names/paths
  • Update functional and kuttl tests

Jira: OSPRH-33504
Jira: OSPRH-33503

Depends-On: openstack-k8s-operators/lib-common#728

@openshift-ci

openshift-ci Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: stuggi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved label Aug 6, 2026
func getVolumes(name string) []corev1.Volume {
var scriptsVolumeDefaultMode int32 = 0755
var configMode int32 = 0640
var configMode int32 = 0440

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is both more and less secure then before.

we intenally made show the placemetn cofnig were owned by placment adn restricted to 0600 by kolla

i guess group access is ok
so 0440 is now makign the etc mounts readony but expanding it to the group

we dintentally didnt do this before bacue the projected files will be owned as root whcih sis incorrect

now we can use

fsGroup: 1001 in the securityContext

to change the group fo the projected fiels to bey placement gid
and 0440 will then allow placement to read it

but this sitll change the cofnig to be owned as root which while it would work is not a good security practice in general.

as far as im aware there is no way to set the user of the fiels natively in k8s so this is a trade off we can make but the other reason we didnt do this in the past was we want the same behvior on the edpm nodes

so if we are entrusting this to k8s to do via projecting the volume with specific permission we will need to restructure the ansibel to ensure the same is done on the edpm node via bind mounts and chowing the files on the host.

doable but we should not keep kolla in one place and remvoe it in the other. that does not impact placmenet but its a wider condieration

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes correct, using the fsgroup in the context is what we do. I just see that I pass the UID, which works as long the UID/GID are the same, but I should update the RestrictivePodSecurityContext helper func to also take the GID.

we discussed the consistency topic re ctlplane/edpm in yesterdays tech call and the consent was that when we do the move away from kolla on the ctlplane side to be able to not have to run the deployments with elevated permissions, which kolla need to run sudo, we should also do it on the edpm side, to be consistent. jslagle took the item to create the tracker for it and follow up.

Eliminate kolla_start/kolla_set_configs and the config.json staging
pattern for the placement service. Configs are now mounted directly
to final paths via SubPath volume mounts, removing the need for root
privilege escalation at container startup. Second operator in the
remove-kolla effort, following keystone-operator.

Key changes:
- Delete placement-api-config.json / placement-dbsync-config.json
- Mount configs via SubPath to /etc/placement/, /etc/httpd/, /etc/my.cnf
- Add emptyDir volumes for writable paths (/run/httpd, /tmp,
  /var/log/httpd) using lib-common volume.WritableDirVolume() and
  volume.WritableDirVolumeMount() helpers with named constants
  (RunHttpdVolumeName, TmpVolumeName, VarLogHttpdVolumeName); the
  pre-existing /var/log/placement "logs" emptyDir (used by the oslo.log
  tailing sidecar) is unrelated to kolla and unchanged
- Standardize config-data Secret DefaultMode to 0440 (was 0640),
  matching the lib-common configSecretMode convention
- Replace kolla_start with /usr/sbin/httpd -DFOREGROUND (api container)
  and placement-manage db sync (db-sync job)
- Remove KOLLA_CONFIG_STRATEGY and KOLLA_BOOTSTRAP env vars
- Use pod.RestrictiveSecurityContext and RestrictivePodSecurityContext
  from lib-common on all 3 containers (-log, -api, -db-sync), setting
  ReadOnlyRootFilesystem, dropping ALL capabilities, and enabling
  seccomp RuntimeDefault
- Set AutomountServiceAccountToken=false on both workloads
- Import PlacementUID from lib-common modules/serviceuser
- Override TLS CertMount/KeyMount to /etc/pki/tls/ for Apache
- Migrate placement's SCC from anyuid to nonroot-v2 -- additive change
  in this multi-controller operator: nova and cyborg keep their own
  anyuid markers/rbacRules untouched, config/rbac/role.yaml gains a
  new nonroot-v2 rule alongside the existing anyuid one
- httpd.conf: User/Group placement, PidFile /run/httpd/httpd.pid,
  re-enable Include conf.d/*.conf (previously disabled due to a
  syntax error that no longer applies once kolla's generated ssl.conf
  is what's actually loaded), vhost ErrorLog to stderr
- policy.yaml mounted conditionally on DefaultConfigOverwrite, matching
  prior (narrow) kolla behavior -- no scope expansion
- Adopt lib-common volume helpers (volume.WritableDirVolume,
  volume.WritableDirVolumeMount) for all emptyDir volumes and mounts,
  using constants for run-httpd, var-log-httpd, and tmp names/paths
- Update functional and kuttl tests

Jira: OSPRH-33504    
Jira: OSPRH-33503    

Depends-On: openstack-k8s-operators/lib-common#728

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Martin Schuppert <mschuppert@redhat.com>
@stuggi
stuggi force-pushed the remove-kolla-placement branch from 3efbf27 to bb15f3f Compare August 7, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants