Upstream 16555 - Replace 4-way OR with UNION in unified job list RBAC query

[cigamit]

Upstream Summary

• Rewrites UnifiedJobAccess.filtered_queryset() to use UNION instead of a 4-way OR
• Each RBAC branch (template read_role, inventory update, ad-hoc command, org auditor) becomes a separate queryset combined with .union(), giving PostgreSQL an independent optimal plan per branch
• Under Scale Lab load, the unified job RBAC query accounts for ~35 hours of DB time per 30-minute window; the OR prevents branch-specific index usage and forces suboptimal plan choices

Resolves: AAP-81173

Classification

New or Enhanced Feature

** Note, this will need rebasing after #643 is merged

[Copilot]

Pull request overview

This PR optimizes unified job list RBAC filtering by restructuring UnifiedJobAccess.filtered_queryset() to use a UNION-based shape (instead of a 4-way OR) to improve PostgreSQL query planning and index usage under load. It also introduces a dedicated paginator for the unified job list endpoint that returns an unfiltered count, and adds unit/functional coverage for both the RBAC query shape and pagination behavior.

Changes:

• Refactors unified job RBAC filtering to build four independent PK query branches and combine them via UNION for better planner choices.
• Adds UnifiedJobPagination / UnifiedJobPaginator and wires it into the unified job list API endpoint.
• Adds new unit and functional tests to validate the paginator behavior and that the RBAC query uses UNION.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
awx/main/access.py	Rewrites unified job RBAC filtering to use UNION across PK subqueries instead of a 4-way OR.
awx/api/pagination.py	Adds UnifiedJobPaginator (unfiltered count) and UnifiedJobPagination; fixes paginator restoration logic.
awx/api/views/init.py	Applies the new UnifiedJobPagination to the unified job list endpoint.
awx/main/tests/unit/api/test_pagination.py	Unit tests for unfiltered count behavior and paginator-class restoration semantics.
awx/main/tests/functional/test_rbac_unified_jobs.py	Functional tests asserting UNION usage and verifying RBAC visibility + pagination count behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[TheWitness]

Some copilot comments but I'll leave it to you

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

awx/api/pagination.py:39

• UnifiedJobPaginator.count returns UnifiedJob.objects.count() (unfiltered table count), which means the count field in unified_job_list responses will no longer represent the number of results matching the user's RBAC + query filters. That’s a behavioral/API contract change (and potentially leaks total job volume to unprivileged users). Consider keeping the accurate filtered count by default and only using an unfiltered/approximate count behind an explicit opt-in query param (or rely on the existing count_disabled option to omit the count).

class UnifiedJobPaginator(DjangoPaginator):
    """Use unfiltered table count for unified job pagination.

    The RBAC-filtered COUNT query is prohibitively slow on large unified job
    tables; an approximate over-count is harmless for pagination UI.
    """

    @cached_property
    def count(self):
        return UnifiedJob.objects.count()

awx/main/access.py:2597

• PR title/description says UnifiedJobAccess.filtered_queryset() is being rewritten to use UNION instead of a 4-way OR, but this implementation explicitly keeps an OR shape (and the comments explain why UNION is avoided). This makes the PR intent unclear and risks landing the opposite of the stated performance fix; please align the implementation/tests/comments with the PR goal (either actually switch to UNION, or update the PR title/description to match the OR-based approach).

        # Pre-compute the user's direct role memberships once and pass them
        # as literal parameters to each RBAC branch, and use OR (not UNION)
        # so the database can single-pass filter with an early LIMIT exit.
        # UNION forces materialization of every accessible job id before
        # ordering/LIMIT can apply, and repeats the role-membership subquery
        # in every branch.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (3)

awx/api/pagination.py:91

• UnifiedJobPagination currently uses UnifiedJobPaginator.count (an unfiltered table count). DRF’s PageNumberPagination uses the paginator count/num_pages to decide has_next(), so a user with zero visible jobs can still get a non-null next link and an inflated page count, which is inconsistent with the returned result set.

class UnifiedJobPagination(Pagination):
    django_paginator_class = UnifiedJobPaginator

awx/main/access.py:2596

• This change (and the inline comment) explicitly keeps a 4-way OR in UnifiedJobAccess.filtered_queryset(), but the PR title/description say it should be replaced with UNION. Either the implementation or the PR metadata/tests need to be updated so they describe the same approach/performance rationale.

        # Pre-compute the user's direct role memberships once and pass them
        # as literal parameters to each RBAC branch, and use OR (not UNION)
        # so the database can single-pass filter with an early LIMIT exit.
        # UNION forces materialization of every accessible job id before
        # ordering/LIMIT can apply, and repeats the role-membership subquery

awx/main/tests/functional/test_rbac_unified_jobs.py:22

• This test hard-codes that the unified job RBAC SQL must not contain UNION, which conflicts with the PR’s stated goal of replacing the 4-way OR with UNION. If UNION is the desired behavior, this assertion (and the docstring/name) should be inverted/updated so the test enforces the intended query shape.

@pytest.mark.django_db
def test_unified_job_list_uses_or_not_union(user, organization, inventory, get):
    """The unified job list RBAC query uses OR-based filtering, not UNION."""
    org_admin = user('uj-org-admin')
    organization.admin_role.members.add(org_admin)