gitstream pr context variable doesn't contain all pr checks

[dorongutman]

Describe the bug

In my PR there are checks showing the status of external CI systems runs, like the docker build, or whether tests passed or failed.
However, when I iterate over pr.checks in gitstream these statuses don't show.

To Reproduce

See full repro in dorongutman/reproduce-gitstream-main#2

I manually created a status check with:

curl -X POST \
  -H "Authorization: token $GITHUB_TOKEN" \
  -H "Accept: application/vnd.github.v3+json" \
  https://api.github.com/repos/dorongutman/reproduce-gitstream-main/statuses/e87a06aa1a29e94e89876248dd1e644fc6aadba8 \
  -d '{
    "state": "success",
    "target_url": "https://example.com",
    "description": "External check running",
    "context": "my-external-service"
  }'

and I used this gitstream.cm file to dump the contents of pr.checks (I tried the entire pr object and also the entire variable contexts in the playground - same thing):

# -*- mode: yaml -*-

manifest:
  version: 1.0
automations:
  dump_checks:
    on:
      - comment_added
    if:
      - true
    run:
      - action: add-comment@v1
        args:
          comment: |
            CHECKS DUMP:
            ```yaml
            {{ pr.checks | dump(2) | indent(12) | safe }}
            ```

Expected behavior

I expect to have all of the checks in pr.checks so I could approve the gitstream check upon successful build and/or tests passing.

[vim-zz]

Hey @dorongutman the scenario you provided shows a check, but under the hood in GitHub this is a commit status as checks are only generated by GitHub apps, thereof it is not collected. Saying that, we do see the added value of collecting commits status as well, and we'll add to the roadmap (I cannot commit on a date at the moment, but we plan to add this)

[MishaKav]

While thise feature on roadmap, you can achieve the same functionality by creating a custom plugin that fetches commit statuses via the GitHub API and makes them available in your gitStream automations.

Custom Plugin Implementation

Create a file .cm/plugins/getCommitStatuses/index.js:

const { Octokit } = require('@octokit/rest');

/**
 * Get CLIENT_PAYLOAD from environment variables
 * @returns {Object} - The client payload object
 */
const getClientPayloadParsed = () => {
  const afterOneParsing = JSON.parse(process.env.CLIENT_PAYLOAD || '{}');

  if (typeof afterOneParsing === 'string') {
    return JSON.parse(afterOneParsing);
  }

  return afterOneParsing;
};

/**
 * Retrieves commit statuses for a commit by fetching status checks from GitHub API
 * Gets commit SHA, repository info, and GitHub token from client payload
 *
 * @function getCommitStatuses
 * @param {*} _ - Unused parameter (context data is retrieved from client payload)
 * @param {Function} callback - Callback function to return results
 * @returns {Array<Object>} - Array of commit status checks with name, status, conclusion, title, completedAt, startedAt
 */
const getCommitStatuses = async (_, callback) => {
  try {
    const clientPayload = getClientPayloadParsed();
    const { githubToken, headSha: commit_sha, owner, repo } = clientPayload;

    if (!githubToken || typeof githubToken !== 'string') {
      console.log(`missing githubToken or bad format`);
      return callback(null, '[]');
    }

    const octokit = new Octokit({ auth: githubToken });

    // get commit statuses for the commit
    const { data: commitStatuses } = await octokit.request(
      'GET /repos/{owner}/{repo}/commits/{commit_sha}/status',
      {
        headers: { 'X-GitHub-Api-Version': '2022-11-28' },
        owner,
        repo,
        commit_sha
      }
    );

    // format the response to match the original checks structure
    const formattedChecks = commitStatuses.statuses.map(status => ({
      name: status.context,
      status: status.state === 'success' ? 'completed' : status.state,
      conclusion: status.state,
      title: status.description,
      completedAt: status.updated_at,
      startedAt: status.created_at
    }));

    return callback(null, JSON.stringify(formattedChecks));
  } catch (error) {
    console.log(error?.message);
    return callback(null, error?.message);
  }
};

module.exports = {
  async: true,
  immediate: true,
  filter: getCommitStatuses
};

Here's how to use the plugin in your .cm file:

# -*- mode: yaml -*-
manifest:
  version: 1.0

automations:
  dump_checks:
    if:
      - true
    run:
      - action: add-comment@v1
        args:
          comment: |
            CHECKS DUMP:
            ```yaml
            {{ pr.checks | dump(2) | indent(12) }}
            ```

            COMMIT STATUSES DUMP:
            ```yaml
            {{ COMMIT_STATUSES | dump(2) | indent(12) }}
            ```

COMMIT_STATUSES: {{ "" | getCommitStatuses }}

Examples of reuslts:

Screenshot	Screenshot

P.S. pay attention that that the commit statuses and cheks comes in different format, i add simple formating to align this.