Backtestshugorun-20261024-162305#15261

Issue #15261 Merged fix PR #15262 Run 24 Oct 2026 · run-20261024-162305 gohugoio/hugo

github: Fix workflow for stale issues

Worse

The merged fix was judged better than our patch.

In this judgment, Patch A = our agent, Patch B = the merged fix.

The failure is an input-validation bug in the pinned dessant/lock-threads action (it rejects tokens over 100 chars). Patch B addresses the root cause by bumping the pinned action to a newer release (v6.0.2) that lifts that limit, keeping all existing inputs, behavior, and the SHA-pinning convention intact. It is a one-line, low-risk change that matches the issue's ask exactly. Patch A instead rips out the action and reimplements its logic as an ad hoc bash script against the REST API. That is a far broader redesign than the issue calls for and introduces real behavioral and reliability risk: it caps each run at 50 items per type via per_page=50 with no pagination, so a backlog would never be cleared; the search query semantics (updated:<, is:unlocked, archived:false) only approximate the action's inactivity logic; it hardcodes lock_reason=resolved, which the original did not specify; and it changes the stale step to run with !cancelled(), masking failures in the new script. It also loses the maintained, pinned third-party behavior in favor of untested shell. Neither patch includes tests, which is expected for a workflow-only change. On correctness, scope, and craft, B is the better fix.

Four judges ruled on this ticket in parallel, and the verdict comes from counting their votes. Each saw the issue and the two patches, labelled A and B in random order (two of them saw ours as A), with issue links and version numbers removed. Where tests could check the fix, they also saw the results of tests a separate agent wrote for the issue, run on both patches. None was told which patch was ours. The reasoning above is one judge's, shown verbatim.

Solve

2m 40s

Wall-clock time in the container

Full test suite

Not run

The harness did not reach the test stage

Contamination audit

Clean

No sign the agent saw the real fix

Patch size

47vs 2 merged

Changed lines, ours vs merged

The workflow began failing on 26 May 2026:

<https://github.com/gohugoio/hugo/actions/runs/26486945788>

"github-token" length must be less than or equal to 100 characters long

The two patches

Our patch+41 −6
.github/workflows/stale.ymlindex 89dbdde7d..131e4c22d 100644@@ -12,19 +12,54 @@ jobs:       pull-requests: write     runs-on: ubuntu-latest     steps:-      - uses: dessant/lock-threads@f5f995c727ac99a91dec92781a8e34e7c839a65e # v6.0.0-        with:-          issue-inactive-days: 21-          add-issue-labels: 'Outdated'-          issue-comment: >+      # Lock closed issues and PRs with the gh CLI. dessant/lock-threads rejects+      # GitHub tokens longer than 100 characters, which the current token format exceeds.+      - name: Lock closed issues and PRs+        env:+          GH_TOKEN: ${{ github.token }}+          GH_REPO: ${{ github.repository }}+          ISSUE_INACTIVE_DAYS: 21+          ISSUE_LABELS: 'Outdated'+          ISSUE_COMMENT: >             This issue has been automatically locked since there             has not been any recent activity after it was closed.             Please open a new issue for related bugs.-          pr-comment: >+          PR_INACTIVE_DAYS: 365+          PR_LABELS: ''+          PR_COMMENT: >             This pull request has been automatically locked since there             has not been any recent activity after it was closed.             Please open a new issue for related bugs.+        run: |+          set -euo pipefail++          lock_threads() {+            local type="$1" days="$2" labels="$3" comment="$4"+            local before numbers number label+            before="$(date -u -d "-${days} days" +%Y-%m-%dT%H:%M:%SZ)"+            # Collect the numbers before modifying anything, as locking changes the search results.+            numbers="$(gh api -X GET search/issues \+              -f q="repo:${GH_REPO} is:${type} is:closed is:unlocked archived:false updated:<${before}" \+              -f sort=updated -f order=desc -F per_page=50 \+              --jq '.items[].number')"+            for number in ${numbers}; do+              echo "Locking ${type} #${number}"+              IFS=',' read -ra label_list <<< "${labels}"+              for label in "${label_list[@]}"; do+                gh api -X POST "repos/${GH_REPO}/issues/${number}/labels" -f "labels[]=${label}" > /dev/null+              done+              if [ -n "${comment}" ]; then+                gh api -X POST "repos/${GH_REPO}/issues/${number}/comments" -f body="${comment}" > /dev/null+              fi+              gh api -X PUT "repos/${GH_REPO}/issues/${number}/lock" -f lock_reason=resolved > /dev/null+            done+          }++          lock_threads issue "${ISSUE_INACTIVE_DAYS}" "${ISSUE_LABELS}" "${ISSUE_COMMENT}"+          lock_threads pr "${PR_INACTIVE_DAYS}" "${PR_LABELS}" "${PR_COMMENT}"       - uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0+        # Run even if locking failed (e.g. on a rate limit), so stale handling isn't skipped.+        if: ${{ !cancelled() }}         with:           operations-per-run: 999           days-before-issue-stale: 365 
Merged fix · PR #15262+1 −1
.github/workflows/stale.ymlindex 89dbdde7d..73e31f024 100644@@ -12,7 +12,7 @@ jobs:       pull-requests: write     runs-on: ubuntu-latest     steps:-      - uses: dessant/lock-threads@f5f995c727ac99a91dec92781a8e34e7c839a65e # v6.0.0+      - uses: dessant/lock-threads@89ae32b08ed1a541efecbab17912962a5e38981c # v6.0.2         with:           issue-inactive-days: 21           add-issue-labels: 'Outdated'