Engineering Outer Loop Skill
Turn diffs, red CI logs and merged PRs into review-ready write-ups with the risk named.
von GetKits
Sieh den Unterschied
Messy branch to a PR description and review focus
Vorher
Branch feat/ENG-482-webhook-retries. Commits: "wip retries", "fix tests". Diff adds retry columns and CREATE INDEX on webhook_deliveries, 30 s doubling backoff up to 8 attempts, and a worker behind WEBHOOK_RETRY_ENABLED. No CI run shared.
Nachher
Summary: Failed webhook deliveries are now retried, up to 8 attempts with delays doubling from 30 seconds (about 64 minutes), behind WEBHOOK_RETRY_ENABLED, off by default. Risk: High. New index on webhook_deliveries (size unknown), and delivery becomes at-least-once. Rollback: Set WEBHOOK_RETRY_ENABLED=false. Safe. Must review 1. [Tier 1] retry-worker.ts: FOR UPDATE SKIP LOCKED runs outside a transaction, so two runs can send the same delivery twice.
Auth PR to a risk-ranked review checklist
Vorher
Orbit PR #2214 "api keys per org" by Jonas, 9 files: - migration creating api_keys (org_id, hashed_key, scopes, last_used_at) - POST /v1/api-keys and DELETE /v1/api-keys/:id - middleware that finds the key and sets req.org; raw keys compared with === - logs the key prefix on auth failure - OpenAPI additions and a regenerated client in sdk/ - 4 tests, all happy path
Nachher
REVIEW PREP PR: #2214 api keys per org Change types: Auth, Migration, API contract Risk: High because a DELETE lookup without an org check would let any customer revoke another customer's keys. Size: M, about 260 meaningful lines in 9 files. Review time about 30 to 45 minutes Must review (ranked): 1. [Tier 1] DELETE /v1/api-keys/:id handler. Confirm the lookup is scoped to req.org (WHERE id = $1 AND org_id = $2). A lookup by id alone lets one org revoke another org's keys. 2. [Tier 1] Auth middleware, raw keys compared with ===. That leaks timing. Look up by SHA-256 hash in the database, or use crypto.timingSafeEqual. 3. [Tier 1] Auth middleware, failure log. A prefix is fine only if it is a non-secret identifier. Confirm the full key and Authorization header never reach logs, including error paths. 4. [Tier 1] api_keys migration. Confirm hashed_key has a unique index and no column stores the raw key. 5. [Tier 2] POST /v1/api-keys. Confirm only admins can create keys and scopes cannot exceed the creator's own permissions. 6. [Tier 2] Revocation. Is a deleted key rejected on the next request, or can a cache keep it valid? State the maximum delay. 7. [Tier 3] Tests are happy path only. Add: org A cannot delete org B's key; revoked key returns 401; non-admin gets 403 on create. Can skim: OpenAPI additions (new endpoints only, additive), generated client in sdk/. Questions for the author: - Is the key shown exactly once at creation, and where is it hashed? - What rate limit applies to failed key lookups? Missing: negative tests, audit log entries for key create and delete, endpoint docs. Suggested reviewers: [confirm] (no CODEOWNERS shared)
Red e2e job on a README PR to a flaky verdict
Vorher
CI / e2e (chromium) failed on PR #1987 "docs: fix typo in README" at 5d2e9b1. Running 214 tests using 4 workers 1) [chromium] > checkout.spec.ts:88:5 > applies discount code Error: locator.click: Timeout 30000ms exceeded. waiting for getByRole('button', { name: 'Apply' }) 1 failed, 213 passed (6.1m) ##[error]Process completed with exit code 1. From gh run list on main: checkout.spec.ts:88 also failed in runs #5098, #5104 and #5117 out of the last 20, each green on rerun.
Nachher
CI TRIAGE Run: CI / e2e (chromium), PR #1987 at 5d2e9b1 Verdict: FLAKY (confidence high) Signature: FLAKY-E2E Evidence: e2e (chromium) | Error: locator.click: Timeout 30000ms exceeded. e2e (chromium) | waiting for getByRole('button', { name: 'Apply' }) Likely cause: The PR only changes README.md, and the same test failed on main in 3 of the last 20 runs (#5098, #5104, #5117), each passing on rerun. The Apply button is likely clicked before the discount form finishes loading. First bad commit: Not this PR. The failure predates it on main. Smallest next step: Rerun the e2e job once to unblock #1987. Open an issue to quarantine checkout.spec.ts:88 with an owner and a due date. Owner: checkout.spec.ts code owner [confirm] Do not: Do not rerun more than once. Do not raise the 30 s timeout; wait on the discount form's network response instead.
13 merged PRs to release notes, breaking first
Vorher
Merged into main since v2.13.2 (Orbit, 2026-09-15 to 2026-09-29): #1102 feat(api)!: remove deprecated GET /v1/reports (body: deprecated since v2.9, use /v2/reports) #1105 feat(dashboard): add csv export to ReportTable #1107 fix: tz bug weekly (body: week started a day early for negative UTC offsets) #1108 chore(deps): bump axios from 1.7.2 to 1.7.7 #1110 refactor(reports): split builder into modules #1111 perf(api): memoize report totals 60s #1113 fix (body: handle null owner) #1114 ci: run e2e on merge queue only #1116 feat(cli): add orbit reports schedule #1117 test: fiscal year fixtures #1119 fix(auth): SSO session ttl 15m instead of 12h #1120 chore: update CODEOWNERS #1121 build: engines node >=20
Nachher
RELEASE NOTES Version: v3.0.0 suggested (breaking changes present). Range: v2.13.2...main, 13 PRs, 2026-09-15 to 2026-09-29 Breaking changes - GET /v1/reports is removed and returns 404. Integrations still calling it must switch to GET /v2/reports, deprecated since v2.9. (#1102) - The CLI now requires Node.js 20 or later. Upgrade Node before updating. (#1121) New - Export any report as CSV from the dashboard. (#1105) - Schedule recurring reports from the CLI with orbit reports schedule. (#1116) Improved - Report totals load faster. Totals can lag new data by up to 60 seconds. (#1111) Fixed - Weekly reports no longer start one day early for workspaces west of UTC. (#1107) - SSO sessions now last the configured 12 hours instead of ending after 15 minutes. (#1119) Internal (5 changes): axios 1.7.7, report builder refactor, e2e on merge queue, fiscal year fixtures, CODEOWNERS (#1108, #1110, #1114, #1117, #1120) Needs confirmation - #1113 fix ("handle null owner"): user impact unclear. Which screen failed, and for whom? Check: 13 PRs in, 13 accounted for (2 breaking, 2 new, 1 improved, 2 fixed, 5 internal, 1 to confirm)
Das steckt drin
- engineering-outer-loop.zip
Über dieses Kit
Paste a diff, a failing CI log, a list of merged PRs or an incident channel export, and get back the finished write-up: a PR description with risk, test plan and rollback, a risk-ranked review checklist, a CI verdict with the smallest next step, release notes with breaking changes first, or a blameless incident summary. Built for engineers and tech leads working in Claude, ChatGPT, Codex or Cursor. It installs in five minutes, and in an agent with repo access it gathers its own input with read-only git and gh commands.
What you get
- The
engineering-outer-loopskill with five modes, each returning one fixed block: PR DESCRIPTION, REVIEW PREP, CI TRIAGE, RELEASE NOTES and INCIDENT SUMMARY. The same blocks are used by the Engineering Outer Loop Bot and Dot, so drafts move between tools unchanged. - Review checklists by change type: migrations (Postgres lock behavior,
CONCURRENTLY, expand and contract), auth (object-level checks, token validation, OAuth redirects), API contracts (what breaks old mobile clients), dependency bumps (supply chain and runtime changes), infra and CI (Terraform replacements,pull_request_target, IAM), plus concurrency and retries. - A CI failure taxonomy of 22 failure types with the exact log lines that identify them across Jest, pytest, Go, Gradle, Playwright and GitHub Actions, each with a verdict and the smallest next step, plus a flake quarantine policy and a method for finding the first bad commit on main.
- A release notes mapping from conventional commits and labels to sections, with rewrite rules, a phrase bank, security embargo handling and a ready
.github/release.yml. - A PR template and rollout patterns for nine change types, including when rollback stops being safe.
- A blameless postmortem template with a severity rubric, blameless rewrites, follow-up quality rules and a 30-minute review agenda.
- A worked example: a real-shaped webhook retry diff in, PR description and an eight-item review prep out, with the reasoning behind each call.
Who it is for
- Engineers who want PR descriptions that reviewers can act on, without writing them from scratch.
- Reviewers who get a 400-line PR and need to know which 20 lines matter.
- Whoever is on CI duty and needs to tell a flaky test from a real break in two minutes.
- Tech leads who ship release notes every release and run postmortems that produce real follow-ups.
What it will not do
- Write or fix product code. It describes and checks changes.
- Approve PRs or replace a human reviewer. REVIEW PREP points a reviewer at the risk; it never says "LGTM".
- Push, merge, comment, rerun CI or publish releases on its own. Those need your explicit request and confirmation.
- Invent ticket numbers, owners, test results or impact numbers. Missing facts come back as
[confirm]. - Audit security. The auth checklist catches common diff-level mistakes, not design flaws.
Bewertungen
Noch keine Bewertungen.
Mehr Kits für Entwicklung
Engineering Outer Loop Kit
Run your team's outer loop: daily triage, CI verdicts, release notes and postmortems.
Engineering Outer Loop Bot
A Grok teammate that triages your repos at 9:00 and drafts release notes. It never merges.
Engineering Outer Loop Dot
Your ChatGPT dot keeps PRs, CI and Linear tidy and brings you only the decisions.