regression-bisect

Automatically identify the commit that introduced a regression using git bisect.

Triggers

Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

"git bisect" → binary search for regression commit
"when did this break" → regression commit discovery

Purpose

This skill automates regression root cause analysis by:

Using git bisect to binary search commit history
Running automated tests at each bisect point
Identifying the exact commit that introduced a failure
Providing blame information and diff context
Suggesting potential fixes based on the breaking change

Behavior

When triggered, this skill:

Validates prerequisites:
- Confirms reproducible failure exists
- Verifies test automation is available
- Ensures clean working directory
- Identifies good and bad commits
Executes git bisect:
- Starts bisect with known good/bad commits
- Runs test suite at each bisect point
- Marks commits as good/bad based on test results
- Continues until breaking commit is found
Analyzes breaking commit:
- Shows commit message and metadata
- Displays full diff of the breaking change
- Identifies changed files and functions
- Maps to requirements/issues if available
Provides context:
- Shows blame information for affected code
- Lists related commits in the area
- Identifies potential reviewers (commit authors)
- Suggests rollback or fix strategies
Documents findings:
- Creates regression analysis report
- Links to issue tracker if available
- Updates regression baseline with findings
- Generates timeline of when regression was introduced

Bisect Configuration

Manual Bisect

bisect_manual:
  mode: interactive
  good_commit: v1.2.0  # Known good state
  bad_commit: HEAD     # Known bad state
  test_command: "npm test -- auth.test.ts"

  workflow:
    - git bisect start
    - git bisect bad HEAD
    - git bisect good v1.2.0
    - git bisect run npm test -- auth.test.ts

Automated Bisect

bisect_automated:
  mode: automated
  detect_good: auto  # Use last passing CI build
  detect_bad: auto   # Use current failing state
  test_command: "npm test"
  timeout: 600       # 10 minutes per commit test

  auto_detection:
    - check_ci_history
    - find_last_green_build
    - set_as_good_commit
    - use_current_as_bad

Bisect with Custom Test

bisect_custom:
  mode: custom
  test_script: scripts/regression-test.sh
  success_exit_code: 0
  failure_exit_code: 1

  test_script_template: |
    #!/bin/bash
    # Custom regression test
    npm run build || exit 1
    npm test -- specific-test.ts || exit 1
    ./validate-output.sh || exit 1
    exit 0

Bisect Report Format

# Regression Bisect Report

**Date**: 2026-01-28
**Issue**: Authentication fails on login
**Analyzer**: regression-bisect skill

## Executive Summary

**Breaking Commit**: abc123def456
**Author**: John Developer <john@example.com>
**Date**: 2026-01-15 14:32:00
**Bisect Duration**: 8 commits tested, 12 minutes

**Root Cause**: Token validation logic changed to require `iss` claim

## Bisect Results

### Commit Range

- **Good**: v1.2.0 (2026-01-10) - 50 commits ago
- **Bad**: HEAD (2026-01-28) - Current state
- **Breaking**: abc123def456 (2026-01-15) - 25 commits into range

### Bisect Log

| Commit | Status | Test Result | Duration |
|--------|--------|-------------|----------|
| abc123 | ❌ Bad | auth.test.ts failed | 45s |
| def789 | ❌ Bad | auth.test.ts failed | 42s |
| ghi012 | ✅ Good | all tests passed | 48s |
| jkl345 | ✅ Good | all tests passed | 46s |
| mno678 | ❌ Bad | auth.test.ts failed | 44s |
| **pqr901** | **❌ Breaking** | **First failure** | **43s** |
| stu234 | ✅ Good | all tests passed | 47s |
| vwx567 | ✅ Good | all tests passed | 45s |

## Breaking Commit Analysis

### Commit Details

commit pqr901xyz Author: John Developer john@example.com Date: Mon Jan 15 14:32:00 2026 +0000

feat(auth): add JWT issuer validation

- Require `iss` claim in JWT tokens
- Validate issuer against whitelist
- Update token generation to include issuer

Closes #456


### Files Changed

| File | Changes | Impact |
|------|---------|--------|
| src/auth/validate-token.ts | +15/-5 | High - Core validation logic |
| src/auth/generate-token.ts | +8/-2 | Medium - Token generation |
| test/auth/token.test.ts | +25/-0 | Low - Test updates |

### Diff Analysis

```diff
diff --git a/src/auth/validate-token.ts b/src/auth/validate-token.ts
index abc123..def456 100644
--- a/src/auth/validate-token.ts
+++ b/src/auth/validate-token.ts
@@ -15,6 +15,11 @@ export function validateToken(token: string): TokenPayload {
     throw new Error('Invalid token signature');
   }

+  // NEW: Require issuer claim
+  if (!payload.iss) {
+    throw new Error('Token missing issuer claim');
+  }
+
   return payload;
 }

Breaking Change: The new validation requires iss claim, but existing tokens don't include it.

Root Cause

The commit added mandatory iss (issuer) claim validation without:

Updating existing token generation (added in same commit but missed migration)
Providing backward compatibility for existing tokens
Coordinating with deployment to invalidate old sessions

Impact: All existing user sessions became invalid immediately.

Affected Components

Authentication: Token validation fails for all existing sessions
API Gateway: 401 errors on all authenticated endpoints
User Experience: Force logout of all active users

Related Information

Traceability

Issue: #456 - "Add JWT issuer validation for security"
Requirement: @.aiwg/requirements/use-cases/UC-AUTH-003-jwt-security.md
Review: PR #789 - Merged without integration testing

Similar Issues

Git history shows related changes:

commit xyz789 (2026-01-10): Initial JWT implementation
commit uvw456 (2026-01-12): Add JWT expiration validation
commit pqr901 (2026-01-15): Add JWT issuer validation ← BREAKING

Code Ownership

src/auth/validate-token.ts:
  85% John Developer <john@example.com>
  10% Jane Reviewer <jane@example.com>
   5% Bob Maintainer <bob@example.com>

Recommendations

Immediate (Hotfix)

Option A: Revert commit pqr901 to restore service
Option B: Make iss validation optional with feature flag
Option C: Deploy token refresh that includes iss claim

Recommended: Option B - Feature flag allows gradual rollout

Short-term (This Sprint)

Add integration tests for auth changes
Add backward compatibility tests for token validation
Update deployment runbook for auth changes
Add session migration logic for breaking changes

Ongoing

Require integration tests for auth PRs
Add "breaking change" checklist to PR template
Schedule auth regression suite in CI
Document token format in API docs

Fix Strategy

Proposed Fix

// src/auth/validate-token.ts
export function validateToken(token: string): TokenPayload {
  const payload = jwt.verify(token, SECRET_KEY);

  if (!payload.sub) {
    throw new Error('Invalid token signature');
  }

  // FIXED: Make issuer validation optional during migration
  if (featureFlags.requireIssuer && !payload.iss) {
    throw new Error('Token missing issuer claim');
  }

  return payload;
}

Migration Path

Deploy fix with requireIssuer: false
Update token generation to include iss
Wait for all old tokens to expire (24 hours)
Enable requireIssuer: true
Remove feature flag after validation

Timeline

2026-01-10: v1.2.0 released (working)
2026-01-15: Commit pqr901 merged (breaking change introduced)
2026-01-15: No CI failure (tests didn't catch integration issue)
2026-01-16: Deployed to production
2026-01-16: User reports started (authentication failures)
2026-01-28: Bisect identified root cause

Time to Detection: 13 days Time to Root Cause: Immediate (via bisect)

Prevention

This regression could have been prevented by:

Integration test covering token validation end-to-end
Staged rollout with feature flag
Backward compatibility review in PR checklist
Session invalidation notice to users

References

Bisect documentation: docs/debugging/git-bisect-guide.md
Breaking change policy: docs/development/breaking-changes.md
Auth architecture: @.aiwg/architecture/auth-architecture.md


## Usage Examples

### Basic Regression Bisect

User: "Find which commit broke authentication"

Skill executes:

Identify current failure: auth tests failing
Find last known good: v1.2.0 (CI history)
Run git bisect with test: npm test -- auth.test.ts
Test 8 commits in 12 minutes
Identify breaking commit: pqr901

Output: "Regression Bisect Complete

Breaking Commit: pqr901xyz Author: John Developer Date: 2026-01-15 14:32:00 Message: feat(auth): add JWT issuer validation

Root Cause: Required iss claim without backward compatibility

Files Changed:

src/auth/validate-token.ts (+15/-5)
src/auth/generate-token.ts (+8/-2)

Recommendation: Make validation optional with feature flag

Full report: .aiwg/testing/regression-bisect-auth.md"


### Bisect with Custom Test

User: "Bisect the payment processing regression"

Skill asks: "What is a good commit (when it worked)?" User: "v2.1.0"

Skill executes:

git bisect start
git bisect bad HEAD
git bisect good v2.1.0
git bisect run scripts/payment-test.sh

Output: "Breaking Commit: def456 Broke payment refund logic Changed: src/payments/refund.ts

Bisected 45 commits in 32 minutes See .aiwg/testing/regression-bisect-payment.md"


### Automatic Good/Bad Detection

User: "When did the API start returning 500 errors?"

Skill analyzes:

Current state: 500 errors on /api/users
CI history: Last green build was 2 days ago
Auto-detect good: commit xyz789 (last green)
Auto-detect bad: HEAD (current failing)

Runs bisect automatically.

Output: "Regression introduced in commit abc123 2 days ago by Jane Developer Changed database query logic

Breaking change: Added JOIN without NULL check See full analysis in .aiwg/testing/regression-api-500.md"


## Integration

This skill uses:
- `regression-baseline`: Load known good states
- `project-awareness`: Detect test commands and CI configuration
- `test-coverage`: Identify relevant test suites
- Git tools: For bisect operations and history analysis

## Agent Orchestration

```yaml
agents:
  analysis:
    agent: debugger
    focus: Root cause analysis and blame

  testing:
    agent: test-engineer
    focus: Test execution and validation

  documentation:
    agent: technical-writer
    focus: Report generation

Configuration

Bisect Presets

bisect_presets:
  auth_regression:
    test_command: npm test -- auth.test.ts
    timeout: 300
    auto_detect_good: true

  api_regression:
    test_command: npm run test:api
    timeout: 600
    auto_detect_good: true

  performance_regression:
    test_command: npm run benchmark
    threshold: "response_time < 1000ms"
    timeout: 900

Timeout Strategy

timeout_config:
  per_commit_test: 600     # 10 minutes
  total_bisect: 3600       # 1 hour max
  on_timeout: skip_commit  # or fail_bisect

Error Handling

Unstable Tests

If test results are inconsistent:

Run test 3 times per commit
Require 2/3 passes to mark as good
Flag flaky tests in report
Recommend test stabilization

Build Failures

If commit doesn't build:

Skip commit and mark as bad
Continue bisect with next commit
Flag build issues in report
Note potential false positive

Timeout Exceeded

If bisect takes too long:

Stop bisect at timeout
Report progress so far
Suggest narrowing commit range
Offer to resume later

Output Locations

Bisect reports: .aiwg/testing/regression-bisect-{issue}.md
Bisect logs: .aiwg/testing/bisect-logs/
Fix recommendations: .aiwg/working/regression-fixes/

References

@$AIWG_ROOT/agentic/code/frameworks/sdlc-complete/schemas/test/regression-schema.yaml
@$AIWG_ROOT/agentic/code/frameworks/sdlc-complete/commands/regression-test.md
@$AIWG_ROOT/agentic/code/frameworks/sdlc-complete/agents/debugger.md