How to Handle Failing Tests Caused by a Known Bug

A question came up on a developer forum recently for a solution to a problem that occurs in almost every engineering team eventually:

"If a test has already found a bug, one option is to comment the test out until the issue is fixed. However, this has to be done manually, and it becomes time-consuming and hard to manage when there are many tests. How do you handle this in your workflow?"

The fact that commenting it out is the assumed default is why I wanted to write this article. Commenting out often feels like the obvious move: the test is noisy, you can't fix the bug right now, so you silence it and move on. However, those with experience know that decision has consequences that only become visible weeks or months later when you've forgotten the test ever existed.

There's a better pattern to temporarily skip or disable your tests, and every major test framework already supports it.

The Three Wrong Answers

As a test engineer, I want all my bugs fixed as soon as I find them, but in a practical sense that isn't always possible. In Kanban iterations and Scrum team sprints there may not be enough capacity in the maintenance or bug fix bucket to address bugs triaged below the line.

So when a test is failing due to a confirmed bug that won't be fixed this sprint, there are four options: leave it failing, comment it out, delete it, or skip (disable) it. The first three are wrong. Let's explore why.

Why Leaving a Failing Test in CI Breaks Your Build Signal

The test documents a failing test so leave the build failing until its resolved since it reflects reality

While one could argue it makes sense to keep the test failing until the bug its detecting is resolved, in practice, its a terrible idea to check in a known failing test.

• It breaks your build pipeline
• The defect may not be fixable for a long time due to priorities or complexity
• An always red, broken, build gets ignored and let's more bugs sneak in

A red build that everyone knows is "just that known bug" trains the team to ignore red builds. It's like your house alarm going off because someone smashed a window. If you leave the alarm going without fixing anything, you won't notice when someone kicks in the backdoor and robs you again. A failing test everyone ignores is a disabled alarm and a low severity defect in a complex area, for example, may sit unresolved for months given real sprint priorities. The build can't stay red that entire time.

With the skip pattern, that we'll discuss, it silences the noise deliberately and intentionally, with a paper trail, so the alarm means something again.

With that said, there are exceptions. For example, if existing tests fail due to a code change, breaking functionality the tests are covering, the build should stay red until the change is reverted or bug that was introduced is fixed. This is different than adding a known failing test to an otherwise green build.

Delete the Test

Another approach would be to delete the failing test, but I've almost never seen this done in practice.

• You lose coverage
• Someone has to write or put back the test again later, wasteful and error prone
• Easy to forget about

Again, the skip pattern is the better approach to disable the test.

Why Commenting Out a Failing Test Is Worse Than It Seems

Commenting out the test seems like a natural way of handling this. Teams do it all the time when temporarily disabling code for debugging. It seems natural to do it for the tests as well. You can just uncomment it later, but those who've worked in legacy code bases know how they are graveyards of forgotten code comments. Tests can have the same fate.

Commented-out test code is invisible to your tooling, silently rots, and is almost guaranteed to be forgotten. Outside of maybe, TODO: patterns, there is no reminder in your codebase to reenable them nor how many have accumulated.

I've seen this play out directly: a test was commented out when a bug was discovered, and it stayed that way until a major cleanup initiative was launched specifically to find dead code and commented-out blocks. When the team went to re-enable it, the codebase had drifted so far that the test was no longer compatible. It had to be rewritten from scratch, not simply re-enabled. The original time investment in writing it produced zero long-term value, and there was no way to know how long that coverage gap had existed or what had shipped during it.

Now, let's discuss the correct way of handling failing tests for bugs that can't be fixed quickly.

How to Skip a Failing Test the Right Way

Every major test framework has a built-in skip mechanism for this very scenario. Use it.

1. Create a bug ticket for the issue in your team's bug tracking system.
2. Note the defect number.
3. Use the test.skip syntax for your test framework to disable/skip the test programmatically
4. Include a TODO comment to unskip or reenable the test once the bug is resolved.
5. Note the location of the test in the bug ticket with instructions to enable and run the test to verify the defect is resolved and to check in the test update with the bug fix.

// Don't do this: invisible, rots silently, easy to forget
// test('user can reset password', async () => { ... })

// Do this: explicit, visible in reports, linked to the bug

// TODO: Test finding BUG#4521 - password reset endpoint returns 500, re-enable when fixed
test.skip('user can reset password', async () => { ... })

Some test frameworks also allow inline comments as a test.skip or test disable parameter alleviating the need for a seperate TODO comment line

Unlike commented-out code, a skipped test still surfaces in your run reports:

12 passed, 0 failed, 1 skipped

That count is a standing reminder that something needs to come back. It shows up on every run, in every CI report, without anyone having to go looking for it.

Why Skip Beats Commenting Out

• Commented-out tests are completely invisible. No skipped count, no reason string, no indication in test output that anything is missing. The gap is hidden from anyone reviewing CI results.
• Comments don't surface in TODO tracking. IDEs and code review tools can surface // TODO comments as actionable items. A commented-out test block is dead code. It won't appear in any report or task list prompting someone to revisit it.
• Commented-out code goes stale silently. As the codebase evolves, commented-out tests develop broken syntax, outdated method calls, and references to renamed or removed APIs. Nobody notices because the code never has to compile. When someone eventually tries to re-enable it, they're restoring broken code.
• Skipped tests still compile. A skipped test is live code. In typed languages, if a method is renamed or a parameter type changes, the skipped test will surface a compile error immediately. The breakage is caught, not hidden.
• Skip reasons are searchable. Searching the codebase for a ticket number instantly finds every test gated on that bug.

Linking Skipped Tests to Bug Tickets

The skip pattern only closes the loop if both sides reference each other:

1. The skip reason includes the bug ticket number or URL
2. The bug ticket description references the test file and test name
3. Re-enabling the test is an explicit step in the bug fix, not an afterthought

When the bug is fixed, the developer checks the ticket, finds the test reference, re-enables it, and verifies it passes before closing. This makes test restoration a first-class step in the fix workflow rather than something that gets remembered, or more often, forgotten.

A Fair Counterpoint

A commenter responding to the forum thread made a fair point: the skip pattern is technically the right answer, but it still requires discipline. Skipped tests are easy to ignore. It takes active effort to monitor the skipped count, prioritize the underlying bugs, and actually re-enable tests when fixes land. Otherwise, skipped tests accumulate and become their own form of technical debt.

That's true. But the same discipline argument applies even more strongly to commented-out tests. A skipped count is visible in every CI run: it's a number that can be tracked, trended, and reviewed in sprint planning. A commented-out test shows up nowhere. If discipline is the concern, the approach that provides the most visibility is the better starting point.

Using a CI Gate to Enforce a Skipped Test Threshold

If skipped count drift is a real concern for your team, you can turn discipline into policy with a CI gate that fails the build if the skipped count exceeds a defined threshold.

To my knowledge neither Jest nor JUnit have a built-in threshold option for this, but there is a practical, framework-agnostic, approach using a two-step GitHub Actions pattern: parse your JUnit XML test output to extract the skipped count, then fail the step if it exceeds your threshold.

- uses: mikepenz/action-junit-report@v4
  id: junit
  with:
    report_paths: '**/test-results/*.xml'

- name: Fail if skipped tests exceed threshold
  if: fromJson(steps.junit.outputs.skipped) > 5
  run: |
    echo "Skipped test count (${{ steps.junit.outputs.skipped }}) exceeds threshold of 5"
    exit 1

This works for any framework that outputs JUnit XML: Jest via jest-junit, Playwright via its built-in JUnit reporter, pytest via pytest-junit, and JUnit 5 natively. The threshold should reflect what's acceptable for your team. Even setting it generously and trending the number over sprints is more actionable than having no visibility at all.

Critically, this kind of gate is only possible with skips. You cannot gate on commented-out tests because your tooling has no visibility into them.

For teams using Jest, the eslint-plugin-jest/no-disabled-tests linting rule is a useful complement. It catches test.skip() at code review time, before it reaches CI.

Test Skip Syntax by Framework

Jest and Vitest

// Skip a single test
test.skip('user can reset password', () => {
  // Bug #4521: password reset endpoint returns 500
})

// Skip a suite
describe.skip('Password Reset', () => { ... })

// Older alias syntax, both work
xit('user can reset password', () => { ... })
xdescribe('Password Reset', () => { ... })

Vitest uses identical syntax to Jest. test.skip and describe.skip work the same way.

Docs: Jest skip · Vitest skip

Playwright

// Skip unconditionally
test.skip('user can reset password', async ({ page }) => {
  // Bug #4521: password reset endpoint returns 500
})

// Skip conditionally, useful for browser-specific bugs
test('user can reset password', async ({ page, browserName }) => {
  test.skip(browserName === 'webkit', 'Bug #4521: fails on Safari only')
  // ...
})

// test.fixme: skips the test but signals it urgently needs attention
// Shows up differently in the Playwright HTML report
test.fixme('user can reset password', async ({ page }) => {
  // Bug #4521: password reset endpoint returns 500
})

test.fixme behaves like test.skip but communicates more urgency. Use it when the test needs to come back soon rather than being parked indefinitely.

Docs: Playwright test annotations

Cypress

// Skip a single test
it.skip('user can reset password', () => {
  // Bug #4521: password reset endpoint returns 500
})

// Skip a suite
describe.skip('Password Reset', () => { ... })

Docs: Cypress test structure

pytest

import pytest

# Skip unconditionally with reason
@pytest.mark.skip(reason="Bug #4521: password reset endpoint returns 500")
def test_user_can_reset_password():
    ...

# Skip conditionally, useful for environment-specific bugs
@pytest.mark.skipif(os.getenv("ENV") == "staging", reason="Bug #4521: only affects staging")
def test_user_can_reset_password():
    ...

# xfail: marks as expected failure, test still runs
# Use when you want the test to run but not break the build
@pytest.mark.xfail(reason="Bug #4521: known failure, fix in progress")
def test_user_can_reset_password():
    ...

pytest.mark.xfail is a useful middle ground. The test still runs, but a failure is expected and won't break the build. Use it when you want visibility that the test is currently broken without silencing it entirely.

Docs: pytest skip reference

JUnit 5

import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;

// Skip a single test
@Disabled("Bug #4521: password reset endpoint returns 500, fix pending")
@Test
void userCanResetPassword() {
    // ...
}

// Skip an entire test class
@Disabled("Bug #4521: all password reset tests affected")
class PasswordResetTests {
    // ...
}

Docs: JUnit disabling tests

NUnit (.NET)

[Test]
[Ignore("Bug #4521: password reset endpoint returns 500, fix pending")]
public void UserCanResetPassword()
{
    // ...
}

Docs: NUnit Ignore attribute

xUnit (.NET)

[Fact(Skip = "Bug #4521: password reset endpoint returns 500, fix pending")]
public void UserCanResetPassword()
{
    // ...
}

Docs: xUnit Skip

RSpec (Ruby)

# Skip with reason
it 'allows user to reset password', :skip => 'Bug #4521: password reset returns 500' do
  # ...
end

# pending: similar to xfail, marks as pending, body is not executed
pending 'Bug #4521: password reset returns 500' do
  # ...
end

Docs: RSpec pending and skipped examples

Nightwatch

module.exports = {
  '@disabled': true, // This will prevent the test module from running.
  
  'sample test': function (browser) {
    // test code
  }
};

describe('homepage test with describe', function() {
  
  // skipped testcase: equivalent to: test.skip(), it.skip(), and xit()
  it.skip('async testcase', async browser => {
    const result = await browser.getText('#navigation');
    console.log('result', result.value)
  });
});

Docs: Nightwatch skipping and disabling tests

Common Mistakes When Disabling Tests for Known Bugs

• Don't comment out. Invisible in reports, won't surface in any tracking system, and goes stale silently as the codebase changes around it.
• Don't delete. The coverage is gone permanently. Someone has to rewrite the test from scratch when the bug is fixed, assuming anyone remembers it existed.
• Don't leave it failing. A red build everyone ignores is a disabled alarm. When a real regression slips through, nobody notices.
• Don't skip without a reason. A bare test.skip('user can reset password') with no context is almost as bad as a comment. There's no ticket reference, no way to know why it was skipped, and no path back to re-enabling it.

Conclusion

The skip pattern costs almost nothing to apply. It takes maybe thirty seconds longer than commenting out. What it buys you is a test that stays in the codebase, stays visible in reports, stays linked to the bug that caused it, and is right there waiting to be re-enabled when the fix lands.

The ticket reference is what closes the loop. Without it, skipped tests are marginally better than commented-out ones, still visible but still forgotten. With it, restoring the test becomes a natural last step in fixing the bug rather than something that has to be remembered.