[{"data":1,"prerenderedAt":6303},["ShallowReactive",2],{"content:\u002Fsoftware-testing\u002Fframeworks\u002Fplaywright\u002Frun-parallel-and-serial-tests-in-playwright":3,"category:\u002Fsoftware-testing\u002Fframeworks\u002Fplaywright\u002Frun-parallel-and-serial-tests-in-playwright":6,"read-next:\u002Fsoftware-testing\u002Ftest-automation\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky,\u002Fsoftware-testing\u002Ftest-automation\u002Frun-fewer-tests-catch-the-same-bugs,\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-playwright-ai-cost-efficient-testing,\u002Fsoftware-testing\u002Ftest-automation\u002Fbest-websites-for-practicing-test-automation":3904},{"id":4,"title":5,"bmcUsername":6,"body":7,"cover":3894,"date":3895,"description":3896,"draft":3897,"extension":3898,"features":6,"githubRepo":6,"headline":6,"highlight":6,"icon":6,"meta":3899,"navigation":175,"npmPackage":6,"order":6,"path":3900,"seo":3901,"stem":3902,"__hash__":3903},"content\u002Fsoftware-testing\u002Fframeworks\u002Fplaywright\u002Frun-parallel-and-serial-tests-in-playwright.md","Run Parallel and Serial Tests Together in Playwright",null,{"type":8,"value":9,"toc":3873},"minimark",[10,14,17,25,28,31,36,72,75,84,88,99,305,316,677,680,684,687,716,731,737,746,750,757,771,1032,1046,1063,1087,1090,1095,1103,1236,1337,1348,1361,1365,1378,1471,1478,1650,1654,1667,1673,1680,1683,1865,1872,1875,1879,1888,2036,2039,2165,2175,2178,2233,2236,2244,2248,2251,2254,2257,2272,2795,2809,2850,2864,2870,3070,3076,3097,3101,3104,3107,3110,3145,3148,3152,3274,3280,3284,3287,3299,3303,3310,3448,3452,3464,3573,3579,3589,3817,3820,3826,3830,3836,3858,3865,3869],[11,12,13],"p",{},"Modern test frameworks like Playwright allow you to run your tests in parallel instead of serially, allowing for faster feedback loops in your automated test runs. Serial test execution is many times slower than parallel runs because you are running tests one at a time. Every test you add slows your test pipeline down by the duration of that test. Parallel execution of your test automation lets you concurrently execute tests, where, with enough workers, your test pipeline would only be as slow as the longest-running test in the test suite.",[11,15,16],{},"This article covers a common problem that keeps test suites from running fully in parallel: tests that change global state to set up preconditions, which breaks other tests running at the same time that expect different preconditions.",[11,18,19,20,24],{},"This is what I am currently dealing with. Our test framework before Playwright had one big limitation: everything ran serially, or across several isolated test environments. Some tests changed global settings in ",[21,22,23],"code",{},"web.config",". Others alter global state by moving the server date to set up a precondition, like a billing rollover at month end. Run any of those next to another test and you break the other test. The entire legacy test suite was built around serial execution concepts that led to inefficient feedback loops that slowed down the Software Development Life Cycle (SDLC).",[11,26,27],{},"While modernizing and converting the legacy tests to Playwright, I was able to rework most scenarios to run in parallel without collisions or flakiness, but a few scenarios still required changing global state settings on the server, and they couldn't stay in the suite without making all the tests run serially.",[11,29,30],{},"Then I learned Playwright projects can separate parallel-safe tests from serial-only ones and run the serial ones on their own.",[32,33,35],"h2",{"id":34},"three-ways-to-run-serial-and-parallel-tests-together","Three Ways to Run Serial and Parallel Tests Together",[37,38,39,52,62],"ul",{},[40,41,42,51],"li",{},[43,44,45,50],"strong",{},[46,47,49],"a",{"href":48},"#option-a-playwright-project-dependencies-the-quick-win","Option A, project dependencies"," (the quick win):"," Split the tests into two projects and let Playwright run the serial one after the parallel one passes. It's a few lines of config and no CI changes, but there's a catch: one failing parallel test means the serial tests never run.",[40,53,54,61],{},[43,55,56,60],{},[46,57,59],{"href":58},"#option-b-separate-ci-steps-for-playwright-projects-fixing-the-catch","Option B, separate CI steps"," (fixes the catch):"," The same split, with each project run as its own CI step. It takes more wiring, but you get results from both suites on every run.",[40,63,64,71],{},[43,65,66,70],{},[46,67,69],{"href":68},"#option-c-preconfigured-test-environments-my-long-term-goal","Option C, preconfigured environments"," (my long-term goal):"," Stop changing settings from inside the tests. Stand up environments already configured for each precondition and point each test group at its own. It's the fastest and cleanest, but it isn't always feasible because of hosting cost and how easily you can create environments.",[11,73,74],{},"Options A and B split the tests the same way, by filename or by tag.",[11,76,77,78,83],{},"I built a small demo of every approach in this article. It's in the ",[79,80],"external-link",{"href":81,"text":82},"https:\u002F\u002Fgithub.com\u002Freallymello\u002Fplaywright-serial-parallel-dependencies-demo","playwright-serial-parallel-dependencies-demo"," repo, and the results below come from running it.",[32,85,87],{"id":86},"demoing-the-problem-playwright-tests-that-change-shared-state","Demoing the Problem: Playwright Tests That Change Shared State",[11,89,90,91,94,95,98],{},"The demo uses one shared JSON file as a stand-in for global server state, holding a feature flag and a \"server date\". The ",[21,92,93],{},"parallel-safe"," tests in the demo project only read it and expect the defaults. The ",[21,96,97],{},"serial-only"," tests change its state to set up a precondition, then reset it afterward:",[100,101,107],"pre",{"className":102,"code":103,"filename":104,"language":105,"meta":106,"style":106},"language-ts shiki shiki-themes material-theme-lighter github-light-high-contrast github-dark-high-contrast","test.afterEach(async () => {\n  await resetSettings();\n});\n\ntest('server date set to month end triggers billing rollover', async () => {\n  await writeSettings({ featureFlag: 'off', serverDate: '2026-01-31' });\n  expect((await readSettings()).serverDate).toBe('2026-01-31');\n});\n","tests\u002Fserver-settings.serial.spec.ts","ts","",[21,108,109,142,159,170,177,206,254,296],{"__ignoreMap":106},[110,111,114,118,122,126,129,133,136,139],"span",{"class":112,"line":113},"line",1,[110,115,117],{"class":116},"sZ-rw","test",[110,119,121],{"class":120},"sPJuK",".",[110,123,125],{"class":124},"sb1SK","afterEach",[110,127,128],{"class":116},"(",[110,130,132],{"class":131},"stWsX","async",[110,134,135],{"class":120}," ()",[110,137,138],{"class":131}," =>",[110,140,141],{"class":120}," {\n",[110,143,145,149,152,156],{"class":112,"line":144},2,[110,146,148],{"class":147},"sZTni","  await",[110,150,151],{"class":124}," resetSettings",[110,153,155],{"class":154},"sq0XF","()",[110,157,158],{"class":120},";\n",[110,160,162,165,168],{"class":112,"line":161},3,[110,163,164],{"class":120},"}",[110,166,167],{"class":116},")",[110,169,158],{"class":120},[110,171,173],{"class":112,"line":172},4,[110,174,176],{"emptyLinePlaceholder":175},true,"\n",[110,178,180,182,184,188,192,194,197,200,202,204],{"class":112,"line":179},5,[110,181,117],{"class":124},[110,183,128],{"class":116},[110,185,187],{"class":186},"sZi47","'",[110,189,191],{"class":190},"srGNg","server date set to month end triggers billing rollover",[110,193,187],{"class":186},[110,195,196],{"class":120},",",[110,198,199],{"class":131}," async",[110,201,135],{"class":120},[110,203,138],{"class":131},[110,205,141],{"class":120},[110,207,209,211,214,216,219,222,225,228,231,233,235,238,240,242,245,247,250,252],{"class":112,"line":208},6,[110,210,148],{"class":147},[110,212,213],{"class":124}," writeSettings",[110,215,128],{"class":154},[110,217,218],{"class":120},"{",[110,220,221],{"class":154}," featureFlag",[110,223,224],{"class":120},":",[110,226,227],{"class":186}," '",[110,229,230],{"class":190},"off",[110,232,187],{"class":186},[110,234,196],{"class":120},[110,236,237],{"class":154}," serverDate",[110,239,224],{"class":120},[110,241,227],{"class":186},[110,243,244],{"class":190},"2026-01-31",[110,246,187],{"class":186},[110,248,249],{"class":120}," }",[110,251,167],{"class":154},[110,253,158],{"class":120},[110,255,257,260,263,266,269,272,274,277,279,281,284,286,288,290,292,294],{"class":112,"line":256},7,[110,258,259],{"class":124},"  expect",[110,261,262],{"class":154},"((",[110,264,265],{"class":147},"await",[110,267,268],{"class":124}," readSettings",[110,270,271],{"class":154},"())",[110,273,121],{"class":120},[110,275,276],{"class":116},"serverDate",[110,278,167],{"class":154},[110,280,121],{"class":120},[110,282,283],{"class":124},"toBe",[110,285,128],{"class":154},[110,287,187],{"class":186},[110,289,244],{"class":190},[110,291,187],{"class":186},[110,293,167],{"class":154},[110,295,158],{"class":120},[110,297,299,301,303],{"class":112,"line":298},8,[110,300,164],{"class":120},[110,302,167],{"class":116},[110,304,158],{"class":120},[11,306,307,308,311,312,315],{},"To see the shared state collision problem you can run ",[21,309,310],{},"npm run test:collision"," which evaluates to ",[21,313,314],{},"playwright test -c playwright.collision.config.ts",". The output below is trimmed to the first failure's details.",[100,317,322],{"className":318,"code":319,"filename":320,"language":321,"meta":106,"style":106},"language-ansi shiki shiki-themes material-theme-lighter github-light-high-contrast github-dark-high-contrast","npm run test:collision\n\n> playwright-serial-parallel-dependencies-demo@1.0.0 test:collision\n> playwright test -c playwright.collision.config.ts\n\n\n\u001b[2mRunning \u001b[22m11\u001b[2m tests using \u001b[22m5\u001b[2m workers\u001b[22m\n\n  \u001b[32m✓\u001b[39m  \u001b[2m 1 \u001b[22mtests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 4 uses default settings\u001b[2m (319ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 2 \u001b[22mtests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 1 uses default settings\u001b[2m (319ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 3 \u001b[22mtests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 3 uses default settings\u001b[2m (318ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 4 \u001b[22mtests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 2 uses default settings\u001b[2m (312ms)\u001b[22m\n  \u001b[32m-\u001b[39m  \u001b[2m 9 \u001b[22m\u001b[36mtests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1)\u001b[39m\n  \u001b[32m✓\u001b[39m  \u001b[2m 5 \u001b[22mtests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 1 renders with default settings\u001b[2m (311ms)\u001b[22m\n  \u001b[31m✘\u001b[39m  \u001b[2m 8 \u001b[22m\u001b[31mtests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings\u001b[39m\u001b[2m (305ms)\u001b[22m\n  \u001b[31m✘\u001b[39m  \u001b[2m 7 \u001b[22m\u001b[31mtests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings\u001b[39m\u001b[2m (305ms)\u001b[22m\n  \u001b[31m✘\u001b[39m  \u001b[2m 6 \u001b[22m\u001b[31mtests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings\u001b[39m\u001b[2m (307ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m10 \u001b[22mtests\\server-settings.serial.spec.ts:13:5 › feature flag on enables new checkout\u001b[2m (515ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m11 \u001b[22mtests\\server-settings.serial.spec.ts:19:5 › server date set to month end triggers billing rollover\u001b[2m (512ms)\u001b[22m\n\n\n\u001b[31m  1) tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings \u001b[2m──\u001b[22m\u001b[39m\n\n    Error: \u001b[2mexpect(\u001b[22m\u001b[31mreceived\u001b[39m\u001b[2m).\u001b[22mtoBe\u001b[2m(\u001b[22m\u001b[32mexpected\u001b[39m\u001b[2m) \u002F\u002F Object.is equality\u001b[22m\n\n    Expected: \u001b[32m\"o\u001b[7mff\u001b[27m\"\u001b[39m\n    Received: \u001b[31m\"o\u001b[7mn\u001b[27m\"\u001b[39m\n\n    ...\n\n\u001b[31m  3 failed\u001b[39m\n\u001b[31m    tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings \u001b[2m───\u001b[22m\u001b[39m\n\u001b[31m    tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings \u001b[2m───\u001b[22m\u001b[39m\n\u001b[31m    tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings \u001b[2m───\u001b[22m\u001b[39m\n\u001b[33m  1 skipped\u001b[39m\n\u001b[32m  7 passed\u001b[39m\u001b[2m (1.7s)\u001b[22m\n","Terminal","ansi",[21,323,324,330,334,339,344,348,352,370,374,390,403,417,431,444,458,474,487,501,515,529,534,539,549,554,579,584,600,614,619,625,630,636,645,653,661,668],{"__ignoreMap":106},[110,325,326],{"class":112,"line":113},[110,327,329],{"class":328},"sOsy2","npm run test:collision\n",[110,331,332],{"class":112,"line":144},[110,333,176],{"emptyLinePlaceholder":175},[110,335,336],{"class":112,"line":161},[110,337,338],{"class":328},"> playwright-serial-parallel-dependencies-demo@1.0.0 test:collision\n",[110,340,341],{"class":112,"line":172},[110,342,343],{"class":328},"> playwright test -c playwright.collision.config.ts\n",[110,345,346],{"class":112,"line":179},[110,347,176],{"emptyLinePlaceholder":175},[110,349,350],{"class":112,"line":208},[110,351,176],{"emptyLinePlaceholder":175},[110,353,354,358,361,364,367],{"class":112,"line":256},[110,355,357],{"class":356},"sFDEC","Running ",[110,359,360],{"class":328},"11",[110,362,363],{"class":356}," tests using ",[110,365,366],{"class":328},"5",[110,368,369],{"class":356}," workers\n",[110,371,372],{"class":112,"line":298},[110,373,176],{"emptyLinePlaceholder":175},[110,375,377,381,384,387],{"class":112,"line":376},9,[110,378,380],{"class":379},"ssIYW","  ✓",[110,382,383],{"class":356},"   1 ",[110,385,386],{"class":328},"tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 4 uses default settings",[110,388,389],{"class":356}," (319ms)\n",[110,391,393,395,398,401],{"class":112,"line":392},10,[110,394,380],{"class":379},[110,396,397],{"class":356},"   2 ",[110,399,400],{"class":328},"tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 1 uses default settings",[110,402,389],{"class":356},[110,404,406,408,411,414],{"class":112,"line":405},11,[110,407,380],{"class":379},[110,409,410],{"class":356},"   3 ",[110,412,413],{"class":328},"tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 3 uses default settings",[110,415,416],{"class":356}," (318ms)\n",[110,418,420,422,425,428],{"class":112,"line":419},12,[110,421,380],{"class":379},[110,423,424],{"class":356},"   4 ",[110,426,427],{"class":328},"tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 2 uses default settings",[110,429,430],{"class":356}," (312ms)\n",[110,432,434,437,440],{"class":112,"line":433},13,[110,435,436],{"class":379},"  -",[110,438,439],{"class":356},"   9 ",[110,441,443],{"class":442},"sCdEk","tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1)\n",[110,445,447,449,452,455],{"class":112,"line":446},14,[110,448,380],{"class":379},[110,450,451],{"class":356},"   5 ",[110,453,454],{"class":328},"tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 1 renders with default settings",[110,456,457],{"class":356}," (311ms)\n",[110,459,461,465,468,471],{"class":112,"line":460},15,[110,462,464],{"class":463},"sDbPz","  ✘",[110,466,467],{"class":356},"   8 ",[110,469,470],{"class":463},"tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings",[110,472,473],{"class":356}," (305ms)\n",[110,475,477,479,482,485],{"class":112,"line":476},16,[110,478,464],{"class":463},[110,480,481],{"class":356},"   7 ",[110,483,484],{"class":463},"tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings",[110,486,473],{"class":356},[110,488,490,492,495,498],{"class":112,"line":489},17,[110,491,464],{"class":463},[110,493,494],{"class":356},"   6 ",[110,496,497],{"class":463},"tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings",[110,499,500],{"class":356}," (307ms)\n",[110,502,504,506,509,512],{"class":112,"line":503},18,[110,505,380],{"class":379},[110,507,508],{"class":356},"  10 ",[110,510,511],{"class":328},"tests\\server-settings.serial.spec.ts:13:5 › feature flag on enables new checkout",[110,513,514],{"class":356}," (515ms)\n",[110,516,518,520,523,526],{"class":112,"line":517},19,[110,519,380],{"class":379},[110,521,522],{"class":356},"  11 ",[110,524,525],{"class":328},"tests\\server-settings.serial.spec.ts:19:5 › server date set to month end triggers billing rollover",[110,527,528],{"class":356}," (512ms)\n",[110,530,532],{"class":112,"line":531},20,[110,533,176],{"emptyLinePlaceholder":175},[110,535,537],{"class":112,"line":536},21,[110,538,176],{"emptyLinePlaceholder":175},[110,540,542,545],{"class":112,"line":541},22,[110,543,544],{"class":463},"  1) tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings ",[110,546,548],{"class":547},"smSMi","──\n",[110,550,552],{"class":112,"line":551},23,[110,553,176],{"emptyLinePlaceholder":175},[110,555,557,560,563,566,569,571,573,576],{"class":112,"line":556},24,[110,558,559],{"class":328},"    Error: ",[110,561,562],{"class":356},"expect(",[110,564,565],{"class":463},"received",[110,567,568],{"class":356},").",[110,570,283],{"class":328},[110,572,128],{"class":356},[110,574,575],{"class":379},"expected",[110,577,578],{"class":356},") \u002F\u002F Object.is equality\n",[110,580,582],{"class":112,"line":581},25,[110,583,176],{"emptyLinePlaceholder":175},[110,585,587,590,593,597],{"class":112,"line":586},26,[110,588,589],{"class":328},"    Expected: ",[110,591,592],{"class":379},"\"o",[110,594,596],{"class":595},"sBsy4","ff",[110,598,599],{"class":379},"\"\n",[110,601,603,606,608,612],{"class":112,"line":602},27,[110,604,605],{"class":328},"    Received: ",[110,607,592],{"class":463},[110,609,611],{"class":610},"sYBWh","n",[110,613,599],{"class":463},[110,615,617],{"class":112,"line":616},28,[110,618,176],{"emptyLinePlaceholder":175},[110,620,622],{"class":112,"line":621},29,[110,623,624],{"class":328},"    ...\n",[110,626,628],{"class":112,"line":627},30,[110,629,176],{"emptyLinePlaceholder":175},[110,631,633],{"class":112,"line":632},31,[110,634,635],{"class":463},"  3 failed\n",[110,637,639,642],{"class":112,"line":638},32,[110,640,641],{"class":463},"    tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings ",[110,643,644],{"class":547},"───\n",[110,646,648,651],{"class":112,"line":647},33,[110,649,650],{"class":463},"    tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings ",[110,652,644],{"class":547},[110,654,656,659],{"class":112,"line":655},34,[110,657,658],{"class":463},"    tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings ",[110,660,644],{"class":547},[110,662,664],{"class":112,"line":663},35,[110,665,667],{"class":666},"spoJ1","  1 skipped\n",[110,669,671,674],{"class":112,"line":670},36,[110,672,673],{"class":379},"  7 passed",[110,675,676],{"class":356}," (1.7s)\n",[11,678,679],{},"As shown above, running them together in a parallel pool introduces failures when the tests that expect the default state evaluate when the serial tests change the state before the teardown runs. The demo holds the changed state for half a second to make the collision likely, but a real backend process reacting to a changed date, feature flag, or web.config setting fails the same way, just less predictably.",[32,681,683],{"id":682},"serial-means-three-different-things-in-playwright","\"Serial\" Means Three Different Things in Playwright",[11,685,686],{},"Playwright has a few different ways of configuring serial runs depending on your use case and intention.",[37,688,689,697,710],{},[40,690,691,696],{},[43,692,693],{},[21,694,695],{},"workers: 1"," limits how many tests run at once. It says nothing about order, and one failure doesn't affect the other tests.",[40,698,699,704,705,709],{},[43,700,701],{},[21,702,703],{},"test.describe.configure({ mode: 'serial' })"," groups tests that depend on each other. ",[79,706],{"href":707,"text":708},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Ftest-parallel","The Playwright docs"," say: \"If one of the serial tests fails, all subsequent tests are skipped.\" That's right for a chain of dependent steps, and wrong for independent tests that each change some state.",[40,711,712,715],{},[43,713,714],{},"A serial project"," is a separate set of tests, selected by a matcher, that runs apart from the parallel-safe set. This is the one that solves the mixed-suite problem.",[11,717,718,719,721,722,726,727,730],{},"Even in parallel test runs many of my longer system testing user-journey type scenarios will be broken into individual tests in the same file that need to run serially, sequentially. For example, the first test may create an applicant and then the test after uses that applicant to apply for an account and so on. These test spec files use ",[21,720,703],{}," so that the tests in the spec run sequentially and skip tests that appear after the failing test ",[723,724,725],"em",{},"in that spec file",". So for my example, if the applicant creation test fails there is no point running the next test where that applicant is used to apply for an account. Playwright will still parallelize the spec files in that project while running tests ",[723,728,729],{},"within"," the spec files serially with that setting applied.",[11,732,733,734,736],{},"Using ",[21,735,695],{}," on the project will prevent Playwright from running the spec files in parallel as well.",[11,738,739,740,742,743,745],{},"Options A and B that we will cover in the following article sections put the state-changing tests in a ",[21,741,97],{}," project, and ",[21,744,695],{}," inside that project makes them run one at a time, serially.",[32,747,749],{"id":748},"split-tests-with-playwright-projects-and-matchers","Split Tests with Playwright Projects and Matchers",[11,751,752,756],{},[79,753],{"href":754,"text":755},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Ftest-projects","Playwright projects"," let you run different subsets of your tests with different settings. We will use this feature to determine which tests to run and when in the different solutions we will be showing to solve the parallel + serial test problem.",[11,758,759,760,763,764,767,768,224],{},"First, name the state-changing files ",[21,761,762],{},"*.serial.spec.ts"," and match on that using the ",[21,765,766],{},"projects"," setting in ",[21,769,770],{},"playwright.config.ts",[100,772,774],{"className":102,"code":773,"filename":770,"language":105,"meta":106,"style":106},"import { defineConfig } from '@playwright\u002Ftest';\n\nexport default defineConfig({\n  testDir: '.\u002Ftests',\n  projects: [\n    {\n      name: 'parallel-safe',\n      testMatch: \u002F.*\\.spec\\.ts\u002F,\n      testIgnore: \u002F.*\\.serial\\.spec\\.ts\u002F,\n      fullyParallel: true,\n    },\n    {\n      name: 'serial-only',\n      testMatch: \u002F.*\\.serial\\.spec\\.ts\u002F,\n      workers: 1, \u002F\u002F per-project workers requires Playwright 1.52+\n    },\n  ],\n});\n",[21,775,776,801,805,822,839,849,854,869,902,932,945,950,954,968,996,1013,1017,1024],{"__ignoreMap":106},[110,777,778,781,784,787,789,792,794,797,799],{"class":112,"line":113},[110,779,780],{"class":147},"import",[110,782,783],{"class":120}," {",[110,785,786],{"class":116}," defineConfig",[110,788,249],{"class":120},[110,790,791],{"class":147}," from",[110,793,227],{"class":186},[110,795,796],{"class":190},"@playwright\u002Ftest",[110,798,187],{"class":186},[110,800,158],{"class":120},[110,802,803],{"class":112,"line":144},[110,804,176],{"emptyLinePlaceholder":175},[110,806,807,810,813,815,818],{"class":112,"line":161},[110,808,809],{"class":147},"export",[110,811,812],{"class":147}," default",[110,814,786],{"class":124},[110,816,128],{"class":817},"sLHsM",[110,819,821],{"class":820},"sCRTB","{\n",[110,823,824,827,829,831,834,836],{"class":112,"line":172},[110,825,826],{"class":154},"  testDir",[110,828,224],{"class":120},[110,830,227],{"class":186},[110,832,833],{"class":190},".\u002Ftests",[110,835,187],{"class":186},[110,837,838],{"class":820},",\n",[110,840,841,844,846],{"class":112,"line":179},[110,842,843],{"class":154},"  projects",[110,845,224],{"class":120},[110,847,848],{"class":116}," [\n",[110,850,851],{"class":112,"line":208},[110,852,853],{"class":120},"    {\n",[110,855,856,859,861,863,865,867],{"class":112,"line":256},[110,857,858],{"class":154},"      name",[110,860,224],{"class":120},[110,862,227],{"class":186},[110,864,93],{"class":190},[110,866,187],{"class":186},[110,868,838],{"class":120},[110,870,871,874,876,879,882,886,890,893,895,897,900],{"class":112,"line":298},[110,872,873],{"class":154},"      testMatch",[110,875,224],{"class":120},[110,877,878],{"class":186}," \u002F",[110,880,121],{"class":881},"spMcu",[110,883,885],{"class":884},"sE6rD","*",[110,887,889],{"class":888},"szoTG","\\.",[110,891,892],{"class":190},"spec",[110,894,889],{"class":888},[110,896,105],{"class":190},[110,898,899],{"class":186},"\u002F",[110,901,838],{"class":120},[110,903,904,907,909,911,913,915,917,920,922,924,926,928,930],{"class":112,"line":376},[110,905,906],{"class":154},"      testIgnore",[110,908,224],{"class":120},[110,910,878],{"class":186},[110,912,121],{"class":881},[110,914,885],{"class":884},[110,916,889],{"class":888},[110,918,919],{"class":190},"serial",[110,921,889],{"class":888},[110,923,892],{"class":190},[110,925,889],{"class":888},[110,927,105],{"class":190},[110,929,899],{"class":186},[110,931,838],{"class":120},[110,933,934,937,939,943],{"class":112,"line":392},[110,935,936],{"class":154},"      fullyParallel",[110,938,224],{"class":120},[110,940,942],{"class":941},"sTqCK"," true",[110,944,838],{"class":120},[110,946,947],{"class":112,"line":405},[110,948,949],{"class":120},"    },\n",[110,951,952],{"class":112,"line":419},[110,953,853],{"class":120},[110,955,956,958,960,962,964,966],{"class":112,"line":433},[110,957,858],{"class":154},[110,959,224],{"class":120},[110,961,227],{"class":186},[110,963,97],{"class":190},[110,965,187],{"class":186},[110,967,838],{"class":120},[110,969,970,972,974,976,978,980,982,984,986,988,990,992,994],{"class":112,"line":446},[110,971,873],{"class":154},[110,973,224],{"class":120},[110,975,878],{"class":186},[110,977,121],{"class":881},[110,979,885],{"class":884},[110,981,889],{"class":888},[110,983,919],{"class":190},[110,985,889],{"class":888},[110,987,892],{"class":190},[110,989,889],{"class":888},[110,991,105],{"class":190},[110,993,899],{"class":186},[110,995,838],{"class":120},[110,997,998,1001,1003,1007,1009],{"class":112,"line":460},[110,999,1000],{"class":154},"      workers",[110,1002,224],{"class":120},[110,1004,1006],{"class":1005},"s6g51"," 1",[110,1008,196],{"class":120},[110,1010,1012],{"class":1011},"s_gjE"," \u002F\u002F per-project workers requires Playwright 1.52+\n",[110,1014,1015],{"class":112,"line":476},[110,1016,949],{"class":120},[110,1018,1019,1022],{"class":112,"line":489},[110,1020,1021],{"class":116},"  ]",[110,1023,838],{"class":820},[110,1025,1026,1028,1030],{"class":112,"line":503},[110,1027,164],{"class":820},[110,1029,167],{"class":817},[110,1031,158],{"class":120},[11,1033,1034,1035,1038,1039,1041,1042,1045],{},"Note the use of ",[21,1036,1037],{},"testIgnore",". ",[21,1040,762],{}," also matches ",[21,1043,1044],{},".*\\.spec\\.ts",", so without it the serial files would land in both projects and run in parallel too.",[11,1047,1048,1049,1051,1052,1055,1056,1059,1060,121],{},"The ",[21,1050,93],{}," project will contain tests that ",[723,1053,1054],{},"play nicely with others",": they can run in parallel without colliding with other tests. There we set ",[21,1057,1058],{},"fullyParallel"," to ",[21,1061,1062],{},"true",[11,1064,1048,1065,1067,1068,1059,1071,1074,1075,1078,1079,1082,1083,1086],{},[21,1066,97],{}," project will hold the tests that change things that would create failures or flakiness in other tests if run at the same time. There we set the ",[21,1069,1070],{},"workers",[21,1072,1073],{},"1"," to ensure only one test runs at a time if they are in that project. These tests that ",[723,1076,1077],{},"don't"," play nicely with others will have the file suffix ",[21,1080,1081],{},".serial.spec.ts"," so we can use the ",[21,1084,1085],{},"testMatch"," parameter to find them.",[11,1088,1089],{},"The different solutions we will be going over use the project classifications in different ways to solve the parallel vs. serial testing problem.",[1091,1092,1094],"h3",{"id":1093},"match-on-tags-instead-of-filenames","Match on Tags Instead of Filenames",[11,1096,1097,1098,1102],{},"A filename convention forces you to keep state-changing tests in their own files. If you'd rather keep related tests together, tag the serial ones (see ",[79,1099],{"href":1100,"text":1101},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Ftest-annotations","Playwright test tags",") and match on the tag:",[100,1104,1107],{"className":102,"code":1105,"filename":1106,"language":105,"meta":106,"style":106},"test.describe('checkout with mutated server state', { tag: '@serial' }, () => {\n  test.describe.configure({ mode: 'serial' });\n\n  test('feature flag on enables new checkout', async () => {\n    \u002F\u002F ...\n  });\n});\n","tests-tagged\u002Fcheckout.spec.ts",[21,1108,1109,1152,1187,1191,1214,1219,1228],{"__ignoreMap":106},[110,1110,1111,1113,1115,1118,1120,1122,1125,1127,1129,1131,1134,1136,1138,1141,1143,1146,1148,1150],{"class":112,"line":113},[110,1112,117],{"class":116},[110,1114,121],{"class":120},[110,1116,1117],{"class":124},"describe",[110,1119,128],{"class":116},[110,1121,187],{"class":186},[110,1123,1124],{"class":190},"checkout with mutated server state",[110,1126,187],{"class":186},[110,1128,196],{"class":120},[110,1130,783],{"class":120},[110,1132,1133],{"class":154}," tag",[110,1135,224],{"class":120},[110,1137,227],{"class":186},[110,1139,1140],{"class":190},"@serial",[110,1142,187],{"class":186},[110,1144,1145],{"class":120}," },",[110,1147,135],{"class":120},[110,1149,138],{"class":131},[110,1151,141],{"class":120},[110,1153,1154,1157,1159,1161,1163,1166,1168,1170,1173,1175,1177,1179,1181,1183,1185],{"class":112,"line":144},[110,1155,1156],{"class":116},"  test",[110,1158,121],{"class":120},[110,1160,1117],{"class":116},[110,1162,121],{"class":120},[110,1164,1165],{"class":124},"configure",[110,1167,128],{"class":154},[110,1169,218],{"class":120},[110,1171,1172],{"class":154}," mode",[110,1174,224],{"class":120},[110,1176,227],{"class":186},[110,1178,919],{"class":190},[110,1180,187],{"class":186},[110,1182,249],{"class":120},[110,1184,167],{"class":154},[110,1186,158],{"class":120},[110,1188,1189],{"class":112,"line":161},[110,1190,176],{"emptyLinePlaceholder":175},[110,1192,1193,1195,1197,1199,1202,1204,1206,1208,1210,1212],{"class":112,"line":172},[110,1194,1156],{"class":124},[110,1196,128],{"class":154},[110,1198,187],{"class":186},[110,1200,1201],{"class":190},"feature flag on enables new checkout",[110,1203,187],{"class":186},[110,1205,196],{"class":120},[110,1207,199],{"class":131},[110,1209,135],{"class":120},[110,1211,138],{"class":131},[110,1213,141],{"class":120},[110,1215,1216],{"class":112,"line":179},[110,1217,1218],{"class":1011},"    \u002F\u002F ...\n",[110,1220,1221,1224,1226],{"class":112,"line":208},[110,1222,1223],{"class":120},"  }",[110,1225,167],{"class":154},[110,1227,158],{"class":120},[110,1229,1230,1232,1234],{"class":112,"line":256},[110,1231,164],{"class":120},[110,1233,167],{"class":116},[110,1235,158],{"class":120},[100,1237,1240],{"className":102,"code":1238,"filename":1239,"language":105,"meta":106,"style":106},"projects: [\n  { name: 'parallel-safe', grepInvert: \u002F@serial\u002F, fullyParallel: true },\n  { name: 'serial-only', grep: \u002F@serial\u002F, workers: 1 },\n],\n","playwright.tagged.config.ts",[21,1241,1242,1251,1292,1330],{"__ignoreMap":106},[110,1243,1244,1247,1249],{"class":112,"line":113},[110,1245,766],{"class":1246},"sA8fK",[110,1248,224],{"class":120},[110,1250,848],{"class":116},[110,1252,1253,1256,1259,1261,1263,1265,1267,1269,1272,1274,1276,1278,1280,1282,1285,1287,1289],{"class":112,"line":144},[110,1254,1255],{"class":120},"  {",[110,1257,1258],{"class":154}," name",[110,1260,224],{"class":120},[110,1262,227],{"class":186},[110,1264,93],{"class":190},[110,1266,187],{"class":186},[110,1268,196],{"class":120},[110,1270,1271],{"class":154}," grepInvert",[110,1273,224],{"class":120},[110,1275,878],{"class":186},[110,1277,1140],{"class":190},[110,1279,899],{"class":186},[110,1281,196],{"class":120},[110,1283,1284],{"class":154}," fullyParallel",[110,1286,224],{"class":120},[110,1288,942],{"class":941},[110,1290,1291],{"class":120}," },\n",[110,1293,1294,1296,1298,1300,1302,1304,1306,1308,1311,1313,1315,1317,1319,1321,1324,1326,1328],{"class":112,"line":161},[110,1295,1255],{"class":120},[110,1297,1258],{"class":154},[110,1299,224],{"class":120},[110,1301,227],{"class":186},[110,1303,97],{"class":190},[110,1305,187],{"class":186},[110,1307,196],{"class":120},[110,1309,1310],{"class":154}," grep",[110,1312,224],{"class":120},[110,1314,878],{"class":186},[110,1316,1140],{"class":190},[110,1318,899],{"class":186},[110,1320,196],{"class":120},[110,1322,1323],{"class":154}," workers",[110,1325,224],{"class":120},[110,1327,1006],{"class":1005},[110,1329,1291],{"class":120},[110,1331,1332,1335],{"class":112,"line":172},[110,1333,1334],{"class":116},"]",[110,1336,838],{"class":120},[11,1338,1339,1340,1343,1344,1347],{},"You can skip projects entirely and filter on the command line with ",[21,1341,1342],{},"--grep @serial"," and ",[21,1345,1346],{},"--grep-invert @serial",". The demo has that version too.",[11,1349,1350,1351,1354,1355,1357,1358,1360],{},"One trap: the demo keeps its tagged tests in a separate ",[21,1352,1353],{},"tests-tagged\u002F"," folder. A tagged file inside the folder your filename-based projects already scan would be picked up by the ",[21,1356,93],{}," project, and the ",[21,1359,1140],{}," tests would run in parallel anyway.",[32,1362,1364],{"id":1363},"option-a-playwright-project-dependencies-the-quick-win","Option A: Playwright Project Dependencies (The Quick Win)",[11,1366,1367,1368,1371,1372,1374,1375,1377],{},"Once the tests are split, you can use the Playwright ",[21,1369,1370],{},"dependencies"," project configuration setting to mark that the ",[21,1373,97],{}," project depends on the ",[21,1376,93],{}," project finishing first.",[100,1379,1382],{"className":102,"code":1380,"filename":1381,"language":105,"meta":106,"style":106},"{\n  name: 'serial-only',\n  testMatch: \u002F.*\\.serial\\.spec\\.ts\u002F,\n  dependencies: ['parallel-safe'], \u002F\u002F the magic happens here\n  workers: 1,\n},\n","playwright.dependencies.config.ts",[21,1383,1384,1388,1403,1432,1455,1466],{"__ignoreMap":106},[110,1385,1386],{"class":112,"line":113},[110,1387,821],{"class":120},[110,1389,1390,1393,1395,1397,1399,1401],{"class":112,"line":144},[110,1391,1392],{"class":1246},"  name",[110,1394,224],{"class":120},[110,1396,227],{"class":186},[110,1398,97],{"class":190},[110,1400,187],{"class":186},[110,1402,838],{"class":120},[110,1404,1405,1408,1410,1412,1414,1416,1418,1420,1422,1424,1426,1428,1430],{"class":112,"line":161},[110,1406,1407],{"class":1246},"  testMatch",[110,1409,224],{"class":120},[110,1411,878],{"class":186},[110,1413,121],{"class":881},[110,1415,885],{"class":884},[110,1417,889],{"class":888},[110,1419,919],{"class":190},[110,1421,889],{"class":888},[110,1423,892],{"class":190},[110,1425,889],{"class":888},[110,1427,105],{"class":190},[110,1429,899],{"class":186},[110,1431,838],{"class":120},[110,1433,1434,1437,1439,1442,1444,1446,1448,1450,1452],{"class":112,"line":172},[110,1435,1436],{"class":1246},"  dependencies",[110,1438,224],{"class":120},[110,1440,1441],{"class":154}," [",[110,1443,187],{"class":186},[110,1445,93],{"class":190},[110,1447,187],{"class":186},[110,1449,1334],{"class":154},[110,1451,196],{"class":120},[110,1453,1454],{"class":1011}," \u002F\u002F the magic happens here\n",[110,1456,1457,1460,1462,1464],{"class":112,"line":179},[110,1458,1459],{"class":1246},"  workers",[110,1461,224],{"class":120},[110,1463,1006],{"class":1005},[110,1465,838],{"class":120},[110,1467,1468],{"class":112,"line":208},[110,1469,1470],{"class":120},"},\n",[11,1472,1473,1474,1477],{},"This is a quick and easy way to solve the problem. The parallel tests run first, and the serial ones run after they finish, so nothing collides. In the demo project, running ",[21,1475,1476],{},"npm run test:deps"," will run with this configuration-based solution.",[100,1479,1481],{"className":318,"code":1480,"filename":320,"language":321,"meta":106,"style":106},"npx playwright test -c playwright.dependencies.config.ts\n\n\u001b[2mRunning \u001b[22m11\u001b[2m tests using \u001b[22m9\u001b[2m workers\u001b[22m\n\n  \u001b[32m-\u001b[39m  \u001b[2m 9 \u001b[22m\u001b[36m[parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1)\u001b[39m\n  \u001b[32m✓\u001b[39m  \u001b[2m 1 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 2 uses default settings\u001b[2m (309ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 2 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 4 uses default settings\u001b[2m (320ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 4 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 1 uses default settings\u001b[2m (307ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 3 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 3 uses default settings\u001b[2m (316ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 5 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 1 renders with default settings\u001b[2m (319ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 6 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings\u001b[2m (313ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 7 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings\u001b[2m (311ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 8 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings\u001b[2m (319ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m10 \u001b[22m[serial-only] › tests\\server-settings.serial.spec.ts:13:5 › feature flag on enables new checkout\u001b[2m (517ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m11 \u001b[22m[serial-only] › tests\\server-settings.serial.spec.ts:19:5 › server date set to month end triggers billing rollover\u001b[2m (508ms)\u001b[22m\n\n\u001b[33m  1 skipped\u001b[39m\n\u001b[32m  10 passed\u001b[39m\u001b[2m (1.9s)\u001b[22m\n",[21,1482,1483,1488,1492,1505,1509,1518,1530,1542,1553,1565,1576,1588,1599,1610,1622,1634,1638,1642],{"__ignoreMap":106},[110,1484,1485],{"class":112,"line":113},[110,1486,1487],{"class":328},"npx playwright test -c playwright.dependencies.config.ts\n",[110,1489,1490],{"class":112,"line":144},[110,1491,176],{"emptyLinePlaceholder":175},[110,1493,1494,1496,1498,1500,1503],{"class":112,"line":161},[110,1495,357],{"class":356},[110,1497,360],{"class":328},[110,1499,363],{"class":356},[110,1501,1502],{"class":328},"9",[110,1504,369],{"class":356},[110,1506,1507],{"class":112,"line":172},[110,1508,176],{"emptyLinePlaceholder":175},[110,1510,1511,1513,1515],{"class":112,"line":179},[110,1512,436],{"class":379},[110,1514,439],{"class":356},[110,1516,1517],{"class":442},"[parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1)\n",[110,1519,1520,1522,1524,1527],{"class":112,"line":208},[110,1521,380],{"class":379},[110,1523,383],{"class":356},[110,1525,1526],{"class":328},"[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 2 uses default settings",[110,1528,1529],{"class":356}," (309ms)\n",[110,1531,1532,1534,1536,1539],{"class":112,"line":256},[110,1533,380],{"class":379},[110,1535,397],{"class":356},[110,1537,1538],{"class":328},"[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 4 uses default settings",[110,1540,1541],{"class":356}," (320ms)\n",[110,1543,1544,1546,1548,1551],{"class":112,"line":298},[110,1545,380],{"class":379},[110,1547,424],{"class":356},[110,1549,1550],{"class":328},"[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 1 uses default settings",[110,1552,500],{"class":356},[110,1554,1555,1557,1559,1562],{"class":112,"line":376},[110,1556,380],{"class":379},[110,1558,410],{"class":356},[110,1560,1561],{"class":328},"[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 3 uses default settings",[110,1563,1564],{"class":356}," (316ms)\n",[110,1566,1567,1569,1571,1574],{"class":112,"line":392},[110,1568,380],{"class":379},[110,1570,451],{"class":356},[110,1572,1573],{"class":328},"[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 1 renders with default settings",[110,1575,389],{"class":356},[110,1577,1578,1580,1582,1585],{"class":112,"line":405},[110,1579,380],{"class":379},[110,1581,494],{"class":356},[110,1583,1584],{"class":328},"[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings",[110,1586,1587],{"class":356}," (313ms)\n",[110,1589,1590,1592,1594,1597],{"class":112,"line":419},[110,1591,380],{"class":379},[110,1593,481],{"class":356},[110,1595,1596],{"class":328},"[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings",[110,1598,457],{"class":356},[110,1600,1601,1603,1605,1608],{"class":112,"line":433},[110,1602,380],{"class":379},[110,1604,467],{"class":356},[110,1606,1607],{"class":328},"[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings",[110,1609,389],{"class":356},[110,1611,1612,1614,1616,1619],{"class":112,"line":446},[110,1613,380],{"class":379},[110,1615,508],{"class":356},[110,1617,1618],{"class":328},"[serial-only] › tests\\server-settings.serial.spec.ts:13:5 › feature flag on enables new checkout",[110,1620,1621],{"class":356}," (517ms)\n",[110,1623,1624,1626,1628,1631],{"class":112,"line":460},[110,1625,380],{"class":379},[110,1627,522],{"class":356},[110,1629,1630],{"class":328},"[serial-only] › tests\\server-settings.serial.spec.ts:19:5 › server date set to month end triggers billing rollover",[110,1632,1633],{"class":356}," (508ms)\n",[110,1635,1636],{"class":112,"line":476},[110,1637,176],{"emptyLinePlaceholder":175},[110,1639,1640],{"class":112,"line":489},[110,1641,667],{"class":666},[110,1643,1644,1647],{"class":112,"line":503},[110,1645,1646],{"class":379},"  10 passed",[110,1648,1649],{"class":356}," (1.9s)\n",[1091,1651,1653],{"id":1652},"the-gotcha-a-failing-test-skips-the-serial-suite","The Gotcha: A Failing Test Skips the Serial Suite",[11,1655,1656,1657,1660,1661,1663,1664,224],{},"There is a ",[723,1658,1659],{},"gotcha"," or trade-off to leveraging the ",[21,1662,1370],{}," setting to separate and run the serial tests after the parallel tests finish. Per the ",[79,1665],{"href":754,"text":1666},"Playwright documentation",[1668,1669,1670],"blockquote",{},[11,1671,1672],{},"\"If the tests from a dependency fails then the tests that rely on this project will not be run.\"",[11,1674,1675,1676,1679],{},"So if there is a test failure in the parallel tests the serial tests ",[723,1677,1678],{},"will not run"," so you'd lose visibility and coverage for that run for your serial tests since they are skipped.",[11,1681,1682],{},"The demo project uses this DEMO_FAIL test that has been getting skipped in the earlier runs to show this downside in action.",[100,1684,1686],{"className":318,"code":1685,"filename":320,"language":321,"meta":106,"style":106},"DEMO_FAIL=1 npx playwright test -c playwright.dependencies.config.ts\n\n\u001b[2mRunning \u001b[22m11\u001b[2m tests using \u001b[22m9\u001b[2m workers\u001b[22m\n\n  \u001b[31m✘\u001b[39m  \u001b[2m 9 \u001b[22m\u001b[31m[parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1)\u001b[39m\u001b[2m (4ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 3 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 4 renders with default settings\u001b[2m (305ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 1 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 2 uses default settings\u001b[2m (312ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 5 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 3 uses default settings\u001b[2m (317ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 4 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 4 uses default settings\u001b[2m (316ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 2 \u001b[22m[parallel-safe] › tests\\pricing.spec.ts:8:9 › pricing (parallel-safe) › quote 1 uses default settings\u001b[2m (322ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 7 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 2 renders with default settings\u001b[2m (310ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 8 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 3 renders with default settings\u001b[2m (310ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 6 \u001b[22m[parallel-safe] › tests\\reports.spec.ts:6:9 › reports (parallel-safe) › report 1 renders with default settings\u001b[2m (316ms)\u001b[22m\n\n\n\u001b[31m  1) [parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1) \u001b[39m\n\n    Error: forced failure to demonstrate dependency blocking\n\n    ...\n\n\u001b[31m  1 failed\u001b[39m\n\u001b[31m    [parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1) \u001b[39m\n\u001b[33m  2 did not run\u001b[39m\n\u001b[32m  8 passed\u001b[39m\u001b[2m (656ms)\u001b[22m\n",[21,1687,1688,1693,1697,1709,1713,1725,1735,1745,1756,1766,1777,1788,1798,1808,1812,1816,1821,1825,1830,1834,1838,1842,1847,1852,1857],{"__ignoreMap":106},[110,1689,1690],{"class":112,"line":113},[110,1691,1692],{"class":328},"DEMO_FAIL=1 npx playwright test -c playwright.dependencies.config.ts\n",[110,1694,1695],{"class":112,"line":144},[110,1696,176],{"emptyLinePlaceholder":175},[110,1698,1699,1701,1703,1705,1707],{"class":112,"line":161},[110,1700,357],{"class":356},[110,1702,360],{"class":328},[110,1704,363],{"class":356},[110,1706,1502],{"class":328},[110,1708,369],{"class":356},[110,1710,1711],{"class":112,"line":172},[110,1712,176],{"emptyLinePlaceholder":175},[110,1714,1715,1717,1719,1722],{"class":112,"line":179},[110,1716,464],{"class":463},[110,1718,439],{"class":356},[110,1720,1721],{"class":463},"[parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1)",[110,1723,1724],{"class":356}," (4ms)\n",[110,1726,1727,1729,1731,1733],{"class":112,"line":208},[110,1728,380],{"class":379},[110,1730,410],{"class":356},[110,1732,1596],{"class":328},[110,1734,473],{"class":356},[110,1736,1737,1739,1741,1743],{"class":112,"line":256},[110,1738,380],{"class":379},[110,1740,383],{"class":356},[110,1742,1526],{"class":328},[110,1744,430],{"class":356},[110,1746,1747,1749,1751,1753],{"class":112,"line":298},[110,1748,380],{"class":379},[110,1750,451],{"class":356},[110,1752,1561],{"class":328},[110,1754,1755],{"class":356}," (317ms)\n",[110,1757,1758,1760,1762,1764],{"class":112,"line":376},[110,1759,380],{"class":379},[110,1761,424],{"class":356},[110,1763,1538],{"class":328},[110,1765,1564],{"class":356},[110,1767,1768,1770,1772,1774],{"class":112,"line":392},[110,1769,380],{"class":379},[110,1771,397],{"class":356},[110,1773,1550],{"class":328},[110,1775,1776],{"class":356}," (322ms)\n",[110,1778,1779,1781,1783,1785],{"class":112,"line":405},[110,1780,380],{"class":379},[110,1782,481],{"class":356},[110,1784,1584],{"class":328},[110,1786,1787],{"class":356}," (310ms)\n",[110,1789,1790,1792,1794,1796],{"class":112,"line":419},[110,1791,380],{"class":379},[110,1793,467],{"class":356},[110,1795,1607],{"class":328},[110,1797,1787],{"class":356},[110,1799,1800,1802,1804,1806],{"class":112,"line":433},[110,1801,380],{"class":379},[110,1803,494],{"class":356},[110,1805,1573],{"class":328},[110,1807,1564],{"class":356},[110,1809,1810],{"class":112,"line":446},[110,1811,176],{"emptyLinePlaceholder":175},[110,1813,1814],{"class":112,"line":460},[110,1815,176],{"emptyLinePlaceholder":175},[110,1817,1818],{"class":112,"line":476},[110,1819,1820],{"class":463},"  1) [parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1) \n",[110,1822,1823],{"class":112,"line":489},[110,1824,176],{"emptyLinePlaceholder":175},[110,1826,1827],{"class":112,"line":503},[110,1828,1829],{"class":328},"    Error: forced failure to demonstrate dependency blocking\n",[110,1831,1832],{"class":112,"line":517},[110,1833,176],{"emptyLinePlaceholder":175},[110,1835,1836],{"class":112,"line":531},[110,1837,624],{"class":328},[110,1839,1840],{"class":112,"line":536},[110,1841,176],{"emptyLinePlaceholder":175},[110,1843,1844],{"class":112,"line":541},[110,1845,1846],{"class":463},"  1 failed\n",[110,1848,1849],{"class":112,"line":551},[110,1850,1851],{"class":463},"    [parallel-safe] › tests\\reports.spec.ts:15:7 › reports (parallel-safe) › deliberately failing test (only when DEMO_FAIL=1) \n",[110,1853,1854],{"class":112,"line":556},[110,1855,1856],{"class":666},"  2 did not run\n",[110,1858,1859,1862],{"class":112,"line":581},[110,1860,1861],{"class":379},"  8 passed",[110,1863,1864],{"class":356}," (656ms)\n",[11,1866,1867,1868,1871],{},"The two serial tests never ran as indicated by ",[21,1869,1870],{},"2 did not run"," in the terminal output. In a real suite, it means one failing test in a large parallel-safe project hides the results of the entire serial project. It also changes what \"the suite ran\" means when someone triages a red build. Every failure has to be fixed before you can see whether the second half is healthy at all. In other words, you don't know what other issues are hiding until you fix the early test failures which can make troubleshooting frustrating if you fix one test, rerun, and find another issue that was hiding behind it.",[11,1873,1874],{},"For my suite that's a dealbreaker. The serial tests cover some of the more important financial transactions in the systems that are part of the critical path. I want to know about a failure there on the same run where a parallel test fails, not one fix cycle later.",[32,1876,1878],{"id":1877},"option-b-separate-ci-steps-for-playwright-projects-fixing-the-catch","Option B: Separate CI Steps for Playwright Projects (Fixing the Catch)",[11,1880,1881,1882,1884,1885,1887],{},"As mentioned, Option A, leveraging Playwright ",[21,1883,1370],{},", is an easy solution to keep the serial and parallel tests separated, but has the trade-off where an upstream failure in the parallel test project causes your serial tests to be skipped. The fix in Option B is to keep leveraging Playwright projects to subset the parallel and serial tests, but drop ",[21,1886,1370],{}," and let CI do the ordering. The projects stay in the config, but each gets its own invocation:",[100,1889,1891],{"className":102,"code":1890,"filename":770,"language":105,"meta":106,"style":106},"projects: [\n  { name: 'parallel-safe', testMatch: \u002F.*\\.spec\\.ts\u002F, testIgnore: \u002F.*\\.serial\\.spec\\.ts\u002F, fullyParallel: true },\n  { name: 'serial-only', testMatch: \u002F.*\\.serial\\.spec\\.ts\u002F, workers: 1 },\n  \u002F\u002F no `dependencies` between them\n],\n",[21,1892,1893,1901,1975,2025,2030],{"__ignoreMap":106},[110,1894,1895,1897,1899],{"class":112,"line":113},[110,1896,766],{"class":1246},[110,1898,224],{"class":120},[110,1900,848],{"class":116},[110,1902,1903,1905,1907,1909,1911,1913,1915,1917,1920,1922,1924,1926,1928,1930,1932,1934,1936,1938,1940,1943,1945,1947,1949,1951,1953,1955,1957,1959,1961,1963,1965,1967,1969,1971,1973],{"class":112,"line":144},[110,1904,1255],{"class":120},[110,1906,1258],{"class":154},[110,1908,224],{"class":120},[110,1910,227],{"class":186},[110,1912,93],{"class":190},[110,1914,187],{"class":186},[110,1916,196],{"class":120},[110,1918,1919],{"class":154}," testMatch",[110,1921,224],{"class":120},[110,1923,878],{"class":186},[110,1925,121],{"class":881},[110,1927,885],{"class":884},[110,1929,889],{"class":888},[110,1931,892],{"class":190},[110,1933,889],{"class":888},[110,1935,105],{"class":190},[110,1937,899],{"class":186},[110,1939,196],{"class":120},[110,1941,1942],{"class":154}," testIgnore",[110,1944,224],{"class":120},[110,1946,878],{"class":186},[110,1948,121],{"class":881},[110,1950,885],{"class":884},[110,1952,889],{"class":888},[110,1954,919],{"class":190},[110,1956,889],{"class":888},[110,1958,892],{"class":190},[110,1960,889],{"class":888},[110,1962,105],{"class":190},[110,1964,899],{"class":186},[110,1966,196],{"class":120},[110,1968,1284],{"class":154},[110,1970,224],{"class":120},[110,1972,942],{"class":941},[110,1974,1291],{"class":120},[110,1976,1977,1979,1981,1983,1985,1987,1989,1991,1993,1995,1997,1999,2001,2003,2005,2007,2009,2011,2013,2015,2017,2019,2021,2023],{"class":112,"line":161},[110,1978,1255],{"class":120},[110,1980,1258],{"class":154},[110,1982,224],{"class":120},[110,1984,227],{"class":186},[110,1986,97],{"class":190},[110,1988,187],{"class":186},[110,1990,196],{"class":120},[110,1992,1919],{"class":154},[110,1994,224],{"class":120},[110,1996,878],{"class":186},[110,1998,121],{"class":881},[110,2000,885],{"class":884},[110,2002,889],{"class":888},[110,2004,919],{"class":190},[110,2006,889],{"class":888},[110,2008,892],{"class":190},[110,2010,889],{"class":888},[110,2012,105],{"class":190},[110,2014,899],{"class":186},[110,2016,196],{"class":120},[110,2018,1323],{"class":154},[110,2020,224],{"class":120},[110,2022,1006],{"class":1005},[110,2024,1291],{"class":120},[110,2026,2027],{"class":112,"line":172},[110,2028,2029],{"class":1011},"  \u002F\u002F no `dependencies` between them\n",[110,2031,2032,2034],{"class":112,"line":179},[110,2033,1334],{"class":116},[110,2035,838],{"class":120},[11,2037,2038],{},"In GitHub Actions, run the parallel-safe step first, let the serial step run whether or not the first one failed, then fail the job if either did:",[100,2040,2045],{"className":2041,"code":2042,"filename":2043,"language":2044,"meta":106,"style":106},"language-yaml shiki shiki-themes material-theme-lighter github-light-high-contrast github-dark-high-contrast","- name: Parallel-safe tests\n  id: parallel\n  run: npx playwright test --project=parallel-safe\n  continue-on-error: true\n\n- name: Serial-only tests\n  id: serial\n  run: npx playwright test --project=serial-only --workers=1\n  continue-on-error: true\n\n- name: Fail job if either suite failed\n  if: steps.parallel.outcome == 'failure' || steps.serial.outcome == 'failure'\n  run: exit 1\n",".github\u002Fworkflows\u002Fsplit-steps.yml","yaml",[21,2046,2047,2060,2070,2080,2090,2094,2105,2114,2123,2131,2135,2146,2156],{"__ignoreMap":106},[110,2048,2049,2052,2055,2057],{"class":112,"line":113},[110,2050,2051],{"class":120},"-",[110,2053,1258],{"class":2054},"saWzx",[110,2056,224],{"class":120},[110,2058,2059],{"class":190}," Parallel-safe tests\n",[110,2061,2062,2065,2067],{"class":112,"line":144},[110,2063,2064],{"class":2054},"  id",[110,2066,224],{"class":120},[110,2068,2069],{"class":190}," parallel\n",[110,2071,2072,2075,2077],{"class":112,"line":161},[110,2073,2074],{"class":2054},"  run",[110,2076,224],{"class":120},[110,2078,2079],{"class":190}," npx playwright test --project=parallel-safe\n",[110,2081,2082,2085,2087],{"class":112,"line":172},[110,2083,2084],{"class":2054},"  continue-on-error",[110,2086,224],{"class":120},[110,2088,2089],{"class":941}," true\n",[110,2091,2092],{"class":112,"line":179},[110,2093,176],{"emptyLinePlaceholder":175},[110,2095,2096,2098,2100,2102],{"class":112,"line":208},[110,2097,2051],{"class":120},[110,2099,1258],{"class":2054},[110,2101,224],{"class":120},[110,2103,2104],{"class":190}," Serial-only tests\n",[110,2106,2107,2109,2111],{"class":112,"line":256},[110,2108,2064],{"class":2054},[110,2110,224],{"class":120},[110,2112,2113],{"class":190}," serial\n",[110,2115,2116,2118,2120],{"class":112,"line":298},[110,2117,2074],{"class":2054},[110,2119,224],{"class":120},[110,2121,2122],{"class":190}," npx playwright test --project=serial-only --workers=1\n",[110,2124,2125,2127,2129],{"class":112,"line":376},[110,2126,2084],{"class":2054},[110,2128,224],{"class":120},[110,2130,2089],{"class":941},[110,2132,2133],{"class":112,"line":392},[110,2134,176],{"emptyLinePlaceholder":175},[110,2136,2137,2139,2141,2143],{"class":112,"line":405},[110,2138,2051],{"class":120},[110,2140,1258],{"class":2054},[110,2142,224],{"class":120},[110,2144,2145],{"class":190}," Fail job if either suite failed\n",[110,2147,2148,2151,2153],{"class":112,"line":419},[110,2149,2150],{"class":2054},"  if",[110,2152,224],{"class":120},[110,2154,2155],{"class":190}," steps.parallel.outcome == 'failure' || steps.serial.outcome == 'failure'\n",[110,2157,2158,2160,2162],{"class":112,"line":433},[110,2159,2074],{"class":2054},[110,2161,224],{"class":120},[110,2163,2164],{"class":190}," exit 1\n",[11,2166,2167,2170,2171,2174],{},[21,2168,2169],{},"continue-on-error: true"," keeps a failed step from stopping the job, and the ",[21,2172,2173],{},"outcome"," check in the last step reads the result of each step before that setting is applied. The last step is what turns the build red.",[11,2176,2177],{},"Run the same forced failure locally as two separate commands and you get the result the dependency version couldn't:",[100,2179,2183],{"className":2180,"code":2181,"filename":320,"language":2182,"meta":106,"style":106},"language-zsh shiki shiki-themes material-theme-lighter github-light-high-contrast github-dark-high-contrast","DEMO_FAIL=1 npx playwright test --project=parallel-safe\n# 1 failed, 8 passed\n\nnpx playwright test --project=serial-only\n# 2 passed\n","zsh",[21,2184,2185,2207,2212,2216,2228],{"__ignoreMap":106},[110,2186,2187,2190,2193,2195,2198,2201,2204],{"class":112,"line":113},[110,2188,2189],{"class":116},"DEMO_FAIL",[110,2191,2192],{"class":884},"=",[110,2194,1073],{"class":190},[110,2196,2197],{"class":1246}," npx",[110,2199,2200],{"class":190}," playwright",[110,2202,2203],{"class":190}," test",[110,2205,2206],{"class":881}," --project=parallel-safe\n",[110,2208,2209],{"class":112,"line":144},[110,2210,2211],{"class":1011},"# 1 failed, 8 passed\n",[110,2213,2214],{"class":112,"line":161},[110,2215,176],{"emptyLinePlaceholder":175},[110,2217,2218,2221,2223,2225],{"class":112,"line":172},[110,2219,2220],{"class":1246},"npx",[110,2222,2200],{"class":190},[110,2224,2203],{"class":190},[110,2226,2227],{"class":881}," --project=serial-only\n",[110,2229,2230],{"class":112,"line":179},[110,2231,2232],{"class":1011},"# 2 passed\n",[11,2234,2235],{},"The parallel step fails and the serial step still runs and passes. The order is still preserved, because the serial tests only start after the parallel step finishes. The two suites just no longer depend on each other's results with no coverage compromise this time.",[11,2237,2238,2239,1343,2241,121],{},"The tag version works with the same two steps. Change the commands to ",[21,2240,1346],{},[21,2242,2243],{},"--grep @serial --workers=1",[32,2245,2247],{"id":2246},"option-c-preconfigured-test-environments-my-long-term-goal","Option C: Preconfigured Test Environments (My Long-Term Goal)",[11,2249,2250],{},"Both options so far keep the underlying problem. The tests still change global state on the fly, so they still need the environment to themselves. That covers things like the server date, web server settings, language settings, and feature flags.",[11,2252,2253],{},"What I'd rather do is stand up test environments that are already configured with those settings. It's less invasive and tends to be less of a flaky pattern than changing major server settings mid-test. It also lessens the amount of tests in the serial project grouping if groups of tests with similar state settings can run together on isolated environments.",[11,2255,2256],{},"One environment would have its server date set to a month end. One has its language set to Spanish. One has the new-checkout feature flag on. Tests that need a given precondition run against the environment that already has it, so nothing has to change a setting mid-run. That lets each group of tests run isolated from the rest and in parallel with them, with no ordering step in CI at all.",[11,2258,2259,2260,2263,2264,2267,2268,2271],{},"Playwright supports this with per-project ",[21,2261,2262],{},"use"," options. ",[79,2265],{"href":2266,"text":708},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Ftest-use-options"," say you can override options for a specific project, and ",[21,2269,2270],{},"baseURL"," is one of them. Name the test files after the environment they need, then match on that naming convention:",[100,2273,2275],{"className":102,"code":2274,"filename":770,"language":105,"meta":106,"style":106},"import { defineConfig } from '@playwright\u002Ftest';\nimport dotenv from 'dotenv';\nimport path from 'path';\n\ndotenv.config({ path: path.resolve(__dirname, '.env') });\n\nexport default defineConfig({\n  testDir: '.\u002Ftests',\n  fullyParallel: true,\n  projects: [\n    {\n      name: 'parallel-safe',\n      testMatch: \u002F.*\\.spec\\.ts\u002F,\n      testIgnore: \u002F.*\\.(month-end|spanish|new-checkout)\\.spec\\.ts\u002F,\n      use: { baseURL: process.env.DEFAULT_BASE_URL },\n    },\n    {\n      name: 'month-end-date',\n      testMatch: \u002F.*\\.month-end\\.spec\\.ts\u002F,\n      use: { baseURL: process.env.MONTH_END_BASE_URL },\n    },\n    {\n      name: 'spanish-locale',\n      testMatch: \u002F.*\\.spanish\\.spec\\.ts\u002F,\n      use: { baseURL: process.env.SPANISH_BASE_URL },\n    },\n    {\n      name: 'new-checkout-flag',\n      testMatch: \u002F.*\\.new-checkout\\.spec\\.ts\u002F,\n      use: { baseURL: process.env.NEW_CHECKOUT_BASE_URL },\n    },\n  ],\n});\n",[21,2276,2277,2297,2316,2334,2338,2384,2388,2400,2414,2425,2433,2437,2451,2475,2519,2549,2553,2557,2572,2600,2625,2629,2633,2648,2676,2701,2705,2709,2724,2752,2777,2781,2787],{"__ignoreMap":106},[110,2278,2279,2281,2283,2285,2287,2289,2291,2293,2295],{"class":112,"line":113},[110,2280,780],{"class":147},[110,2282,783],{"class":120},[110,2284,786],{"class":116},[110,2286,249],{"class":120},[110,2288,791],{"class":147},[110,2290,227],{"class":186},[110,2292,796],{"class":190},[110,2294,187],{"class":186},[110,2296,158],{"class":120},[110,2298,2299,2301,2304,2307,2309,2312,2314],{"class":112,"line":144},[110,2300,780],{"class":147},[110,2302,2303],{"class":116}," dotenv ",[110,2305,2306],{"class":147},"from",[110,2308,227],{"class":186},[110,2310,2311],{"class":190},"dotenv",[110,2313,187],{"class":186},[110,2315,158],{"class":120},[110,2317,2318,2320,2323,2325,2327,2330,2332],{"class":112,"line":161},[110,2319,780],{"class":147},[110,2321,2322],{"class":116}," path ",[110,2324,2306],{"class":147},[110,2326,227],{"class":186},[110,2328,2329],{"class":190},"path",[110,2331,187],{"class":186},[110,2333,158],{"class":120},[110,2335,2336],{"class":112,"line":172},[110,2337,176],{"emptyLinePlaceholder":175},[110,2339,2340,2342,2344,2347,2349,2351,2354,2356,2358,2360,2363,2366,2368,2370,2373,2375,2378,2380,2382],{"class":112,"line":179},[110,2341,2311],{"class":116},[110,2343,121],{"class":120},[110,2345,2346],{"class":124},"config",[110,2348,128],{"class":116},[110,2350,218],{"class":120},[110,2352,2353],{"class":154}," path",[110,2355,224],{"class":120},[110,2357,2353],{"class":116},[110,2359,121],{"class":120},[110,2361,2362],{"class":124},"resolve",[110,2364,2365],{"class":116},"(__dirname",[110,2367,196],{"class":120},[110,2369,227],{"class":186},[110,2371,2372],{"class":190},".env",[110,2374,187],{"class":186},[110,2376,2377],{"class":116},") ",[110,2379,164],{"class":120},[110,2381,167],{"class":116},[110,2383,158],{"class":120},[110,2385,2386],{"class":112,"line":208},[110,2387,176],{"emptyLinePlaceholder":175},[110,2389,2390,2392,2394,2396,2398],{"class":112,"line":256},[110,2391,809],{"class":147},[110,2393,812],{"class":147},[110,2395,786],{"class":124},[110,2397,128],{"class":817},[110,2399,821],{"class":820},[110,2401,2402,2404,2406,2408,2410,2412],{"class":112,"line":298},[110,2403,826],{"class":154},[110,2405,224],{"class":120},[110,2407,227],{"class":186},[110,2409,833],{"class":190},[110,2411,187],{"class":186},[110,2413,838],{"class":820},[110,2415,2416,2419,2421,2423],{"class":112,"line":376},[110,2417,2418],{"class":154},"  fullyParallel",[110,2420,224],{"class":120},[110,2422,942],{"class":941},[110,2424,838],{"class":820},[110,2426,2427,2429,2431],{"class":112,"line":392},[110,2428,843],{"class":154},[110,2430,224],{"class":120},[110,2432,848],{"class":116},[110,2434,2435],{"class":112,"line":405},[110,2436,853],{"class":120},[110,2438,2439,2441,2443,2445,2447,2449],{"class":112,"line":419},[110,2440,858],{"class":154},[110,2442,224],{"class":120},[110,2444,227],{"class":186},[110,2446,93],{"class":190},[110,2448,187],{"class":186},[110,2450,838],{"class":120},[110,2452,2453,2455,2457,2459,2461,2463,2465,2467,2469,2471,2473],{"class":112,"line":433},[110,2454,873],{"class":154},[110,2456,224],{"class":120},[110,2458,878],{"class":186},[110,2460,121],{"class":881},[110,2462,885],{"class":884},[110,2464,889],{"class":888},[110,2466,892],{"class":190},[110,2468,889],{"class":888},[110,2470,105],{"class":190},[110,2472,899],{"class":186},[110,2474,838],{"class":120},[110,2476,2477,2479,2481,2483,2485,2487,2489,2491,2494,2497,2500,2502,2505,2507,2509,2511,2513,2515,2517],{"class":112,"line":446},[110,2478,906],{"class":154},[110,2480,224],{"class":120},[110,2482,878],{"class":186},[110,2484,121],{"class":881},[110,2486,885],{"class":884},[110,2488,889],{"class":888},[110,2490,128],{"class":186},[110,2492,2493],{"class":190},"month-end",[110,2495,2496],{"class":884},"|",[110,2498,2499],{"class":190},"spanish",[110,2501,2496],{"class":884},[110,2503,2504],{"class":190},"new-checkout",[110,2506,167],{"class":186},[110,2508,889],{"class":888},[110,2510,892],{"class":190},[110,2512,889],{"class":888},[110,2514,105],{"class":190},[110,2516,899],{"class":186},[110,2518,838],{"class":120},[110,2520,2521,2524,2526,2528,2531,2533,2536,2538,2541,2543,2547],{"class":112,"line":460},[110,2522,2523],{"class":154},"      use",[110,2525,224],{"class":120},[110,2527,783],{"class":120},[110,2529,2530],{"class":154}," baseURL",[110,2532,224],{"class":120},[110,2534,2535],{"class":116}," process",[110,2537,121],{"class":120},[110,2539,2540],{"class":116},"env",[110,2542,121],{"class":120},[110,2544,2546],{"class":2545},"sQ79N","DEFAULT_BASE_URL",[110,2548,1291],{"class":120},[110,2550,2551],{"class":112,"line":476},[110,2552,949],{"class":120},[110,2554,2555],{"class":112,"line":489},[110,2556,853],{"class":120},[110,2558,2559,2561,2563,2565,2568,2570],{"class":112,"line":503},[110,2560,858],{"class":154},[110,2562,224],{"class":120},[110,2564,227],{"class":186},[110,2566,2567],{"class":190},"month-end-date",[110,2569,187],{"class":186},[110,2571,838],{"class":120},[110,2573,2574,2576,2578,2580,2582,2584,2586,2588,2590,2592,2594,2596,2598],{"class":112,"line":517},[110,2575,873],{"class":154},[110,2577,224],{"class":120},[110,2579,878],{"class":186},[110,2581,121],{"class":881},[110,2583,885],{"class":884},[110,2585,889],{"class":888},[110,2587,2493],{"class":190},[110,2589,889],{"class":888},[110,2591,892],{"class":190},[110,2593,889],{"class":888},[110,2595,105],{"class":190},[110,2597,899],{"class":186},[110,2599,838],{"class":120},[110,2601,2602,2604,2606,2608,2610,2612,2614,2616,2618,2620,2623],{"class":112,"line":531},[110,2603,2523],{"class":154},[110,2605,224],{"class":120},[110,2607,783],{"class":120},[110,2609,2530],{"class":154},[110,2611,224],{"class":120},[110,2613,2535],{"class":116},[110,2615,121],{"class":120},[110,2617,2540],{"class":116},[110,2619,121],{"class":120},[110,2621,2622],{"class":2545},"MONTH_END_BASE_URL",[110,2624,1291],{"class":120},[110,2626,2627],{"class":112,"line":536},[110,2628,949],{"class":120},[110,2630,2631],{"class":112,"line":541},[110,2632,853],{"class":120},[110,2634,2635,2637,2639,2641,2644,2646],{"class":112,"line":551},[110,2636,858],{"class":154},[110,2638,224],{"class":120},[110,2640,227],{"class":186},[110,2642,2643],{"class":190},"spanish-locale",[110,2645,187],{"class":186},[110,2647,838],{"class":120},[110,2649,2650,2652,2654,2656,2658,2660,2662,2664,2666,2668,2670,2672,2674],{"class":112,"line":556},[110,2651,873],{"class":154},[110,2653,224],{"class":120},[110,2655,878],{"class":186},[110,2657,121],{"class":881},[110,2659,885],{"class":884},[110,2661,889],{"class":888},[110,2663,2499],{"class":190},[110,2665,889],{"class":888},[110,2667,892],{"class":190},[110,2669,889],{"class":888},[110,2671,105],{"class":190},[110,2673,899],{"class":186},[110,2675,838],{"class":120},[110,2677,2678,2680,2682,2684,2686,2688,2690,2692,2694,2696,2699],{"class":112,"line":581},[110,2679,2523],{"class":154},[110,2681,224],{"class":120},[110,2683,783],{"class":120},[110,2685,2530],{"class":154},[110,2687,224],{"class":120},[110,2689,2535],{"class":116},[110,2691,121],{"class":120},[110,2693,2540],{"class":116},[110,2695,121],{"class":120},[110,2697,2698],{"class":2545},"SPANISH_BASE_URL",[110,2700,1291],{"class":120},[110,2702,2703],{"class":112,"line":586},[110,2704,949],{"class":120},[110,2706,2707],{"class":112,"line":602},[110,2708,853],{"class":120},[110,2710,2711,2713,2715,2717,2720,2722],{"class":112,"line":616},[110,2712,858],{"class":154},[110,2714,224],{"class":120},[110,2716,227],{"class":186},[110,2718,2719],{"class":190},"new-checkout-flag",[110,2721,187],{"class":186},[110,2723,838],{"class":120},[110,2725,2726,2728,2730,2732,2734,2736,2738,2740,2742,2744,2746,2748,2750],{"class":112,"line":621},[110,2727,873],{"class":154},[110,2729,224],{"class":120},[110,2731,878],{"class":186},[110,2733,121],{"class":881},[110,2735,885],{"class":884},[110,2737,889],{"class":888},[110,2739,2504],{"class":190},[110,2741,889],{"class":888},[110,2743,892],{"class":190},[110,2745,889],{"class":888},[110,2747,105],{"class":190},[110,2749,899],{"class":186},[110,2751,838],{"class":120},[110,2753,2754,2756,2758,2760,2762,2764,2766,2768,2770,2772,2775],{"class":112,"line":627},[110,2755,2523],{"class":154},[110,2757,224],{"class":120},[110,2759,783],{"class":120},[110,2761,2530],{"class":154},[110,2763,224],{"class":120},[110,2765,2535],{"class":116},[110,2767,121],{"class":120},[110,2769,2540],{"class":116},[110,2771,121],{"class":120},[110,2773,2774],{"class":2545},"NEW_CHECKOUT_BASE_URL",[110,2776,1291],{"class":120},[110,2778,2779],{"class":112,"line":632},[110,2780,949],{"class":120},[110,2782,2783,2785],{"class":112,"line":638},[110,2784,1021],{"class":116},[110,2786,838],{"class":820},[110,2788,2789,2791,2793],{"class":112,"line":647},[110,2790,164],{"class":820},[110,2792,167],{"class":817},[110,2794,158],{"class":120},[11,2796,2797,2798,2800,2801,2803,2804,2808],{},"Each environment flavor gets its own base URL in a ",[21,2799,2372],{}," file, which the config loads with ",[21,2802,2311],{}," as the ",[79,2805],{"href":2806,"text":2807},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Ftest-parameterize","Playwright docs"," suggest:",[100,2810,2812],{"className":2180,"code":2811,"filename":2372,"language":2182,"meta":106,"style":106},"DEFAULT_BASE_URL=https:\u002F\u002Ftest.example.com\nMONTH_END_BASE_URL=https:\u002F\u002Ftest-month-end.example.com\nSPANISH_BASE_URL=https:\u002F\u002Ftest-es.example.com\nNEW_CHECKOUT_BASE_URL=https:\u002F\u002Ftest-new-checkout.example.com\n",[21,2813,2814,2823,2832,2841],{"__ignoreMap":106},[110,2815,2816,2818,2820],{"class":112,"line":113},[110,2817,2546],{"class":116},[110,2819,2192],{"class":884},[110,2821,2822],{"class":190},"https:\u002F\u002Ftest.example.com\n",[110,2824,2825,2827,2829],{"class":112,"line":144},[110,2826,2622],{"class":116},[110,2828,2192],{"class":884},[110,2830,2831],{"class":190},"https:\u002F\u002Ftest-month-end.example.com\n",[110,2833,2834,2836,2838],{"class":112,"line":161},[110,2835,2698],{"class":116},[110,2837,2192],{"class":884},[110,2839,2840],{"class":190},"https:\u002F\u002Ftest-es.example.com\n",[110,2842,2843,2845,2847],{"class":112,"line":172},[110,2844,2774],{"class":116},[110,2846,2192],{"class":884},[110,2848,2849],{"class":190},"https:\u002F\u002Ftest-new-checkout.example.com\n",[11,2851,2852,2853,2856,2857,2860,2861,2863],{},"A test file such as ",[21,2854,2855],{},"billing.month-end.spec.ts"," then uses relative navigation like ",[21,2858,2859],{},"page.goto('\u002Fbilling')",", and ",[21,2862,2270],{}," decides which environment it hits. In CI you'd set the same variables from your environment or secrets instead of a file.",[11,2865,2866,2867,2869],{},"The demo repo runs this configuration in one command with no ",[21,2868,1370],{}," and no separate steps:",[100,2871,2873],{"className":318,"code":2872,"filename":320,"language":321,"meta":106,"style":106},"npx playwright test -c playwright.environments.config.ts\n◇ injected env (4) from .env.example\n\n\u001b[2mRunning \u001b[22m10\u001b[2m tests using \u001b[22m10\u001b[2m workers\u001b[22m\n\n◇ injected env (0) from .env.example\n◇ injected env (0) from .env.example\n◇ injected env (0) from .env.example\n  \u001b[32m✓\u001b[39m  \u001b[2m 1 \u001b[22m[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 1 runs against the default environment\u001b[2m (310ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 2 \u001b[22m[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 4 runs against the default environment\u001b[2m (313ms)\u001b[22m\n◇ injected env (0) from .env.example\n◇ injected env (0) from .env.example\n◇ injected env (0) from .env.example\n  \u001b[32m✓\u001b[39m  \u001b[2m 3 \u001b[22m[month-end-date] › tests-environments\\billing.month-end.spec.ts:7:9 › billing rollover (month-end environment) › rollover scenario 1\u001b[2m (319ms)\u001b[22m\n◇ injected env (0) from .env.example\n  \u001b[32m✓\u001b[39m  \u001b[2m 4 \u001b[22m[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 3 runs against the default environment\u001b[2m (314ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 5 \u001b[22m[month-end-date] › tests-environments\\billing.month-end.spec.ts:7:9 › billing rollover (month-end environment) › rollover scenario 2\u001b[2m (314ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 6 \u001b[22m[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 2 runs against the default environment\u001b[2m (311ms)\u001b[22m\n◇ injected env (0) from .env.example\n  \u001b[32m✓\u001b[39m  \u001b[2m 7 \u001b[22m[spanish-locale] › tests-environments\\account.spanish.spec.ts:7:9 › account page (Spanish environment) › account scenario 2\u001b[2m (316ms)\u001b[22m\n◇ injected env (0) from .env.example\n  \u001b[32m✓\u001b[39m  \u001b[2m 8 \u001b[22m[new-checkout-flag] › tests-environments\\checkout.new-checkout.spec.ts:7:9 › checkout (new-checkout flag environment) › checkout scenario 1\u001b[2m (306ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m 9 \u001b[22m[new-checkout-flag] › tests-environments\\checkout.new-checkout.spec.ts:7:9 › checkout (new-checkout flag environment) › checkout scenario 2\u001b[2m (305ms)\u001b[22m\n◇ injected env (0) from .env.example\n  \u001b[32m✓\u001b[39m  \u001b[2m10 \u001b[22m[spanish-locale] › tests-environments\\account.spanish.spec.ts:7:9 › account page (Spanish environment) › account scenario 1\u001b[2m (306ms)\u001b[22m\n\n\u001b[32m  10 passed\u001b[39m\u001b[2m (726ms)\u001b[22m\n",[21,2874,2875,2880,2885,2889,2902,2906,2911,2915,2919,2930,2941,2945,2949,2953,2964,2968,2980,2991,3002,3006,3017,3021,3033,3044,3048,3059,3063],{"__ignoreMap":106},[110,2876,2877],{"class":112,"line":113},[110,2878,2879],{"class":328},"npx playwright test -c playwright.environments.config.ts\n",[110,2881,2882],{"class":112,"line":144},[110,2883,2884],{"class":328},"◇ injected env (4) from .env.example\n",[110,2886,2887],{"class":112,"line":161},[110,2888,176],{"emptyLinePlaceholder":175},[110,2890,2891,2893,2896,2898,2900],{"class":112,"line":172},[110,2892,357],{"class":356},[110,2894,2895],{"class":328},"10",[110,2897,363],{"class":356},[110,2899,2895],{"class":328},[110,2901,369],{"class":356},[110,2903,2904],{"class":112,"line":179},[110,2905,176],{"emptyLinePlaceholder":175},[110,2907,2908],{"class":112,"line":208},[110,2909,2910],{"class":328},"◇ injected env (0) from .env.example\n",[110,2912,2913],{"class":112,"line":256},[110,2914,2910],{"class":328},[110,2916,2917],{"class":112,"line":298},[110,2918,2910],{"class":328},[110,2920,2921,2923,2925,2928],{"class":112,"line":376},[110,2922,380],{"class":379},[110,2924,383],{"class":356},[110,2926,2927],{"class":328},"[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 1 runs against the default environment",[110,2929,1787],{"class":356},[110,2931,2932,2934,2936,2939],{"class":112,"line":392},[110,2933,380],{"class":379},[110,2935,397],{"class":356},[110,2937,2938],{"class":328},"[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 4 runs against the default environment",[110,2940,1587],{"class":356},[110,2942,2943],{"class":112,"line":405},[110,2944,2910],{"class":328},[110,2946,2947],{"class":112,"line":419},[110,2948,2910],{"class":328},[110,2950,2951],{"class":112,"line":433},[110,2952,2910],{"class":328},[110,2954,2955,2957,2959,2962],{"class":112,"line":446},[110,2956,380],{"class":379},[110,2958,410],{"class":356},[110,2960,2961],{"class":328},"[month-end-date] › tests-environments\\billing.month-end.spec.ts:7:9 › billing rollover (month-end environment) › rollover scenario 1",[110,2963,389],{"class":356},[110,2965,2966],{"class":112,"line":460},[110,2967,2910],{"class":328},[110,2969,2970,2972,2974,2977],{"class":112,"line":476},[110,2971,380],{"class":379},[110,2973,424],{"class":356},[110,2975,2976],{"class":328},"[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 3 runs against the default environment",[110,2978,2979],{"class":356}," (314ms)\n",[110,2981,2982,2984,2986,2989],{"class":112,"line":489},[110,2983,380],{"class":379},[110,2985,451],{"class":356},[110,2987,2988],{"class":328},"[month-end-date] › tests-environments\\billing.month-end.spec.ts:7:9 › billing rollover (month-end environment) › rollover scenario 2",[110,2990,2979],{"class":356},[110,2992,2993,2995,2997,3000],{"class":112,"line":503},[110,2994,380],{"class":379},[110,2996,494],{"class":356},[110,2998,2999],{"class":328},"[parallel-safe] › tests-environments\\pricing.spec.ts:8:9 › pricing (default environment) › quote 2 runs against the default environment",[110,3001,457],{"class":356},[110,3003,3004],{"class":112,"line":517},[110,3005,2910],{"class":328},[110,3007,3008,3010,3012,3015],{"class":112,"line":531},[110,3009,380],{"class":379},[110,3011,481],{"class":356},[110,3013,3014],{"class":328},"[spanish-locale] › tests-environments\\account.spanish.spec.ts:7:9 › account page (Spanish environment) › account scenario 2",[110,3016,1564],{"class":356},[110,3018,3019],{"class":112,"line":536},[110,3020,2910],{"class":328},[110,3022,3023,3025,3027,3030],{"class":112,"line":541},[110,3024,380],{"class":379},[110,3026,467],{"class":356},[110,3028,3029],{"class":328},"[new-checkout-flag] › tests-environments\\checkout.new-checkout.spec.ts:7:9 › checkout (new-checkout flag environment) › checkout scenario 1",[110,3031,3032],{"class":356}," (306ms)\n",[110,3034,3035,3037,3039,3042],{"class":112,"line":551},[110,3036,380],{"class":379},[110,3038,439],{"class":356},[110,3040,3041],{"class":328},"[new-checkout-flag] › tests-environments\\checkout.new-checkout.spec.ts:7:9 › checkout (new-checkout flag environment) › checkout scenario 2",[110,3043,473],{"class":356},[110,3045,3046],{"class":112,"line":556},[110,3047,2910],{"class":328},[110,3049,3050,3052,3054,3057],{"class":112,"line":581},[110,3051,380],{"class":379},[110,3053,508],{"class":356},[110,3055,3056],{"class":328},"[spanish-locale] › tests-environments\\account.spanish.spec.ts:7:9 › account page (Spanish environment) › account scenario 1",[110,3058,3032],{"class":356},[110,3060,3061],{"class":112,"line":586},[110,3062,176],{"emptyLinePlaceholder":175},[110,3064,3065,3067],{"class":112,"line":602},[110,3066,1646],{"class":379},[110,3068,3069],{"class":356}," (726ms)\n",[11,3071,3072,3073,3075],{},"All four projects ran together, each handing its own ",[21,3074,2270],{}," to its tests. The demo's tests check that each project provides a set, valid base URL that differs from every other environment's, but they don't prove an environment is actually configured, since the example URLs don't point at real servers.",[11,3077,1048,3078,3081,3082,3084,3085,3088,3089,3092,3093,3096],{},[21,3079,3080],{},"injected env"," lines are printed by ",[21,3083,2311],{},", not Playwright. The demo loads ",[21,3086,3087],{},".env.example"," so it runs out of the box. The ",[21,3090,3091],{},"(4)"," is the main process loading the four base URLs, and the ",[21,3094,3095],{},"(0)"," lines appear as each worker starts and loads the config.",[1091,3098,3100],{"id":3099},"preconfigured-test-environments-the-cost","Preconfigured Test Environments: The Cost",[11,3102,3103],{},"Preconfigured test environments remove the need for serial tests since you can run those tests in parallel on isolated environments preconfigured with their preconditions where they won't pollute other tests running at the same time on other servers. This is the most performant of the solutions, but may not be possible for everyone.",[11,3105,3106],{},"For example, I want to use this approach, but I want these tests running on every pull request. We may have multiple pull requests that come in at a time and these preconfigured environments, which we provision from a base image with the PR applied, may need 5 or more flavors so that would mean creating 25 environments to support 5 pull requests. Since the system under test requires VMs instead of containers this is pretty expensive in provisioning time and quota.",[11,3108,3109],{},"Below are some things to evaluate:",[37,3111,3112,3118,3124,3130,3139],{},[40,3113,3114,3117],{},[43,3115,3116],{},"Hosting:"," Every flavor is another environment to pay for, and your hosting quota may not stretch to it.",[40,3119,3120,3123],{},[43,3121,3122],{},"Creating environments:"," It only works if you can stand up a preconfigured environment easily and repeatably. If a new environment takes days of manual setup, you won't keep several of them healthy.",[40,3125,3126,3129],{},[43,3127,3128],{},"Drift:"," Each flavor has to stay close to the baseline environment. A month-end environment that has quietly fallen behind the default one tests a different system.",[40,3131,3132,3135,3136,3138],{},[43,3133,3134],{},"Hardcoded URLs:"," A test that calls an absolute URL ignores ",[21,3137,2270],{},", and so does any test that reaches the server some other way, like a database connection.",[40,3140,3141,3144],{},[43,3142,3143],{},"Provisioning:"," Playwright only routes the tests. You still have to build and configure the environments yourself.",[11,3146,3147],{},"If you can't afford it, options A and B above are still solid. They're slower, because the serial tests wait for the parallel ones and run one at a time, but they need no extra environments.",[32,3149,3151],{"id":3150},"which-approach-to-use-for-serial-and-parallel-playwright-tests","Which Approach to Use for Serial and Parallel Playwright Tests",[3153,3154,3155,3175],"table",{},[3156,3157,3158],"thead",{},[3159,3160,3161,3164,3169,3172],"tr",{},[3162,3163],"th",{},[3162,3165,3166,3167],{},"Option A: Project ",[21,3168,1370],{},[3162,3170,3171],{},"Option B: Separate CI steps",[3162,3173,3174],{},"Option C: Preconfigured environments",[3176,3177,3178,3192,3205,3217,3233,3246,3260],"tbody",{},[3159,3179,3180,3184,3187,3189],{},[3181,3182,3183],"td",{},"Tests change global state on the fly",[3181,3185,3186],{},"Yes",[3181,3188,3186],{},[3181,3190,3191],{},"No, the environment is already set up",[3159,3193,3194,3197,3199,3202],{},[3181,3195,3196],{},"Runs state-changing tests after parallel ones",[3181,3198,3186],{},[3181,3200,3201],{},"Yes, by step order",[3181,3203,3204],{},"No ordering needed, everything runs together",[3159,3206,3207,3210,3213,3215],{},[3181,3208,3209],{},"Runs state-changing tests when a parallel test fails",[3181,3211,3212],{},"No",[3181,3214,3186],{},[3181,3216,3186],{},[3159,3218,3219,3226,3228,3231],{},[3181,3220,3221,3222,3225],{},"Single ",[21,3223,3224],{},"npx playwright test"," command",[3181,3227,3186],{},[3181,3229,3230],{},"No, one command per project",[3181,3232,3186],{},[3159,3234,3235,3238,3241,3243],{},[3181,3236,3237],{},"Extra infrastructure",[3181,3239,3240],{},"None",[3181,3242,3240],{},[3181,3244,3245],{},"One environment per flavor",[3159,3247,3248,3251,3254,3257],{},[3181,3249,3250],{},"Speed",[3181,3252,3253],{},"Slowest",[3181,3255,3256],{},"Slow",[3181,3258,3259],{},"Fastest",[3159,3261,3262,3265,3268,3271],{},[3181,3263,3264],{},"Best when",[3181,3266,3267],{},"The serial tests only make sense if the rest passed, like a smoke gate",[3181,3269,3270],{},"The two suites are independent and you can't add environments",[3181,3272,3273],{},"You can stand up preconfigured environments at a reasonable cost",[11,3275,3276,3277,3279],{},"If the serial tests build on the parallel ones, ",[21,3278,1370],{}," is the simpler option and skipping them on failure is the right behavior. For tests that only need the environment to themselves, use separate CI steps. If you can afford the environments, use those.",[32,3281,3283],{"id":3282},"reduce-how-many-tests-need-to-run-serially","Reduce How Many Tests Need to Run Serially",[11,3285,3286],{},"Splitting the suites fixes when the tests run. It doesn't make the state-changing tests any less fragile, and options A and B still need the environment to themselves. Fewer tests in the serial project means faster runs, so it's worth asking of each one whether it really needs to change global state or could be rewritten to leverage Playwright features that can emulate some of these things.",[11,3288,3289,3290,3293,3294,3298],{},"For example, if a test only depends on the browser's clock, Playwright has a ",[21,3291,3292],{},"page.clock"," API for controlling it (see the ",[79,3295],{"href":3296,"text":3297},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Fclock","Playwright Clock docs","). I haven't used it. It wouldn't help my case anyway, because the date I care about is on the server and triggers backend business processes, but it might solve some time travel test cases for others.",[1091,3300,3302],{"id":3301},"testinfooutputpath-when-multiple-tests-save-the-same-file","testInfo.outputPath: When Multiple Tests Save the Same File",[11,3304,3305,3306,3309],{},"If you are running into situations where parallel files touch the same output file you can leverage Playwright ",[21,3307,3308],{},"testInfo.outputPath"," command which returns a path scoped to the current test to prevent parallel collisions.",[100,3311,3313],{"className":102,"code":3312,"filename":770,"language":105,"meta":106,"style":106},"import { test } from '@playwright\u002Ftest';\n\ntest('exports rates PDF', async ({ page }, testInfo) => {\n  \u002F\u002F ...some sort of file download scenario\n  const file = testInfo.outputPath('insurance-rates.pdf'); \u002F\u002F Scoped, won't conflict with other tests downloading the same-named file with different contents\n  await download.saveAs(filePath);\n  \u002F\u002F ...verify the file contents\n});\n",[21,3314,3315,3335,3339,3374,3379,3414,3435,3440],{"__ignoreMap":106},[110,3316,3317,3319,3321,3323,3325,3327,3329,3331,3333],{"class":112,"line":113},[110,3318,780],{"class":147},[110,3320,783],{"class":120},[110,3322,2203],{"class":116},[110,3324,249],{"class":120},[110,3326,791],{"class":147},[110,3328,227],{"class":186},[110,3330,796],{"class":190},[110,3332,187],{"class":186},[110,3334,158],{"class":120},[110,3336,3337],{"class":112,"line":144},[110,3338,176],{"emptyLinePlaceholder":175},[110,3340,3341,3343,3345,3347,3350,3352,3354,3356,3359,3363,3365,3368,3370,3372],{"class":112,"line":161},[110,3342,117],{"class":124},[110,3344,128],{"class":116},[110,3346,187],{"class":186},[110,3348,3349],{"class":190},"exports rates PDF",[110,3351,187],{"class":186},[110,3353,196],{"class":120},[110,3355,199],{"class":131},[110,3357,3358],{"class":120}," ({",[110,3360,3362],{"class":3361},"s2xgV"," page",[110,3364,1145],{"class":120},[110,3366,3367],{"class":3361}," testInfo",[110,3369,167],{"class":120},[110,3371,138],{"class":131},[110,3373,141],{"class":120},[110,3375,3376],{"class":112,"line":172},[110,3377,3378],{"class":1011},"  \u002F\u002F ...some sort of file download scenario\n",[110,3380,3381,3384,3387,3390,3392,3394,3397,3399,3401,3404,3406,3408,3411],{"class":112,"line":179},[110,3382,3383],{"class":131},"  const",[110,3385,3386],{"class":2545}," file",[110,3388,3389],{"class":884}," =",[110,3391,3367],{"class":116},[110,3393,121],{"class":120},[110,3395,3396],{"class":124},"outputPath",[110,3398,128],{"class":154},[110,3400,187],{"class":186},[110,3402,3403],{"class":190},"insurance-rates.pdf",[110,3405,187],{"class":186},[110,3407,167],{"class":154},[110,3409,3410],{"class":120},";",[110,3412,3413],{"class":1011}," \u002F\u002F Scoped, won't conflict with other tests downloading the same-named file with different contents\n",[110,3415,3416,3418,3421,3423,3426,3428,3431,3433],{"class":112,"line":208},[110,3417,148],{"class":147},[110,3419,3420],{"class":116}," download",[110,3422,121],{"class":120},[110,3424,3425],{"class":124},"saveAs",[110,3427,128],{"class":154},[110,3429,3430],{"class":116},"filePath",[110,3432,167],{"class":154},[110,3434,158],{"class":120},[110,3436,3437],{"class":112,"line":256},[110,3438,3439],{"class":1011},"  \u002F\u002F ...verify the file contents\n",[110,3441,3442,3444,3446],{"class":112,"line":298},[110,3443,164],{"class":120},[110,3445,167],{"class":116},[110,3447,158],{"class":120},[1091,3449,3451],{"id":3450},"test-locks-when-only-a-few-tests-share-the-resource","Test Locks: When Only a Few Tests Share the Resource",[11,3453,3454,3455,3459,3460,3463],{},"Playwright 1.63 added ",[79,3456],{"href":3457,"text":3458},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Ftest-parallel#test-locks","test locks",". Give the tests that touch the same shared resource the same named ",[21,3461,3462],{},"lock",", and Playwright never runs them at the same time, across files, workers and projects, while every other test keeps running in parallel.",[100,3465,3468],{"className":102,"code":3466,"filename":3467,"language":105,"meta":106,"style":106},"test('feature flag on enables new checkout', { lock: 'server-settings' }, async () => {\n  \u002F\u002F ...\n});\n\ntest('server date set to month end triggers billing rollover', { lock: 'server-settings' }, async () => {\n  \u002F\u002F ...\n});\n","tests-locks\u002Fmutators.spec.ts",[21,3469,3470,3508,3513,3521,3525,3561,3565],{"__ignoreMap":106},[110,3471,3472,3474,3476,3478,3480,3482,3484,3486,3489,3491,3493,3496,3498,3500,3502,3504,3506],{"class":112,"line":113},[110,3473,117],{"class":124},[110,3475,128],{"class":116},[110,3477,187],{"class":186},[110,3479,1201],{"class":190},[110,3481,187],{"class":186},[110,3483,196],{"class":120},[110,3485,783],{"class":120},[110,3487,3488],{"class":154}," lock",[110,3490,224],{"class":120},[110,3492,227],{"class":186},[110,3494,3495],{"class":190},"server-settings",[110,3497,187],{"class":186},[110,3499,1145],{"class":120},[110,3501,199],{"class":131},[110,3503,135],{"class":120},[110,3505,138],{"class":131},[110,3507,141],{"class":120},[110,3509,3510],{"class":112,"line":144},[110,3511,3512],{"class":1011},"  \u002F\u002F ...\n",[110,3514,3515,3517,3519],{"class":112,"line":161},[110,3516,164],{"class":120},[110,3518,167],{"class":116},[110,3520,158],{"class":120},[110,3522,3523],{"class":112,"line":172},[110,3524,176],{"emptyLinePlaceholder":175},[110,3526,3527,3529,3531,3533,3535,3537,3539,3541,3543,3545,3547,3549,3551,3553,3555,3557,3559],{"class":112,"line":179},[110,3528,117],{"class":124},[110,3530,128],{"class":116},[110,3532,187],{"class":186},[110,3534,191],{"class":190},[110,3536,187],{"class":186},[110,3538,196],{"class":120},[110,3540,783],{"class":120},[110,3542,3488],{"class":154},[110,3544,224],{"class":120},[110,3546,227],{"class":186},[110,3548,3495],{"class":190},[110,3550,187],{"class":186},[110,3552,1145],{"class":120},[110,3554,199],{"class":131},[110,3556,135],{"class":120},[110,3558,138],{"class":131},[110,3560,141],{"class":120},[110,3562,3563],{"class":112,"line":208},[110,3564,3512],{"class":1011},[110,3566,3567,3569,3571],{"class":112,"line":256},[110,3568,164],{"class":120},[110,3570,167],{"class":116},[110,3572,158],{"class":120},[11,3574,3575,3576,3578],{},"This is a good fit when a few tests only conflict with each other, like several tests that edit the same account setting. Those can stay in the main parallel suite instead of moving to the ",[21,3577,97],{}," project. In the demo, the two tests above overwrote each other's state and failed in every one of 5 runs when I removed the lock, and passed in every one of 5 runs with it.",[11,3580,3581,3582,3584,3585,3588],{},"It doesn't solve my problem, though. A lock only keeps apart the tests that hold it. The ",[21,3583,93],{}," tests that read the settings don't hold it, so they keep running while a locked test has the state changed. That's what ",[21,3586,3587],{},"npm run test:locks:partial"," shows in the demo:",[100,3590,3592],{"className":318,"code":3591,"filename":320,"language":321,"meta":106,"style":106},"npm run test:locks:partial\n\n> playwright-serial-parallel-dependencies-demo@1.0.0 test:locks:partial\n> playwright test -c playwright.locks.config.ts mutators readers-unlocked\n\n\n\u001b[2mRunning \u001b[22m6\u001b[2m tests using \u001b[22m5\u001b[2m workers\u001b[22m\n\n  \u001b[31m✘\u001b[39m  \u001b[2m3 \u001b[22m\u001b[31mtests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 3 renders with default settings\u001b[39m\u001b[2m (320ms)\u001b[22m\n  \u001b[31m✘\u001b[39m  \u001b[2m1 \u001b[22m\u001b[31mtests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 2 renders with default settings\u001b[39m\u001b[2m (322ms)\u001b[22m\n  \u001b[31m✘\u001b[39m  \u001b[2m5 \u001b[22m\u001b[31mtests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 1 renders with default settings\u001b[39m\u001b[2m (315ms)\u001b[22m\n  \u001b[31m✘\u001b[39m  \u001b[2m4 \u001b[22m\u001b[31mtests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 4 renders with default settings\u001b[39m\u001b[2m (322ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m2 \u001b[22mtests-locks\\mutators.spec.ts:13:5 › feature flag on enables new checkout\u001b[2m (518ms)\u001b[22m\n  \u001b[32m✓\u001b[39m  \u001b[2m6 \u001b[22mtests-locks\\mutators.spec.ts:19:5 › server date set to month end triggers billing rollover\u001b[2m (503ms)\u001b[22m\n\n\n\u001b[31m  1) tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 1 renders with default settings \u001b[39m\n\n    Error: \u001b[2mexpect(\u001b[22m\u001b[31mreceived\u001b[39m\u001b[2m).\u001b[22mtoBe\u001b[2m(\u001b[22m\u001b[32mexpected\u001b[39m\u001b[2m) \u002F\u002F Object.is equality\u001b[22m\n\n    Expected: \u001b[32m\"o\u001b[7mff\u001b[27m\"\u001b[39m\n    Received: \u001b[31m\"o\u001b[7mn\u001b[27m\"\u001b[39m\n\n    ...\n\n\u001b[31m  4 failed\u001b[39m\n\u001b[31m    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 1 renders with default settings \u001b[39m\n\u001b[31m    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 2 renders with default settings \u001b[39m\n\u001b[31m    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 3 renders with default settings \u001b[39m\n\u001b[31m    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 4 renders with default settings \u001b[39m\n\u001b[32m  2 passed\u001b[39m\u001b[2m (1.3s)\u001b[22m\n",[21,3593,3594,3599,3603,3608,3613,3617,3621,3634,3638,3650,3662,3675,3687,3700,3713,3717,3721,3726,3730,3748,3752,3762,3772,3776,3780,3784,3789,3794,3799,3804,3809],{"__ignoreMap":106},[110,3595,3596],{"class":112,"line":113},[110,3597,3598],{"class":328},"npm run test:locks:partial\n",[110,3600,3601],{"class":112,"line":144},[110,3602,176],{"emptyLinePlaceholder":175},[110,3604,3605],{"class":112,"line":161},[110,3606,3607],{"class":328},"> playwright-serial-parallel-dependencies-demo@1.0.0 test:locks:partial\n",[110,3609,3610],{"class":112,"line":172},[110,3611,3612],{"class":328},"> playwright test -c playwright.locks.config.ts mutators readers-unlocked\n",[110,3614,3615],{"class":112,"line":179},[110,3616,176],{"emptyLinePlaceholder":175},[110,3618,3619],{"class":112,"line":208},[110,3620,176],{"emptyLinePlaceholder":175},[110,3622,3623,3625,3628,3630,3632],{"class":112,"line":256},[110,3624,357],{"class":356},[110,3626,3627],{"class":328},"6",[110,3629,363],{"class":356},[110,3631,366],{"class":328},[110,3633,369],{"class":356},[110,3635,3636],{"class":112,"line":298},[110,3637,176],{"emptyLinePlaceholder":175},[110,3639,3640,3642,3645,3648],{"class":112,"line":376},[110,3641,464],{"class":463},[110,3643,3644],{"class":356},"  3 ",[110,3646,3647],{"class":463},"tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 3 renders with default settings",[110,3649,1541],{"class":356},[110,3651,3652,3654,3657,3660],{"class":112,"line":392},[110,3653,464],{"class":463},[110,3655,3656],{"class":356},"  1 ",[110,3658,3659],{"class":463},"tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 2 renders with default settings",[110,3661,1776],{"class":356},[110,3663,3664,3666,3669,3672],{"class":112,"line":405},[110,3665,464],{"class":463},[110,3667,3668],{"class":356},"  5 ",[110,3670,3671],{"class":463},"tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 1 renders with default settings",[110,3673,3674],{"class":356}," (315ms)\n",[110,3676,3677,3679,3682,3685],{"class":112,"line":419},[110,3678,464],{"class":463},[110,3680,3681],{"class":356},"  4 ",[110,3683,3684],{"class":463},"tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 4 renders with default settings",[110,3686,1776],{"class":356},[110,3688,3689,3691,3694,3697],{"class":112,"line":433},[110,3690,380],{"class":379},[110,3692,3693],{"class":356},"  2 ",[110,3695,3696],{"class":328},"tests-locks\\mutators.spec.ts:13:5 › feature flag on enables new checkout",[110,3698,3699],{"class":356}," (518ms)\n",[110,3701,3702,3704,3707,3710],{"class":112,"line":446},[110,3703,380],{"class":379},[110,3705,3706],{"class":356},"  6 ",[110,3708,3709],{"class":328},"tests-locks\\mutators.spec.ts:19:5 › server date set to month end triggers billing rollover",[110,3711,3712],{"class":356}," (503ms)\n",[110,3714,3715],{"class":112,"line":460},[110,3716,176],{"emptyLinePlaceholder":175},[110,3718,3719],{"class":112,"line":476},[110,3720,176],{"emptyLinePlaceholder":175},[110,3722,3723],{"class":112,"line":489},[110,3724,3725],{"class":463},"  1) tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 1 renders with default settings \n",[110,3727,3728],{"class":112,"line":503},[110,3729,176],{"emptyLinePlaceholder":175},[110,3731,3732,3734,3736,3738,3740,3742,3744,3746],{"class":112,"line":517},[110,3733,559],{"class":328},[110,3735,562],{"class":356},[110,3737,565],{"class":463},[110,3739,568],{"class":356},[110,3741,283],{"class":328},[110,3743,128],{"class":356},[110,3745,575],{"class":379},[110,3747,578],{"class":356},[110,3749,3750],{"class":112,"line":531},[110,3751,176],{"emptyLinePlaceholder":175},[110,3753,3754,3756,3758,3760],{"class":112,"line":536},[110,3755,589],{"class":328},[110,3757,592],{"class":379},[110,3759,596],{"class":595},[110,3761,599],{"class":379},[110,3763,3764,3766,3768,3770],{"class":112,"line":541},[110,3765,605],{"class":328},[110,3767,592],{"class":463},[110,3769,611],{"class":610},[110,3771,599],{"class":463},[110,3773,3774],{"class":112,"line":551},[110,3775,176],{"emptyLinePlaceholder":175},[110,3777,3778],{"class":112,"line":556},[110,3779,624],{"class":328},[110,3781,3782],{"class":112,"line":581},[110,3783,176],{"emptyLinePlaceholder":175},[110,3785,3786],{"class":112,"line":586},[110,3787,3788],{"class":463},"  4 failed\n",[110,3790,3791],{"class":112,"line":602},[110,3792,3793],{"class":463},"    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 1 renders with default settings \n",[110,3795,3796],{"class":112,"line":616},[110,3797,3798],{"class":463},"    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 2 renders with default settings \n",[110,3800,3801],{"class":112,"line":621},[110,3802,3803],{"class":463},"    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 3 renders with default settings \n",[110,3805,3806],{"class":112,"line":627},[110,3807,3808],{"class":463},"    tests-locks\\readers-unlocked.spec.ts:9:9 › reports (no lock) › report 4 renders with default settings \n",[110,3810,3811,3814],{"class":112,"line":632},[110,3812,3813],{"class":379},"  2 passed",[110,3815,3816],{"class":356}," (1.3s)\n",[11,3818,3819],{},"All four unlocked tests failed in each of 5 runs on my machine. You could give those tests the same lock and everything passes, but then they run one at a time too: the demo took about 2.6 seconds that way, against 1.3 seconds with the readers unlocked. When a change affects every test, like the server date, putting a lock on everything is a serial run again so locking should be used sparingly.",[11,3821,3822,3823,3825],{},"Also note that a lock on any test in a spec file is held for the duration of the whole file. That means if one test in a spec file needs the lock, the other tests in that file count as holding it too, and a locked test in another file has to wait until the whole spec file finishes. Keeping the tests that need a lock in their own file, or turning on ",[21,3824,1058],{},", avoids that.",[32,3827,3829],{"id":3828},"takeaway-run-serial-and-parallel-playwright-tests-together-without-failures","Takeaway: Run Serial and Parallel Playwright Tests Together Without Failures",[11,3831,3832,3833,3835],{},"Tests that change global state don't have to force your whole suite to run serially. Mark them, match them with a filename or a tag, and give them their own project with ",[21,3834,695],{},". Then choose how the projects relate:",[37,3837,3838,3846,3852],{},[40,3839,3840,3845],{},[43,3841,3842,3843,196],{},"Option A, ",[21,3844,1370],{}," is the quick win when the serial tests should only run once everything else has passed.",[40,3847,3848,3851],{},[43,3849,3850],{},"Option B, separate CI steps,"," takes a little more wiring, but a red parallel run never hides your serial results.",[40,3853,3854,3857],{},[43,3855,3856],{},"Option C, preconfigured environments,"," is my long-term goal: if you can afford them, the tests never have to change global state at all.",[11,3859,3860,3861,121],{},"The demo repo has every variant if you want to try them. For more, see my ",[46,3862,3864],{"href":3863},"\u002Fsoftware-testing\u002Fframeworks\u002Fplaywright","Playwright articles",[3866,3867],"read-next",{":items":3868},"[\"\u002Fsoftware-testing\u002Ftest-automation\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Frun-fewer-tests-catch-the-same-bugs\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-playwright-ai-cost-efficient-testing\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fbest-websites-for-practicing-test-automation\"]",[3870,3871,3872],"style",{},"html pre.shiki code .sZ-rw,html code.shiki .sZ-rw{--shiki-light:#90A4AE;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sPJuK,html code.shiki .sPJuK{--shiki-light:#39ADB5;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sb1SK,html code.shiki .sb1SK{--shiki-light:#6182B8;--shiki-default:#622CBC;--shiki-dark:#DBB7FF}html pre.shiki code .stWsX,html code.shiki .stWsX{--shiki-light:#9C3EDA;--shiki-default:#A0111F;--shiki-dark:#FF9492}html pre.shiki code .sZTni,html code.shiki .sZTni{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#A0111F;--shiki-default-font-style:inherit;--shiki-dark:#FF9492;--shiki-dark-font-style:inherit}html pre.shiki code .sq0XF,html code.shiki .sq0XF{--shiki-light:#E53935;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sZi47,html code.shiki .sZi47{--shiki-light:#39ADB5;--shiki-default:#032563;--shiki-dark:#ADDCFF}html pre.shiki code .srGNg,html code.shiki .srGNg{--shiki-light:#91B859;--shiki-default:#032563;--shiki-dark:#ADDCFF}html .light .shiki span{color:var(--shiki-light);background:var(--shiki-light-bg);font-style:var(--shiki-light-font-style);font-weight:var(--shiki-light-font-weight);text-decoration:var(--shiki-light-text-decoration)}html.light .shiki span{color:var(--shiki-light);background:var(--shiki-light-bg);font-style:var(--shiki-light-font-style);font-weight:var(--shiki-light-font-weight);text-decoration:var(--shiki-light-text-decoration)}html .default .shiki span{color:var(--shiki-default);background:var(--shiki-default-bg);font-style:var(--shiki-default-font-style);font-weight:var(--shiki-default-font-weight);text-decoration:var(--shiki-default-text-decoration)}html .shiki span{color:var(--shiki-default);background:var(--shiki-default-bg);font-style:var(--shiki-default-font-style);font-weight:var(--shiki-default-font-weight);text-decoration:var(--shiki-default-text-decoration)}html .dark .shiki span{color:var(--shiki-dark);background:var(--shiki-dark-bg);font-style:var(--shiki-dark-font-style);font-weight:var(--shiki-dark-font-weight);text-decoration:var(--shiki-dark-text-decoration)}html.dark .shiki span{color:var(--shiki-dark);background:var(--shiki-dark-bg);font-style:var(--shiki-dark-font-style);font-weight:var(--shiki-dark-font-weight);text-decoration:var(--shiki-dark-text-decoration)}html pre.shiki code .sOsy2,html code.shiki .sOsy2{--shiki-light:#90A4AE;--shiki-default:#0e1116;--shiki-dark:#f0f3f6}html pre.shiki code .sFDEC,html code.shiki .sFDEC{--shiki-light:#90A4AE80;--shiki-default:#0e111680;--shiki-dark:#f0f3f680}html pre.shiki code .ssIYW,html code.shiki .ssIYW{--shiki-light:#91B859;--shiki-default:#024c1a;--shiki-dark:#26cd4d}html pre.shiki code .sCdEk,html code.shiki .sCdEk{--shiki-light:#39ADB5;--shiki-default:#1b7c83;--shiki-dark:#39c5cf}html pre.shiki code .sDbPz,html code.shiki .sDbPz{--shiki-light:#E53935;--shiki-default:#a0111f;--shiki-dark:#ff9492}html pre.shiki code .smSMi,html code.shiki .smSMi{--shiki-light:#E5393580;--shiki-default:#a0111f80;--shiki-dark:#ff949280}html pre.shiki code .sBsy4,html code.shiki .sBsy4{--shiki-light:#FAFAFA;--shiki-light-bg:#91B859;--shiki-default:#ffffff;--shiki-default-bg:#024c1a;--shiki-dark:#0a0c10;--shiki-dark-bg:#26cd4d}html pre.shiki code .sYBWh,html code.shiki .sYBWh{--shiki-light:#FAFAFA;--shiki-light-bg:#E53935;--shiki-default:#ffffff;--shiki-default-bg:#a0111f;--shiki-dark:#0a0c10;--shiki-dark-bg:#ff9492}html pre.shiki code .spoJ1,html code.shiki .spoJ1{--shiki-light:#E2931D;--shiki-default:#3f2200;--shiki-dark:#f0b72f}html pre.shiki code .sLHsM,html code.shiki .sLHsM{--shiki-light:#90A4AE;--shiki-default:#702C00;--shiki-dark:#FFB757}html pre.shiki code .sCRTB,html code.shiki .sCRTB{--shiki-light:#39ADB5;--shiki-default:#702C00;--shiki-dark:#FFB757}html pre.shiki code .spMcu,html code.shiki .spMcu{--shiki-light:#91B859;--shiki-default:#023B95;--shiki-dark:#91CBFF}html pre.shiki code .sE6rD,html code.shiki .sE6rD{--shiki-light:#39ADB5;--shiki-default:#A0111F;--shiki-dark:#FF9492}html pre.shiki code .szoTG,html code.shiki .szoTG{--shiki-light:#90A4AE;--shiki-light-font-weight:inherit;--shiki-default:#024C1A;--shiki-default-font-weight:bold;--shiki-dark:#72F088;--shiki-dark-font-weight:bold}html pre.shiki code .sTqCK,html code.shiki .sTqCK{--shiki-light:#FF5370;--shiki-default:#023B95;--shiki-dark:#91CBFF}html pre.shiki code .s6g51,html code.shiki .s6g51{--shiki-light:#F76D47;--shiki-default:#023B95;--shiki-dark:#91CBFF}html pre.shiki code .s_gjE,html code.shiki .s_gjE{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#66707B;--shiki-default-font-style:inherit;--shiki-dark:#BDC4CC;--shiki-dark-font-style:inherit}html pre.shiki code .sA8fK,html code.shiki .sA8fK{--shiki-light:#E2931D;--shiki-default:#702C00;--shiki-dark:#FFB757}html pre.shiki code .saWzx,html code.shiki .saWzx{--shiki-light:#E53935;--shiki-default:#024C1A;--shiki-dark:#72F088}html pre.shiki code .sQ79N,html code.shiki .sQ79N{--shiki-light:#90A4AE;--shiki-default:#023B95;--shiki-dark:#91CBFF}html pre.shiki code .s2xgV,html code.shiki .s2xgV{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#702C00;--shiki-default-font-style:inherit;--shiki-dark:#FFB757;--shiki-dark-font-style:inherit}",{"title":106,"searchDepth":144,"depth":144,"links":3874},[3875,3876,3877,3878,3881,3884,3885,3888,3889,3893],{"id":34,"depth":144,"text":35},{"id":86,"depth":144,"text":87},{"id":682,"depth":144,"text":683},{"id":748,"depth":144,"text":749,"children":3879},[3880],{"id":1093,"depth":161,"text":1094},{"id":1363,"depth":144,"text":1364,"children":3882},[3883],{"id":1652,"depth":161,"text":1653},{"id":1877,"depth":144,"text":1878},{"id":2246,"depth":144,"text":2247,"children":3886},[3887],{"id":3099,"depth":161,"text":3100},{"id":3150,"depth":144,"text":3151},{"id":3282,"depth":144,"text":3283,"children":3890},[3891,3892],{"id":3301,"depth":161,"text":3302},{"id":3450,"depth":161,"text":3451},{"id":3828,"depth":144,"text":3829},"\u002Fimages\u002Fposts\u002Frun-parallel-and-serial-tests-in-playwright\u002Frun-parallel-and-serial-tests-in-playwright-cover.webp","2026-09-20","Playwright tests that change global state break parallel test runs. Learn different ways to run serial and parallel tests together without failures.",false,"md",{},"\u002Fsoftware-testing\u002Fframeworks\u002Fplaywright\u002Frun-parallel-and-serial-tests-in-playwright",{"title":5,"description":3896},"software-testing\u002Fframeworks\u002Fplaywright\u002Frun-parallel-and-serial-tests-in-playwright","7UYaA6wuodZYYwHt2TfjNCuO5SPqAQW_G_KY0lObN8s",[3905,4286,4850,6000],{"id":3906,"title":3907,"bmcUsername":6,"body":3908,"cover":4278,"date":4279,"description":4280,"draft":3897,"extension":3898,"features":6,"githubRepo":6,"headline":6,"highlight":6,"icon":6,"meta":4281,"navigation":175,"npmPackage":6,"order":6,"path":4282,"seo":4283,"stem":4284,"__hash__":4285},"content\u002Fsoftware-testing\u002Ftest-automation\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky.md","What Would You Stop Doing When UI Tests Are Flaky?",{"type":8,"value":3909,"toc":4265},[3910,3917,3922,3933,3936,3939,3942,3946,3950,3965,3972,3974,3978,3999,4014,4017,4024,4027,4029,4033,4036,4063,4066,4073,4075,4079,4086,4089,4093,4096,4099,4103,4106,4118,4129,4134,4138,4149,4152,4156,4159,4162,4165,4168,4171,4174,4178,4181,4188,4191,4205,4208,4212,4215,4218,4221,4223,4227,4230,4240,4250,4256,4262],[11,3911,3912,3913,3916],{},"This question ",[723,3914,3915],{},"about"," an interview question was recently posted in a QA forum, and the discussion it generated is more interesting than the question itself:",[1668,3918,3919],{},[11,3920,3921],{},"\"What would you stop doing when UI tests are flaky?\"",[11,3923,3924,3925,3928,3929,3932],{},"The phrasing trips people up. Most interview questions ask what you ",[723,3926,3927],{},"would do",", essentially what's your process, how do you handle it, what tools do you reach for. This one inverts it. It's asking about habits to ",[723,3930,3931],{},"eliminate",", which implies the interviewer already assumes you have them. It's also, perhaps intentionally, phrased awkwardly.",[11,3934,3935],{},"I've spent over 20 years in software testing across fintech, SaaS HCM, and insurtech and currently serve as the Director of Quality Engineering at my current employer. I haven't been asked this question in exactly this phrasing, but I've used similar ones from the other side of the table. I know what this type of question is designed to surface.",[11,3937,3938],{},"Before we get to the answer, let's look at what the QA community said. See if you can guess or click reveal to see all the survey responses.",[3940,3941],"hr",{},[32,3943,3945],{"id":3944},"survey-says-what-the-qa-community-answered-this-interview-question","Survey Says — What the QA Community Answered This Interview Question",[3947,3948],"flaky-test-survey",{":answers":3949},"[{\"text\":\"Stop using sleep() \u002F fix timing and waits\",\"votes\":25,\"keywords\":[\"sleep\",\"pause\",\"timing\",\"wait\",\"thread.sleep\",\"time.sleep\"]},{\"text\":\"Investigate root cause first\",\"votes\":11,\"keywords\":[\"investigate\",\"root cause\",\"diagnose\",\"why\",\"cause\",\"reason\"]},{\"text\":\"Quarantine tests from CI\",\"votes\":3,\"keywords\":[\"quarantine\",\"mute\",\"skip\",\"disable\",\"isolate\"]},{\"text\":\"Stop automating an unstable UI\",\"votes\":2,\"keywords\":[\"unstable\",\"automat\",\"flaky ui\",\"not ready\"]},{\"text\":\"Stop adding more tests\",\"votes\":2,\"keywords\":[\"adding\",\"add test\",\"more test\",\"new test\",\"expand\"]},{\"text\":\"Stop running tests in parallel\",\"votes\":1,\"keywords\":[\"parallel\",\"concurrent\",\"simultaneously\"]}]",[11,3951,3952,3953,3956,3957,3960,3961,3964],{},"The most popular community answers were technical and relatable — ",[723,3954,3955],{},"stop using sleep()",", ",[723,3958,3959],{},"fix timing and waits"," — the instinctive responses from anyone who has spent time debugging intermittent failures. ",[723,3962,3963],{},"Investigate root cause first"," ranked lower by sheer volume but drew the most endorsement from people who paused to think about what was actually being asked.",[11,3966,3967,3968,3971],{},"I also ran a LinkedIn poll with the same question. It had 357 impressions and only 5 votes — low participation — but those 5 voters unanimously chose ",[723,3969,3970],{},"investigate root cause first",". The gap between the free-comment community vote pattern and the forced-choice poll result is itself telling: when people had to commit to one answer, they chose the diagnostic approach. When free-commenting, they led with the most relatable war story.",[3940,3973],{},[32,3975,3977],{"id":3976},"why-most-candidates-answer-the-wrong-question","Why Most Candidates Answer the Wrong Question",[11,3979,3980,3981,3956,3984,3956,3987,3990,3991,3994,3995,3998],{},"Here's what's worth pausing on: many of the most popular community answers — including ",[723,3982,3983],{},"quarantine tests from CI",[723,3985,3986],{},"add retry logic",[723,3988,3989],{},"report flakiness to the dev team"," — are valid responses to \"what would you ",[723,3992,3993],{},"do"," about flaky tests.\" They are not answers to \"what would you ",[723,3996,3997],{},"stop"," doing.\"",[11,4000,4001,4002,4005,4006,4009,4010,4013],{},"Quarantining is an action you ",[723,4003,4004],{},"add"," to your process. Retries are something you ",[723,4007,4008],{},"implement",". Reporting is something you ",[723,4011,4012],{},"start"," doing. None of these are things you stop.",[11,4015,4016],{},"The community's own discussion demonstrated the exact failure mode the question is designed to surface: answering a different question than the one being asked.",[11,4018,4019,4020,4023],{},"This is worth a conscious moment when you're in an interview seat. Before diving in, restate the question: ",[723,4021,4022],{},"\"So you're asking what habits I'd stop — not what I'd add to my process?\""," That one sentence signals precision under pressure, and precision matters.",[11,4025,4026],{},"When I'm conducting an interview, if a candidate is giving an answer that feels off, I'll ask them to repeat back their understanding of the question. Sometimes they're just wrong, but more often they didn't fully process it in the moment due to nerves, language barrier, or, in the case of remote interviews, dropped audio packets. The candidates who handle interviews best are the ones who preemptively restate their understanding before answering. It reads as both confident and careful (good qualities for testers and quality engineers).",[3940,4028],{},[32,4030,4032],{"id":4031},"what-this-flaky-test-interview-question-is-actually-testing","What This Flaky Test Interview Question Is Actually Testing",[11,4034,4035],{},"This question tests at least four things at once:",[4037,4038,4039,4045,4051,4057],"ol",{},[40,4040,4041,4044],{},[43,4042,4043],{},"Technical knowledge"," — Do you know the common anti-patterns that cause flaky UI tests?",[40,4046,4047,4050],{},[43,4048,4049],{},"Diagnostic thinking"," — Can you reason about root causes rather than recite a fix list?",[40,4052,4053,4056],{},[43,4054,4055],{},"Listening comprehension"," — Did you actually process what was asked?",[40,4058,4059,4062],{},[43,4060,4061],{},"Confidence to challenge ambiguity"," — Will the candidate accept the awkwardly worded question or point that out and ask for clarification?",[11,4064,4065],{},"A junior answer names tactics: stop using sleep, fix your waits, add retries. Not wrong, but symptom-level.",[11,4067,4068,4069,4072],{},"An experienced answer narrates a ",[723,4070,4071],{},"thought process"," — how you'd identify what's causing the flakiness before deciding what to change. The \"stop doing\" framing is a clue. It's asking which habits you've already had to unlearn, implying you've operated at enough scale to have learned them the hard way.",[3940,4074],{},[32,4076,4078],{"id":4077},"what-to-stop-doing-when-ui-tests-are-flaky-the-full-answer","What to Stop Doing When UI Tests Are Flaky: The Full Answer",[11,4080,4081,4082,4085],{},"If asked this question in an interview, I'd clarify the framing first: ",[723,4083,4084],{},"\"Are you asking about common anti-patterns that lead to flakiness, or more about how I'd approach the investigation?\""," That distinction matters, and asking it signals diagnostic thinking before the answer even starts.",[11,4087,4088],{},"If they want the approach angle, this is how I'd answer.",[1091,4090,4092],{"id":4091},"stop-adding-tests-to-an-unstable-suite","Stop Adding Tests to an Unstable Suite",[11,4094,4095],{},"This would be my first answer, and I'd lead with it.",[11,4097,4098],{},"Adding tests to a flaky suite compounds the problem. Every new test inherits the instability of the environment it runs in. Before expanding coverage, you need to stop the bleeding and understand whether the flakiness lives in the test code, the application behavior, or the infrastructure. That distinction determines the shape of your fix.",[1091,4100,4102],{"id":4101},"stop-using-sleep-and-pause-statements","Stop Using sleep() and pause Statements",[11,4104,4105],{},"This is the answer that generates the most community agreement, and for good reason — it's the most widespread bad habit in UI test automation.",[11,4107,4108,1343,4111,4114,4115,121],{},[21,4109,4110],{},"sleep()",[21,4112,4113],{},"pause"," are blunt instruments. They wait a fixed amount of time regardless of whether the condition they're waiting for became true a second in or never became true at all. They're slow, brittle, and mask the real problem: the test doesn't know what it's waiting ",[723,4116,4117],{},"for",[11,4119,4120,4121,4124,4125,4128],{},"This is so well understood that Playwright formally marks ",[21,4122,4123],{},"page.waitForTimeout()"," as ",[723,4126,4127],{},"Discouraged"," in their own API docs:",[1668,4130,4131],{},[11,4132,4133],{},"\"Never wait for timeout in production. Tests that wait for time are inherently flaky. Use Locator actions and web assertions that wait automatically.\"",[79,4135],{"href":4136,"text":4137},"https:\u002F\u002Fplaywright.dev\u002Fdocs\u002Fapi\u002Fclass-page#page-wait-for-timeout","Playwright docs — page.waitForTimeout()",[11,4139,4140,4141,4144,4145,4148],{},"I've mandated the removal of pause statements from test suites I've managed and replaced them with explicit wait patterns — ",[21,4142,4143],{},"waitForElementPresent",", custom polling waits — anything that returns as soon as the condition is true rather than waiting out a fixed interval. I've added lint rules to prevent ",[21,4146,4147],{},".pause"," commands from being checked in at all. On one large serial suite, removing sleep and pause statements alone saved over an hour off the total test run time.",[11,4150,4151],{},"One practical detail: when setting a max wait timeout, I set it to roughly twice what I'd expect the worst case to be. CI environments consistently run slower than local development in ways that aren't always predictable. A wait that looks generous locally can time out under CI load.",[1091,4153,4155],{"id":4154},"stop-assuming-the-problem-is-in-the-test-code","Stop Assuming the Problem Is in the Test Code",[11,4157,4158],{},"Some flakiness isn't in the test at all.",[11,4160,4161],{},"I had a test that failed intermittently depending on what time of day the build kicked off. After investigation, the root cause was a timezone mismatch between the server under test and the system running the tests. A validation rule in the application behaved differently at a specific hour because of this offset — the test was faithfully catching real behavior, but it looked like random flakiness until you looked closely enough. The initial investigation was tricky because it would pass during normal business hours when we tried to reproduce the failure in the first place!",[11,4163,4164],{},"The fix was a conditional branch in the test to account for the business rule at that magic hour. I generally avoid conditional branched logic in tests — it adds complexity and makes tests harder to reason about. But we couldn't time-travel or alter system clocks, and the conditional was the honest solution.",[11,4166,4167],{},"The point: before assuming the test is broken, determine whether you're dealing with test code, an application bug, or an infrastructure mismatch. The investigation approach is different for each.",[11,4169,4170],{},"It's also worth noting that some intermittent failures aren't flakiness at all — they're the test catching a real intermittent bug in the application. A test that fails once and passes on the next re-run looks identical to a flaky test on the surface. One is noise; the other is a signal you're about to dismiss. This is why every failure deserves investigation before it gets written off.",[11,4172,4173],{},"The goal is a suite trustworthy enough that the team's first instinct when a test fails is \"it found something\" — not \"ugh, it's flaky, just re-run it.\" The moment re-running becomes the default response, it becomes an annoying car alarm at 3 AM instead of a useful tool.",[1091,4175,4177],{"id":4176},"stop-running-tests-in-parallel-without-isolating-shared-state","Stop Running Tests in Parallel Without Isolating Shared State",[11,4179,4180],{},"Parallelism is worth pursuing — the time savings on a large suite are significant, and it's one of the highest-leverage improvements you can make to CI feedback time. The problem isn't parallelism itself; it's running tests in parallel that were never designed for it.",[11,4182,4183,4184,4187],{},"Tests that share data, database state, or external resources become order-dependent and environment-dependent the moment you parallelize them. A suite that runs cleanly in serial can look deeply flaky in parallel for no obvious reason — because the flakiness is in the ",[723,4185,4186],{},"interaction"," between tests, not in any individual test.",[11,4189,4190],{},"The practical solution is to stop treating your suite as a single homogeneous run and start thinking in terms of what can safely run concurrently:",[37,4192,4193,4199],{},[40,4194,4195,4198],{},[43,4196,4197],{},"Read-only tests"," — tests that only query state without mutating it — are natural candidates for parallel execution. They can't interfere with each other.",[40,4200,4201,4204],{},[43,4202,4203],{},"Write operations, state-dependent flows, and anything touching shared fixtures"," are better kept in a serial suite until you've isolated their data properly (unique test data per run, dedicated test accounts, isolated environments).",[11,4206,4207],{},"A combined approach — a parallel suite for safe tests and a serial suite for the rest — gets you most of the speed benefit while keeping the flakiness surface small. Once the serial tests are properly isolated with their own data, you can graduate them into the parallel suite over time.",[1091,4209,4211],{"id":4210},"stop-treating-flakiness-as-normal","Stop Treating Flakiness as Normal",[11,4213,4214],{},"The most damaging thing a team can do with a flaky test is shrug and accept it.",[11,4216,4217],{},"Flakiness trains everyone to ignore failures. Once the build becomes a noise generator instead of a signal, real regressions slip through unchallenged. A test suite that cries wolf is functionally worse than no test suite, because it creates false confidence.",[11,4219,4220],{},"I've used flakiness scoring in both BitBucket and BrowserStack Test Analytics to identify and mute the worst offenders. Muting is not the same as deleting: the test still runs, it just doesn't fail the build while it's under investigation. That distinction matters — it preserves your ability to track whether improvements helped without letting the instability contaminate every build in the meantime.",[3940,4222],{},[32,4224,4226],{"id":4225},"how-to-answer-flaky-ui-test-interview-questions","How to Answer Flaky UI Test Interview Questions",[11,4228,4229],{},"A few framing notes regardless of how you structure your answer:",[11,4231,4232,4235,4236,4239],{},[43,4233,4234],{},"Restate first."," Before diving in, confirm you understood the question. ",[723,4237,4238],{},"\"So you're asking what habits I'd stop, not what I'd add to my process?\""," One sentence of confirmation demonstrates careful listening — which is arguably what the question is testing most.",[11,4241,4242,4245,4246,4249],{},[43,4243,4244],{},"Narrate, don't list."," A list of tactics sounds like you memorized a checklist. A thought process — ",[723,4247,4248],{},"\"I'd start by determining whether this is test code, application behavior, or environment, because the fix is different for each\""," — sounds like someone who has actually dealt with this at scale.",[11,4251,4252,4255],{},[43,4253,4254],{},"Distinguish the problem type."," Not all flakiness has the same root cause. Timing issues, shared state, environment inconsistency, and automating an unstable UI are four different problems with four different fixes. Showing you can distinguish them is what separates a good answer from a more experienced one.",[11,4257,4258,4261],{},[43,4259,4260],{},"Own a specific example."," The most memorable interview answers are concrete. If you've refactored a suite full of sleep statements, or tracked down a timezone mismatch that looked like random flakiness for weeks, say so. Specific experience is more credible than correct-sounding generalizations.",[3866,4263],{":items":4264},"[\"\u002Fsoftware-testing\u002Ftest-automation\u002Fhow-to-handle-failing-tests-caused-by-known-bugs\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fai-in-testing-2026-state-of-the-industry\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Frun-fewer-tests-catch-the-same-bugs\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fai-performance-testing-guardrails\"]",{"title":106,"searchDepth":144,"depth":144,"links":4266},[4267,4268,4269,4270,4277],{"id":3944,"depth":144,"text":3945},{"id":3976,"depth":144,"text":3977},{"id":4031,"depth":144,"text":4032},{"id":4077,"depth":144,"text":4078,"children":4271},[4272,4273,4274,4275,4276],{"id":4091,"depth":161,"text":4092},{"id":4101,"depth":161,"text":4102},{"id":4154,"depth":161,"text":4155},{"id":4176,"depth":161,"text":4177},{"id":4210,"depth":161,"text":4211},{"id":4225,"depth":144,"text":4226},"\u002Fimages\u002Fposts\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky-cover.webp","2026-05-16","Most QA engineers answer this interview question confidently wrong. Here's what \"What would you stop doing when UI tests are flaky?\" is actually testing and what an experienced answer sounds like.",{},"\u002Fsoftware-testing\u002Ftest-automation\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky",{"title":3907,"description":4280},"software-testing\u002Ftest-automation\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky","J3cN721HjJWyvMOPlZArPiG3YREFE1qMDtp5Lcsm2n4",{"id":4287,"title":4288,"bmcUsername":6,"body":4289,"cover":4842,"date":4843,"description":4844,"draft":3897,"extension":3898,"features":6,"githubRepo":6,"headline":6,"highlight":6,"icon":6,"meta":4845,"navigation":175,"npmPackage":6,"order":6,"path":4846,"seo":4847,"stem":4848,"__hash__":4849},"content\u002Fsoftware-testing\u002Ftest-automation\u002Frun-fewer-tests-catch-the-same-bugs.md","Test Selection: Run Fewer Tests, Catch the Same Bugs",{"type":8,"value":4290,"toc":4830},[4291,4294,4298,4301,4309,4312,4316,4319,4323,4329,4335,4339,4342,4360,4363,4448,4451,4455,4460,4465,4470,4475,4478,4482,4485,4488,4495,4498,4502,4505,4512,4519,4522,4529,4532,4536,4539,4542,4545,4548,4554,4635,4752,4758,4766,4775,4778,4781,4787,4797,4800,4804,4807,4810,4821,4824,4827],[11,4292,4293],{},"An 8+ hour CI feedback loop is the kind of pain that makes any promise of faster tests sound appealing, and at a past job the company adopted a black-box predictive test-selection tool that trained on historical pass and fail data to guess which tests a given change was likely to break. While seemingly effective initially, it surfaced a real problem almost immediately, just not the one we were hoping to fix: our UI suite had hidden dependencies, tests relying on shared, warmed-up page state or on data another test had mutated earlier in the run. Predictive test selection, by not always running things in the suite's original order, introduced flakiness that proved difficult to troubleshoot due to different tests running each build. We were never able to fully resolve it, which never allowed us to realize the benefits of predictive test selection.",[32,4295,4297],{"id":4296},"an-ai-test-selection-framework-that-shows-its-work","An AI Test Selection Framework That Shows Its Work",[11,4299,4300],{},"That experience made me skeptical of any tool promising to safely trim a test suite, especially one that couldn't explain itself. So when I saw a StarEast 2026 session titled \"Dear AI, Which Tests Should We Run Now?\" promising to go deeper into the actual mechanics behind test selection, I made a point of attending. I wanted to know whether the field had advanced since my own bad experience, and whether I could actually learn the theory behind these approaches instead of trusting a vendor's black box to get it right.",[11,4302,4303,4304,4308],{},"The session was presented by ",[79,4305],{"href":4306,"text":4307},"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Felmar-juergens\u002F","Dr. Elmar Jürgens",", co-founder of CQSE GmbH, makers of the TeamScale software-quality-analysis platform, who holds a PhD in software quality analysis himself. His talk covers four distinct approaches to picking a smaller, faster subset of a large test suite that still catches nearly as many bugs as running everything, each with its tradeoffs measured and stated plainly rather than buried inside a product.",[11,4310,4311],{},"What made this talk earn my trust, in a way my prior experience with an opaque tool never did, is that he shows his work. Every approach comes with a measured tradeoff, a named limitation, and an honest account of when it doesn't apply, instead of a black box you're asked to take on faith. He's also explicit that he doesn't care whether a given approach technically counts as AI: \"I care about whether the approaches find more bugs more quickly.\" That's the right question and I appreciated him being up front about it since speakers at these events tend to just rename their presentation from a past year with this year's trending keywords.",[32,4313,4315],{"id":4314},"four-test-selection-approaches","Four Test Selection Approaches",[11,4317,4318],{},"Jürgens builds up to four approaches by crossing two questions. The first is which situation you're in.",[1091,4320,4322],{"id":4321},"question-1-do-you-know-what-changed","Question 1: Do You Know What Changed?",[11,4324,4325,4328],{},[43,4326,4327],{},"Quality Gate"," means you have a large suite and a small time budget, but you don't know what changed in the software since you last ran tests. One of Jürgens' customers, a German pension fund, ran a 20-hour test suite only on weekends because there wasn't time to run it more often. By the time a build reached that gate, days of accumulated commits sat behind it, no single diff to point to, just a build that needed a yes-or-no answer in a fraction of the time the full suite would take.",[11,4330,4331,4334],{},[43,4332,4333],{},"Continuous Integration",", by contrast, means you already know exactly what changed, because you have a diff or a pull request in hand. Dolby, another of Jürgens' customers, dealt with a combinatorial explosion of audio bitrate and sample-rate combinations to test and wanted CI feedback on every pull request within 10 minutes, even though their full suite took 3 to 12 hours to run. Each PR came with a specific, known diff to work from, a very different starting point than the pension fund's accumulated pile of weekend changes.",[1091,4336,4338],{"id":4337},"question-2-coverage-based-or-content-based-test-selection","Question 2: Coverage-Based or Content-Based Test Selection?",[11,4340,4341],{},"The second question is how much you're willing to invest in setup to get a better number. Coverage-based approaches require measuring test-case-specific code coverage first, real upfront effort, but they pay it back with a better result. Content-based approaches skip that setup entirely and work directly from the test and change content, in exchange for a slightly worse number. Jürgens presents this as a real choice inside each situation, not a fixed rule: for Quality Gate, coverage-based Pareto Optimization versus content-based AI Test Clustering; for Continuous Integration, coverage-based Test Impact Analysis versus content-based Similarity Scoring. Two situations, two ways to solve each one.",[4343,4344,4345,4352],"figure",{},[11,4346,4347],{},[4348,4349],"img",{"alt":4350,"src":4351},"Dr. Elmar Jürgens presenting his four test-selection approaches at StarEast 2026","\u002Fimages\u002Fposts\u002Frun-fewer-tests-catch-the-same-bugs\u002Felmar-juergens-stareast-2026-test-selection-presentation.webp",[4353,4354,4359],"figcaption",{"className":4355},[4356,4357,4358],"text-sm","text-muted","mt-2","Jürgens' summary slide: all four approaches, and what percentage of full suite runtime it takes each one to find 90% of the bugs the full suite would find.",[11,4361,4362],{},"Cross those two questions and you get the four approaches, each measured the same way: what percentage of full suite runtime it takes to find 90% of the bugs the full suite would find. The quick reference, then what each one actually does:",[3153,4364,4365,4384],{},[3156,4366,4367],{},[3159,4368,4369,4372,4375,4378,4381],{},[3162,4370,4371],{},"Approach",[3162,4373,4374],{},"Category",[3162,4376,4377],{},"Use Case",[3162,4379,4380],{},"Runtime Required to Find 90% of Bugs",[3162,4382,4383],{},"Setup Effort",[3176,4385,4386,4402,4418,4433],{},[3159,4387,4388,4391,4394,4396,4399],{},[3181,4389,4390],{},"Test Impact Analysis",[3181,4392,4393],{},"Coverage-based",[3181,4395,4333],{},[3181,4397,4398],{},"⭐⭐⭐⭐⭐ (2%)",[3181,4400,4401],{},"🕐🕐 about half a year",[3159,4403,4404,4407,4410,4412,4415],{},[3181,4405,4406],{},"Similarity Scoring",[3181,4408,4409],{},"Content-based",[3181,4411,4333],{},[3181,4413,4414],{},"⭐⭐⭐⭐☆ (4%)",[3181,4416,4417],{},"🕐 about a day",[3159,4419,4420,4423,4425,4427,4430],{},[3181,4421,4422],{},"Pareto Optimization",[3181,4424,4393],{},[3181,4426,4327],{},[3181,4428,4429],{},"⭐⭐⭐☆☆ (11%)",[3181,4431,4432],{},"🕐🕐🕐🕐 about a year",[3159,4434,4435,4438,4440,4442,4445],{},[3181,4436,4437],{},"AI Test Clustering",[3181,4439,4409],{},[3181,4441,4327],{},[3181,4443,4444],{},"⭐⭐☆☆☆ (13%)",[3181,4446,4447],{},"🕐 about two days",[11,4449,4450],{},"Test Impact Analysis (TIA) is the most speed optimized, but Similarity Scoring gets you close in about a day instead of half a year.",[1091,4452,4454],{"id":4453},"predictive-test-selection-approaches-explained","Predictive Test Selection Approaches Explained",[11,4456,4457,4459],{},[43,4458,4422],{}," measures per-test code coverage for the whole suite, then greedily orders tests by how much previously-uncovered code each one covers per second of execution time, cutting off once you hit your time budget. The clearest illustration of why this beats running tests in whatever order they were written came from PixelitOr, an open-source paint program Jürgens uses as a fully visualizable example. Running four UI tests in sequence, Gaussian Blur, Motion Blur, Lens Blur, Smart Blur, each new test lights up progressively less new code and re-covers more of what the earlier blur tests already exercised. His team checked whether that redundant coverage was actually worthless by tracking real and injected bugs: if a bug lives in that repeated code, typically either all of those blur tests find it or none of them do. Running all four buys you almost nothing over running one, and a greedy, coverage-per-second reordering naturally spreads the budget across dissimilar tests instead of exhausting it inside one redundant cluster.",[11,4461,4462,4464],{},[43,4463,4437],{}," gets you a similar diversity-driven selection without needing any coverage data at all. Every test gets represented as a point in a high-dimensional vector space, generated from a large language model's embedding of the test's actual content, its code if it's automated, or even a plain-English Given\u002FWhen\u002FThen description if it's a manual test case. Tests that exercise similar functionality land close together in that space; tests that do genuinely different things land far apart. The selection algorithm then greedily picks whichever remaining test is furthest from everything already chosen, which naturally avoids getting stuck resampling one redundant cluster the way running tests in file order would. Jürgens showed this working on a 3D projection of around 2,000 real customer test cases: the tests visibly clustered into dense groups, and the selected subset landed inside essentially every visible cluster rather than missing whole regions.",[11,4466,4467,4469],{},[43,4468,4390],{}," exploits something the other two approaches don't have access to: you already know exactly what changed, because you have a diff or a PR in hand. Combine that with coverage data recorded from prior test runs, and you can directly compute which historically-recorded tests actually execute the changed lines, then skip everything that provably can't be affected by this specific change. On a real change to Jürgens' own team's codebase, out of 5,000 automated tests, only 4 actually executed the changed code. The other 4,996 categorically could not have caught a bug introduced by that change, because they never touch it.",[11,4471,4472,4474],{},[43,4473,4406],{}," gets a comparable result to Test Impact Analysis without needing coverage data, by treating test selection like a search engine query. Every test's content gets indexed into a document database, the way a search engine indexes web pages. A code change then becomes the search query, built from the changed identifiers plus their surrounding context, not just the raw diff lines. Retrieval works conceptually like TF-IDF (term frequency, inverse document frequency), the same scoring method search engines use: a test scores higher the more it mentions terms that appear in the change, weighted down for terms so common across the whole suite that they don't tell you much. In Jürgens' worked example, a change to a bank transfer method matched correctly against a Cucumber test, a Robot Framework test, and even a plain manual test description, three completely different formats, all identified as relevant through shared identifiers alone.",[11,4476,4477],{},"Counterintuitively, both AI Test Clustering and Similarity Scoring need more suite runtime than their coverage-based counterparts to find 90% of the defects a full run would find. He recommends starting with them anyway, since coverage-based approaches can take months to a year to set up on a large industrial system, while the content-based ones can be running in a day or two.",[1091,4479,4481],{"id":4480},"how-test-gap-analysis-complements-predictive-test-selection","How Test Gap Analysis Complements Predictive Test Selection",[11,4483,4484],{},"All four predictive test selection approaches assume somewhere in your suite a test already exists that's capable of catching a bug in the code being changed. You can't optimize tests that don't exist yet. Jürgens described a related technique, Test Gap Analysis. It works by overlaying two maps: which code changed recently, and which code has ever been executed by any test, unit, integration, manual, anything at all. Wherever those two maps don't overlap, you have changed code with missing test coverage.",[11,4486,4487],{},"He mentioned at one large software company, running this the night before a scheduled release revealed entire multi-year components that had never been touched by a single test. The release was postponed three weeks.",[11,4489,4490,4491,4494],{},"Initially I thought, \"Isn't this what our SonarQube pull request Quality Gates do for us when we set coverage floors?\" Jürgens' version has a broader scope and combines the coverage data from JaCoCo, Istanbul, Jest, etc. with manual and exploratory coverage data as well to get a full picture. Beyond that though, this lets one see that there are changes in code where your testing has blind spots. ",[723,4492,4493],{},"This is where you would want to focus on adding coverage"," because these are areas that may or may not have been working before, but now they've been modified. Without tests you don't know if this introduced a regression in this area or not.",[11,4496,4497],{},"Since uncovered areas may be large, Jürgens recommends a risk-based prioritization pass, tackling the gaps sitting in your highest-domain-risk code first. I think this is a great technique, but getting that full picture is a prerequisite and test-case-specific coverage recorded and persisted across every kind of testing you do is realistically its own months-to-a-year project.",[32,4499,4501],{"id":4500},"the-one-prerequisite-hidden-test-dependencies","The One Prerequisite: Hidden Test Dependencies",[11,4503,4504],{},"I attended the session because this is something we actually tried with a commercial product, Launchable, years ago at a previous employer with inconclusive results. I wanted to see what Jürgens' experience was and what has changed in the space since, especially with advances in AI.",[11,4506,4507,4508,4511],{},"The problem we were trying to solve by trialing Launchable was our extremely long feedback loop. It took 8-10 hours to see our test results after the code was committed. As a result, if a developer introduced a breaking change during their workday they wouldn't know before they left for the day. Launchable predictively selected tests based on our code coverage, drastically reducing the number of tests required to run, but caused the test suite to become unreliable. Tests were ",[723,4509,4510],{},"supposed"," to be written without dependencies on other tests, but over the years they naturally formed dependencies due to innocent mistakes like test engineers assuming certain data was pre-seeded in our baseline where in reality the data was inserted by upstream tests.",[11,4513,4514,4515,4518],{},"With Launchable running different tests each build it would expose these issues. We'd end up fixing the issue, but since the next run may not execute that test again it was hard to confirm the fix organically. Worse, the next run might surface other tests with the same issue. This led to expensive delays trying to get a ",[723,4516,4517],{},"green"," build.",[11,4520,4521],{},"During the Q&A, I described my experience. Jürgens' answer generalized my specific pain point and gave it a name, the test-dependency-chain problem, and it applies to all four approaches to predictive test selection.",[11,4523,4524,4525,4528],{},"If your tests form a hidden sequential chain, test A sets up state that test B needs, which a batch job then depends on as a prerequisite for test C, none of these approaches can safely select \"test C\" in isolation. Jürgens uses the German term ",[723,4526,4527],{},"Testkappen"," for this pattern. His rule is direct: the entire chain has to be treated and selected as a single atomic unit. If your whole system is effectively one long chain end to end, none of these approaches are useful at all, no matter how good the underlying algorithm is.",[11,4530,4531],{},"So the takeaway is you need to verify how interwoven your tests are, and how well they'd handle being reordered or omitted altogether from build to build, before adopting any of these approaches, whether through a commercial tool or your own implementation of what Jürgens described.",[32,4533,4535],{"id":4534},"how-id-implement-similarity-scoring","How I'd Implement Similarity Scoring",[11,4537,4538],{},"I started to research which of these techniques would be the most feasible at my current employer and narrowed it down to content-based approaches: Similarity Scoring or AI Test Clustering. While we have extensive code coverage from our unit tests for some parts of the codebase, large sections are XML-driven and tested through special tooling where we don't have the coverage data we'd need to get started quickly with coverage-based approaches. Mapping the UI tests back through Test Impact Analysis, as mentioned earlier, would take a longer time investment relative to the content-based approaches.",[11,4540,4541],{},"I thought using Claude would quickly get us to AI Test Clustering, but initial research showed Claude doesn't provide the vector embeddings this approach depends on directly, Anthropic points people to third-party providers like Voyage AI for that instead.",[11,4543,4544],{},"Similarity Scoring, by contrast, needs no embedding infrastructure to get started. Its crudest version, literal keyword overlap between a test's content and the identifiers in a change, is plain text matching I could build with a basic search index. It's also easier to inspect and explain than a black box that would trade one set of problems for another, which I consider valuable for a toe-dip investigation. Weighing my options, Similarity Scoring is what I'd start with.",[11,4546,4547],{},"It comes down to two decisions.",[11,4549,4550,4551,224],{},"First, I'd need to decide what counts as a \"test document,\" the thing Similarity Scoring compares a change against. It's not just the test's own code, it's every identifier that test touches: selectors, page-object property names, the API endpoints or service methods it exercises, even indirectly through a fixture or helper. Say a Playwright spec calls a page-object method that, several layers down, calls a service method named ",[21,4552,4553],{},"ProcessRefund",[100,4555,4558],{"className":102,"code":4556,"filename":4557,"language":105,"meta":106,"style":106},"test('customer can request a refund', async ({ page }) => {\n  const checkoutPage = new CheckoutPage(page);\n  await checkoutPage.submitRefundRequest();\n});\n","refund.spec.ts",[21,4559,4560,4588,4612,4627],{"__ignoreMap":106},[110,4561,4562,4564,4566,4568,4571,4573,4575,4577,4579,4581,4584,4586],{"class":112,"line":113},[110,4563,117],{"class":124},[110,4565,128],{"class":116},[110,4567,187],{"class":186},[110,4569,4570],{"class":190},"customer can request a refund",[110,4572,187],{"class":186},[110,4574,196],{"class":120},[110,4576,199],{"class":131},[110,4578,3358],{"class":120},[110,4580,3362],{"class":3361},[110,4582,4583],{"class":120}," })",[110,4585,138],{"class":131},[110,4587,141],{"class":120},[110,4589,4590,4592,4595,4597,4600,4603,4605,4608,4610],{"class":112,"line":144},[110,4591,3383],{"class":131},[110,4593,4594],{"class":2545}," checkoutPage",[110,4596,3389],{"class":884},[110,4598,4599],{"class":884}," new",[110,4601,4602],{"class":124}," CheckoutPage",[110,4604,128],{"class":154},[110,4606,4607],{"class":116},"page",[110,4609,167],{"class":154},[110,4611,158],{"class":120},[110,4613,4614,4616,4618,4620,4623,4625],{"class":112,"line":161},[110,4615,148],{"class":147},[110,4617,4594],{"class":116},[110,4619,121],{"class":120},[110,4621,4622],{"class":124},"submitRefundRequest",[110,4624,155],{"class":154},[110,4626,158],{"class":120},[110,4628,4629,4631,4633],{"class":112,"line":172},[110,4630,164],{"class":120},[110,4632,167],{"class":116},[110,4634,158],{"class":120},[100,4636,4639],{"className":102,"code":4637,"filename":4638,"language":105,"meta":106,"style":106},"export class CheckoutPage {\n  async submitRefundRequest() {\n    await this.page.click('#refund-button');\n    await this.api.post('\u002Frefunds', { handler: 'ProcessRefund' });\n  }\n}\n","checkout-page.ts",[21,4640,4641,4652,4665,4696,4742,4747],{"__ignoreMap":106},[110,4642,4643,4645,4648,4650],{"class":112,"line":113},[110,4644,809],{"class":147},[110,4646,4647],{"class":131}," class",[110,4649,4602],{"class":1246},[110,4651,141],{"class":120},[110,4653,4654,4657,4661,4663],{"class":112,"line":144},[110,4655,4656],{"class":131},"  async",[110,4658,4660],{"class":4659},"stzDA"," submitRefundRequest",[110,4662,155],{"class":120},[110,4664,141],{"class":120},[110,4666,4667,4670,4674,4676,4678,4680,4683,4685,4687,4690,4692,4694],{"class":112,"line":161},[110,4668,4669],{"class":147},"    await",[110,4671,4673],{"class":4672},"sPxkN"," this",[110,4675,121],{"class":120},[110,4677,4607],{"class":116},[110,4679,121],{"class":120},[110,4681,4682],{"class":124},"click",[110,4684,128],{"class":154},[110,4686,187],{"class":186},[110,4688,4689],{"class":190},"#refund-button",[110,4691,187],{"class":186},[110,4693,167],{"class":154},[110,4695,158],{"class":120},[110,4697,4698,4700,4702,4704,4707,4709,4712,4714,4716,4719,4721,4723,4725,4728,4730,4732,4734,4736,4738,4740],{"class":112,"line":172},[110,4699,4669],{"class":147},[110,4701,4673],{"class":4672},[110,4703,121],{"class":120},[110,4705,4706],{"class":116},"api",[110,4708,121],{"class":120},[110,4710,4711],{"class":124},"post",[110,4713,128],{"class":154},[110,4715,187],{"class":186},[110,4717,4718],{"class":190},"\u002Frefunds",[110,4720,187],{"class":186},[110,4722,196],{"class":120},[110,4724,783],{"class":120},[110,4726,4727],{"class":154}," handler",[110,4729,224],{"class":120},[110,4731,227],{"class":186},[110,4733,4553],{"class":190},[110,4735,187],{"class":186},[110,4737,249],{"class":120},[110,4739,167],{"class":154},[110,4741,158],{"class":120},[110,4743,4744],{"class":112,"line":179},[110,4745,4746],{"class":120},"  }\n",[110,4748,4749],{"class":112,"line":208},[110,4750,4751],{"class":120},"}\n",[11,4753,4754,4755,4757],{},"That spec's test document would include ",[21,4756,4553],{},", even though the name never appears in the spec file itself, because the spec's actual behavior depends on it:",[100,4759,4764],{"className":4760,"code":4762,"language":4763},[4761],"language-text","test_document[\"refund.spec.ts\"] = [\n  \"customer can request a refund\",\n  \"CheckoutPage\",\n  \"submitRefundRequest\",\n  \"#refund-button\",\n  \"\u002Frefunds\",\n  \"ProcessRefund\"\n]\n","text",[21,4765,4762],{"__ignoreMap":106},[11,4767,4768,4769,1059,4771,4774],{},"Second, I'd need to decide what counts as a \"query,\" the thing built from an incoming change to search those test documents against. That's one combined query per pull request, built from the diff plus the surrounding context of whatever the changed code calls or is called by, not just the raw changed lines. If a PR renames ",[21,4770,4553],{},[21,4772,4773],{},"ProcessRefundRequest",", a query built from only the two literal diff lines would miss every test that reaches the old name through a helper several layers away. A query built from the diff's surrounding context catches those too.",[11,4776,4777],{},"With both of those defined, the mechanism connecting them is just comparison, not anything more exotic: score every test document against the query, rank the results, and take the highest-scoring tests, or everything above some threshold. The only real decision left is what that scoring function actually is, and that's also where I'd start simple.",[11,4779,4780],{},"As a first pass, before trying TF-IDF (term frequency, inverse document frequency) weighting, I'd try literal keyword overlap to match on, in other words, does the test document contain the changed identifier strings. This would be a quick-to-implement form of Similarity Scoring:",[100,4782,4785],{"className":4783,"code":4784,"language":4763},[4761],"score(test) = count of identifiers shared between test.document and query.identifiers\n\nselected_tests = tests\n    .where(score(test) > 0)\n    .order_by(score, descending)\n",[21,4786,4784],{"__ignoreMap":106},[11,4788,4789,4790,4793,4794,4796],{},"That ranked list still has to turn into something a CI job can actually run. Each test document maps back to a real Playwright spec file, so ",[21,4791,4792],{},"selected_tests"," becomes a list of file paths handed straight to ",[21,4795,3224],{},", the same command any CI job already invokes, just pointed at a smaller, targeted list instead of the whole suite.",[11,4798,4799],{},"Before trusting any of this in a real CI gate, I'd validate it the way Jürgens' own team did: pull a handful of recent pull requests that caused a regression someone caught later, and check whether my selected subset would have included the test that actually caught it. I'd also heavily leverage mutation testing to see how well this approach holds up before trusting it, and follow Jürgens' example of always running test suites that cover critical business functionality.",[32,4801,4803],{"id":4802},"takeaway-what-test-selection-actually-requires","Takeaway: What Test Selection Actually Requires",[11,4805,4806],{},"After attending, I learned that my own experience, predictive test selection working in theory but introducing too much flakiness into the pipeline, wasn't unique to me.",[11,4808,4809],{},"Predictive test selection is not just something that is an academic exercise, but requires:",[4037,4811,4812,4815,4818],{},[40,4813,4814],{},"A healthy test suite without hidden interdependencies between tests",[40,4816,4817],{},"Coverage instrumentation, if you're going the coverage-based route, content-based approaches skip this entirely",[40,4819,4820],{},"Time to invest in building out the ecosystem (or funding to buy it)",[11,4822,4823],{},"Jürgens' session was valuable because it laid out the real pros and cons of each approach, letting you match one to your own tech stack and appetite for investment. Versus a black box, he lays out their weaknesses and trade-offs up front, rather than leaving you to discover where they fall apart mid-evaluation.",[3866,4825],{":items":4826},"[\"\u002Fsoftware-testing\u002Ftest-automation\u002Freuse-playwright-tests-for-load-testing\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fai-performance-testing-guardrails\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fwhat-would-you-stop-doing-when-ui-tests-are-flaky\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-playwright-ai-cost-efficient-testing\"]",[3870,4828,4829],{},"html pre.shiki code .sb1SK,html code.shiki .sb1SK{--shiki-light:#6182B8;--shiki-default:#622CBC;--shiki-dark:#DBB7FF}html pre.shiki code .sZ-rw,html code.shiki .sZ-rw{--shiki-light:#90A4AE;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sZi47,html code.shiki .sZi47{--shiki-light:#39ADB5;--shiki-default:#032563;--shiki-dark:#ADDCFF}html pre.shiki code .srGNg,html code.shiki .srGNg{--shiki-light:#91B859;--shiki-default:#032563;--shiki-dark:#ADDCFF}html pre.shiki code .sPJuK,html code.shiki .sPJuK{--shiki-light:#39ADB5;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .stWsX,html code.shiki .stWsX{--shiki-light:#9C3EDA;--shiki-default:#A0111F;--shiki-dark:#FF9492}html pre.shiki code .s2xgV,html code.shiki .s2xgV{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#702C00;--shiki-default-font-style:inherit;--shiki-dark:#FFB757;--shiki-dark-font-style:inherit}html pre.shiki code .sQ79N,html code.shiki .sQ79N{--shiki-light:#90A4AE;--shiki-default:#023B95;--shiki-dark:#91CBFF}html pre.shiki code .sE6rD,html code.shiki .sE6rD{--shiki-light:#39ADB5;--shiki-default:#A0111F;--shiki-dark:#FF9492}html pre.shiki code .sq0XF,html code.shiki .sq0XF{--shiki-light:#E53935;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sZTni,html code.shiki .sZTni{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#A0111F;--shiki-default-font-style:inherit;--shiki-dark:#FF9492;--shiki-dark-font-style:inherit}html .light .shiki span{color:var(--shiki-light);background:var(--shiki-light-bg);font-style:var(--shiki-light-font-style);font-weight:var(--shiki-light-font-weight);text-decoration:var(--shiki-light-text-decoration)}html.light .shiki span{color:var(--shiki-light);background:var(--shiki-light-bg);font-style:var(--shiki-light-font-style);font-weight:var(--shiki-light-font-weight);text-decoration:var(--shiki-light-text-decoration)}html .default .shiki span{color:var(--shiki-default);background:var(--shiki-default-bg);font-style:var(--shiki-default-font-style);font-weight:var(--shiki-default-font-weight);text-decoration:var(--shiki-default-text-decoration)}html .shiki span{color:var(--shiki-default);background:var(--shiki-default-bg);font-style:var(--shiki-default-font-style);font-weight:var(--shiki-default-font-weight);text-decoration:var(--shiki-default-text-decoration)}html .dark .shiki span{color:var(--shiki-dark);background:var(--shiki-dark-bg);font-style:var(--shiki-dark-font-style);font-weight:var(--shiki-dark-font-weight);text-decoration:var(--shiki-dark-text-decoration)}html.dark .shiki span{color:var(--shiki-dark);background:var(--shiki-dark-bg);font-style:var(--shiki-dark-font-style);font-weight:var(--shiki-dark-font-weight);text-decoration:var(--shiki-dark-text-decoration)}html pre.shiki code .sA8fK,html code.shiki .sA8fK{--shiki-light:#E2931D;--shiki-default:#702C00;--shiki-dark:#FFB757}html pre.shiki code .stzDA,html code.shiki .stzDA{--shiki-light:#E53935;--shiki-default:#622CBC;--shiki-dark:#DBB7FF}html pre.shiki code .sPxkN,html code.shiki .sPxkN{--shiki-light:#39ADB5;--shiki-default:#023B95;--shiki-dark:#91CBFF}",{"title":106,"searchDepth":144,"depth":144,"links":4831},[4832,4833,4839,4840,4841],{"id":4296,"depth":144,"text":4297},{"id":4314,"depth":144,"text":4315,"children":4834},[4835,4836,4837,4838],{"id":4321,"depth":161,"text":4322},{"id":4337,"depth":161,"text":4338},{"id":4453,"depth":161,"text":4454},{"id":4480,"depth":161,"text":4481},{"id":4500,"depth":144,"text":4501},{"id":4534,"depth":144,"text":4535},{"id":4802,"depth":144,"text":4803},"\u002Fimages\u002Fposts\u002Frun-fewer-tests-catch-the-same-bugs\u002Frun-fewer-tests-catch-the-same-bugs-cover.webp","2026-09-05","A framework for running fewer tests without missing real bugs, and the one prerequisite that has to be true first. StarEast 2026 lessons.",{},"\u002Fsoftware-testing\u002Ftest-automation\u002Frun-fewer-tests-catch-the-same-bugs",{"title":4288,"description":4844},"software-testing\u002Ftest-automation\u002Frun-fewer-tests-catch-the-same-bugs","dx_k1CWyKJGJnlLB7rFIeHbcWe6GZ2YCaZlDXBjGkz8",{"id":4851,"title":4852,"bmcUsername":6,"body":4853,"cover":5992,"date":5993,"description":5994,"draft":3897,"extension":3898,"features":6,"githubRepo":6,"headline":6,"highlight":6,"icon":6,"meta":5995,"navigation":175,"npmPackage":6,"order":6,"path":5996,"seo":5997,"stem":5998,"__hash__":5999},"content\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-playwright-ai-cost-efficient-testing.md","Playwright AI Testing on a Budget: Locators vs. Computer Vision — StarEast 2026",{"type":8,"value":4854,"toc":5984},[4855,4858,4879,4882,4888,4892,4895,4898,4903,4906,4909,4914,4921,4925,4932,4938,4945,4948,5672,5687,5691,5694,5697,5710,5715,5718,5724,5729,5762,5773,5776,5779,5799,5805,5812,5819,5824,5827,5859,5862,5867,5870,5874,5881,5886,5889,5894,5897,5902,5909,5913,5916,5921,5924,5927,5934,5937,5941,5944,5970,5978,5981],[11,4856,4857],{},"Andy Knight's half-day StarEast 2026 tutorial, officially titled \"Top-Notch Web Testing with Playwright and AI,\" was billed as a hands-on walkthrough, and for most of its four hours, that's exactly what it was. Two claims kept it from being just another how-to for me. Playwright's MCP server can burn through an AI testing budget fast enough to matter (one joke about a junior developer's $5,000 month illustrated that point), and computer vision based testing, despite what a different StarEast tutorial argued the day before, is unlikely to replace locator-based Playwright tests anytime soon.",[11,4859,4860,4861,4865,4866,1343,4870,4874,4875,4878],{},"Knight, who goes by Pandy or Automation Panda depending on which corner of the testing internet you found him in, is an actual ",[79,4862],{"href":4863,"text":4864},"https:\u002F\u002Fautomationpanda.com","Playwright Ambassador",". His session was the third of four StarEast 2026 tutorials I attended over two days, the first two are their own write-ups, on ",[46,4867,4869],{"href":4868},"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-getting-started-ai-driven-automation","getting started with AI-driven automation and AI vision testing",[46,4871,4873],{"href":4872},"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-getting-dirty-ai-testing","evals, vibe coding, and prompt engineering",". Knight acknowledged near the end that the class hadn't gotten through the whole tutorial repository live, \"we only got through about half of what's in the tutorial repository.\" Part of that had a funny explanation: Knight assumed most of the class had simply ignored the prerequisite machine setup instructions he'd sent out ahead of time. It turned out the StarEast organizers never actually emailed those instructions to anyone. So the room spent a chunk of class scrambling to install several hundred megabytes of Playwright's browser dependencies over the now-saturated conference Wi-Fi. The organizers only figured out what happened when they noticed the network anomaly and mentioned it to Knight, at which point I felt vindicated, I'd been ",[723,4876,4877],{},"certain"," no such instructions were ever sent and had assumed I'd just failed to do my homework.",[11,4880,4881],{},"Everything below is what we actually built and discussed in the room, plus what I read in his written tutorial chapters afterward to fill in gaps.",[11,4883,4884],{},[4348,4885],{"alt":4886,"src":4887},"Andy Knight presenting his Playwright and AI tutorial at StarEast 2026","\u002Fimages\u002Fposts\u002Fstareast-2026-playwright-ai-cost-efficient-testing\u002Fandy-knight-stareast-2026.webp",[32,4889,4891],{"id":4890},"playwright-vs-selenium-what-actually-got-fixed","Playwright vs. Selenium: What Actually Got Fixed",[11,4893,4894],{},"Knight opened by asking the room what makes test automation hard, and the answers came fast: tests are slow, brittle, flaky, don't make sense when you read them back, don't make money (a real line, \"we're not shipping tests to customers\"), and force a context switch every time you flip from building a feature to testing it.",[11,4896,4897],{},"The classic fix for this was the Testing Pyramid, lots of cheap unit tests at the base, fewer expensive UI tests at the top, because UI tests were \"big, slow, and expensive.\" Knight's pushback wasn't that the pyramid's diagnosis was wrong. It was that the diagnosis got blamed on the wrong cause:",[1668,4899,4900],{},[11,4901,4902],{},"\"End-to-end tests can be very valuable. Unfortunately, the Testing Pyramid labeled them as 'difficult' and 'bad' primarily due to poor practices and tool shortcomings.\"",[11,4904,4905],{},"He had a punchier name for what should replace pyramid-style thinking (\"we don't build pyramids anymore, we build skyscrapers\"). We'll revisit that line in a later section because I don't think it holds up quite as cleanly as it sounded in the room at the time.",[11,4907,4908],{},"What does hold up is the tooling argument. Playwright's actual fix for \"UI tests are slow and flaky\" is architectural: one browser instance per worker, with each test pulling its own isolated browser context out of that instance (\"akin to an incognito session, or a mini container in your browser\"), and each context holding one or more pages. Spinning up a context is nearly instant, which is the opposite of Selenium's per-test full-browser-relaunch model. Knight's own story below, about discovering this, resonated with me because I had a similar reaction when using Playwright for the first time.",[1668,4910,4911],{},[11,4912,4913],{},"\"I remember the first time I used Playwright, this was back in late 2021... I quickly bang out about a dozen tests or so... I go to the terminal, I'm like npx Playwright test, run it, hit it, and then within a second it comes back and it says 12 tests passed. And I'm like, no, no, no, no, no, it didn't find the tests, it didn't run the tests, it skipped it, something went wrong... then I run it in headed mode, and it was so fast... I was expecting each test to take about a minute, because I came from Selenium, but it's like when I say it's freaky fast man, it is, it screams.\"",[11,4915,4916,4917,4920],{},"Playwright avoids the behavior that gives Selenium its flaky reputation by, among other things, polling automatically: locators and assertions keep rechecking until they succeed or time out, instead of failing the instant they're called, if misaligned. Selenium does the opposite by default, checking once, so a test that forgets to include explicit waits fails the moment the page hasn't caught up yet. Playwright's defaults give that polling a generous window: locator actions retry for 30 seconds, ",[21,4918,4919],{},"expect"," assertions for 5, enough slack to absorb a slower page load between runs without anyone configuring a thing. Knight was fair to say, \"Selenium itself is not flaky, it's the tests that people write with it.\" Playwright's real contribution is removing a specific set of execution-speed and tooling-friction problems that made E2E testing painful for the last decade, not inventing testing concepts from scratch.",[32,4922,4924],{"id":4923},"from-codegen-to-a-real-test","From Codegen to a Real Test",[11,4926,4927,4928,4931],{},"The hands-on portion started with ",[21,4929,4930],{},"npx playwright codegen"," against a local Trello-style Kanban app (a clone built by Filip Hric, used with permission). Codegen records your clicks and fills into a script, and the output is rough on purpose, Knight's framing: \"there's a difference between a script and a test case... we can use this to ruthlessly refine it into a better test case.\"",[11,4933,4934],{},[4348,4935],{"alt":4936,"src":4937},"Trello app being tested","\u002Fimages\u002Fposts\u002Fstareast-2026-playwright-ai-cost-efficient-testing\u002Ftrello-app-under-test-listview.webp",[11,4939,4940,4941,4944],{},"Refining it meant three things: trimming the clicks codegen over-records (you don't need to click an input before typing into it), picking stable locators (",[21,4942,4943],{},"data-testid"," attributes if you control the app, \"these are very nice test hooks to have\"), and adding the assertions codegen never gives you, since codegen only captures interactions, not verifications.",[11,4946,4947],{},"We iterated from the raw click events through refining the flow so it could be run repeatedly by adding things like pre and post test hooks to ensure the test launches in the correct state and doesn't leave behind past entries that would cause different state between runs. Here's my own rough version of that test, written live in the room.:",[100,4949,4954],{"className":4950,"code":4951,"filename":4952,"language":4953,"meta":106,"style":106},"language-typescript shiki shiki-themes material-theme-lighter github-light-high-contrast github-dark-high-contrast","import { test, expect } from '@playwright\u002Ftest';\n\ntest.beforeEach(async ({ page, request }) => {\n  \u002F\u002F Added this reset endpoint to erase the board and then naivate to the app at the start of each test run\n  await request.post('http:\u002F\u002Flocalhost:3000\u002Fapi\u002Freset');\n  await page.goto('http:\u002F\u002Flocalhost:3000\u002F');\n});\n\ntest.afterEach(async ({ request }) => {\n  \u002F\u002F Added this explicit reset after each test to erase the board (belt and suspenders with the beforeEach's erase)\n  await request.post('http:\u002F\u002Flocalhost:3000\u002Fapi\u002Freset');\n});\n\ntest.afterAll(async ({ browser }) => {\n  \u002F\u002F Added to close down the browser after all the tests complete\n  await browser.close();\n});\n\ntest('Create a new board with list and cards', async ({ page }) => {\n  \u002F\u002F You'll notice the selector repetition and lack of page objects which we didn't get to during the session \u002F wasn't a primary focus\n  await page.getByTestId('first-board').click();\n  await page.getByTestId('first-board').fill('chores');\n  await page.getByTestId('first-board').press('Enter');\n\n  expect(page.getByTestId('first-board')).toHaveValue('chores');\n\n  await page.getByTestId('add-list-input').click();\n  await page.getByTestId('add-list-input').fill('todo');\n  await page.getByRole('button', { name: 'Add list' }).click();\n  await page.getByTestId('new-card').click();\n  await page.getByTestId('new-card-input').fill('walk the dog');\n  await page.getByTestId('new-card-input').click();\n  await page.getByTestId('new-card-input').fill('mow the lawn');\n  await page.getByTestId('home').click();\n\n  \u002F\u002F Didn't have a chance to add more assertions, was helping classmates with setup.\n});\n","trello.spec.ts","typescript",[21,4955,4956,4981,4985,5013,5018,5041,5065,5073,5077,5099,5104,5126,5134,5138,5162,5167,5182,5190,5194,5221,5226,5256,5294,5332,5336,5376,5380,5409,5446,5493,5522,5560,5588,5625,5654,5658,5663],{"__ignoreMap":106},[110,4957,4958,4960,4962,4964,4966,4969,4971,4973,4975,4977,4979],{"class":112,"line":113},[110,4959,780],{"class":147},[110,4961,783],{"class":120},[110,4963,2203],{"class":116},[110,4965,196],{"class":120},[110,4967,4968],{"class":116}," expect",[110,4970,249],{"class":120},[110,4972,791],{"class":147},[110,4974,227],{"class":186},[110,4976,796],{"class":190},[110,4978,187],{"class":186},[110,4980,158],{"class":120},[110,4982,4983],{"class":112,"line":144},[110,4984,176],{"emptyLinePlaceholder":175},[110,4986,4987,4989,4991,4994,4996,4998,5000,5002,5004,5007,5009,5011],{"class":112,"line":161},[110,4988,117],{"class":116},[110,4990,121],{"class":120},[110,4992,4993],{"class":124},"beforeEach",[110,4995,128],{"class":116},[110,4997,132],{"class":131},[110,4999,3358],{"class":120},[110,5001,3362],{"class":3361},[110,5003,196],{"class":120},[110,5005,5006],{"class":3361}," request",[110,5008,4583],{"class":120},[110,5010,138],{"class":131},[110,5012,141],{"class":120},[110,5014,5015],{"class":112,"line":172},[110,5016,5017],{"class":1011},"  \u002F\u002F Added this reset endpoint to erase the board and then naivate to the app at the start of each test run\n",[110,5019,5020,5022,5024,5026,5028,5030,5032,5035,5037,5039],{"class":112,"line":179},[110,5021,148],{"class":147},[110,5023,5006],{"class":116},[110,5025,121],{"class":120},[110,5027,4711],{"class":124},[110,5029,128],{"class":154},[110,5031,187],{"class":186},[110,5033,5034],{"class":190},"http:\u002F\u002Flocalhost:3000\u002Fapi\u002Freset",[110,5036,187],{"class":186},[110,5038,167],{"class":154},[110,5040,158],{"class":120},[110,5042,5043,5045,5047,5049,5052,5054,5056,5059,5061,5063],{"class":112,"line":208},[110,5044,148],{"class":147},[110,5046,3362],{"class":116},[110,5048,121],{"class":120},[110,5050,5051],{"class":124},"goto",[110,5053,128],{"class":154},[110,5055,187],{"class":186},[110,5057,5058],{"class":190},"http:\u002F\u002Flocalhost:3000\u002F",[110,5060,187],{"class":186},[110,5062,167],{"class":154},[110,5064,158],{"class":120},[110,5066,5067,5069,5071],{"class":112,"line":256},[110,5068,164],{"class":120},[110,5070,167],{"class":116},[110,5072,158],{"class":120},[110,5074,5075],{"class":112,"line":298},[110,5076,176],{"emptyLinePlaceholder":175},[110,5078,5079,5081,5083,5085,5087,5089,5091,5093,5095,5097],{"class":112,"line":376},[110,5080,117],{"class":116},[110,5082,121],{"class":120},[110,5084,125],{"class":124},[110,5086,128],{"class":116},[110,5088,132],{"class":131},[110,5090,3358],{"class":120},[110,5092,5006],{"class":3361},[110,5094,4583],{"class":120},[110,5096,138],{"class":131},[110,5098,141],{"class":120},[110,5100,5101],{"class":112,"line":392},[110,5102,5103],{"class":1011},"  \u002F\u002F Added this explicit reset after each test to erase the board (belt and suspenders with the beforeEach's erase)\n",[110,5105,5106,5108,5110,5112,5114,5116,5118,5120,5122,5124],{"class":112,"line":405},[110,5107,148],{"class":147},[110,5109,5006],{"class":116},[110,5111,121],{"class":120},[110,5113,4711],{"class":124},[110,5115,128],{"class":154},[110,5117,187],{"class":186},[110,5119,5034],{"class":190},[110,5121,187],{"class":186},[110,5123,167],{"class":154},[110,5125,158],{"class":120},[110,5127,5128,5130,5132],{"class":112,"line":419},[110,5129,164],{"class":120},[110,5131,167],{"class":116},[110,5133,158],{"class":120},[110,5135,5136],{"class":112,"line":433},[110,5137,176],{"emptyLinePlaceholder":175},[110,5139,5140,5142,5144,5147,5149,5151,5153,5156,5158,5160],{"class":112,"line":446},[110,5141,117],{"class":116},[110,5143,121],{"class":120},[110,5145,5146],{"class":124},"afterAll",[110,5148,128],{"class":116},[110,5150,132],{"class":131},[110,5152,3358],{"class":120},[110,5154,5155],{"class":3361}," browser",[110,5157,4583],{"class":120},[110,5159,138],{"class":131},[110,5161,141],{"class":120},[110,5163,5164],{"class":112,"line":460},[110,5165,5166],{"class":1011},"  \u002F\u002F Added to close down the browser after all the tests complete\n",[110,5168,5169,5171,5173,5175,5178,5180],{"class":112,"line":476},[110,5170,148],{"class":147},[110,5172,5155],{"class":116},[110,5174,121],{"class":120},[110,5176,5177],{"class":124},"close",[110,5179,155],{"class":154},[110,5181,158],{"class":120},[110,5183,5184,5186,5188],{"class":112,"line":489},[110,5185,164],{"class":120},[110,5187,167],{"class":116},[110,5189,158],{"class":120},[110,5191,5192],{"class":112,"line":503},[110,5193,176],{"emptyLinePlaceholder":175},[110,5195,5196,5198,5200,5202,5205,5207,5209,5211,5213,5215,5217,5219],{"class":112,"line":517},[110,5197,117],{"class":124},[110,5199,128],{"class":116},[110,5201,187],{"class":186},[110,5203,5204],{"class":190},"Create a new board with list and cards",[110,5206,187],{"class":186},[110,5208,196],{"class":120},[110,5210,199],{"class":131},[110,5212,3358],{"class":120},[110,5214,3362],{"class":3361},[110,5216,4583],{"class":120},[110,5218,138],{"class":131},[110,5220,141],{"class":120},[110,5222,5223],{"class":112,"line":531},[110,5224,5225],{"class":1011},"  \u002F\u002F You'll notice the selector repetition and lack of page objects which we didn't get to during the session \u002F wasn't a primary focus\n",[110,5227,5228,5230,5232,5234,5237,5239,5241,5244,5246,5248,5250,5252,5254],{"class":112,"line":536},[110,5229,148],{"class":147},[110,5231,3362],{"class":116},[110,5233,121],{"class":120},[110,5235,5236],{"class":124},"getByTestId",[110,5238,128],{"class":154},[110,5240,187],{"class":186},[110,5242,5243],{"class":190},"first-board",[110,5245,187],{"class":186},[110,5247,167],{"class":154},[110,5249,121],{"class":120},[110,5251,4682],{"class":124},[110,5253,155],{"class":154},[110,5255,158],{"class":120},[110,5257,5258,5260,5262,5264,5266,5268,5270,5272,5274,5276,5278,5281,5283,5285,5288,5290,5292],{"class":112,"line":541},[110,5259,148],{"class":147},[110,5261,3362],{"class":116},[110,5263,121],{"class":120},[110,5265,5236],{"class":124},[110,5267,128],{"class":154},[110,5269,187],{"class":186},[110,5271,5243],{"class":190},[110,5273,187],{"class":186},[110,5275,167],{"class":154},[110,5277,121],{"class":120},[110,5279,5280],{"class":124},"fill",[110,5282,128],{"class":154},[110,5284,187],{"class":186},[110,5286,5287],{"class":190},"chores",[110,5289,187],{"class":186},[110,5291,167],{"class":154},[110,5293,158],{"class":120},[110,5295,5296,5298,5300,5302,5304,5306,5308,5310,5312,5314,5316,5319,5321,5323,5326,5328,5330],{"class":112,"line":551},[110,5297,148],{"class":147},[110,5299,3362],{"class":116},[110,5301,121],{"class":120},[110,5303,5236],{"class":124},[110,5305,128],{"class":154},[110,5307,187],{"class":186},[110,5309,5243],{"class":190},[110,5311,187],{"class":186},[110,5313,167],{"class":154},[110,5315,121],{"class":120},[110,5317,5318],{"class":124},"press",[110,5320,128],{"class":154},[110,5322,187],{"class":186},[110,5324,5325],{"class":190},"Enter",[110,5327,187],{"class":186},[110,5329,167],{"class":154},[110,5331,158],{"class":120},[110,5333,5334],{"class":112,"line":556},[110,5335,176],{"emptyLinePlaceholder":175},[110,5337,5338,5340,5342,5344,5346,5348,5350,5352,5354,5356,5359,5361,5364,5366,5368,5370,5372,5374],{"class":112,"line":581},[110,5339,259],{"class":124},[110,5341,128],{"class":154},[110,5343,4607],{"class":116},[110,5345,121],{"class":120},[110,5347,5236],{"class":124},[110,5349,128],{"class":154},[110,5351,187],{"class":186},[110,5353,5243],{"class":190},[110,5355,187],{"class":186},[110,5357,5358],{"class":154},"))",[110,5360,121],{"class":120},[110,5362,5363],{"class":124},"toHaveValue",[110,5365,128],{"class":154},[110,5367,187],{"class":186},[110,5369,5287],{"class":190},[110,5371,187],{"class":186},[110,5373,167],{"class":154},[110,5375,158],{"class":120},[110,5377,5378],{"class":112,"line":586},[110,5379,176],{"emptyLinePlaceholder":175},[110,5381,5382,5384,5386,5388,5390,5392,5394,5397,5399,5401,5403,5405,5407],{"class":112,"line":602},[110,5383,148],{"class":147},[110,5385,3362],{"class":116},[110,5387,121],{"class":120},[110,5389,5236],{"class":124},[110,5391,128],{"class":154},[110,5393,187],{"class":186},[110,5395,5396],{"class":190},"add-list-input",[110,5398,187],{"class":186},[110,5400,167],{"class":154},[110,5402,121],{"class":120},[110,5404,4682],{"class":124},[110,5406,155],{"class":154},[110,5408,158],{"class":120},[110,5410,5411,5413,5415,5417,5419,5421,5423,5425,5427,5429,5431,5433,5435,5437,5440,5442,5444],{"class":112,"line":616},[110,5412,148],{"class":147},[110,5414,3362],{"class":116},[110,5416,121],{"class":120},[110,5418,5236],{"class":124},[110,5420,128],{"class":154},[110,5422,187],{"class":186},[110,5424,5396],{"class":190},[110,5426,187],{"class":186},[110,5428,167],{"class":154},[110,5430,121],{"class":120},[110,5432,5280],{"class":124},[110,5434,128],{"class":154},[110,5436,187],{"class":186},[110,5438,5439],{"class":190},"todo",[110,5441,187],{"class":186},[110,5443,167],{"class":154},[110,5445,158],{"class":120},[110,5447,5448,5450,5452,5454,5457,5459,5461,5464,5466,5468,5470,5472,5474,5476,5479,5481,5483,5485,5487,5489,5491],{"class":112,"line":621},[110,5449,148],{"class":147},[110,5451,3362],{"class":116},[110,5453,121],{"class":120},[110,5455,5456],{"class":124},"getByRole",[110,5458,128],{"class":154},[110,5460,187],{"class":186},[110,5462,5463],{"class":190},"button",[110,5465,187],{"class":186},[110,5467,196],{"class":120},[110,5469,783],{"class":120},[110,5471,1258],{"class":154},[110,5473,224],{"class":120},[110,5475,227],{"class":186},[110,5477,5478],{"class":190},"Add list",[110,5480,187],{"class":186},[110,5482,249],{"class":120},[110,5484,167],{"class":154},[110,5486,121],{"class":120},[110,5488,4682],{"class":124},[110,5490,155],{"class":154},[110,5492,158],{"class":120},[110,5494,5495,5497,5499,5501,5503,5505,5507,5510,5512,5514,5516,5518,5520],{"class":112,"line":627},[110,5496,148],{"class":147},[110,5498,3362],{"class":116},[110,5500,121],{"class":120},[110,5502,5236],{"class":124},[110,5504,128],{"class":154},[110,5506,187],{"class":186},[110,5508,5509],{"class":190},"new-card",[110,5511,187],{"class":186},[110,5513,167],{"class":154},[110,5515,121],{"class":120},[110,5517,4682],{"class":124},[110,5519,155],{"class":154},[110,5521,158],{"class":120},[110,5523,5524,5526,5528,5530,5532,5534,5536,5539,5541,5543,5545,5547,5549,5551,5554,5556,5558],{"class":112,"line":632},[110,5525,148],{"class":147},[110,5527,3362],{"class":116},[110,5529,121],{"class":120},[110,5531,5236],{"class":124},[110,5533,128],{"class":154},[110,5535,187],{"class":186},[110,5537,5538],{"class":190},"new-card-input",[110,5540,187],{"class":186},[110,5542,167],{"class":154},[110,5544,121],{"class":120},[110,5546,5280],{"class":124},[110,5548,128],{"class":154},[110,5550,187],{"class":186},[110,5552,5553],{"class":190},"walk the dog",[110,5555,187],{"class":186},[110,5557,167],{"class":154},[110,5559,158],{"class":120},[110,5561,5562,5564,5566,5568,5570,5572,5574,5576,5578,5580,5582,5584,5586],{"class":112,"line":638},[110,5563,148],{"class":147},[110,5565,3362],{"class":116},[110,5567,121],{"class":120},[110,5569,5236],{"class":124},[110,5571,128],{"class":154},[110,5573,187],{"class":186},[110,5575,5538],{"class":190},[110,5577,187],{"class":186},[110,5579,167],{"class":154},[110,5581,121],{"class":120},[110,5583,4682],{"class":124},[110,5585,155],{"class":154},[110,5587,158],{"class":120},[110,5589,5590,5592,5594,5596,5598,5600,5602,5604,5606,5608,5610,5612,5614,5616,5619,5621,5623],{"class":112,"line":647},[110,5591,148],{"class":147},[110,5593,3362],{"class":116},[110,5595,121],{"class":120},[110,5597,5236],{"class":124},[110,5599,128],{"class":154},[110,5601,187],{"class":186},[110,5603,5538],{"class":190},[110,5605,187],{"class":186},[110,5607,167],{"class":154},[110,5609,121],{"class":120},[110,5611,5280],{"class":124},[110,5613,128],{"class":154},[110,5615,187],{"class":186},[110,5617,5618],{"class":190},"mow the lawn",[110,5620,187],{"class":186},[110,5622,167],{"class":154},[110,5624,158],{"class":120},[110,5626,5627,5629,5631,5633,5635,5637,5639,5642,5644,5646,5648,5650,5652],{"class":112,"line":655},[110,5628,148],{"class":147},[110,5630,3362],{"class":116},[110,5632,121],{"class":120},[110,5634,5236],{"class":124},[110,5636,128],{"class":154},[110,5638,187],{"class":186},[110,5640,5641],{"class":190},"home",[110,5643,187],{"class":186},[110,5645,167],{"class":154},[110,5647,121],{"class":120},[110,5649,4682],{"class":124},[110,5651,155],{"class":154},[110,5653,158],{"class":120},[110,5655,5656],{"class":112,"line":663},[110,5657,176],{"emptyLinePlaceholder":175},[110,5659,5660],{"class":112,"line":670},[110,5661,5662],{"class":1011},"  \u002F\u002F Didn't have a chance to add more assertions, was helping classmates with setup.\n",[110,5664,5666,5668,5670],{"class":112,"line":5665},37,[110,5667,164],{"class":120},[110,5669,167],{"class":116},[110,5671,158],{"class":120},[11,5673,5674,5675,5678,5679,5682,5683,5686],{},"Test data was the other rough edge. The app resets its entire backend through a ",[21,5676,5677],{},"\u002Fapi\u002Freset"," endpoint, called via Playwright's ",[21,5680,5681],{},"request"," fixture, and Knight was explicit that this was a deliberate, temporary shortcut: \"Remember, this is a tutorial, friends. Don't do this for real... Do not say automation panda told me to drop my whole database as test setup. No, he did not.\" The honest cost of that shortcut showed up immediately: resetting the whole database before every test means tests can't run in parallel, so the class was capped at ",[21,5684,5685],{},"--workers 1"," for the rest of the session. Fixing that properly (per-test data instead of a global wipe) is exactly the kind of thing that's covered in the tutorial's later, unreached chapters, more on that near the end of this article.",[32,5688,5690],{"id":5689},"the-efficient-ai-workflow-playwright-cli-vs-mcp","The Efficient AI Workflow: Playwright CLI vs. MCP",[11,5692,5693],{},"Coming into this session, I'd already absorbed the soundbite that Playwright's CLI is more token-efficient than its MCP server, but nobody had explained why, and I had a more basic confusion sitting underneath that one: the CLI is just terminal commands, so in what sense is that even \"AI\"? Knight's session got me most of the way to an answer. It didn't fully click until I went and read more on my own afterward.",[11,5695,5696],{},"Once the manual test was working, Knight pivoted to AI, with an important framing up front: \"Playwright doesn't bring its own model, it doesn't bring its own magic. Basically what it does is it brings tooling to integrate into existing AI coding agents.\" You still need Claude, Cursor, Copilot, or Codex. Playwright gives that agent two different ways to actually drive a browser.",[11,5698,5699,5702,5703,1343,5706,5709],{},[43,5700,5701],{},"MCP"," (Model Context Protocol) exposes structured tools like ",[21,5704,5705],{},"browser_navigate",[21,5707,5708],{},"browser_snapshot"," to your coding agent. It works well, and it's expensive. Knight's framing of why, in full:",[1668,5711,5712],{},[11,5713,5714],{},"\"There's a problem with MCP. Does anybody know the problem with MCP? Burns a lot of tokens. It burns a heckin' ton of tokens... Intelligence is a utility. You pay a power bill, you pay a water bill. Guess what we're all paying for next? An intelligence bill.\"",[11,5716,5717],{},"The joke that opened this article followed directly: a junior developer who ran up a $5,000 month using MCP without understanding the cost. The mechanism, explained later in the session, isn't about which model you use, it's that MCP's tool schemas and structured page snapshots eat far more context window per step than a plain terminal command does, which forces more turns, which burns more tokens.",[11,5719,5720,5723],{},[43,5721,5722],{},"Playwright's CLI"," does the same browser-driving job as MCP, as plain terminal commands instead of structured tool calls, and according to Knight, \"uses a tenth of the tokens.\" His actual decision rule, given directly in response to \"why would you ever use MCP if the CLI is so much cheaper\":",[1668,5725,5726],{},[11,5727,5728],{},"\"The CLI is really good if you are doing the workflow that we are doing, for test developers, for grinding out some code, with coding agents CLI is better. But let's say that you wanted a more agentic workflow that wasn't you coding. Let's say you had to use Playwright as a browser automation tool in some way, writing a web scraper or web browser. In those cases the MCP is going to be better than the CLI. Because the MCP can be hosted on a network that you can reach out to it back and forth. CLI is all local to your machine.\"",[11,5730,5731,5732,5735,5736,5739,5740,3956,5742,5745,5746,5749,5750,5753,5754,5757,5758,5761],{},"Here's the part that actually answered both of my questions, the AI-or-not question and the why-tokens question, together. Both MCP and the CLI are AI-driven, in both cases the coding agent itself is deciding what to do and reading the result back. ",[43,5733,5734],{},"The difference is just what vocabulary it uses to act."," MCP issues ",[43,5737,5738],{},"structured tool calls"," (",[21,5741,5705],{},[21,5743,5744],{},"browser_click",") over a protocol built on JSON-RPC, so the call and its ",[43,5747,5748],{},"full response travel through the model's context every time",". The ",[43,5751,5752],{},"CLI"," has the agent run ",[43,5755,5756],{},"literal shell commands"," against itself, something like ",[21,5759,5760],{},"playwright-cli click e21",", the same way it would run any other terminal command in a coding session.",[11,5763,5764,5765,5768,5769,5772],{},"That's also where the token savings actually come from. ",[43,5766,5767],{},"MCP has to keep the page's structure resident in the session's context for as long as the agent is working with it."," The CLI's skills are markdown files sitting on disk, ",[43,5770,5771],{},"read in only when something needs them",", then left there. One holds everything it might need in memory the whole time. The other fetches what it needs and sets it back down.",[11,5774,5775],{},"That also sharpens Knight's own rule (local machine versus network-hosted) into something more concrete. The CLI needs a real terminal, a filesystem, and the ability to spawn its own processes, exactly what you have during local development, and exactly what you don't have everywhere else. MCP doesn't need any of that, which is why it's the better fit in more locked-down or remote contexts: AI-assisted CI failure triage running inside a pipeline with no terminal session attached, for instance, or a low-code product where an agent runs server-side and a non-technical user just describes a test case in plain English, with no shell ever exposed to that agent at all.",[11,5777,5778],{},"Three more habits from the session genuinely earn their place under an efficiency banner, each backed by Knight's own stated reasoning rather than just a vibe:",[37,5780,5781,5787,5793],{},[40,5782,5783,5786],{},[43,5784,5785],{},"Skills over re-explaining."," Installing CLI skills (markdown files that teach the agent what commands exist) means you're not \"pasting huge help text into every prompt.\" It's explicitly part of why the CLI uses fewer tokens than MCP in his own comparison, skills are loaded only when needed instead of being baked into every tool call.",[40,5788,5789,5792],{},[43,5790,5791],{},"Save state to markdown instead of letting it evaporate."," When Knight had the agent save a generated test plan to a file rather than leaving it in chat, his reasoning doubled as a genuinely good explanation of why: \"Your context window is only so big... if I didn't save my test plan in this markdown file, I'd have to make it regenerate the test plan again. That sucks.\" He compared it to saving progress in an old Super Nintendo game before your context window (or your save file) gets wiped.",[40,5794,5795,5798],{},[43,5796,5797],{},"Inside-out test generation."," Rather than guessing a locator, running the test, watching it crash, and correcting, Playwright's CLI and MCP tooling let the agent build a session step by step, discovering real locators as it goes. \"That usually leads to very short loops, not having to repeat a lot of loops.\" It's a real efficiency argument and it's specific to how Playwright's own tooling is built, not a generic prompting tip.",[11,5800,5801,5802,5804],{},"Knight also argued that AI-assisted test generation cuts maintenance cost, since a broken locator can trigger \"a little bit of agentic maintenance... a healing loop, commit that fix back in.\" I think that may be oversold, or at least dependent on your engineering practices. Maybe this has more of an ROI on pages undergoing rapid prototyping or constant redesigns, but outside of those scenarios, I find locators remain relatively stable once they're set up in a page object model, assuming you're using ID attributes (if they aren't randomly generated) or something like ",[21,5803,4943],{},". Playwright also has modern locator strategies that preclude a lot of the problems people used to get themselves into with XPath or text-based locators.",[11,5806,5807,5808,5811],{},"The live demos backed up the rest. One had the agent open the app, create a board, add a list, and invent three plausible user stories from a single plain-English prompt, no locators, no Playwright code written by hand. Another had it explore the app, propose a test plan, save that plan to a markdown file, and then generate full ",[21,5809,5810],{},"*.spec.ts"," files from it, self-healing failures as it ran, ending at 74 passed and 1 skipped. Knight's own retrospective on that second demo is worth keeping, because it's a caution about scope, not about cost: \"I would not recommend doing what I showed here, big asks. I would recommend many small asks.\" Review the output like a teammate's pull request, not like a vending machine.",[11,5813,5814,5815,5818],{},"I liked that Knight acknowledged the reality of the quality of test you get straight from AI with a prompt like this. The generated code was unoptimized and raw, similar to what the earlier codegen example created when we recorded our manual steps through the application to build a test case. You ",[723,5816,5817],{},"would not"," want to use these tests in your final test suite as-is:",[1668,5820,5821],{},[11,5822,5823],{},"\"There's no page objects here. There's no real library abstraction... these names aren't great.\"",[11,5825,5826],{},"Here's the clean version of the prompt he used, taken from his tutorial notes rather than transcribed live:",[100,5828,5832],{"className":5829,"code":5830,"language":5831,"meta":106,"style":106},"language-txt shiki shiki-themes material-theme-lighter github-light-high-contrast github-dark-high-contrast","Using playwright-cli, open http:\u002F\u002Flocalhost:3000\u002F, reset data if needed via API, then walk through\nthe \"create board → add list → add cards → go home\" flow. Use snapshots to pick stable locators.\nThen add a new Playwright TypeScript test under `tests\u002F` that matches our existing style:\n`test.beforeAll` or `beforeEach` for \u002Fapi\u002Freset, clear test name, getByRole\u002FgetByPlaceholder,\nand expect assertions. Reuse patterns from our existing trello spec if present.\n","txt",[21,5833,5834,5839,5844,5849,5854],{"__ignoreMap":106},[110,5835,5836],{"class":112,"line":113},[110,5837,5838],{},"Using playwright-cli, open http:\u002F\u002Flocalhost:3000\u002F, reset data if needed via API, then walk through\n",[110,5840,5841],{"class":112,"line":144},[110,5842,5843],{},"the \"create board → add list → add cards → go home\" flow. Use snapshots to pick stable locators.\n",[110,5845,5846],{"class":112,"line":161},[110,5847,5848],{},"Then add a new Playwright TypeScript test under `tests\u002F` that matches our existing style:\n",[110,5850,5851],{"class":112,"line":172},[110,5852,5853],{},"`test.beforeAll` or `beforeEach` for \u002Fapi\u002Freset, clear test name, getByRole\u002FgetByPlaceholder,\n",[110,5855,5856],{"class":112,"line":179},[110,5857,5858],{},"and expect assertions. Reuse patterns from our existing trello spec if present.\n",[11,5860,5861],{},"With more deliberate prompt engineering, Claude could have produced a cleaner first draft. But the rawer version is what actually demonstrated the accelerated-scaffolding benefit, and it set up a natural case for why prompt engineering matters in the first place:",[1668,5863,5864],{},[11,5865,5866],{},"\"If I were to do full context engineering, I would have my rules for Playwright tests, and I would say things like, use page object model.\"",[11,5868,5869],{},"Left on its own, a prompt like this gets you a fast, working first draft, not a finished one. The written version of this tutorial has a fair name for that tradeoff: \"accelerated scaffolding, not a substitute for judgment.\" Same deal as raw codegen output earlier in this piece, a working draft far faster than typing it by hand, just not something you'd commit as-is.",[32,5871,5873],{"id":5872},"why-locators-still-beat-computer-vision","Why Locators Still Beat Computer Vision",[11,5875,5876,5877,5880],{},"The day before Knight's session, Dionny Santiago's StarEast 2026 tutorial made close to the opposite argument about how AI should interact with a web page. I wrote about ",[46,5878,5879],{"href":4868},"his case for AI vision testing over brittle CSS and XPath selectors"," in more detail, but the short version is direct: \"Computer vision is the evolution of the CSS selectors and the XPath selectors,\" reading a page the way a person does instead of hunting for a class name or test ID. Knight never mentioned Santiago's session, and might not have even been aware of it. An audience member raised a version of it anyway, describing tools that skip \"element work\" entirely in favor of a vision-based approach, and Knight disagreed without hesitating:",[1668,5882,5883],{},[11,5884,5885],{},"\"I disagree with that. Because even with AI superpowers, image matching is still going to be expensive. Whereas locators are very cheap and quick.\"",[11,5887,5888],{},"He built a full historical case for why, the kind of argument worth quoting at length because it's the most fully-reasoned claim in the entire session. The short version: programming has only ever moved toward higher abstraction (assembly to Fortran and C to Java, Python, and TypeScript), because each higher layer let us trust the layer below it without reading it. His extension of that idea to AI:",[1668,5890,5891],{},[11,5892,5893],{},"\"AI is the new compiler. Source code in TypeScript and Java and Python is the new assembly code... It will not be much longer that we still have to dance down at those levels because it's going to get so good. We still have to today because it's not as good yet.\"",[11,5895,5896],{},"Then the part that actually settles the locators-versus-vision question, mapping compiled-versus-interpreted execution onto test automation directly:",[1668,5898,5899],{},[11,5900,5901],{},"\"What I showed you before with, hey, let's just explore the app with Playwright CLI and just let it go and not record anything, that was equivalent to an interpreter. That's very slow. That's token heavy. Your image matching thing when it comes to test execution is also going to be inherently slow. Always, because if you're looking at something, you have to image match in the moment... that grinding can never not be done in that kind of model. So that's why I don't think the image matching of locators is ever really going to happen.\"",[11,5903,5904,5905,5908],{},"The distinction matters for accuracy: this is about test ",[723,5906,5907],{},"execution",", how the automation decides where to click while a test runs, not about visual regression tools that diff screenshots to catch rendering bugs. Knight never argues against that second category at all. Within the category he's actually addressing, his case is the more convincing one between these two tutorials. Generating a locator-based script costs tokens once. Running it costs almost nothing, over and over. Vision-based execution pays the image-matching cost every single run, forever, no matter how good the underlying model gets. That's a structural cost difference, not a current-capability gap that better models eventually close.",[32,5910,5912],{"id":5911},"test-pyramids-skyscrapers-and-the-gap-nobody-closed","Test Pyramids, Skyscrapers, and the Gap Nobody Closed",[11,5914,5915],{},"Back to the line I deferred earlier. Here's Knight's full skyscraper pivot, verbatim:",[1668,5917,5918],{},[11,5919,5920],{},"\"Today we don't build pyramids anymore. We build skyscrapers. Look up to testing skyscrapers. We need to reframe what we think of for testing in modern times because the world has changed since that previous mental model was created.\"",[11,5922,5923],{},"It's a good line, and it's worth being precise about what it actually claims. Knight never says UI tests are better than unit tests, the literal claim is narrower: \"UI tests are not bad. All tests are good because they mitigate different kinds of risks.\" That's an argument against rigid proportions, not a reordering of the hierarchy. He also never builds out the metaphor itself, there's no mapping of \"floors\" to test types anywhere in the session, the slides, or the written tutorial chapters. The skyscraper is a mood, not a blueprint.",[11,5925,5926],{},"His actual defense for ditching the pyramid's bias against UI tests is the tooling argument from earlier in this piece: Playwright's architecture fixed the execution speed and flakiness problems that gave UI tests their bad reputation. That's a real, demonstrated improvement. What it doesn't touch is the part of the pyramid's logic that was never about execution speed at all. A unit test calling a function in-process will always be faster than even the fastest browser context, that's a difference in kind, not in tooling. Unit tests also stay directly traceable to source lines and branches in a way browser-driven tests can't. Neither Playwright nor the AI tooling covered in this session does anything about that gap, and it never came up once in either half of the tutorial.",[11,5928,5929,5930,5933],{},"The AI-assisted authoring material from the previous section actually extends Knight's case further than he extended it himself, just not far enough to close that gap. If AI assistance genuinely lowers the cost of writing and maintaining E2E tests (and the token and time savings shown live back that up, even if the maintenance claim is softer), that addresses the ",[723,5931,5932],{},"other"," half of the pyramid's original justification, the cost of producing and keeping E2E tests working, which his own tooling argument never reached. So the fuller, more honest position: the case for de-emphasizing strict pyramid proportions is stronger than Knight made it sound, once you add AI-assisted authoring on top of Playwright's execution-speed fix. It's still not a full rebuttal of the pyramid, because the one gap that was never about tooling in the first place is still sitting there untouched.",[11,5935,5936],{},"Page objects, splitting one big test into independent behavior tests, and the parallel-safe test data strategy that actually fixes the \"drop the whole database\" shortcut from earlier in this piece are all covered in Knight's written tutorial chapters, just not in the room.",[32,5938,5940],{"id":5939},"my-takeaways-on-playwright-and-ai-testing","My Takeaways on Playwright and AI Testing",[11,5942,5943],{},"A few things I'm taking back with me:",[37,5945,5946,5952,5958,5964],{},[40,5947,5948,5951],{},[43,5949,5950],{},"Default to the CLI over MCP for routine test-development work."," Reach for MCP only when local terminal and filesystem access isn't an option in the first place, not just because it feels more capable.",[40,5953,5954,5957],{},[43,5955,5956],{},"Treat AI-generated tests as scaffolding, not a finished product."," The first draft comes out raw, the same as old-school codegen output, so the cleanup step (page objects, naming, structure) isn't optional, it's the rest of the job.",[40,5959,5960,5963],{},[43,5961,5962],{},"Locator-based testing wins for driving test execution, and I don't expect that to change as models improve."," The cost gap is structural, not a capability gap that better models eventually close. (Visual regression testing is a different problem, and a fair use case for vision-based tools.)",[40,5965,5966,5969],{},[43,5967,5968],{},"Playwright and AI assistance narrow the case for the old Testing Pyramid, but they don't close it."," Knight's argument only ever answered the execution-speed half of the pyramid's old bias against UI tests; AI-assisted authoring answers some of the authoring-cost half too. Neither touches the one gap that was never about tooling: a unit test will always run faster and trace more directly to source than any browser-driven test.",[11,5971,5972,5973,5977],{},"If testing AI systems themselves (not just using AI to write tests) is more your focus right now, ",[46,5974,5976],{"href":5975},"\u002Fsoftware-testing\u002Ftest-automation\u002Fhow-to-test-ai-chatbots-and-agents","how I approached evals on a real agentic chatbot engagement"," is a related read you may find useful.",[3866,5979],{":items":5980},"[\"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-getting-started-ai-driven-automation\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-getting-dirty-ai-testing\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fplaywright-accessibility-testing-axe-lighthouse-limitations\",\"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-prompt-engineering-techniques\"]",[3870,5982,5983],{},"html pre.shiki code .sZTni,html code.shiki .sZTni{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#A0111F;--shiki-default-font-style:inherit;--shiki-dark:#FF9492;--shiki-dark-font-style:inherit}html pre.shiki code .sPJuK,html code.shiki .sPJuK{--shiki-light:#39ADB5;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sZ-rw,html code.shiki .sZ-rw{--shiki-light:#90A4AE;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html pre.shiki code .sZi47,html code.shiki .sZi47{--shiki-light:#39ADB5;--shiki-default:#032563;--shiki-dark:#ADDCFF}html pre.shiki code .srGNg,html code.shiki .srGNg{--shiki-light:#91B859;--shiki-default:#032563;--shiki-dark:#ADDCFF}html pre.shiki code .sb1SK,html code.shiki .sb1SK{--shiki-light:#6182B8;--shiki-default:#622CBC;--shiki-dark:#DBB7FF}html pre.shiki code .stWsX,html code.shiki .stWsX{--shiki-light:#9C3EDA;--shiki-default:#A0111F;--shiki-dark:#FF9492}html pre.shiki code .s2xgV,html code.shiki .s2xgV{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#702C00;--shiki-default-font-style:inherit;--shiki-dark:#FFB757;--shiki-dark-font-style:inherit}html pre.shiki code .s_gjE,html code.shiki .s_gjE{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#66707B;--shiki-default-font-style:inherit;--shiki-dark:#BDC4CC;--shiki-dark-font-style:inherit}html pre.shiki code .sq0XF,html code.shiki .sq0XF{--shiki-light:#E53935;--shiki-default:#0E1116;--shiki-dark:#F0F3F6}html .light .shiki span{color:var(--shiki-light);background:var(--shiki-light-bg);font-style:var(--shiki-light-font-style);font-weight:var(--shiki-light-font-weight);text-decoration:var(--shiki-light-text-decoration)}html.light .shiki span{color:var(--shiki-light);background:var(--shiki-light-bg);font-style:var(--shiki-light-font-style);font-weight:var(--shiki-light-font-weight);text-decoration:var(--shiki-light-text-decoration)}html .default .shiki span{color:var(--shiki-default);background:var(--shiki-default-bg);font-style:var(--shiki-default-font-style);font-weight:var(--shiki-default-font-weight);text-decoration:var(--shiki-default-text-decoration)}html .shiki span{color:var(--shiki-default);background:var(--shiki-default-bg);font-style:var(--shiki-default-font-style);font-weight:var(--shiki-default-font-weight);text-decoration:var(--shiki-default-text-decoration)}html .dark .shiki span{color:var(--shiki-dark);background:var(--shiki-dark-bg);font-style:var(--shiki-dark-font-style);font-weight:var(--shiki-dark-font-weight);text-decoration:var(--shiki-dark-text-decoration)}html.dark .shiki span{color:var(--shiki-dark);background:var(--shiki-dark-bg);font-style:var(--shiki-dark-font-style);font-weight:var(--shiki-dark-font-weight);text-decoration:var(--shiki-dark-text-decoration)}",{"title":106,"searchDepth":144,"depth":144,"links":5985},[5986,5987,5988,5989,5990,5991],{"id":4890,"depth":144,"text":4891},{"id":4923,"depth":144,"text":4924},{"id":5689,"depth":144,"text":5690},{"id":5872,"depth":144,"text":5873},{"id":5911,"depth":144,"text":5912},{"id":5939,"depth":144,"text":5940},"\u002Fimages\u002Fposts\u002Fstareast-2026-playwright-ai-cost-efficient-testing\u002Fstareast-2026-playwright-ai-cost-efficient-testing-cover.webp","2026-06-22","Playwright's MCP server can burn your AI budget fast. StarEast 2026 lessons on efficient AI testing, and why locators still beat computer vision.",{},"\u002Fsoftware-testing\u002Ftest-automation\u002Fstareast-2026-playwright-ai-cost-efficient-testing",{"title":4852,"description":5994},"software-testing\u002Ftest-automation\u002Fstareast-2026-playwright-ai-cost-efficient-testing","joMPpMIfdSO7ZR8LztFrojUyxQeGStMo6mi6RIG0EI4",{"id":6001,"title":6002,"bmcUsername":6,"body":6003,"cover":6295,"date":6296,"description":6297,"draft":3897,"extension":3898,"features":6,"githubRepo":6,"headline":6,"highlight":6,"icon":6,"meta":6298,"navigation":175,"npmPackage":6,"order":6,"path":6299,"seo":6300,"stem":6301,"__hash__":6302},"content\u002Fsoftware-testing\u002Ftest-automation\u002Fbest-websites-for-practicing-test-automation.md","Best Websites for Practicing Test Automation",{"type":8,"value":6004,"toc":6288},[6005,6009,6012,6015,6035,6038,6041,6045,6059,6062,6074,6077,6086,6089,6098,6101,6110,6113,6122,6125,6134,6137,6146,6149,6160,6169,6175,6189,6193,6196,6205,6208,6217,6220,6229,6232,6241,6244,6248,6256,6259,6267,6270,6278,6282,6285],[32,6006,6008],{"id":6007},"why-use-demo-websites-for-test-automation-practice","Why use demo websites for test automation practice",[11,6010,6011],{},"Before investing time in learning and building out a test suite against a new automation framework it is a good idea to try it out against different kinds of websites to ensure the framework performs reliably and is easy to work with. For example, if you are using Selenium currently, but are interested in trying Cypress, Playwright, WebDriverIO, or Nightwatch try doing more than just the basic example against these sites. This way you don't get too invested to find out your chosen automation framework doesn't work on some important area you need to write automation for.",[11,6013,6014],{},"You'll want to try out your test automation framework against things like:",[37,6016,6017,6020,6023,6026,6029,6032],{},[40,6018,6019],{},"iFrames",[40,6021,6022],{},"Modal dialogs",[40,6024,6025],{},"New tabs",[40,6027,6028],{},"Multiple windows",[40,6030,6031],{},"Asynchronous loading of content",[40,6033,6034],{},"File upload dialogs",[11,6036,6037],{},"You may not have one or all of these scenarios in your current automated software testing project. These demo sites become useful to provide somewhere to practice skills around these harder scenarios if you don't get exposure in your normal everyday feature testing.",[11,6039,6040],{},"I've compiled a list of the best websites I've come across to practice or demo test automation frameworks against in the list below. They include the harder to automate scenarios just mentioned.",[32,6042,6044],{"id":6043},"recommended-demo-websites-for-test-automation-practice","Recommended demo websites for test automation practice",[4037,6046,6047],{},[40,6048,6049,6050,6054,6055],{},"\"The Internet\" ",[79,6051],{"href":6052,"text":6053},"https:\u002F\u002Fgithub.com\u002Fsaucelabs\u002Fthe-internet","github"," or ",[79,6056],{"href":6057,"text":6058},"http:\u002F\u002Fthe-internet.herokuapp.com\u002F","live site",[11,6060,6061],{},"This GitHub repository is a collection of common test automation scenarios including hard to automate situations; nested frames, shadow DOM, keypresses, and complicated DOMs. The hosting on this site can be unreliable so fall back to their github if it happens to be down.",[4037,6063,6064],{"start":144},[40,6065,1048,6066,6069,6070],{},[723,6067,6068],{},"Automation Exercise"," clothing store ",[79,6071],{"href":6072,"text":6073},"https:\u002F\u002Fwww.automationexercise.com\u002F","automationexercise.com",[11,6075,6076],{},"This example clothing storefront has both a web front end to test against and APIs. The site even includes test cases to guide you.",[4037,6078,6079],{"start":161},[40,6080,6081,6082],{},"LetCode ",[79,6083],{"href":6084,"text":6085},"https:\u002F\u002Fletcode.in\u002Ftest","letcode.in",[11,6087,6088],{},"This site isolates examples around common DOM elements letting you practice on specific examples of web elements like inputs, tables, alerts, slides, calendars, and more.",[4037,6090,6091],{"start":172},[40,6092,6093,6094],{},"UI Test Automation Playground ",[79,6095],{"href":6096,"text":6097},"http:\u002F\u002Fwww.uitestingplayground.com\u002F","uitestingplayground.com",[11,6099,6100],{},"Smaller site, but it contains edge cases for load delays, mouse over behavior, dynamic IDs, and automation issues arising from hidden layers.",[4037,6102,6103],{"start":179},[40,6104,6105,6106],{},"SwagLabs ",[79,6107],{"href":6108,"text":6109},"https:\u002F\u002Fwww.saucedemo.com\u002F","saucedemo.com",[11,6111,6112],{},"Another demo web storefront useful for testing login and shopping cart flows. A key distinction on this site is that it has 4 different logins you can use for different experiences for the same site; normal, locked out, problem user, and performance glitch user. Maintained by the folks at SauceLabs.",[4037,6114,6115],{"start":208},[40,6116,6117,6118],{},"Angular Banking Site ",[79,6119],{"href":6120,"text":6121},"https:\u002F\u002Fwww.globalsqa.com\u002FangularJs-protractor\u002FBankingProject","GlobalsQA",[11,6123,6124],{},"Very small example bank website written in Angular for testing your automation framework against a website written in Angular. The site has login scenarios as well as transaction listing, deposits, and withdrawal behavior workflows.",[4037,6126,6127],{"start":256},[40,6128,6129,6130],{},"Forms sandbox ",[79,6131],{"href":6132,"text":6133},"https:\u002F\u002Fautomatenow.io\u002Fsandbox\u002F","AutomateNow",[11,6135,6136],{},"Small, straight-forward single page site for practicing interactions with typical form controls, search, pop up dialogs, and a map.",[4037,6138,6139],{"start":298},[40,6140,6141,6142],{},"Automation Practice ",[79,6143],{"href":6144,"text":6145},"https:\u002F\u002Fqa-practice.razvanvancea.ro\u002F","QA-Practice",[11,6147,6148],{},"Clean site that combines several different testing exercises in a small e-commerce site:",[37,6150,6151,6154,6157],{},[40,6152,6153],{},"Common web elements",[40,6155,6156],{},"API with Swagger documentation",[40,6158,6159],{},"Buggy web form you can use to challenge candidates to see how many bugs they can discover.",[4037,6161,6162],{"start":376},[40,6163,6164,6165],{},"The Sweet Shop ",[79,6166],{"href":6167,"text":6168},"https:\u002F\u002Fsweetshop.netlify.app\u002F","sweetshop.netlify.app",[11,6170,6171,6174],{},[723,6172,6173],{},"The Sweet Shop"," is a fake e-commerce front end for a candy store with hidden defects and opportunity to automate:",[37,6176,6177,6180,6183,6186],{},[40,6178,6179],{},"Account login",[40,6181,6182],{},"Shopping cart",[40,6184,6185],{},"Product pages",[40,6187,6188],{},"Non-US currency",[32,6190,6192],{"id":6191},"recommended-api-testing-sites","Recommended API testing sites",[11,6194,6195],{},"If you are learning SuperTest + Mocha for API Test automation or learning Postman the following sites are handy REST APIs to practice tests against.",[4037,6197,6198],{},[40,6199,6200,6201],{},"Swagger Petstore ",[79,6202],{"href":6203,"text":6204},"https:\u002F\u002Fpetstore.swagger.io\u002F","petstore.swagger.io",[11,6206,6207],{},"Manage an example pet store inventory through this API. This site shows off the usefulness in Swagger when documenting REST APIs, but you can also practice API testing with it.",[4037,6209,6210],{"start":144},[40,6211,6212,6213],{},"Restful Booker ",[79,6214],{"href":6215,"text":6216},"https:\u002F\u002Frestful-booker.herokuapp.com\u002Fapidoc\u002Findex.html","restful-booker.herokuapp.com",[11,6218,6219],{},"Test all kinds of CRUD (Create, Update, Delete) REST API scenarios using this example booking API.",[4037,6221,6222],{"start":161},[40,6223,6224,6225],{},"FakeRestAPI ",[79,6226],{"href":6227,"text":6228},"https:\u002F\u002Ffakerestapi.azurewebsites.net\u002Findex.html","fakerestapi.azurewebsites.net",[11,6230,6231],{},"More practice REST endpoints around a book store inventory API. This one lacks Auth operations so you can't practice token creation. However, Restful Booker (above), does support this.",[4037,6233,6234],{"start":172},[40,6235,6236,6237],{},"Star Wars API ",[79,6238],{"href":6239,"text":6240},"https:\u002F\u002Fswapi.dev\u002F","SWAPI",[11,6242,6243],{},"A well-documented public API for Star Wars data that can be used for practicing building your own Postman collection or automated API test suite.",[32,6245,6247],{"id":6246},"other-lists-more-reading","Other Lists \u002F More Reading",[4037,6249,6250],{},[40,6251,6252],{},[79,6253],{"href":6254,"text":6255},"https:\u002F\u002Fgithub.com\u002FBMayhew\u002Fawesome-sites-to-test-on","Awesome Sites to Test On",[11,6257,6258],{},"Butch Mayhew maintains a list of more sites categorized by security, mobile, performance, and web.",[4037,6260,6261],{"start":144},[40,6262,6263],{},[79,6264],{"href":6265,"text":6266},"https:\u002F\u002Fautomationpanda.com\u002F2021\u002F12\u002F29\u002Fwant-to-practice-test-automation-try-these-demo-sites\u002F","Automation Panda",[11,6268,6269],{},"Recommended sites by Andrew Knight.",[4037,6271,6272],{"start":161},[40,6273,6274],{},[46,6275,6277],{"href":6276},"\u002Fsoftware-testing\u002Fframeworks\u002Fnightwatch\u002Fapi-testing-with-nightwatch-supertest","API Testing in Nightwatch.js",[32,6279,6281],{"id":6280},"know-of-a-good-one","Know of a good one?",[11,6283,6284],{},"If you know of any other good sites I should include please reach out through my social links below 👇",[3866,6286],{"to":6287},"\u002Fsoftware-testing\u002Ftest-automation",{"title":106,"searchDepth":144,"depth":144,"links":6289},[6290,6291,6292,6293,6294],{"id":6007,"depth":144,"text":6008},{"id":6043,"depth":144,"text":6044},{"id":6191,"depth":144,"text":6192},{"id":6246,"depth":144,"text":6247},{"id":6280,"depth":144,"text":6281},"\u002Fimages\u002Fposts\u002Fbest-websites-for-practicing-test-automation\u002Fbest-sites-for-test-practice.jpg","2026-03-12","Looking for example demo sites to practice your test automation against with Selenium, Playwright, or Cypress? Here is a list of the top demo websites for practicing your test automation skills against.",{},"\u002Fsoftware-testing\u002Ftest-automation\u002Fbest-websites-for-practicing-test-automation",{"title":6002,"description":6297},"software-testing\u002Ftest-automation\u002Fbest-websites-for-practicing-test-automation","BPGWHU_1LTRwQJQp50G1zM3QKu7eqfqP2aNRmJp3puc",1789952825760]