New headless chrome randomly hangs at the end #27264
Comments
|
@maciejmrozinski headless=new is what is run in Cypress by default since 12.15.0, so you don't need to pass the workaround tp get the new headless. Does turning off headless=new resolve your issue? Pass the code below, let us know. setupNodeEvents: function setupNodeEvents(on, config) {
on('before:browser:launch', (browser = {}, launchOptions) => {
if (browser.name === 'chrome' && browser.isHeadless) {
launchOptions.args.push('--headless');
}
return launchOptions;
});
} |
|
Yes, I've already tested this before and using old headless mode is resolving the issue. |
|
@maciejmrozinski We'll need a test to run to reproduce this behavior fully. We haven't observed this behavior on our test suite, so there is something particular about your run that's causing the hang that we'd need to be able to run so we can track down the issue. |
|
It's also applicable to my project. |
|
We have the same issue, will there be a fix in the future? |
|
I have this issue as well on CI (CircleCI) I get When I try I get |
|
Is there update for this issue? |
See also Google Chrome Developer > Try out the new Headless
|
|
We're also encountering this issue with our cypress runs in CI. We were using the We use parallelization via DeploySentinel. You can see that one of the parallel runs in this example (#5) drops to 0% CPU and hangs, producing now output, eventually being cancelled by CircleCI due to inactivity. I'll enable verbose debug logging on the process handler and the chrome browser launch to get chrom stderr and report back if that reveals anything. |
|
We need a reproducible example provided so that we can narrow down the cause of hanging for some users with the new headless behavior. Please can someone provide one. |
|
The problem is that it's not reproducible, it's seemingly random. We're not sure what triggers it. |
|
We also still have this issue. Currently no clue on how to reproduce it, as it occurs randomly |
|
This issue is random, it seems it is hanging with one of the longest test or with the last instance. But this seems to get resolved after upgraded to the latest |
|
@jennifer-shehane we don't have a reproducible example but we did manage to capture debug logs for a run that this happened to. See attached file. This is with the following: The logs that look promising there to me are shortly before the tests just hang we get: And then the chrome process is missing from the process list that immediately follows. It pops up again later but maybe cypress isn't reconnecting to the new chrome process and hence the hanging? |
|
We did notice that peak memory usage was around 3.5 GB with new, whereas old uses about 2.4 GB. Our github actions runner has a 4GB resource limit. Perhaps chrome is running out of memory and is making cypress hang (pure guess). We are using the following github actions config: test:
runs-on: [self-hosted, prod]
container:
image: cypress/included:cypress-13.2.0-node-20.6.1-chrome-116.0.5845.187-1-ff-117.0-edge-116.0.1938.76-1
options: --ipc=hostand the following cypress.config.ts: e2e: {
setupNodeEvents(_on, _config) {
_on('before:browser:launch', (browser, launchOptions) => {
if (browser.name === 'chrome') {
launchOptions.args.push('--disable-dev-shm-usage');
launchOptions.args.push('--headless=old');
}
console.log(launchOptions.args);
return launchOptions;
});
}, |
|
Hi there, I'm also facing same issue with cypress version 12.7.4 and chrome version 112 when try to execute the tests in parallel. I have tried using above '--headless=old' but no luck. Please can someone help? Thanks |
|
Our issue was resolved by itself after upgrade to v13 |
|
Unfortunately, I can't upgrade to v13 so need some workaround please |
|
We have observed a slowdown in performance for one project when using headless=new. We're still interested in having examples that show this behavior so that we can narrow down the issue. We suspect there's likely a bug in Chrome headless, but it's specific to some situation. |
|
Not sure if it's really helpful but it could support the hypothesis that it's a bug in Chrome headless. We're having the same issue that Chrome with new headless modes just hangs randomly after running all tests. But with a whole different stack: We're on Ruby and using RSpec/Capybara. |
jennifer-shehane
added
the
stage: needs information
|
@jennifer-shehane I'm facing same issue with electron browser. Cypress tests hangs up sometimes at the end and sometimes before starting. In cypress cloud it show "This spec does not have any test results because it timed out". I've faced same issue from cypress 12.13.0 to 12.17.0 and for some people it hangs in firefox too. I'm mentioning this issue for reference |
|
Is this still occuring for people? We haven't had comments for a couple of months. |
|
Yes, myself and other users of my project are still seeing headless chrome randomly hang before exit, even when it's run directly via CLI outside of cypress. I'm almost positive it's an upstream chrome bug. Rebooting often fixes it, waiting an hour and trying again sometimes fixes it, force-reinstalling chrome also often fixes it, which puts it squarely into heisenbug territory. Related: |
|
@pirate What version of Chrome are you using? Have you tried updating? |
|
This issue has been present as far back as It's intermittent and hard to verify sometimes, so many issues I've found about it on related projects have gotten closed as "cannot reproduce". I've just confirmed it's happening particularly consistently with There are also widespread reports of similar issues with the two telltale symptoms:
Possibly related reports:
It's possible some of these issues ^ are unrelated, but it's also possible they all stem from the same underlying issue of child chromium processes not exiting correctly. The problem is widespread enough that many of the tools that use chrome headless have implemented hacky workarounds like this: https://devforth.io/blog/how-to-simply-workaround-ram-leaking-libraries-like-puppeteer-universal-way-to-fix-ram-leaks-once-and-forever/ (spawning chrome under a child process then doing killasgroup -9 after every run) I tried again just for fun and managed to reproduce this on the first try! I didn't even add any of the extra args we usually use ( 
This dispelled the last of my doubts, I think this is 100% an upstream Chromium bug and has nothing to do with Cypress/Playwright/Puppeteer/ArchiveBox/any driver. I just opened an upstream bug report on the Chromium bug tracker, follow over there for progress: https://issues.chromium.org/issues/327583144 👾 
|
|
@pirate Thanks for the detailed writeup and opening an issue with chromium. We'll take a look. It is extremely difficult to track down with all the variables involved as you explained. Is there a way to provide the project you're running where you got it to hang immediately? |
|
It's not in any project, it's just raw chromium headless from the command line, no extra env vars, hidden CLI flags, or profile directory provided: # using chromium downloaded via puppeteer
# recommended by: https://www.chromium.org/getting-involved/download-chromium/
$ npx @puppeteer/browsers install [email protected]
$ ~/chrome/mac_arm-121.0.6167.57/chrome-mac-arm64/Google\ Chrome\ for\ Testing.app/Contents/MacOS/Google\ Chrome\ for\ Testing --headless=new --screenshot 'https://example.com'
[63086:259:0301/184829.306692:ERROR:policy_logger.cc(156)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(161) Cloud management controller initialization aborted as CBCM is not enabled. Please use the `--enable-chrome-browser-cloud-management` command line flag to enable it if you are not using the official Google Chrome build.
72602 bytes written to file screenshot.png
# ... hangs indefinitely ...
^C⏎
[138.101s]
# OR equivalent using playwright's chromium
$ pip install --upgrade playwright
$ playwright install --with-deps chromium
$ ~/Library/Caches/ms-playwright/chromium-1097/chrome-mac/Chromium.app/Contents/MacOS/Chromium --headless=new --screenshot 'https://example.com'
[63478:259:0301/185212.347544:ERROR:policy_logger.cc(156)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(161) Cloud management controller initialization aborted as CBCM is not enabled. Please use the `--enable-chrome-browser-cloud-management` command line flag to enable it if you are not using the official Google Chrome build.
72602 bytes written to file screenshot.png
# ... hangs indefinitely ...
^C⏎
[241.309s]
Hung on the first try for both methods. Other versions besides |
|
This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided. |
|
This issue has been closed due to inactivity. |
Current behavior
Using new headless mode in Chrome it sometimes hangs after running test file (also mentioned in this comment #25972 (comment))
Desired behavior
Consistent, correct behaviour just like with headless=old
Test code to reproduce
There is no simple repo to reproduce. From my observation it hangs mostly if test flow contains the page that has external (iframe) loaded content (in my case it's google recaptcha). CI runs are more vulnerable to this issue, but I was able to reproduce it few times locally inside docker container.
Simple test code that sometimes fails:
Cypress Version
12.17.0
Node version
18.16.0
Operating System
Docker image: cypress/browsers:node-18.16.0-chrome-113.0.5672.92-1-ff-113.0-edge-113.0.1774.35-1
Debug Logs
Other
No response
The text was updated successfully, but these errors were encountered: