a3238eb494
This commit takes a screenshot of the active browser window when an end2end test fails. When running on CI a zip file of screenshots will be attached to the run summary as an artifact. When run locally screenshots will be left in /$TMPDIR/pytest-screenshots/. The screenshot is of the Xvfb screen that the tests are running under. If there are multiple windows open it will likely only show the active window because a) we aren't running with a window manager b) the Xvfb display is, by default, the same size as the browser window. I'm not sure if xvfb is used on the Window runs in CI. We could fall back to trying to take screenshots if not running under xvfb but I'm a bit wary of an automatic feature that takes screenshots of people's desktops when running locally. Even if they just to to /tmp/ it might be surprising. We can change it later if it turns out we need to try to take screenshots in more cases. I'm using pillow ImageGrab, the same as pyvirtualdisplay.smartdisplay. I'm getting the display number from the pytest-xvfb plugin and formatting it appropriately (pyvirtualdisplay has an already formatted one which is used by the smartdisplay, but that's not accessible). Pillow is now a requirement for running the tests. I thought about making it gracefully not required but I'm not sure how to inform the user with a warning from pytest, or if they would even want one. Maybe we could add a config thing to allow not taking screenshots? I had to bump the colordepth config for pytest-xvfb otherwise pillow complained that the default 16bit color depth wasn't supported as it only supports 24bit, see https://github.com/python-pillow/Pillow/blob/1138ea5370cbda5eb328ec949 8c314d376c81265/src/display.c#L898 I'm saving screenshots to a temp dir because I don't want to put them in my workdir when running locally. I want to clear the directory for each run so that you don't get confused by looking at old images. I'm not 100% sure about the lifecycle of the process classes though. Eg if we have two processes they might both try to create the output directory. I'm using pytest.session.stash to save the directory so perhaps the lifecycle of the stash will handle that? Not sure. Ideally the images would be uploaded somewhere where we could click through and open them in the browser without having to download a zip file, but I'm not sure how to achieve that. It would be nice to print in the test logs that a screenshot was saved and where to. Just so you could copy paste the filename instead of having to match the sanitized filename against failing test names. But I don't know how to log stuff from this stage in the pytest lifecycle. TODO: * I'm not sure that the screenshot captures the whole browser window? Maybe the browser windows is bigger than the X11 display? Closes: #7625 |
||
|---|---|---|
| .. | ||
| dev | ||
| testbrowser | ||
| __init__.py | ||
| asciidoc2html.py | ||
| cycle-inputs.js | ||
| dictcli.py | ||
| hist_importer.py | ||
| hostblock_blame.py | ||
| importer.py | ||
| keytester.py | ||
| link_pyqt.py | ||
| mkvenv.py | ||
| open_url_in_instance.sh | ||
| opengl_info.py | ||
| setupcommon.py | ||
| utils.py | ||