This commit takes a screenshot of the active browser window when an
end2end test fails. When running on CI a zip file of screenshots will be
attached to the run summary as an artifact. When run locally screenshots
will be left in /$TMPDIR/pytest-screenshots/.
The screenshot is of the Xvfb screen that the tests are running under.
If there are multiple windows open it will likely only show the active
window because a) we aren't running with a window manager b) the Xvfb
display is, by default, the same size as the browser window.
I'm not sure if xvfb is used on the Window runs in CI. We could fall
back to trying to take screenshots if not running under xvfb but I'm a
bit wary of an automatic feature that takes screenshots of people's
desktops when running locally. Even if they just to to /tmp/ it might be
surprising. We can change it later if it turns out we need to try to
take screenshots in more cases.
I'm using pillow ImageGrab, the same as pyvirtualdisplay.smartdisplay. I'm
getting the display number from the pytest-xvfb plugin and formatting it
appropriately (pyvirtualdisplay has an already formatted one which is used by
the smartdisplay, but that's not accessible).
Pillow is now a requirement for running the tests. I thought about making
it gracefully not required but I'm not sure how to inform the user with
a warning from pytest, or if they would even want one. Maybe we could
add a config thing to allow not taking screenshots?
I had to bump the colordepth config for pytest-xvfb otherwise pillow
complained that the default 16bit color depth wasn't supported as it
only supports 24bit, see https://github.com/python-pillow/Pillow/blob/1138ea5370cbda5eb328ec949
8c314d376c81265/src/display.c#L898
I'm saving screenshots to a temp dir because I don't want to put them in
my workdir when running locally. I want to clear the directory for each
run so that you don't get confused by looking at old images. I'm not
100% sure about the lifecycle of the process classes though. Eg if we
have two processes they might both try to create the output directory.
I'm using pytest.session.stash to save the directory so perhaps the
lifecycle of the stash will handle that? Not sure.
Ideally the images would be uploaded somewhere where we could click
through and open them in the browser without having to download a zip
file, but I'm not sure how to achieve that.
It would be nice to print in the test logs that a screenshot was saved
and where to. Just so you could copy paste the filename instead of
having to match the sanitized filename against failing test names. But I
don't know how to log stuff from this stage in the pytest lifecycle.
TODO:
* I'm not sure that the screenshot captures the whole browser window?
Maybe the browser windows is bigger than the X11 display?
Closes: #7625
My virtualenv I used to run webkit has rotted long ago and I don't remember
how I set it up. There is a PyQtWebKit project on PyPI but I don't know
who that's published by.
So I figured I would write some notes for myself on using the docker container
used for CI instead. I chose to mount the current directory (which is
presumably a qutebrowser checkout!) directly into the container instead of
cloning it so I could have quicker feedback between making code changes and
running tests.
Then there's a couple of things that stem from that. Since the user in the
container is different from the one in the host we have to move some things
that are normally written to the current directory to be written elsewhere.
There are other ways to approach this (eg you can add `-u $(id -u)` to the
docker command line, although that makes things a bit confusing in the
container) but arguably it's good for the container not to be able to write to
the host, hence making that volume read only.
The TOX_WORK_DIR trick is from
[here](https://github.com/tox-dev/tox/issues/20), apart from with
`{toxinidir}` in it too because the pyroma env was failing with just
`.tox`, saying the pyroma binary needed to be in the allowlist, possibly
it was doing full path matching without normalizing.
The hypothesis folks
[here](https://github.com/HypothesisWorks/hypothesis/issues/2367#issuecomment-595524571)
say if you want to override the examples DB location with an env var to
do it yourself. It's actually only a warning from hypothesis, it says it
falls back to an in-memory DB, but I guess the tests run with
warnings-are-errors. You can also pass `database=None` to make
hypothesis skip example storage altogether.
I'm using tox to run commands in a virtualenv with the right stuff in it
because, uh, because I was copying the CI workflow actually. I just found out
about the `exec` subcommand to override the `commands` defined for the env,
neat! One point of awkwardness about that is that since we are using the
PyQt from the OS we need any virtualenv we use to have access to the OS
packages, which isn't the default for virtualenvs created by tox. The
text envs use the link_pyqt script for that but if you are using this
container and the first thing you do is run `tox exec` then that
wouldn't have been run. So I'm setting `VIRTUALENV_SYSTEM_SITE_PACKAGES`
to tell tox to always make the system packages available in the
virtualenvs it manages.
I did try using the mkvenv script instead of tox but it complained when
trying to install the current directory in editable mode because
setup.py tries to write to a git-commit-id file.
6.7 is released now, some distros are already shipping it!
This commit:
1. adds a new 6.7 requirements file (the plain 6 one has already been
updated by the bot)
2. adds a new tox env referring to the new requirements file
3. updates the mac and windows installer jobs to run with pyqt67 with the
assumption we'll be including that in our next release
4. adds two new CI environments for 6.7, one each for python 3.11 and 3.12
(3.12 only came out like 6 months ago)
5. updates a couple of references to the py37 tox env that looked like they
were missed, 3.7 support was dropped in 93c7fdd
6. updates various ubuntu containers to the latest LTS instead of the previous
related one - this is quite unrelated to this change but I thought I would
give it a go, no need to use the old one unless we are specifically testing
on it?
- linters - they use tox but probably use system libraries
- these all run in nested containers anyway, should be fully isolated
- codeql - eh, uses a third party action, check the docs if it fails
- irc - as above
We got a `DeprecationWarning` during the package build, which we were not able to
reproduce locally. For now we just don't turn this particular `DeprecationWarning` into
an exception to not fail CI.
This is still *very* basic, but it serves its purpose of failing for warnings during
package build.
I verified that `tox -e package` is failing by introducing some warnings with this change:
```diff
diff --git a/setup.py b/setup.py
index feb949595..6810eaf1e 100755
--- a/setup.py
+++ b/setup.py
@@ -51,8 +51,7 @@ def _get_constant(name):
try:
common.write_git_file()
setuptools.setup(
- packages=setuptools.find_namespace_packages(include=['qutebrowser',
- 'qutebrowser.*']),
+ packages=setuptools.find_namespace_packages(include=['qutebrowser']),
include_package_data=True,
entry_points={'gui_scripts':
['qutebrowser = qutebrowser.qutebrowser:main']},
```
Since we are pulling down PyInstaller off of the develop branch we need
to recompile the bootloader, because upstream only commits a new one
back to the branch on releases. Luckily all the compiler requirements
seem to already be installed on CI.
For the record the macOS CI is currently failing with
dlopen: dlopen(/Users/runner/work/qutebrowser/qutebrowser/dist/qutebrowser.app/Contents/MacOS/libpython3.10.dylib, 10): image not found
And upon inspection of dist/ that file seems to be at
./qutebrowser.app/Contents/Resources/libpython3.10.dylib
./qutebrowser.app/Contents/Frameworks/libpython3.10.dylib
./qutebrowser/_internal/libpython3.10.dylib
Otherwise we run into a Python 3.12 incompatibility with pkg_resources:
https://github.com/pypa/pip/issues/11501
Also needs PIP_REQUIRE_VIRTUALENV=0 because otherwise pip seems to
falsely assume it's installing things system-wide, weirdly.
Should probably be removed once there is a newer virtualenv, which
vendors a newer pip, which vendors a newer pkg_resources...
See #7727
- Make sure we still test Python 3.7 and 3.8 after dropping old PyQt
versions in c5a51eb0bc
- Keep a modern Python version (3.11) with Qt 5 around, however
- Make sure we still test Python 3.10 too
- Also start testing the Python 3.12 alpha
On CI were were getting "Could not import sip" because link_pyqt was
looking for PyQt5.sip.
I made that look at QUTE_QT_WRAPPER since that's being set already on
tox.ini
There are probably a few other changes around link_pyqt and the makefile
etc we need to change when we switch the default wrapper.
I overrode the default `py` tox environment with py-qt6 to override
those wrapper related variables. I probably could have done something
sneaky with curly braces to make it so we don't have to add a few more
lines to the file. But in my opinion in config file is far to obfuscated
and hard to maintain already.
I changed the docker file to call the new py-qt6 env if it's a qt6
container. I'm not 100% sure that is required though since there is also
a tox invocation in the GH action definition, maybe that overrides the
container entrypoint? Also changed the indentation in the dockerfile
template a bit to make it easier to see where the conditionals start and
end.
Speaking of which I changed the matrix definition and tox invocation to
match a later one to hopefully make it so we can invoke different tox
environments in the containers without having to rebuild the containers.
Not sure I did that right, I'll see soon.
I added the unstable-qt6 container generation line so we can use it in
the future, and to match the not-qt6 one. I'm not switching to that in
CI though because the pyqt used by that is broken at the moment
(ref https://www.riverbankcomputing.com/pipermail/pyqt/2023-March/045214.html)
Also fixed the vim modeline in generate.py so my syntax highlighting
works.
Florian Bruhin
mathis renaud
toofar
Philipp Albrecht
arza