With the upgrade to MarkupSafe 3.0, something funny happened when trying to pass
the GUIProcess object to jinja after launching a userscript:
[...]
File "[...]/qutebrowser/browser/qutescheme.py", line 291, in qute_process
src = jinja.render('process.html', title=f'Process {pid}', proc=proc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "[...]/qutebrowser/utils/jinja.py", line 123, in render
return environment.get_template(template).render(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...]
File "html/process.html", line 11, in block 'content'
File "[...]/lib/python3.11/site-packages/markupsafe/__init__.py", line 42, in escape
if hasattr(s, "__html__"):
^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: wrapped C/C++ object of type GUIProcess has been deleted
This can be reproduced with:
qutebrowser --temp-basedir ':cmd-later 0 spawn -u -o /bin/echo test'
We pass the `GUIProcess` to the Jinja template as `proc`, which then formats it as
`{{ proc }}`` (to stringify it). For some reason, with the newest MarkupSafe/Jinja
versions, this now triggers the `if hasattr(s, "__html__")` check in MarkupSafe
(which has been around for a while). That then presumably causes PyQt to try and
access the underlying C++ object for `GUIProcess``, but that has already been
deleted.
But why is it deleted in the first place, if we keep track of even completed
processes data ever since we added `:process` in a3adba81c? It looks like the Qt
parent-child relationship is the culprit here: When we pass a parent to the
`GUIProcess`` from the userscript runner, it will get deleted as soon as said
runner is cleaned up (which happens after the userscript has finished).
We probably never noticed this before because we only accessed data from the
Python wrapper and not from the C++ side, but it still seems like a good idea
to avoid passing a parent for a long-lived object (with time-based cleanup) in
the first place.
This flag is vital for the allow-list configuration to be picked up.
It should be set globally and `dev-qt/qtwebengine` should be
recompiled after it's enabled.
Ref #8313.
With PyQt 6, this gets represented as
QWebEnginePage.RenderProcessTerminationStatus(-1)
which is != -1, thus leading to a KeyError.
Updating to a RendererProcessTerminationStatus
enum value works fine on both PyQt5 and PyQt6.
Will be dropped on GitHub Actions tomorrow:
https://github.blog/changelog/2024-05-20-actions-upcoming-changes-to-github-hosted-macos-runners/
For unit tests, we now run them on macOS 13 instead, thus testing on all three
macOS versions we currently support.
For releases, this forces us to now support macOS 12 as the oldest supported
version and drop macOS 11 support. Thus, we should not have a v3.2.2 release.
Not backporting this commit so CI fails there rather than silently bumping up
requirements.
My virtualenv I used to run webkit has rotted long ago and I don't remember
how I set it up. There is a PyQtWebKit project on PyPI but I don't know
who that's published by.
So I figured I would write some notes for myself on using the docker container
used for CI instead. I chose to mount the current directory (which is
presumably a qutebrowser checkout!) directly into the container instead of
cloning it so I could have quicker feedback between making code changes and
running tests.
Then there's a couple of things that stem from that. Since the user in the
container is different from the one in the host we have to move some things
that are normally written to the current directory to be written elsewhere.
There are other ways to approach this (eg you can add `-u $(id -u)` to the
docker command line, although that makes things a bit confusing in the
container) but arguably it's good for the container not to be able to write to
the host, hence making that volume read only.
The TOX_WORK_DIR trick is from
[here](https://github.com/tox-dev/tox/issues/20), apart from with
`{toxinidir}` in it too because the pyroma env was failing with just
`.tox`, saying the pyroma binary needed to be in the allowlist, possibly
it was doing full path matching without normalizing.
The hypothesis folks
[here](https://github.com/HypothesisWorks/hypothesis/issues/2367#issuecomment-595524571)
say if you want to override the examples DB location with an env var to
do it yourself. It's actually only a warning from hypothesis, it says it
falls back to an in-memory DB, but I guess the tests run with
warnings-are-errors. You can also pass `database=None` to make
hypothesis skip example storage altogether.
I'm using tox to run commands in a virtualenv with the right stuff in it
because, uh, because I was copying the CI workflow actually. I just found out
about the `exec` subcommand to override the `commands` defined for the env,
neat! One point of awkwardness about that is that since we are using the
PyQt from the OS we need any virtualenv we use to have access to the OS
packages, which isn't the default for virtualenvs created by tox. The
text envs use the link_pyqt script for that but if you are using this
container and the first thing you do is run `tox exec` then that
wouldn't have been run. So I'm setting `VIRTUALENV_SYSTEM_SITE_PACKAGES`
to tell tox to always make the system packages available in the
virtualenvs it manages.
I did try using the mkvenv script instead of tox but it complained when
trying to install the current directory in editable mode because
setup.py tries to write to a git-commit-id file.
With GitHub Actions now providing macOS 14 runners with M1 chips, we can
build a separate Apple Silicon release there and upload it.
Universal wheels are currently not possible, see #8229 for details.
Closes#6478
I was getting crash reports from someone about this. Not sure what's going wrong
there (hence the additional information in the exception).
What's clear however is that we're raising ParseError, but only handling that
when actually parsing. The code calling copy_/_find_webengine_resources only
handles OSError. So let's raise a FileNotFoundError instead.
When qutebrowser is running but its installation has been deleted/moved, it
fails in somewhat mysterious but predictable ways. This is e.g. the case
currently, when people upgrade their Archlinux packages and upgrade from Python
3.11 to 3.12. When doing that with qutebrowser open, on the next page load, it
will:
- Have a crashed renderer process, because (assumingly) the process executable
is gone on disk.
- Which then causes us trying to render an error page, but that fails due to
broken_qutebrowser_logo.png being gone from disk.
- The FileNotFoundError then causes jinja2 to import jinja2.debug at runtime,
but that *also* fails because the jinja2 package is gone.
We work around this by loading the PNG into RAM early, and then using the cached
version instead. This amends b4a2352833 which did
the same with HTML/JS resources, but never for this PNG, which (looking at crash
logs) seems to be a somewhat common breakage.
Alternatives I've considered:
- Catching the FileNotFoundError and not showing an error page at all.
- Generating a PNG with an explanatory text via QPainter and returning that.
However, with the renderer process crash happening in the first place for
unknown reasons, it's unclear if the error page ever gets actually displayed...
Let's roll with this for now, and if this causes a repeating renderer process
crash, fix that separately (also see #5108).
Florian Bruhin
qutebrowser bot
toofar
Sviatoslav Sydorenko
Fernando Ramos
owl
Evan Chen