0c3807b04a
I would like it to be obvious to contributors who run the tests locally that there are screenshots of the processes under test that they can examine. I don't think it's obvious that there could be useful files sitting round in a temp directory. This commit adds the screenshot file paths to a user property on failed tests then adds a custom report section that pulls that lists those properties. That way when there is errors users will get the paths to the images printed out alongside the report of failed tests. I find it difficult to navigate the internals of pytest. I tried various ways of printing information and getting that information to methods that could do the printing but couldn't get anything to work. I ended up entirely copying this SO post which worked really well for attaching information to test results in a place that is accessable to the reporting hook: https://stackoverflow.com/a/64822668 It's added to the end of the existing terminal report hook, because while it seems you can have two of those hooks, things can get pretty confusing with interleaved reports and not all of them running every time. --------------------- End2end screenshots available in: /tmp/pytest-of-user/pytest-56/pytest-screenshots --------------------- 2024-08-17T14_49_35.896940-tests_end2end_features_test_utilcmds_bdd.py__test_cmdrepeatlast_with_modeswitching_command_deleting.png 2024-08-17T14_49_37.391229-tests_end2end_features_test_completion_bdd.py__test_deleting_an_open_tab_via_the_completion.png =================================================== short test summary info ==================================================== FAILED tests/end2end/features/test_utilcmds_bdd.py::test_cmdrepeatlast_with_modeswitching_command_deleting - AssertionError: assert 'http://local...ata/hello.txt' == 'http://local...ata/sello.txt' FAILED tests/end2end/features/test_completion_bdd.py::test_deleting_an_open_tab_via_the_completion - AssertionError: assert 'http://local...ata/hello.txt' == 'http://local...ata/sello.txt' ====================================================== 2 failed in 5.18s ======================================================= From adding debug messages I can see: RUNNER_TEMP=/home/runner/work/_temp /tmp/pytest-of-runner/pytest-0/pytest-screenshots Means that I don't think GHA will be able to collect the temp files because they are not being written to the temp dir being mounted in from outside. I think that's the case anyway. Might have to pass --basetemp=$RUNNER_TEMP to pytest or set TEMP or something. TODO |
||
|---|---|---|
| .. | ||
| end2end | ||
| helpers | ||
| manual | ||
| unit | ||
| conftest.py | ||
| test_conftest.py | ||