tests: wait for flaky iframes to load via JS
TL;DR: I think I stabilized a couple of hinting iframe tests and applied that change to all the iframe usages in the end2end tests in the hopes that it would resolve some other flaky tests. I was facing some tests that hinted elements in iframes failing intermittently recently. They were most consistently failing on the windows runner. This is similar to the trend described in the comment linked from #1525. The tests failing recently where: tests (py39-pyqt515, windows-2019, 3.9) test_using_hintfollow_inside_an_iframe test_using_hintfollow_inside_an_iframe_button tests (py311-pyqt515, ubuntu-20.04, 3.11) test_using_hintfollow_inside_an_iframe test_using_hintfollow_inside_an_iframe_button They are failing because hints don't get generated for the elements in the iframe. I can see hints do get generated for the iframe itself though. Examining the logs in 5.15 it seems that there is a `cur_load_finished(True) (tab *)` log line after we run :hint. I suspect that this is the load finished signal for the iframe. I attempted to change the bdd open_path function to wait for the current message it's looking for and then additionally look for that cur_load_finished signal, but then it starts failing locally when the iframe load signal actually comes before the parent frame one. Just looking for the cur_load_finished signal itself also always found it immediately ("already found"), probably from the parent frame. So instead of trying to deal with that indeterminate ordering or trying to change the signals to say what frame they are coming from, I added javascript to all the pages used in iframes that run on load, and changed all the tests to watch for log messages from the JS. It's not the most maintainable solution, perhaps if we generated our test files with jinja we could have some "subframe loaded" boilerplate, get a message to log from a query param, look at metadata from test files for the open_path checking etc. I then searched for all the iframe usages in the test data and applied that change to all of those files and all the tests that used them. A few of those tests also had `flaky` annotations on them. I removed those annotations for now in the hopes that there were affected by the same problem. And that I actually managed to work around it correctly. ref: #7621
This commit is contained in:
toofar
parent
e03b81cb35
commit
cb0b51c094
|
|
@ -7,7 +7,7 @@
|
|||
<meta charset="utf-8">
|
||||
<title>Link wrapped across multiple lines</title>
|
||||
</head>
|
||||
<body>
|
||||
<body onload="console.log('wrapped loaded')">
|
||||
<div style="width: 20em;">
|
||||
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis <a href="/data/hello.txt">nostrud exercitation</a> ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
<meta charset="utf-8">
|
||||
<title>Button link wrapped across multiple lines</title>
|
||||
</head>
|
||||
<body>
|
||||
<body onload="console.log('wrapped_button loaded')">
|
||||
<div style="width: 20em;">
|
||||
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis <button id="myButton" class="float-left submit-button" >nostrud exercitation</button> ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@
|
|||
<title>Simple input</title>
|
||||
<script type="text/javascript">
|
||||
function setup_event_listener() {
|
||||
console.log('input loaded')
|
||||
var elem = document.getElementById('qute-input-existing');
|
||||
console.log(elem);
|
||||
elem.addEventListener('input', function() {
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
<head>
|
||||
<title>Issue 3711</title>
|
||||
</head>
|
||||
<body>
|
||||
<body onload="console.log('issue3711 loaded')">
|
||||
<!--
|
||||
Verify no hint error occurs when hinting input range elements in iframes on qt5.9
|
||||
Possibly an issue in chrome.
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
}
|
||||
</script>
|
||||
</head>
|
||||
<body>
|
||||
<body onload="console.log('simple loaded')">
|
||||
<a href="/data/hello.txt" id="link">Just a link</a>
|
||||
<button>blub</button>
|
||||
<p>Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Vestibulum tortor quam, feugiat vitae, ultricies eget, tempor sit amet, ante. Donec eu libero sit amet quam egestas semper. Aenean ultricies mi vitae est. Mauris placerat eleifend leo. Quisque sit amet est et sapien ullamcorper pharetra. Vestibulum erat wisi, condimentum sed, commodo vitae, ornare sit amet, wisi. Aenean fermentum, elit eget tincidunt condimentum, eros ipsum rutrum orci, sagittis tempus lacus enim ac dui. Donec non enim in turpis pulvinar facilisis. Ut felis. Praesent dapibus, neque id cursus faucibus, tortor neque egestas augue, eu vulputate magna eros eu erat. Aliquam erat volutpat. Nam dui mi, tincidunt quis, accumsan porttitor, facilisis luctus, metus</p>
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
<meta charset="utf-8">
|
||||
<title>Searching text on the page</title>
|
||||
</head>
|
||||
<body>
|
||||
<body onload="console.load('search loaded')">
|
||||
<p>
|
||||
foo<br/>
|
||||
Foo<br/>
|
||||
|
|
|
|||
|
|
@ -251,25 +251,28 @@ Feature: Using hints
|
|||
### iframes
|
||||
Scenario: Using :hint-follow inside an iframe
|
||||
When I open data/hints/iframe.html
|
||||
And I wait for "* wrapped loaded" in the log
|
||||
And I hint with args "links normal" and follow a
|
||||
Then "navigation request: url http://localhost:*/data/hello.txt, type Type.link_clicked, *" should be logged
|
||||
|
||||
Scenario: Using :hint-follow inside an iframe button
|
||||
When I open data/hints/iframe_button.html
|
||||
And I wait for "* wrapped_button loaded" in the log
|
||||
And I hint with args "all normal" and follow s
|
||||
Then "navigation request: url http://localhost:*/data/hello.txt, *" should be logged
|
||||
|
||||
Scenario: Hinting inputs in an iframe without type
|
||||
When I open data/hints/iframe_input.html
|
||||
And I wait for "* input loaded" in the log
|
||||
And I hint with args "inputs" and follow a
|
||||
And I wait for "Entering mode KeyMode.insert (reason: clicking input)" in the log
|
||||
And I run :mode-leave
|
||||
# The actual check is already done above
|
||||
Then no crash should happen
|
||||
|
||||
@flaky # FIXME https://github.com/qutebrowser/qutebrowser/issues/1525
|
||||
Scenario: Using :hint-follow inside a scrolled iframe
|
||||
When I open data/hints/iframe_scroll.html
|
||||
And I wait for "* simple loaded" in the log
|
||||
And I hint with args "all normal" and follow a
|
||||
And I run :scroll bottom
|
||||
And I hint with args "links normal" and follow a
|
||||
|
|
@ -291,11 +294,13 @@ Feature: Using hints
|
|||
|
||||
Scenario: Clicking on iframe with :hint all current
|
||||
When I open data/hints/iframe.html
|
||||
And I wait for "* wrapped loaded" in the log
|
||||
And I hint with args "all current" and follow a
|
||||
Then no crash should happen
|
||||
|
||||
Scenario: No error when hinting ranged input in frames
|
||||
When I open data/hints/issue3711_frame.html
|
||||
And I wait for "* issue3711 loaded" in the log
|
||||
And I hint with args "all current" and follow a
|
||||
Then no crash should happen
|
||||
|
||||
|
|
|
|||
|
|
@ -124,24 +124,25 @@ Feature: Javascript stuff
|
|||
# https://github.com/qutebrowser/qutebrowser/issues/1190
|
||||
# https://github.com/qutebrowser/qutebrowser/issues/2495
|
||||
|
||||
@flaky
|
||||
Scenario: Have a GreaseMonkey script run at page start
|
||||
When I have a GreaseMonkey file saved for document-start with noframes unset
|
||||
And I run :greasemonkey-reload
|
||||
And I open data/hints/iframe.html
|
||||
And I wait for "* wrapped loaded" in the log
|
||||
Then the javascript message "Script is running on /data/hints/iframe.html" should be logged
|
||||
|
||||
Scenario: Have a GreaseMonkey script running on frames
|
||||
When I have a GreaseMonkey file saved for document-end with noframes unset
|
||||
And I run :greasemonkey-reload
|
||||
And I open data/hints/iframe.html
|
||||
And I wait for "* wrapped loaded" in the log
|
||||
Then the javascript message "Script is running on /data/hints/html/wrapped.html" should be logged
|
||||
|
||||
@flaky
|
||||
Scenario: Have a GreaseMonkey script running on noframes
|
||||
When I have a GreaseMonkey file saved for document-end with noframes set
|
||||
And I run :greasemonkey-reload
|
||||
And I open data/hints/iframe.html
|
||||
And I wait for "* wrapped loaded" in the log
|
||||
Then the javascript message "Script is running on /data/hints/html/wrapped.html" should not be logged
|
||||
|
||||
Scenario: Per-URL localstorage setting
|
||||
|
|
|
|||
|
|
@ -363,20 +363,20 @@ Feature: Searching on a page
|
|||
- data/search.html
|
||||
- data/hello.txt (active)
|
||||
|
||||
# Too flaky
|
||||
@qtwebkit_skip: Not supported in qtwebkit @skip
|
||||
Scenario: Follow a searched link in an iframe
|
||||
When I open data/iframe_search.html
|
||||
And I wait for "* search loaded" in the log
|
||||
And I run :tab-only
|
||||
And I run :search follow
|
||||
And I wait for "search found follow" in the log
|
||||
And I run :selection-follow
|
||||
Then "navigation request: url http://localhost:*/data/hello.txt, type Type.link_clicked, is_main_frame False" should be logged
|
||||
|
||||
# Too flaky
|
||||
@qtwebkit_skip: Not supported in qtwebkit @skip
|
||||
Scenario: Follow a tabbed searched link in an iframe
|
||||
When I open data/iframe_search.html
|
||||
And I wait for "* search loaded" in the log
|
||||
And I run :tab-only
|
||||
And I run :search follow
|
||||
And I wait for "search found follow" in the log
|
||||
|
|
|
|||
Loading…
Reference in New Issue