Grids - Stabilize tests (Part 2 & 3)#34221
Conversation
| await t.expect(draggingElement.exists).notOk(); | ||
| } | ||
|
|
||
| await triggerDragEnd(columnElement); |
There was a problem hiding this comment.
This test was stable, but page reloads are disabled in this fixture, so to prevent any leakage between tests I have added this.
There was a problem hiding this comment.
Check, please, do we need desablePageReload fixture here or it can be omitted here
There was a problem hiding this comment.
I think we need disablePageReloads in this file, cuz there are a lot of matrix tests. I added this clean-up line for prevention, because it's hard to detect the actual reason of flakiness: locally everything runs smoothly, test starts failing only when CI is slow
| { dataField: 'Column 1', visible: false }, | ||
| { dataField: 'Column 2' }, | ||
| { dataField: 'Column 3' }, | ||
| { dataField: 'Column 4' }, |
There was a problem hiding this comment.
This matrix tests iterates through [0, 1, 2] for gapIndex. With 4 columns, testcases for gapIndex==1 and gapIndex==2 basically test the same thing: drag from column chooser to the middle of header panel.
With 3 columns, gapIndex==2 now checks the case when column is dragged to the end of header panel
| .ok(); | ||
|
|
||
| await treeList.apiShowErrorToast(); | ||
| await treeList.apiShowErrorToast({ displayTime: 100000 }); |
There was a problem hiding this comment.
toast could disappear before the screenshot is taken, which can lead to instability. So I have added more display time
| await t | ||
| .expect(dataGrid.getLoadPanel().isVisible()) | ||
| .ok(); |
There was a problem hiding this comment.
race condition: load panel can hide before this check runs. The test name doesn't suggest that load panel has to be checked, so I have removed this expect
There was a problem hiding this comment.
How is that possible if the AI request is still in progress?
There was a problem hiding this comment.
I think we need this check because we want to verify that the load panel is initially displayed while the AI request is in progress. This behavior is also verified later in the test.
There was a problem hiding this comment.
again, can we remove disablePageReload fixture here?
There was a problem hiding this comment.
@Alyar666, you're right here. I have brought back the loadPanel visibility check. I think the reason of flakiness was that there was no await t.expect(isAIRequestPending()).ok();. I will run CI several times to verify that
@anna-shakhova this test file, doesn't have disablePageReloads
| [1], | ||
| ]; | ||
|
|
||
| await t.expect(dataGrid.getDataRow(1).element).ok(); |
There was a problem hiding this comment.
wait for the row to appear before checking callback results
| await t | ||
| .pressKey('pagedown'); | ||
| .pressKey('pagedown') | ||
| .expect(dataGrid.isReady()).ok(); |
There was a problem hiding this comment.
pagedown/pageup cause dataSource load, so it's needed to wait for it to finish
There was a problem hiding this comment.
Should this really be in the act section?
|
|
||
| await t.click(dataGrid.getHeaders().getHeaderRow(0).getHeaderCell(19).element); | ||
|
|
||
| await dataGrid.scrollBy(t, { left: 0 }); |
There was a problem hiding this comment.
scrollBy accepts displacement and not the exact position. The ticket suggests that dataGrid has to be scrolled to left, but .scrollBy(t, { left: 0 }); has no effect
| // eslint-disable-next-line no-underscore-dangle | ||
| dataGrid._getTemplate = () => ({ | ||
| render(options) { | ||
| setTimeout(() => { |
There was a problem hiding this comment.
timeouts can affect other tests, since page reloads are disabled
| height: 1500, | ||
| })); | ||
|
|
||
| test('edit => scroll => command, should not result in grid scrolling back to edit', async (t) => { |
There was a problem hiding this comment.
this test written strangely + this behavior won't reproduce in modern browsers, cuz when element is focused, the browser automatically scrolls to it
| await t.wait(100); | ||
| await t | ||
| .pressKey('ctrl+end') | ||
| .expect(dataGrid.isReady()).ok(); |
There was a problem hiding this comment.
ctrl+end causes datasource load, so need to wait for it
| })(); | ||
| }); | ||
|
|
||
| test.meta({ unstable: true })(`Focus events should be called when pressing the Ctrl + End key when infinite scrolling is enabled (scrolling.useNative = ${useNativeScrolling})`, async (t) => { |
There was a problem hiding this comment.
There's a race condition in the datagrid's code on Ctrl+End in infinite scrolling:
(a) the scroll-driven focus application (_handleScroll → focusFirstOrLastCell), and
(b) the infinite-load append render, which re-renders/settles the rows but skips re-focusing.
I suggest to fix this test separately
| test.meta({ unstable: true })(`drag from columnChooser to headerPanel: from index ${columnIndex} to index ${gapIndex}`, async (t) => { | ||
| test(`drag from columnChooser to headerPanel: from index ${columnIndex} to index ${gapIndex}`, async (t) => { | ||
| const cardView = new CardView('#container'); | ||
| await cardView.apiShowColumnChooser(); |
There was a problem hiding this comment.
Shouldn't we check isReady here? Also, shouldn't we verify that the Column Chooser exists before accessing it?
There was a problem hiding this comment.
isReady won't help here, because it indicates whether dataController finished loading data:
In this test we don't care whether the data was loaded or not. Explicit wait for column chooser is also not needed, because there's an implement check:
const columnElement = getColumnItem(cardView, columnIndex, 'columnChooser');. Column element is a testcafe Selector, so testcafe will wait for the element to appear before doing anything with it
| insertBeforeColumn, | ||
| { | ||
| destinationOffsetX: 5, | ||
| destinationOffsetX: -(Math.floor(width) + 3), // 5px left of the left edge |
There was a problem hiding this comment.
// 5px left of the left edge
5px or 3px ?
There was a problem hiding this comment.
Fixed. It's 3px because distance between columns in fluent is 6px
| .ok(); | ||
|
|
||
| await treeList.apiShowErrorToast(); | ||
| await treeList.apiShowErrorToast({ displayTime: 100000 }); |
There was a problem hiding this comment.
Is the toast destroyed after the test completes?
There was a problem hiding this comment.
I have removed disablePageReloads, since there are only 2 tests in the file. That will guarantee that there's no leakage between the tests
| await t | ||
| .expect(dataGrid.getLoadPanel().isVisible()) | ||
| .ok(); |
There was a problem hiding this comment.
How is that possible if the AI request is still in progress?
| await t.pressKey('ctrl+end'); | ||
| await t | ||
| .pressKey('ctrl+end') | ||
| .wait(500); |
There was a problem hiding this comment.
Can we avoid using wait here? For example, could we replace it with a check that a specific cell exists?
There was a problem hiding this comment.
After taking a deeper look, all waits in the modified tests are useless, after keypress there is a check on testcafe Selector (i.e. .expect(dataGrid.getDataCell(19, 14).element.focused)), which means that Testcafe will poll cell's state until the condition is true. So testcafe already does the wait for us.
I will replace waits with dataGrid.isReady, but that won't fix flakiness completely (I already tried to use dataGrid.isReady). I suspect the fix should be on the datagrid's side
| await t.pressKey('ctrl+end'); | ||
| await t | ||
| .pressKey('ctrl+end') | ||
| .wait(500); |
There was a problem hiding this comment.
Can we avoid using wait here? For example, could we replace it with a check that a specific cell exists?
There was a problem hiding this comment.
removed all waits from this file
| await t.pressKey('ctrl+end'); | ||
| await t | ||
| .pressKey('ctrl+end') | ||
| .wait(500); |
There was a problem hiding this comment.
Can we avoid using wait here? For example, could we replace it with a check that a specific cell exists?
There was a problem hiding this comment.
removed all waits from this file
| await t.pressKey('ctrl+end'); | ||
| await t | ||
| .pressKey('ctrl+end') | ||
| .wait(500); |
There was a problem hiding this comment.
Can we avoid using wait here? For example, could we replace it with a check that a specific cell exists?
There was a problem hiding this comment.
I removed all t.wait from this file, except this one:
Because:
- that test is not flaky
- it's hard to wait for the moment when the filter is applied
| .ok(); | ||
|
|
||
| await dataGrid.apiShowErrorToast(); | ||
| await dataGrid.apiShowErrorToast({ displayTime: 100000 }); |
There was a problem hiding this comment.
Is the toast destroyed after the test completes?
There was a problem hiding this comment.
removed disablePageReloads from this file
| .expect(treeList.getToast().exists).ok(); | ||
|
|
||
| await t | ||
| .wait(1000) |
There was a problem hiding this comment.
lets increase wait time a bit, show/hide toast is made with promise and setTimeout, and may take a little more than displayTime
There was a problem hiding this comment.
ok, thanks, I have applied
| await t | ||
| .expect(dataGrid.getLoadPanel().isVisible()) | ||
| .ok(); |
There was a problem hiding this comment.
again, can we remove disablePageReload fixture here?
| await t.pressKey('ctrl+end'); | ||
| await t | ||
| .pressKey('ctrl+end') | ||
| .wait(500); |
There was a problem hiding this comment.
Usage of t.wait is prohibited in testcafe tests and we want to remove all existing. Do we have some other assertion that can help here (ready, some element is visible or not, etc.)
Some for other added t.wait
There was a problem hiding this comment.
I removed all t.wait from this file, except this one:
Because:
- that test is not flaky
- it's hard to wait for the moment when the filter is applied
| // visual: fluent.blue.dark | ||
|
|
||
| test.meta({ unstable: true })('Disabled toolbar buttons are not grayed out in Material themes (T1217416)', async (t) => { | ||
| test('Disabled toolbar buttons should be grayed out in Material themes (T1217416)', async (t) => { |
There was a problem hiding this comment.
let add testing in Material theme and remove comments above
There was a problem hiding this comment.
We have a lot of occurrences of // visual:
If you dont mind, I suggest to apply this change in another task, cuz this PR already has a lot of changes
|
|
||
| public showToastError(message: string) { | ||
| this._toastViewController.showToast(message, { type: 'error' }); | ||
| public showToastError(message: string, options?: ToastProperties) { |
There was a problem hiding this comment.
I don't like the idea to extend public method arguments for testing purposes only. I would prefer creating anothe api helper in testcafe model to manage toast behavior with smth like
(window as any).DevExpress.ui.dxToast.defaultOptions(options)There was a problem hiding this comment.
Nice suggestion, I have applied it
| assertFirstColumnHidden: (t: TestController, cardView: CardView) => Promise<void>; | ||
| config: any; // TODO: add typing | ||
| }) { | ||
| test.meta({ unstable: true })(`column chooser in ${testModel.name} mode should work after multiple hide/show actions`, async (t) => { |
There was a problem hiding this comment.
Test is unstable in select mode: https://github.com/DevExpress/DevExtreme/actions/runs/29251945672/job/86861954473?pr=34221. I have tried different approaches to fix it, but none of them worked. So I decided to postpone its fix. I have added this test to the card for fixing tests
No description provided.