[FLAKY ON MKI] `x-pack/test_serverless/functional/test_suites/common/discover/x_pack/reporting.ts`

by ADMIN 99 views

FLAKY ON MKI: Investigating Issues in x-pack/test_serverless/functional/test_suites/common/discover/x_pack/reporting.ts

Summary

This article discusses two failing tests in the x-pack/test_serverless/functional/test_suites/common/discover/x_pack/reporting.ts file on the MKI environment. The tests are related to the discover feature in Kibana, specifically the CSV export functionality. We will delve into the error messages, command lines, and screenshots provided to identify the root cause of the issues and propose potential solutions.

Test 1: discover Discover CSV Export Generate CSV: new search generates a large export

Command Line

node scripts/functional_test_runner.js --config x-pack/test_serverless/functional/test_suites/search/common_configs/config.group6.ts --exclude-tag skipMKI --exclude-tag failsOnMKI --exclude-tag skipServerless --exclude-tag skipCloud --exclude-tag skipSvlSearch --info

Error Message

Error: Test report failed: Unknown error: Error: retry.tryForTime reached timeout 120000 ms
TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
Wait timed out after 81635ms
    at /root/.qaf/data/git/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:22
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at ReportingPageObject.getReportURL (reporting_page.ts:58:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at getReport (reporting.ts:40:17)
    at Context. (reporting.ts:141:35)
    at Object.apply (wrap_function.js:74:16) {
  [cause]: Error: retry.tryForTime reached timeout 120000 ms
  TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
  Wait timed out after 81635ms
      at /root/.qaf/data/git/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:22
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
      at onFailure (retry_for_success.ts:18:9)
      at retryForSuccess (retry_for_success.ts:86:7)
      at RetryService.tryForTime (retry.ts:38:12)
      at TestSubjects.getAttribute (test_subjects.ts:263:12)
      at ReportingPageObject.getReportURL (reporting_page.ts:44:19)
      at getReport (reporting.ts:40:17)
      at Context. (reporting.ts:141:35)
      at Object.apply (wrap_function.js:74:16)
}

Screenshot

Image

The error message indicates that the test is waiting for the "downloadCompletedReportButton" element to be located, but it times out after 81635ms. This suggests that the element is not being loaded or is not present on the page.

Test 2: discover Discover CSV Export Generate CSV: new search generates a report from a new search with data: default

Command Line

node scripts/functional_test_runner.js --config x-pack/test_serverless/functional/test_suites/security/common_configs/config.group6.ts --exclude-tag skipMKI --exclude-tag failsOnMKI --exclude-tag skipServerless --exclude-tag skipCloud --exclude-tag skipSvlSec --info

Error Message

Error: Test report failed: Unknown error: Error: retry.tryForTime reached timeout 120000 ms
TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
Wait timed out after 61176ms
    at /root/.qaf/data/git/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:22
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at ReportingPageObject.getReportURL (reporting_page.ts:58:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at getReport (reporting.ts:40:17)
    at Context. (reporting.ts:112:21)
    at Object.apply (wrap_function.js:74:16) {
  [cause]: Error: retry.tryForTime reached timeout 120000 ms
  TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="downloadCompletedReportButton"])
  Wait timed out after 61176ms
      at /root/.qaf/data/git/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:22
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
      at onFailure (retry_for_success.ts:18:9)
      at retryForSuccess (retry_for_success.ts:86:7)
      at RetryService.tryForTime (retry.ts:38:12)
      at TestSubjects.getAttribute (test_subjects.ts:263:12)
      at ReportingPageObject.getReportURL (reporting_page.ts:44:19)
      at getReport (reporting.ts:40:17)
      at Context. (reporting.ts:112:21)
      at Object.apply (wrap_function.js:74:16)
}

Screenshot

Image

The error message for this test is similar to the previous one, indicating that the test is waiting for the "downloadCompletedReportButton" element to be located, but it times out after 61176ms.

Potential Solutions

Based on the error messages and screenshots, it appears that the issue is related to the "downloadCompletedReportButton" element not being loaded or present on the page. Here are some potential solutions:

  1. Check the element's presence: Verify that the "downloadCompletedReportButton" element is present on the page and is not being loaded dynamically.
  2. Increase the timeout: Increase the timeout value for the test to allow more time for the element to be loaded.
  3. Use a more robust waiting mechanism: Use a more robust waiting mechanism, such as waiting for the element to be visible or enabled, instead of just waiting for it to be present.
  4. Check for any errors in the console: Check the console for any errors that may be related to the element not being loaded.
  5. Verify the test configuration: Verify that the test configuration is correct and that the test is running with the correct environment and settings.

By investigating the error messages and screenshots, and considering the potential solutions, we can identify the root cause of the issues and propose a plan to resolve them.
FLAKY ON MKI: Investigating Issues in x-pack/test_serverless/functional/test_suites/common/discover/x_pack/reporting.ts - Q&A

Q: What is the issue with the discover feature in Kibana?

A: The issue is that the "downloadCompletedReportButton" element is not being loaded or is not present on the page, causing the test to timeout.

Q: What is the cause of the issue?

A: The cause of the issue is not immediately clear, but it may be related to the test configuration, the environment, or the code itself.

Q: How can I troubleshoot the issue?

A: To troubleshoot the issue, you can try the following:

  1. Check the element's presence: Verify that the "downloadCompletedReportButton" element is present on the page and is not being loaded dynamically.
  2. Increase the timeout: Increase the timeout value for the test to allow more time for the element to be loaded.
  3. Use a more robust waiting mechanism: Use a more robust waiting mechanism, such as waiting for the element to be visible or enabled, instead of just waiting for it to be present.
  4. Check for any errors in the console: Check the console for any errors that may be related to the element not being loaded.
  5. Verify the test configuration: Verify that the test configuration is correct and that the test is running with the correct environment and settings.

Q: What are some potential solutions to the issue?

A: Some potential solutions to the issue include:

  1. Check the element's presence: Verify that the "downloadCompletedReportButton" element is present on the page and is not being loaded dynamically.
  2. Increase the timeout: Increase the timeout value for the test to allow more time for the element to be loaded.
  3. Use a more robust waiting mechanism: Use a more robust waiting mechanism, such as waiting for the element to be visible or enabled, instead of just waiting for it to be present.
  4. Check for any errors in the console: Check the console for any errors that may be related to the element not being loaded.
  5. Verify the test configuration: Verify that the test configuration is correct and that the test is running with the correct environment and settings.

Q: How can I prevent similar issues in the future?

A: To prevent similar issues in the future, you can:

  1. Regularly review and update test configurations: Regularly review and update test configurations to ensure that they are correct and up-to-date.
  2. Use robust waiting mechanisms: Use robust waiting mechanisms, such as waiting for elements to be visible or enabled, instead of just waiting for them to be present.
  3. Check for errors in the console: Regularly check the console for any errors that may be related to test failures.
  4. Verify test results: Verify test results to ensure that they are accurate and reliable.
  5. Continuously monitor and improve test infrastructure: Continuously monitor and improve test infrastructure to ensure that it is reliable and efficient.

Q: What are some best practices for testing discover features in Kibana?

A: Some best practices for testing discover features in Kibana include:

  1. Use robust testing frameworks: Use robust testing frameworks, such as Selenium or Cypress, to ensure tests are accurate and reliable.
  2. Use realistic test data: Use realistic test data to ensure that tests are representative of real-world scenarios.
  3. Test for edge cases: Test for edge cases, such as invalid input or unexpected behavior, to ensure that the feature is robust and reliable.
  4. Verify test results: Verify test results to ensure that they are accurate and reliable.
  5. Continuously monitor and improve test infrastructure: Continuously monitor and improve test infrastructure to ensure that it is reliable and efficient.

By following these best practices and troubleshooting techniques, you can ensure that your discover features in Kibana are accurate, reliable, and efficient.