Commit Graph

181 Commits

Author SHA1 Message Date
Richard Hansen c803ec81f1 remote_runner: Handle webdriver errors 2021-05-03 01:48:08 -04:00
Richard Hansen 7f57b17b2e remote_runner: Use Error objects to convey pass/fail 2021-05-03 01:48:08 -04:00
Richard Hansen 4ec02a9af9 remote_runner: Simplify finished test check 2021-05-03 01:48:08 -04:00
Richard Hansen b0e367a982 remote_runner: Simplify logging of console text 2021-05-03 01:48:08 -04:00
Richard Hansen 08856fe42e remote_runner: Move `logIndex` updates into `printLog()` 2021-05-03 01:48:08 -04:00
Richard Hansen a12c475776 remote_runner: Use an options object to create webdriver object 2021-05-03 01:48:08 -04:00
Richard Hansen 925f789d4c remote_runner: Simplify logging 2021-05-03 01:48:08 -04:00
Richard Hansen 014e19cf7d remote_runner: `await` each browser test 2021-05-03 01:48:07 -04:00
Richard Hansen 1f3a831cc3 remote_runner: Avoid duplication in task list 2021-05-03 01:48:07 -04:00
Richard Hansen 59be8d5c05 remote_runner: Delete commented-out browsers
Also delete useless and incorrect browser comments.
2021-05-03 01:48:07 -04:00
Richard Hansen a58fa4a2c5 tests: Add tests for `SkipList.atOffset()` 2021-05-03 01:42:03 -04:00
Richard Hansen e2eb7327c2 skiplist: Sanity check inserted entries 2021-05-03 01:42:03 -04:00
webzwo0i 9408d4395f remove custom timeouts 2021-04-07 17:47:11 -04:00
Richard Hansen e4754eb9df tests: Fix race in `timeslider_revisions.js` 2021-04-02 15:46:27 +02:00
Richard Hansen 27e5373050 tests: Fix race in `change_user_name.js` 2021-04-02 15:46:27 +02:00
Richard Hansen 58dac4c0fc tests: Fix races in `inner_height.js` 2021-04-02 15:46:27 +02:00
Richard Hansen 4ad80d4072 tests: Delete overly aggressive frontend test timeouts
This should reduce test flakiness.
2021-04-02 15:46:27 +02:00
Richard Hansen 7cbb3f565d tests: Speed up `helper.edit()` and `helper.clearPad()` 2021-04-02 15:46:27 +02:00
Richard Hansen 7a154b1e1d tests: Wait for commit instead of sleep in `timeslider_revisions.js` 2021-04-02 15:46:27 +02:00
Richard Hansen d15ff9ce8d tests: Add missing `await`s to `change_user_name.js`
Also increase the timeouts.
2021-04-02 15:46:27 +02:00
Richard Hansen bbf89dfcf9 tests: Refine frontend tests
* Switch from `helper.newPad()` to `helper.aNewPad()`.
  * Promisify.
  * Delete redundant logic.
  * Lint fixes.
2021-04-02 15:46:27 +02:00
Richard Hansen 3790c0e41c tests: Use `async`/`await` instead of returning Promises
This has a few benefits:
  * It's more readable: It's easier for a user of the function to know
    that they should use `await` when calling the function.
  * Stack traces are more useful.
  * Some code (e.g., the async npm package) uses introspection to
    determine if a function is `async` vs. takes a callback.
2021-04-02 15:46:27 +02:00
Richard Hansen b164a34e64 lint: Fix ESLint error in `helper/methods.js` 2021-04-02 15:46:27 +02:00
Richard Hansen 62403159df tests: Invert conditions to improve readability 2021-04-02 15:46:27 +02:00
Richard Hansen dd9c08d821 tests: Wait for commit rather than sleep 2021-04-01 14:31:56 +02:00
Richard Hansen 2776946627 tests: Use cookie libraries to manipulate cookies 2021-04-01 14:31:56 +02:00
Richard Hansen aeee5c0b69 tests: Fix cookie name in `helper.js` tests 2021-04-01 14:31:56 +02:00
Richard Hansen 137fa89d2a tests: Always set cookie path to / (to match non-test behavior) 2021-04-01 14:31:56 +02:00
Richard Hansen 5666c34061 tests: Fix encoding of prefs cookie 2021-04-01 14:31:56 +02:00
Richard Hansen 63e6e163b7 tests: Promisify some of the `helper.js` tests 2021-04-01 14:31:56 +02:00
Richard Hansen 701a40ac13 tests: Promisify `multiple_authors_clear_authorship_colors.js` 2021-04-01 14:31:56 +02:00
Richard Hansen 8b43f9eb5f tests: Promisify `authorship_of_editions.js` 2021-04-01 14:31:56 +02:00
Richard Hansen 056939cd22 tests: Refine `helper/multipleUsers.js`
* Rename "current"/"other" to "user0"/"user1".
  * Delete unnecessary `_createTokenFor*` functions.
  * Rename helper functions to remove unnecessary leading underscore
    and for brevity.
  * Use jQuery's `.attr()` to build the second iframe.
  * Use js-cookie to manipulate the token cookie.
  * Don't attempt to set the token cookie if the pad isn't loaded.
  * Use the token generated by the pad.
  * Only clear the token cookie at path=/.
2021-04-01 14:31:56 +02:00
Guilherme Goncalves f2034ad368 tests: Add regression tests for character composition race
See: https://github.com/ether/etherpad-lite/issues/4978
2021-03-30 16:42:53 -04:00
Richard Hansen 718da6fc1b tests: New `helper.aNewPad()` (promisified `newPad()`) 2021-03-29 18:40:05 -04:00
Richard Hansen ec76a6548f tests: Make the `helper.newPad()` callback optional 2021-03-29 18:40:05 -04:00
Richard Hansen 27b35699ea tests: Fix `helper.newPad()` retries
* Pass retry count in options object so that each pad has its own
    retry count.
  * Delete useless `origPadName` variable.
2021-03-29 18:40:05 -04:00
webzwo0i b250ceaae7 add test for wrong clientHeight 2021-03-20 16:41:15 +00:00
John McLear 5ac90ab30f tests: Allow time for minification to complete
Minification happens after the initial visit and request to pages.
2021-03-05 07:11:42 +00:00
Richard Hansen 8971166c58 lint: Set up Cypress config and fix issues 2021-03-02 05:49:48 +00:00
John McLear 64e9e7fcda
tests: Frontend test Windows ZIP (#4894)
* tests:  Frontend test Windows ZIP

This PR introduces Frontend testing within Github actions!

We're depending a lot on saucelabs recently and that's fine but sometimes we just want to quickly do a frontend simple test on a weird environment (IE windows build) so this PR solves that problem.

Things to note.

    It still builds the windows .zip if the cypress tests fail.
    It does not add any heavy deps to Etherpad as cypress must be installed in CI.
    Cypress is responsible for running the Etherpad instance.

It's up to us how much we use this or not, I know it introduces a bunch of technical debt but I tried to keep that a minimum by compartmentalizing things and documenting where required.

* Update .github/workflows/windows-zip.yml

Co-authored-by: Richard Hansen <rhansen@rhansen.org>

* remove timeouts

* Move folder structure up a level

* Update windows-zip.yml

* Update test.js

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
2021-03-01 14:31:55 +00:00
John McLear 35ae08ca77
tests: run a long(1+ hr) load test 2021-02-28 16:05:16 +00:00
John McLear c7511b2f2e tests: change timeout on bold and embed_value test to support Win10 FF84 2021-02-23 19:53:03 +00:00
John McLear 72d9c8fef9 tests: Use more modern relevant Firefox version in Sauce Labs
Firefox 52 has issues with rendering SVG animations which caused random tests to fail.  Less than 2% of total Firefox users now use Firefox 52 so we're safe to drop testing for it.
2021-02-23 19:53:03 +00:00
John McLear 8364546e70 tests: fix importexport tests
The testing approach was redone to fix numerous issues:
  * Even if the tests had been working, none of them would have caught
    https://github.com/ether/etherpad-lite/issues/4808 because they
    didn't exercise the client-side import logic. Now they do.
  * Follow-up logic was not in the `helper.waitFor()` callback like it
    should have been. Now the code uses `async` and `await` to ensure
    proper execution order.
  * All `$.ajax()` calls used `async: false`. Now they're properly
    asynchronous.
  * The `helper.waitFor()` condition callbacks threw instead of
    returning false.
  * The string comparisons didn't allow for different attribute
    order (e.g., `<ol start="1" class="list-number1">` vs. `<ol
    class="list-number1" start="1">`). Now `Node.isEqualNode()` is
    used to reduce fragility. (`Node.isEqualNode()` is not perfect, so
    the tests are still a bit fragile: If class names or style strings
    are in a different order then `Node.isEqualNode()` will return
    false even if the nodes are semantically equivalent.)

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
2021-02-22 18:20:24 -05:00
Richard Hansen 3ca1589885 Revert "tests: fix importexport frontend tests (#4827)"
I'm going to split this into separate commits.

This reverts commit 9b03f8f6ab.
2021-02-22 18:19:48 -05:00
John McLear 0ac33d2085 tests: timeslider_follow increase test timeout for firefox. 2021-02-22 14:30:19 +00:00
Richard Hansen 773959ec57 CI: Delete old Sauce Connect logic 2021-02-22 11:25:21 +00:00
John McLear 9b03f8f6ab
tests: fix importexport frontend tests (#4827)
* CI: Leave log level at INFO for frontend tests

* CI: Disable frontend admin tests for non-admin workflow

* CI: Disable import/export rate limiting for frontend tests

* tests: fix importexport tests

The testing approach was redone to fix numerous issues:
  * Even if the tests had been working, none of them would have caught
    https://github.com/ether/etherpad-lite/issues/4808 because they
    didn't exercise the client-side import logic. Now they do.
  * Follow-up logic was not in the `helper.waitFor()` callback like it
    should have been. Now the code uses `async` and `await` to ensure
    proper execution order.
  * All `$.ajax()` calls used `async: false`. Now they're properly
    asynchronous.
  * The `helper.waitFor()` condition callbacks threw instead of
    returning false.
  * The string comparisons didn't allow for different attribute
    order (e.g., `<ol start="1" class="list-number1">` vs. `<ol
    class="list-number1" start="1">`). Now `Node.isEqualNode()` is
    used to reduce fragility. (`Node.isEqualNode()` is not perfect, so
    the tests are still a bit fragile: If class names or style strings
    are in a different order then `Node.isEqualNode()` will return
    false even if the nodes are semantically equivalent.)

Co-authored-by: Richard Hansen <rhansen@rhansen.org>

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
2021-02-22 10:40:38 +00:00
Richard Hansen 85231cb774 tests: More descriptive Sauce Labs name 2021-02-22 03:36:12 -05:00
John McLear 1b8cd0747d
Move vendor libraries to /vendors folder and exclude from LGTM 2021-02-21 15:07:39 +00:00
Richard Hansen 8ef0860e8b
tests: Restore `runnerBackend.sh` (#4803)
* tests: Restore `runnerBackend.sh`

`runnerBackend.sh` was deleted in commit
7dae5e3db8 but plugins still need it
until their GitHub workflow definitions have been updated.

Co-authored-by: John McLear <john@mclear.co.uk>
2021-02-17 04:35:57 -05:00
Richard Hansen 7dae5e3db8 tests: Use the supertest agent from `common.js` for backend tests 2021-02-16 21:13:35 -05:00
John McLear 2e92e8e9d0 tests: frontend test button increase timeouts 2021-02-16 16:38:59 +00:00
Richard Hansen bdd32f8915 Replace `node_modules/ep_etherpad-lite` with `src` 2021-02-16 10:35:05 +00:00
John McLear f59e0993a6
tests: test runner output HTML and CSS improvements to show duration (#4775) 2021-02-14 08:04:50 +00:00
Richard Hansen e674d9789e
express: Change `httpUptime` to `httpStartTime` (#4777)
It's better to provide a primitive value and let the consumer of the
metric do math if desired.

Co-authored-by: John McLear <john@mclear.co.uk>
2021-02-14 07:50:10 +00:00
John McLear e585d321f9
tests: timeouts for tests (#4773) 2021-02-13 19:00:06 +00:00
Richard Hansen f9ec49d7ac tests: Improve `/admin/settings` restart test 2021-02-13 10:02:28 +00:00
Richard Hansen 09e9c36098 tests: Accept async condition functions for `helper.waitFor()` 2021-02-13 08:12:53 +00:00
Richard Hansen 8dca4cb16f tests: Give `helper.waitFor()` timeout errors a useful stack trace 2021-02-13 08:12:53 +00:00
Richard Hansen 1c8a913411 lint: Delete unnecessary `eslint-disable-line` comment 2021-02-13 08:12:53 +00:00
Richard Hansen eb9d5bb470 lint: `src/tests/frontend/helper.js` and friends 2021-02-13 00:46:30 -05:00
John McLear fcd9adf20d
tests: admin test timeout increase, bugfix and removal of buggy test (#4762) 2021-02-12 20:04:42 +00:00
Richard Hansen 085ab452a0 chat tests: Give some time to process each of the 140 chat messages
This avoids overflowing socket.io's receive buffer.
2021-02-11 21:01:47 -05:00
Richard Hansen 020df75677 chat tests: Asyncify 2021-02-11 21:01:47 -05:00
Richard Hansen 0fae34009d chat tests: Delete unnecessary `expect()` 2021-02-11 21:01:47 -05:00
John McLear cc7f11560f
Fix OL list items not increasing (#4749)
* tests: additional test coverage for OL items

* parseInt means we can do a proper check

* tests: use ol check value test for #4748
2021-02-11 13:26:37 -05:00
Richard Hansen cb1116607e Revert "backend tests: change loglevel to WARN (#4514)"
Logging verbosity of the openapi handlers was turned down so GitHub
should be happier with INFO now. This makes it easier to troubleshoot
problems.

This reverts commit b98aaf4904.
2021-02-09 07:24:31 +00:00
Richard Hansen 714e099b2a tests: Fix accidental functions in `ordered_list.js`
This fixes a bug introduced in commit
e9bb2c410e.
2021-02-09 07:23:38 +00:00
Richard Hansen e9bb2c410e tests: Fix erroneous use of `waitForPromise()` in `ordered_list.js`
`waitForPromise()` should always be used with `await` (either directly
or with a later `await` on the returned Promise). In this case,
the condition should be immediately true so `waitForPromise()` is not
the right tool here.
2021-02-08 18:24:11 +00:00
John McLear 7baa0cda02
tests: disabled checks shouldnt change behavior (#4729) 2021-02-08 11:07:07 +00:00
John McLear 2b112ac851
tests: Admin Frontend Test Coverage(#4717)
Covers all frontend admin operations, runs separated in CI.
2021-02-07 11:32:57 +00:00
Richard Hansen 294f2a251f lint: Fix bugs and style issues introduced in PR #4718
This fixes issues introduced in commit
f8a19c4527.
2021-02-07 07:51:35 +00:00
Richard Hansen 0ff8274d2e tests: Fix `waitForPromise()` in `enter.js` 2021-02-07 07:13:04 +00:00
Richard Hansen c625c611d2 tests: delete `src/tests/frontend/specs/caret.js`
All of the tests in this file are commented out so this file does
nothing. We can uncomment the code and clean it up, but the approach
taken in these tests will never work: For security reasons, browsers
do not allow synthetic key events to perform the default
behavior (such as moving the carent when an arrow key is pressed).

There are two ways to test responses to navigation keys:
  * Use WebDriver to create "genuine" keyboard events.
  * Suppress the default behavior and implement caret movement
    ourselves. This is tremendously complicated, especially arrow
    up/down.
2021-02-07 07:02:23 +00:00
John McLear 1f0cb01110 tests: fix enter test 2021-02-07 07:00:12 +00:00
John McLear f8a19c4527
lint: lint and various fixes of frontend test specs 2021-02-07 06:39:03 +00:00
John McLear 4862d6fa9c
editor: fix enter key keep line in view (#4639) 2021-02-06 19:56:59 +00:00
Richard Hansen 8b28e00784 restructure: Prefix `bin/` and `tests/` with `src/`
This is a follow-up to commit
2ea8ea1275.
2021-02-05 21:52:08 +00:00
John McLear 2ea8ea1275 restructure: move bin/ and tests/ to src/
Also add symlinks from the old `bin/` and `tests/` locations to avoid
breaking scripts and other tools.

Motivations:

  * Scripts and tests no longer have to do dubious things like:

        require('ep_etherpad-lite/node_modules/foo')

    to access packages installed as dependencies in
    `src/package.json`.

  * Plugins can access the backend test helper library in a non-hacky
    way:

        require('ep_etherpad-lite/tests/backend/common')

  * We can delete the top-level `package.json` without breaking our
    ability to lint the files in `bin/` and `tests/`.

    Deleting the top-level `package.json` has downsides: It will cause
    `npm` to print warnings whenever plugins are installed, npm will
    no longer be able to enforce a plugin's peer dependency on
    ep_etherpad-lite, and npm will keep deleting the
    `node_modules/ep_etherpad-lite` symlink that points to `../src`.

    But there are significant upsides to deleting the top-level
    `package.json`: It will drastically speed up plugin installation
    because `npm` doesn't have to recursively walk the dependencies in
    `src/package.json`. Also, deleting the top-level `package.json`
    avoids npm's horrible dependency hoisting behavior (where it moves
    stuff from `src/node_modules/` to the top-level `node_modules/`
    directory). Dependency hoisting causes numerous mysterious
    problems such as silent failures in `npm outdated` and `npm
    update`. Dependency hoisting also breaks plugins that do:

        require('ep_etherpad-lite/node_modules/foo')
2021-02-04 17:15:08 -05:00