Commit graph

5493 commits

Author SHA1 Message Date
John McLear
61f0b5db0b tests: enable sauce labs tests again 2020-03-23 23:51:51 +01:00
John McLear
88d05d3e66 tests: configure the OS/Browsers versions on which tests will be run
For all the supported operating systems, the latest version of each browser is
supported, with the Exception of Win 8 / IE 10, which has an explicit version
pinned.
2020-03-23 23:51:51 +01:00
John McLear
3085f46f32 tests: explicit tunnel version URI in sauce_tunnel.sh
For an explanation, see bf0bb58c70.
2020-03-23 23:51:51 +01:00
muxator
f6436be486 tests: quote bash variables in sauce_tunnel.sh 2020-03-23 23:51:51 +01:00
Chocobozzz
43ca44e444 minify: make conditional requests work. No HTTP/304 was ever generated and file were reminified uselessly.
By specification [0], the if-modified-since HTTP header sent by browsers does
not include milliseconds.

Before this patch, let's say a file was generate at time:
    t_real-file         = 2020-03-22T02:15:53.548Z (note the fractional seconds)

When issuing a conditional request, the browser would truncate the fractional
part, and only request an if-modified-since with this contents:
    t_if-modified-since = 2020-03-22T02:15:53.000Z

The minify() function would return HTTP/304 only if
t_if-modified-since >= t_real-file, but this would never be true unless, by
chance, a file was generated at XX.000Z.

This resulted in that file being minified/compressed again and resent to the
client for no reason. After this patch, the server correctly responds with
HTTP/304 without doing any computation, and the browser uses the cached file.

[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
2020-03-23 11:34:01 +01:00
translatewiki.net
2bce9f5c38 Localisation updates from https://translatewiki.net. 2020-03-23 14:58:30 +01:00
John McLear
71fa852dfb dependencies: update cheerio 0.20.0 -> 0.22.0
No variation in # of security vulnerabilities.

The output of `npm audit` goes from this:
  found 7 vulnerabilities (5 low, 2 high) in 13881 scanned packages
    7 vulnerabilities require manual review. See the full report for details.

To this:
  found 7 vulnerabilities (5 low, 2 high) in 13795 scanned packages
    7 vulnerabilities require manual review. See the full report for details.
2020-03-15 14:36:42 +00:00
muxator
b075d1fd7c dependencies: update mocha 5.2.0 -> 7.1.1
This is a dev dependency. No impact in production.
After this upgrade the backend tests ("cd src; npm test") still pass.

This fixes 2 reported security vulnerabilities.

The output of `npm audit` goes from this:
  found 9 vulnerabilities (7 low, 2 high) in 13707 scanned packages
    run `npm audit fix` to fix 1 of them.
    1 vulnerability requires semver-major dependency updates.
    7 vulnerabilities require manual review. See the full report for details.

To this:
  found 7 vulnerabilities (5 low, 2 high) in 13881 scanned packages
    7 vulnerabilities require manual review. See the full report for details.
2020-03-22 00:33:22 +01:00
muxator
dabc1cae56 dependencies: update nyc 14.1.0 -> 15.0.0
This is a dev dependency. No impact in production.
After this upgrade the backend tests ("cd src; npm test") still pass.

This fixes 1 reported security vulnerability.

The output of `npm audit` goes from this:
  found 10 vulnerabilities (8 low, 2 high) in 13390 scanned packages
    run `npm audit fix` to fix 2 of them.
    1 vulnerability requires semver-major dependency updates.
    7 vulnerabilities require manual review. See the full report for details.

To this:
  found 9 vulnerabilities (7 low, 2 high) in 13707 scanned packages
    run `npm audit fix` to fix 1 of them.
    1 vulnerability requires semver-major dependency updates.
    7 vulnerabilities require manual review. See the full report for details.
2020-03-22 00:33:22 +01:00
muxator
4cbbf9893f dependencies: update supertest 3.0.0 -> 4.0.2
This is a dev dependency. No impact in production. After this upgrade the
backend tests ("cd src; npm test") still pass.
2020-03-22 00:33:22 +01:00
muxator
6a55e7e03f dependencies: upgrade npm 6.13.4 -> 6.14.3
This fixes 107 reported security vulnerabilites.

The output of `npm audit` goes from this:
  found 117 vulnerabilities (115 low, 2 high) in 13365 scanned packages
    run `npm audit fix` to fix 6 of them.
    1 vulnerability requires semver-major dependency updates.
    110 vulnerabilities require manual review. See the full report for details.

To this:
  found 10 vulnerabilities (8 low, 2 high) in 13390 scanned packages
    run `npm audit fix` to fix 2 of them.
    1 vulnerability requires semver-major dependency updates.
    7 vulnerabilities require manual review. See the full report for details.

The changelog for npm can be read at https://github.com/npm/cli/releases
2020-03-22 00:33:22 +01:00
muxator
40d3d400a1 fonts: remove the execution bit from the font files
Monsterrat was added in 8bd8be97b9
Roboto Mono was added in 7029248a76
2020-03-22 00:33:22 +01:00
muxator
ca35488273 minify: upgrade clean-css 3.4.19 -> 4.2.3. Adapt to the semver major change
CleanCSS 3.4.19 had a Regex Denial of Service vulnerability and has to be
updated. The major version bump requires the following changes:

1. Disabling rebase is necessary because otherwise the URLs for the web fonts
   become wrong;

   EXAMPLE 1:
       /static/css/src/static/font/fontawesome-etherpad.woff
     instead of
       /static/font/fontawesome-etherpad.woff

   EXAMPLE 2 (this is more surprising):
       /p/src/static/font/opendyslexic.otf
     instead of
       /static/font/opendyslexic.otf

2. CleanCSS.minify() can either receive a string containing the CSS, or an array
   of strings. In that case each array element is interpreted as an absolute
   local path from which the CSS file is read.

   In version 4.x, CleanCSS API was simplified, eliminating the relativeTo
   parameter, and thus we cannot use our already loaded "content" argument, but
   we have to wrap the absolute path to the CSS in an array and ask the library
   to read it by itself.

Fixes #3616.
2020-03-22 00:33:22 +01:00
muxator
2c44a0f71e package-lock: preliminary recomputation before updating a dependency
The next commit will update a dependency in package.json. This commit only
refreshes package-lock.json recalculating it at the current date.
In this way we will be sure that any modifications in package-lock.json in the
next commit will be only due to the package.json change.

Should this commit introduce any regression, we would know for certain that one
of our dependencies is not correctly honoring their semver promises.
2020-03-22 00:00:45 +01:00
muxator
a1978d2245 Minify: on errors, generate logs instead of simply silencing them
This will help when we'll have to update clean-css.
2020-03-20 22:46:39 +01:00
John McLear
c316402d86 PadMessageHandler: use a predefined color when authorInfo.colorId is not defined
For some reason authorInfo is sometimes null, and therefore it is not possible
to get colorId from it.

This resulted in the following stack trace:
    [2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'colorId' of null
    at <BASEDIR>/src/node/handler/PadMessageHandler.js:1199:37
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Promise.all (index 0)
    at async handleClientReady (<BASEDIR>/src/node/handler/PadMessageHandler.js:1171:5)
    [2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 76)
    [2020-03-16 09:27:19.034] [WARN] message - Dropped message, USERINFO_UPDATE Session not ready.[object Object]

Which is due to a bug in Etherpad that we are not going to solve now.

As a workaround, when this happens, let's set the username to "Anonymous" (if
it is not already set), and colorId to the fixed value "#daf0b2". Warning
messages are written in the logs to signal this condition.

This is no definitive solution, but fixes #3612 (via a workaround).
2020-03-20 22:32:06 +01:00
John McLear
b480416375 toolbar: the "star" button no longer disappears when visiting a read-only pad
Before this patch, visiting the read-only URL for a random pad would remove
the "Save Revision" (the "star" icon) from all the other RW pads. The only way
to make it appear again was to restart the server.

This change does not fix the underlying bug: after visiting a read only link
the "star" button would still disapper, but it is explictly reinserted via an
ad-hoc condition.

Fixes #3702
2020-03-19 22:42:22 +01:00
muxator
840b4a0988 contentcollector: backed out changeset 3292429ab3
That commit (merged recently with PR #3622) was part of an effort to fix #3620,
but introduced a very bad bug that broke the cursor behaviour when pressing
space, making the program unusable.

This commit completes the revert of PR #3622 and fixes #3728.

--HG--
branch : revert-3622
2020-03-19 02:53:41 +01:00
muxator
c382ba35c9 tests: backed out changeset 155a895604
This was a preparatory commit for 3292429ab3 (which introduced a bug, see
issue #3728) and modified the tests for issue #3620.

Commit 155a895604 per se did not introduce any bugs, but was difficult to
inspect because of its size. For this, the corresponding PR (#3622) should not
have been accepted.

--HG--
branch : revert-3622
2020-03-19 02:53:41 +01:00
John McLear
8261229323 pass file ending 2020-03-17 13:08:46 +00:00
Tudor Constantin
28102d8e1f ImportHandler: importing files with unknown extension work again when allowUnknownFileEnds is true
By specification, when settings.allowUnknownFileEnds is true and the user tries
to import a file with an unknown extension (this includes no extension),
Etherpad tries to import it as txt.

This broke in Etherpad 1.8.0, that abruptly terminates the processing with an
UnhandledPromiseRejectionWarning.

This patch restores the intended behaviour, and allows to import as text a file
with an unknown extension (on no extension).

In order to catch the UnhandledPromiseRejectionWarning we had to use
fsp_rename(), which is declared earlier in the code and is promised based
instead of fs.rename(), which is callback based.

Fixes #3710.
2020-03-17 12:41:18 +01:00
John McLear
a0579c90db APIHandler: return HTTP/404 when non existing API methods are invoked
Before this change, invoking a non existing API method would return an HTTP/200
response with a JSON payload {"code":3,"message":"no such function"}.

This commit changes the HTTP status code to 404, leaving the payload as-is.

Before:
   curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"
  < HTTP/1.1 200 OK
  < X-Powered-By: Express
  [...]
  {"code":3,"message":"no such function","data":null}

After:
   curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"

   < HTTP/1.1 404 OK
   < X-Powered-By: Express
   [...]
   {"code":3,"message":"no such function","data":null}

Fixes #3546.
2020-03-15 09:26:44 +00:00
Sebastian Castro
0d61d6bb13 ui: on mobile, move the right toolbar to the bottom and make the top toolbar scrollable
Before this change there was always a single toolbar on the top, with both
Colibris and the legacy skin. When the screen size was reduced:

- the legacy skin would compact the icons in the toolbar (this was fine,
  indeed);
- Colibris would hide some formatting icons. This would hamper the functionality
  for mobile users.

After this change both the skins work in the same way, which is the following:
- when the screen gets smaller the right toolbar (the one with "export",
  "timeslider", and other buttons) goes to the bottom of the screen;
- when there are many icons, the toolbar keeps all of them, and to see them the
  user must drag the toolbar.

This behaviour will probably be changed before release, opting instead to show
a "+" button when there is an overflow, since this appears to be more
discoverable (see the discusison in #3697).

Do not tested with custom toolbar elements (toolbar.left and toolbar.right
configuration items in settings.json).

Fixes #3697.
2020-03-12 18:47:12 +01:00
translatewiki.net
6323f59c3a Localisation updates from https://translatewiki.net. 2020-03-16 15:41:54 +01:00
Tom Briles
3292429ab3 trim text entries upon import. Fixes: #3620 2020-03-15 11:35:08 +00:00
Tom Briles
155a895604 Test for Issue-3620: space in the HTML for an unordered list creates an extra list item 2020-03-15 11:35:08 +00:00
John McLear
467fc11b72 fix 2020-03-14 21:58:26 +01:00
translatewiki.net
bb868beb9c Localisation updates from https://translatewiki.net. 2020-02-24 15:39:25 +01:00
translatewiki.net
5773e6cea0 Localisation updates from https://translatewiki.net. 2020-02-20 15:58:56 +01:00
John McLear
74fa47e295
Pretty sure you need elevated command prompt in windows..
To do anything nowadays.
2020-02-14 23:36:13 +00:00
translatewiki.net
7950d336eb Localisation updates from https://translatewiki.net. 2020-02-10 14:50:52 +01:00
translatewiki.net
ffcf22e4ac Localisation updates from https://translatewiki.net. 2020-01-30 15:49:13 +01:00
translatewiki.net
de3a677df5 Localisation updates from https://translatewiki.net. 2020-01-20 11:32:11 +01:00
translatewiki.net
bac58a7391 Localisation updates from https://translatewiki.net. 2020-01-13 16:08:24 +01:00
Sebastian Castro
6d4ea36646 skin colibris: Fix table of content with ep_resizable_bar 2020-01-03 15:35:12 -03:00
muxator
3b24c97d1e db/SecurityManager.js: accessing without session a public group pad no longer causes a crash
Steps to reproduce (via HTTP API):
1. create a group via createGroup()
2. create a group pad inside that group via createGroupPad()
3. make that pad public calling setPublicStatus(true)
4. access the pad via a clean web browser (with no sessions)
5. UnhandledPromiseRejectionWarning: apierror: sessionID does not exist

This was due to an overlook in 769933786c: "apierror: sessionID does not
exist" may be a legal condition if we are also visiting a public pad. The
function that could throw that error was sessionManager.getSessionInfo(), and
thus it needed to be inside the try...catch block.

Please note that calling getText() on the pad always return the pad contents,
*even for non-public pads*, because the API bypasses the security checks and
directly talks to the DB layer.

Fixes #3600.
2019-12-26 00:30:43 +01:00
Pierre Prinetti
0b3cf7cc96 docker: Add Run with volume example
Supersedes https://github.com/ether/etherpad-lite/pull/3631

Co-authored-by: RaymondCavallaro <RaymondCavallaro@users.noreply.github.com>
2019-12-25 00:48:30 +01:00
Pierre Prinetti
92f07a544b ci: test basic application response of the docker build
Note by muxator:
This commit introduced a copied & modified version of the testing files
loadSettings.js and pad.js.

It's Christmas night, and we want to shipt this feature, so I merged it anyway,
adding a note in both the original and copied files so that hopefully someone
in the distant future is going to merge them back again.
2019-12-25 00:28:38 +01:00
Pierre Prinetti
69fd393708 ci: test the Dockerfile
Add a `Test Dockerfile` job to Travis that checks the `docker build` exit
code.
More useful tests can be added later.
2019-12-25 00:28:38 +01:00
muxator
5b4bca2c6e ci: move the existing test under a jobs section, so we can add others later
In the next commit Pierre will start adding tests for the docker build, and this
lays out the structure for doing that.
No functional changes.

The relevant TravisCI docs that motivates moving under a jobs section is
https://docs.travis-ci.com/user/build-matrix/

> There are two ways to specify multiple parallel jobs (what we call the build
> matrix) with a single .travis.yml configuration file:
>
> * combine a language-and-environment dependent set of configuration options to
>   automatically create a matrix of all possible combinations. This is called
>   matrix expansion. For example, the following configuration produces a build
>   matrix that expands to 8 individual (2 * 2 * 2) jobs
>   [...]
>
> * specify the exact combination of configurations you want in jobs.include.
>   For example, if not all of those combinations are interesting, you can
>   specify just the combinations you want
2019-12-25 00:28:38 +01:00
muxator
bf0bb58c70 ci: no need to include java
The dependency on java was introduced in 2012 (c021cf52d8) to start
Sauce-Connect from sauce labs.

Probably at the time it was a runtime dependency, but it is no longer the case
today. It is possible that java was already not needed when db003a1460 changed
from downloading Sauce-Connect-latest.zip to sc-latest-linux.tar.gz.

Moreover, I am quite sure tests/frontend/travis/sauce_tunnel.sh no longer works
today, because tests/frontend/travis/sauce_tunnel.sh downloads from an url that
gives HTTP/404 now: sc-latest-linux.tar.gz if no longer a valid file name, we
would need to explicitly download a specific version.
2019-12-25 00:28:38 +01:00
muxator
cffa3e0a5d ci: trivial reformatting in preparation for next commits.
This is just needed to slim up the diffs for the next commits.
Non functional changes.
2019-12-25 00:28:38 +01:00
muxator
fe0cf4bdb0 tests: reorganize some files, because we are going to copy & paste them.
In the following commits Pierre is going to copy & modify some files.
This commit prepares the source files in order to minimize those differences,
so we can re-unify them as soon as possible.

No functional changes.
2019-12-25 00:28:38 +01:00
muxator
5bcc5a3be0 windows: bump the node version included in the prebuilt package: 10.16.3 -> 10.18.0
This is the latest version as of today.
2019-12-18 02:00:08 +01:00
muxator
140d5c4b41 dependencies: upgrade npm 6.13.1 -> 6.13.4
This fixes some security vulnerabilites, among them an arbitrary file overwrite.


The output of `npm audit` goes from this:
  found 17 vulnerabilities (15 low, 2 high) in 13344 scanned packages
    run `npm audit fix` to fix 6 of them.
    1 vulnerability requires semver-major dependency updates.
    10 vulnerabilities require manual review. See the full report for details.

To this:
  found 5 vulnerabilities (3 low, 2 high) in 13370 scanned packages
    1 vulnerability requires semver-major dependency updates.
    4 vulnerabilities require manual review. See the full report for details.


Changelog:
- https://github.com/npm/cli/releases


6.13.4 (2019-12-11)
    BUGFIXES
    320ac9aee npm/bin-links#12 npm/gentle-fs#7 Do not remove global bin/man links inappropriately (@isaacs)

    DEPENDENCIES
    52fd21061 gentle-fs@2.3.0 (@isaacs)
    d06f5c0b0 bin-links@1.1.6 (@isaacs)

6.13.3 (2019-12-09)
    DEPENDENCIES
    19ce061a2 bin-links@1.1.5 Properly normalize, sanitize, and verify bin entries in package.json.
    59c836aae npm-packlist@1.4.7
    fb4ecd7d2 pacote@9.5.11
        5f33040 #476 npm/pacote#22 npm/pacote#14 fix: Do not drop perms in git when not root (isaacs, @darcyclarke)
        6f229f7 sanitize and normalize package bin field (isaacs)
    1743cb339 read-package-json@2.1.1

6.13.2 (2019-12-03)
    BUG FIXES
    4429645b3 #546 fix docs target typo (@richardlau)
    867642942 #142 fix(packageRelativePath): fix 'where' for file deps (@larsgw)
    d480f2c17 #527 Revert "windows: Add preliminary WSL support for npm and npx" (@craigloewen-msft)
    e4b97962e #504 remove unnecessary package.json read when reading shrinkwrap (@Lighting-Jack)
    1c65d26ac #501 fix(fund): open url for string shorthand (@ruyadorno)
    ae7afe565 #263 Don't log error message if git tagging is disabled (@woppa684)
    4c1b16f6a #182 Warn the user that it is uninstalling npm-install (@Hoidberg)
2019-12-18 01:17:35 +01:00
translatewiki.net
b6105d8c75 Localisation updates from https://translatewiki.net. 2019-12-16 15:54:40 +01:00
muxator
70bc71c0c3 skins: make "colibris" the default skin for new installations
Colibris skin was first introduced in 1.7.5 and received some bugfixes in 1.8.0.
It is now time to make it the default for new installs.
2019-12-08 00:32:03 +01:00
muxator
1dfd52bcce release: prepare for 1.8.0 2019-12-07 18:55:07 +01:00
muxator
a817acbbcc security: when served over https, set the "secure" flag for "express_sid" and "language" cookie
The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.

The "secure" flag is set if one of these is true:

1. we are directly serving Etherpad over SSL using the native nodejs
   functionality, via the "ssl" options in settings.json

2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
   proxy for terminating the SSL for us;
   In this case, the user has to be instructed to properly set trustProxy: true
   in settings.json, and the information wheter the application is over SSL or
   not will be extracted from the X-Forwarded-Proto HTTP header.

Please note that this will not be compatible with applications being served over
http and https at the same time.

The change on webaccess.js amends 009b61b338, which did not work when the SSL
termination was performed by a reverse proxy.

Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecure

Closes #3561.
2019-12-07 04:36:01 +01:00
muxator
b82816c774 express: reformat session configuration in preparation for the next commit
No functional changes.
2019-12-07 04:22:54 +01:00