In the next commit Pierre will start adding tests for the docker build, and this
lays out the structure for doing that.
No functional changes.
The relevant TravisCI docs that motivates moving under a jobs section is
https://docs.travis-ci.com/user/build-matrix/
> There are two ways to specify multiple parallel jobs (what we call the build
> matrix) with a single .travis.yml configuration file:
>
> * combine a language-and-environment dependent set of configuration options to
> automatically create a matrix of all possible combinations. This is called
> matrix expansion. For example, the following configuration produces a build
> matrix that expands to 8 individual (2 * 2 * 2) jobs
> [...]
>
> * specify the exact combination of configurations you want in jobs.include.
> For example, if not all of those combinations are interesting, you can
> specify just the combinations you want
The dependency on java was introduced in 2012 (c021cf52d8) to start
Sauce-Connect from sauce labs.
Probably at the time it was a runtime dependency, but it is no longer the case
today. It is possible that java was already not needed when db003a1460 changed
from downloading Sauce-Connect-latest.zip to sc-latest-linux.tar.gz.
Moreover, I am quite sure tests/frontend/travis/sauce_tunnel.sh no longer works
today, because tests/frontend/travis/sauce_tunnel.sh downloads from an url that
gives HTTP/404 now: sc-latest-linux.tar.gz if no longer a valid file name, we
would need to explicitly download a specific version.
In the following commits Pierre is going to copy & modify some files.
This commit prepares the source files in order to minimize those differences,
so we can re-unify them as soon as possible.
No functional changes.
This fixes some security vulnerabilites, among them an arbitrary file overwrite.
The output of `npm audit` goes from this:
found 17 vulnerabilities (15 low, 2 high) in 13344 scanned packages
run `npm audit fix` to fix 6 of them.
1 vulnerability requires semver-major dependency updates.
10 vulnerabilities require manual review. See the full report for details.
To this:
found 5 vulnerabilities (3 low, 2 high) in 13370 scanned packages
1 vulnerability requires semver-major dependency updates.
4 vulnerabilities require manual review. See the full report for details.
Changelog:
- https://github.com/npm/cli/releases
6.13.4 (2019-12-11)
BUGFIXES
320ac9aee npm/bin-links#12 npm/gentle-fs#7 Do not remove global bin/man links inappropriately (@isaacs)
DEPENDENCIES
52fd21061 gentle-fs@2.3.0 (@isaacs)
d06f5c0b0 bin-links@1.1.6 (@isaacs)
6.13.3 (2019-12-09)
DEPENDENCIES
19ce061a2 bin-links@1.1.5 Properly normalize, sanitize, and verify bin entries in package.json.
59c836aae npm-packlist@1.4.7
fb4ecd7d2 pacote@9.5.11
5f33040 #476npm/pacote#22npm/pacote#14 fix: Do not drop perms in git when not root (isaacs, @darcyclarke)
6f229f7 sanitize and normalize package bin field (isaacs)
1743cb339 read-package-json@2.1.1
6.13.2 (2019-12-03)
BUG FIXES
4429645b3 #546 fix docs target typo (@richardlau)
867642942 #142 fix(packageRelativePath): fix 'where' for file deps (@larsgw)
d480f2c17 #527 Revert "windows: Add preliminary WSL support for npm and npx" (@craigloewen-msft)
e4b97962e #504 remove unnecessary package.json read when reading shrinkwrap (@Lighting-Jack)
1c65d26ac #501 fix(fund): open url for string shorthand (@ruyadorno)
ae7afe565 #263 Don't log error message if git tagging is disabled (@woppa684)
4c1b16f6a #182 Warn the user that it is uninstalling npm-install (@Hoidberg)
The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.
The "secure" flag is set if one of these is true:
1. we are directly serving Etherpad over SSL using the native nodejs
functionality, via the "ssl" options in settings.json
2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
proxy for terminating the SSL for us;
In this case, the user has to be instructed to properly set trustProxy: true
in settings.json, and the information wheter the application is over SSL or
not will be extracted from the X-Forwarded-Proto HTTP header.
Please note that this will not be compatible with applications being served over
http and https at the same time.
The change on webaccess.js amends 009b61b338, which did not work when the SSL
termination was performed by a reverse proxy.
Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecureCloses#3561.
The "io" cookie is created by socket.io, and its purpose is to offer an handle
to perform load balancing with session stickiness when the library falls back to
long polling or below.
In Etherpad's case, if an operator needs to load balance, he can use the
"express_sid" cookie, and thus "io" is of no use.
Moreover, socket.io API does not offer a way of setting the "secure" flag on it,
and thus is a liability.
Let's simply nuke it.
References:
https://socket.io/docs/using-multiple-nodes/#Sticky-load-balancinghttps://github.com/socketio/socket.io/issues/2276#issuecomment-147184662 (not totally true, actually, see above)
Before this change, the docker user had home in a directory it had no
permissions on. The inability of creating a cache directory in `$HOME`
prevented npm to work properly.
Additionally, the `node_modules` in the base working directory had its
owner set to root, preventing further changes.
With this change, the `etherpad` user has a home directory.
Additionally, `npm i` is now run by `etherpad` rather than the root
user; this way, it is possible to dynamically change the `node_modules`
content in day 2 operations.
Note that while switching to the `useradd` builtin, a conflict was
discovered with the GID 65534 that was previously used. This change is
changing the `etherpad` user's UID to 5001 to avoid said conflict. As a
consequence, a `chmod -R 5001:5001` must be run prior to attaching
volumes created from previous Etherpad versions.
Revision 5879037ddc fixed a security bug, but introduced a regression, where
on page load the js console showed:
ReferenceError: require is not defined
The reason was that the fix called require('../static/js/pad_utils') to load a
module at a time when require() was still not defined.
This change anticipates the loading of require-kernel, and manually loads
pad_utils.
The fix proposed in #3670 by aaron-costello, which seemed to do the right
thing, anticipating the configuration phase of require-kernel, did not work.
It had to be declined and replaced by this (less elegant) change.
This upgrade solves the high-severity vulnerabilities regarding
https-proxy-agent that were still present in 8e6bca456f.
The output of `npm audit` goes from this:
found 29 vulnerabilities (3 low, 26 high) in 13338 scanned packages
run `npm audit fix` to fix 4 of them.
1 vulnerability requires semver-major dependency updates.
24 vulnerabilities require manual review. See the full report for details.
To this:
found 5 vulnerabilities (3 low, 2 high) in 13338 scanned packages
1 vulnerability requires semver-major dependency updates.
4 vulnerabilities require manual review. See the full report for details.
Changelog:
- https://github.com/npm/cli/releases
6.13.1 (2019-11-18)
BUG FIXES
938d6124d #472 fix(fund): support funding string shorthand (@ruyadorno)
b49c5535b #471 should not publish tap-snapshot folder (@ruyadorno)
3471d5200 #253 Add preliminary WSL support for npm and npx (@infinnie)
3ef295f23 #486 print quick audit report for human output (@isaacs)
TESTING
dbbf977ac #278 added workflow to trigger and run benchmarks (@mikemimik)
b4f5e3825 #457 feat(docs): adding tests and updating docs to reflect changes in registry teams API. (@nomadtechie)
454c7dd60 #456 fix git configs for git 2.23 and above (@isaacs)
DEPENDENCIES
661d86cd2 make-fetch-happen@5.0.2 (@claudiahdz)
6.13.0 (2019-11-05)
NEW FEATURES
4414b06d9 #273 add fund command (@ruyadorno)
BUG FIXES
e4455409f #281 delete ps1 files on package removal (@NoDocCat)
cd14d4701 #279 update supported node list to remove v6.0, v6.1, v9.0 - v9.2 (@ljharb)
DEPENDENCIES
a37296b20 pacote@9.5.9
d3cb3abe8 read-cmd-shim@1.0.5
TESTING
688cd97be #272 use github actions for CI (@JasonEtco)
9a2d8af84 #240 Clean up some flakiness and inconsistency (@isaacs)
Added `rel="noreferrer"` to automatically generated links in the main pad window
as well as the chat window.
`rel="noreferrer"` is part of the HTML5 standard. While browser support isn't
100%, it's better than nothing. Future alternative solutions with wider browser
support, such as intermediary redirect pages, are unaffected by this change.
https://html.spec.whatwg.org/multipage/links.html#link-type-noreferrer
This commit was originally part of https://github.com/ether/etherpad-lite/pull/2498
With this change, the Dockerfile builds the Docker image from the code
checked out in the local filesystem, instead of downloading a revision
from git.
Implements #3657
This change amends eea99fe507.
https://docs.docker.com/engine/reference/builder/#user
USER <user>[:<group>] or
USER <UID>[:<GID>]
The USER instruction sets the user name (or UID) and optionally the user group
(or GID) to use when running the image and for any RUN, CMD and ENTRYPOINT
instructions that follow it in the Dockerfile.
The change that implemented #3648 (7c099fef5e) was incorrect, and resulted
in disabling every user at startup.
The problem was twofold:
1. _.filter() on an object returns an array of the object's enumerable values
and strips out the keys, see: https://stackoverflow.com/questions/11697702/how-to-use-underscore-js-filter-with-an-object
To filter an object, the function that needs to be used is _.pick();
2. The logic condition on userProperties.password was plain wrong (it should
have been an AND instead of an OR).
This change corrects 1) and 2), and writes more specific logs when something
goes wrong.
Closes#3661.
This change reverts c4918efc1b, and basically negates what was done for #3396,
but aligns better with current practices in the nodejs ecosystem.
Pragmatically speaking, this will allow users, if they want, to use
npm-force-resolutions (https://github.com/rogeriochaves/npm-force-resolutions)
to manually fix security vulnerabilities.
We had a problem for that (see #3598), and - given the fragmented nature of
the nodejs ecosystem - it is reasonable to expect more issues like that one,
so it's better to be prepared.
Closes#3659.
The previous attempt to directly release 1.8.0 had to be hold back, and indeed
1.8.0 was never tagged.
Since 1.8.0 contains many changes, let's do a prerelease instead.
Closes#3660
A Docker base image without version is a bit of a moving target. Buster-slim,
for example, is currently based on nodejs 12.
For now, let's base our official Docker image on nodejs 10 (an LTS, non at End
of Life, which we explicitly mention in the documentation).
Amends a9a3bf9bd2 and the corresponding PR #3646.
Without this, on nodejs 10 and 12 (and maybe 8, not tested), Etherpad failed to
start, throwing the following error:
[2019-10-22 19:01:01.439] [ERROR] console - exception thrown: Maximum call stack size exceeded
[2019-10-22 19:01:01.439] [INFO] console - RangeError: Maximum call stack size exceeded
at Function.[Symbol.hasInstance] (<anonymous>)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:194:14)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
at ReadStream (/opt/etherpad-lite/src/node_modules/graceful-fs/graceful-fs.js:195:28)
Fixes#3654.
This upgrade should be backward compatible, but still suffers form major
vulnerabilities in its https-proxy-agent transitive dependency (see
https://www.npmjs.com/advisories/1184).
Changelog:
- https://github.com/npm/cli/releases
6.12.0 (2019-10-08):
Now npm ci runs prepare scripts for git dependencies, and respects the
--no-optional argument. Warnings for engine mismatches are printed again.
Various other fixes and cleanups.
BUG FIXES
890b245dc #252 ci: add dirPacker to options (@claudiahdz)
f3299acd0 #257 npm.community#4792 warn message on engine mismatch
(@ruyadorno)
bbc92fb8f #259 npm.community#10288 Fix figgyPudding error in npm token
(@benblank)
70f54dcb5 #241 doctor: Make OK more consistent (@gemal)
FEATURES
ed993a29c #249 Add CI environment variables to user-agent (@isaacs)
f6b0459a4 #248 Add option to save package-lock without formatting Adds a new
config --format-package-lock, which defaults to true.
(@bl00mber)
DEPENDENCIES
0ca063c5d npm-lifecycle@3.1.4:
fix: filter functions and undefined out of makeEnv (@isaacs)
5df6b0ea2 libcipm@4.0.4:
fix: pack git directories properly (@claudiahdz)
respect no-optional argument (@cruzdanilo)
7e04f728c tar@4.4.12
5c380e5a3 stringify-package@1.0.1 (@isaacs)
62f2ca692 node-gyp@5.0.5 (@isaacs)
0ff0ea47a npm-install-checks@3.0.2 (@isaacs)
f46edae94 hosted-git-info@2.8.5 (@isaacs)
TESTING
44a2b036b #262 fix root-ownership race conditions in meta-test (@isaacs)
6.11.3 (2019-09-03):
Fix npm ci regressions and npm outdated depth.
BUG FIXES
235ed1d28 #239 Don't override user specified depth in outdated. Restores
ability to update packages using --depth as suggested by npm audit. (@G-Rath)
1fafb5151 #242 npm.community#9586 Revert "install: do not descend into
directory deps' child modules" (@isaacs)
cebf542e6 #243 npm.community#9720 ci: pass appropriate configs for file/dir
modes (@isaacs)
DEPENDENCIES
e5fbb7ed1 read-cmd-shim@1.0.4 (@claudiahdz)
23ce65616 npm-pick-manifest@3.0.2 (@claudiahdz)
6.11.2 (2019-08-22):
Fix a recent Windows regression, and two long-standing Windows bugs. Also,
get CI running on Windows, so these things are less likely in the future.
DEPENDENCIES
9778a1b87 cmd-shim@3.0.3: Fix regression where shims fail to preserve exit
code (@isaacs)
bf93e91d8 npm-package-arg@6.1.1: Properly handle git+file: urls on Windows
when a drive letter is included. (@isaacs)
BUGFIXES
6cc4cc66f escape args properly on Windows Bash Despite being bash, Node.js
running on windows git mingw bash still executes child processes
using cmd.exe. As a result, arguments in this environment need to
be escaped in the style of cmd.exe, not bash. (@isaacs)
TESTS
291aba7b8 make tests pass on Windows (@isaacs)
fea3a023a travis: run tests on Windows as well (@isaacs)
6.11.1 (2019-08-20):
Fix a regression for windows command shim syntax.
37db29647 cmd-shim@3.0.2 (@isaacs)
v6.11.0 (2019-08-20):
A few meaty bugfixes, and introducing peerDependenciesMeta.
FEATURES
a12341088 #224 Implements peerDependenciesMeta (@arcanis)
2f3b79bba #234 add new forbidden 403 error code (@claudiahdz)
BUGFIXES
24acc9fc8 and 45772af0d #217 npm.community#8863 npm.community#9327 do not
descend into directory deps' child modules, fix shrinkwrap files
that inappropriately list child nodes of symlink packages (@isaacs
and @salomvary)
50cfe113d #229 fixed typo in semver doc (@gall0ws)
e8fb2a1bd #231 Fix spelling mistakes in CHANGELOG-3.md (@XhmikosR)
769d2e057 npm/uid-number#7 Better error on invalid --user/--group configs.
This addresses the issue when people fail to install binary
packages on Docker and other environments where there is no
'nobody' user. (@isaacs)
8b43c9624 nodejs/node#28987 npm.community#6032 npm.community#6658
npm.community#6069 npm.community#9323 Fix the regression where
random config values in a .npmrc file are not passed to lifecycle
scripts, breaking build processes which rely on them. (@isaacs)
8b85eaa47 save files with inferred ownership rather than relying on SUDO_UID
and SUDO_GID. (@isaacs)
b7f6e5f02 Infer ownership of shrinkwrap files (@isaacs)
54b095d77 #235 Add spec to dist-tag remove function (@theberbie)
DEPENDENCIES
dc8f9e52f pacote@9.5.7: Infer the ownership of all unpacked files in
node_modules, so that we never have user-owned files in root-owned
folders, or root-owned files in user-owned folders. (@isaacs)
bb33940c3 cmd-shim@3.0.0:
9c93ac3 #2 npm#3380 Handle environment variables properly (@basbossink)
2d277f8 #25#36#35 Fix 'no shebang' case by always providing $basedir
in shell script (@igorklopov)
adaf20b #26 Fix $* causing an error when arguments contain parentheses
(@satazor)
49f0c13 #30 Fix paths for MSYS/MINGW bash (@dscho)
51a8af3 #34 Add proper support for PowerShell (@ExE-Boss)
4c37e04 #10 Work around quoted batch file names (@isaacs)
a4e279544 npm-lifecycle@3.1.3 (@isaacs):
fail properly if uid-number raises an error
7086a1809 libcipm@4.0.3 (@isaacs)
8845141f9 read-package-json@2.1.0 (@isaacs)
51c028215 bin-links@1.1.3 (@isaacs)
534a5548c read-cmd-shim@1.0.3 (@isaacs)
3038f2fd5 gentle-fs@2.2.1 (@isaacs)
a609a1648 graceful-fs@4.2.2 (@isaacs)
f0346f754 cacache@12.0.3 (@isaacs)
ca9c615c8 npm-pick-manifest@3.0.0 (@isaacs)
b417affbf pacote@9.5.8 (@isaacs)
TESTS
b6df0913c #228 Proper handing of /usr/bin/node lifecycle-path test (@olivr70)
aaf98e88c npm-registry-mock@1.3.0 (@isaacs)
This upgrade should be backward compatible.
Changelog:
- https://github.com/expressjs/session/blob/master/HISTORY.md
1.17.0 / 2019-10-10
deps: cookie@0.4.0
Add SameSite=None support
deps: safe-buffer@5.2.0
1.16.2 / 2019-06-12
Fix restoring cookie.originalMaxAge when store returns Date
deps: parseurl@~1.3.3
This upgrade should be backward compatible.
Changelogs:
- https://expressjs.com/en/changelog/4x.html
- https://github.com/expressjs/express/blob/master/History.md#4171--2019-05-25
4.17.1 - Release date: 2019-05-25
The 4.17.1 patch release includes one bug fix:
The change to the res.status() API has been reverted due to causing
regressions in existing Express 4 applications.
4.17.0 - Release date: 2019-05-16
The 4.17.0 minor release includes bug fixes and some new features, including:
The express.raw() and express.text() middleware have been added to provide
request body parsing for more raw request payloads. This uses the
expressjs/body-parser module module underneath, so apps that are currently
requiring the module separately can switch to the built-in parsers.
The res.cookie() API now supports the "none" value for the sameSite option.
When the "trust proxy" setting is enabled, the req.hostname now supports
multiple X-Forwarded-For headers in a request.
Starting with this version, Express supports Node.js 10.x and 12.x.
The res.sendFile() API now provides and more immediate and easier to
understand error when a non-string is passed as the path argument.
The res.status() API now provides and more immediate and easier to
understand error when null or undefined is passed as the argument.
This makes the package smaller.
Before this change:
added 981 packages from 1497 contributors [...]
53M etherpad-lite-win.zip
After this change:
added 734 packages from 1043 contributors [...]
43M etherpad-lite-win.zip
Closes#3651.
Nodejs 8 will be EOLed on December 31th, 2019 (https://github.com/nodejs/Release).
This means any future Etherpad version released from 2020 on should require at
least the next LTS (10.13.0). Let's keep some margin and decide that the first
Etherpad version dropping node 8 compatibility will be 1.8.3.
Closes#3650.
Disclosure of vulnerabilities should be handled by a dedicated, multi-person
group, which for now does not exist. I take charge (hopefully temporarily) of
being that point of contact for now.
Tried to simplify the document, to reduce unneded info, and to use a less
informal language.
For example, the introductory links describing git made sense 10 years ago.
Today they are not needed to understand what Etherpad is.
And mercurial was always better than git, anyways :)
The mailing list and the IRC channel seem pretty dead by now. Let's just
move everything to Github issues, which was the de facto situation anyways.
About the donation links: I am the maintainer, and I do not know the identity of
the owners of the donation links, so it is correct to remove them. The same was
done on the website three months ago:
https://github.com/ether/ether.github.com/commit/d4ef04605da5
