ueberDB2 can return either undefined or null for a missing key, depending on
which DB driver is used. This patch changes the promise version of the API so
that it will always return null.
some code chunks previously used `async.parallel` but if you
use `await` that forces them to be run serially. Instead,
you can initiate the operation (getting a Promise) and then
_later_ `await` the result of that Promise.
If you use `await` inside a loop it makes the loop inherently serial.
If you omit the `await` however, the tasks will all start but the loop
will finish while the tasks are still being scheduled.
So, to make a set of tasks run in parallel but then have the
code block after the loop once all the tasks have been completed
you have to get an array of Promises (one for each iteration) and
then use `Promise.all()` to wait for those promises to be resolved.
Using `Array#map` is a convenient way to go from an array of inputs
to the require array of Promises.
NB1: needs additional review and testing - no abiword available on my test bed
NB2: in ImportHandler.js, directly delete the file, and handle the eventual
error later: checking before for existence is prone to race conditions,
and does not handle any errors anyway.
- removed possible issue with failing to sanitize `padName` if `padId` was also
supplied
- removed unnecessary `try` block
- simplified API and function name matching tests
Also converted the handler functions that depend on checkAccess() into async
functions too.
NB: this commit needs specific attention to it because it touches a lot of
security related code!
This patch also contains significant refactoring relating to error checking of
arguments supplied to the functions (e.g. rev) facilitated by use of `throw`
instead of nodeback errors.
1. This module was not migrated to Promises, because it is only used via
express-session, which can't actually use promises anyway.
2. all(), clear() and length() depend on the presence of the `db.forEach()`
function, which in ueberdb2 doesn't even exist.
Fortunately those three methods are optional, so I made their existence
conditional on the presence of `db.forEach`.
3. in SessionStore.clear(), replaced a call to db.db.remove() with db.remove()
Changed two occurrences of:
process.nextTick(function() {
if (fn) fn();
});
with
if (fn) {
process.nextTick(fn);
}
i.e. such that no function even gets added to the `nextTick` queue unless
there's actually a function to be called.
Extracted from Ray's work.
`getPadAccess()` (src/node/padaccess.js) is now "promise only", resolving to
`true` or `false` as appropriate, and throwing an exception if there's an
error.
The two call sites (padreadonly.js and importexport.js) updated to match.
Use real `async` instead of async.js where applicable.
The `getPluginTests()` function was never truly async anyway because it only
contains calls to synchronous `fs` modules.
Since this code can end up loaded in browsers when using client side plugins,
avoid use of ES6 syntax features such as arrow functions until MSIE support is
finally dropped.
This change is in preparation of the future async refactoring by Ray. It tries
to extract as many changes in boolean conditions as possible, in order to make
more evident identifying eventual logic bugs in the future work.
This proved already useful in at least one case.
BEWARE: this commit exposes an incoherency in the DB API, in which, depending
on the driver used, some functions can return null or undefined. This condition
will be externally fixed by the final commit in this series ("db/DB.js: prevent
DB layer from returning undefined"). Until that commit, the code base may have
some bugs.
This change extracts the grammar correction performed on the async branch,
anticipating them in a single commit. It cannot be folded with the previous
one, as it is not purely cosmetic.
This change is only cosmetic. Its aim is do make it easier to understand the
async changes that are going to be merged later on. It was extracted from the
original work from Ray Bellis.
To verify that nothing has changed, you can run the following command on each
file touched by this commit:
npm install uglify-es
diff --unified <(uglify-js --beautify bracketize <BEFORE.js>) <(uglify-js --beautify bracketize <AFTER.js>)
This is a complete script that does the same automatically (works from a
mercurial clone):
```bash
#!/usr/bin/env bash
set -eu
REVISION=<THIS_REVISION>
PARENT_REV=$(hg identify --rev "${REVISION}" --template '{p1rev}')
FILE_LIST=$(hg status --no-status --change ${REVISION})
UGLIFYJS="node_modules/uglify-es/bin/uglifyjs"
for FILE_NAME in ${FILE_LIST[@]}; do
echo "Checking ${FILE_NAME}"
diff --unified \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${PARENT_REV}" "${FILE_NAME}")) \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${REVISION}" "${FILE_NAME}"))
done
```
This is documented to be more performant.
The substitution was made on frontend code, too (i.e., the one in /static),
because Date.now() is supported since IE 9, and we are life supporting only
IE 11.
Commands:
find . -name *.js | xargs sed --in-place "s/new Date().getTime()/Date.now()/g"
find . -name *.js | xargs sed --in-place "s/(new Date()).getTime()/Date.now()/g"
Not done on jQuery.
The guard condition on count being non negative and < 100 used the wrong
boolean operator. In its form it was impossible.
This error was introduced in 2013, in 5592c4b0fe.
Fixes#3499
The HTTP API doesn't ever omit arguments, it always passes `undefined` for a
parameter that wasn't supplied in the request.
The functions that were simplified are:
- getRevisionChangeset()
- getText()
- getHTML()
- saveRevision()
The only function still supporting optional arguments is getPadSafe(), which is
only called from this module.
This function was simulating two overloads:
1. copy(destinationID, force, callback)
2. copy(destinationID, callback), in this case "force" would be assumed false
But all the call sites always used the version with arity 3.
Thus, we can remove that optionality and always assume that the funcion will be
called with three parameters. This will simplify future work.
Moving classes to html tag so it can be used to style other part of template depending on plugins like #users, #chat etc...
Rename plugin class with "plugin-" prefix, because there were conflicts with some plugins using the same .ep_font_color class to apply css rules
- path.exists() is no longer part of nodejs
- fs.exists() is deprecated (as of nodejs >= 8)
- checking a file for existence before using it is open to raca condition. It is
preferable to go ahead and use the file, and eventually handle the error
- we can afford two simple synchronous fs operations here
Next version will be Etherpad 1.8. As planned in #3424, we are going to require
NodeJS >=8.9.0 and npm >= 6.4.
This commit implements that change and updates documentation and scripts.
Subsequent changes will get rid of old idioms, dating back to node < 0.7, that
still survive in the code.
Once migrated to NodeJS 8, we will be able to start working on migrating the
code base from callbacks to async/await, greatly simplifying legibility (see
#3540).
Closes#3557
Until Etherpad 1.7.5, process.on('SIGTERM') and process.on('SIGINT') were not
hooked up under Windows, because old nodejs versions did not support them.
This excluded the possibility of doing a graceful shutdown of the database
connection under that platform.
According to nodejs 6.x documentation, it is now safe to do so. This allows to
gracefully close the DB connection when hitting CTRL+C under Windows, for
example.
Source: https://nodejs.org/docs/latest-v6.x/api/process.html#process_signal_events
- SIGTERM is not supported on Windows, it can be listened on.
- SIGINT from the terminal is supported on all platforms, and can usually be
generated with <Ctrl>+C (though this may be configurable). It is not
generated when terminal raw mode is enabled.
A Windows manual install has the same directory layout of a normal Unix one
(e.g. the nice symlink node_modules/ep_etherpad-lite -> ../src).
Only when running from the pre-built Windows package the directory layout is
different (e.g. src is physically copied into node_modules/ep_etherpad-lite).
The previous version of the code wrongly assumed that all Windows installs would
be run from the pre-built pakage.
In this version the path search is the same on all platform. If it fails, and we
are on Windows, there is a fallback for the specific case of the pre-built
package.
Fixes#3550
This commit vastly shortens (and simplifies) the version table within
handler/APIHandler.js by building each version's entry incrementally based off
the previous version.
The resulting table has been validated by comparing the "before" and "after"
output of the following loop on both versions of the code (albeit with an
intermediate "sort" step to account for the different insertion order)
for (let v in version) {
let m = version[v];
for (let [k, a] of Object.entries(m)) {
console.log(v, k, a);
}
}
The patch also fixes a few typos, and removes a duplicate definition of
getChatHistory which in each applicable version was defined with two different
parameter lists, but where only the second would be used.
Compatibility with IE11 regressed in 23eab79946 while working for #3488.
That commit made use of modern js syntax, not supported by IE11.
- Removed arrow functions, replaced with normal functions.
- Removed the spread operator (<...iterable>) and the "new Set()" construct,
replaced with _.uniq()
At some point IE11 compatibility will be dropped.
Ditching it now, for such a small gain, is not wise.
Fixes#3500.
This is just a dev dependency, so no real risks, but it's better not to scare
users.
Reported vulnerability before this change:
$ npm audit
=== npm audit security report ===
# Run npm update cryptiles --depth 4 to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High │ Insufficient Entropy │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package │ cryptiles │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ wd [dev] │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path │ wd > request > hawk > cryptiles │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info │ https://nodesecurity.io/advisories/720 │
└───────────────┴──────────────────────────────────────────────────────────────┘
Etherpad-lite relies on the user's browser to generate a random pad
name, but the current solution is not safe against collisions. In order
to generate unique pad names, the following modifications are made:
* use a PRNG instead of Math.random() and ensure uniform distribution
when selecting chars.
* choose the pad name length to achieve a specific number of bits of
security.
Closes: #3516
a) these rules:
[class^="icon-"]:before
[class*=" icon-"]:before
b) were the same as this one:
[data-icon]:before
except the rules in b) had a "content: attr(data-icon)" rule, too.
This commit groups all of them together, and gets rid of the "attr(data-icon)".
The commit that introduced these rules in the first place, and that are now
partially reverted, was 9aea689438 (move tiny bit
of font awesome we actually use into pad.css) from 2014-11-19.
Preparatory work for introducing colibris skin
This commit implements the following behaviour:
1. adds a function clientPluginNames() to hooks.js (mimicking what is done in
static.js), which returns an array containing the list of currently installed
client side plugins. The array is eventually empty.
2. calls that function in pad.html at rendering time (thus server-side) to
populate a class attribute.
Example results:
- with no client-side plugins installed:
<div id="editorcontainerbox" class="">
- with some client-side plugins installed:
<div id="editorcontainerbox" class="ep_author_neat ep_adminpads">
Looking at the existing code (src/node/hooks/express/static.js#L39-L57), a
client-side plugin is defined as a plugin that implements at least a client side
hook.
NOTE: there is currently no support for notifying plugin removal/installation
to the connected clients: for now, in order to get an updated class list,
the clients will have to refresh the page.
Fixes#3488
Since the original comparison compared for truthy and not for "===", and it's
3 AM now, I blindly negated it, in order to show how fragile it was in the first
instance.
No functional changes.
This is the final commit of this refactoring series.
Get rid of an else branch to simplify code layout. No functional changes at all.
==============
This series is an attempt to reduce the control structure depth of the code
base, maintaining at the same time its exact same behaviour, bugs included. It
is, in a sense, an initial attempt at a refactoring in the spirit of its
original definition [0].
The idea beyond this refactoring is that reducing the code depth and, sometimes,
inverting some conditions, bugs and logic errors may become easier to spot, and
the code easier to read.
When looked at ignoring whitespace changes, all of these diffs should appear
trivial.
[0] https://refactoring.com/
It's just synctactic sugar, but it is always better than executing string
concatenations in one's mind.
Do not do this with files in src/static, because we want to keep IE 11
compatibility.
The old "static/custom" directory is replaced by "static/skins/<skinName>",
where <skinName> is taken from settings.json.
When no value is found, a default of "no-skin" is assumed, so that backward
compatibility is maintained.
The most evident security concerns have been addressed.
Closes#3471.
skinName must be a single string (no directory separators in it) pointing to an
existing directory under /src/static/skins.
In case these conditions are not met, its value is rewritten to "no-skin".
Also, the value of skinName if sent to the client via clientVars for allowing
its use it in the browser.
Currently, an Etherpad skin requires the existence of 6 files:
- index.{css,js}
- pad.{css,js}
- timeslider.{css,js}
In the default empty skin (in static/custom), there were 2 small placeholders
({js,css}.template) to be copied in place by the startup script in case no skin
was in use.
Now that we are moving to multiple directories (see #3471) we can simply commit
the example files and remove the copying code from the startup script.
Not performing encoding/decoding when traversing logical domains is a security
risk.
String concatenation is not great, too, but this change is just focused on
allowing the implementation of skin support.
If you edit `src/templates/export_html.html` to remove the
`<meta name="changedby" content="Etherpad">` tag[1], PDF export with
soffice has a bug: the first word of the pad is deleted and a blank page
is inserted as first page (the pad's text begins on the second page).
The `--writer` soffice option avoids that bug.
[1] you may want to delete that tag since it is inserted as a comment in
.doc or .odt soffice export.
