diff --git a/CHANGELOG.md b/CHANGELOG.md index 74d06f45..02accf8c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,17 @@ +# 1.6.4 + * SECURITY: exploitable /admin access - CVE-2018-9845 + * SECURITY: DoS with pad exports - CVE-2018-9327 + * SECURITY: Remote Code Execution - CVE-2018-9326 + * SECURITY: Pad data leak - CVE-2018-9325 + * Fix: Admin redirect URL + * Fix: Various script Fixes + * Fix: Various CSS/Style/Layout fixes + * NEW: Improved Pad contents readability + * NEW: Hook: onAccessCheck + * NEW: SESSIONKEY and APIKey customizable path + * NEW: checkPads script + * NEW: Support "cluster mode" + # 1.6.3 * SECURITY: Update ejs * SECURITY: xss vulnerability when reading window.location.href @@ -56,7 +70,7 @@ * NEW: Allow LibreOffice to be used when exporting a pad * NEW: Create hook exportHtmlAdditionalTagsWithData * NEW: Improve DB migration performance - * NEW: allow settings to be applied from the filesystem + * NEW: allow settings to be applied from the filesystem * NEW: remove applySettings hook and allow credentials.json to be part of core * NEW: Use exec to switch to node process * NEW: Validate incoming color codes @@ -85,7 +99,7 @@ * Fix: switchToPad method * Fix: Dead keys * Fix: Preserve new lines in copy-pasted text - * Fix: Compatibility mode on IE + * Fix: Compatibility mode on IE * Fix: Content Collector to get the class of the DOM-node * Fix: Timeslider export links * Fix: Double prompt on file upload @@ -212,7 +226,7 @@ * Fix: Session Deletion error * Fix: Allow browser tabs to be cycled when focus is in editor * Fix: Various Editor issues with Easysync potentially entering forever loop on bad changeset - + # 1.4 * NEW: Disable toolbar items through settings.json * NEW: Internal stats/metrics engine @@ -244,7 +258,7 @@ # 1.3 * NEW: We now follow the semantic versioning scheme! * NEW: Option to disable IP logging - * NEW: Localisation updates from http://translatewiki.net. + * NEW: Localisation updates from http://translatewiki.net. * Fix: Fix readOnly group pads * Fix: don't fetch padList on every request @@ -337,7 +351,7 @@ * NEW: Add authorId to chat and userlist as a data attribute * NEW: Refactor and fix our frontend tests * NEW: Localisation updates - + # 1.2.81 * Fix: CtrlZ-Y for Undo Redo @@ -377,7 +391,7 @@ * Other: Change loading message asking user to please wait on first build * Other: Allow etherpad to use global npm installation (Safe since node 6.3) * Other: Better documentation for log rotation and log message handling - + # 1.2.7 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 09ddc286..66946080 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,4 @@ -# Developer Guidelines +# Contributor Guidelines (Please talk to people on the mailing list before you change this page, see our section on [how to get in touch](https://github.com/ether/etherpad-lite#get-in-touch)) ## How to write a bug report @@ -35,7 +35,7 @@ The logfile location is defined in startup script or the log is directly shown i To make sure everybody is going in the same direction: * easy to install for admins and easy to use for people * easy to integrate into other apps, but also usable as standalone -* using less resources on server side +* lightweight and scalable * extensible, as much functionality should be extendable with plugins so changes don't have to be done in core. Also, keep it maintainable. We don't wanna end up as the monster Etherpad was! @@ -92,3 +92,19 @@ You can build the docs e.g. produce html, using `make docs`. At some point in th ## Testing Front-end tests are found in the `tests/frontend/` folder in the repository. Run them by pointing your browser to `/tests/frontend`. + +## Things you can help with +Etherpad is much more than software. So if you aren't a developer then worry not, there is still a LOT you can do! A big part of what we do is community engagement. You can help in the following ways + * Triage bugs (applying labels) and confirming their existance + * Testing fixes (simply applying them and seeing if it fixes your issue or not) - Some git experience required + * Notifying large site admins of new releases + * Writing Changelogs for releases + * Creating Windows packages + * Creating releases + * Bumping dependencies periodically and checking they don't break anything + * Write proposals for grants + * Co-Author and Publish CVEs + * Work with SFC to maintain legal side of project + * Maintain TODO page - https://github.com/ether/etherpad-lite/wiki/TODO#IMPORTANT_TODOS + * Replying to messages on IRC / The Mailing list / Emails + diff --git a/README.md b/README.md index d0d22433..d8d7b621 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,43 @@ +### This project is looking for a new project lead. If you wish to help steer Etherpad forward please email contact@etherpad.org + +[![Deps](https://david-dm.org/ether/etherpad-lite.svg?branch=develop)](https://david-dm.org/ether/etherpad-lite) +[![NSP Status](https://nodesecurity.io/orgs/etherpad/projects/635f6185-35c6-4ed7-931a-0bc62758ece7/badge)](https://nodesecurity.io/orgs/etherpad/projects/635f6185-35c6-4ed7-931a-0bc62758ece7) + # A really-real time collaborative word processor for the web -![alt text](https://i.imgur.com/zYrGkg3.gif "Etherpad in action on PrimaryPad") +![Demo Etherpad Animated Jif](https://i.imgur.com/zYrGkg3.gif "Etherpad in action on PrimaryPad") # About -Etherpad is a really-real time collaborative editor maintained by the Etherpad Community. +Etherpad is a really-real time collaborative editor scalable to thousands of simultanious real time users. Unlike all other collaborative tools Etherpad provides full fidelity data export and portability making it fully GDPR compliant. -Etherpad is written in JavaScript (99.9%) on both the server and client so it's easy for developers to maintain and add new features. Because of this Etherpad has tons of customizations that you can leverage. - -Etherpad is designed to be easily embeddable and provides a [HTTP API](https://github.com/ether/etherpad-lite/wiki/HTTP-API) -that allows your web application to manage pads, users and groups. It is recommended to use the [available client implementations](https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries) in order to interact with this API. - -There is also a [jQuery plugin](https://github.com/ether/etherpad-lite-jquery-plugin) that helps you to embed Pads into your website. - -There's also a full-featured plugin framework, allowing you to easily add your own features. By default your Etherpad is rather sparse and because Etherpad takes a lot of its inspiration from WordPress, plugins are really easy to install and update. Once you have Etherpad installed you should visit the plugin page and take control. - -Finally, Etherpad comes with translations into most languages! Users are automatically delivered the correct language for their local settings. - - -**Visit [beta.etherpad.org](http://beta.etherpad.org) to test it live.** - -Also, check out the **[FAQ](https://github.com/ether/etherpad-lite/wiki/FAQ)**, really! +**[Try it out](http://beta.etherpad.org)** # Installation -Etherpad works with node v0.10+ (except 6.0 and 6.1). +## Uber-Quick Ubuntu +``` +curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash - +sudo apt-get install -y nodejs +git clone https://github.com/ether/etherpad-lite.git && cd etherpad-lite && bin/run.sh +``` + +## GNU/Linux and other UNIX-like systems +You'll need gzip, git, curl, libssl develop libraries, python and gcc. +- *For Debian/Ubuntu*: `apt install gzip git curl python libssl-dev pkg-config build-essential` +- *For Fedora/CentOS*: `yum install gzip git curl python openssl-devel && yum groupinstall "Development Tools"` +- *For FreeBSD*: `portinstall node, npm, curl, git (optional)` + +Additionally, you'll need [node.js](https://nodejs.org) installed, Ideally the latest stable version, we recommend installing/compiling nodejs from source (avoiding apt). + +**As any user (we recommend creating a separate user called etherpad):** + +1. Move to a folder where you want to install Etherpad. Clone the git repository `git clone git://github.com/ether/etherpad-lite.git` +2. Change into the new directory containing the cloned source code `cd etherpad-lite` + +Now, run `bin/run.sh` and open in your browser. + +Update to the latest version with `git pull origin`. The next start with bin/run.sh will update the dependencies. + +[Next steps](#next-steps). ## Windows @@ -52,27 +67,6 @@ If cloning to a subdirectory within another project, you may need to do the foll 2. Edit the db `filename` in `settings.json` to the relative directory with the file (e.g. `application/lib/etherpad-lite/var/dirty.db`) 3. Add auto-generated files to the main project `.gitignore` -[Next steps](#next-steps). - -## GNU/Linux and other UNIX-like systems -You'll need gzip, git, curl, libssl develop libraries, python and gcc. -- *For Debian/Ubuntu*: `apt install gzip git curl python libssl-dev pkg-config build-essential` -- *For Fedora/CentOS*: `yum install gzip git curl python openssl-devel && yum groupinstall "Development Tools"` -- *For FreeBSD*: `portinstall node, npm, curl, git (optional)` - -Additionally, you'll need [node.js](https://nodejs.org) installed, Ideally the latest stable version, we recommend installing/compiling nodejs from source (avoiding apt). - -**As any user (we recommend creating a separate user called etherpad):** - -1. Move to a folder where you want to install Etherpad. Clone the git repository `git clone git://github.com/ether/etherpad-lite.git` -2. Change into the new directory containing the cloned source code `cd etherpad-lite` - -Now, run `bin/run.sh` and open in your browser. - -Update to the latest version with `git pull origin`. The next start with bin/run.sh will update the dependencies. - -You like it? [Next steps](#next-steps). - # Next Steps ## Tweak the settings @@ -85,7 +79,7 @@ You should use a dedicated database such as "mysql", if you are planning on usin Etherpad is very customizable through plugins. Instructions for installing themes and plugins can be found in [the plugin wiki article](https://github.com/ether/etherpad-lite/wiki/Available-Plugins). ## Helpful resources -The [wiki](https://github.com/ether/etherpad-lite/wiki) is your one-stop resource for Tutorials and How-to's, really check it out! Also, feel free to improve these wiki pages. +The [wiki](https://github.com/ether/etherpad-lite/wiki) is your one-stop resource for Tutorials and How-to's. Documentation can be found in `doc/`. @@ -100,26 +94,38 @@ You can debug Etherpad using `bin/debugRun.sh`. If you want to find out how Etherpad's `Easysync` works (the library that makes it really realtime), start with this [PDF](https://github.com/ether/etherpad-lite/raw/master/doc/easysync/easysync-full-description.pdf) (complex, but worth reading). -## Getting started -You know all this and just want to know how you can help? - -Look at the [TODO list](https://github.com/ether/etherpad-lite/wiki/TODO) and our [Issue tracker](https://github.com/ether/etherpad-lite/issues). (Please consider using [jshint](http://www.jshint.com/about/), if you plan to contribute code.) - -Also, and most importantly, read our [**Developer Guidelines**](https://github.com/ether/etherpad-lite/blob/master/CONTRIBUTING.md), really! +## Contributing +Read our [**Developer Guidelines**](https://github.com/ether/etherpad-lite/blob/master/CONTRIBUTING.md) # Get in touch -Join the [mailinglist](https://groups.google.com/group/etherpad-lite-dev) and make some noise on our busy freenode irc channel [#etherpad-lite-dev](https://webchat.freenode.net?channels=#etherpad-lite-dev)! +[mailinglist](https://groups.google.com/group/etherpad-lite-dev) +[#etherpad-lite-dev freenode IRC](https://webchat.freenode.net?channels=#etherpad-lite-dev)! -# Modules created for this project +# Languages +Etherpad is written in JavaScript on both the server and client so it's easy for developers to maintain and add new features. -* [ueberDB](https://github.com/Pita/ueberDB) "transforms every database into a object key value store" - manages all database access -* [channels](https://github.com/Pita/channels) "Event channels in node.js" - ensures that ueberDB operations are atomic and in series for each key -* [async-stacktrace](https://github.com/Pita/async-stacktrace) "Improves node.js stacktraces and makes it easier to handle errors" +# HTTP API +Etherpad is designed to be easily embeddable and provides a [HTTP API](https://github.com/ether/etherpad-lite/wiki/HTTP-API) +that allows your web application to manage pads, users and groups. It is recommended to use the [available client implementations](https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries) in order to interact with this API. + +# jQuery plugin +There is a [jQuery plugin](https://github.com/ether/etherpad-lite-jquery-plugin) that helps you to embed Pads into your website. + +# Plugin Framework +Etherpad offers a plugin framework, allowing you to easily add your own features. By default your Etherpad is extremely light-weight and it's up to you to customize your experience. Once you have Etherpad installed you should visit the plugin page and take control. + +# Translations / Localizations (i18n / l10n) +Etherpad comes with translations into all languages thanks to the team at TranslateWiki. + +# FAQ +Visit the **[FAQ](https://github.com/ether/etherpad-lite/wiki/FAQ)**. # Donate! * [Flattr](https://flattr.com/thing/71378/Etherpad-Foundation) * Paypal - Press the donate button on [etherpad.org](http://etherpad.org) * [Bitcoin](https://coinbase.com/checkouts/1e572bf8a82e4663499f7f1f66c2d15a) +All donations go to the Etherpad foundation which is part of Software Freedom Conservency + # License [Apache License v2](http://www.apache.org/licenses/LICENSE-2.0.html) diff --git a/bin/checkAllPads.js b/bin/checkAllPads.js new file mode 100644 index 00000000..90cb1527 --- /dev/null +++ b/bin/checkAllPads.js @@ -0,0 +1,145 @@ +/* + This is a debug tool. It checks all revisions for data corruption +*/ + +if(process.argv.length != 2) +{ + console.error("Use: node bin/checkAllPads.js"); + process.exit(1); +} + +//initalize the variables +var db, settings, padManager; +var npm = require("../src/node_modules/npm"); +var async = require("../src/node_modules/async"); + +var Changeset = require("../src/static/js/Changeset"); + +async.series([ + //load npm + function(callback) { + npm.load({}, callback); + }, + //load modules + function(callback) { + settings = require('../src/node/utils/Settings'); + db = require('../src/node/db/DB'); + + //initalize the database + db.init(callback); + }, + //load pads + function (callback) + { + padManager = require('../src/node/db/PadManager'); + + padManager.listAllPads(function(err, res) + { + padIds = res.padIDs; + callback(err); + }); + }, + function (callback) + { + async.forEach(padIds, function(padId, callback) + { + padManager.getPad(padId, function(err, pad) { + if (err) { + callback(err); + } + + //check if the pad has a pool + if(pad.pool === undefined ) + { + console.error("[" + pad.id + "] Missing attribute pool"); + callback(); + return; + } + + //create an array with key kevisions + //key revisions always save the full pad atext + var head = pad.getHeadRevisionNumber(); + var keyRevisions = []; + for(var i=0;i -1; + if(deniedByHook) + { + callback(null, {accessStatus: "deny"}); + return; + } + // a valid session is required (api-only mode) if(settings.requireSession) { diff --git a/src/node/handler/APIHandler.js b/src/node/handler/APIHandler.js index 179c2b40..05e14705 100644 --- a/src/node/handler/APIHandler.js +++ b/src/node/handler/APIHandler.js @@ -24,17 +24,19 @@ var fs = require("fs"); var api = require("../db/API"); var padManager = require("../db/PadManager"); var randomString = require("../utils/randomstring"); +var argv = require('../utils/Cli').argv; //ensure we have an apikey var apikey = null; +var apikeyFilename = argv.apikey || "./APIKEY.txt"; try { - apikey = fs.readFileSync("./APIKEY.txt","utf8"); + apikey = fs.readFileSync(apikeyFilename,"utf8"); } catch(e) { apikey = randomString(32); - fs.writeFileSync("./APIKEY.txt",apikey,"utf8"); + fs.writeFileSync(apikeyFilename,apikey,"utf8"); } //a list of all functions diff --git a/src/node/handler/ImportHandler.js b/src/node/handler/ImportHandler.js index 6aa94e64..3e3dc195 100644 --- a/src/node/handler/ImportHandler.js +++ b/src/node/handler/ImportHandler.js @@ -90,7 +90,7 @@ exports.doImport = function(req, res, padId) //this allows us to accept source code files like .c or .java function(callback) { var fileEnding = path.extname(srcFile).toLowerCase() - , knownFileEndings = [".txt", ".doc", ".docx", ".pdf", ".odt", ".html", ".htm", ".etherpad"] + , knownFileEndings = [".txt", ".doc", ".docx", ".pdf", ".odt", ".html", ".htm", ".etherpad", ".rtf"] , fileEndingKnown = (knownFileEndings.indexOf(fileEnding) > -1); //if the file ending is known, continue as normal diff --git a/src/node/hooks/express/admin.js b/src/node/hooks/express/admin.js index 70539f0c..0884cde5 100644 --- a/src/node/hooks/express/admin.js +++ b/src/node/hooks/express/admin.js @@ -2,7 +2,7 @@ var eejs = require('ep_etherpad-lite/node/eejs'); exports.expressCreateServer = function (hook_name, args, cb) { args.app.get('/admin', function(req, res) { - if('/' != req.path[req.path.length-1]) return res.redirect('/admin/'); + if('/' != req.path[req.path.length-1]) return res.redirect('./admin/'); res.send( eejs.require("ep_etherpad-lite/templates/admin/index.html", {}) ); }); } diff --git a/src/node/hooks/express/webaccess.js b/src/node/hooks/express/webaccess.js index 29475037..4b9ad4eb 100644 --- a/src/node/hooks/express/webaccess.js +++ b/src/node/hooks/express/webaccess.js @@ -36,13 +36,16 @@ exports.basicAuth = function (req, res, next) { var userpass = new Buffer(req.headers.authorization.split(' ')[1], 'base64').toString().split(":") var username = userpass.shift(); var password = userpass.join(':'); - - if (settings.users[username] != undefined && settings.users[username].password === password) { - settings.users[username].username = username; - req.session.user = settings.users[username]; - return cb(true); - } - return hooks.aCallFirst("authenticate", {req: req, res:res, next:next, username: username, password: password}, hookResultMangle(cb)); + var fallback = function(success) { + if (success) return cb(true); + if (settings.users[username] != undefined && settings.users[username].password == password) { + settings.users[username].username = username; + req.session.user = settings.users[username]; + return cb(true); + } + return cb(false); + }; + return hooks.aCallFirst("authenticate", {req: req, res:res, next:next, username: username, password: password}, hookResultMangle(fallback)); } hooks.aCallFirst("authenticate", {req: req, res:res, next:next}, hookResultMangle(cb)); } @@ -126,4 +129,3 @@ exports.expressConfigure = function (hook_name, args, cb) { args.app.use(exports.basicAuth); } - diff --git a/src/node/utils/Cli.js b/src/node/utils/Cli.js index 9419ed26..154590dc 100644 --- a/src/node/utils/Cli.js +++ b/src/node/utils/Cli.js @@ -39,5 +39,15 @@ for ( var i = 0; i < argv.length; i++ ) { exports.argv.credentials = arg; } + // Override location of settings.json file + if ( prevArg == '--sessionkey' || prevArg == '-k' ) { + exports.argv.sessionkey = arg; + } + + // Override location of settings.json file + if ( prevArg == '--apikey' || prevArg == '-k' ) { + exports.argv.apikey = arg; + } + prevArg = arg; } diff --git a/src/node/utils/Settings.js b/src/node/utils/Settings.js index cf7fea80..08ace60c 100644 --- a/src/node/utils/Settings.js +++ b/src/node/utils/Settings.js @@ -476,11 +476,12 @@ exports.reloadSettings = function reloadSettings() { } if (!exports.sessionKey) { + var sessionkeyFilename = argv.sessionkey || "./SESSIONKEY.txt"; try { - exports.sessionKey = fs.readFileSync("./SESSIONKEY.txt","utf8"); + exports.sessionKey = fs.readFileSync(sessionkeyFilename,"utf8"); } catch(e) { exports.sessionKey = randomString(32); - fs.writeFileSync("./SESSIONKEY.txt",exports.sessionKey,"utf8"); + fs.writeFileSync(sessionkeyFilename,exports.sessionKey,"utf8"); } } else { console.warn("Declaring the sessionKey in the settings.json is deprecated. This value is auto-generated now. Please remove the setting from the file."); diff --git a/src/package.json b/src/package.json index 6aa17a69..bf2f8858 100644 --- a/src/package.json +++ b/src/package.json @@ -17,7 +17,7 @@ "etherpad-require-kernel" : "1.0.9", "resolve" : "1.1.7", "socket.io" : "1.7.3", - "ueberdb2" : "0.3.6", + "ueberdb2" : "0.3.8", "express" : "4.13.4", "express-session" : "1.13.0", "cookie-parser" : "1.3.4", @@ -28,7 +28,7 @@ "log4js" : "0.6.35", "cheerio" : "0.20.0", "async-stacktrace" : "0.0.2", - "npm" : "4.0.2", + "npm" : ">=4.0.2", "ejs" : "2.5.7", "graceful-fs" : "4.1.3", "slide" : "1.1.6", @@ -55,6 +55,6 @@ "repository" : { "type" : "git", "url" : "http://github.com/ether/etherpad-lite.git" }, - "version" : "1.6.3", + "version" : "1.6.4", "license" : "Apache-2.0" } diff --git a/src/static/css/iframe_editor.css b/src/static/css/iframe_editor.css index 9aa003aa..757bfa96 100644 --- a/src/static/css/iframe_editor.css +++ b/src/static/css/iframe_editor.css @@ -31,13 +31,17 @@ body { body.grayedout { background-color: #eee !important } #innerdocbody { - font-size: 12px; /* overridden by body.style */ + font-size: 16px; /* overridden by body.style */ font-family:Arial, sans-serif; /* overridden by body.style */ line-height: 16px; /* overridden by body.style */ background-color: white; color: black; } +.innerdocbody>div{ + padding: 1px; +} + body.doesWrap { /* white-space: pre-wrap; */ @@ -58,9 +62,11 @@ body.doesWrap { white-space: normal; } -body.doesWrap:not(.noprewrap) > div{ - /* Related to #1766 */ - white-space: pre-wrap; +@-moz-document url-prefix() { + body.doesWrap:not(.noprewrap) > div{ + /* Related to #1766 */ + white-space: pre-wrap; + } } #innerdocbody { diff --git a/src/static/css/pad.css b/src/static/css/pad.css index cabde7ef..484e6f2a 100644 --- a/src/static/css/pad.css +++ b/src/static/css/pad.css @@ -3,8 +3,9 @@ html, body, p { margin: 0; - padding: 0; + padding: 0px; } + .clear { clear: both } @@ -1071,9 +1072,9 @@ input[type=checkbox] { overflow: auto; } #mycolorpicker { - left: -73px; - top:auto !important; - bottom:33px !important; + left: 0px; + top:37px !important; + position:fixed; /* #mycolorpicker: width -#users: width */; } #editorcontainer { diff --git a/src/static/js/ace2_inner.js b/src/static/js/ace2_inner.js index df9c9642..90cefa50 100644 --- a/src/static/js/ace2_inner.js +++ b/src/static/js/ace2_inner.js @@ -5404,8 +5404,8 @@ function Ace2Inner(){ // height is taken to be the top offset of the next line. If we // didn't do this special case, we would miss out on any top margin // included on the first line. The default stylesheet doesn't add - // extra margins, but plugins might. - h = b.nextSibling.offsetTop; + // extra margins/padding, but plugins might. + h = b.nextSibling.offsetTop - window.getComputedStyle(doc.body).getPropertyValue("padding-top"); } else { h = b.nextSibling.offsetTop - b.offsetTop; } diff --git a/src/templates/export_html.html b/src/templates/export_html.html index b29941c9..b8893b71 100644 --- a/src/templates/export_html.html +++ b/src/templates/export_html.html @@ -139,6 +139,5 @@ ol > ol > ol > ol > ol > ol > ol > ol > ol > ol > ol > ol > ol > ol > ol > ol { <%- body %> -
JavaScript license information