webstrates / file-system Goto Github PK

Mounts a Webstrate document as a file on disk, propagating changes between the file and document.

JavaScript 100.00%

file-system's Introduction

Webstrates

With Webstrates, webpages become collaboratively editable in real-time. Changes to the Document Object Model (DOM) of a page persist and are synchronised to all connected clients of the same page using Operational Transformation through ShareDB.

Webstrates can be used to develop interactive software where collaboration-support is the norm rather than the exception.

Go to the documentation to get started using webstrates, for user guide and developer guide.

Visit webstrates.net for background on the Webstrates project, academic papers and videos of Webstrates in action.

License

This work is licenced under the Apache License, Version 2.0.

file-system's People

Contributors

Stargazers

Watchers

Forkers

raedle eiselmayer eaganj mwangey

file-system's Issues

HTML comments can cause the output to be garbled

In certain cases HTML comments in the input can cause the output to be almost entirely discarded with the exception of the contents of the comment. Example filesystem edit:

<!DOCTYPE html>
<!-- This shouldn't be the output -->
<html>
<body>
<div>TODO write content</div>
</body>
</html>

returns the following output in the webstrate:

and nothing else.

Attributes are lowercased

This impacts HTML5 with inline SVGs which require the viewBox attribute to be camel cased. Example:

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 11.8 12.9">
    <path d="M0 0h3.2v3.5H0zm4.3 0h3.2v3.5H4.3zm4.3 0h3.2v3.5H8.6zM0 4.7h3.2v3.5H0zm4.3 0h3.2v3.5H4.3zm4.3 0h3.2v3.5H8.6zM0 9.4h3.2v3.5H0zm4.3 0h3.2v3.5H4.3zm4.3 0h3.2v3.5H8.6z"/>
</svg>

Name Collision with Windows Fax and Scan when installed globally with npm on Win10

When installing file-system through npm install -g on Windows 10, the wfs command does not reference webstrates file-system. Instead, Windows Fax and Scan is started. This service is installed on every Windows 10 machine and will be launched when running wfs from both cmd and git-bash.

A quick workaround that worked for me was to rename the files named wfs and wfs.cmd in C:\Users\<User>\AppData\Roaming\npm. As a permanent fix, I recommend renaming the path variable in the bin property in package.json.

Oneshot read/write

A low-prio feature request for three commandline arguments that instruct the file-system script how to behave in one-way scenarios where either the webstrate or the file is "authoritative".

--readonly : to only ever read the current webstrate into the file regardless of any local changes to the file made by other programs
--writeonly: to only write the file into the webstrate but ignore any changes made to the webstrate
--oneshot: immediately exit after updating either the file or the webstrate rather than continuing to monitor the file/webstrate.

The primary use-case for this will be to embed the file-system script into continuous integration servers or similar scripting environments where webstrates can be either deployed or archived automatically.

--oneshot doesn't properly cover assets

Since the implementation stops execution after covering the main index.html file, assets are randomly left out, depending on whether they're processed before or after index.html.

One-shot should probably not be implemented as part of the watcher, but instead just upload all files.

HTML entities are acting up

A test-case with the document in the bottom of this issue displays as
© & > < æ ø å
locally but displays as
© & > < æ ø å
when uploaded via the filesystem script. Note that ampersand, greater than and less than are correctly shown but not other entities; the other entities are doubly escaped.

<!DOCTYPE html>
<html>
    <head></head>
    <body>&copy; &amp; &gt; &lt; &aelig; &oslash; &aring;</body>
</html>

File change detection corrupts file

For large files where it takes more than a few nanosecs to save the file from an editor, the file-system script will detect that the file has changed already while it is being saved. It will then read the (halfway written) file and construct a partial DOM from it and write back the file to the file system. The file is now no longer correct and much smaller than it was supposed to be. Both the webstrate and the file is now corrupt.

In our testcase it took 3,8ms to commit a 300KB file to disk but a change was detected by the script already after the first 8KB were written and immediately after the editor was done saving the file the file-system script truncates the file to around 8KB albeit with a "valid" DOM tree.

The issue can be worked around from the editor by writing to a tmp file and then atomically replacing the file system document file with the new copy, but not all editors do this. Most simply rewrite the existing file.

Occasional crashes when files update a lot

Getting some crashes in the file-system mapping service when the file is changed locally, but it only happens somewhat rarely - and typically when in some kind of coding frenzy, so maybe this is timing related. It is always the same error, though:
This is the index.js stack trace from the file-system module:

 $ node ./index.js --id=party --host=ws://192.168.42.110:7007
Connecting to ws://192.168.42.110:7007...
Connected.
~/file-system/node_modules/jsonml-parse/index.js:71
            if (data[0] === "\n" && data[1]) data = data[1];
                    ^

TypeError: Cannot read property '0' of undefined
    at ~/file-system/node_modules/jsonml-parse/index.js:71:21
    at ConcatStream.<anonymous> (~/file-system/node_modules/concat-stream/index.js:36:43)
    at emitNone (events.js:91:20)
    at ConcatStream.emit (events.js:185:7)
    at finishMaybe (~/file-system/node_modules/readable-stream/lib/_stream_writable.js:511:14)
    at endWritable (~/file-system/node_modules/readable-stream/lib/_stream_writable.js:521:3)
    at ConcatStream.Writable.end (~/file-system/node_modules/readable-stream/lib/_stream_writable.js:491:41)
    at JSONMLParser.onend (_stream_readable.js:511:10)
    at JSONMLParser.g (events.js:291:16)
    at emitNone (events.js:91:20)

No error when connection not established

If the service can't connect to the given Webstrates server, it quits silently.

<transient> tag causes issues

Creating a <transient> tag in a document conflicts with the <transient> implementation in Webstrates as these tags are never supposed to be shared or persisted.

Adding a <transient> tag in the document causes it to be created on other clients, but any changes made to it on other clients are not saved. Any subsequent changes made to the tag from the file system cause errors on the client, because the <transient> tag rightfully aren't in the Webstrate PathTree.

This could be solved by not letting <transient> tags get sent to the server.

Installation fail with node v11.11.0, npm 6.9.0, macOS 10.14.3

I ran into the following trying to install wfs on a fresh node install
I attempted the installation with and without sudo.
wfs-install-fail.txt

2019-03-18T10_58_03_960Z-debug.log
2019-03-18T10_58_42_196Z-debug.log