The storage service requires a relatively complex setup, since it should be compatible with real-time collaboration and sharing features. Also it should work in a non-blocking manner with a scenario with several open tabs.
Several options were taken into account for the synchronization logic and a number of requirements and constraints were defined (see #21 ).
The final setup includes the following:
Implement a VCS system, where the domain store state (pages/notes) is persisted in a repository. The latter holds commits (git-like). When the store is altered - a commit is generated with a content hash (based on the store contents) and a unique id (not based on the content or parent commit ids like in git).
Define a synchronisation scheme - The repo may have multiple branches, where each branch corresponds to a device. Each device should write only to its own branch. The repo info can be synced without conflicts in this manner - each device gets all commits and branch info. The merging of remote changes happens by seniority, where junior devices merge changes from the more senior ones and senior devices ignore conflicting changes.
The frontend code should not be concerned with that - the frontend operates by using the Frontend domain store (FDS) - an in-memory store, that is used synchronously in the actions triggered by the user.
Changes in the FDS trigger commit creation in the storage service. And remote changes received by the storage service are pushed to the frontend domain store and the GUI respectively.
The storage service should run in a service worker whenever possible in order to avoid blocking the main thread and to avoid memory/network overhead when many tabs are open and remote sync is happening. That's why it's separated into a StorageService (wrapper class) and a StorageServiceActual (that can be instantiated in a worker). The two communicate via a messaging interface.
The storage interface is implemented on the repo/project level. But the user may open several projects (in different tabs, etc). So the storage manager can load/unload Project storage managers (in the StorageServiceActual)
Each Project storage manager is instantiated with an adapter config - i.e. typically local persistence (indexedDB adapter) and in the future - a webRTC adapter (for real-time collab, sync) and/or cloud storage.
Overview of the save/receive-update flow:
Acceptance criteria:
Changes persist over separate sessions.
Fix the domain store related code - it will be used for propagating the changes to the in-mem storage or other (persistence) adapters.. also services like search/undo-redo. Let's leave the mobx store to be used only for views. That's what it's for. Other stuff will be handled by the custom store and channels. Ok, so if i'll pass evereything through the pametCRUD anyway - I might as well make the .note accessor methods return a note copy to be modified in the actions (e.g. refactor the .note to note(), which under the hood calls the computed _note mobx property or whatever). then the updated note/arrow/element gets passed to pametCRUD via the facade methods. I'm taking ownage of the domain store management, so only add a mapping function to update view states on note changes (synchronously, in-action, in the facade, right after the inMemStore update).
setCurrentPage action to directly create (or reuse) the PVS , instead of just chanigng the currentPageId...done
Reintroduce the InMemoryStore (move methods from facade?)...done
Fix facade methods to use the InMemRepo...done
Load duplicate errors...done
Decide on PametInMemRepo implementation - compose, rather than inherit...not needed. The facade does that. In other words: done
applyChangesToViewStates(app_state, changes) reducer-like function will go through all the ViewStates in the app state and swap the data objects on update (fcit no diffing needed. It's only for actual entity changes).
add/remove/update EVS methods in the page view state...done
Implement applyChangesToViewStates...done
Call it after pushing the changes (synchronously) to the inMemStore (so that that mapping happens in-action, and without increased complexity, side-effects etc)...done
Test with note creation...done
Image note update after loading is broken...resolved in meantime
New note change is not reflected...done
refactor NVS.note to note() + get _note():computed
Rename sync repo to store if it still seems appropriate...not for now
It will be nice if you can ask Mila to make some simple icon for the project that will be seen in GitHub, Taskbar, Desktop etc.
She will be able to add "Works on several open source projects." to her CV then ;)
Note editing, at least for the core note types, will happen via a single note edit component. Any interactive integrations will open in a separate modal window.
Design the window...done, mostly reusing the desktop design
Consider integrations (e.g. script note, audio, video) settings - as a separate panel?...no, as a modal window
Consider the mobile interface...done
Implement note text editing
Implement a minimal component/window...done
Add the edit window state as part of the page view state...done
Structure the component hierarchy app-mid?-page...done
Implement an action to show the edit window and call it on the N button press...done
Should be a command, so setup the commands lib...done
Mouse position is needed, so setup tracking in the page view state...done
Fix the basic layout...done
Implement window dragging...done
Implement restrictions on drag/resize positions and sizes. Hopefully in css...done, not in css but accetably easy
add close button, title and move handle icon...done
Text edit related
Add text area and fix main content layout...done
Pass the note data and populate the text area...done
Fix react image element duplicates...done
Actions to save/cancel editing...done
Add a note() method to the editWindowState...done
Implement the saveEditedNote action
Add default method to TextNote...done
Refactor own/page id storage logic...done
Autosize function translate to ts...done
Implement Size class...done
Note does not update on size change...will be fixed in another issue
Add autosize command and shortcut (also a large size shortcut for testing)...done
Add E edit window shortcut in page...done
Add Esc shortcut for edit window close...done
Shortcuts interfere with text typing in the edit window...done
Scroll events should also probably be stopped to not bubble...done
A key-value base class, and a proper class with property access methods. With adapters for localStorage and in-memory storage (the latter for testing).
PixiJS research...no, doesn't include path intersection API, and is heavier
Optimize rendering...done
Drawing cache service...done
Urgent rendering logic...done
Testing...done
Draw cache rect and add sanity check logging in the untested functions. It's probably the projection matrix...done
Test arrow disappearance - probably bad ctx management...done
Fix dpr rendering issues...done
Fix viewport projection calculation - makes no sense to keep the 0,0 real coords in the viewport center...done
Project to topleft in Viewport...done
Init vieportCenter in PVS to the element center...done
Fix realTopLeft argument in calls...done
Fix denovo updates on translate (bad cache rect I guess)...done
No patterns appear ever....done
Cache rects are variable on DPR!=1 (notes wiggle on panning)...done
Add DPR to the Viewport ...done
Don't cache on closeup...done
There's still rendering artefacts on some scales. Translation invariant...done , fixed by increasing the cache rect padding
Initial render update not happening...done
Remove PaperJS (and pixi), replace bezier intersection functionality...no, there's no alternative and I fixed performance issues and memory leaks...done
Skip redundant queued renders...done
Note rendering
TextLayout remove empty lines...done
Fix notes with new lines...done
Call next render if unfinished items...done
Plan image rendering...done
Take advantage of browser cache
Add a lazy image loading service - notes will query it when rendering and will provide fallback container size. The service will either return the image or a placeholder depiction
The lazy image service keeps its state in the appState (queued downloads)
Well the whole service has to be a queue with cache. I.e. keep most frequently accessed in memory (that is optional actually, but preferable) and regardless if retrieved from cache or abroad - deliver each item to be "consumed" by the renderer. This may be implicitly achieved through the image access method. This means the image disappears on scaling tho.. But only if there's too many images (>cache), which is ok. Which means I don't need a strict consumption mechanism/queue, neither the implicit freeing. Just a cache, and an on-screen warning for "too many images. Will not render all".
Alternatively: Since I'll have a boundary on the acceptable image count (for full rendering) - I can implement that in react and not deal with the service. I only need to synchronize react rendering with the canvas rendering to avoid artefacts
Final? a lazy image service that queues images for download (+/- in a service worker) upon calling the access method. The service has a view-state: on each finished download a counter is incremented to trigger page rerender, a flag indicates if the queue is empty (shows a spinner otherwise?)
Final:
Upon further consideration - the max note size is 1920x1920. That will be the max image preview size too, which with adequate compression means acceptable memory requirements. The server will provide images with the size of the note (on each note resize - the backend updates its cache). On the cloud/desktop backend the original file is kept (subject to size restrictions). On the PWA backend - files are resized on upload (to 1920x1920, lossy) and kept in the indexDB. On page render - the notes with images will be added as hidden elements (with a placeholder fallback) and their references will be mapped to their NVS/url in the canvas-renderer. The canvasDraw will accept not ctx, but the renderer as a parameter (and will access images by retrieving its ref via some method)
since we'll be keeping references (i.e. note view react state) - we can go ahead and add all notes for SEO (invisible) and call the canvasDraw in the component... NO, the two renderers (react, and my canvas-renderer) are separate concerns
*On the PWA backend - files are resized on upload (to 1920x1920, lossy) and kept in the indexDB... NO , i don't want to deal with caring for the image life cycle
Write down the plan in a structured manner...done
Render the img tags for the media components with react
Will just create the elements hidden with react and query them at render time via the document API...done
parseMediaUrl error (mutable data mistake?)...done
Implement add/get/clear image element reference...no
Put note rendering logic in CanvasElementView components - functional (the simpler option) + leave only NoteViewState (no note-type-specific states). The mapping will be on the View level
Add ElementView registration...done
Rename the page renderer...done
Implement ImageView...done
Remove card notes?...yes
InternalLinkNote...done
Test...done
Implement CardView...no
Draw link borders when appropriate...done
Arrow rendering...done
Draw arrow head...done
Acceptance criteria:
Correct rendering for all types of notes and for arrows
Undo/redo - not really, will be handled on the change-set level
Backups - not really, could use commits, but should be handled in a separate on-device (non-synced) service
Full change history retention - not really, should be on-device, non-synced. Again - possibly a separate service or part of the backup
Media sync
The fetch source can be swapped by intercepting the requests in a service worker. So the browser cache will be used naturally, while the source of the media can be swapped.
Constraints/requirements
Possibly without accurate time sync for every node
Able to tollerate long offline periods and multiple changes (offline work)
Multi device multi-user state syncing with automatic conflict resolution. Data loss on conflict is acceptable and resolved on a node seniority principle
Worst case scenario merkle-tree hashing tests (200 changed hashes for 750char content leafs + 15000 unchanged content leafs to be recalculated into node-hashes) on "low-end mobile hardware" (chrome dev-tools throttling) take ~160ms. Therefore commit creation cannot be in the main-thread, but can be done on every commit
Story points:
XL / 13 (were expected to be M to L)
Navigation lag noticeable in large note files. In theory pure translate/scale transformations should be efficient, so the lag may be alleviated via restricting state-tree diffing operations for the page.
Test if pure css var changes reduce the lag...yes, but on low-end devices and large pages that's not enough
Will have to implement canvas/webgl rendering instead of react components for notes/arrows