erangell/articles

Markdown source for articles

articles's Issues

Node JS release cadence

See https://github.com/nodejs/release for chart of each major version and upcoming versions.
Production should run on the latest Active LTS version.
Dev and Test environments should run on the Current version.
Note that dependencies for the hyper stack will also need to upgrade as Maintenance support ends for older releases.

Comments on article about decentralized tech for Education

"So many of these solutions feel like they're appropriate for a camp atmosphere. Not school. They almost always assume that the end-user is self-motivated and ignore that the end-user is most likely a disinterested cheater."

ER: I grew up in a different environment, which many today call "privileged". In 7th grade I was given a 2 year assignment to write a research paper about a topic I was interested in, and a book of steps to follow (use the Reader's Guide to Periodical Literature at the library, take notes on index cards, write an outline, write about each point, rearrange it, and type it on a typewriter - this was before my family had a computer at home). I chose the topic of video games, which at the time were new and debates were raging about their effect on kids. But it was going through the research process that mattered - the self discovery, reading, evaluating what I read, and learning how to think on my own that were the lasting lessons. So in these articles I am proposing alternatives that can be sprinkled in where they can over time as our current methods evolve.

Namespaces for collections of dimensions

The current method in Gzz of rotating across all dimensions is cumbersome. Dimensions should be logically grouped together into namespaces that are relevant for specific applications and even users. This can help avoid name collisions when the same dimension name makes sense in multiple contexts. The current workaround for this is to use multi-dotted dimension names ( ex: d.erangell.primesieve.mersenne ) Mappings of dimension namespaces across users and applications would also be needed, which could be supported by transclusion links.

Exports to standard semantic web formats

Data created in dZez should be portable if users decide to use it in other systems.
So export processes should be developed to serialize all data in the zzstructure to a graph structure.
The graph can then be exported to standard formats (ex: graphML, RDF, gunjs, etc).

How to be protocol agnostic?

Browsers such as Agregore ( https://github.com/AgregoreWeb/agregore-browser ) support multiple decentralized protocols. So if a cell contains a decentralized URL of any supported protocol, the data can be retrieved with such a browser. Agregore will be supporting web archives, which would be a good format that can be added to a dzez hyperdrive so the content can be accessed when disconnected from the internet.

API design

In the decentralized web, APIs can be written in Javascript run on client machines instead of servers. An API should be designed to encapsulate access to dZeZ data structures stored in hypercores so that any front end application can call the API operations to access and manipulate the graphs or zzstructures in the hyperdrives (or other protocols). So an interface should be defined that protocol handlers can implement to provide such API access. This needs to be well thought out before implementing the first version, because once people start using it, breaking changes cannot be made.

Linking cells across dimensions in hyperdrives

Dimensions are defined in subdirectories under the 0 directory of a user's hyperdrive. Cells should be able to live anywhere and be referenced by their tumblers. Ranks in dimensions have json files that create a linked list pointing backward to the head cell. So the json files can include rich links to point to cells. Should each json file in a dimension be a rich link?

Links to comments received about the "dzezthoughts" article

While I am using the dzez directory as an example of how comments on the article might be managed in a zzstructure, I am posting links to comments here for convenience:

https://via.hypothes.is/https://github.com/erangell/Articles/blob/master/dzezthoughts.md

Playlist for videos explaining dZeZ ideas

https://www.youtube.com/playlist?list=PLOW4_Hp8_9101YA30gGhV3AfjOEIJLw8Z

Auto-cloning and always-transclude

Quality of life features for users. Linking a cell to a cell that is already connected in the desired dimension should create a clone and link the clone to the desired cell. The system should never have explicit links to external content. There should always be a transclusion link - a copy of the content should be imported to the hyperdrive and linked with the source (URL, book, etc).

EDL structure and implications for performance

Various forms of EDLs have appeared over the years. If we define it as a list of tumblers, where each tumbler locates a span of content, then it can be a JSON file with an array of tumblers. Indexing within the target file/blob would be a range of bytes. We will want to take advantage of sparse querying provided by hypercore/hyperdrive structures. So maybe hypercores can be used for blobs and textual documents can be stored in hyperdrives. Need to think through various use cases.

Use of encryption for confidential information

Certain material may be shared confidentially among peers and should be saved but not disclosed.
Some material may only be disclosed to members of a web of trust.
The sharing is an act of trust which has the possibility of being broken by any recipient at any time, which causes some type of harm.
This information is part of a person's personal knowledge repository and should be saved somewhere for future reference.
Many encryption products exist which create a keypair and encrypt messages with a public key that can only be decrypted by the holder of the private key. So storage of encrypted files in a hyperdrive is one use case, and re-encrypting files for sharing with a web of trust is another that can be either features or add-on extensions. Encryption technologies will evolve, so periodic re-encryption is required, perhaps by a smart contract periodically reminding the owner or team to do so. The web of trust model can also be used to enforce transcopyright, where upon receipt of a micropayment a document is re-encrypted with the public key of the recipient, thus allowing the transcluded material to be mounted in the user's hyperdrive (or equivalent technology).

Time Locks: Cam a smart contract keep documents with confidential information encrypted for a period of time before disclosure becomes public domain by decrypting it with the author's private key?

Operating system path limits may be an issue for long tumbler directories

Long filenames and path names cause issues over a certain length cause issues on certain operating systems.
For an experimental system this is OK - don't use it for important data.
There is also the issue of logical units of work, where multiple files may need to get updated as a unit in order to avoid data anomalies or inconsistent index structures. Code can be structured so an exception writing any file of a logical unit of work (LUW) invokes compensation actions to rollback changes made to other files in the LUW.

This issue may force the design to use a graph database. It can also be a relational database. However one goal is not to require the user to depend on any specific vendor products. So open source databases can be used if the procedures for maintenance are automated and documented well enough for users to understand.

Documentation of test cases

Test cases can be created and documented in Gzz.
Zip files of Gzz test data will be posted here: https://github.com/erangell/Articles/tree/master/dZeZtestCases

Desired enhancements to Gzz

• warning when an operation you are about to do will break a rank or disconnect a cell
• interference issues with certain keystroke commands I repeatedly hit wrong