nebulouslabs/skynet-js

A Javascript module made to simplify communication with Sia Skynet portals from the browser.

skynet-js's Issues

parseSkylink regex parsing issue

bug recreated in test case #94

Improve skylink param handling

Improve skylink param handling:

• validate skylink [a-zA-Z0-9_-]{46}
• allow sia://<skylink> (strip sia://)

Directory uplaod returns Undefined

https://siasky.net/docs/?javascript--browser#uploading-a-directory

async function uploadDirectoryExample() {
	try {
		var files = []
		var f1 = document.getElementById('profilePicture').files[0]
		f1.webkitRelativePath = '1.jpg'
		files.push(f1)

		const directory = files.reduce((accumulator, file) => {
			const path = skyid.getRelativeFilePath(file);

			return { ...accumulator, [path]: file };
		}, {});

		console.log('directory', directory)
		const { skylink } = await skyid.skynetClient.uploadDirectory(directory, 'asd');
		console.log('skylink', skylink)
	} catch (error) {
		console.log(error);
	}
}

client.db and client.registry and associated functions are not typed properly

#141 (comment)

Set default maxBodyLength to Infinity

We should set maxBodyLength and maxContentLength to Infinity when making axios calls. See NebulousLabs/nodejs-skynet#49

v2.1.1 uploadFile returning undefined

Using the code from the SDK documentation

async function uploadExample() {
  try {
    const { skylink } = await client.uploadFile(file);
  } catch (error) {
    console.log(error)
  }
}

return undefined on successful upload

`dataKey` is passed twice into `skynetClient.registry.setEntry(...)`

dataKey is passed twice into skynetClient.registry.setEntry(...)

example from https://siasky.net/docs/#setting-data-on-the-registry

import { SkynetClient, keyPairFromSeed } from "skynet-js";

const client = new SkynetClient();
const { publicKey, privateKey } = keyPairFromSeed("this seed should be fairly long for security");

const dataKey = "foo";
const data = "bar";
const revision = 0;
const entry = { datakey, data, revision };

async function setEntryExample() {
  try {
    await client.registry.setEntry(privateKey, dataKey, entry);
  } catch (error) {
    console.log(error);
  }
}

Variable dataKey is passed once to create entry

const entry = { datakey, data, revision };

and then also as a second parameter to setEntry

client.registry.setEntry(privateKey, dataKey, entry);

Solution proposals

// (my choice)
const entry = { datakey, data, revision };
client.registry.setEntry(privateKey, entry);

or

const entry = { data, revision };
client.registry.setEntry(privateKey, dataKey, entry);

or

client.registry.setEntry(privateKey, dataKey, data, revision);

Replace mime-db dependency

Inspired by #319 I looked into mime-db -- it seems like the original usage was prompted by an issue with react-dropzone on the web portal homepage.

This dependency is just a large JSON file of all known MIME-types. It seems like this would be a developer's responsibility to manage, and at 200kb unpacked, it's almost 20% of the space in our code.

Option 1: Fix react-dropzone and drop dependency.

Option 2: Follow their documentation's advice:

If you're crazy enough to use this in the browser, you can just grab the JSON file using jsDelivr. It is recommended to replace master with a release tag as the JSON format may change in the future.

We could host the file on skynet and have the client load it from the current portal if an upload of unknown MIME-type ever occurs.

Option 3: Consider a library like mime. It's a self-contained module bundled with a pre-optimized version of the mime-db dataset, and intended for browser use. mime-db, when minified and compressed, is 18KB, mime is 8KB, mime/lite is 2KB.

Change request: keyPairFromSeed and generateKeyPairAndSeed should return keys as strings

Functions keyPairFromSeed and generateKeyPairAndSeed should return keys as strings, not buffers, otherwise if I want to print the publickey to user, I need to install a buffer package and use toString("hex") myself which is an extra step and I think totally unnecessary.

closes #165

Reorganize utils

The utils.ts file is stuffed with a bunch of different functions by this point. I think it would be itneresting to organize them into a utils directory with different files (for example, theres a few functions for manipulating URLs that can go into utils/url.ts).

Add support for production build

We could add support for production build that would allow people to use https://unpkg.com/

#131 (comment)

getJSON and getEntry should throw on real errors

Right now, we consider any rejected response from api to be "empty record" for both getEntry and getJSON due to the fact that we had to reject it ourselves on timeout and couldn't rely on a specific error code from siad. Now that #218 implements timeout properly, we should be returning empty records only in an event when siad rejects with the not found error, otherwise we should be rethrowing the original api error.

Method to detect if user is using a portal account

Pending either a response header or an API endpoint on Web Portal:
https://github.com/NebulousLabs/skynet-webportal/issues/657

Skapps that expect their storage to be "long-term" may want to indicate to users that they are not logged into the portal using an account. Even "free" should be fine.

Include credentials with Axios call

GET and POST calls need cookies sent with Accounts.

fetch("https://siasky.xyz/AACU3d_HIQ8jkSPdQwn415O0AVvvP6jo9RALPXWUltwcfQ", { credentials: 'include'})

For axios:
https://stackoverflow.com/a/43178070

Some functions have first letter capitalised

functions like HashAll, HashDataKey, NewHash, HashRegistryEntry should have first letter lowercase

Encrypted upload/download is not encrypted

I think the required skykeyName is just ignored when uploading encrypted file.
https://siasky.net/docs/?javascript--browser#uploading-with-encryption

(Would be great to be able to encrypt SkyDB JSON files also)

Add method to client for identifying current Skynet portal URL

We need a way for Skapps to reliably know what web portal they're being accessed through.

This will need to use headers the portal passes or an endpoint accessible even when at an HNS subdomain. The first seems far preferable.

Inconsistency in method naming

generateKeyPairAndSeed and keyPairFromSeed

One starts with a verb generate, the other doesn't. Since we're doing some breaking changes anyway, let's change keyPairFromSeed to generateKeyPairFromSeed.

Typescript support

Add typescript support.

Base32 encoding and hns subdomains support

We introduced 2 new features recently:

Base32 skylink subdomain access

You can reencode skylink with Base32 and use it as a subdomain to access your content on siasky.net:

• Regular skylink: https://siasky.net/XABvi7JtJbQSMAcDwnUnmp2FKDPjg8_tTTFP4BwMSxVdEg
• Base32 skylink in subdomain: https://bg06v2tidkir84hg0s1s4t97jaeoaa1jse1svrad657u070c9calq4g.siasky.net

Converter: https://convert-skylink.hns.siasky.net/ (conversion explained and documented on GitHub)

We should add support to either return regular skylink url or the subdomain skylink url when returning skylink url in SDK. preferably regular skylink url by default and subdomain skylink url if some optional param is set to true.

We should also add a utility function that converts Base64 (regular) skylink (46 chars) to Base32 skylink. There are already decoder and encoder functions here https://github.com/kwypchlo/base32/blob/master/src/crypto.ts and usage example is https://github.com/kwypchlo/base32/blob/master/src/App.tsx#L24-L25 but I guess it could be one function in the SDK (instead of two).

Handshake subdomain access

You can access your content bound to the hns domain using *.hns.siasky.net subdomain:

Regular hns: https://siasky.net/hns/convert-skylink/
New hns subdomain: https://convert-skylink.hns.siasky.net/

We should add a param to function that return hns url to either return regular one (default) or the one in subdomain (optional param set to true).

Modify SkyDB to optimize initial read/write times

Current db.setJSON does the following in series:

1. Tries to read if registry entry exists (5 second timeout)
2. Upload Skyfile (~2 seonds)
3. POSTs new registry entry

• Option 1: The first 2 calls are very expensive and can be done in parallel.
  https://siasky.net/IADUtc2kNnhgWVISPvel9H_Lazb8RjOghSYFgn_mIkqcTg
  Finished in #305

• Option 2: Idea needs tested, but providing a "skylink" instead of a JSON object to setJSON and getJSON. Then "initial value" skylinks can be used to try to initialize the SkyDB entry at registry rev0 without checking if the registry entry already exists or not. Also, this bypasses the first skyfile upload to save time early in the user experience and ensures that any subsequent visit to the site will not result in the registry read having to timeout.

new method request: getRegistryEntryUrl

I need to print a url after creating the registry entry. Currently I do it like this:

const encodedPublicKey = encodeURIComponent(`ed25519:${publicKey}`);
const encodedDataKey = encodeURIComponent(Buffer.from(HashDataKey(dataKey)).toString("hex"));

console.log(`https://siasky.net/skynet/registry?publickey=${encodedPublicKey}&datakey=${encodedDataKey}`);

I would like a function called getRegistryEntryUrl that would allow me to modify the above code to be

console.log(skynetClient.registry.getEntryUrl(publicKey, dataKey));

Follow-up about content type

From #239 (comment):

What happens if we don't specify the content type? Would it be detected?
No need to cover this now but it might be interesting to verify the behaviour.

Change request: `keyPairFromSeed` should include `specifier` in returned value

Function keyPairFromSeed should also include specifier in the response object, set to "ed25519" (it wouldn't be needed if private key and public key would be strings in the response so close this one if #162 is accepted).

Change request: `skynetClient.registry.getEntry` should return `{ entry: null }` on entry not found

Function skynetClient.registry.getEntry now returns null if entry is not found so we need to write

const entry = skynetClient.registry.getEntry(...);

if(entry) {
  console.log(entry.entry.data);
}

and it works fine (not even mentioning weird entry.entry chain) but I think usually people would write

const { entry } = skynetClient.registry.getEntry(...);

but it will throw an error if getEntry returns null.

I think we should return { entry: null } if the entry is not found instead of just null so it's easier for people to destructure the result and saves us weird accessing entry.entry.

parseSkylink should return null on not found

From #141 (comment).

Do not set https://siasky.net as a default portal

It is easy to obscure the otherwise explicit definition of the portal if we make our portal a default. This could cause bad user experience if people start relying on https://siasky.net free service as a de facto way to use the library and our portal starts having problems or we take it down for updates or sth. We should include it in the examples but the actual library should stay portal agnostic.

@m-cat FLW: the same should apply to all our SDKs

Originally posted by @kwypchlo in #37 (comment)

Include the new functions in usage examples

FLW: We should include the new functions in README.md examples

Originally posted by @kwypchlo in #37 (comment)

genKeyPairFromSeed() returns bytearray instead of HEX

genKeyPairFromSeed() returns something weird bytearray instead of HEX

Solution: Use this function instead of .toString('hex')

Investigate large webpack bundles for skapps

When a skapp imports skynet-js and builds with Webpack (what we recommend in tutorials), the resulting bundle pulls in the entire SDK, even if the skapp only used a single function from skynet-js (e.g. defaultPortalUrl()).

build pipeline overwrites Babel output with tsc

Currently build runs babel and then tsc to generate .js files. tsc actually overwrites the babel generated files, meaning we should either:

1. Add "emitDeclarationOnly": true, to tsconfig.build.json as addressed here.
2. Remove Babel from the build process, since we're not using a mixed JS/TS codebase (and since we haven't been using it for npm for quite awhile.)

getSkylinkUrl with subdomain: true doesn't work from base32 skapps

Example:

My skapp https://vg4hm678pvjltuh5nmmfimdi9jud7tggoda3eo4651uioqrii5oali0.siasky.net calls getSkylinkUrl("_AkeLeS84huwnOKEdP9o36dE1qJkmwHCVpzyFcMhv5d33A", { subdomain: true })

The result is https://vg4o10vtuj8j5np6rm4ujohmpaoksi9f9ei5ecnkguch74qgepbiobg.vg4hm678pvjltuh5nmmfimdi9jud7tggoda3eo4651uioqrii5oali0.siasky.net/.

The expected result is https://vg4o10vtuj8j5np6rm4ujohmpaoksi9f9ei5ecnkguch74qgepbiobg.siasky.net/.

This is probably impossible until https://github.com/NebulousLabs/skynet-webportal/issues/423 is implemented.

requestFile Function

Currently, the SDK uses very similar ways of directing the browser to "download" or "open" files, both by calling the window object.

As I work more with skynet-js, I'm wondering if "requesting" a file and returning the data of the Axios response (and perhaps skylink metadata) makes sense, especially for txt, json or other filetypes.

I'd propose getFile for this, but that overlaps with SkyDB function names, so perhaps client.requestFile(skylink)?

I'd be happy to do the initial implementation.

Don't export default options

The default options are being exported for compatibility with Go, but they shouldn't be because the user has no use for them and they only clutter the API

Add download option to support no-response-metadata

The download options object should be expanded with a new option to pass the no-response-metadata HTTP parameter

parseSkylink returns false positive when running from app with a skylink in url

When running app in a browser from an url with a skylink (for example when your app is uploaded to skynet), parseSkylink returns that skylink from the browser url even if the argument passed to parseSkylink should throw an error (ex parseSkylink("foo")).