oceanprotocol/market

🧜‍♀️ THE Data Market

market's Issues

publish: make it work against Rinkeby

Publishing an asset via https://market-six.now.sh needs to work

Web3Feedback: add loading state

Upon activating the wallet users see the message "Not Connected to Pacific" right away although they are connected to Pacific.

This message will stay there for as long as it takes for the app to connect to the Ocean components. While technically correct, it is giving users the impression they have to do something. We should add a loading state for the time between activating wallet and showing the network connection state.

Flag a dataset and give notification to marketplace

kremalicious commented 26 days ago •
Define "Flag". Define "notification". What does a user see? What does a user do? What is happening in background when user does it?

People can upload data easily (and we don't store it)

Summary
Provide publishers with an easy solution to drag & drop some files during publish flow and in background they are added to decentralized storage. But where we don't store it or pay for storage.

Motivation
Status quo: publishers must a provide url for their data, which takes technical savvy. Removing this friction will help adoption.

In Ocean Commons, we had where people could "upload to IPFS" for a smoother flow. They would drag and drop the file, and it would get uploaded. However since IPFS does not do storage, we were actually doing the storage on Pacific nodes. This is not a good idea because then we inadvertently start hosting a ton of files for free.

Target outcome

• have a GUI / UX similar to what we had in Ocean Commons
• but store somewhere that we don't have to pay for.

Where to store

• Recommended: Arweave - pay once, store forever
• (Backup: Filecoin - pay once, store for a long time)
• IPFS could still be used as the intermediary ("IPFS pinning"). Data would be encrypted.

Publish: Include summary pop up with data asset details after Submit

-- kremalicious

This sounds like having a confirmation dialogue with a summary before we start the publish process? Cause after publish we redirect to the actual asset page.

This would imply a slightly different UI flow for the whole publishing, which then should happen in that confirmation popup, including the wallet connection. Would actually make more sense blocking the UI during publish with a popup, think we require people to stay on page during the publish process.

---

i think it would be better a step/wizard approach than a popup

Add custom provider endpoint during publish

In Ocean Market, OPF has to run Provider and therefore holds the data service decryption key. This means that OPF has custody of these keys.

What to do for this issue:

• In Ocean Market, give an option for the user to provide an endpoint of the Provider. Therefore the user can run their own Provider, and retain decentralization. We can expect enterprises and others to do this. Handled in #702
• In Ocean Market, if the user chooses to use the OPF-run Provider, have a GUI affordance to set the expectation that this Provider will only be run until some specific cut-off date: Dec 31, 2021. This ensures that OPF is not bound to some weird long-term commitment that it didn't mean to make. Handled in #703
• Allow external provider in ocean-lib-js Issue
• Allow external provider in ocean-lib-py Issue

Note: in the future we will want more decentralized storage of private keys. See related tech spike.

Note: moved from this issue

fallback for pool fetching

Just like price display, fetching pools should have a fallback for non-web3 browsers and disconnected wallets.

Add support for adding multiple files on asset

It might be useful for some use cases to allow users to add more than one file when publishing an asset

publish file error when adding

When you add a file in publish you get an error : " unexpected end of file"

As a <Publisher>, I want to bulk publish data with APIs / Scripts so that I can save time with an automated process

web3connect update to web3modal

Should we update to https://github.com/Web3Modal/web3modal since web3connect is deprecated?

wallet - disconnect button

Should we have a disconnect button? If the user wants to disconnect. He can do so from metamask but there are several clicks involved.

Wallet interaction UI/UX

I really like how they handle the wallet
https://gnosis-safe.io/app/#/welcome

wallet UI long balance

If you are rich the wallet doesn't resize properly. So we can do one of the following:

a. Shorten the balance, but how i understood from previous discussions the size is ok based on exchanges
b. Make the wallet size dinamic
c. Make the wallet wider so that the longest value looks ok.

completely implement ocean-mock

ocean-mock has a minimal implementation so that tests run but stories don't reflect the actual scenarios. This impacts mostly compute and job related components.

-----kremalicious
good point, because of the way jest works we should rather change the scope to mocking squid. We have to create tests/mocks/@oceanprotocol/squid.ts and this will automatically be picked up by all the tests and just with ocean-mock we can use it in the stories too.

asset detail URLs throw 404 on Vercel

When accessing an asset url directly, Vercel throws their own 404 error.

E.g. going to this asset from front-page works correctly but accessing it directly does not:
https://market-six.now.sh/asset/did:op:7df3412a97ab4f8ab6dc5115f27c9f09b384fdf080e84103a225a77bdfaf9277

Publish - when adding file check HEAD then OPTIONS

A lot of common api's don't support HEAD, so in case this fails check OPTIONS. If both fail give a more explicit error

for example https://finances.worldbank.org/resource/xnse-jvg2.json

web3-react

Is it worth implementing https://github.com/NoahZinsmeister/web3-react/ instead of classic web3? Not sure it is..

Transactions -show created/finished time in jobs

If you have multiple jobs on the same item on the same day you need to identify them properly.

Wallet - ocean and eth precision

We should show only a fixed number of digits in the balance so it doesn't look like this

"License" needs a commercial option as default

Ocean marketplaces, including Ocean Market, are meant for data that's for sale.

In Ocean Market, there's a dropdown for "License". Currently all the options are for free / open data. But this is a marketplace for commercial data. Therefore we need an appropriate license. And, it should be the default.

We should be able to take the terms developed for ascribe, and use them here.

I also question whether we want to offer any of the free / open licenses (Creative Commons). They're for open documents. Historically we had Ocean Commons, which was for open data, but it was really a demonstrator, with no real USP. If people want free / open data then they can simply use Web protocols directly, no need for extra Ocean machinery in between.

Pool default weight and LP fee

Currently the default (hardcoded) values for weight and fee are 9 and 0.03. We discussed the 90% weight but we never discussed the fee for liquidity providers (LPs) to the pool. I randomly set it to 0.03 , I don't even know what % this means

user preferences for fiat currency

1. global store for user preferences, connected to localStorage
2. User preferences UI somewhere, with currency selection as first option
3. Modify price conversion fetches to include more currencies, and pick the one based on user preference

Sample File upload for data assets

As a i want to add a sample file when i publish so people can see the structure and quality of the data

negative prices

Only happens when multiple prices are fetched at same time. Clicking on asset with negative price then will show correct price on asset details.

Wallet - burnet wallet to show prices without connecting

A quick solution to always show prices ( even to users that didn't connect a web3 wallet) is to add a default burner wallet that will be used to get prices.

[EPIC] that price thing

Figure out the best UI to get the values needed for publishing:

Battle plan

• new UI for fixed price
• new UI for dynamic price
• new price form widget
• dynamic price: collect and pass correct values to publish method
• output price on asset details page
• output price in asset teasers
• fixed price: wait for new fixed contract, and addition to lib-js
• fixed price: collect and pass correct values to publish method
• visual distinction between fixed/dynamic price on Price output
• output price in asset preview during publish
• fix price display when user is not connected to correct network (should get fixed with #51)
• figure out UI for liquidity/pool infos/providing liquidity/removing liquidity
• output all possible fees during asset creation

User Stories

1. As a publisher I want to receive 10 OCEAN when someone downloads my data set.
2. As a publisher I want to receive 10 OCEAN when someone runs a compute job on my data set.
   ...

Ideas

• Should we have a distinction between Use (Download/Compute) & Trade, like we prepared for asset details already?
• create data token for each data set?
• have user decide on data token ticker name?

IPFS - add gateway to sample ( in display component)

Improve hint about why Publish Submit is disabled when Wallet is not activated

As stated in this comment from @maxieprotocol :

Visually, there is no clear correlation b/w form submit and activate wallet section.

Perhaps we shouldn't keep it disabled but rather provide feedback to the user that he must activate the wallet in order to submit the form the same way as we do with the rest of mandatory fields by
implementing the snack bar warning for wallet activation (that would be the faster solution to fix that)
Later the wallet activation would be a part of the wizard step

Not sure if this is an issue anymore, but we need to take this in consideration in the new design for the market

Transactions - button to refresh jobs

Publish errors and current step

We need to handler errors better and show status of publishing

switch form library

This is a proposal to kick out react-jsonschema-form if we decide to continue to polish the POC after the final deadline. Going forward this should provide a more future-proof way of solving all those remaining and newly found issues with current publish form.

Background
Almost everything we want to do (with one form only so far) requires like 1 week of research and 1 week of implementation for minimal changes. With that speed we would take weeks to get the publish form in a user-friendly state. As of right now, nothing fits together visually with countless interaction and usability problems. Live validation is not working. Quickly creating a new form which works out of the box is not possible. Etc.

This is all because react-jsonschema-form tries to solve too many things at once without a clear separation of concerns. It has its own vocabulary and system (Field, Widget, Template, Object Field Template, Schema, UI Schema, what?), uses its own custom UI system and markup, which we constantly try to overwrite, not just in this app. This is also reflected in the library size, almost 100KB when gzipped(!), so we just add to that when doing more and more customizations on top of it. It is simply too inflexible and requires tremendous amount of tweaks to make it work in our projects so far.

This opens up too many unknowns e.g. when dealing with a new project too. What if we want to use another UI framework? Then we would start the research again from the beginning to somehow fiddle together the UI.

Proposal
I think a form library should only deal with handling form data, capturing, validating, saving. That's it, only some minimal API to do just that. Pure functionality, no opinionated UI or markup.

We will always put in the UI, we never want pre-styled forms we then have to figure out how to overwrite. If data management and UI are separated, we reduce the risk of ending up in dead ends where something we need from a form can't be done because library xyz does not support it.

Formik comes to mind, which has a really minimal API, and can be connected with Material UI too. All with 7KB gzipped.

We could then create our own Input/Form component on top of it, which opens up possibility to use any UI in any app in a clean way, without overwriting styles. And we have full control over markup and components. We could still use a json file to create a form with that.

Any other suggestions?

Include summary pop up with data asset details after Submit (publish page)

Tags autocomplete

The UI draft implies some sort of autocomplete for tags during publishing flow. This is very useful to unify all tag names so we don't end up with different cased variations of the same tag.

This needs multiple things to work:

1. UI: a tag within the input itself is rendered as a small, dismissible Material UI Chip
2. UI: the Tags field needs an autocomplete UI where users start typing a tag and a dropdown appears showing all available tags based on user input.
3. Aquarius: to make the UI work, we need a list of all tags in the system which ideally would come from Aquarius, like as described in oceanprotocol/aquarius#223

Another workaround for the consolidated metadata could be to construct and maintain that on frontend, where we have some provider which constantly queries all assets in Aquarius and returns that consolidated metadata based on that. This should only be last resort though, the client RAM usage probably goes through the roof with that.

This is a copy paste from daimler, we will probably not be using material ui

publish form: add entry for date created

Port over the date picker for allowing publishers to set dateCreated. This could follow the pattern we already had where publishers can choose a specific date or a date range for when the data was collected.

When going with date range, the output of dateCreated needs to be updated too.

Consumed whitelisted assets don't shown up in Transactions/Downloaded table

unjapones commented 7 days ago
Downloadable assets that have been whitelisted and downloaded for a given account, do not show up as an entry on the Transactions page, table Downloaded.

kremalicious commented 7 days ago
we get consumed assets from chain and that requires an existing agreement for every asset consumption.

So we either:

actually create an agreement with 0 price when whitelisted users consume an asset
track downloads in a database for each account
Given that 2. is rather big to setup, we might want to look into 1. again

Price without connected wallet

I've been thinking about this and the conclusion is that it's not possible in the current state of the app. The problem is that you need a web3 connection and you need an ocean instance. To get a price you need to know the desired network.

My suggestion is that for now we have the constraint that to see prices you need to connect a wallet. When we go on the mainnet we can make a default dummy wallet that has the mainnet config hardcoded

I added the blocked label because we don't have a proper mainnet config

Drag & drop into decentralized storage

Goal should be to provide publishers with an easy solution to drag and drop some files during publish flow and in background they are added to a decentralized storage network.

This could start as providing an IPFS upload for sample files, since they can be public.

Storing actual data set files needs some more advanced solution to preserve data privacy.

Add file fails if resource doesn't support HEAD method

for example https://finances.worldbank.org/resource/xnse-jvg2.json

let publisher choose data token name & symbol

During publish we want users to choose a name & symbol for their to-be-minted data token.

minimum viable search

Quick wins with current technology where we use Aquarius nativeSearch in the /search route. This also means all sorting & filtering actions fire a server search to Aquarius:

Sorting

• by default, sort results by date created, newest first
• sort results by liquidity, from highest to lowest (ddo.price.ocean)
• sort results by price, from highest to lowest (ddo.price.value)
• needs UI pattern for sorting actions

Filtering

• filter to show only pools/only exchanges
• needs UI pattern for filter actions

Currency value display

We use https://github.com/coingecko/cryptoformat for formatting our values but that works weirdly in some places:

• numbers below 1 get a lot of decimals, no matter the currency
• a full number like 1 will be displayed with lots of 0 decimals too

Requirements

We either hack current usage into following those rules, or use another library.

• For fiat: never more than 2 decimals
• For crypto: never more than 5 decimals
• 1.00 needs to be displayed as 1
• 0.100000 needs to be displayed as 0.1

Fixes #53

Have a Sort function (by Price, Data Name, Ratings) (explore page)

Sort assets by liquidity

1.Sort by liquidity
2. Have tabs to filter AMM/FixedRate
3. Sort by latest pools

reset form action does not clear localStorage correctly

Clicking RESET FORM correctly resets the Formik state for the form, but formik-persist seems to fail clearing the localStorage entries properly, resulting in the form values being restored after browsing away and back to the publish page.

price component should handle all price logic

note to self: the price output component should be made into a more functional component which itself fetches the price, and handles all the visual states based on response

Review T&C

We need to review the Terms & Condition, they were copy pasted all the time.

wallet UI glitches

Some weird visuals in wallet popover when not connected to correct network:

• there never should be NaN
• spacing above network warning is too much

As for balance this is tricky. Technically it should not matter to which network I'm connected to, cause I can have a balance in each of those. The wallet UI should actually show those balances then, based on selected network in MetaMask.

But we use ocean to get the balances. If we only use web3 functionality on frontend this would actually work.

Publish - incorrect balance validation

If I try to publish an asset with dynamic pricing and I add more than 4 ocean tokens I get the following error:

Publish - you don't get any feedback if you cancel sign in metamask