cambiatus/backend

Cambiatus GraphQL API

backend's Issues

Remove `isVerified` from the Upsert KYC mutation

We currently allow for users to specify if their KYC is verified or not. This should be an internal process, must be removed for now

Review tests

Our backend tests and structure are a little bit stale and unreliable. We need to improve the overall quality and cohesion of the app

Update the GraphQL API to include the new photo proof fields

Make objectives be completed on the backend

• Adicionar coluna is_completed na tabela de objetivos
• Adicionar tipos GraphQL para essa alteracao
• Atualizar tela de Community.elm para não mostrar objetivos completados
• Atualizar esses dois objetivos acima como completados

Add support contact information

Is your feature request related to a problem? Please describe.
During the development of cambiatus/frontend#270 we noticed that we need a way to allow users to contact support. Currently there is no way to add community support information

Describe the solution you'd like
Adding a setting for adding a WhatsApp / Telegram group and email. Then displaying this on the screens that need it

Describe alternatives you've considered
Maybe have a global hardcoded support channel, but looks less flexible and it would add more stuff for our team to handle, I think it would be better to share support with community leaders, as a lot of stuff is more related to teaching the users than actual bugs

Additional context
N/A

Add the ability to add your phone number

Use the existing update profile mutation.

Also find a way to check if the user has Telegram and/or WhatsApp and flag it somehow

Create sharable invite links

Is your feature request related to a problem? Please describe.
Users have been requesting for a while now a way to invite people to their community. Although we do have a way of doing that it requires you to fill in an email. Maybe it is best for us to create just a sharable link that allow users to register. It will be easier to share via WhatsApp and other messengers apps, and it can still work with email.

Check the `invitations` field from the Profile GraphQl query

Go to https://staging.cambiatus.io/api/graphiql and try to run this query:

{
  profile(input: {account: "lucca"}) {
    invitations
  }
}

Get the 500 error. This must be a cast problem.

Create timeline

We need to create the necessary structure for the new timeline feature, that can be see on this Figma design: https://www.figma.com/file/nEPG6Dkns8hDVCudVUTEzc/Dashboard?node-id=564%3A3

This is a very complex screen and in order to complete it we will need to do a few steps:

• Design a table that need to store all the events on the users's history on Cambiatus. It should allow us to have frontend translatable descriptions, date, relatable data (such as other users, sales, claims, communities, etc). It will also need to have a type so the frontend nows how to render it properly
• Create database triggers that automatically fill in that table depending on what has happened. For bidirectional stuff (such as transfers or shop buys) we need to make sure every user will have its own entry.
• Add a GraphQL entities for the Timeline
• Add GraphQL queries for the timeline. Make sure it works with pagination, as the frontend will need it
• Add GraphQL subscriptions for the timeline. It will also involve database triggers to trigger the subscription

Maybe we will need to check event-source or our smart contract to allow for all the cases:

• Join community
• Create shop sales
• Send/Receive transfers
• Open Claims
• Analyse claims
• Receive rewards for analysing claims
• Receive rewards for having their claim approved

Add account creation validations

Add sign up validation for the account creation process, now we don't make any validations for the input values.

Change database to allow for profile types 30

Users must be a company or a person. We will have to ask for addresses if its a company

• Create migrations for user type
• Add address table
• Make sure Ecto/Postgres complains if there is no address
• Make sure Address table is international (must at least work with Costa Rica)
• This table must allow us to pin point an address on a map later

Investigate memory errors on production

Due to some errors in production, happening during database operations, we have to update our dependencies and OTP, the logs from the errors are very obfuscated so lets try to just update everything as a first try before diving deeper into what's causing this.

API for Addresses auto complete 7

Create an API endpoint to serve a raw JSON with all the locations used for auto complete

Can't access verifications

As a verifier I can see that a verification is pending on my dashboard, but when I click on it, is showing me this error message

This error was reported while testing https://borboins.cambiatus.io

Constraint error when using "upsertKyc" mutation

Error:

constraint error when attempting to insert struct: * kyc_data_account_id_fkey (foreign_key_constraint)If you would like to stop this constraint violation from raising anexception and instead add it as an error to your changeset, pleasecall foreign_key_constraint/3 on your changeset with the constraint:name as an option.The changeset has not defined any constraint.

Query:

{"query":"mutation {\n upsertKyc2069213328: upsertKyc(input: {accountId: "rodrigoky14", countryId: "1", document: "1231231231", documentType: "nite", phone: "11948340013", userType: "natural"}) {\n __typename0: __typename\n }\n}"}

Create Community configuration

We need to create boolean configurations for the community. On the new dashboard community admins can choose what features will be enabled on their community. The designs can be seen here: https://www.figma.com/file/nEPG6Dkns8hDVCudVUTEzc/Dashboard?node-id=1046%3A22

We need to build only a few configurations for now, the are all toggles:

• shop
• objectives & actions
• show community stats

The steps to accomplish this will be roughly:

• Create a new migration that stores those toggles in the database (please lets discuss what's the best strategy before implementing)
• Make sure the migration fill in default values for that table on existing communities (table shouldn't allow NULL values)
• Add Ecto fields corresponding to the new table fields
• Add Absinthe fields corresponding to the Ecto fields (make sure to design NULLs properly)

Add KYC feature toggle 5

This will live alongside with the other feature toggles in our database, and will be used by the frontend to control certain flows

• Add migration
• Add GraphQL types

Create Shop orders 🧽

Is your feature request related to a problem? Please describe.
Today we have products and a very basic structure called orders. Before it was called sale_history because it was really only a registration of buying something from the shop. This proposal suggests a new structure to make the order abstraction more powerful and more useful.

Describe the solution you'd like
First we need to make the order have status. Orders always relate to trying to buy products from the same seller, or user. An order looks like this:

• Product ID list which contains a list of the product IDs that the buyer wants
• Quantity
• Amount
• Status status of the order. This first draft will allow only the basic ones: created, completed
• Payment a JSON or even only a transaction ID. allow us to link an order with a transfer
• Timestamps for creation/updates

Another change is that today, orders are created by listening to this contract call. We should change this to make the creation of an Order to be an GraphQL call. After the order is created we can update it with the transaction ID to link it with the payment. This process can later be expanded to include several payments, until the oder is completed.

With that we can completely remove the transfer_sale call and properly using the order abstraction.

Orders will also need to be approved by the creator of the offer/product. This is because change is to be expected and orders must be accepted

We will need the following screens:

• My orders (as a seller)
• My orders (as a buyer)
• Community orders (for admins)
• Community stats of orders
• Individual order
  • Notion of quantities for products
  • Notion of payments on its own area
  • Contact information
  • Status of the order (a way for the user to predictably know what's next -- wait for the seller, etc)

The new features users may gain

• This will allow us to create interfaces to both seller and buyer to know how their orders are.
• It will be easy to add steps related to shipping or partial payments too
• This will also allow sellers to have greater control, with dedicated shop pages to search and see their orders
• This will make the timeline possible, with our current structure we cannot achieve the result shown in the designs

Describe alternatives you've considered
We could start working on the idea of expanding possible statuses. We could choose to track partial payments (useful for payments with fiat and crypto). We could add tracking capabilities to allow users to inform delivering/posting information and also add stuff like canceled or other abstractions

Additional context
Add any other context or screenshots about the feature request here.

Add staging deploy instructions

(Decide which repo it should be in)

• Add hints on logs
• Add instructions how builds
• Show how to attach a console
• Show how to debug errors on production
• Show how to work with pm2

Add API timeout on all requests

This is avoid us to have hanging requests hitting our servers

Implement multi domain support

We need to support multi domains on our backend.

• Communities should be able to choose a subdomain (we need to have a blacklist filled from the migration)
• Backend should take the host into consideration when receiving requests

Context

cambiatus/improvement-proposals#35

TODO

Handle an error message for wrong invitation id format

Problem

• Try to register with some wrong invitation id, like /register/abc.
• Get the Internal Server Error from the backend.

It looks like this error raises from lib/cambiatus/auth/invitation_id.ex:18, where Hashids.decode is called. Here's the Sentry report.

Possible Solution

Send the "Invitation not found" or maybe "Wrong invitation id format" message instead of 500 error.

It also would be handy to show the appropriate message on the frontend, because it's possible for the User to enter the invitation id by hand and make mistakes. Now it looks like this:

Add GraphQL instructions

Add to README.md:

• Sample queries
• URL

Add subscription for changes in the configurations on a community

Create a subscription that send events when a community configuration is changed, input is a community

Use Waffle on the backend

With extra dip

Wrong link to EOS repo

The Readme file has the incorrect link to EOS project.

Improve Authentication

Add the following features:

• Sessions table, storing expiration, last time requested and validity. Use that to serve and validate session tokens.
• Add validation of message. Client signs a suggested message with their private key and we validate its ownership using the account's public key. Maybe NIFs are the way to go
• Store failed login attempts
• Add auto-invite configurations migrations / mutations
• Adjust the signUp function to respect that auto invite, always using a) the invitation b) the request header's url

For session tokens we must have a way to invalidate certain user access to the database

• Add testing
• Security audit
• Update elixir #49

Some images won't upload from iPhones

Some photos from the iPhone photo gallery won't upload to xes.cambiatus.io, app.cambiatus.io, and staging.cambiatus.io. This Slack thread has an example of a "broken" photo. But the same photo could be uploaded form the localhost (:3000 frontend, :4000 backend).

While trying to upload a "broken" photo from any browser this error appears and the photo won't be saved to S3:

Possible Problem

Some of the problematic photos were autocorrected (adjusted brightness, contract, etc.). Probably these photos have the metadata which breaks something.

I also suspect iCloud. On the iPhone, there’s an option to store only previews on the phone and preload originals from iCloud by the need. All problematic photos were loaded from iCloud right before uploading it to the website.

How to Fix the Photo on the iPhone

This problem could be solved by choosing image size other than actual (iPhone has this option in the select image dialog for upload forms):

How to Fix on the backend

It seems that processing/resizing images before uploading to S3 will help. I think we need to do that because now it’s pretty common for us to have a 2000px wide picture for 80x80px avatar. The same is for the Shop images. We can save much space on S3 and significantly speed up the loading of the website!

As an experiment, I tried to convert the problematic images with Mogrify (a wrapper around ImageMagick) in Elixir and the processed image was uploaded correctly.

Improve error handling on backend

absinthe-graphql/absinthe_plug#138

Add mutation to allow for existing users to update or fill in their KYC informations 25

Users will need to be able to fill in their KYC information at any time, we need to provide a GraphQL mutation for that

We must just make sure we have the right mutation depending on which if its a company or person

Claim analysis history query is missing some results

Some users are not being able to see some existing pending claims on their claim analysis history page (/dashboard/analysis)

That page uses this function to process the request. It is a composable query that allows the user to add status and claimer filters.

One of the reasons it don't work is because of this line:

 where: fragment("?.validator_id = ? OR ?.claim_id IS NULL", ch, ^account, ch),

It makes sure to find only claims whose checks haven't been voted by the logged user yet. This is not the best way to do it. The best way is to add this SQL query instead:

select count(*) from checks where claim_id = claims.id and checks.validator_id = 'USER ACCOUNT HERE'

or in Ecto query:

|> where(
      [c, a, o, v],
      o.community_id == ^community_id and v.validator_id == ^account and c.status == "pending" and
        fragment(
          "select count(*) from checks b where b.claim_id = ?.id and b.validator_id = ?",
          c,
          ^account
        ) == 0
    )

The problem is to add those filters only depending on what the user choose. Today we simply add the status to the query:

  def claim_filter_status(query, status) do
    query |> where(status: ^status)
  end

This function should be smarter to know that if its pending and not voted, the status should be different.

Move away from IPFS

Is your feature request related to a problem? Please describe.
It is very frustrating to use images in the platform. Images take a while to load, sometimes they don’t appear even if the request of saving it worked.

Make sure to migrate shop images in MUDA

Describe the solution you'd like
Just use S3

Describe alternatives you've considered
none

Implement Search

It will be a new GraphQL Object that for now only searches for the shop, it should have a structure to differentiate from other results later

It will not be paginated and will always take two params: communityId and query

Allow users to delete their KYC information 6

We need to add a mutation to delete KYC information from the user profile

Improve error reporting in upsert KYC/Address mutations

Show appropriate error messages when invalid data is sent to the mutations. See this Slack thread for details. Also, consider this comment.

Improve Setup documentation

Add Wiki documentation to the backend repository:

• Instructions for setting up the dev env
• Adding debug techniques and hints
• Local variables (difference between dev.exs and config.exs)
• Instructions to don't commit certain files
• Links for Ecto instructions
• How to setup local ssh tunnels to point local endpoint to staging
• How to dump and restore data for local development

Add subscriptions when something has to be written to the blockchain

Is your feature request related to a problem? Please describe.
In a number of situtations our application has to write information the the blockchain then that has to be transferred to the backend and finally the front end can show it.

Sometimes it's misleading to show a user stale information such as when an action is being created, redirecting back to the objective too soon would make i it look like the action creation failed

Describe the solution you'd like
We need to list the updates that go through the blockchain and make subscriptions for them so that we can have the frontend wait for their creation before it can show a user some results

Describe alternatives you've considered
None

Additional context
Data needs to go to the blockchain then it can be broadcasted to the backend, this usually takes a few seconds and we should initiate waits for this

@lucca65 I think a while ago you listed these scenarios in slack but I can't seem to find the message in question

Add mutation for creating new users 25

We currently have a Rest API for sign_up new users. We'll need to create a new mutation to be used on the new sign_up process. That process will need to have the same functionality we have today (Invitation support and blockchain's netlink call) and also will need to have support for all the information related to KYC -- if the community enables it -- validate it and store it on our servers

Please be aware that Costa Rica has 3 different types of physical person and 3 different types of companies, we will need to find out the data format and validate everything

Misleading instructions in README

I cloned the project and followed the instructions from "Building and running the application locally" section in Readme. I failed because this instructions are outdated. Then I found .github/setup.md and run the project successfully.

How it can be improved
The section with outdated instructions could be replaced with the link to the right ones. Something like this:

## Building and running the application locally
Follow [these steps](.github/setup.md) to run the project on your computer.

Fix phone validation regex

Fix the phone regex for phone/whatsapp contact validation in lib/cambiatus/accounts/contact.ex to support brazilian numbers in the format +55 xx 9xxxxxxxx

Improve backend Sign Up process

• Make sure to validate the KYC data/address before netlink
• Allow KYC/Address mutation to happen in a single request with the signup

Define data structures to hold new information

Create a migration on our Postgres database to hold all the new information related to the claims with images

• On claims:
  • proof_number;
  • proof_photo_url;
• On action:
  • has_proof_photo;
  • has_proof_number;
  • photo_proof_tips;

Images are uploaded without Content-Type

This can potentially cause issues on browser who don't try and interpret images regardless of Content-Type (maily Safari). We should detect the appropriate type from the file's extension and upload the file appropriately.

Remove docker configs/files

Store KYC information 40

We need to find out all the fields necessary and store them the database. It must be on a separated table (so its easier to erase if the user ask to)

• Create migrations for KYC table with all the fields
• Make sure to validate all the data with Costa Rica rules
• That validation must be parametrised, by allowing us to send a country to it. The same must be done on the KYC table
• Make sure we have a flag saying if it is validated or not
• Add GraphQL Types for all user types and queries

Sort objectives

Is your feature request related to a problem? Please describe.
During community actions/objective management the order of the objectives is inconsistent and keeps changing after every update

Describe the solution you'd like
Sort the objectives by creation date, older ones last

Describe alternatives you've considered
Maybe allowing manual control over the order, maybe?

Add Authentication to the API

This issue must hold a discussion on how we should approach this. This will be necessary but it must be done right, after the blockchain receives a signed transaction it must create a new token to be used by the frontend. All logins now will need to include that new transaction, and the backend will need to check if the request made is actually valid

Sign up mutation doesn't return an error when account already exists

Instead it returns a 500 Internal Server Error, leaving the frontend with no information on how to handle it.

Update Elixir version to the latest

make sure our docker build process via make build still working properly

TODO

• Test locally
• Update elixir version to 1.9 in staging
• Update CI config
• Update elixir version in prod

Add support to slug lookup

• Create column with unique slug
• Migrate existing to slug
• Add GraphQL query that allows us to query using slug