tmilar/meli-manager

Node.js + MercadoLibre API + Google Spreadsheets API = 🎉

meli-manager's Issues

Further clarify setup instructions in README.md + other issues

Some project instructions are not entirely clear yet. Items I can think of right now:

• How to get Google Spreadsheet API keys (as a private app, for the time being)
• Decide and describe which license to use
• Add Travis CI badge
• any other?

Treat Spreadsheet sheets as an user resource, not as app configuration

Implement a REST API for user sheets.
Stop referencing sheets in app configuration env variables.

Also, 'active' sheets should be re-assignable.

Extract MeLi API access logic

Many places will need to use MercadoLibre API (read + write), while supporting multiple accounts aggregation.

There doesn't seem to exist any good MeLi node client implementation for this task, so it might be a good idea to extract refactor all MeLi API access to a centralized MeliClient implementation.

When two or more notifications of a same new order arrive (almost at the same time), duplicated rows are written.

Sometimes, when more than one notification for one (not yet stored) Order are received, both are written in spreadsheet almost simultaneously, which leads to duplicated row

Possible solution:
Serialize the spreadsheet writing requests to prevent duplicated entries. For example, by enqueing the writing requests, then writing one at a time. Or at least one at a time per resource (order) id.

As an User I want to be able to authenticate so I can manage my own resources such as meli accounts, spreadsheets and more.

Suggested: passport authentication which is easy to integrate with current express structure.
Preferred Google Authentication since we'll also need to integrate with Google Spreadsheets OAuth.

Use snapshot testing for the help flags/options (meli-accounts CLI)

Related: vercel/micro#345

Originally posted by @fmiras in https://github.com/tmilar/meli-manager/pull/40/files

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

As a developer I want to create, register and authenticate/refresh MeLi Test accounts to test different parts of the app.

Currently a test user account can be created using Mercadolibre API, but this retrieves only a {nickname, password} pair, and has some problems:

• The same test account can't be retrieved ever again
• Creation limited to 10 accounts per Client Owner (that is, per Application / MERCADOLIBRE_CLIENT_ID)
  • Accounts are only deleted automatically after 60 days of inactivity
• The Account access token can only be refreshed using client owner Application secret id.

Provide a tool/script to manage Mercadolibre Test accounts easily.

Use valid JSDoc comments or remove the invalid ones.

There are a lot of invalid JSDoc comment blocks that need to be fixed, or else removed.
Running xo will complain about this with warnings.
Expected: xo doesn't complain about this.

Example of wrong case:

/**
 * Callback after Mercadolibre authorization.
 * Uses default meliAuth.onAuth() callback, but it can be overwritten for custom logic ie. CLI app.
 *
 * @param accessToken
 * @param refreshToken
 * @param profile
 * @param done
 * @returns {Promise<*>}
 */
const authorizedCb = async (accessToken, refreshToken, profile, done) => {
  // ...
}

Example of correct case:

/**
 *
 * @param {Object} loggedUser             - the mercadolibre logged user info
 * @param {Object} loggedUser.profile     - mercadolibre account profile info
 * @param {Object} loggedUser.tokens.<accessToken, refreshToken>
 *                                        - mercadolibre account tokens
 *
 * @returns {Promise<void>} - exec promise
 */
async function registerAccount({profile, tokens}) {
  // ...
}

Originally commented by @fmiras in https://github.com/tmilar/meli-manager/pull/40/files#r224301316

Optimize handling of entity row Save vs entity row Update

Right now, the entire row is being written in the spreadsheet.

If it's a Save operation, it will append the row in the end of the sheet document.
If it's an Update operation, it will overwrite the row that matches the 'id' field. But, in this case, we probably should only write the desired/updatable fields, not all of them?

For example: if a question/answer text is banned/deleted, we probably should not overwrite the info from the spreadsheet when updating the row.

This would require a refactor in the current flow.

improve API docs

API docs are not 100% complete, ie. response info is missing.
Some guidelines:
https://gist.github.com/iros/3426278
https://bocoup.com/blog/documenting-your-api

orders spreadsheets data is incomplete and inconsistent

There are some fields that are relevant to Order entity that are being ignored such as shipping costs, order id, buyer id, and more.

Also order status and payment status is broken in some scenarios.

Long spreadsheet writing operations can turn into an unhandled exception.

The current spreadsheet library doesn't handle a rare scenario when the spreadsheet given locks are lost.

To reproduce, call /order?store=true endpoint (with no dates limiter) to get a large operations writing request, then wait for this case to hit.

Also, the same library not only doesn't handle this error, but also uses some weird setTimeout callbacks in it's implementation, therefore, using this after promisifying doesn't allow to handle this exception which bubbles to the event loop itself and breaks the application.
.
I'll provide logs/screenshots/more references later.

MeLi Accounts CLI Tool needs better naming + better features documentation

This package name does not describe at all what it actually does, re-think it!
Originally posted by @fmiras in https://github.com/tmilar/meli-manager/pull/40/files/ded8c9013e1ce09734cd0ddac85d435d5e7060fa

Startup app services in order, exit early on failure

The db connection, and other services such as SheetsService need to start up before the express server should be able to listen for requests.

---

This is a must have connection for the app, you should try this connection before setting up the express server, otherwise you're running a server and executing more code in vane since you already know that there's a critical error. Also show a more friendly error, a simple process.exit(1) just sucks

Originally posted by @fmiras in https://github.com/tmilar/meli-manager/pull/40/files/ded8c9013e1ce09734cd0ddac85d435d5e7060fa

Write spreadsheet rows in parallel instead of in series

Until now, rows are being written in series to:

• allow recalculation of sheet size between rows
• check of errors between each write.
• prevent collision with other writing sources ie. manual inputs

But, so far, there have been almost no errors in this process, and the sheet size re-calculation can be done in advance with some extra care (some 'lock' on the rows to be written, maybe a temporal cells paint to 'reserve' them).

It can be a good time to work on parallel writing.

As an user, when my MeLi Account authorization expires/invalidates, I want to be informed to decide what to do (ie. login again or remove the account)

When a MeLi account authorization (not to be confused with the access_token) expires, we currently have no way of noticing it.
A solution could be:

1. handle MeLi API response auth errors (except regular access_token expiration, in that case we just refresh)
2. store the auth error in the Account document
3. all further requests for that account must be ignored/fail prematurely.

Somewhere:

1. inform the auth error to the user, suggest to login again

When user account logs in and authorizes again:

1. remove stored authorization errors