stellar-deprecated/stex

DEPRECATED. An opinionated framework built on top of expressjs used to build better web services for stellar

stex's Issues

A argument for an auto-loading stex

tl;dr; We should both standardize the directory layout of stex for both "model" and "controller" files and automatically load them at stex initialization

Problem

The CommonJS model for adding module dependencies is nice for being explicit about inter-module dependencies and has overall led to much better javascript code. However, it can lead to very large sections of "require" blocks at the top of each file that can distract from the function of the module, and more importantly it leaves each module the freedom to name the symbols that it imports, leading to naming inconsistencies:

// in module 1:
var wallet = require('wallet');

// in module 2:
var Wallet = require('wallet');

Furthermore, within a single node project you are forced to use relative requires, leading to friction when you restructure your project or rename a file:

// when in a sibling module
var wallet = require("./wallet")

// when in a child module
var wallet = require("../wallet")

Proposal

There exists a set of modules within a stex application that are used quite often throughout nearly all the other modules in a stex application: the "models" and the "controllers". By automatically loading these files we will reduce duplication and increase clarity. I propose we automatically load files underneath /lib/controllers and /lib/models. For example, the routes file (app.js) could go from this:

var walletsCtrl = require("./controllers/wallets")
var totpCtrl    = require("./controllers/totp")
var recoverCtrl = require("./controllers/recovery")

router.post("/v2/wallets/create", walletsCtrl.create);
router.post("/v2/wallets/update", walletsCtrl.update);
router.post("/v2/totp/enable",    totpCtrl.enable);
// etc.

to:

router.post("/v2/wallets/create", stex.controllers.wallets.create)
router.post("/v2/wallets/update", stex.controllers.wallets.update)
router.post("/v2/totp/enable",    stex.controllers.totp.enable);
// etc.

Migrate gulp dependencies to peer dependencies

Investigate incorporating pm2 into stex

https://github.com/Unitech/pm2 is pretty badass. Lets see if we can leverage it in the context of stex.

Provide a "unique request id" in stex.log

When a stex application recieves a request, it would be nice to attach a unique id to every log that arrises from that request. that way logs could be tracked by request id. the asynchronous nature of node makes this hard as you lose context in the callback after the event loop.

here's a potential solution or model we could use https://datahero.com/blog/2014/05/22/node-js-preserving-data-across-async-callbacks/

Ideally most of the functionality would be abstracted into Stex, and all the client would need to do is add some stex speicific middleware to their request chain, and any further call to Stex.log will automatically add that request id to the log.

redis authentication

is it possible to use remote authenticated redis DB's with stex? error is:

Error: Ready check failed: NOAUTH Authentication required.

publish stex to npm

Stex is not yet published on the public npm repository. Would it be possible to add it ?

Add “workerCounter” config variable, override-able on the cli, that controls how many forks we make when launching the server

Add "stex.reportError" helper function

It's come to my attention that in stellar-api we're forwarding any call to stex.error.log to sentry. This is not a good situation. It means that error reports in stellar-api are actually just log lines that alias most of the useful information encoded into a Error object. It seems that this solution was chosen because we did not have defined with stex a reusable system to report errors.

We should add this functionality at stex.reportError which should behave similar to the stex errors middleware: It logs as well as reports to sentry as peer operations, not dependent ones. From the code:

    var reportErrorDirectly = function(err) {
        log.error(err.message);
        // report to sentry if we are enabled
        if(sentry) {
            sentry.captureError(err);
        }
    };

I propose we extract that functionality to a method stex.reportError, such that the above code becomes:

    var reportErrorDirectly = function(err) {
        stex.reportError(err);
    };

New Initialization system

Similar to the rails initializer system, the stex initializer system will allow applications to define custom "startup" and "shutdown" functions as well as a dependency mechanism to allow code inside an initializer to leverage other portions of the framework

Domain-based error handling for background tasks

Fix CORS domain wildcard matching

Use url-pattern instead of minimatch for domain matching.
https://github.com/snd/url-pattern#match-domains

Fix Crash-on-startup issues

Currently, a stex service looks like it is running, even if every worker process immediately dies.

Yeoman generator for new stex applications

We should build a generator for http://yeoman.io/ that will bootstrap a new stex-based project.

Unrecoverable Error: Cannot enqueue Query after fatal error.

If a connection is killed by mysql, or the running stex application otherwise gets its connection into a fatal state, stex does not properly recover and the service requires a restart to resume normal function.

Instead, stex needs to validate/reconnect to the db at the HTTP request boundary if no connection is available, and we need to close connections after we receive a fatal error.

Import email infrastructure from fbgive into stex

Add "scripts" support

STORY

As a developer building an application on top of stex, I would like to be able to easily defined multiple entrypoints for my application, so that I can build one-time processes, cron-based repeated jobs, or multiple web service APIs on top of one codebase

RATIONALE

Presently, there are only 2 types of processes that can be started on top of stex... the www process started via stex run www and the repl process started via stex console. The goal of this issue is to unify how we launch processes within stex as well as enable an application to define their own entrypoints.

For example, you may have a custom job queue running process that you want run within the stex ecosystem (giving access to the db,log,conf globals). Or you may have written a database cleaning process to resolve some prior corruption.

DESIGN IDEAS

The following paragraph describes a possible design:

Every stex application has a top-level folder called scripts. By default, it is populated with 2 files: www.js, and console.js. These two scripts can be launched with either stex run www or stex run console respectively.

A script launched in this manner (via stex run NAME) will execute after having the stex ecosystem loaded and initialized.

A developer can define additional scripts simply by creating new files within the scripts folder, which can then be run as expected by executing stex run. In this way, stex run NAME is just an alias for:
node scripts/NAME.js.

Add Git sha and instance ID to response headers

Previous discussion: stellar/stellar-api#54

Change mocha reporter

Current one looks bad in Travis.

Switch query logging to debug

See 01df521#commitcomment-8995392 for context. I would just change it, but don't have stuff set up for local testing yet, so filing it for now.

Promise-enabled REPL

STORY

As a developer running in a stex console process, I want to be able to execute javascript that returns a promise which is then resolved before printing, so that I can have a more friendly asynchronous experience when using the REPL.

RATIONALE

Presently, the stex repl process is very bare-bones... simply using the built in node 'repl' module with no customization. Given that most of the useful code in node is asynchronous (and for stex, promise-based), the repl is quite hard to use because you pretty much end up having to write lines like: db("wallets").where("id", id).select().then(console.log)

This is a pain and slows down the explorative experience of a repl, IMO. It would nicer instead if we built a repl that saw the return value of a statement was a Promise, resolve it, then printout the value for the developer... essentially automating the .then(console.log) part.