brave-intl / bat-ledger Goto Github PK
View Code? Open in Web Editor NEWA BAT-based micropayments system for users and publishers
License: Mozilla Public License 2.0
A BAT-based micropayments system for users and publishers
License: Mozilla Public License 2.0
will require the cards:write
scope
setup scripts have been abstracted to tests. remove setup folder, update readme
To avoid unexpected interactions and allow use of fixtures, tests should clean some or all database data during setup or teardown.
cf., http://swaggerstats.io/apidoc.html#operation/%2Fswagger-stats%2Fmetrics
Name | Type | Labels | Description |
---|---|---|---|
api_all_request_total | counter | - | The total number of all API requests received |
api_all_success_total | counter | - | The total number of all API requests with success response |
api_all_errors_total | counter | - | The total number of all API requests with error response |
api_all_client_error_total | counter | - | The total number of all API requests with client error response |
api_all_server_error_total | counter | - | The total number of all API requests with server error response |
api_all_request_in_processing_total | gauge | - | The total number of all API requests currently in processing (no response yet) |
nodejs_process_memory_rss_bytes | gauge | - | Node.js process resident memory (RSS) bytes |
nodejs_process_memory_heap_total_bytes | gauge | - | Node.js process memory heapTotal bytes |
nodejs_process_memory_heap_used_bytes | gauge | - | Node.js process memory heapUsed bytes |
nodejs_process_memory_external_bytes | gauge | - | Node.js process memory external bytes |
nodejs_process_cpu_usage_percentage | gauge | - | Node.js process CPU usage percentage |
api_request_total | counter | methodpathcode | The total number of all API requests |
api_request_duration_milliseconds | histogram | methodpathle | API requests duration |
api_request_size_bytes | histogram | methodpathle | API requests size |
api_response_size_bytes | histogram | methodpathle | API response size |
and have bat-ledger
and bat-eyeshade
talk to it instead of the exchange rate services directly
the /v*/publisher/*
endpoints are read-only for the browser, so it's a good thing to move over.
there is an invalid error check that occurs on ledger/controllers/registrar.js:164
when hitting /v1/rates
immediately after the server starts up, a subset of rate conversions are sent back, instead of the full 20, which is denoted in the helper tests.
To reproduce, shut everything down, start up the servers, and run the helper
integration test and see it fail the first time because only 8 values are sent back. Sometimes 9 are sent back, depending on if other tests are running in parallel.
when starting up the extractor worker, the relation download errs in
npm run start-extractor-worker
saying that it does not exist.
during the test for settling a grant, the tester should not have to manage the db before running the tests. manual management can cause the db to be in poor state and for tests to fail. instead, automate by making sure that during a test, a single grant is added to the db, it is used, and subsequently flushed from the redis db by the end of the test.
In the Brave browser, the payments tab has publisher verified green checkmarks. This green checkmark is queried from Ledger.
Instead of Ledger fetching this information from Eyeshade, we're going to fetch the verified_status for an owner from the publishers api.
We'll be disabling the data propagation in a separate task.
might be hard to pull off due to latencies, but I think there's a potential side channel attack
@diracdeltas fyi
for batching voting surveyor
cc: @evq @davidtemkin
currently only non-website channels are verified (all are sent by publishers), but with brave-intl/creators-private-issues#940 we will need this change to keep things in sync until we can remove eyeshade's copy of the channel/owner relationships
e.g., DASH, BCH, BTG.
this requires adding the appropriate network address. we probably want to rework the algorithm so that an address generation request is sent to uphold only if the user selects a particular alt-currency (at present, a wallet creation gets addresses for BTC, ETH, and LTC).
Publishers are reporting that their statements are not generating.
See: https://twitter.com/robek_world/status/981563504686784517
Many publishers have requested that their accounts be deleted. We do not have an easy way to do this. Right now we must manually delete the records from both eyeshade and publishers. This is error prone and not scalable.
We should create some endpoint on eyeshade that publishers can hit that deletes all records associated with an owner (owner, their channels, token, etc). Something like this would work:
DELETE /v2/owners/{owner}
Related to https://github.com/brave-intl/creators-private-issues/issues/969
see https://github.com/brave-intl/bat-ledger/blob/master/ledger/controllers/wallet.js#L244, we use the client provided surveyor. depending on client behavior perhaps we can reject payments where the surveyorId does not match that of the current surveyor
The root cause appears to be fixed in bitgojs
BitGo/BitGoJS@6b2296f
we should update when a new release is available
currently we expose the list of currencies that the user has a card for, we should add possible currencies such that the response includes both lists
If you've registered potato.com and become a Brave publisher; and potato.com expires and Cate Camper registers it right away, contributions to potato.com should no longer flow to you.
eyeshade, as the canonical registration checker, should be the service which checks periodically.
it also needs communicate this info with publishers: https://github.com/brave-intl/creators-private-issues/issues/1211
use -- to pass flags to the following scripts
lint
docker-up
To simplify Eyeshade and dedupe data, we decided to move the Site channel verification flow so it's entirely within Publishers (https://github.com/brave-intl/creators-private-issues/issues/940). The Publishers task and this one need to completed together.
As a result, Eyeshade would no longer have verification tokens or unverified channels.
(TODO: Check if unverified channels exist in Eyeshade).
there are a variety of improvements that could be made to make the code more legible and testable
these items can be addressed in any order, as well as broken up by server / worker
blacklisting publishers currently happens manually on voting document basis
mark publisher's excluded boolean if they are registered in our blacklist
if excluded is true, do not include in payout information
take the publisher, check the blacklist, flip the bool if
create mongodb indeces for blacklist and query them
eyeshade/workers/wallet.js:voting-report
mark excluded
eyeshade/workers/reports.js:report-publishers-contributions
sanity check to make sure no publishers in report have
Deprecate these calls in the ledger:
GET /v2/publisher/identity
GET /v2/publisher/identity/verified
Move these calls from the ledger to the eyeshade server:
POST /v2/publisher/ruleset
PATCH /v2/publisher/rulesets
PUT /v2/publisher/ruleset/{publisher}
DELETE /v2/publisher/ruleset/{publisher}
GET /v2/reports/publisher/rulesets
Add messages from eyeshade to ledger that provide the functionality in the five calls above.
currently we have a number of ledger clients stuck using old surveyors instead of the current one. it seems most likely that this is caused by brave-intl/bat-client#35 in combination with the fact that some clients reconcile date was not pushed back on grant claim and grants were made unredeemable before their maturity date.
we will fix the client side - to go out in a browser-laptop release but as a quick fix we should unstick clients by upgrading old surveyors servers side on demand.
while it is implicit currently, the promotion creation under the npm script create-promotion should be a test that can be isolated and integrated with the rest of the testing flow
TypeError: reconnect is not a function
we were sending unix time (as is incorporated into grants), but the browser assumes javascript style ms since epoch. changed in the server promotion payload since that doesn't require a new release.
we should retire github auth in eyeshade and other services where possible, we're moving toward a publishers based admin panel and our github auth has known issues, e.g. it does not work across dynos:
if the first hit to v1/login (redirects the user to github) and the second (user is redirected back from github) happen on different dynos, the login does not succeed
apps we could remove it for currently:
bring up all relevant interoperating services
ledger services, grant server, referral server and publishers
To implement https://github.com/brave-intl/creators-private-issues/issues/960 publishers needs to know the total amount an owner earned during the last settlement.
It appears as though a settlement
document only has the information for a single channel
, and we are only returning one settlement
per owner
in this request.
bat-ledger/eyeshade/controllers/owners.js
Line 467 in 4f7562a
cc @evq
isolate why this is happening and allow tests to turn off / reduce time during testing
during the script
npm run test-integration
or, if using the docker container
docker-compose run --rm -v $(pwd)/test:/usr/src/app/test ledger-web npm run test-integration
the following error is received. if the endpoint is hit too often
status: 429
http://ledger-web:3001/v1/grants/d6a9c116-5779-4442-a0dc-6237e295ae29
{ statusCode: 429,
error: 'Too Many Requests',
message: 'try again in 86400 seconds' }
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.