Giter Site home page Giter Site logo

stronk-dev / livepeerevents Goto Github PK

View Code? Open in Web Editor NEW
4.0 2.0 2.0 18.54 MB

API which aggregates and caches in one central location: coin prices, blockchain data, ENS data, Livepeer smart contract events and monthy statistics

Home Page: https://stronk.rocks/livepeer

License: The Unlicense

JavaScript 95.36% HTML 0.38% CSS 4.25%
javascript livepeer lpt nodejs

livepeerevents's Introduction

What

This project shows a live feed of events happening on the Livepeer BondingManager smart contract. This way it is able to track:

  • (De)activations of Orchestrators on the Livepeer network
  • Orchestrators calling reward
  • Orchestrators updating their commission rates
  • Delegators earning staking and fee rewards
  • Delegators delegating or moving their delegated stake
  • Delegators unbonding their stake

Orchestrators can be inspected by clicking on their address, showing all of their delegators, earned fees and stake

API endpoints:

View the Wiki for a list of all available endpoints, their parameters and an example of their output

How

Frontend

The frontend can be used on it's own as a basis for hosting your own Orchestrator website

Running Standalone

By default the website will forward all API requests to http://localhost:42609 Meaning, it will look for the backend at the same place where the website is running! If you want to run this frontend without running your own backend, you have to configure it to always pull it's data from the existing API at https://nframe.nl/ Modify package.json to do this

  • Edit the line containing "proxy": "http://localhost:42609" and replace "http://localhost:42609" with "https://nframe.nl/"

Dependencies

nginx, certbot, certbot-nginx, npm

HTTPS using nginx

replace /etc/nginx/nginx.conf with supplied one, certbot will upgrade it to HTTPS

  • systemctl enable --now nginx.service
  • certbot --nginx

Initial Config

Download copy of repository and change directory to it

  • git clone https://github.com/stronk-dev/LivepeerEvents.git /var/www
  • cd /var/www

Download all external dependencies we are using

  • npm install

Edit default Orchestrator Address

If you are not running your own backend, by default the backend will return my Orchestrator In order to show your own Orchestrator on the Grafana page, you need to edit src/util/livepeer.js Edit the line containing:

export const getCurrentOrchestratorInfo = () => (
  fetch("api/livepeer/getOrchestrator", {
    method: "GET",
    headers: {
      "Content-Type": "application/json"
    }
  })
);

And change this to (replace with your own address):

export const getCurrentOrchestratorInfo = () => (
  fetch("api/livepeer/getOrchestrator/0x847791cbf03be716a7fe9dc8c9affe17bd49ae5e", {
    method: "GET",
    headers: {
      "Content-Type": "application/json"
    }
  })
);

All Grafana panels point towards where I am hosting my own Grafana dashboard. You need to edit this to pull the panels from your own Grafana instance For each panel you want to replace:

  • Open your own Grafana page
  • Click on the menu of any Panel and click share
  • Open the Embed tab
  • Copy the entire Embed HTML bit
  • Edit src/grafana.js:
  • Replace any iframe block with the one you copied and edit the height to your liking. The width parameter can be left out so that they scale to the width of the parent element

Note that you can specify the timerange and update frequency in the URL by using URL parameters, like

  • from=now-2d&to=now&refresh=5s

Developing

Open the directory of your frontend and run

  • npm start

Your local environtment is available at http://localhost:3000/

Running or updating in Production

  • cd /var/www
  • npm run build

(Optional) Run your own Backend

Note that running your own backend is not required, since you can also use the existing API at nframe.nl Nonetheless, running your own backend is recommended so you can make changes as well as keep things quick if you are far away from Western Europe, where nframe.nl is hosted

Dependencies

nodejs, npm, pm2, mongodb

Standalone for Prometheus/Grafana

The backend can be run in 'simple' mode. In this mode the entire API stays functioning, except for the /getEvents API call. Perfect for using the API just for pulling the data as JSON or using the Prometheus endpoint and requires much let configuration to set up.

Example Prometheus.yml entry for accessing the API using Prometheus: - job_name: 'livepeer' scheme: https scrape_interval: 30s metrics_path: /api/livepeer/prometheus/0x847791cbf03be716a7fe9dc8c9affe17bd49ae5e static_configs: - targets: ['nframe.nl']

MongoDB Cloud

If not running the backend in simple mode, the free tier of MongoDB Cloud Services is recommended in order to store blockchain Events persistently. Otherwise the backend has to parse all blockchain events each time it boots

Initial Config

Download copy of repository and change directory to it

  • git clone https://github.com/stronk-dev/LivepeerEvents.git /var/www
  • cd /var/www/backend

Download all external dependencies we are using

  • npm install

Set configuration variables

  • nano /var/www/backend/src/config.js
  • PORT can be left as is, defines at what port the backend will accept API requests
  • NODE_ENV can be left as is, defines the default way to connect to the database
  • MONGO_URI should be edited to the URL of your MongoDB instance. This is where the backend stores it's data when it is running in production mode
  • MONGO_URI_DEV should be edited to the URL of your MongoDB instance. This is where the backend stores it's data when it is running in development mode
  • MONGO_URI_LOCAL can be left as is, this is where the backend stores it's data when it is running in local mode
  • API_CMC should be edited to contain your own coinmarketcap api key
  • API_L1_HTTP should be edited to the HTTP url of a L1 ethereum node
  • API_L2_HTTP should be edited to the HTTP url of a L2 ethereum node
  • API_L2_WS should be edited to the WS url of an Arbitrum mainnet Alchemy node
  • CONF_DEFAULT_ORCH should be edited to the public address of your own Orchestrator. This is the default Orchestrator the backend will return
  • CONF_SIMPLE_MODE if set to true, will disable live smart contract monitoring. Also disabled all MONGO related functionalities. Config flags API_L2_WS, MONGO_URI, MONGO_URI_DEV and MONGO_URI_LOCAL can be ignored if this flag is set to true.
  • CONF_TIMEOUT_CMC time in milliseconds of how long coinmarketcap data is kept in the cache before a new request is made. Recommended is around 5-6 minutes in order to stay below the default daily soft cap
  • CONF_TIMEOUT_ALCHEMY time in milliseconds of how long blockchain information, like gas prices, is being kept in the cache
  • CONF_TIMEOUT_LIVEPEER time in milliseconds of how long livepeer data, like orchestrator and delegator information, is being kept in the cache
  • CONF_DISABLE_SYNC set to true to disable syncing events and tickets at boot
  • CONF_DISABLE_DB set to false to disable persistent storage of events and tickets into a mongodb database
  • CONF_DISABLE_CMC set to false to disable coin quotes using CMC

Developing

Open the directory of your backend and run

  • npm run dev

Running in Production

Open the directory of your backend and run the backend using pm2

  • cd /var/www/backend
  • pm2 start ecosystem.config.js --env production

Save the current pm2 process list configuration

  • pm2 save

Automatically start the backend when the server reboots

  • pm2 startup

Updating in Production

  • pm2 restart backend

Other commands

View logs

  • pm2 log backend

Stop the entire thing

  • pm2 stop backend

livepeerevents's People

Contributors

stronk-dev avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

livepeerevents's Issues

Add filter buttons

Pass a name to the eventViewer in order to filter out specific transactions

Add buttons to livepeer.js in order to set this to ALL, TranscoderActivated, TranscoderUpdate, Reward, WithdrawStake

TBD event types, which we might throw away or not:
EarningsClaimed
Unbond
TransferBond
Bond
Rebond

Rework filterbuttons

Normal: inset effect
When active: hide that type of event and grey out the color

Do something with the filtered out events

At least check if we are not missing any events like stake moving around

Events we are skipping:
// Filter
if (eventObj.name === "WithdrawFees" || eventObj.name === "TransferBond"
|| eventObj.name === "Rebond" || eventObj.name === "Unbond" || eventObj.name === "EarningsClaimed") {
return;
}

Fix unstake events

https://nframe.nl/livepeer?orchAddr=0x08f10D03A0CF7a9eADdc7EacD4cf135a07A0feff

has a 10K unstake from https://explorer.livepeer.org/accounts/0x5ec2be1adc70bc338471277c2dcc183b0b2c91be/delegating

{
"address":"0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash":"0x8bec6edb47f5085c61abf28e0ff9a70280929fc7d9cc95b615df485881361d8d",
"transactionUrl":"https://arbiscan.io/tx/0x8bec6edb47f5085c61abf28e0ff9a70280929fc7d9cc95b615df485881361d8d",
"name":"Unbond",
"data":{
"0":"0x08f10D03A0CF7a9eADdc7EacD4cf135a07A0feff",
"1":"0x5Ec2be1aDC70Bc338471277c2dCc183b0b2C91be",
"2":"1",
"3":"10000000000000000000000",
"4":"2491",
"delegate":"0x08f10D03A0CF7a9eADdc7EacD4cf135a07A0feff",
"delegator":"0x5Ec2be1aDC70Bc338471277c2dCc183b0b2C91be",
"unbondingLockId":"1",
"amount":"10000000000000000000000",
"withdrawRound":"2491"
}
},
{
"address":"0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash":"0x8bec6edb47f5085c61abf28e0ff9a70280929fc7d9cc95b615df485881361d8d",
"transactionUrl":"https://arbiscan.io/tx/0x8bec6edb47f5085c61abf28e0ff9a70280929fc7d9cc95b615df485881361d8d",
"name":"EarningsClaimed",
"data":{
"0":"0x08f10D03A0CF7a9eADdc7EacD4cf135a07A0feff",
"1":"0x5Ec2be1aDC70Bc338471277c2dCc183b0b2C91be",
"2":"118964187072793932562",
"3":"0",
"4":"2481",
"5":"2484",
"delegate":"0x08f10D03A0CF7a9eADdc7EacD4cf135a07A0feff",
"delegator":"0x5Ec2be1aDC70Bc338471277c2dCc183b0b2C91be",
"rewards":"118964187072793932562",
"fees":"0",
"startRound":"2481",
"endRound":"2484"
}
},

Replace imported draggable scroller package

Currrent scroller is very simple.
Try something which handles large list sizes better, since dragging when there are many elements becomes very slow
Would be nice if you can scroll like on mobile, where it slowy comes to a stop when releasing the drag button

Add timestamp to events

Display datetime in locale format

Would require some major modifications, since the backend does not even attach a date. Maybe just add the time an event was inserted into the database?

Add a simple mode for the backend

Leave out all of the event stuff, only use the backend as a caching mechanism for the CMC, L1, L2 and thegraph stuff. Only stuff we want to leave in are the JSON outputs and the prometheus integration

Limit of 100 delegators in the Orchestrator Viewers

https://api.thegraph.com/subgraphs/name/livepeer/arbitrum-one/graphql
Due to a limitation in gql queries, it is only getting max 100 delegators. Something like this does not work:
{
transcoders(limit: 2,
where: {id: "0x525419ff5707190389bfb5c87c375d710f5fcb0e"}) {
activationRound
deactivationRound
active
lastRewardRound {
id
length
startBlock
endBlock
mintableTokens
volumeETH
volumeUSD
totalActiveStake
totalSupply
participationRate
movedStake
newStake
}
rewardCut
feeShare
pendingFeeShare
pendingRewardCut
totalStake
totalVolumeETH
totalVolumeUSD
serviceURI
delegators {
id
bondedAmount
startRound
}
delegator {
id
bondedAmount
startRound
}
}
}

Maybe we have to try a different query, or see if this is a limitation in the api itself:
https://github.com/livepeer/livepeerjs/tree/master/packages/subgraph
Maybe we need to create a PR there to get more than 100 results, which seems to be the default limit

Convert components into more subcomponents

All components are currently just in the root directory
A lot of components also have conditional subcomponents which should be split off into separate components
Todo: organize the components and split off large components into multiple subcomponents

Fix Bond eventButton

Doesn't display the right information right now. Should parse nested obj.subEvents in order to display better info

Find a way to render a large list of Events

When fully synced, we have thousands of events

Loads in a few seconds, but can be quicker if we simply don't render any event which is too old

Biggest problem being the filters and search function, which should function on the entire set of events

This is a problem since right now the render functions determines the type of the event bundle

Split transaction into multiple sub-transactions

Right now in a transaction events like moving stake take priority over claiming eanings. During some rebond events claiming and moving stake happens in 1 transaction. We should split a transaction into possible subtransactions, or just leave out earnings all together

[
{
"address": "0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash": "0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"transactionUrl": "https://arbiscan.io/tx/0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"name": "WithdrawFees",
"data": {
"0": "0xDFc452Da1fA53f5DCD1EBc63A3E5f6C5F9139F76",
"1": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"2": "6915574260931571",
"delegator": "0xDFc452Da1fA53f5DCD1EBc63A3E5f6C5F9139F76",
"recipient": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"amount": "6915574260931571"
}
},
{
"address": "0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash": "0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"transactionUrl": "https://arbiscan.io/tx/0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"name": "Rebond",
"data": {
"0": "0x525419FF5707190389bfb5C87c375D710F5fCb0E",
"1": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"2": "0",
"3": "13175701400114191076727",
"delegate": "0x525419FF5707190389bfb5C87c375D710F5fCb0E",
"delegator": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"unbondingLockId": "0",
"amount": "13175701400114191076727"
}
},
{
"address": "0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash": "0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"transactionUrl": "https://arbiscan.io/tx/0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"name": "EarningsClaimed",
"data": {
"0": "0x0000000000000000000000000000000000000000",
"1": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"2": "0",
"3": "0",
"4": "1",
"5": "2483",
"delegate": "0x0000000000000000000000000000000000000000",
"delegator": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"rewards": "0",
"fees": "0",
"startRound": "1",
"endRound": "2483"
}
},
{
"address": "0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash": "0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"transactionUrl": "https://arbiscan.io/tx/0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"name": "TransferBond",
"data": {
"0": "0xDFc452Da1fA53f5DCD1EBc63A3E5f6C5F9139F76",
"1": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"2": "161",
"3": "0",
"4": "13175701400114191076727",
"oldDelegator": "0xDFc452Da1fA53f5DCD1EBc63A3E5f6C5F9139F76",
"newDelegator": "0x2cC53Ae643038a51C0F10964efEdb04285607771",
"oldUnbondingLockId": "161",
"newUnbondingLockId": "0",
"amount": "13175701400114191076727"
}
},
{
"address": "0x35Bcf3c30594191d53231E4FF333E8A770453e40",
"transactionHash": "0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"transactionUrl": "https://arbiscan.io/tx/0x23b41d7f7fd388766484d7288ec164171a185e15d09a694270c091164ecdbafe",
"name": "Unbond",
"data": {
"0": "0x525419FF5707190389bfb5C87c375D710F5fCb0E",
"1": "0xDFc452Da1fA53f5DCD1EBc63A3E5f6C5F9139F76",
"2": "161",
"3": "13175701400114191076727",
"4": "2490",
"delegate": "0x525419FF5707190389bfb5C87c375D710F5fCb0E",
"delegator": "0xDFc452Da1fA53f5DCD1EBc63A3E5f6C5F9139F76",
"unbondingLockId": "161",
"amount": "13175701400114191076727",
"withdrawRound": "2490"
}
}
]

Show Orchestrator info when clicking on a delegator

Right now if you click on a delegator in the eventViewer, it clears your current selected orchestrator info

On clicking an address in the event viewer, set a loading screen in the orchestrator viewer, if getOrchestrator returns an empty list, getOrchestratorByDelegator (separate issue) and remove the loading screen once we have an orchestrator. If we do not have an O at this point, show an error

In src/livepeer.js render the OrchestratorViewer if we have a loaded Orchestrator object, else the text

Rework loading screen

We will be increasing the amount of processing done in the loading screen

Also show a progress indicator or an error thingy if there is no backend connection

Create new layout for /livepeer page

Maybe make the topbar a side bar? find a way to make the feed taller at least
Would also make scaling for mobile easier if all components are less wide

Think making events 3 lines tall and very narrow?

Fix Mobile scaling

livepeer.js header, as well as the event viewer and buttons do not scale for mobile devices yet

[Backend] Expand fullSync option

Since watching for events can cause you to miss one or two

Rather than start from scratch, keep track of what range of blocks have been added live and what range of blocks have been synced by getting all events

Then on boot continueSync all blocks which were added live

Any events which come in live do not get added to the database, but only cached serverside

Add DB Fullsync optie

Manier om de Event DB helemaal opnieuw te syncen, alle events uitlezen

Bij booten doen we fullsync totdat alle blokken behandeld zijn
Dus een teller voor block verwerkt, een lege collectie, en dan alle events niet cachen, alleen aanmaken in de DB
Tijdens de fullsync, geen contract listener aanhebben
Als afgerond, gehele lijst ophalen en in de cache stoppen. De timeout op 1x per uur zetten ofzo
Contract listener vervangen door een timed event die blocks in de gaten houdt

Add support for Livepeer Subgraph

https://thegraph.com/hosted-service/subgraph/livepeer/livepeer

We can then create an endpoint in the API to get all details on a given Orchestrator:
{
transcoders(where: {id: "0x847791cbf03be716a7fe9dc8c9affe17bd49ae5e"}) {
activationRound
deactivationRound
active
lastRewardRound {
id
length
startBlock
endBlock
mintableTokens
volumeETH
volumeUSD
totalActiveStake
totalSupply
participationRate
movedStake
newStake
}
rewardCut
feeShare
pricePerSegment
pendingPricePerSegment
pendingFeeShare
pendingRewardCut
totalStake
totalVolumeETH
totalVolumeUSD
serviceURI
delegators {
id
bondedAmount
startRound
}
delegator {
id
bondedAmount
startRound
}
}
}

New Readme

Should contain more info on what the program does, requirements, how to install, how to update, other useful commands

Make Guide for hosting your own frontend for your own Orchestrator

Make a separate guide on how to use just the frontend and my API in order to easily host a website for your own orchestrator

steps:
installing dependencies
cloning repo
modifying my orch address to theirs
modifying grafana panel links
build the website
explanation on updating and and resetting branch to master
modifying nginx config
certbot

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.