cgewecke/eth-gas-reporter

Gas usage per unit test. Average gas usage per method. A mocha reporter.

eth-gas-reporter's Issues

Only supports unique method signatures

@iamchrissmith raised this. They're trying to compare two erc20 token implementations side by side. At the moment we track method data by signature hash (using abi-decoder) and if there are duplicates, all invocations are collected under it, even if they belong to different contracts.

Enabling this reporter severely decreases the ability to debug failing tests

Without this reporter, a failing test will print out the error messages as soon as the run finishes. Was it a revert, a opcode or invalid jump, etc. Stuff that really helps figuring out the issue.

With the reporter, it does no longer print out these errors. Which means that I can use it as long as my tests all pass, but as soon as one fails, I have to disable it to fix it.

Duplicate method entries?

Byzantium upgrades

• testrpc dev dep to v6
• Pure / View
• Check tx status to filter thrown tx's

Table not produced when tests fail

Output test duration

hey, firstly: thanks for this tool!

was wondering whether there was a way to output the test duration using this reporter.
trying to identify our bottlenecks in a quite large test suite

Support Web3 1.0

The gas reporter inherits web3 from the Truffle test environment and it's likely this will be upgraded to Web3 1.0 soon. Sync web3 methods will be async.

(Suggested via email by Sai)

Show price/eth and gwei gas price in table header.

Error: Synchronous requests are not supported.

Seen at Neufund, when using the provider in truffle.js. . . the pattern looks like this:

    inprocess: {
      network_id: "*",
      provider: TestRPC.provider({
        accounts: Array(10).fill({ balance: "12300000000000000000000000" })
      })
    },

The errors then come from our use of sync web3 methods like web3.eth.getBlock('latest)

NaN ?

Internal functions are not estimated.

Hi dear @cgewecke,
first of all thanks for the great tool!

Maybe I'm wrong or missing something but I've noticed that internal functions are not estimated in the report but they are in the inline test report.

I've been working on this little project TokenLocker contract where I've implemented this pattern:

contract TokenLocker is Ownable {
....
function release(address _to) internal ... { ... }
function() public {
        release(msg.sender);
    }
}

The tests are running cool
(e.g ✓ should release the tokens to the first beneficiary expecting the LogRelease event (47139 gas))

but in the resume table at the end I got 0 estimation of the release function
| TokenTimelock · release · - · - · - · 0 · - │

It could be my lack of experience.
Thanks in advance!

Start deployments data search at block zero?

Maybe this should be an option for CI or something. Otherwise lots of old data in a long running testrpc instance might corrupt the vals. On the other hand, lots of data is missing if you deploy in the migrations and use those instances.

Investigate gas cost variances btw geth / ethereumjs-vm

Still not alphabetically listed!

Profile reporter to find out why it's slow

Config

• gas price and eth price
• if these are present, calculate cost per test. This is useful if you want to approximate costs of contract deployment in a dedicated sub-suite. Or show these values in CI.
• currency code (list the available codes in a doc).

Add option to output to markdown

Probably:

• decolorized
• to file

Hide zero usage in the units logging

Transitive dependency contains a CVE

Mocha 3.5.3 depends on node-growl 1.9.2 which contains this CVE: https://nvd.nist.gov/vuln/detail/CVE-2017-16042

Recommended action is to update to node-growl 1.10.3, which means updating mocha to 4.x as 3.5.3 was the last 3.x release.

I tested with mocha 4.1.0 and everything seems to work fine.

Collect gas data for specified tests only

This would useful for speed improvements alone. It's common to want to collect data for a discrete simulation that sits within a larger test corpus. Might be convenient to be able to use Mocha's @test flagging to be more selective.

(Suggested via email by Sai)

Support Libraries

truffle-debugger has a nice binary identification algo . . . that I will steal.

Picture link

Add median stat, use min gas value to calculate the currency cost

1. In any well written test suite a substantial number (if not most) of the tests will be about ensuring that error conditions throw.
2. We're at a turning point with testrpc where shortly it will begin refunding unused gas for most of the transactions that fail.
3. Averages are useless for both of these cases - right now the vals are getting skewed way high, going forward they will skew way low.
4. Atm we should foreground the min as the most plausible 'real' cost?
5. In future we should foreground the max?

Add Travis

Test stop at string.match(regex) when a certain contract is in the test

Hey there,

I am trying to connect the reporter to daostak/arc (https://github.com/daostack/arc) but for some reason, the test just stops.
We tried to debug it and found that this line causes the reporter operation to stop:
const regExp = new RegExp(`^${binary}`); at getStats.js line 76.
This only happens when the UController.sol contract is in the project when we remove this contract the reporter starts to work again.

Would appreciate any help on the matter.

Filtering out unused contract methods

It is possible that a lot of methods implemented in contracts will be not covered (for different reasons: small coverage, optional tests called, etc.). For those cases, I think having only called Methods in table desirable.
Let's have this as an option in .ethgas.js

Add examples in docs

Mocha has own implementation of parameters for reporters

Subj. Why using .ethgas.js needed? Parameters are stored under reporterOptions
For example: https://github.com/michaelleeallen/mocha-junit-reporter

Empty test cases report gas consumed

Consider the following diff creating an empty test case:

diff --git a/mock/test/metacoin.js b/mock/test/metacoin.js
index f7e6397..b146518 100644
--- a/mock/test/metacoin.js
+++ b/mock/test/metacoin.js
@@ -34,6 +34,7 @@ contract('MetaCoin', function (accounts) {
       assert.equal(metaCoinEthBalance, 2 * metaCoinBalance, 'Library function returned unexpected function, linkage may be broken')
     })
   })
+  it('should not consume any gas when an empty test case runs', function() {});
   it('should send coin correctly', function () {
     var meta

We should not expect this test case to report gas being consumed. However, it does:

  Contract: MetaCoin
    ✓ should put 10000 MetaCoin in the first account (347432 gas)
    ✓ should call a function that depends on a linked library (347432 gas)
    ✓ should not consume any gas when an empty test case runs (347432 gas)
    ✓ should send coin correctly (398425 gas)

You can also see that the previous test which only uses calls and doesn't send any transactions also reports gas usage even though the calls do not consume gas.

Tested on macOS 10.13.5, truffle v4.1.11, eth-gas-reporter v0.1.17 (and v0.1.8)

Add number of calls column

Round avg gas

Add some basic tests

Doesn't work with truffle develop

On 4.0.0-beta.2 the tests don't execute and an empty table is printed.

On 4.0.1 mocha reports reporter not found:

truffle(develop)> test
"eth-gas-reporter" reporter not found
Error: invalid reporter "eth-gas-reporter"

Tried installing the reporter globally and got the same error.

Report not available for calls for contracts created by other contracts

No results are shown for contract created by another contract in test. Example is the ReputationMiningCycle contract calls in
https://github.com/JoinColony/colonyNetwork/pull/179/files#diff-16a74db0d2b552059a5414340b64eafdR186

Error: Could not find artifacts for ... from any sources

From Aragon. Apparently it's not a given that artifacts are generated for everything?

Report table doesn't count calls for other contracts

I have a bug with the report table, it displays incorrect gas usage. I have a contract which in one of its functions calls the decreaseApproval function of a StandardToken contract (https://github.com/OpenZeppelin/openzeppelin-solidity/blob/master/contracts/token/ERC20/StandardToken.sol) and fires an event.
The cost which the table displayed for calling that function was 22,204, this is about the cost of the basic tx fee and a single event, it looks like it didn't count the call to the other contract at all.

Add gasStats

Alan Lu wrote these really cool gasStats methods for Gnosis that show average gas usage for each method across a full test run.

They won't really work for the reporter because the execution context is different. They also require installation on a per test basis in the before blocks. However, truffle's artifacts is exposed as a global in the reporter and there's another way to do this with abi-encoder.

• List all the .sols in contracts
• artifact.require all the contracts
• create a mapping of the method hashes using abi-encoder
• for every block, get the tx(s) & get the method hash from the code + gas spent.
• add to map, as array of expenditures.
• process the averages using Alan's technique here
• publish as a table.

Add deployment subsection to table

Algo for this:

• At mapMethodsToContracts get all the artifact instances and return them to the runner.
• At hook start get a deployments specific startBlock
• At hook end
  • for each block find contract creation transactions and get their address
  • for each artifact try to generate an instance at that address
  • for each successfully created instance produce { contract: , gasUsed: }
• At generateGasStatsReport print a separate alphabetically sorted deployments section

Neufund's % of limit

They have a nice gas logging utility in their tests that looks like this:

387% of the limit. The source data for the gasPrice and ether price is static.

`syncRequest` does not support / crashes with web3 websocket provider

This is ok for ganache since it seems to be protocol agnostic. (Geth is another story.)
We have to identify the case where web3.currentProvider.host is undefined, extract the url from

web3.currentProvider.connection.url

and replace the ws:// protocol with http://

Unit test gas costs should exclude test harness setup cost

Gas costs for each unit test should not include the gas spent on setting up the test harness in before and beforeEach hooks.

Error: Content needs to be a primitive, got: function

// ^^ a bunch of tests passing, and then...  ^^
     ✓ should be able to withdraw via core as COO (45404 gas)
Gas spent on test suite: 44334524000000879275

/Users/primary/Projects/cryptokitties-bounty/node_modules/truffle/build/cli.bundled.js:320098
        throw reason;
        ^

Error: Content needs to be a primitive, got: function
    at Cell.setOptions (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/cell.js:27:11)
    at new Cell (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/cell.js:12:8)
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/layout-manager.js:157:16
    at arrayMap (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:1406:25)
    at Function.map (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:6710:14)
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/layout-manager.js:156:16
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:2275:27
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:3049:15
    at baseMap (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:2274:7)
    at Function.map (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:6710:14)
npm ERR! Test failed.  See above for more details.```

Doesn't support the truffle --working_directory flag

Sent #90

Add gas-to-currency conversion e.g (23000 gas) (USD 0.04)

There's an api endpoint exposed here with latest market data.

Also add a config that lets you set the currency.

This is probably most useful for the table proposed in #6.

To get the Dispaly of gas Used for each transaction

I Installed eth-gas-reporter and npm install -g eth-gas-reporter
How can I get the gas report from the terminal

Optional output to file

May we allow user to save table output to file? That may be useful for automation of this reports.

Derive block gas limit from testrpc

Blockcypher `safe low` gas price has no relationship to reality.

Stuck at 5 gwei while eth gas station reports 36 gwei due to cryptokitty insanity.

Fix by using: https://ethgasstation.info/json/ethgasAPI.json

{
 "speed": 0.9603672373790222, 
 "safeLow": 360.0, 
 "safeLowWait": 9.2, 
 "block_time": 13.806122448979592, 
 "average": 400.0, 
 "avgWait": 1.7, 
 "fastestWait": 0.5, 
 "blockNum": 4704355, 
 "fast": 510.0, 
 "fastest": 550.0, 
 "fastWait": 0.6
}

Alphabetize contracts, then methods by contract.

Colors should be optional

There is no way now to switch off colors.