decred/dcrticketbuyer

An automated smart ticket purchaser.

dcrticketbuyer - OBSOLETE

This package has been superseded by the built-in ticketbuyer in dcrwallet

dcrticketbuyer's Issues

dcrticketbuyer has been replaced by the builtin ticketbuyer in dcrwallet. repo should say so

use average window ticket price instead of average pool ticket price

When volatility is high, tickets are bought at the cheap price, and then 3 windows pass with very few tickets being bought. This causes a majority of the tickets in the pool to be priced low, so the average ticket price is a much lower figure then the price that would be the stable ticket price. A calculation is needed to determine the stable price of tickets. This should be derivable from looking at the price used for the previous ticket windows instead of looking at the bought tickets themselves.

Update glide.lock for 0.5.0 release

This will just be for dcr* suite deps:

automatically adjust the value of maxperblock

If there are 100 blocks remaining in the window, and the buyer is planning to purchase 10 tickets within the current window (or for minpricescale, to the proportional amount of the ideal ticket price), then dcrticketbuyer would buy approximately every 10 blocks. It would recalculate and update the equation after every ticket purchasing so that it adjust as it goes along in order to take account for delays in purchases being mined as well as any change in available spending balance.

Benefits:
-The user wont have to frequently adjust maxperblock by hand to make sure the right number of tickets are purchased. Less manual configuration required.
-Keeps ticket fees more stable since all tickets are not trying to get into the earlier blocks.
-Would help keep the expected estimatestakediff accurate during the window.
-Useful when minpricescale activates. It is better to let others purchase tickets who want to first, rather then raising the price higher then it needs to go if others buy after you. It helps reduce the dilemma of people trying to hold out to see if someone else will raise the price first. A better functioning minpricescale encourges users to keep it enabled. Then the default could also potentialy be set higher to 0.95 instead of 0.7, helping to stabilize the price.

wrong final spendable balance

The 'Final spendable balance' that dcrticketbuyer shows is not the same as what 'getbalance' shows from dcrctl.

This is a recurrence of #21

Misses first buying opportunity

Does not start buying until after a block is received.

dcrd cert file error on launch

[z@mac decred-v0.1.5]$ ./dcrticketbuyer
Failed to read dcrd cert file at /home/z/.dcrd: read /home/z/.dcrd: is a directory

exceeding max allowed in mempool

Tickets are always bought in increments of maxperblock, and buys will exceed maxinmempool. Example: maxperblock = 10, maxinmempool = 12, result, in mempool: 20.

do not buy if exceeds your max fee

Currently dcrticketbuyer will (try to) buy ticket even when there is no chance of the tickets getting mined. This happens when the ticket price is low with lots of buying competition which has driven the fees up. If your maxfee is set low, dcrticketbuyer will waste money creating transactions/purchase attempts that will never go through. The correct behavior would be to not create transactions/purchases if the blocks are full (20 purchases already being mined) and the lowest mined purchase fee is higher then your maxfee.

pricetarget redundant with minpricescale

It seems like pricetarget may not be needed. The desired effect can be achieved by setting setting minpricescale to a value greater then 1. It may be possible to make an argument that they behave in different ways, but I see no benefit to using pricetarget. minpricescale seems preferable.

Consider available funds to tune buying aggressiveness

This is an enhancement. Currently dcrticketbuyer applies a penalty to the number of tickets to purchase based on how high the current stake price is above the average price. This penalty could also consider the amount of funds available for purchasing tickets. A ratio of the users funds which are available but not locked can be found from 'getbalance' / 'getbalance * 0 all'. For example, I would reduce the penalty by half for every tenth of the users total funds that are available for ticket purchasing. (a tenth is 2.8 days worth based on a 28 day maturity time).

avgpricemode dual is far off

These are the numbers being reported for the target price. Dual is far off from what is expected.

avgpricemode=vwap 24.97240151
avgpricemode=pool 24.97060493
avgpricemode=dual 11.31349543

smart input selection

Instead of creating a new transaction output for each ticket purchase, first check if we already have outputs that are the correct size. This is common because of ticket purchases that expired before they were mined.

More advanced. Check for outputs that are larger then needed. Sort smallest usable first. Calculate if it is efficient to use by looking at maxperblock and expirydelta. If maxperblock is 3 and expiredelta is 16, then 16 * 3 * ticketprice is the amount of reserve funds needed. If wallets spendable balance is greater then the reserve funds needed minus the large output, then the output can be used directly. Also, if the output is only roughly 10% larger then the ticket price it should just be used since that amount is likely not enough to get a ticket with anyways.

Question: Does getting change increase the size of a transaction? From the looks of the block explorer, all tickets include a sstxchange regardless of if it is used.

Remove -v to unclutter travis logs

graphs don't display due to local file access via javascript

Reported here:

https://forum.decred.org/threads/dd-15-v0-3-0-08-15-16.3960/page-2#post-21758

I can reproduce locally.

It ends up generating javascript calls with local paths like:

d3.csv("/home/myuser/decred/webui/prices.csv")

These fail due to the browser's javascript security model (though kind of odd that it doesn't trigger a visible warning).

It should just be:

d3.csv("webui/prices.csv")

I believe this worked originally due to the default config but a generated config from dcrinstall broke it since it expects a relative path and not absolute one.

Tickets not being bought

I set highpricepenalty=1.0 (default is 1.3) so that there would be no penalty and tickets would continue being purchased to use up all my funds. balancetomaintain is set to 0.0 and maxpricescale is set to 10.0 so that again, nothing would stand in the way of buying tickets and using all my available funds. maxpriceabsolute was set well above the current price. I have enough funds for many buys remaining. debuglevel is set to trace and I see "[TRC] TKBY: All tickets have been purchased, aborting further ticket purchases". The 12 hour window passes and my funds are not spent.

overbuying the target price

When buying is activated by minpricescale to maintain the ideal ticket price, the number of tickets bought it always equal to maxperblock which sometimes means raising the price higher then minpricescale specifies. The expected behavior is for the number of tickets bought to be only the amount needed to reach the (proportional) ideal ticket price.

enhance information output

Even with debuglevel=debug, it doesnt say a whole lot about what is going on. This issue is to track improving the normal on screen information that is output, as well as adding to the debug output.

Update glide.lock for 0.6.0 release

This should include all deps outside of the dcr* suite

maxinmempool ambiguous

The description of maxinmempool in --help and in the example.conf is ambiguous because it is not clear if it is tracking just your tickets in the mempool or anyones tickets in the mempool. I actually thought it meant all tickets, but after reviewing the debug logs, it appears its only your own tickets.

Related to this is the dontwaitfortickets option which says "Don't wait until your last round of tickets have entered the blockchain to attempt to purchase more" but its not clear what a round is. What if only half of your tickets get mined? Will it send more tickets or wait till all of your tickets get mined?

invalid expirydelta should fail

[ERR] TKBY: Failed to purchase tickets this round: -4: need expiry that is beyond next height

Looks like expirydelta=1 is not valid. Should probably error and stop at startup since its not a valid config setting, instead of running and displaying a not immediately obvious error when it tries to buy.

maxpricescale buys too early

The purpose of minpricescale is to raise the expected ticket price for the next window if it looks like it will fall too low. The default is 0.7. Raising ones default to 0.8 means that you will likely be the only one buying tickets, because the majority will still be using the default value of 0.7 and will not be buying tickets at all because you are the only one holding the price up. This is a disincentive for deviating from the default and from helping to stabilize the price.

A way to fix this would be to make mixpricescale wait till the end of the ticket purchasing window to buy. By calculating the number of tickets purchase slots remaining in a window, the amount of tickets you have funds to purchase, the number of tickets purchases needed to raise the estimate price to the target amount, and the maxperblock setting, it would be possible to wait towards the end of the ticket window to determine that it is time to buy for minpricescale. This would create a group effect where more participants would be helping to raise the ticket price, as well as not buying too early (before other parties make purchases) and causing the ticket price to go too high.

No tickets bought with maxperblock=1

When maxperblock is set to 1, it will not buy any tickets. The expected behavior is to buy one ticket per block.

allow maxpricerelative

Right now maxpriceabsolute get configured to a set price. What is needed is a maxpricerelative which is relative to the ideal price.

This is primarily because of minpricescale which "Forces the wallet to purchase tickets if the price is estimated to fall below this proportional amount of the ideal ticket price." In times of high volatility minpricescale can cause the inadvertent purchase of very expensive tickets. maxpriceabsolute protects from this, but requires adjustment by the user.

'# Never spend more than 30% above the ideal ticket price.
maxpricerelative = 1.3

csv file location

csv files being created (fees.csv, mempool.csv, prices.csv, purchased.csv) are not static, so it seems like they belong in the ~/.dcrticketbuyer/ folder instead of the /webui/ folder.

highpricepenalty is not useful

I propose that the highpricepenalty feature of dcrticketbuyer be removed because it serves no useful purpose. It spends money on expensive tickets haphazardly. Not buying overpriced tickets is effectively managed by the use of maxpriceabsolute.

dynamic max fee

Set the fee dynamically. The dynamic fee is based on profitability and is calculated as follows.

20 (cur price) / 28 (avg price) = 0.714 (percent reduced price)

1.6944 (reward) * 0.714 = 1.2098 DCR (equivalent reward per cost)

1.6944 - 1.2098 = 0.4846 DCR (extra profit that can be used for the fee)

0.4846 DCR is the max fee spend for profitability that is equivalent to buying at the avg price.

When there is a rush on cheap tickets, the fee would get bid up until it becomes the equivalent profit as just buying tickets at the average price.

When the current price is equal to or greater then the average price, the fee would stay set to the minimum required.

Check if wallet is locked on startup

A minor UX suggestion: we can use the walletislocked rpc command check if wallet is locked to remind the user that the wallet must be unlocked to purchase tickets.

Problem buying tickets

My testnet dcrticketbuyer bought maybe 2 dozen tickets just fine and then stopped with errors about expiry showing up on each block:

11:18:41 2016-08-19 [INF] TKBY: Purchased ticket 84af33691ed150074759a9c6c42a698c6ade1ac2cd1fe0656b56541916fe1272 at stake difficulty 17.03362513 (0.01 fees per KB used)
11:19:13 2016-08-19 [INF] DCRD: Block height 144047 connected
11:26:59 2016-08-19 [INF] DCRD: Block height 144048 connected
11:33:38 2016-08-19 [ERR] TKBY: Failed to purchase tickets this round: -4: need expiry that is beyond next height (given: 144064, next height 144064)

I've restarted dcrticketbuyer and will see if it starts buying again.

Reorgs aren't handled at all

The OnBlockConnected notification should on trigger once per height. If there are reorganizations, it may trigger more than once. A map should be added and checked so that the software only triggers once per height.

Tickets to buy doesnt consider updated balance

Tickets are only queued for purchase once, which seems to be at the beginning of the stake window or when when when the ticket buyer is launched. When the wallet balance changes during the window, the number of tickets to buy is not updated.

Update to use latest dcrwallet

pkg ticketbuyer is now part of dcrwallet which uses it to make automatic ticket purchases.

dcrticketbuyer should be updated to make use of the pkg via RPCs to make configuration changes and toggle purchasing.

Consolidate webui folder

As @jolan pointed out in #1 there are some ways to include the contents of the webui folder in the binary (and then it could write the files to ~/.dcrticketbuyer/webui/).

Including assets in source code is yucky but so is depending on a folder in a known place.

So we should at least consider this (or if there are other options think about them).

better price targeting defaults

Current defaults are
maxpricescale=2.0
minpricescale=0.7
This is a 130% default price fluctuation.
This does not help stabilize the ticket price. minpricescale holds the price up at 70% of the average price. That is a price that has fallen 70% below the average ticket price, which starts a massive ticket buying flood for cheap tickets and then causes the price to go up really high. Once the price is really high, then a few windows have to pass for the price to fall, and it falls really far often overshooting the avg price, and the cycle repeats never to stabilize. maxpricescale 200% does the same thing. It allows the ticket price to go to double the avg price, and keeps the huge volatility.

The question is how much do we want to default targeting to fluctuate by?
5% ? 10% ? Okay, then

10% default fluctuation.
maxpricescale = 1.05
minpricescale = 0.95

5% default fluctuation.
maxpricescale = 1.025
minpricescale = 0.975

max min target refinement

Currently the max and min target options work based on estimate price, even though estimatestakediff shows the min and max ticket price for the next window. Min target should be watching the min price for the next window and buy based on that instead of the estimate price. That way the users ticket buyer will not contribute to raising the price beyond the min target that was specified, and same for max. For this to all go down well, it will also need to use a dynamic # of tickets to buy per window, so that if the number of blocks remaining in the window is small, then the buyer will be able to buy as many tickets as needed to get the the specified target price. Also of note here, is that target price should take a static price setting in addition how it is currently done using a percentage.

fee per KB calculation wrong

The fee per KB amount is showing different amounts when comparing the output from dcrticketbuyer to what it says when you pull up the transaction on the https://mainnet.decred.org/ . As far as if it is dcrticketbuyer or the block explorer that should be fixed, from my calculations it looks like they are both off.

https://mainnet.decred.org/tx/5161b980936ee934f0fc0025395bb91fdcd9b5fd89b8a77cc375c51159e7a46c

FEE: 0.07500000 DCR , SIZE: 297 BYTES , FEE RATE:0.25252525 DCR PER KB

0.25252525 * (297/1024) = 0.07324218

or are we suppose to divide by 1000?

0.25252525 * (297/1000) = 0.07499999925

but that is just for insight. dcrticketbuyer is still over 1% off from that.

paying higher fees even when blocks are not full

Dcrticketbuyer uses the mean/median ticket fee for the last x blocks to set the ticket fee value you will be paying. This ends up in a situation where everyone continues paying a progressively higher ticket fee during a window, even when less then 20 tickets are being purchased per block. The desired behavior is to calculate what fee needs to be paid to get into a block and minimize costs.

buys happen every time dcrticketbuyer is restarted

Instead of buying once in the first round, dcrticketbuyer will buy every time it it restarted, even several times in the same window.

This is the inversion of #20

"Tickets purchased in this window" is not persistent

Quit dcrtickerbuyer. Restart. This counter starts over at zero. Restarting it will cause it to buy more tickets after it has already maxed out.

buy tickets in the first possible block

Referencing this comment that suggested it,
decred/dcrd#394 (comment)

"The way to fix this would be to make a new ticket buying mode in wallet that slightly overpays and creates a number of outputs based on the max estimated ticket difficulty before the new difficulty block is mined, and then spends those outputs immediate and gives change back to the wallet."

Tickets remaining to be purchased in window should reflect possible tickets left in the window.

16:40:14 2016-06-03 [DBG] TKBY: Tickets remaining to be purchased in this window: 96622

This is far above any possible amount of tickets that a window could include.

So at the most it should be 2880 (144 * 20) then go down from there

maxinmempool defaults to 0 instead of 40

https://github.com/decred/dcrticketbuyer/blob/master/config.go#L68

https://github.com/decred/dcrticketbuyer/blob/master/ticketbuyer-example.conf#L55

fee price bids up against oneself

feetargetscaling bids above the mean price, even against your own tickets, causing a perpetually rising fee even when you are the only one buying tickets.

Fails to buy, conflicting log msgs

It says it has tickets queued, and then aborts saying all tickets have been purchased. This happens with every new block.

09:40:53 2016-11-10 [DBG] TKBY: The stake difficulty 22.23241839 Coin was below the target penalty cutoff 27.03236773; 60 many tickets have been queued for purchase

09:40:56 2016-11-10 [TRC] TKBY: All tickets have been purchased, aborting further ticket purchases

Also, the word 'many' is redundant grammar.