univocity/univocity-trader

open-source trading framework for java, supports backtesting and live trading with exchanges

univocity-trader's Issues

Add integration with Coinbase

Create another sub-project like univocity-trader-binance but for Coinbase.

Notice their fee schedule is a bit more intricate than what the SimpleTradingFees does so a custom implementation of TradingFees will have to be made.

Any help in making this work is appreciated.

Unresolved dependencies when build with maven after cloning project

I get the following maven message:

[ERROR] Failed to execute goal on project univocity-trader-examples: Could not resolve dependencies for project com.univocity:univocity-trader-examples:jar:1.0.0-SNAPSHOT: Failure to find com.univocity:univocity-trader-interactivebrokers:jar:1.0.0-SNAPSHOT in https://dl.bintray.com/testng-team/testng/ was cached in the local repository, resolution will not be reattempted until the update interval of testng has elapsed or updates are forced -> [Help 1]

Support bracket orders

It's basically submitting an order at a price with two defined exits (sell if price hits a profit target, or sell at a maximum loss). The opposite when shorting.

As per interactive brokers docs:

https://interactivebrokers.github.io/tws-api/bracket_order.html

Support margin accounts

Margin works like this:

1. account balance is X

2. deposit some or all X into a margin account

3. optionally borrow extra amounts via margin account (2x, 5x, 100x, whatever) - with interest

4. buy/sell whatever you have: margin amount + borrowed amount

5. close the trade by selling/buying back.

6. close the loan and pay whatever interest was charged.

Interest might be charged per hour/day, etc. Binance charges anything from 0.01% per day to 0.3% depending on the asset. Interactive Brokers charges roughly 4% to 2%/year depending on amount borrowed, also charges interest daily.

Need to first incorporate interest calculation to simulations, then move on to live trading - consider that the exchange may not provide an API to open and close margin accounts (binance does).

Deprecations of Binance APIs

As per : https://binance-docs.github.io/apidocs/spot/en/#change-log

By end of Q1 2020, the following endpoints will be removed from the API. The documentation has been updated to use the v3 versions of these endpoints.

GET api/v1/depth
GET api/v1/historicalTrades
GET api/v1/aggTrades
GET api/v1/klines
GET api/v1/ticker/24hr
GET api/v1/ticker/price
GET api/v1/exchangeInfo
POST api/v1/userDataStream
PUT api/v1/userDataStream
GET api/v1/ping
GET api/v1/time
GET api/v1/ticker/bookTicker

These are also changing:

GET api/v1/ticker/allPrices	GET api/v3/ticker/price

GET api/v1/ticker/allBookTickers	GET api/v3/ticker/bookTicker

TD Ameritrade interface

Hi Jeronimo - I'm interested in adding a TD Ameritrade interface to the project for personal use. The TDA API uses JSON schemas to define requests for orders, price history, etc. so for implementation I was thinking of storing the schemas in the project, and then using a serialization library to generate classes as needed.

Any thoughts on this approach? And do you have a preferred JSON serialization library?

Broken build

PR #116 introduces half-made changes including breaking changes in interfaces, commented code including constructors and methods that are called by other classes.

Ideally the user who submitted the PR should complete the changes to fix the compilation issues. For now the only workaround is to exclude module univocity-trader-binance-future from maven build, so maybe that PR should be reverted until a new PR with the completed code is re-submitted.

In addition to that, test MarketSimulation.testMarketOrders is failing

Results :

Failed tests:   testMarketOrders(com.univocity.trader.scenarios.MarketSimulation): expected:<1225.636493> but was:<1000.0>

Save trades to google sheet

Based on OrderExecutionToLog and OrderExecutionToEmail implement an OrderExecutionToGoogleSheet that will be updated every time a buy/sell is made.

Support short selling

For live trading with stocks this is one of the common processes:

1. need to have X amount in account.

2. short sell up to the equivalent X amount. Additional fees may apply, also might need to close trade before end of trading session to avoid paying interest.

For Binance (and possibly others), you can only do that with a margin account. I'll open a separate issue for this.

Emulate order fills using forward history

Right now any buy/sell on a simulation simply considers the closing price of the candle. It's not realistic enough and each order should be "filled" based on the candlesticks that come up AFTER the order is placed. That should use volume, high/low and close prices of every tick until the order fills or expires).

Support for embedded database

I was considering to brute-force parameters with backtesting data on a AWS EC2 instance, one of the instances with 96 cores, or maybe one of the instances with 128 cores (although the latter has almost 2TB of RAM so they're like 3x more expensive than the ones with 96 cores).

Roughly the ones with 96 cores cost like $4-$5 an hour, and the ones with 128 cores like $12/$13 an hour.
This price is very expensive if it's going to run 24/7 for a month, but it's a perfectly acceptable price if you're going to run your simulations for a limited time, @jbax mentioned that "I can run the process described above in 20 minutes on a threadripper 3990x (we're talking about billions of candles processes over and over) ." And TR 3990x has 64 cores, so with 96 or 128 cores would be even better.

I was considering how could I maximize performance and minimize cost. I thought about using docker for a database and bring it up but it would take too much time to import a dump and then get the new data etc. Then I thought about using Amazon RDS but if I only need to run the process for a short time then I'd have to pay for the database to hold my data without using it.

I considered implementing support for reading binary data (as it would be faster to read than text files), and I saw ActiveJ Serializer as a very fast option, but it required adding annotations to the model and it was overhead.

Then I thought that maybe an embedded database could be perfect. I considered Nitrite, a NoSql embedded database based on H2, but probably SQL based was less work since most of the database code would already be the same. Nitrite also requires some annotations on the models.

The best fit is H2, is written in Java and is very fast. It would be great for unit tests to use in-memory, and for heavy simulations on AWS would be easy to just copy the H2 database file with all the backtesting data and the jar of the application. It would be great to run it in-memory in one of the AWS instances with lots of RAM, but then we would have to get all the backtesting data. It's more convenient just uploading a database file.

I tried in a branch and is very easy to setup, I'll create a PR soon for that.

Setup a discord channel for developers

Please setup a discord channel where developers can interact and discuss more needed features. I can help managing the discord channel.

Add support for BInance Spot TestNet

On the Binance Public API you can see a mention to the SPOT Test Network (https://binance-docs.github.io/apidocs/spot/en/#spot-testnet)

On the Test net, you use realtime data but you don't use real money.

We need a way to use the Test Net

Attachment orders missing in LiveTrading?

Not sure if I'm missing something but I can't seem to find the attachment equivalent code in the Live trader mode:

univocity-trader/univocity-trader-core/src/main/java/com/univocity/trader/simulation/SimulatedClientAccount.java

Line 112 in a81fe3f

if (order != null) {

Is this an omission or is it handled elsewhere?

Add integration with OANDA

It looks like it should be simple since they provide a java library: https://github.com/oanda/v20-java

Examples:
https://github.com/oanda/v20-java-samples/tree/master/src/main/java/com/oanda/v20/v20sample

Setter for simulation backfill fails to set value

A minor PR for a small bug:

#52

waitForFill looses track of modified order

Hi,

Thanks for the great project :-)

In waitForFill method, there's the following:

Order updated = updateOrder(order, null);
if (updated.isFinalized()) {
             return;
}

We are in a thread there so the modified updated value (the one with a new order ID when an order gets filled) never gets sent back to the caller. Because of this, the executeOrder in AccountManager never gets the updated order so when it gets the FILLED notification, it fails to remove the updated order from the pending queue.

I'm not quite sure how best to fix this, it's easy to just say if (updated.isFinalized()) {... remove from queue) in the snippet above but the actual object itself is not being updated elsewhere so for example if it needs to get logged it won't have the "latest and greatest" values. An alternate way would be to set the order handle properties to the new one but that means we've lost immutability.

Buy/Sell signal with amount?

First of all i love this project. Great work! (and sorry for my English! )

I need to buy/sell signal with amount. For example,

My Strategy :
Price 1000$ -> Buy 100 (amount)
if Price down 900$ -> Buy 200 (amount)

My target is reduce my avarage cost.

Is there any way achieve this strategy?

Build fails if you don't have Interactive Brokers jar

When trying to run the build without having the Interactive Brokers jar on the classpath you get this error message

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for univocity-trader 1.0.0-SNAPSHOT:
[INFO]
[INFO] univocity-trader ................................... SUCCESS [ 1.213 s]
[INFO] univocity-trader-core .............................. SUCCESS [ 11.431 s]
[INFO] univocity-trader-binance ........................... SUCCESS [ 1.321 s]
[INFO] univocity-trader-binance-futures ................... SUCCESS [ 1.690 s]
[INFO] univocity-trader-iqfeed ............................ SUCCESS [ 0.890 s]
[INFO] univocity-trader-examples .......................... FAILURE [ 0.046 s]
[INFO] univocity-trader-chart ............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.760 s
[INFO] Finished at: 2021-03-01T23:55:41Z
[INFO] ------------------------------------------------------------------------

Given that jar can't be bundled since you have to download and accept terms and conditions, and that some users may not be interested in using IB, you have to comment pom.xml in examples module (reference to IB dependency) and imports in a couple of classes.

That means that by defauilt, after project checkout, doing a mvn clean install or mvn clean package will fail, and you'll have to figure out why. You have to comment things related to IB in 3 files, so it's easy to fix, but it's not obvious and it's not documented on README.md

I wonder if we could conditionally exclude that from compilation, for example by using profiles or the maven-build-helper.

CrossOver function

Hi,

I'm very interest by what you made, and i'm looking for the way to check when a indicator crossover up or down an other indicator or a value.

Thanks

Implement weigthed linear regression

Create another LinearRegression implementation that adds more weight to the most recent values added to it.

I suck at math at this level so if anyone could lend a hand here that would be great.

Adding interface for Interactive Brokers

Any plans for this yet?

Add a new indicator

Hello, the author. Your framework is great. Whether a risk indicator can be transplanted. I'm not very good at it. The following documents can be provided for your reference https://adrian.ng/java/var/

Add support for IEX cloud

These guys provide a Java lib that we can import and use:

https://github.com/WojciechZankowski/iextrading4j

That makes life easier.

Adding integration for Apha Vantage

They have a pretty decently documented API: https://www.alphavantage.co/documentation/
Not sure if there are any java libs out there that implement most of the support for that, if there is then it should not be too hard to create the Exchange and ClientAccount implementation for them.

Hope to support cross market Pairs trading

Hello, I've been following your trading framework for a long time

I find that the framework is very advantageous for trend trading, but it is not easy to handle the cross market hedging of a single strategy

e.g

Cross market hedging. Trading Pairs Binance btc_usdt,huobi btc_usdt

Cross market triangular hedging. Trading Pairs Binance btc_usdt,eth_usdt; huobi eth_btc

com.univocity.trader.strategy.Strategy#getSignal We need a pairs. We need a market order In this way, we can judge whether the three comparisons are profitable or not, and it is not necessary to use indicators comparison

Thanks!

Add javadocs to everything

No public method/class in this project should be left without proper javadocs that actually explain how things work.

This is going to take a while, but I'll get there.

Build fails because of maven-bundle-plugin

Just after cloning the project and trying to build it, you'll get this error

[INFO] 
[INFO] --- maven-bundle-plugin:4.2.1:manifest (bundle-manifest) @ univocity-trader-core ---
[ERROR] An internal error occurred
java.util.ConcurrentModificationException
    at java.util.TreeMap.callMappingFunctionWithCheck (TreeMap.java:742)
    at java.util.TreeMap.computeIfAbsent (TreeMap.java:558)
    at aQute.bnd.osgi.Jar.putResource (Jar.java:288)
    at aQute.bnd.osgi.Jar$1.visitFile (Jar.java:202)
    at aQute.bnd.osgi.Jar$1.visitFile (Jar.java:177)
    at java.nio.file.Files.walkFileTree (Files.java:2804)
    at aQute.bnd.osgi.Jar.buildFromDirectory (Jar.java:176)
    at aQute.bnd.osgi.Jar.<init> (Jar.java:119)
    at aQute.bnd.osgi.Jar.<init> (Jar.java:172)
    at org.apache.felix.bundleplugin.BundlePlugin.getOSGiBuilder (BundlePlugin.java:604)
    at org.apache.felix.bundleplugin.ManifestPlugin.getAnalyzer (ManifestPlugin.java:285)
    at org.apache.felix.bundleplugin.ManifestPlugin.execute (ManifestPlugin.java:111)
    at org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java:364)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:64)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:564)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for univocity-trader 1.0.0-SNAPSHOT:
[INFO] 
[INFO] univocity-trader ................................... SUCCESS [  1.355 s]
[INFO] univocity-trader-core .............................. FAILURE [  4.879 s]
[INFO] univocity-trader-binance ........................... SKIPPED
[INFO] univocity-trader-binance-futures ................... SKIPPED
[INFO] univocity-trader-iqfeed ............................ SKIPPED
[INFO] univocity-trader-examples .......................... SKIPPED
[INFO] univocity-trader-chart ............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.404 s
[INFO] Finished at: 2021-03-02T00:06:13Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.felix:maven-bundle-plugin:4.2.1:manifest (bundle-manifest) on project univocity-trader-core: Internal error in maven-bundle-plugin: ConcurrentModificationException -> [Help 1]
[ERROR] 

After some investigation (see FasterXML/oss-parent#27) it looks like the maven-bundle-plugin is culprit, and that upgrading that to 5.1.1 will fix it.

Try to migrate broker integrations from https://github.com/cryo8822/trader

As described in the link: https://github.com/cryo8822/trader

Can trade through InteractiveBrokers and MB Trading. Can receive feeds from Yahoo and ActiveTick.

The caveat here is that the Groovy code must be converted to pure Java.

Running Chart window

Please provide univocity-trader.properties file to run sample chart window.

Connection times out when unused for a while

When a signal is triggered, I often get the following exception. I put in an emergency exit (System.exit) upon repeated failures and restart the app at which point the connection works straight away. I haven't yet been able to track this one down yes -- I've already added a keep alive timer that also pings the connection. Leaving this bug report here until I can nail this one down (or perhaps open a new restClient() connection each time?)

   java.io.IOException: java.util.concurrent.TimeoutException
	at org.asynchttpclient.extras.retrofit.AsyncHttpClientCall.toIOException(AsyncHttpClientCall.java:295)
	at org.asynchttpclient.extras.retrofit.AsyncHttpClientCall.execute(AsyncHttpClientCall.java:133)
	at retrofit2.OkHttpCall.execute(OkHttpCall.java:190)
	at com.univocity.trader.exchange.binance.api.client.impl.BinanceApiServiceGenerator.executeSync(BinanceApiServiceGenerator.java:79)
	at com.univocity.trader.exchange.binance.api.client.impl.BinanceApiRestClientImpl.getAllPrices(BinanceApiRestClientImpl.java:102)
	at com.univocity.trader.exchange.binance.BinanceExchange.getLatestPrices(BinanceExchange.java:153)
	at com.univocity.trader.account.TradingManager.getAllPrices(TradingManager.java:107)
	at com.univocity.trader.account.AccountManager.getTotalFundsIn(AccountManager.java:365)
	at com.univocity.trader.account.AccountManager.executeAllocateFunds(AccountManager.java:277)
	at com.univocity.trader.account.AccountManager.allocateFunds(AccountManager.java:325)
	at com.univocity.trader.account.AccountManager.allocateFunds(AccountManager.java:347)
	at com.univocity.trader.account.TradingManager.allocateFunds(TradingManager.java:192)
	at com.univocity.trader.account.Trader.prepareTrade(Trader.java:370)
	at com.univocity.trader.account.Trader.buy(Trader.java:418)
	at com.univocity.trader.account.Trader.processBuy(Trader.java:201)
	at com.univocity.trader.account.Trader.trade(Trader.java:170)
	at com.univocity.trader.strategy.TradingEngine.process(TradingEngine.java:93)
	at com.univocity.trader.candles.CandleProcessor.processCandle(CandleProcessor.```



 

Implement Zig Zag indicator

About: https://www.investopedia.com/terms/z/zig_zag_indicator.asp

Take profit

Read the docs but couldn't find a clear way to set take profit or stop loss levels. Is that done through the Monitor? How can I set say 10% TP with 5% SL?

Implement Waddah attar explosion

title

Migrate every single indicator in ta4J lib to this project

There are plenty of indicators in ta4j library, with unit tests and everything, but it's not a library that can be used with this framework. The code in there needs to be migrated so it can work here.

Note that every effort is made here to prevent maintaining a list of candles in memory. New indicators must extend from one of the abstract classes in package com.univocity.trader.indicators.base.

Nothing prevents one to create their own Indicator implementation, but in such case care must be taken to properly initialize the Aggregator that is responsible for aggregating candles (for example, converting five 1 minute candles into a single 5 minute candle)

Bracket orders for Binance

#56

Refer to : #50

I'm uncertain how best to pass in TradingFees since there doesn't seem to be an easy way to obtain this in the LiveTrader -- for now this is hard-coded to 0.1%

Last candles, price signal

Hello,

The signal is sent every candle, but is there a signal sent every price change ?

Also, I need the 500 last candles everytime, do I need to wait 500 candles and save them one by one to use my indicator ?

Cache indicator values

As suggested in issue #1:

One thing you may want to add in the future for better performance is a caching mechanism for your indicators to reduce repeated computation time. Here's a Java TA library which does this as an example:

https://github.com/ta4j/ta4j

Previous candles

Hi, within a strategy's getSignal is it possible to get previous candles?

VWMA

@jbax I'm trying to add the VWMA indicator but I'm not able to get it to work just yet. My code is below:

public class VolumeWeightedMovingAverage extends SingleValueIndicator {

  private double value = 0;
  private int length = 0;
  private final CircularList price;
  private final CircularList volume;

  public VolumeWeightedMovingAverage(int length, TimeInterval interval) {
    this(length, interval, c -> c.close);
  }

  public VolumeWeightedMovingAverage(int length, TimeInterval timeInterval, ToDoubleFunction<Candle> valueGetter) {
    super(timeInterval, valueGetter);
    this.length = length;
    price = new CircularList(length);
    volume = new CircularList(length);
  }

  @Override
  protected Indicator[] children() {
    return new Indicator[0];
  }

  @Override
  protected boolean process(Candle candle, double value, boolean updating) {
    if (updating) {
      price.update(candle.close);
      volume.update(candle.volume);
    } else {
      price.add(candle.close);
      volume.add(candle.volume);
    }
    this.value = this.calculate(updating);
    return true;
  }

  private double calculate(boolean updating) {
    if (getAccumulationCount() <= this.length) {
      return 0;
    }

    double top = 0;
    double bottom = 0;
    int index = 0;
    for (double px : price.values) {
      top += price.get(index) * volume.get(index);
      bottom += volume.get(index);
      index++;
    }
    return top/bottom;
  }

  @Override
  public double getValue() {
    return this.value;
  }

  @Override
  public String signalDescription() {
    return super.signalDescription();
  }
}

I see there are various classes to extend for implementation but not too sure what to use for this indicator.

Loading currently open positions and overriding method

Is there a way to load currently open positions preferabbly as a separate object?

The method which loads the positions can be made overridable so if we have multiple strategies running on one account, and say 2 strategies have positions on asset X with quantity 10 and 20 each then we can determine what is the position for the current strategy in asset X by subtracting the total open position on the asset X in our account by a value we stored locally.

The value could have been saved in the last run of our current strategy when it received fill status.

So this way we know that even though the account has 30 shares of X, we have to only see it as 10(30 - 20) if we are inside the second strategy.

Interactive Brokers resolution error in maven

I have cloned this repo and trying to get it working but I keep getting this error in my IDE:

Cannot resolve com.interactivebrokers:twsapi:9.76

It's related to how IB api is imported in maven in the ib module:

                <dependency>
			<groupId>com.interactivebrokers</groupId>
			<artifactId>twsapi</artifactId>
			<version>9.76</version>
			<scope>system</scope>
			<systemPath>${user.home}/dev/IBJts/source/JavaClient/TwsApi_debug.jar</systemPath>
		</dependency>

Where can we get this jar? I am unable to find the jars for TWS 9.76 online.

OrderExecutionLine strips away original data types

OrderExecutionLine should maintain the original data types because it currently assumes a text-based output. For the output to Google Sheets, it helps to transmit the actual types so that the sheet values are formatted correctly.

I've done the required modifications( here: https://github.com/wwadge/univocity-trader-1/tree/feature/fix-google-logging-types) but this works by exposing a getFormattedXXX in to keep the existing String-based formatting.

Is there a way to to tell CSV parser to tell it that field X should call getFormattedX() instead of getting the field value?

Support themes for the user interface

The ChartWindow will fire up a test UI I'm writing in plain java swing.

All colors, fonts and other settings used to plot the charts are currently hard coded in the controller classes. Would be great to be able to save any changes made via UI into a new theme file as well.

These should, in addition to using the given defaults, load a properties or json or maybe a limited CSS file (whatever makes sense) which define a theme. Ideally the user could load the preferred theme on the fly with the program running. This is partly supported already through implementations of the UpdateProcessor interface.

I'm opening this issue if anyone wants to lend a hand here.