univocity / univocity-trader Goto Github PK
View Code? Open in Web Editor NEWopen-source trading framework for java, supports backtesting and live trading with exchanges
open-source trading framework for java, supports backtesting and live trading with exchanges
They have a pretty decently documented API: https://www.alphavantage.co/documentation/
Not sure if there are any java libs out there that implement most of the support for that, if there is then it should not be too hard to create the Exchange
and ClientAccount
implementation for them.
Hi,
Thanks for the great project :-)
In waitForFill method, there's the following:
Order updated = updateOrder(order, null);
if (updated.isFinalized()) {
return;
}
We are in a thread there so the modified updated value (the one with a new order ID when an order gets filled) never gets sent back to the caller. Because of this, the executeOrder
in AccountManager never gets the updated order so when it gets the FILLED notification, it fails to remove the updated order from the pending queue.
I'm not quite sure how best to fix this, it's easy to just say if (updated.isFinalized()) {... remove from queue) in the snippet above but the actual object itself is not being updated elsewhere so for example if it needs to get logged it won't have the "latest and greatest" values. An alternate way would be to set the order handle properties to the new one but that means we've lost immutability.
It looks like it should be simple since they provide a java library: https://github.com/oanda/v20-java
Examples:
https://github.com/oanda/v20-java-samples/tree/master/src/main/java/com/oanda/v20/v20sample
Right now any buy/sell on a simulation simply considers the closing price of the candle. It's not realistic enough and each order should be "filled" based on the candlesticks that come up AFTER the order is placed. That should use volume, high/low and close prices of every tick until the order fills or expires).
Please setup a discord channel where developers can interact and discuss more needed features. I can help managing the discord channel.
Hi, within a strategy's getSignal
is it possible to get previous candles?
These guys provide a Java lib that we can import and use:
https://github.com/WojciechZankowski/iextrading4j
That makes life easier.
First of all i love this project. Great work! (and sorry for my English! )
I need to buy/sell signal with amount. For example,
My Strategy :
Price 1000$ -> Buy 100 (amount)
if Price down 900$ -> Buy 200 (amount)
My target is reduce my avarage cost.
Is there any way achieve this strategy?
For live trading with stocks this is one of the common processes:
need to have X amount in account.
short sell up to the equivalent X amount. Additional fees may apply, also might need to close trade before end of trading session to avoid paying interest.
For Binance (and possibly others), you can only do that with a margin account. I'll open a separate issue for this.
PR #116 introduces half-made changes including breaking changes in interfaces, commented code including constructors and methods that are called by other classes.
Ideally the user who submitted the PR should complete the changes to fix the compilation issues. For now the only workaround is to exclude module univocity-trader-binance-future
from maven build, so maybe that PR should be reverted until a new PR with the completed code is re-submitted.
In addition to that, test MarketSimulation.testMarketOrders
is failing
Results :
Failed tests: testMarketOrders(com.univocity.trader.scenarios.MarketSimulation): expected:<1225.636493> but was:<1000.0>
I have cloned this repo and trying to get it working but I keep getting this error in my IDE:
Cannot resolve com.interactivebrokers:twsapi:9.76
It's related to how IB api is imported in maven in the ib module:
<dependency>
<groupId>com.interactivebrokers</groupId>
<artifactId>twsapi</artifactId>
<version>9.76</version>
<scope>system</scope>
<systemPath>${user.home}/dev/IBJts/source/JavaClient/TwsApi_debug.jar</systemPath>
</dependency>
Where can we get this jar? I am unable to find the jars for TWS 9.76 online.
There are plenty of indicators in ta4j library, with unit tests and everything, but it's not a library that can be used with this framework. The code in there needs to be migrated so it can work here.
Note that every effort is made here to prevent maintaining a list of candles in memory. New indicators must extend from one of the abstract classes in package com.univocity.trader.indicators.base.
Nothing prevents one to create their own Indicator
implementation, but in such case care must be taken to properly initialize the Aggregator
that is responsible for aggregating candles (for example, converting five 1 minute candles into a single 5 minute candle)
Just after cloning the project and trying to build it, you'll get this error
[INFO]
[INFO] --- maven-bundle-plugin:4.2.1:manifest (bundle-manifest) @ univocity-trader-core ---
[ERROR] An internal error occurred
java.util.ConcurrentModificationException
at java.util.TreeMap.callMappingFunctionWithCheck (TreeMap.java:742)
at java.util.TreeMap.computeIfAbsent (TreeMap.java:558)
at aQute.bnd.osgi.Jar.putResource (Jar.java:288)
at aQute.bnd.osgi.Jar$1.visitFile (Jar.java:202)
at aQute.bnd.osgi.Jar$1.visitFile (Jar.java:177)
at java.nio.file.Files.walkFileTree (Files.java:2804)
at aQute.bnd.osgi.Jar.buildFromDirectory (Jar.java:176)
at aQute.bnd.osgi.Jar.<init> (Jar.java:119)
at aQute.bnd.osgi.Jar.<init> (Jar.java:172)
at org.apache.felix.bundleplugin.BundlePlugin.getOSGiBuilder (BundlePlugin.java:604)
at org.apache.felix.bundleplugin.ManifestPlugin.getAnalyzer (ManifestPlugin.java:285)
at org.apache.felix.bundleplugin.ManifestPlugin.execute (ManifestPlugin.java:111)
at org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java:364)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:64)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:564)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for univocity-trader 1.0.0-SNAPSHOT:
[INFO]
[INFO] univocity-trader ................................... SUCCESS [ 1.355 s]
[INFO] univocity-trader-core .............................. FAILURE [ 4.879 s]
[INFO] univocity-trader-binance ........................... SKIPPED
[INFO] univocity-trader-binance-futures ................... SKIPPED
[INFO] univocity-trader-iqfeed ............................ SKIPPED
[INFO] univocity-trader-examples .......................... SKIPPED
[INFO] univocity-trader-chart ............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.404 s
[INFO] Finished at: 2021-03-02T00:06:13Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.felix:maven-bundle-plugin:4.2.1:manifest (bundle-manifest) on project univocity-trader-core: Internal error in maven-bundle-plugin: ConcurrentModificationException -> [Help 1]
[ERROR]
After some investigation (see FasterXML/oss-parent#27) it looks like the maven-bundle-plugin is culprit, and that upgrading that to 5.1.1 will fix it.
Based on OrderExecutionToLog
and OrderExecutionToEmail
implement an OrderExecutionToGoogleSheet
that will be updated every time a buy/sell is made.
It's basically submitting an order at a price with two defined exits (sell if price hits a profit target, or sell at a maximum loss). The opposite when shorting.
As per interactive brokers docs:
https://interactivebrokers.github.io/tws-api/bracket_order.html
Hi Jeronimo - I'm interested in adding a TD Ameritrade interface to the project for personal use. The TDA API uses JSON schemas to define requests for orders, price history, etc. so for implementation I was thinking of storing the schemas in the project, and then using a serialization library to generate classes as needed.
Any thoughts on this approach? And do you have a preferred JSON serialization library?
Hello,
The signal is sent every candle, but is there a signal sent every price change ?
Also, I need the 500 last candles everytime, do I need to wait 500 candles and save them one by one to use my indicator ?
Is there a way to load currently open positions preferabbly as a separate object?
The method which loads the positions can be made overridable so if we have multiple strategies running on one account, and say 2 strategies have positions on asset X with quantity 10 and 20 each then we can determine what is the position for the current strategy in asset X by subtracting the total open position on the asset X in our account by a value we stored locally.
The value could have been saved in the last run of our current strategy when it received fill status.
So this way we know that even though the account has 30 shares of X, we have to only see it as 10(30 - 20) if we are inside the second strategy.
Read the docs but couldn't find a clear way to set take profit or stop loss levels. Is that done through the Monitor? How can I set say 10% TP with 5% SL?
OrderExecutionLine should maintain the original data types because it currently assumes a text-based output. For the output to Google Sheets, it helps to transmit the actual types so that the sheet values are formatted correctly.
I've done the required modifications( here: https://github.com/wwadge/univocity-trader-1/tree/feature/fix-google-logging-types) but this works by exposing a getFormattedXXX in to keep the existing String-based formatting.
Is there a way to to tell CSV parser to tell it that field X should call getFormattedX() instead of getting the field value?
Margin works like this:
account balance is X
deposit some or all X into a margin account
optionally borrow extra amounts via margin account (2x, 5x, 100x, whatever) - with interest
buy/sell whatever you have: margin amount + borrowed amount
close the trade by selling/buying back.
close the loan and pay whatever interest was charged.
Interest might be charged per hour/day, etc. Binance charges anything from 0.01% per day to 0.3% depending on the asset. Interactive Brokers charges roughly 4% to 2%/year depending on amount borrowed, also charges interest daily.
Need to first incorporate interest calculation to simulations, then move on to live trading - consider that the exchange may not provide an API to open and close margin accounts (binance does).
@jbax I'm trying to add the VWMA indicator but I'm not able to get it to work just yet. My code is below:
public class VolumeWeightedMovingAverage extends SingleValueIndicator {
private double value = 0;
private int length = 0;
private final CircularList price;
private final CircularList volume;
public VolumeWeightedMovingAverage(int length, TimeInterval interval) {
this(length, interval, c -> c.close);
}
public VolumeWeightedMovingAverage(int length, TimeInterval timeInterval, ToDoubleFunction<Candle> valueGetter) {
super(timeInterval, valueGetter);
this.length = length;
price = new CircularList(length);
volume = new CircularList(length);
}
@Override
protected Indicator[] children() {
return new Indicator[0];
}
@Override
protected boolean process(Candle candle, double value, boolean updating) {
if (updating) {
price.update(candle.close);
volume.update(candle.volume);
} else {
price.add(candle.close);
volume.add(candle.volume);
}
this.value = this.calculate(updating);
return true;
}
private double calculate(boolean updating) {
if (getAccumulationCount() <= this.length) {
return 0;
}
double top = 0;
double bottom = 0;
int index = 0;
for (double px : price.values) {
top += price.get(index) * volume.get(index);
bottom += volume.get(index);
index++;
}
return top/bottom;
}
@Override
public double getValue() {
return this.value;
}
@Override
public String signalDescription() {
return super.signalDescription();
}
}
I see there are various classes to extend for implementation but not too sure what to use for this indicator.
Not sure if I'm missing something but I can't seem to find the attachment equivalent code in the Live trader mode:
Is this an omission or is it handled elsewhere?
No public method/class in this project should be left without proper javadocs that actually explain how things work.
This is going to take a while, but I'll get there.
Create another LinearRegression implementation that adds more weight to the most recent values added to it.
I suck at math at this level so if anyone could lend a hand here that would be great.
When trying to run the build without having the Interactive Brokers jar on the classpath you get this error message
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for univocity-trader 1.0.0-SNAPSHOT:
[INFO]
[INFO] univocity-trader ................................... SUCCESS [ 1.213 s]
[INFO] univocity-trader-core .............................. SUCCESS [ 11.431 s]
[INFO] univocity-trader-binance ........................... SUCCESS [ 1.321 s]
[INFO] univocity-trader-binance-futures ................... SUCCESS [ 1.690 s]
[INFO] univocity-trader-iqfeed ............................ SUCCESS [ 0.890 s]
[INFO] univocity-trader-examples .......................... FAILURE [ 0.046 s]
[INFO] univocity-trader-chart ............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.760 s
[INFO] Finished at: 2021-03-01T23:55:41Z
[INFO] ------------------------------------------------------------------------
Given that jar can't be bundled since you have to download and accept terms and conditions, and that some users may not be interested in using IB, you have to comment pom.xml in examples module (reference to IB dependency) and imports in a couple of classes.
That means that by defauilt, after project checkout, doing a mvn clean install
or mvn clean package
will fail, and you'll have to figure out why. You have to comment things related to IB in 3 files, so it's easy to fix, but it's not obvious and it's not documented on README.md
I wonder if we could conditionally exclude that from compilation, for example by using profiles or the maven-build-helper.
As per : https://binance-docs.github.io/apidocs/spot/en/#change-log
By end of Q1 2020, the following endpoints will be removed from the API. The documentation has been updated to use the v3 versions of these endpoints.
GET api/v1/depth
GET api/v1/historicalTrades
GET api/v1/aggTrades
GET api/v1/klines
GET api/v1/ticker/24hr
GET api/v1/ticker/price
GET api/v1/exchangeInfo
POST api/v1/userDataStream
PUT api/v1/userDataStream
GET api/v1/ping
GET api/v1/time
GET api/v1/ticker/bookTicker
These are also changing:
GET api/v1/ticker/allPrices | GET api/v3/ticker/price |
---|
GET api/v1/ticker/allBookTickers | GET api/v3/ticker/bookTicker |
---|
Hello, the author. Your framework is great. Whether a risk indicator can be transplanted. I'm not very good at it. The following documents can be provided for your reference https://adrian.ng/java/var/
Any plans for this yet?
Create another sub-project like univocity-trader-binance but for Coinbase.
Notice their fee schedule is a bit more intricate than what the SimpleTradingFees does so a custom implementation of TradingFees will have to be made.
Any help in making this work is appreciated.
When a signal is triggered, I often get the following exception. I put in an emergency exit (System.exit) upon repeated failures and restart the app at which point the connection works straight away. I haven't yet been able to track this one down yes -- I've already added a keep alive timer that also pings the connection. Leaving this bug report here until I can nail this one down (or perhaps open a new restClient() connection each time?)
java.io.IOException: java.util.concurrent.TimeoutException
at org.asynchttpclient.extras.retrofit.AsyncHttpClientCall.toIOException(AsyncHttpClientCall.java:295)
at org.asynchttpclient.extras.retrofit.AsyncHttpClientCall.execute(AsyncHttpClientCall.java:133)
at retrofit2.OkHttpCall.execute(OkHttpCall.java:190)
at com.univocity.trader.exchange.binance.api.client.impl.BinanceApiServiceGenerator.executeSync(BinanceApiServiceGenerator.java:79)
at com.univocity.trader.exchange.binance.api.client.impl.BinanceApiRestClientImpl.getAllPrices(BinanceApiRestClientImpl.java:102)
at com.univocity.trader.exchange.binance.BinanceExchange.getLatestPrices(BinanceExchange.java:153)
at com.univocity.trader.account.TradingManager.getAllPrices(TradingManager.java:107)
at com.univocity.trader.account.AccountManager.getTotalFundsIn(AccountManager.java:365)
at com.univocity.trader.account.AccountManager.executeAllocateFunds(AccountManager.java:277)
at com.univocity.trader.account.AccountManager.allocateFunds(AccountManager.java:325)
at com.univocity.trader.account.AccountManager.allocateFunds(AccountManager.java:347)
at com.univocity.trader.account.TradingManager.allocateFunds(TradingManager.java:192)
at com.univocity.trader.account.Trader.prepareTrade(Trader.java:370)
at com.univocity.trader.account.Trader.buy(Trader.java:418)
at com.univocity.trader.account.Trader.processBuy(Trader.java:201)
at com.univocity.trader.account.Trader.trade(Trader.java:170)
at com.univocity.trader.strategy.TradingEngine.process(TradingEngine.java:93)
at com.univocity.trader.candles.CandleProcessor.processCandle(CandleProcessor.```
Hello, I've been following your trading framework for a long time
I find that the framework is very advantageous for trend trading, but it is not easy to handle the cross market hedging of a single strategy
e.g
Cross market hedging. Trading Pairs Binance btc_usdt,huobi btc_usdt
Cross market triangular hedging. Trading Pairs Binance btc_usdt,eth_usdt; huobi eth_btc
com.univocity.trader.strategy.Strategy#getSignal We need a pairs. We need a market order In this way, we can judge whether the three comparisons are profitable or not, and it is not necessary to use indicators comparison
Thanks!
Hi,
I'm very interest by what you made, and i'm looking for the way to check when a indicator crossover up or down an other indicator or a value.
Thanks
I was considering to brute-force parameters with backtesting data on a AWS EC2 instance, one of the instances with 96 cores, or maybe one of the instances with 128 cores (although the latter has almost 2TB of RAM so they're like 3x more expensive than the ones with 96 cores).
Roughly the ones with 96 cores cost like $4-$5 an hour, and the ones with 128 cores like $12/$13 an hour.
This price is very expensive if it's going to run 24/7 for a month, but it's a perfectly acceptable price if you're going to run your simulations for a limited time, @jbax mentioned that "I can run the process described above in 20 minutes on a threadripper 3990x (we're talking about billions of candles processes over and over) ." And TR 3990x has 64 cores, so with 96 or 128 cores would be even better.
I was considering how could I maximize performance and minimize cost. I thought about using docker for a database and bring it up but it would take too much time to import a dump and then get the new data etc. Then I thought about using Amazon RDS but if I only need to run the process for a short time then I'd have to pay for the database to hold my data without using it.
I considered implementing support for reading binary data (as it would be faster to read than text files), and I saw ActiveJ Serializer as a very fast option, but it required adding annotations to the model and it was overhead.
Then I thought that maybe an embedded database could be perfect. I considered Nitrite, a NoSql embedded database based on H2, but probably SQL based was less work since most of the database code would already be the same. Nitrite also requires some annotations on the models.
The best fit is H2, is written in Java and is very fast. It would be great for unit tests to use in-memory, and for heavy simulations on AWS would be easy to just copy the H2 database file with all the backtesting data and the jar of the application. It would be great to run it in-memory in one of the AWS instances with lots of RAM, but then we would have to get all the backtesting data. It's more convenient just uploading a database file.
I tried in a branch and is very easy to setup, I'll create a PR soon for that.
On the Binance Public API you can see a mention to the SPOT Test Network (https://binance-docs.github.io/apidocs/spot/en/#spot-testnet)
On the Test net, you use realtime data but you don't use real money.
We need a way to use the Test Net
I get the following maven message:
[ERROR] Failed to execute goal on project univocity-trader-examples: Could not resolve dependencies for project com.univocity:univocity-trader-examples:jar:1.0.0-SNAPSHOT: Failure to find com.univocity:univocity-trader-interactivebrokers:jar:1.0.0-SNAPSHOT in https://dl.bintray.com/testng-team/testng/ was cached in the local repository, resolution will not be reattempted until the update interval of testng has elapsed or updates are forced -> [Help 1]
As suggested in issue #1:
One thing you may want to add in the future for better performance is a caching mechanism for your indicators to reduce repeated computation time. Here's a Java TA library which does this as an example:
title
The ChartWindow will fire up a test UI I'm writing in plain java swing.
All colors, fonts and other settings used to plot the charts are currently hard coded in the controller classes. Would be great to be able to save any changes made via UI into a new theme file as well.
These should, in addition to using the given defaults, load a properties or json or maybe a limited CSS file (whatever makes sense) which define a theme. Ideally the user could load the preferred theme on the fly with the program running. This is partly supported already through implementations of the UpdateProcessor
interface.
I'm opening this issue if anyone wants to lend a hand here.
Please provide univocity-trader.properties file to run sample chart window.
As described in the link: https://github.com/cryo8822/trader
Can trade through InteractiveBrokers and MB Trading. Can receive feeds from Yahoo and ActiveTick.
The caveat here is that the Groovy code must be converted to pure Java.
A minor PR for a small bug:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.