tradytics / eiten Goto Github PK
View Code? Open in Web Editor NEWStatistical and Algorithmic Investing Strategies for Everyone
Home Page: https://www.tradytics.com/
License: GNU General Public License v3.0
Statistical and Algorithmic Investing Strategies for Everyone
Home Page: https://www.tradytics.com/
License: GNU General Public License v3.0
I went through the Genetic Algorithm Strategy code and it seems to me to be actually random. The purpose of selecting the top K is that we want to preserve their qualities and use them in crossover. However, that preservation does not occur since their are straight crossed. Further, there is no probability involved into mutation.
Point being: The qualities of top k are most likely lost because the code does not preserve them, and does not prioritize them into crossover (we want to reproduce its qualities).
future_prices
is initialized to None
, https://github.com/tradytics/eiten/blob/master/data_loader.py#L51
If is_test
is not 1
, future_prices
doesn't change, https://github.com/tradytics/eiten/blob/master/data_loader.py#L99
future_prices
as None
is called with .values
, https://github.com/tradytics/eiten/blob/master/data_loader.py#L111
Hello,
I was wondering how to save this to a JSON file for algo-trading purposes
On the subject.
Additionally, would one be able to integrate realtime data, plus auto buy/sell?
There is a serious issue for the portfolio weights. When run for long only, there is not code do divide sum(weights). Total weights is almost round 2.5 to 4. So the result scaled 2.5 to 4. Because the project use the bull market data. So we got a pretty beautiful cumsum curve.
Fresh install and run the README command โฐโ python portfolio_manager.py --is_test 1 --future_bars 90 --data_granularity_minutes 3600 --history_to_use all --apply_noise_filtering 1 --market_index QQQ --only_long 1 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
Then got the following error, is this because yfinance is having issue?
Loading data for all stocks...
0%| | 0/6 [00:00<?, ?it/s]Exception in thread Thread-2:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/multitasking/init.py", line 102, in _run_via_pool
return callee(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/yfinance/multi.py", line 166, in _download_one_threaded
data = _download_one(ticker, start, end, auto_adjust, back_adjust,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/yfinance/multi.py", line 178, in _download_one
return Ticker(ticker).history(period=period, interval=interval,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/yfinance/base.py", line 155, in history
data = data.json()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/requests/models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
This is all on Google Colab.
I tried showing plot but I get this:
<Figure size 1200x600 with 1 Axes>
So I tried doing:
%matplotlib inline
!python portfolio_manager.py --is_test 1 --future_bars 90 --data_granularity_minutes 3600 --history_to_use all --apply_noise_filtering 1 --market_index QQQ --only_long 1 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
But still no output.
Any ideas why?
Hi,
I'm building portfolios for my home country's stock market.
However there is limited options for shorting stocks.
Is it possible to run Eiten without allowing negative weight (shorting)?
Does the portfolio backtesting backtest the current portfolios or portfolios constructed at the time of backtesting?
Thanks!
When i use the integer(for example 100) as value for for --history_to_use, error happens.
File "C:\Users\hello\Repo\eiten\argchecker.py", line 8, in __init__
self.check_arguments(args)
File "C:\Users\hello\Repo\eiten\argchecker.py", line 20, in check_arguments
assert not(args.history_to_use != "all" and int(args.history_to_use_int) <
AttributeError: 'Namespace' object has no attribute 'history_to_use_int'
args.history_to_use_int should be the reason, there isn't history_to_use_int argument parameter
Running this command from the examples:
python portfolio_manager.py --is_test 0 --future_bars 30 --data_granularity_minutes 60 --history_to_use all --apply_noise_filtering 1 --market_index QQQ --only_long 1 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
returns:
Traceback (most recent call last):
File "portfolio_manager.py", line 37, in
main()
File "portfolio_manager.py", line 33, in main
eiten.run_strategies()
File "Work/eiten/eiten.py", line 107, in run_strategies
historical_price_info, future_prices, symbol_names, predicted_return_vectors, returns_matrix, returns_matrix_percentages = self.load_data()
File "Work/eiten/eiten.py", line 88, in load_data
self.data_dictionary = self.dataEngine.collect_data_for_all_tickers()
File "Work/eiten/data_loader.py", line 153, in collect_data_for_all_tickers
historical_price, future_price, symbol_names)
File "Work/eiten/data_loader.py", line 171, in remove_bad_data
most_common_length = length_dictionary[0]
IndexError: list index out of range
Any thoughts on what I'm doing wrong?
There is probably an error in the choice of stocks because the choice is made on stocks of which we know the good behavior
for the period 2015/01 2020/01 the proposed portfolio (AAPL, AMD, AMZN, FB, MSFT, NFLX, NVDA, TSLA) with equal weights and buy and hold has a performance of approximately 1000% !!.
Eiten GA returns 450%. Not so good.
If you randomly choose stocks from 2015, the result is not good.
When I run this from the examples:
python portfolio_manager.py --is_test 1 --future_bars 90 --data_granularity_minutes 3600 --history_to_use all --apply_noise_filtering 1 --market_index QQQ --only_long 1 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
I get:
Traceback (most recent call last):
File "portfolio_manager.py", line 37, in
main()
File "portfolio_manager.py", line 33, in main
eiten.run_strategies()
File "/Users/matt/Desktop/eiten-master/eiten.py", line 118, in run_strategies
returns_matrix)
File "/Users/matt/Desktop/eiten-master/strategy_manager.py", line 59, in random_matrix_theory_based_cov
filtered_covariance_matrix = self.strategyHelperFunctions.random_matrix_theory_based_cov(returns_matrix)
File "/Users/matt/Desktop/eiten-master/strategies/strategy_helper_functions.py", line 18, in random_matrix_theory_based_cov
variances = np.diag(np.cov(returns_matrix))
File "<array_function internals>", line 6, in diag
File "/Users/matt/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/twodim_base.py", line 283, in diag
raise ValueError("Input must be 1- or 2-d.")
ValueError: Input must be 1- or 2-d.
I used this just yesterday to do an analysis with no problem. Also, I looked at the other issue in the list that said maybe there's some issue with a stock picked in stocks.txt, so picked two stocks (AAPL and BA) to check if that could be it and it still causes an error.
First of all thank you! This tool was so easy to get up and running, despite the obvious underlying complexity.
After running portfolio_manager.py
on a few different stock lists I am left with the following simple questions:
README
only explains the former, I am having trouble interpreting the latter. They seem to be wildly different!Hi,
I'm building portfolios for my home country's stock market.
However there is limited options for shorting stocks.
Is it possible to run Eiten without allowing negative weight (shorting)?
With the most recent code, I run the following command:
python3 portfolio_manager.py --is_test 1 --future_bars 90 --data_granularity_minutes 3600 --history_to_use all --apply_noise_filtering 1 --market_index QQQ --only_long 1 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
Previously, this worked. Now, I get an error:
Traceback (most recent call last):
File "portfolio_manager.py", line 37, in <module>
main()
File "portfolio_manager.py", line 33, in main
eiten.run_strategies()
File "/Users/jmartinezlago/eiten/eiten.py", line 107, in run_strategies
historical_price_info, future_prices, symbol_names, predicted_return_vectors, returns_matrix, returns_matrix_percentages = self.load_data()
File "/Users/jmartinezlago/eiten/eiten.py", line 95, in load_data
self.data_dictionary[symbol]["historical_prices"])
KeyError: 'historical_prices'
Firstly, thank you for putting this together. I've been working on portfolio optimization through backtesting and this is definitely a simple and great setup. As I was testing your code, I wanted to call out a few things,
The only_long
setting does not work. Irrespective of whether this is set to 0 or 1, the resulting weights are long_short lying between (-1, 1). This is the exact command i used as per your description:
python portfolio_manager.py --is_test 1 --future_bars 90 --data_granularity_minutes 3600 --history_to_use all --apply_noise_filtering 1 --market_index SPY --only_long 0 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
I only changed only_long
to 1 or 0
in my tests and it doesn't alter the weights. Let me know if I'm doing something wrong.
I then used the weights that were returned by your algorithm and plotted it on the efficient frontier created with over 150,000 portfolios (from random weights) on the same ohlc dataset.
Below are the weights from your algorithm,
-------- Weights for Eigen Portfolio --------
Symbol: AAPL, Weight: 0.3492
Symbol: AMD, Weight: -0.7297
Symbol: AMZN, Weight: 0.3650
Symbol: FB, Weight: 0.4094
Symbol: MSFT, Weight: 0.3314
Symbol: NFLX, Weight: 0.7475
Symbol: NVDA, Weight: 1.0672
Symbol: TSLA, Weight: -1.5400
-------- Weights for Maximum Sharpe Portfolio (MSR) --------
Symbol: AAPL, Weight: 0.6987
Symbol: AMD, Weight: -0.1833
Symbol: AMZN, Weight: -0.9613
Symbol: FB, Weight: 0.7741
Symbol: MSFT, Weight: 0.7640
Symbol: NFLX, Weight: -0.2281
Symbol: NVDA, Weight: -0.1171
Symbol: TSLA, Weight: 0.2530
-------- Weights for Genetic Algo (GA) --------
Symbol: AAPL, Weight: -1.2315
Symbol: AMD, Weight: 0.9703
Symbol: AMZN, Weight: 1.3787
Symbol: FB, Weight: -1.1198
Symbol: MSFT, Weight: 1.9202
Symbol: NFLX, Weight: 0.2728
Symbol: NVDA, Weight: 0.9967
Symbol: TSLA, Weight: 0.0455
This is what the efficient frontier looks like for the above weights with the backtest data:
For the portfolios returned by your max sharpe (Eiten max sharpe) and Eigen (Eiten Eigen PF) the sharpe ratios are 0.07 and 0.15 respectively. As you can see, the genetic algorithm weights returned a sharpe ratio of 1.78 and the starred max_sharpe is 1.63 (close, but definitely more room for optimization). The starred max_sharpe is just the best performing random weight portfolio. We can nearly draw a straight line from the risk free return to the GA point cutting through the starred max_sharpe points. I guess the GA has only cranked up the risk for higher returns.
Similary the EF for the future data is as shown below and the max_sharpe in this case is 3.78 (from random weights):
Let me know if i'm doing anything wrong running this as the performance of max sharpe and eigen portfolio from your algorithm does not correspond to the best performing portfolio even in the back testing case. There is still room to reduce risk and increase returns here.
The random_matrix_theory_based_cov
seems to be vanishing with the covariance values. Empirical evidence shows this:
Experiment one:
How Eiten
works in the current push, without matrix normalization:
This is a regular matrix without the noise cleaning method
After transformation the matrix becomes this:
We can see a shadow of red in the diagonal so I though I would plot out of scale. This is the result:
This leads me to question whether there is an implementation issue or if the method is really reliable.
Hi how does one load in this https://finance.yahoo.com/quote/ICNYB.PA?p=ICNYB.PA ?
Tried different combo's of ticker but throws error or ignored (if use ICNYB.PA) ...
Txs
Thanks for making this! Looks really cool and I am excited to try it out.
I would like to use this to backtest a portfolio's asset allocation and compare it to an index.
Example:
Ticker,% of Allocation
AAPL,0.5
TSLA,0.25
MSFT,0.20
AMZN,0.05
Given this fixed asset allocation of the tickers above, how do I backtest this portfolio and compare it against an index?
The key thing to note here is that the allocation is fixed in terms of % of portfolio.
Command to run:
python3 portfolio_manager.py --is_test 1 --future_bars 90 --data_granularity_minutes 3600 --history_to_use all --apply_noise_filtering 1 --market_index QQQ --only_long 1 --eigen_portfolio_number 3 --stocks_file_path stocks/stocks.txt
When loading data for all stocks:
Exception No objects to concatenate
Then, I see only info about 4 stocks, even though there's 15 in my stock file.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.