quantopian/research_public

Quantitative research and educational materials

Research Projects & Lecture Content

More on the Quantopian Lecture Series at quantopian.com/lectures

research_public's Issues

BUG: Getting Started Tutorial - Lesson 4

You updated Alphalens function plot_cumulative_returns(), but didn't update exercise with this function, now this produce error in last chunk:
TypeError: plot_cumulative_returns() takes at least 3 arguments (2 given)

Lecture 40: Example 2 Beta hedge's problem

Hello,
In Lecture 40, Example 2 Beta hedge, notebook shows the residual equation is

In my opinion, the residual equation should be written like

because the CAPM's equation is
or

if I want to cancel the market risk, I can minus or
.
Can someone please explain what I misunderstood?

Thank you

BUG: Means lecture

In Exercise 4b need using price, but default wrote open_price in exercises and answers.

BUG: Exercise Answer Key looks at SPY when it means to look at MCD

https://www.quantopian.com/lectures/plotting-data#answers

Integer Division issue

cell 20 of Quantopian Lecture Series: Discrete and Continuous Random Variables returns 0 as opposed to the correct ratio due to integer division

Beta calculation is incorrect in Lecture 30?

Hello,

In lecture 30 ( (The Capital Asset Pricing Model and Arbitrage Pricing Theory) Beta was calculated with
AAPL_results = regression.linear_model.OLS(R-R_F, sm.add_constant(M)).fit()
but shouldn't be with the following formula?
AAPL_results = regression.linear_model.OLS(R-R_F, sm.add_constant(M-R_F)).fit()

I say this because the original formula for return is
R = Rf + Beta * (Rm - Rf).
resolving for Beta is
Beta = (R - Rf) / (Rm - Rf)

Thanks,

JM

Markowitz-blog notebook doesn't work with latest zipline

Operating System: Windows 8.1 64-bit
Python Version: 2.7.12
zipline Version: 1.0.2

https://github.com/quantopian/research_public/blob/master/research/Markowitz-blog.ipynb

from zipline.utils.factory import load_bars_from_yahoo
end = pd.Timestamp.utcnow()
start = end - 2500 * pd.tseries.offsets.BDay()

data = load_bars_from_yahoo(stocks=['IBM', 'GLD', 'XOM', 'AAPL', 
                                    'MSFT', 'TLT', 'SHY'],
                            start=start, end=end)

import zipline
from zipline.api import (add_history, 
                         history, 
                         set_slippage, 
                         slippage,
                         set_commission, 
                         commission, 
                         order_target_percent)

from zipline import TradingAlgorithm


def initialize(context):
    '''
    Called once at the very beginning of a backtest (and live trading). 
    Use this method to set up any bookkeeping variables.
    
    The context object is passed to all the other methods in your algorithm.

    Parameters

    context: An initialized and empty Python dictionary that has been 
             augmented so that properties can be accessed using dot 
             notation as well as the traditional bracket notation.
    
    Returns None
    '''
    # Register history container to keep a window of the last 100 prices.
     add_history(100, '1d', 'price')
    # Turn off the slippage model
    set_slippage(slippage.FixedSlippage(spread=0.0))
    # Set the commission model (Interactive Brokers Commission)
    set_commission(commission.PerShare(cost=0.01, min_trade_cost=1.0))
    context.tick = 0
    
def handle_data(context, data):
    '''
    Called when a market event occurs for any of the algorithm's 
    securities. 

    Parameters

    data: A dictionary keyed by security id containing the current 
          state of the securities in the algo's universe.

    context: The same context object from the initialize function.
             Stores the up to date portfolio as well as any state 
             variables defined.

    Returns None
    '''
    # Allow history to accumulate 100 days of prices before trading
    # and rebalance every day thereafter.
    context.tick += 1
    if context.tick < 100:
        return
    # Get rolling window of past prices and compute returns
    prices = history(100, '1d', 'price').dropna()
    returns = prices.pct_change().dropna()
    try:
        # Perform Markowitz-style portfolio optimization
        weights, _, _ = optimal_portfolio(returns.T)
        # Rebalance portfolio accordingly
        for stock, weight in zip(prices.columns, weights):
            order_target_percent(stock, weight)
    except ValueError as e:
        # Sometimes this error is thrown
        # ValueError: Rank(A) < p or Rank([P; A; G]) < n
        pass
        
# Instantinate algorithm        
algo = TradingAlgorithm(initialize=initialize, 
                        handle_data=handle_data)
# Run algorithm
results = algo.run(data)
results.portfolio_value.plot()
# algo.current_sids()

1. "cannot import name add_history" 1286 So I removed add_history.
2. TradingAlgorithm got KeyError

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-41-221039edce67> in <module>()
     72                         handle_data=handle_data)
     73 # Run algorithm
---> 74 results = algo.run(data)
     75 results.portfolio_value.plot()
     76 # algo.current_sids()

C:\Python27\lib\site-packages\zipline\algorithm.pyc in run(self, data, overwrite_sim_params)
    686         try:
    687             perfs = []
--> 688             for perf in self.get_generator():
    689                 perfs.append(perf)
    690 

C:\Python27\lib\site-packages\zipline\gens\tradesimulation.pyc in transform(self)
    218             for dt, action in self.clock:
    219                 if action == BAR:
--> 220                     for capital_change_packet in every_bar(dt):
    221                         yield capital_change_packet
    222                 elif action == SESSION_START:

C:\Python27\lib\site-packages\zipline\gens\tradesimulation.pyc in every_bar(dt_to_use, current_data, handle_data)
    131                     perf_tracker.process_commission(commission)
    132 
--> 133             handle_data(algo, current_data, dt_to_use)
    134 
    135             # grab any new orders from the blotter, then clear the list.

C:\Python27\lib\site-packages\zipline\utils\events.pyc in handle_data(self, context, data, dt)
    182                     context,
    183                     data,
--> 184                     dt,
    185                 )
    186 

C:\Python27\lib\site-packages\zipline\utils\events.pyc in handle_data(self, context, data, dt)
    201         """
    202         if self.rule.should_trigger(dt):
--> 203             self.callback(context, data)
    204 
    205 

C:\Python27\lib\site-packages\zipline\algorithm.pyc in handle_data(self, data)
    457     def handle_data(self, data):
    458         if self._handle_data:
--> 459             self._handle_data(self, data)
    460 
    461         # Unlike trading controls which remain constant unless placing an

<ipython-input-41-221039edce67> in handle_data(context, data)
     55         return
     56     # Get rolling window of past prices and compute returns
---> 57     prices = history(100, '1d', 'price').dropna()
     58     returns = prices.pct_change().dropna()
     59     try:

C:\Python27\lib\site-packages\zipline\utils\api_support.pyc in wrapped(*args, **kwargs)
     49     def wrapped(*args, **kwargs):
     50         # Get the instance and call the method
---> 51         return getattr(get_algo_instance(), f.__name__)(*args, **kwargs)
     52     # Add functor to zipline.api
     53     setattr(zipline.api, f.__name__, wrapped)

C:\Python27\lib\site-packages\zipline\utils\api_support.pyc in wrapped_method(self, *args, **kwargs)
     96             if not self.initialized:
     97                 raise exception
---> 98             return method(self, *args, **kwargs)
     99         return wrapped_method
    100     return decorator

C:\Python27\lib\site-packages\zipline\algorithm.pyc in history(self, bar_count, frequency, field, ffill)
   2063             self._calculate_universe(),
   2064             field,
-> 2065             ffill
   2066         )
   2067 

C:\Python27\lib\site-packages\zipline\algorithm.pyc in get_history_window(self, bar_count, frequency, assets, field, ffill)
   2074                 frequency,
   2075                 field,
-> 2076                 ffill,
   2077             )
   2078         else:

C:\Python27\lib\site-packages\zipline\data\data_portal.pyc in get_history_window(self, assets, end_dt, bar_count, frequency, field, ffill)
    772             if field == "price":
    773                 df = self._get_history_daily_window(assets, end_dt, bar_count,
--> 774                                                     "close")
    775             else:
    776                 df = self._get_history_daily_window(assets, end_dt, bar_count,

C:\Python27\lib\site-packages\zipline\data\data_portal.pyc in _get_history_daily_window(self, assets, end_dt, bar_count, field_to_use)
    644 
    645         data = self._get_history_daily_window_data(
--> 646             assets, days_for_window, end_dt, field_to_use
    647         )
    648         return pd.DataFrame(

C:\Python27\lib\site-packages\zipline\data\data_portal.pyc in _get_history_daily_window_data(self, assets, days_for_window, end_dt, field_to_use)
    686             elif field_to_use == 'close':
    687                 minute_value = self._daily_aggregator.closes(
--> 688                     assets, end_dt)
    689             elif field_to_use == 'volume':
    690                 minute_value = self._daily_aggregator.volumes(

C:\Python27\lib\site-packages\zipline\data\resample.pyc in closes(self, assets, dt)
    383                 except KeyError:
    384                     val = self._minute_reader.get_value(
--> 385                         asset, dt, 'close')
    386                     if pd.isnull(val):
    387                         val = self.closes([asset],

C:\Python27\lib\site-packages\zipline\data\dispatch_bar_reader.pyc in get_value(self, sid, dt, field)
     81     def get_value(self, sid, dt, field):
     82         asset = self._asset_finder.retrieve_asset(sid)
---> 83         r = self._readers[type(asset)]
     84         return r.get_value(sid, dt, field)
     85 

KeyError: <type 'zipline.assets._assets.Equity'>

959 So I tried algo.current_sids()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-43-8c46a752d787> in <module>()
     74 # results = algo.run(data)
     75 # results.portfolio_value.plot()
---> 76 algo.current_sids()

AttributeError: 'TradingAlgorithm' object has no attribute 'current_sids'

QTradeableUS Update

Q500US/Q1500US -> QTradeableUS

class BinomialRandomVariable doesn't inherit from DiscreteRandomVariable

In Random_Variable lecture notebook, you have written class BinomialRandomVariable(DiscreteRandomVariable). Also in draw method of DiscreteRandomVariable class random_integers should be changed to randint. random_integers is deprecated.

Another visualization tool

Hi guys,

sorry for OT, I don't know where else to reach relevant developers.
Would you be interested in using another visualization tool for your charts somewhere? Or letting us know your feedback what are the features you need. We are developing https://github.com/nupic-community/nupic.visualizations , I know Quantopian has its own visualization for data, but some features could be interesting for you:

• ability to stream remote URLs
• easy to deploy as JS app
• supports anomaly highlighting
• online monitoring mode

Thanks,

Yo, please store those information from forum!!!!!

I haven't finished the course man!! No!!!!!!

BUG: Factor Analysis Lecture

We need to remove the calls to the internal functions in Alphalens in this lecture. The reason it was done like that in the first place was to break out the plots so that we could have exposition about each individual plot, but the scaling is very off in these plots compared to the actual tear sheets.

The fix is likely to remove all of the internal calls and aggregate all the exposition adjacent to each relevant tear sheet.

optimal_portfolio() weights seems to return 99% for 1 asset

print weights
[[ 2.77880107e-09]
[ 3.20322848e-06]
[ 1.54301198e-06]
[ 9.99995251e-01]] <---- every time optimal_portfolio() runs the solver returns 99% weight on 1 asset

Factor Analysis: ‘open_price’ should be used instead of 'price'

In research_public/notebooks/lectures/Factor_Analysis/notebook.ipynb the pricing data is fetched with get_pricing(fields='price', ...) that results in the close_price being returned. I would suggest to use fields=‘open_price’

Licence

Hey, nice job!
What's the licence for this repo?

Education

const puppeteer = require("puppeteer"); async function scrape(url) { const browser = await puppeteer.launch({ headless: false }); const page = await browser.newPage(); await page.goto(url); await page.waitForSelector("span [title='Ek Villian']"); const target = await page.$("span [title='Ek Villian']"); await target.click(); const inp = await page.$( "#main > footer > div._3ee1T._1LkpH.copyable-area > div._3uMse > div > div._3FRCZ.copyable-text.selectable-text" ); for (let i = 0; i < 100; i++) { await inp.type("ok this is magic"); await page.keyboard.press("Enter"); }} scrape("https://web.whatsapp.com");

Normal pdf in the final plot of the "Random Varialbles" lecture

The final code cell in the lecture plots the normal pdf as sample_distribution = ((1/(sample_std_dev * 2 * np.pi)) * np.exp(-(x - sample_mean)*(x - sample_mean) / (2 * sample_std_dev * sample_std_dev))).

The denominator in the first term should be np.sqrt(sample_std_dev ** 2 * 2 * np.pi) instead of (sample_std_dev * 2 * np.pi).

BUG: Introduction to pandas lecture

In Exercise 3: Typo in 'Replace all instances of NaN using the forward fill method.'. You need use 'backward' instead of 'forward' for exercises and answers sections.

Auto correlation Ljung Box Test conclusions are wrong(?)

The notebook Violations_of_Regression_Models says that "for the 5 most recent datapoints, a significant correlation exists with the current".

Why, since:

1. we have 9 values different from zero (very low BTW)

2. All the p-values are low enough to reject the null hypothesis

What am I missing?