Giter Site home page Giter Site logo

quantstrat's Introduction

Travis Build Status Codecov test coverage

quantstrat

Transaction-oriented infrastructure for constructing trading systems and simulation. Provides support for multi-asset class and multi-currency portfolios for backtesting and other financial research.

Overview

quantstrat provides a generic infrastructure to model and backtest signal-based quantitative strategies. It is a high-level abstraction layer (built on xts, FinancialInstrument, blotter, etc.) that allows you to build and test strategies in very few lines of code. quantstrat is still under heavy development but is being used every day on real portfolios. We encourage you to send contributions and test cases via the appropriate GitHub mediums (Pull requests and Issue tracker).

Installation

In order to install quantstrat from GitHub, you will first need to install devtools and blotter from GitHub.

install.packages("devtools") # if not installed
install.packages("FinancialInstrument") #if not installed
install.packages("PerformanceAnalytics") #if not installed

# next install blotter from GitHub
devtools::install_github("braverock/blotter")
# next install quantstrat from GitHub
devtools::install_github("braverock/quantstrat")

Example: maCross

The demos in the demo folder are great for learning how to use quantstrat specifically. Below is an example of the maCross strategy, a simple moving average strategy using a short-term SMA of 50 days and a long-term SMA of 200 days.

We specify the symbol/s and currency/ies before defining the stock metadata using the stock() function from the FinancialInstrument package, available on CRAN.

stock.str='AAPL' # what are we trying it on
currency('USD')
#> [1] "USD"
stock(stock.str,currency='USD',multiplier=1)
#> [1] "AAPL"

Next we set up the rest of the backtest charateristics:

  • start date
  • initial equity
  • portfolio and account names
  • initialize Portfolio, Account and Orders objects
  • assign strategy object to “stratMACROSS”
startDate="1999-12-31"
initEq=1000000
portfolio.st='macross'
account.st='macross'
initPortf(portfolio.st,symbols=stock.str)
#> [1] "macross"
initAcct(account.st,portfolios=portfolio.st, initEq=initEq)
#> [1] "macross"
initOrders(portfolio=portfolio.st)
stratMACROSS<- strategy(portfolio.st)

We are now ready to add indicators, signals and rules. For more information on the theory of this approach, see below sections “About Signal-Based Strategy Modeling” and “How quantstrat Models Strategies”.

stratMACROSS <- add.indicator(strategy = stratMACROSS, name = "SMA", arguments = list(x=quote(Cl(mktdata)), n=50),label= "ma50" )
stratMACROSS <- add.indicator(strategy = stratMACROSS, name = "SMA", arguments = list(x=quote(Cl(mktdata)[,1]), n=200),label= "ma200")

stratMACROSS <- add.signal(strategy = stratMACROSS,name="sigCrossover",arguments = list(columns=c("ma50","ma200"), relationship="gte"),label="ma50.gt.ma200")
stratMACROSS <- add.signal(strategy = stratMACROSS,name="sigCrossover",arguments = list(column=c("ma50","ma200"),relationship="lt"),label="ma50.lt.ma200")

stratMACROSS <- add.rule(strategy = stratMACROSS,name='ruleSignal', arguments = list(sigcol="ma50.gt.ma200",sigval=TRUE, orderqty=100, ordertype='market', orderside='long'),type='enter')
stratMACROSS <- add.rule(strategy = stratMACROSS,name='ruleSignal', arguments = list(sigcol="ma50.lt.ma200",sigval=TRUE, orderqty='all', ordertype='market', orderside='long'),type='exit')

# if you want a long/short Stops and Reverse MA cross strategy, you would add two more rules for the short side:

# stratMACROSS <- add.rule(strategy = stratMACROSS,name='ruleSignal', arguments = list(sigcol="ma50.lt.ma200",sigval=TRUE, orderqty=-100, ordertype='market', orderside='short'),type='enter')
# stratMACROSS <- add.rule(strategy = stratMACROSS,name='ruleSignal', arguments = list(sigcol="ma50.gt.ma200",sigval=TRUE, orderqty=100, ordertype='market', orderside='short'),type='exit')

Now all we need to do is add our market data before calling the applyStrategy function to initiate the backtest.

getSymbols(stock.str,from=startDate)
#> [1] "AAPL"
for(i in stock.str)
  assign(i, adjustOHLC(get(i),use.Adjusted=TRUE))

start_t<-Sys.time()
out<-applyStrategy(strategy=stratMACROSS , portfolios=portfolio.st)
#> [1] "2001-06-27 00:00:00 AAPL 100 @ 1.443241"
#> [1] "2001-09-07 00:00:00 AAPL -100 @ 1.068518"
#> [1] "2002-01-07 00:00:00 AAPL 100 @ 1.416034"
#> [1] "2002-07-10 00:00:00 AAPL -100 @ 1.070991"
#> [1] "2003-05-16 00:00:00 AAPL 100 @ 1.162508"
#> [1] "2006-06-22 00:00:00 AAPL -100 @ 7.368322"
#> [1] "2006-09-26 00:00:00 AAPL 100 @ 9.598111"
#> [1] "2008-03-07 00:00:00 AAPL -100 @ 15.118788"
#> [1] "2008-05-19 00:00:00 AAPL 100 @ 22.706005"
#> [1] "2008-09-24 00:00:00 AAPL -100 @ 15.917701"
#> [1] "2009-05-14 00:00:00 AAPL 100 @ 15.205353"
#> [1] "2012-12-11 00:00:00 AAPL -100 @ 67.548859"
#> [1] "2013-09-11 00:00:00 AAPL 100 @ 59.474586"
#> [1] "2015-08-31 00:00:00 AAPL -100 @ 104.390999"
#> [1] "2016-08-31 00:00:00 AAPL 100 @ 100.325439"
#> [1] "2018-12-24 00:00:00 AAPL -100 @ 143.924454"
#> [1] "2019-05-07 00:00:00 AAPL 100 @ 199.698502"
end_t<-Sys.time()
print(end_t-start_t)
#> Time difference of 0.1832633 secs

Before we can review results using chart.Posn(), we update the portfolio.

start_t<-Sys.time()
updatePortf(Portfolio='macross',Dates=paste('::',as.Date(Sys.time()),sep=''))
#> [1] "macross"
end_t<-Sys.time()
print("trade blotter portfolio update:")
#> [1] "trade blotter portfolio update:"
print(end_t-start_t)
#> Time difference of 0.03264308 secs

chart.Posn(Portfolio='macross',Symbol=stock.str, TA=c("add_SMA(n=50,col='red')","add_SMA(n=200,col='blue')"))

If you would like to zoom into a particular period, you can use quantmod’s zoomChart().

quantmod::zoomChart()

zoom_Chart('2014::2018')

Warning

A backtest cannot be unseen. In the words of Lopez de Prado from his book Advances in Financial Machine Learning, “Backtesting is one of the most essential, and yet least understood, techniques in the quant arsenal. A common misunderstanding is to think of backtesting as a research tool. Researching and backtesting is like drinking and driving. Do not research under the influence of a backtest. …A good backtest can be extremely helpful, but backtesting well is extremely hard.”

For a comprehensive overview of an hypothesis based approach to research and backtesting, see Developing & Backtesting Systematic Trading Strategies.

Resources

Below is a growing list of resources (some actively being developed) as relates to quantstrat:

About Signal-Based Strategy Modeling

A signal-based strategy model first generates indicators. Indicators are quantitative values derived from market data (e.g. moving averages, RSI, volatility bands, channels, momentum, etc.). Indicators should be applied to market data in a vectorized (for fast backtesting) or streaming (for live execution) fashion, and are assumed to be path-independent (i.e. they do not depend on account / portfolio characteristics, current positions, or trades).

The interaction between indicators and market data are used to generate signals (e.g. crossovers, thresholds, multiples, etc.). These signals are points in time at which you may want to take some action, even though you may not be able to. Like indicators, signals may be applied in a vectorized or streaming fashion, and are assumed to be path-independent.

Rules use market data, indicators, signals, and current account / portfolio characteristics to generate orders. Notice that rules about position sizing, fill simulation, order generation / management, etc. are separate from the indicator and signal generation process. Unlike indicators and signals, rules are generally evaluated in a path-dependent fashion (path-independent rules are supported but are rare in real life) and are aware of all prior market data and current positions at the time of evaluation. Rules may either generate new or modify existing orders (e.g. risk management, fill, rebalance, entry, exit).

How quantstrat Models Strategies

quantstrat uses FinancialInstrument to specify instruments (including their currencies) and uses blotter to keep track of transactions, valuations, and P&L across portfolios and accounts.

Indicators are often standard technical analysis functions like those found in TTR; and signals are often specified by the quantstrat sig* functions (i.e. sigComparison, sigCrossover, sigFormula, sigPeak, sigThreshold). Rules are typically specified with the quantstrat ruleSignal function.

The functions used to specify indicators, signals, and rules are not limited to those mentioned previously. The name parameter to add.indicator, add.signal, and add.rule can be any R function. Because the supporting toolchain is built using xts objects, custom functions will integrate most easily if they return xts objects.

The strategy model is created in layers and makes use of delayed execution. This means strategies can be applied–unmodified–to several different portfolios. Before execution, quantstrat strategy objects do not know what instruments they will be applied to or what parameters will be passed to them.

For example, indicator parameters such as moving average periods or thresholds are likely to affect strategy performance. Default values for parameters may (optionally) be set in the strategy object, or set at call-time via the parameters argument of applyStrategy (parameters is a named list, used like the arguments lists).

quantstrat models orders, which may or may not become transactions. This provides a lot of extra ability to evaluate how the strategy is actually working, not working, or could be improved. For example, the performance of strategies are often affected by how often resting limit orders are changed / replaced / canceled. An order book allows the quantitative strategist to examine market conditions at the time these decisions are made. Also, the order history allows for easy computation of things that are important for many strategies, like order-to-fill ratios.

Contributing

Please see the contributing guide.

quantstrat's People

Contributors

bosma avatar braverock avatar cloudcell avatar erastotle avatar gsee avatar guanm326 avatar ilyakipnis avatar jaymon0703 avatar joshuaulrich avatar llevenson avatar opentrades avatar peterccarl avatar ryogi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

quantstrat's Issues

[R-Forge #5814] store = TRUE not working for add.indicator and add.signal

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

Custom signal and indicator functions are not found when using apply.paramset, even though store=TRUE for both the add.indicator and add.signal calls. The store logic within add.rule is working, so I've attached proposed fixes that also incorporate that logic into add.indicator and add.signal. One other possible bug in the store logic for add.rule was also fixed. If there is a better way to accomplish this, please let me know.

Followups:

Date: 2014-11-06 19:20
Sender: Jonathan Owen
I've attached diff files with proposed updates.

Date: 2014-07-24 16:36
Sender: Jonathan Owen
I've also noticed that a custom function passed to TxnFees during the add.rule call is not stored. Since the TxnFees value is stored as a column in the orderbook, it doesn't look like the same functionality used for add.rule can be used to store the TxnFees function. I've found that this mainly becomes a problem when apply.paramset is run in parallel, and one solution is to add an export parameter to apply.paramset that is passed to the .export parameter of the foreach call.

Date: 2014-07-24 12:19
Sender: Jonathan Owen
Slight correction to the proposed fix. To be consistent with add.rule, add.signal and add.indicator should add a storefun = TRUE parameter instead of using the store parameter.

[R-Forge #1630] Stop loss rule broken?

Submitted by: Heiko Stegmann
Assigned to: Nobody
R-Forge link

After not being busy with quantstrat for some weeks, I updated all TradeAnalytics packages as of Oct 20 (blotter Rev. 778, FinancialInstrument 821, quantstrat 810, RTAQ 784).
Now, scripts that contain a stoplimit rule do not work anymore. The scripts stop with 'Error : Object 'orderprice' not found' when applyStrategy() is called. Try the macd.R demo, for example.

Followups:

Date: 2013-04-24 16:08
Sender: Brian Peterson
Cleaning up, but this has been fixed in svn for a while, and has been recently improved with commits by Josh Ulrich.Brian

Cannot load in quantstrat

I installed quantstrat via this github account, via devtools::install_github("braverock/quantstrat"), and after installation, when I try to load with library(quantstrat) I get this error:

internal error -3 in R_decompress1Error in get(Info[i, 1], envir = env) : lazy-load database '/home/curtis/R/i686-pc-linux-gnu-library/3.3/zoo/R/zoo.rdb' is corrupt Error: package or namespace load failed for ‘quantstrat’

I can load in zoo just fine.

[R-Forge #5784] Error in calculation of order.Price (the trailing stop price) for stoptrailing order updates?

Submitted by: Claymore Marshall
Assigned to: Joshua Ulrich
R-Forge link

In the orderType switch inside the ruleOrderProc function, I think there is a problem with how the trailing stop price is being calculated, for isTRUE(isOHLC) type data.

On lines ~288+, the current code is:

' For orderType == 'stoptrailing'

...
} else {
order.threshold <- as.numeric(ordersubset[ii,
'Order.Threshold'])
if (order.side == 'long') new.order.price <- max(orderPrice,
as.numeric(Hi(mktdataTimestamp)) + order.threshold)
if (order.side == 'short') new.order.price <- min(orderPrice,
as.numeric(Lo(mktdataTimestamp)) + order.threshold)

if (new.order.price != orderPrice) {
....

The problem is that Hi ( ...) returns multiple numbers, if mktdata (mktdataTimestamp) contains two or more columns with 'high' in the names. For example, try running a OHLC series and include the ATR columns as indicators. The Hi(mktdataTimestamp) will operate on the 'High' column AND 'trueHigh.ATR' column. If trueHigh happens to be > High, then you can get an incorrect updated new.order.price for a (long position) trailing stop order.

The mirror image problem applies to the Lo(mktdataTimestamp).

The fix is straightforward (assuming the first high column is always the OHLC high):

} else {
if (order.side == 'long') new.order.price <- max(orderPrice,
as.numeric(Hi(mktdataTimestamp)[, 1]) + order.threshold)
if (order.side == 'short') new.order.price <- min(orderPrice,
as.numeric(Lo(mktdataTimestamp)[, 1]) + order.threshold)

A related cosmetic change (which won't affect the code right now because R only evaluates the boolean for the first returned high/low value in the mktdataTimestamp object), which I think improves the clarity of the code in the same section, is to slightly modify 221-228:

' old code:

else if (isOHLCmktdata) {
order.side <- ordersubset[ii, 'Order.Side']
if (order.side == 'long' && as.numeric(Lo(mktdataTimestamp)) <
orderPrice || order.side == 'short' && as.numeric(Hi(mktdataTimestamp)) >
orderPrice) {
txnprice <- orderPrice
txntime <- timestamp
} else {

' New code:

else if (isOHLCmktdata) {
order.side <- ordersubset[ii, 'Order.Side']
if (order.side == 'long' && as.numeric(Lo(mktdataTimestamp)[, 1]) <
orderPrice || order.side == 'short' && as.numeric(Hi(mktdataTimestamp)[, 1]) >
orderPrice) {
txnprice <- orderPrice
txntime <- timestamp
} else {

Followups:

Date: 2014-08-03 01:05
Sender: Joshua Ulrich
Fixed in r1629.

[R-Forge #5788] tradeGraphs() error

Submitted by: Jonathan Owen
Assigned to: Evelyn Mitchell
R-Forge link

This function calls reshape2::recast and looks for a labels attribute to create the 3d plot. Since recast has been updated to use dcast instead of cast, this attribute is no longer returned. The updated code below appears to fix the error and generates a plot.

data_r = recast(data, as.formula(paste0(var1, ' ~ ', var2)),
id.var=c(var1,var2), measure.var=c(var3))
x <- data_r[, 1]
y <- as.numeric(colnames(data_r)[-1])
z <- unlist(data_r[, -1])

Followups:

Date: 2015-11-14 23:49
Sender: Joshua Ulrich
Fixed in r1624.

Date: 2014-07-01 13:22
Sender: Jonathan Owen
I've attached a file with proposed changes.

Problem when using a Rule of type Order with a Chain Rule

I am trying to use a rule of type: order in my strategy but it is giving an error due to closed.orders object doesn't exist. The reason is in the applyRules function, specifically in the switch case for the rule type, when there is a rule of type order, if (length(strategy$rules[[type]]) >= 1) evaluates to true, which in turn means the else clause does not get processed which is where closed.orders is set. This causes a problem when it goes on to evaluate the rules of type chain, when if (!is.null(closed.orders)) is evaluated, the closed.orders object doesn't exist. I am very unsure of my implementation of the order rule, but this problem makes me think add a rule of type order like a normal enter or exit rule at all? I am trying to use it to allow me to modify open orders (such as profit targets) when a signal occur. Im not even sure if this is the correct approach. Is there any documentation or examples anywhere doing this? I've had a good look but couldn't find anything.

[R-Forge #1328] argument length 0 in > 0 operations

Submitted by: G Lin
Assigned to: Nobody
R-Forge link

When the order book is initialized and order side = NULL, line 115 of tradeRules.R will error out on

'
Error in if (curqty >0) {: argumen tis of length zero

because getPosQty can return a 0 length result.

maybe getPosQty should be changed to return 0 by default? rather than NA as numeric or have tradeRules.R check the output has length > 0.

ruleOrderProc on line 576 has this same issue in orders.R

sessionInfo()
R version 2.12.1 (2010-12-16)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] quantstrat_0.4.0 RBloomberg_0.4-145 rJava_0.8-8 PerformanceAnalytics_1.0.3.2
[5] blotter_0.8 FinancialInstrument_0.4 quantmod_0.3-15 TTR_0.20-3
[9] Defaults_1.1-1 xts_0.7-6.17 zoo_1.6-4

loaded via a namespace (and not attached):
[1] grid_2.12.1 lattice_0.19-13 tools_2.12.1

Followups:

Date: 2011-08-19 13:41
Sender: Brian Peterson
Resolved. Fixed.getPosQty returns a numeric, which may be zero.

[R-Forge #1146] MOO/MOC ordertypes?

Submitted by: Mark Breman
Assigned to: Brian Peterson
R-Forge link

Hi,

I can't find a way to specify in my strategy that orders should be executed at the Open or at the Close.

The ruleSignal() function has these ordertypes (according to the docs): ordertype: one of 'market','limit','stoplimit', or 'stoptrailing'.

Would it be possible to add Market On Open (MOO) and Market On Close (MOC) ordertypes?

Thanks,

-Mark-

Followups:

Date: 2010-10-26 19:00
Sender: Brian Peterson
traceback is:> applyStrategy(strategy=strat, portfolios=portfolio)Error in [.xts(x, , loc) : 'i' or 'j' out of rangeIn addition: Warning message:In applyIndicators(strategy = strategy, mktdata = mktdata, parameters = parameters, : some arguments stored for myindicator do not match> traceback()7: .Call('_do_subset_xts', x, as.integer(i), as.integer(j), drop, PACKAGE = 'xts')6: [.xts(x, , loc)5: x[, loc]4: getPrice(last(mktdata[txntime]), prefer = 'close')3: ruleOrderProc(portfolio = portfolio, symbol = symbol, mktdata = mktdata, timespan = timespan, ...)2: applyRules(portfolio = portfolio, symbol = symbol, strategy = strategy, mktdata = mktdata, Dates = NULL, indicators = sret$indicators, signals = sret$signals, parameters = parameters, ..., path.dep = FALSE)1: applyStrategy(strategy = strat, portfolios = portfolio)definitely a bug, I'll try to stomp it. - Brian

Date: 2010-10-26 18:15
Sender: Brian Peterson
I've replicated your error, I'll try to sort it out.Thanks, - Brian

Date: 2010-10-26 14:09
Sender: Mark Breman
Hi Brian,As a follow-up, and after re-reading your last explanation I'm beginning to doubt if my modification to orders.R is correct. Is it?Anyway, if I run my testscript without the modification I get an error on applyStrategy():....> strat <- add.rule(strat, name='ruleSignal', arguments = list(sigcol='ind.gt.50', sigval=TRUE, orderqty=1, ordertype='market', orderside='long', pricemethod='market', prefer='Open', delay=86400), type='enter', path.dep=TRUE)> > > > out<-try(applyStrategy(strategy=strat, portfolios=portfolio))Error in [.xts(x, , loc) : 'i' or 'j' out of rangeIn addition: Warning message:In applyIndicators(strategy = strategy, mktdata = mktdata, parameters = parameters, : some arguments stored for myindicator do not match> ...Thought to let you know just in case.-Mark-

Date: 2010-10-26 13:58
Sender: Mark Breman
Hi Brian,Ok I made a simplified testscript to reproduce the transactions on non-existing price periods.Before running it you should modify ruleOrderProc() in orders.R as I did to get the price from the order-book instead of always the close price from the pricedata o execution of the order:replace line 423 in orders.R:txnprice=as.numeric(getPrice(last(mktdata[txntime]), prefer='close')) withtxnprice = as.numeric(ordersubset[ii,]$Order.Price) This replacement is correct right?here is the script to run:####################### start script #####################library(blotter)library(quantstrat)try(rm('account.myacct','portfolio.myportf',pos=.blotter),silent=TRUE)try(rm('MYSTOCK','USD',pos=.instrument),silent=TRUE)try(rm('order_book.myportf',pos=.strategy),silent=TRUE)Sys.setenv(TZ='GMT')open = c(23.0, 23.01, 23.02, 23.04, 23.04, 23.05)high = c(23.10, 23.11, 23.12, 23.13, 23.14, 23.15)low = c(22.0, 22.01, 22.02, 22.03, 22.04, 22.05)close = c(22.5, 22.51, 22.52, 22.53, 22.54, 22.55)dates = as.POSIXct(strptime(c('2010-01-01', '2010-01-02', '2010-01-05', '2010-01-06', '2010-01-07', '2010-01-08'), '%Y-%m-%d'))MYSTOCK = xts(data.frame(Open=open, High=high, Low=low, Close=close), dates)myindicator <-function(mktdata) { s = xts(c(F,T,F,F,F,F), dates) colnames(s) = 'myindicator' return(s)}initDate='2009-12-31'currency('USD')stock(primary_id='MYSTOCK', currency='USD', multiplier=1)portfolio='myportf'account='myacct'verbose=TRUEinitPortf(name=portfolio, symbols='MYSTOCK', initDate=initDate)initAcct(name=account, portfolios=portfolio, initDate=initDate, initEq=100)initOrders(portfolio=portfolio, initDate=initDate)strat <- strategy('mystrat')strat <- add.indicator(strategy = strat, name = 'myindicator', arguments = list(x = quote(mktdata)), label='myindicator')strat <- add.signal(strat, name='sigThreshold',arguments = list(threshold=0, column='myindicator' ,relationship='gt', cross=TRUE), label='ind.gt.50')strat <- add.rule(strat, name='ruleSignal', arguments = list(sigcol='ind.gt.50', sigval=TRUE, orderqty=1, ordertype='market', orderside='long', pricemethod='market', prefer='Open', delay=86400), type='enter', path.dep=TRUE)out<-try(applyStrategy(strategy=strat, portfolios=portfolio))#print(getOrderBook(portfolio))updatePortf(Portfolio=portfolio,Dates=paste('::',as.Date(Sys.time()),sep=''))p = getPortfolio(Portfolio=portfolio)p$symbols$MYSTOCK$txnp$symbols$MYSTOCK$posPL################################ end script ###################################The output I get from the last lines is:> p$symbols$MYSTOCK$txn Txn.Qty Txn.Price Txn.Value Txn.Avg.Cost Pos.Qty Pos.Avg.Cost Gross.Txn.Realized.PL Txn.Fees Net.Txn.Realized.PL Con.Mult2009-12-31 0 0.00 0.00 0.00 0 0.00 0 0 0 02010-01-03 1 23.01 23.01 23.01 1 23.01 0 0 0 1> > p$symbols$MYSTOCK$posPL Pos.Qty Con.Mult Ccy.Mult Pos.Value Pos.Avg.Cost Txn.Value Period.Realized.PL Period.Unrealized.PL Gross.Trading.PL Txn.Fees2009-12-31 0 1 1 0.00 0.00 0.00 0 0.00 0.00 02010-01-01 0 1 1 0.00 0.00 0.00 0 0.00 0.00 02010-01-02 0 1 1 0.00 0.00 0.00 0 0.00 0.00 02010-01-03 1 1 1 22.51 23.01 23.01 0 -0.50 -0.50 02010-01-05 1 1 1 22.52 0.00 0.00 0 0.01 0.01 02010-01-06 1 1 1 22.53 0.00 0.00 0 0.01 0.01 02010-01-07 1 1 1 22.54 0.00 0.00 0 0.01 0.01 02010-01-08 1 1 1 22.55 0.00 0.00 0 0.01 0.01 0 Net.Trading.PL2009-12-31 0.002010-01-01 0.002010-01-02 0.002010-01-03 -0.502010-01-05 0.012010-01-06 0.012010-01-07 0.012010-01-08 0.01> As you can see the transaction takes place on 2010-01-03, which is not in the price data.I hope this clarifies the issue.-Mark-

Date: 2010-10-26 12:18
Sender: Brian Peterson
Mark, I don't think that specifying the next day's price will take any other code changes. You will specify delay=86400'delay' has been modeled in quantstrat orders since inception. I think it should do what you want. In high frequency data, you are correct that delay models 'processing and transmission latency', but I think that it serves to model what you want as well.Let me try to provide a little more information. A market order with delay will be entered as a 'market' order at the price that is present when the order is entered. This provides information that may be used to model such things as slippage and model accuracy, questions like 'How good are we at detecting turning points?'If you add a one-day delay, and there is no price data (as for weekends and holidays), it will still execute on Monday's open, for example, since that will be the next bar, and you will have an open order. I think adding more options obfuscates the usage more than utilizing already provided options, no?Could you provide an example of 2) ? Execution should happen on the next bar, so this would be a bug. We have seen in illiquid markets with things like gaps that limit orders may be executed at prices 'in the gap', but this is correct, you limit order would have to be passed through for a market to reach a new level. I haven't seen transaction executed in time periods where there are no bars.I'm also a little concerned with options that are explicitly forward-looking. At least in the things I do, almost all orders are sent to the market as limit orders, not market orders, to prevent slippage. Forward looking options break the explicit 'only information available at the time of the decision' nature that quantstrat tries to achieve, and also mess with the ability to enter orders at decision time, potentially in a live trading environment.I am concerned about the possible bug described by your point 2), so let me know please, and we'll see if we can sort it out.Regards, - Brian

Date: 2010-10-26 07:11
Sender: Mark Breman
Hi Brian,Thanks for fixing the prefer parameter, I have verified that it's now available in ruleSignal() (in traderules.R).I have also verified that if I replace:txnprice=as.numeric(getPrice(last(mktdata[txntime]), prefer='close'))with txnprice = as.numeric(ordersubset[ii,]$Order.Price) in ruleOrderProc() (orders.R), that it will indeed use the price from the order-book, as expected.Now there is one thing left to fix: How to specify in add.rule() that you want the next day's Open or Close instead of todays.You suggested earlier to use the delay parameter for this (using a one day delay in seconds).I had a good look in the code and did some testing, but I'm afraid this might not be a good idea because:1) it looks like the delay parameter is meant to model the latency of an order traveling over the network (the time it takes for it to get on the order-book after submitting), and not the preferred period of execution. Mixing these two in the same parameter will obfuscate the code I think... 2) adding one day (in seconds) as a delay will only work if the periodicity in the price data is consecutive, but most OHLC data series have gaps (only trading days are present). Testing this gave me some funny results like execution of orders on none trading days (days not present in the price series) etc. I think there is no easy way to fix this.I would suggest three other options for implementing this (in order of my preference I think):1) add another parameter 'period.prefer'. This parameter could hold the values: 'current' and 'next' ('current' would be default). In ruleSignal() (traderules.R) and/or addOrder() (orders.R) we could than test for the 'next' value, and if found, we could select the period and price from the next row in mktdata and add these to the order-book. For instance we could change ruleSignal() (line 74 in traderules.R) from: orderprice <- try(getPrice(x=data, prefer=prefer))[as.character(timestamp)]to if(period.prefer == 'next') orderprice <- try(last(first(getPrice(x=data, prefer=prefer))[paste(as.character(timestamp), '::', sep='')], 2)) else orderprice <- try(getPrice(x=data, prefer=prefer))[as.character(timestamp)]2) allow the prefer parameter to hold two new values: 'next.open' and 'next.close', and test for these in ruleSignal() and/or addOrder()...If they are found the value should also be restored to 'open' or 'close' as it's directly used by getPrice().3) We could also use option 2 and change getPrice() to also accept the new values and return the correct period data. In this case the prefer parameter does not have to be restored.Please let me know what you think.-Mark-

Date: 2010-10-25 14:44
Sender: Brian Peterson
Found a bug, 'prefer' still wasn't always being passed. We've explicitly added prefer as an argument to ruleSignal now. r431 always passes it, so nested matching shouldn't be a problem anymore.Regards, - Brian

Date: 2010-10-25 12:56
Sender: Brian Peterson
SVN r430 contains argument handling for 'prefer' explicitly lower down in the chain. I also updated documentation so a few things should be a little easier to sort through.r429 also contains performance enhancements from Josh Ulrich, potentially 40% or more.Looking forward to the results of your testing, then we'll close this request.Regards, - Brian

Date: 2010-10-25 11:27
Sender: Brian Peterson
Mark, thanks.I'm looking at this now, and hopefully will have it passing through the 'prefer' argument soon. - Brian

Date: 2010-10-21 07:33
Sender: Mark Breman
Ok I had a good look at the code after Brian's remark. My conclusions:I think the way to go here would be the 'prefer' parameter to specify 'Open' or 'Close' (the latter is the default) or some other price column which is available in the price data. There is no need for adding an other order type, as they are both 'market' orders.So the add.rule call in the strategy definition would then look like:...strat <- add.rule(strat, name='ruleSignal', arguments = list(sigcol='DVI.gt.50', sigval=TRUE, orderqty=1, ordertype='market', orderside='long', pricemethod='market', prefer='Open', delay=.0001), type='enter', path.dep=TRUE)...This 'prefer' parameter is already handled in the code but it's somewhere lost in all the formal parameter juggling that's taking place. I have verified that it's not set/available in ruleSignal() (in tradeRules.R), while I think it should be because here the order is added to the orderbook (with the actual price to be used). I can't find where the parameter is lost though...ruleOrderProc() (in orders.R) executes the orders. I think there is a mistake in the function when it's handling a market order on a low-frequency time scale (daily, monthly etc): it's not getting the price for the order from the orderbook (as for the other order types), but directly from the mktdata, and always with the close price:...txnprice=as.numeric(getPrice(last(mktdata[txntime]), prefer='close'))...I think this line should be something like:...txnprice = as.numeric(ordersubset[ii,]$Order.Price) ... Please let me know if my findings are correct.-Mark-

Date: 2010-10-14 15:21
Sender: Brian Peterson
On daily OHLC data, the current 'market' order is a MOC order if you don't have any delay set in the order entry. This presumes either that you would have your signal before the close, or that you have access to the intraday cross through your broker.I work in intraday/tick/BBO data at the office, so this isn't a high priority for me personally, but we would certainly entertain/apply a patch to ruleOrderProc for daily OHLC data that wqould support MOC/MOO orders. For MOO orders, you might want to add a 1-day delay (86400) from the current timestamp, to force it to the next bar. If you look at ruleOrderProc in the traderules.R file, you'll see the switch statement that I'm talking about.Regards, - Brian

[R-Forge #1125] add capital/equity-aware order sizing function

Submitted by: Brian Peterson
Assigned to: Joshua Ulrich
R-Forge link

quantstrat currently has two default order sizing functions:

osNoOp (no operation, tests for errors only, otherwise returns the quantity requested)

osMaxPos (allows leveling into positions up to some maximum position)

We should add a function of functions to do order sizing based on account equity.

At it's simplest, I think that this would do sizing based on either initEQ (stored as an attr in the Account), or on the current endEq (if it has been updated by a call to updateAcct)

I suppose several other options could make sense, like handling weights to apply. This could then be used for either true weight-based allocation (think portfolio optimization ala PortfolioAnalytics or LSPM), or for volatility weighting (per the original Turtles strategy, or similar).

Cheers,

  • Brian

[R-Forge #5776] apply.paramset() ignores paramset with a single parameter

Submitted by: Doug Service
Assigned to: Joshua Ulrich
R-Forge link

A paramset with a single parameter passed to apply.paramset() will run the optimization the requested number of times using the default value each time. I traced through the quantstrat code enough to see that if nsamples >0 then select.samples() strips off the parameter name. Otherwise it seems to get stripped off by the iterator before it gets to install.param.combo().

I got around this issue by adding a second parameter with a single value to the paramset. In the normal case of parameter optimization, optimizing a single parameter would most likely not be important as multiple parameters tend to be optimized together as in the samples. However, for new users to quantstrat, it is the most obvious case to start with. First get one parameter optimization working, then add more.

The Faber demo does not do parameter optimization and the luxor demo starts with the two moving averages. I will try to put a stripped down example together, but I wanted to record this while it is fresh in my mind.

Followups:

Date: 2015-01-13 23:04
Sender: Joshua Ulrich
Fixed (again) in r1669. This time for 'chain' components.

Date: 2014-09-06 20:39
Sender: Joshua Ulrich
This iterators buglet has been worked-around in r1630.

Date: 2014-06-26 01:37
Sender: Doug Service
When nsamples=0, it appears to be a problem with iterators package and I sent email with the issue to package maintainer.There appears to be a problem with the iterators package as the following sample program shows. When iterating across rows of a data.frame, the iterator loses the name of the columns when there is only one column. However, it retains the name of the columns when there are two or more columns in the data.frame. > require(iterators)> (param.combos <- data.frame(SMA=2:4)) SMA1 22 33 4> param.combo=iter(param.combos,by='row')> nextElem(param.combo)[1] 2> nextElem(param.combo)[1] 3> (param.combos <- data.frame(SMA=2:4,SMA1=rep(1,3))) SMA SMA11 2 12 3 13 4 1> param.combo=iter(param.combos,by='row')> nextElem(param.combo) SMA SMA11 2 1> nextElem(param.combo) SMA SMA12 3 1This is currently causing an issue in the quantstrat package which uses the iterators package to optimize algorithm parameters in parallel.

Date: 2014-06-25 07:20
Sender: Doug Service
I have added a minimal test case that showsthe problem. It creates a strategy with asingle moving average, creates a paramset witha single parameter, then calls apply.paramset().You can see in the output below that the parametername ends up as r$param.combo and the default valuefor the n parameter to SMA is used for each iteration.I then clear the environment, call exactly the same codeas in the first case, and based on a booleanvalue, add a second moving average to the strategy,add the moving average to the parameter setas a constant, and call apply.paramset(). The second printout below shows that now the proper sequence of optimizationsoccur. The only difference between the two versions is theconditional addition of the second moving average and secondto call to add.distribution(). r$param.combo Net.Trading.PL Avg.Trade.PLGS 2 4270 1.509434GS1 3 4270 1.509434GS2 4 4270 1.509434GS3 5 4270 1.509434GS4 6 4270 1.509434GS5 7 4270 1.509434GS6 8 4270 1.509434GS7 9 4270 1.509434GS8 10 4270 1.509434 SMA SMA1 Net.Trading.PL Avg.Trade.PL1 2 20 15155 85.5026462 3 20 9005 71.6788323 4 20 2055 18.1415934 5 20 720 16.5053765 6 20 7630 97.0689666 7 20 6705 105.9154937 8 20 5510 39.8461548 9 20 3810 -7.1698119 10 20 4270 1.509434

[R-Forge #5802] ruleOrderProc Error in freq$scale : $ operator is invalid for atomic vectors

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

I believe this is being caused by applyRules setting freq = 'daily' when indexClass(mktdata) == 'Date'. I believe the update below in applyRules should fix the error.

if (nrow(mktdata) > 1) {
  freq <- periodicity(mktdata)  # run once and pass to ruleOrderProc
} else if (indexClass(mktdata) == 'Date') {
  freq = list()
  freq$scale = 'daily'
}

Followups:

Date: 2014-08-02 13:14
Sender: Joshua Ulrich
Fixed in r1628.

Date: 2014-07-27 08:49
Sender: Joshua Ulrich
Johnathan, do you have an example that triggers this bug? I believe I have a patch, but I'm not sure how to test it.

[R-Forge #5790] apply.paramset() not working with doParallel under Windows

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

Within apply.paramset(), the applyStrategy() function throws an error with the get(symbol) call, as it cannot find the instrument data. The fix appears to be implementing logic similar to that used in applyParameter(). Here the symbol names are passed to the to the foreach function via .export and then assigned to the .GlobalEnv within the foreach call. This changes appears to resolve the issue on Windows but I'm unable to test for other backends.

Followups:

Date: 2014-11-06 19:34
Sender: Jonathan Owen
I've attached a diff file with some proposed fixes.

Date: 2014-09-13 23:30
Sender: Joshua Ulrich
Attempted to fix in r1631.

Date: 2014-09-13 23:00
Sender: Joshua Ulrich
doParallel also does not work under Ubuntu (when using a socket cluster) for the same reason. I'll test this patch.

Date: 2014-07-01 13:21
Sender: Jonathan Owen
I've attached a file that includes my proposed changes for apply.paramset(), which includes a fix for 5787 and 5790.Sorry, this is my first real use of R-Forge. The network I'm using doesn't allow access to svn, so I'm actually using a git version of this project (R-Finance/quantstrat) to view code and test changes.

Date: 2014-07-01 13:11
Sender: Brian Peterson
Jonathan, Thank you for the report. It would be faster for us to fix the problems you're reporting if you attached proposed patches. That allows us to more quickly evaluate and apply the patch. Regards,Brian

walk.forward analysis: where is test results?

I've tried using the walk.forward() function for walk-forward analysis. I get a list reporting the results of the analysis on training sets but never do I see the results for the test set. I've looked in walk.forward.R and testing is done around line 157 of the file, but I don't see it being stored anywhere except maybe in some blotter environment, but when I try to look at the portfolio using the method used in the demo, luxor.8.walk.forward.R, which is to use tradeStats(portfolio.st), I get NULL. (The demo doesn't work either; I get the error message Error in runSum(x, n) : Invalid 'n', in addition to warning messages. This happens every time. I've had this come up and I think it's related to training or testing periods being smaller than a window size for a moving average being tried.)

Am I missing something or does walk.forward() just not save testing results anywhere?

[R-Forge #2658] error in pair trade demo

Submitted by: Robert Schmidt
Assigned to: Nobody
R-Forge link

First, a big thank you for this wonderful work. I'm in over my head now, LOL. I run R --vanilla in a terminal. Then require(quantstrat) and demo(pair_trade). Here is the output when it gets to applyStrategy:

out1<-applyStrategy(strategy=pairStrat, portfolios=portfolio1.st)
Error in if (length(j) == 0 || (length(j) == 1 && j == 0)) { :
missing value where TRUE/FALSE needed
traceback()
16: [.xts(data, , colNums[1])
15: data[, colNums[1]]
14: do.call(opr, list(data[, colNums[1]] + offset1, data[, colNums[2]] +
offset2))
13: sigComparison(label = label, data = data, columns = columns[c(i,
lng)], relationship = relationship, offset1 = offset1, offset2 = offset2)
12: diff(sigComparison(label = label, data = data, columns = columns[c(i,
lng)], relationship = relationship, offset1 = offset1, offset2 = offset2))
11: withCallingHandlers(expr, warning = function(w) invokeRestart('muffleWarning'))
10: suppressWarnings(ret_sig | diff(sigComparison(label = label,
data = data, columns = columns[c(i, lng)], relationship = relationship,
offset1 = offset1, offset2 = offset2)) == 1)
9: function (label, data = mktdata, columns, relationship = c('gt',
'lt', 'eq', 'gte', 'lte'), offset1 = 0, offset2 = 0)
{
ret_sig = FALSE
lng <- length(columns)
for (i in 1:(lng - 1)) {
ret_sig = suppressWarnings(ret_sig | diff(sigComparison(label = label,
data = data, columns = columns[c(i, lng)], relationship = relationship,
offset1 = offset1, offset2 = offset2)) == 1)
}
is.na(ret_sig) <- which(!ret_sig)
colnames(ret_sig) <- label
return(ret_sig)
}(label = 'cross.up', data = mktdata, columns = c('Ratio', 'up'
), relationship = 'lt', offset1 = 0, offset2 = 0)
8: do.call(fun, .formals)
7: applySignals(strategy = strategy, mktdata = mktdata, sret$indicators,
parameters = parameters, ...)
6: applyStrategy(strategy = pairStrat, portfolios = portfolio1.st) at pair_trade.R#189
5: eval(expr, envir, enclos)
4: eval(ei, envir)
3: withVisible(eval(ei, envir))
2: source(available, echo = echo, max.deparse.length = Inf, keep.source = TRUE)
1: demo(pair_trade)
sessionInfo()
R version 2.15.3 (2013-03-01)
Platform: x86_64-pc-linux-gnu (64-bit)

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=C LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] quantstrat_0.7.7 foreach_1.4.0 blotter_0.8.13
[4] FinancialInstrument_1.1 quantmod_0.4-0 Defaults_1.1-1
[7] TTR_0.22-0 xts_0.9-3 zoo_1.7-9

loaded via a namespace (and not attached):
[1] codetools_0.2-8 grid_2.15.3 iterators_1.0.6 lattice_0.20-13 tools_2.15.3

The column labels for mktdata aren't matching the names in the sigComparison lines. I added label arguments to the add.indicator commands and changed them to match in the 'update' I posted to the message board. I hope this isn't a dumb newbie thing.

Best regards,

Rob

Followups:

Date: 2013-04-02 14:08
Sender: Brian Peterson
It's not you. We changed the way labels worked, as an earlier attempt was a step backwards, not forwards. Hopefully the new way is 'better' than the one that just overwrote the indicator columns entirely.I'll take a look and patch the pair_trade demo, but I'd be lying if I said it was a high priority.If you actually intended to trade pairs, you'd use a spreader. That would mean that in R, for research, you'd construct a single spread series, and backtest on the spread, rather than writing the logic to manual leg into the spread into your backtest. As such, I think that the pair_trade demo shows how to do some interesting things, but isn't a realistic representation of how you would typically research and trade a strategy like this.All that aside, I'll review the column labels and code there and patch, and probably refer to your R-SIG-Finance post.Thanks for the contributions to the community! Keep it up!Regards,Brian

[R-Forge #5773] Documentation is inconsistent on providing return values

Submitted by: Doug Service
Assigned to: Nobody
R-Forge link

The documentation for

add.paramset()
add.constraint()
add.distribution.constraint()

should be updated to be consistent with the documentation for add.signal(), add.indicator() and other quantstrat functions that take a strategy and possibly return one based on the value of the store parameter. For example, the add.signal() documentation has the following return value discussion.

Value

if strategy was the name of a strategy, the name. It it was a strategy, the updated strategy.

[R-Forge #5801] getPrice in applyRules, ruleOrderProc and ruleSignal missing symbol parameter

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

The missing parameter can cause an incorrect order/trade price to be used when mktdata includes more than 1 symbol. The bug can be reproduced by passing a mktdata object including more than 1 symbol to the applyStrategy function. This is not a problem when mktdata=NULL, as the applyStrategy retrieves data for a single symbol using get(symbol).

I believe a fix is to include the symbol=symbol parameter in the getPrice calls.

The function applyRules includes additional price logic that does not use the getPrice function (see posQty and neqQty around line 500 in rules.R). If this logic is still in use, to incorporate filtering by symbol it may be desirable to replace these lines with calls to getPrice, appropriately specifying prefer.

Followups:

Date: 2014-11-06 19:14
Sender: Jonathan Owen
I've added a diff file with proposed updates.

[R-Forge #5990] rule will not be fired if signal is on endpoint of rebalance rule

Submitted by: Offer Markovich
Assigned to: Joshua Ulrich
R-Forge link

When using rebalance rule the mktdata is split to chunks according to the 'rebalance_on' parameter.

if a signal is set on the end of the chunk (endpoint) the rule which is using such signal will not be executed.

Below is:

  1. simple standalone script that demonstrate the problem
  2. The execution output (wrong)
  3. the execution output after applying the attached patch

here we go:

library(quantstrat)
Sys.setenv(TZ='UTC')

symbols=c('SPY')
currency('USD')
stock('SPY',currency='USD')
.from=as.Date('2014-09-01')
.to=Sys.Date()

suppressWarnings(getSymbols(symbols,adjust=T,from=.from,to=.to))
account.st='My Account'
portfolio.st='My Portfolio'
strategy.st='My Strategy'
initEq=100000

initDate=max(start(SPY))
rm.strat(strategy.st)
rm.strat(portfolio.st)
rm.strat(account.st)

initPortf(portfolio.st,symbols='SPY', initDate=initDate, currency='USD')
initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD',initEq=initEq)
initOrders(portfolio.st, initDate=initDate)

strategy(strategy.st, store=TRUE)

SPY$Rebalance=NA
SPY[endpoints(SPY,on='weeks'),'Rebalance']=1
mktdata=SPY

add.indicator(strategy.st, name = 'EMA',arguments = list(x=quote(Cl(SPY)),n = .fast),label='spyFast')
add.indicator(strategy.st, name = 'EMA',arguments = list(x=quote(Cl(SPY)),n = .slow),label='spySlow')

mktdataInd=applyIndicators(strategy.st,mktdata=mktdata)

add.rule(strategy.st, name='ruleSignal',
arguments=list(sigcol='Rebalance' , sigval=1,
orderside='short' ,
ordertype='market', prefer='Open',
orderqty=-11,
#osFUN='posSize',
replace=FALSE
),
type='enter',
label='Short'
)

myrebalance<-function (trade.percent=.02,
...,
longlevels=1,
shortlevels=1,
digits=NULL,
refprice=NULL,
portfolio,
symbol,
timestamp)
{
print(paste(timestamp,'rebelance indicator is ON'))
}

add.rule(strategy.st, 'myrebalance',
arguments=list(rebalance_on='weeks'),
type='rebalance',
label='rebalance')

out<-applyStrategy.rebalancing(getStrategy(strategy.st) , portfolios=portfolio.st)

wrong output:

[1] '2014-09-12 rebelance indicator is ON'
[1] '2014-09-19 rebelance indicator is ON'
[1] '2014-09-26 rebelance indicator is ON'
[1] '2014-10-03 rebelance indicator is ON'
[1] '2014-10-10 rebelance indicator is ON'
[1] '2014-10-17 rebelance indicator is ON'
[1] '2014-10-24 rebelance indicator is ON'
[1] '2014-10-31 rebelance indicator is ON'
[1] '2014-11-07 rebelance indicator is ON'
[1] '2014-11-14 rebelance indicator is ON'
[1] '2014-11-17 rebelance indicator is ON'

correct output after applying the patch:

[1] '2014-09-12 rebelance indicator is ON'
[1] '2014-09-15 00:00:00 SPY -11 @ 198.233376077693'
[1] '2014-09-19 rebelance indicator is ON'
[1] '2014-09-22 00:00:00 SPY -11 @ 200.35'
[1] '2014-09-26 rebelance indicator is ON'
[1] '2014-09-29 00:00:00 SPY -11 @ 196.2'
[1] '2014-10-03 rebelance indicator is ON'
[1] '2014-10-06 00:00:00 SPY -11 @ 197.34'
[1] '2014-10-10 rebelance indicator is ON'
[1] '2014-10-13 00:00:00 SPY -11 @ 190.46'
[1] '2014-10-17 rebelance indicator is ON'
[1] '2014-10-20 00:00:00 SPY -11 @ 188.13'
[1] '2014-10-24 rebelance indicator is ON'
[1] '2014-10-27 00:00:00 SPY -11 @ 195.73'
[1] '2014-10-31 rebelance indicator is ON'
[1] '2014-11-03 00:00:00 SPY -11 @ 201.92'
[1] '2014-11-07 rebelance indicator is ON'
[1] '2014-11-10 00:00:00 SPY -11 @ 203.38'
[1] '2014-11-14 rebelance indicator is ON'
[1] '2014-11-17 00:00:00 SPY -11 @ 203.85'
[1] '2014-11-17 rebelance indicator is ON'

Followups:

Date: 2015-02-01 21:54
Sender: Joshua Ulrich
I disagree with the attached patch. We shouldn't change the primary rule function to correct an issue with the rebalancing functionality. I patched applyStrategy.rebalancing instead.Fixed in revision 1678.

UpdateOrders param default orderset = NULL

I am calling updateOrders with the orderset param left as default which is NULL. Line 524 of orders.R in updateOrders is calling is.na() on the orderset param which is causing an argument is of length zero error.

Specifically in my case, I am calling updateOrders from the ruleRevoke function. If I set orderset = NA then I no longer get the error.

Min Reproducible example is:
add.rule(strategy.st, name="ruleRevoke", arguments=list(sigcol="EOD", sigval=TRUE, portfolio = portfolio.st, symbols = symbols, ruletype = "risk"), type="risk", enabled=TRUE) where "EOD" is a signal column which has a value of TRUE at the End Of Day.

[R-Forge #5028] Caching instrument data to prevent addTxns calling getInstrument each time

Submitted by: Ilya Kipnis
Assigned to: Nobody
R-Forge link

So thanks to help from Josh Ulrich, I was able to get to the bottom of why my relatively simple strategy was taking an obnoxiously long time (or at least one of the reasons), in that addTxns calls getInstrument on each call if ConMult is not passed in as an argument to applyStrategy. Since applyRules already calls getInstrument, I'm hoping there's a way to cache each instrument for the duration of the strategy with only one call to getInstrument for each instrument?

Because for the futures contract I'm testing on, it takes 1.5 seconds for a call to getInstrument to finish execution on it, and when that call is made ~20,000 times over the course of the backtest, that alone takes up more than 8 hours.

Here's the offending chunk of code, in addTxns (near the beginning of the function):

if (is.null(ConMult) | !hasArg(ConMult)) {
tmp_instr <- try(getInstrument(Symbol), silent = TRUE)
if (inherits(tmp_instr, 'try-error') | !is.instrument(tmp_instr)) {
warning(paste('Instrument', Symbol, ' not found, using contract multiplier of 1'))
ConMult <- 1
}
else {
ConMult <- tmp_instr$multiplier
}
}

Calling getInstrument thousands of times IMO is highly excessive, and there is probably a workaround somewhere that reduces this operation to O(k), where k is the number of instruments, rather than O(N), where N is the number of transactions.

[R-Forge #5800] sigTimestamp() cannot find function .firstThreshold

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

I believe .firstThreshold was renamed to .firstCross, so sigTimestamp just needs to be updated to use the new function name.

For a timestamp signal, it may not necessary to re-calculate for each symbol, so I'd propose creating a mechanism to support global signals. Below is a quick-and-dirty attempt at creating a global timestamp signal. For this to work, I also had to update the applySignals function to handle a NULL return value (see attached; most of the logic was already in place).

sigTimestampGlbl = function(label, data=mktdata, ...) {
if(label %in% colnames(data))
return(NULL)
else
sigTimestamp(label=label, data=data, ...)
}

Followups:

Date: 2014-09-13 23:33
Sender: Joshua Ulrich
I'm closing this as fixed. Jonathan, could you please add the global signals mechanism as a feature request?

Date: 2014-07-11 15:51
Sender: Joshua Ulrich
Rename fixed in r1623. Global signal mechanism still open.

[R-Forge #5834] unexpected order date if symbol index is Date class

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

I'm not sure is this is a bug or shows a need for additional input data checks/documentation. When running the attached script (provided by Samo Pahor), which uses daily symbol data with a Date class index and a exit rule delay=1, there is both an order and transaction dated on 2/11, but the transaction uses the price from 2/10.

Signals are generated on 2/9, so using allowMagicalThinking one would expect to see an enter order and transaction on 2/9 and, due to the delay=1, an exit order and transaction on 2/10. It appears that the price logic is working as expected, but the order date (and transaction date) is not. This may have something to do with how the delay is handled in addOrder (lines 386-7) in orders.R--instead of is.timeBased, should the validation be that timestamp is POSIXct? The discrepancy is gone if the index on the symbol data is changed to be POSIXct (see line 37 of attached script).

I think it would make sense to add some logic to applyStrategy that validates that the index class for the symbol and mktdata is POSIXct, and if not either throw a warning and/or convert. Any thoughts?

[R-Forge #6310] rules.R add.rule has a few lines codes may never be used, but will cause error if it is executed.

Submitted by: Kenny Liao
Assigned to: Joshua Ulrich
R-Forge link

I use the source code at https://github.com/R-Finance/quantstrat/blob/master/R/rules.R

at line 90:

This line may never be executed by any user, as everyone is writing 'ruleSignal' correctly. However, when I tried any name other than 'ruleSignal', line 90 will generate an error 'Error in get(name) : object 'signal' not found', making lines 91-97 useless.

at line 95:

'rule$name' could this be a leftover from other files?

Thanks a lot!

Followups:

Date: 2016-04-04 07:21
Sender: Kenny Liao
Thanks @brian Peterson and @joshua Ulrich. I think you solved the issue. I will use the R-Forge version next time.

Date: 2016-04-03 14:53
Sender: Joshua Ulrich
I wouldn't consider this a bug, but it's 'fixed' in r1745 by making the function search more robust. You're still going to get nonsense results if you pass in a non-function for an indicator, signal, rule, etc.Also note that the source you're looking at on GitHub is not authoritative. It's a read-only mirror, and it's not kept up-to-date with the commits in the subversion repository on R-Forge.

Date: 2016-04-03 12:04
Sender: Joshua Ulrich
The NEWS file says get0() was added in 3.2.0. What makes you say the addition of get0() changed the behavior of get()? I just ran is.function(get('nothing')) and is.function(try(get('nothing'))) in R-3.1.0 and I get the same result as in R-3.2.0.

Date: 2016-04-03 11:52
Sender: Brian Peterson
As a little more context, get0() was added to R somewhere in the 3.x tree. It changed the behavior of get(). I found a reference somewhere that get0 was added in 3.2.0, but that seems way too recent. It still wouldn't hurt to require a recent R for quantstrat, and to look for formulations like this which should be changed to use get0 instead of get.

Date: 2016-04-03 11:47
Sender: Brian Peterson
This is likely just very old code.The addition of get0() to R changed the behavior of get()Those get() statements should be changed to get0. This is probably true several other places in quantstrat as well.> is.function(get('nothing'))Error in get('nothing') : object 'nothing' not found> is.function(try(get('nothing')))Error in get('nothing') : object 'nothing' not found[1] FALSE> is.function(get0('nothing'))[1] FALSE> is.function(get0('new.env'))[1] TRUE

Date: 2016-04-03 11:40
Sender: Brian Peterson
it seems most likely that you are receiving an error because the rule function, described by the 'name' of the rule, that you are looking for does not exist.I'll note, for example, that we do not see errors in rebalancing rules where the rebalancing rule function is not 'ruleSignal'

Date: 2016-04-03 11:23
Sender: Brian Peterson
This code looks fine. It tries first to find the function specified by 'name', and then guesses that maybe the name is actually sig.nameIt is possible we could make it more robust with try().if(!is.function(try(get(name)))){Is that what you're suggesting?

Date: 2016-04-03 08:52
Sender: Kenny Liao
from source code lines 90 to 97 if(!is.function(get(name))){ if(!is.function(get(paste('sig',name,sep='.')))){ message(paste('Skipping rule',name,'because there is no function by that name to call')) next() } else { name<-paste('sig',rule$name,sep='.') } }

Error when running walk.forward over a short testing period

The extended GBPUSD data used in the creation of the Luxor Demo includes (at least) an erroneous date (2002/10/27) with only 1 observation which causes an issue if you run the walk.forward function with a test period of 1 day. It tries to run the signal/indicator calculations with a look back period of "n" over this one observation which raises an error. I can also foresee this being an issue when testing longer signal periods on instruments like Crude where they have only a few trading hours on Sunday evenings (UTC).

I solved this by cleaning the data pre-WFA, ensuring only days with a full amount of observations were used, but this is far from optimal!

Traceback:

8: stop("Invalid 'n'")
7: runSum(x, n)
6: runMean(x, n)
5: (function (x, n = 10, ...) 
   {
       ma <- runMean(x, n)
       if (!is.null(dim(ma))) {
       colnames(ma) <- "SMA"
   }
   return(ma)
   })(x = Cl(mktdata)[, 1], n = 25)
4: do.call(indFun, .formals)
3: applyIndicators(strategy = strategy, mktdata = mktdata, parameters = parameters, 
   ...)
2: applyStrategy(strategy, portfolios = portfolio.st, mktdata = symbol[testing.timespan]) at walk.forward.R#122
1: walk.forward(strategy.st, paramset.label = "WFA", portfolio.st = portfolio.st, 
   account.st = account.st, period = "days", k.training = 3, 
   k.testing = 1, obj.func = my.obj.func, obj.args = list(x = quote(result$apply.paramset)), 
   audit.prefix = "wfa", anchored = FALSE, verbose = TRUE)

Support Order Quantity as an Indicator Column in mktdata

Support numeric or name of indicator column in mktdata for orderqty argument in ruleSignal(). The orderqty argument would then be handled the same as the threshold argument. It is reasonable to support this as strategies may size positions based on an indicator (e.g. volatility, signal strength, etc.).

[R-Forge #5038] Run symbols in parallel

Submitted by: Ilya Kipnis
Assigned to: Nobody
R-Forge link

Simple request here...right now, applyStrategy runs all of the symbols in series. I'd appreciate if there'd be an option to run the symbols in parallel, seeing as to how apply.Paramset runs parameter sets in parallel, but if I know my strategy parameters, I'd like for there to be a way of running the symbols in parallel as well.

Thanks.

Followups:

Date: 2014-11-01 18:04
Sender: Ethan Lin
I've hacked this in my own environment, using doSNOW and foreach. You create the required environments to run each symbol on the 'slaves' and the 'slave' threads need to return their portfolio trades and order books back to the 'master' to be added via addTxns to the master portfolio.

[R-Forge #2727] tradeOrderStats() parameter spellings

Submitted by: Scott Schmidt
Assigned to: Brian Peterson
R-Forge link

It seems there may be a couple of parameter misspellings in the tradeOrderStats() function signature. The existing signature is:
tradeOrderStats <- function(portfolio, symbol, ...) {
stats.table <- perTradeStats(Portfolio=portfolio,Symbol=symbol,...)
stats.xts <- as.xts(stats_table, order.by=stats.table$End)
orderbook <- getOrderBook(portfolio=Portfolio)[[Portfolio]][[symbol]]
closed <- orderbook[which(orderbook$Order.Status=='closed'),]
closed.xts <- xts(closed, as.POSIXct(as.vector(closed$Order.StatusTime), format='%Y-%m-%d %H:%M:%OS'))
merged.table <- merge(closed.xts,stats.xts)
merged.closed <- merged.table[index(stats.xts)]
return(merged.closed)
}

Suggested function signature:
tradeOrderStats <- function(portfolio, symbol, ...) {
stats.table <- perTradeStats(Portfolio=portfolio,Symbol=symbol,...)
stats.xts <- as.xts(stats.table, order.by=stats.table$End)
orderbook <- getOrderBook(portfolio=portfolio)[[portfolio]][[symbol]]
closed <- orderbook[which(orderbook$Order.Status=='closed'),]
closed.xts <- xts(closed, as.POSIXct(as.vector(closed$Order.StatusTime), format='%Y-%m-%d %H:%M:%OS'))
merged.table <- merge(closed.xts,stats.xts)
merged.closed <- merged.table[index(stats.xts)]
return(merged.closed)
}

summary:

  1. change stats_table to stats.table
  2. change getOrderBook(portfolio=Portfolio)[[Portfolio]][[symbol]] to
    getOrderBook(portfolio=portfolio)[[portfolio]][[symbol]]

Followups:

Date: 2013-04-24 16:02
Sender: Brian Peterson
fixed in svn r 1447, credited to you.That's what I get for trying to normalize punctuation and capitalization after writing the function.Thanks,Brian

[R-Forge #5792] apply.paramset() does not pass additional parameters to tradeStats()

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

This may be more of an enhancement than a bug, but currently it's not possible to pass additional parameters to the tradeStats() function called during the combine step of apply.paramset(). Updating the call to be r$tradeStats <- tradeStats(r$portfolio.st, ...) may be a solution.

Followups:

Date: 2014-07-01 20:58
Sender: Jonathan Owen
This file includes the proposed change. A little off topic, but for increased logging, it also includes a suggested call to sink() during the foreach loop if verbose == TRUE.

[R-Forge #1063] bug in quantstrat/R/signals.R, sigComparison

Submitted by: Mstislav Elagin
Assigned to: Brian Peterson
R-Forge link

the following code fails:

d <- matrix(1:4, nr=2); colnames(d) <- c('one', 'two')
sigComparison( 'a.label', data=d, columns=c('one', 'two'), rel='gt')

Error in colnames<-(*tmp*, value = 'a.label') :

attempt to set colnames on object with less than two dimensions

This happens because line 172 in signals.R reads:

colnames(ret_sig)<-label

However, ret_sig is a vector and cannot have colnames. I suggest assigning an attribute instead, e.g.:

attr(ret_sig, 'label') <- label

Followups:

Date: 2010-10-07 17:00
Sender: Brian Peterson
Mstislav, I will try to add a data type check, but I think I can close this for now. Let me know if you think differently.Regards, - Brian

Date: 2010-09-20 12:48
Sender: Mstislav Elagin
Oh, alright. You understood my point quite correctly, just the point was not correct, provided you only intend to support xts. Sorry about the unnecessary ticket. Maybe a check should be added to enforce the type of the data argument to sigComparison.BestMstislav

Date: 2010-09-20 12:33
Sender: Brian Peterson
I don't think this is a valid bug.Quantstrat expects, and the documentation reflects, that the data will be xts time series. xts time series always have columns, so colnames will always work.I don't have any interest in supporting non-xts mktdata in quantstrat.If I misunderstand your point, please let me know, but for now, I think this is invalid.Regards, - Brian

[R-Forge #1531] parameterTest and parameterTestMACD demos do not run (v.

Submitted by: Dan Potter
Assigned to: Nobody
R-Forge link

The parameterTest and parameterTestMACD quantstrat demo programs do not work, see below.

(I would like to give a release number but I do not know how to find it. I downloaded the software yesterday from
ttps://r-forge.r-project.org/snapshots.php?group_id=316
and used Rtools to build for for windows.)

demo(package='quantstrat',topic='parameterTestMACD')

demo(parameterTestMACD)
---- ~~~~~~~~~~~~~~~~~

Type to start :

TODO: Add comment

Author: CCD

require(foreach)
Loading required package: foreach
Loading required package: iterators
Loading required package: codetools
foreach: simple, scalable parallel programming from Revolution Analytics
Use Revolution R for scalability, fault tolerance and more.
http://www.revolutionanalytics.com

require(doSMP)
Loading required package: doSMP
Loading required package: revoIPC

workers <- startWorkers(2)

registerDoSMP(workers)

please run macd demo before all these...

paramStructure<-getParameterTable(stratMACD)
Error in getParameterTable(stratMACD) : object 'stratMACD' not found
demo(package='quantstrat',topic='parameterTest')

demo(parameterTest)
---- ~~~~~~~~~~~~~

Type to start :

TODO: Add comment

Author: Yu Chen

please run bbands demo before all these...

paramStructure<-getParameterTable(stratBBands)

tPD<-setParameterDistribution() need no more for initial object.

Do expand test

tPD<-setParameterDistribution(tPD,'indicator',indexnum=1,distribution=list(sd=(1:3)))

tPD<-setParameterDistribution(tPD,'signal',indexnum=2,distribution=list(sd=sample(1:10, size=1, replace=FALSE)))

tPD<-setParameterDistribution(tPD,'signal',indexnum=3,distribution=list(n=sample(1:10, size=1, replace=FALSE)))

update the 3rd slot by using psindex

tPD<-setParameterDistribution(tPD,'signal',indexnum=2,distribution=list(n=c(20,30)),psindex=3)

testPackList<-applyParameter(strategy=stratBBands,portfolios='bbands',parameterPool=tPD,method='expand')

tPD

debug(applyParameter)

undebug(applyParameter)

Just provide leagal values and use random sampling.

tPD<-setParameterDistribution(tPD,'indicator',indexnum=1,distribution=list(sd=(1:3)),weight=c(.25,.25,.5),label='sd')

tPD<-setParameterDistribution(tPD,'signal',indexnum=2,distribution=list(relationship=c('lt','lte')),label='rel')

tPD<-setParameterDistribution(tPD,'signal',indexnum=2,distribution=list(relationship=c('lte')))

tPD<-setParameterDistribution(tPD,'indicator',indexnum=1,distribution=list(n=20:30),label='n')

pConstr<-setParameterConstraint()

pConstraint<-setParameterConstraint(constraintLabel='PC1',paramList=c('sd','n'),relationship='gt')
[1] 'Parameter constraint object initialized...'

testPackList<-applyParameter(strategy=stratBBands,portfolios='bbands',parameterPool=tPD,method='random',sampleSize=2,parameterConstrains=pConstraint)
Error in paramConstraint(label = parameterConstrains[[k]]$constraintLabel, :
comparison of more than two columns not supported, see sigFormula

Followups:

Date: 2011-08-21 14:02
Sender: Brian Peterson
Dan,Glad things are working for you now.I'll try to clean up those demo scripts so that they manage their own side effects when run multiple times. Also, I need to write some code that checks for and registers an appropriate parallel backend depending on OS and what is available. (This type of stuff is why they aren't in the index of demos for quantstrat yet).Regards, - Brian

Date: 2011-08-21 13:14
Sender: Dan Potter
OK. that works if I start clean. I had done a require(foreach) after the %dopar% failure and then tried re-running parameterTest, and also bbands (and then parameterTest) but there are some side effects from the original failure that keep it from working in those cases.Thank you for your help Brian.-Dan

Date: 2011-08-21 12:24
Sender: Brian Peterson
require(foreach)before running the parameterTest demoforeach provides %dopar%If you want things to run in parallel, you need to also register a parallel backend, such as doMC, doSMP, or doRedis.I'll do a little more work to make that as transparent as possible.Brian

Date: 2011-08-21 10:55
Sender: Dan Potter
Hi Brian,It may be my use of the wrong package for %dopar%. What is the prefered one? In general, how do I determine which SVN revision was used in an installed package? (I want to include the quantstrat revision # in my report.)Below are some error examples.Since I am so new to R, the may be very obvious to a more experienced user.-Dan#This is the setup (re-load and rerun this morning)install.packages('quantstrat', repos='http://R-Forge.R-project.org&quot;)demo(package='quantstrat', topic='bbands')demo(package='quantstrat', topic='parameterTest')#ParameterTest errors out with the following> testPackList<-applyParameter(strategy=stratBBands,portfolios='bbands',parameterPool=tPD,method='random',sampleSize=2,parameterConstrains=pConstraint)[1] 'ParamTable generated'[1] 2Error in applyParameter(strategy = stratBBands, portfolios = 'bbands', : could not find function '%dopar%'# So %dopar% is missing - there is one in the foreach pacakage require(foreach)demo(package='quantstrat', topic='parameterTest')#ParameterTest now errors out with> testPackList<-applyParameter(strategy=stratBBands,portfolios='bbands',parameterPool=tPD,method='random',sampleSize=2,parameterConstrains=pConstraint)Error in paramConstraint(label = parameterConstrains[[k]]$constraintLabel, : comparison of more than two columns not supported, see sigFormua

Date: 2011-08-21 01:09
Sender: Brian Peterson
So, what error do you get? If we can narrow in on the exact problem, maybe we can fix it.The bbands parameterTest demo works for me.Regards, - Brian

Date: 2011-08-19 14:10
Sender: Dan Potter
First the bad news, parameterTest.R does not run even if I run bbands beforehand (per the comment) parameterTest.R:#please run bbands demo before all these...Now the good news, I missed that comment in parameterTestMACD.R (thank you for pointing it out). It DOES run forme if run the macd demo first.

Date: 2011-08-19 13:36
Sender: Brian Peterson
I suppose I should just update these in svn, but as noted in the comments for the two parameter test demos, they assume that you have previously run the associated BBands or MACD demo first:> #please run macd demo before all these...I'll update svn, but this should solve your problems.Regards, - Brian

[R-Forge #6306] Quantstart `osMaxPos` source code typo errors?

Submitted by: Kenny Liao
Assigned to: Joshua Ulrich
R-Forge link

I looked into osMaxPos just trying to understand quantstrat a little better, however, there are two lines of codes strikes me as typo errors. Or, maybe I missed something important.

I pasted the source code below, and commented on the two lines where I think are typo errors. Could anyone help me have a look? Thanks a lot!

inside osFUNs.R file of Quantstrat of Version: 0.9.1709 | Last change: 2016-03-11 17:31:38+01 | Rev.: 1734

First typo error I suspected is at line 212:

        if(orderqty+pos>PosLimit[,'MaxPos']) orderqty <- PosLimit[,'MinPos']-pos

the 'MaxPos' does not make sense to me, I wonder it should be 'MinPos' instead?

Second typo error I suspected is at line 226:

        orderqty<-pos #flatten position, don't cross through zero

as we are to 'buy cover short', pos should be negative, and orderqty should be positive, therefore, orderqty should be -pos, not pos.

To find the source with my comments to mark those two lines above, click this link http://stackoverflow.com/questions/36097455/quantstart-osmaxpos-source-code-typo-errors

Followups:

Date: 2016-03-28 03:29
Sender: Joshua Ulrich
osMasPos was refactored in r1738. I couldn't trigger the possible bug caused by the second typo. I'm closing this issue, since unit tests now provide decent coverage that the function is performing correctly.Please report any additional problems in a new issue.

Date: 2016-03-27 00:52
Sender: Kenny Liao
Thank you @brian Peterson and @joshua Ulrich for your kind and helpful replies and advices. I will learn to look into unit tests.

Date: 2016-03-25 20:44
Sender: Joshua Ulrich
The first issue is fixed in r1736. I couldn't seem to trigger any bug caused by the second issue, but I probably couldn't get the right combination of position and order to trigger it. Now that there are several unit tests, maybe you can modify them to trigger a bug caused by the second typo.I suspect the possibility that there's no issue because that code is never exercised. I have in mind a few refactoring steps for osMaxPos that will help make it easier to reason about. That should make it clear whether the code is buggy, or whether it's not even necessary.

Date: 2016-03-25 16:19
Sender: Joshua Ulrich
Thanks for the tests, though it would be better if you test the actual code, rather than some 'simplified' version. I tried to translate your tests into some that test the actual code, and I cannot replicate scenario 3 because the offending code isn't reached if ruletype='risk'. That said, I get -4000 (as in your tests) if ruletype=''.

Date: 2016-03-25 11:33
Sender: Brian Peterson
I'm pretty sure your patch is correct. We should develop a test for the other one as well, if possible, but I'll try to get this committed soon. Thanks!

Date: 2016-03-25 08:47
Sender: Kenny Liao
I have add a simple code example to testify the existence of the second typo error at http://stackoverflow.com/questions/36097455/quantstart-osmaxpos-source-code-second-typo-error-is-testified could anyone help me have a look? Thanks a lot

[R-Forge #5978] add.rule and ruleProc search for function names beginning with 'sig' instead of 'rule'

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

I believe these functions should be searching for functions that begin with 'rule' instead of 'sig'. See lines 91, 95, 652 and 656 of rules.R. Proposed fix attached.

Followups:

Date: 2014-11-06 20:05
Sender: Joshua Ulrich
Thanks for the patch file. Fixed in r1645.

Date: 2014-11-06 19:10
Sender: Jonathan Owen
Let's try this again. I didn't realize you can export just the diff.

Date: 2014-11-06 18:58
Sender: Joshua Ulrich
Thanks for the report. Please confirm that the only thing that needs to be done is change 'sig' to 'rule' on those 4 lines.In the future, can you please just attach the diff, so it's easier to see what's changed? The files you have attached to your reports often have different line endings and spacing, which makes comparing them a bit more difficult.

[R-Forge #5101] sigCol exists

Submitted by: Evelyn Mitchell
Assigned to: Joshua Ulrich
R-Forge link

if sigCol doesn't exist, or more likely, is named incorrectly, the error message thrown by applyStrategy is uninformative:

Error in .xts(e, .index(e1), .indexCLASS = indexClass(e1), .indexFORMAT = indexFormat(e1), :

index length must match number of observation

---- Replicate error
require(quantstrat)
require(quantmod)
require(PerformanceAnalytics)
suppressWarnings(try(rm(list=ls()),silent=TRUE))
currency('USD')
stock('SPY', currency = 'USD', multiplier = 1)

ls(envir=FinancialInstrument:::.instrument)

initDate <- '2009-10-01'
startDate <- '2009-11-10' # after the SMAs start
endDate <- '2013-10-31'
initEq <- 500000

Sys.setenv(TZ='UTC')

getSymbols('SPY', from = startDate, to=endDate, adjust=T)
summary(SPY)
chartSeries(SPY)
chartSeries(SPY, theme=chartTheme('white'))
addSMA(7)
addSMA(15)
SPY$FastSMA <- SMA(Cl(SPY), 7)
SPY$SlowSMA <- SMA(Cl(SPY), 15)
head(SPY)
rm.strat(qs.strategy)

qs.strategy <- 'qsCartwheel'

initPortf(qs.strategy, 'SPY', initDate=initDate)
initAcct(qs.strategy, portfolios=qs.strategy, initDate=initDate, initEq=initEq)

initOrders(portfolio=qs.strategy, initDate=initDate)

strategy(qs.strategy, store=T)

strat <- getStrategy(qs.strategy)
summary(strat)

add.indicator(strategy=qs.strategy,
name = 'SMA',
arguments = list(x=quote(Cl(SPY)), n=15),
parameters = NULL,
label = 'SlowSMA',
enabled = T,
store = T)

add.indicator(strategy=qs.strategy,
name = 'SMA',
arguments = list(x=quote(Cl(SPY)), n=7),
parameters = NULL,
label = 'FastSMA',
enabled = T,
store = T)

str(getStrategy(qs.strategy)$indicators) # works

add.signal(qs.strategy, name='sigCrossover',
arguments = list(columns=c('SlowSMA', 'FastSMA'),
relationship='gt'),
label='Slow.gt.Fast')

add.signal(qs.strategy, name='sigCrossover',
arguments = list(columns=c('SlowSMA', 'FastSMA'),
relationship='lt'),
label='Slow.lt.Fast')

str(getStrategy(qs.strategy)$signals) # works

add.rule(qs.strategy,
name='ruleSignal',
arguments=list(sigcol = 'SlowSMA.lt.FastSMA',
sigval = T,
orderqty = 'all',
ordertype = 'market',
orderside = 'long',
pricemethod = 'market',
replace = F),
parameters = NULL,
type = 'enter',
parent = NULL,
enabled = T,
indexnum = NULL,
path.dep = T,
timespan = NULL,
store = False,
storefun = T)
add.rule(qs.strategy,
name='ruleSignal',
arguments=list(sigcol = 'SlowSMA.gt.FastSMA',
sigval = T,
orderqty = 'all',
ordertype = 'market',
orderside = 'long',
pricemethod = 'market',
replace = F),
parameters = NULL,
type = 'exit',
parent = NULL,
enabled = T,
indexnum = NULL,
path.dep = T,
timespan = NULL,
store = False,
storefun = T)
str(getStrategy(qs.strategy)$rules)
str(getStrategy(qs.strategy))
summary(getStrategy(qs.strategy))
out <- applyStrategy(strategy=qs.strategy, portfolios=qs.strategy)

errors with

> out <- applyStrategy(strategy=qs.strategy, portfolios=qs.strategy)

Error in .xts(e, .index(e1), .indexCLASS = indexClass(e1), .indexFORMAT = indexFormat(e1), :

index length must match number of observations

I've double checked that initDate is prior to startDate, and that startDate is after all SMA columns have values

class(out)
names(out)
names(out$qsCartwheel)
names(out$qsCartwheel$SPY)

---- Fix

add.rule(qs.strategy,
name='ruleSignal',
arguments=list(sigcol = 'Slow.lt.Fast',
sigval = T,
orderqty = 'all',
ordertype = 'market',
orderside = 'long',
pricemethod = 'market',
replace = F),
parameters = NULL,
type = 'enter',
parent = NULL,
enabled = T,
indexnum = NULL,
path.dep = T,
timespan = NULL,
store = False,
storefun = T)
add.rule(qs.strategy,
name='ruleSignal',
arguments=list(sigcol = 'Slow.gt.Fast',
sigval = T,
orderqty = 'all',
ordertype = 'market',
orderside = 'long',
pricemethod = 'market',
replace = F),
parameters = NULL,
type = 'exit',
parent = NULL,
enabled = T,
indexnum = NULL,
path.dep = T,
timespan = NULL,
store = False,
storefun = T)

Followups:

Date: 2015-11-14 15:57
Sender: Joshua Ulrich
Fixed in r1714.

[R-Forge #5888] getOrders timespan ignored when which.i == TRUE causing updateOrders to also ignore timespan

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

Since getOrders ignores the timespan parameter when which.i = TRUE, and the updateOrders function sets which.i = TRUE, the timespan parameter passed to updateOrders is also being ignored.

The function ruleRevoke also calls updateOrders and may therefore also be impacted by this behavior (in this function a timespan parameter is used but the parameter doesn't appear to be passed to the function).

[R-Forge #5787] audit not working for apply.paramset() when single distribution specified

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

When a single distribution is specified for apply.paramset() and an environment is passed to audit, the environment appears to be overwritten for each iteration.

I believe this is caused by the iter method used in the foreach call converting the param.combos data frame to a numeric vector (param.combo) and the rownames of this vector being used to name the portfolio object stored in the audit environment. One possible fix might be to check for is.null(rownames(param.combo)), and if TRUE, use as.character(param.combo) to name the portfolio object.

Followups:

Date: 2014-09-13 22:58
Sender: Joshua Ulrich
Closing as a duplicate of #5776, which is fixed as of r1630.

Date: 2014-07-01 13:28
Sender: Jonathan Owen
Related to 5790--a proposed fix was included with that report. This might be related to 5776.

Date: 2014-06-30 15:25
Sender: Jonathan Owen
One more item related to apply.paramset(). The combine function doesn't currently handle results where 1 or more runs generates an error. One solution might be to wrap the combine function's for loop logic with an error check like: if !is.null(r$message)) { print(r) } else { ... }.

Date: 2014-06-30 13:53
Sender: Jonathan Owen
The install.param.combo() function also relies on the rownames of the para.combo variable, so the fix here may be to check if param.combo is a data.frame at the start of each foreach loop, and if not, convert to a data.frame with a specified row name and column name. Based on my limited testing, this seems to resolve the issue.

[R-Forge #5930] Create global signals mechanism

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

Migrating from #5800.

For a timestamp signal, it may not necessary to re-calculate for each symbol, so I'd propose creating a mechanism to support global signals. Below is a quick-and-dirty attempt at creating a global timestamp signal. For this to work, I also had to update the applySignals function to handle a NULL return value (see attached; most of the logic was already in place).

sigTimestampGlbl = function(label, data=mktdata, ...) {
if(label %in% colnames(data))
return(NULL)
else
sigTimestamp(label=label, data=data, ...)
}

[R-Forge #5895] Stoplimit order execution with OHLC price gaps

Submitted by: Claymore Marshall
Assigned to: Joshua Ulrich
R-Forge link

In ruleOrderProc.R, I think the logic for the stoplimit orders being executed is not quite right.

There is a piece of code which handles price gapping on stoplimit orders. In the case of long stoplimit order and !isBBOmktdata (i.e. using OHLC data), the existing code is this :

if ((orderQty > 0 && orderType != 'stoplimit') || (orderQty < 0 && (orderType == 'stoplimit'))) {
if ((has.Lo(mktdata) && orderPrice > as.numeric(Lo(mktdataTimestamp))) ||
(!has.Lo(mktdata) && orderPrice > as.numeric(getPrice(mktdataTimestamp,
prefer = prefer)[, 1]))) {
txnprice = min(orderPrice, Hi(mktdataTimestamp)[, 1])
txntime = timestamp
} else (next)()
} else ...

The txnprice is the minimum of the orderPrice and the high of the next bar, so if the price drops sharply from one bar to the next (such as from Friday afternoon to market open the next week), crossing through the stoplimit price for the first time, this code says exit at the high price. I think when the stoplimit price is breached due to gapping, the exit should be at the open price of the gapping bar (plus some slippage, but ignore that here), as that is the first time the trade can be exited after the price gap (And a stoplimit is a hard stop which should be exited at all costs asap once the orderPrice has been crossed from above). Exiting at the high price on the gapping bar doesn't make sense to me, because how can one know to exit at the high price, when the high price is only known after the fact once the gapping bar is complete?

The fix is simple, change the txnprice to

txnprice = min(orderPrice, Op(mktdataTimestamp)[, 1])

A similar change is also necessary for the short stoplimit order too (a few lines later in ruleOrderProc). The existing code for the short stoplimit sets

txnprice = max(orderPrice, Lo(mktdataTimestamp)[, 1])

It would become txnprice = max(orderPrice, Op(mktdataTimestamp)[, 1])

These suggested changes are in the attached patch.

Followups:

Date: 2014-09-18 21:17
Sender: Joshua Ulrich
Fixed in r1633.

Date: 2014-09-07 11:19
Sender: Brian Peterson
A resting limit will always execute at the limit. There are cases where a resting limit can execute at a price better than the limit, due to implied fills on some exchanges, but the resting limit will never execute at a price worse than the limit. Most gaps in time series bar data are due to data that is not reflective of the true or complete market. Most equities, for example, trade effectively 24 hours per day, so a limit order would execute 'in the gap'. Even over weekend close/open, the limit would be matched on the open.An iceberg order is a resting limit with a shown quantity and a reload. An exchange iceberg will execute like a limit order. In quantstrat, we model the possibility that you may not fill your entire iceberg by putting in a new order for the reload quantity, making the quantstrat iceberg closer to a synthetic iceberg than an exchange iceberg. So, in the case of a gap, at least the first shown quantity would be filled at the limit price.There is no point in 'filling at the open price' with limit orders, that's not how they work.For stops, I think the changes that have been discussed in this thread make sense.

Date: 2014-09-07 06:25
Sender: Claymore Marshall
Agreed. But what about if you're running an interday strategy on say 30 min bars. Ideally you want to handle the case of filling at the open on moving from the 3.30-4pm bars to the 9.30-10 am bars the next day.It seems like there are two clear situations for limit orders:1) If the buy limit is executed on a bar through the trading day after the first bar of the day, the transaction price is the buy limit price. In particular, this occurs if the time difference between the last bar and the current bar is equal to the periodicity of the mktdata.2) If it is the first bar of the trading day (or week in the case of FX), fill at the open instead, as this is the first available price.These 2 conditions could be allowed for more generally, I think, by looking at the periodicity (length of a bar in seconds) of the mktdata for the symbol, and checking whether the difference in seconds between curIndex and (curIndex -1) is greater than the periodicity of the mktdata (in seconds) by some multiple (possibly user defined, but perhaps this is overengineering).

Date: 2014-09-05 16:49
Sender: Joshua Ulrich
I think we're talking past each other a bit. You seem focused on daily data, when gaps are due to markets being closed. I'm also considering gaps in intraday data, while the market is open.In the latter case, it doesn't matter if the buy limit 50 is an entry or exit. You want to buy at 50 or lower and it's optimistic to assume that the market would gap through your limit, rather than filling it as the market moved through it. If the gap is due to the market being closed, it might be okay/better to assume you get filled at the open of the next observation.Maybe fill simulation on daily OHLC data should be special-cased, since it's potentially very different from intraday OHLC data. Thoughts?

Date: 2014-09-05 15:49
Sender: Claymore Marshall
I guess the frequency of the bar data is a key point on this issue. Being conservative on limit orders on 5 sec bar data is the right thing to do.

Date: 2014-09-05 15:22
Sender: Claymore Marshall
Actually, when you say buy limit, you're not talking about a take profit, you're talking about filling an entry? My misunderstanding in that context. Indeed, I haven't thought about the code in the context of buy limits ...

Date: 2014-09-05 15:16
Sender: Claymore Marshall
I didn't really have in mind iceberg orders when making changes to the code myself with this issue, so I can't comment on these conditions for those orders. In fact, I'd be curious to see examples of ice berg orders used in quantstrat written by the original authors of the ice berg order type; I don't recall seeing any demos, but I digress...My point about 'the high price is only known after the fact once the gapping bar is complete' (As you're most likely aware I suspect, but for the record and other interested readers) was more to say, if we are long, the price is 52 at bar t, and we have a stop limit at 50, and then on bar t+1 the open price is 49, the high is 49.50, then when we loop to bar t +1, it seems optimistic to fill at 50, since we could not exit at 50 during market close, so we exit asap at the open of 49 (as the price could continue to drop hard duing that bar). We don't exit at the 49.50 high, because we don't know 49.50 is going to be the highest price during bar t + 1 to exit at; i.e. we are being optimistic in the backtest.In the case of limit orders, erring on the side of caution and assuming the fill is always at the orderprice seems like a reasonable idea, but I think it is a moot point. In your example, if the very first time you could fill was at the open of 49, you could get that plus some slippage (since filling right on the open, particularly RIGHT on market open in say equity markets is somewhat optimistic as we cant get the open price exactly?). That gap of ~$1 is material profit lost.Consider real world scenarios. Take backtesting a stock like BBRY over the last few years on Earning release mornings. If you were short the trade the day before at say a price of 14 at 4pm, and you have a take profit set at 12 for fill during real time trading hours, and the market the next day at 9.30am opens at a price of 10, you really would be able to take profit at say 10-10.20, and the profit you would make could be substantially higher than filling at 12 (it could make or break a daily/hourly bar strategy that involves holding traders ovenight). In the case of FX, 'quality' FX brokers will fill you above limit prices at market open (in the case of a long trade) on Sunday afternoon (EST) if the market jumps above in your favour. Again, filling right at the limit might be considered too conservative.Admittedly, I don't use the txnprice as is in ruleOrderProc, I add conservatism by applying a slippage function on the txnprice which penalises take profits and stop limits by a multiple of pips.

Date: 2014-09-05 12:28
Sender: Joshua Ulrich
I now do not think this works for limit or iceberg orders. Say we have a buy limit 50, and the current bar low is 51. The next bar open/low is 49. That means, using the attached patch, txnprice = min(50, 49). Assuming we get filled better than our limit price is not conservative, since (in most cases) we would be filled at our limit price as the market moved through it.

Date: 2014-09-04 23:31
Sender: Joshua Ulrich
Note that this section of code handles 3 order types: limit, stoplimit, and iceberg. I can see the argument for the case of stoplimit orders, and I think it also works for limit orders, but I'm not sure it makes sense for iceberg orders (I don't know enough about that order type).For the record, this change was introduced in r1465:https://r-forge.r-project.org/scm/viewvc.php/pkg/quantstrat/R/ruleOrderProc.R?root=blotter&r1=1435&r2=1465Regarding, 'the high price is only known after the fact once the gapping bar is complete': I think you're confusing signal generation and order execution simulation. Note that orderPrice > Lo(mktdata) is doing the same thing that doesn't make sense to you. The strategy is not deciding whether or not to act. Instead, we're trying to determine whether or not the active order would have been executed.

[R-Forge #5884] applyStrategy.rebalancing ignores first date when rebalancing on all dates

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

The applyStrategy.rebalancing function is set-up to rebalance either on all periods or on specified endpoints, but the endpoint loop assumes endpoints were specified. Specifically, the endpoint loop starts with the second index

for(i in 2:length(pindex)

which causes the first date to be skipped when no endpoints are specified (intention being to rebalance on all dates). I believe the endpoints loop and mktdata subset logic should be updated to something the example below

for(i in ifelse(names(plist) == 'all', 1, 2):length(pindex))

...

now subset

if (names(plist) == 'all')
md_subset<-mktdata[as.POSIXct(index(mktdata)) == pindex[i]]
else
md_subset<-mktdata[as.POSIXct(index(mktdata))>pindex[i-1]&as.POSIXct(index(mktdata))<=pindex[i]]

Followups:

Date: 2014-11-06 20:09
Sender: Jonathan Owen
See also #5804. This becomes a bigger concern when rebalancing is run without a rebalancing rule.

Date: 2014-11-06 20:01
Sender: Jonathan Owen
Attaching diff file of proposed updates.

walk.forward.R Filename format issue

Running walk.forward on the GBPUSD 30min data as provided in the Luxor demo generates this error:

Error in gzfile(file, "wb") : cannot open the connection In addition: Warning message: In gzfile(file, "wb") : cannot open compressed file 'wfa.GBPUSD.2002-10-21 00:30:00.2002-10-23 23:30:00.RData', probable reason 'Invalid argument'

I believe that there is some issue with the file name (it's length or special characters?) when the date AND time are used.

A solution proposed on a pull request in the R-Finance/Quantstrat Repo is to wrap the indexes in the as.Date() function to remove the time. This solves the issue but, as suggested by braverock, it could prevent applying new parameters at any other time than midnight.

[R-Forge #6316] add a switch option between value and percentage for CumPL and Drawdown in chart.Posn

Submitted by: Kenny Liao
Assigned to: Nobody
R-Forge link

Currently, CumPL and Drawdown in chart.Posn are measured in cash (or value), but it may seem also helpful for comparison between returns/drawdown and initEquity, if they can be measured in percentage.

I added an argument Value to chart.Posn, and make the default value to be TRUE. Also add the following line below line 58.

if (!Value) {CumPL = round(CumPL/initEq*100, 1) }

The modified version of chart.Posn can be found here https://bitbucket.org/HiLow/hackforgood/src/96161d4f62ada2dd5602a1b3aa2ca15b144b9117/chart.Posn_CumPLDrawdown_percentage.R?at=master&fileviewer=file-view-default

Followups:

Date: 2016-04-08 22:54
Sender: Kenny Liao
Thanks Brian, could you point me to an example using additions and withdrawals? I have not yet come cross this kind of examples before.Thanks.

Date: 2016-04-08 01:01
Sender: Brian Peterson
This won't work if there are any additions or withdrawals.There are functions in blotter already for extracting returns so that you can use e.g. PerformanceAnalytics to analyze the returns.

[R-Forge #5783] save.strategy() does not save position limits

Submitted by: Jonathan Owen
Assigned to: Nobody
R-Forge link

I discovered this while using apply.paramset() for a strategy having a trade rule that specifies osFUN=osMaxPos. The base strategy script has a call to addPosLimit and runs as expected.

When I clear the environment, reload with load.strategy() and then run apply.paramset(), I receive an error message stating 'no position limit defined for portfolio'. Adding a call to addPosLimit in my paramset script gets around this error. It appears that the position limits are saved in the .blotter env instead of .strategy env and are therefore not restored when a strategy is reloaded.

The question is is this as expected? I can see arguments for and against storing the position limits within the strategy. Since osMaxPos is defined within a rule, the strategy is not able to run without these limits being defined, so if this is behavior is as expected, additional documentation or a more direct error message would be useful (this error is buried in the results from the apply.paramset() runs, which can be more challenging identify).

Clarify documentation for osMasPos

Comments on the commit that merged #42 (559d512) highlighted an issue with the documentation in the merged commit(s). To summarize:

The documentation currently says digits is "numeric number of digits to round position size to". But the osMaxPos digits argument is passed directly to round, and the digits argument to round specifies the number of digits after the decimal place to keep, not the number of digits to round to.

For example:

R> round(400.04, 0)  # ?osMaxPos suggests this should be 0 digits, not 3
[1] 400
R> round(400.04, 3)  # ?osMaxPos suggests this should be 3 digits, not 5
[1] 400.04

[R-Forge #6261] bug & a FIX for demo(luxor.sample.walk.forward) in quantstrat 0.9.1709 rev1715

Submitted by: Alex B
Assigned to: Joshua Ulrich
R-Forge link

to see the bug, run the demo (in subj)

fixes are here (as of this moment):
https://github.com/cloudcello/r-data-store-public/tree/master/quantstrat-fix
take the following files

  1. integrate into quantstrat: chart.forward.training_fixed.R (fixed function chart.forward.training )
  2. put this into the demo folder: luxor.sample.walk.forward.R (a fixed version of 'luxor.smp.wfwd' demo)

Followups:

Date: 2015-11-15 18:20
Sender: Joshua Ulrich
Fixed in r1718.

walk.forward uses only the [testing.timespan] subset from: [R-Forge #6258] bug in demo(luxor.8.walk.forward) in quantstrat 0.9.1687

Submitted by: Alex B
Assigned to: Nobody
R-Forge link

demo(luxor.8.walk.forward) from quantstrat 0.9.1687
produces the following bug: 'Error in if (!all(i <= 0)) stop('only zeros may be mixed with negative subscripts') : missing value where TRUE/FALSE needed '

Below is the code reproducing the bug (or simply run the demos in quantstrat up to the demo named luxor.8.walk.forward

#######################################################################
#!/usr/bin/Rscript --vanilla
#
# Jan Humme (@opentrades) - April 2013
#
# Tested and found to work correctly using blotter r1457
#
# After Jaekle & Tamasini: A new approach to system development and portfolio optimisation (ISBN 978-1-905641-79-6)
#
# Paragraph 3.7 walk forward analysis

require(quantstrat)

source(paste0(path.package('quantstrat'),'/demo/luxor.include.R'))
source(paste0(path.package('quantstrat'),'/demo/luxor.getSymbols.R'))

### foreach and doMC

require(foreach)
require(doMC)
registerDoMC(cores=8)

### blotter

initPortf(portfolio.st, symbols='GBPUSD', initDate=initDate, currency='USD')
initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD', initEq=100000)

### quantstrat

initOrders(portfolio.st, initDate=initDate)

load.strategy(strategy.st)

enable.rule(strategy.st, 'chain', 'StopLoss')
#enable.rule(strategy.st, 'chain', 'StopTrailing')
enable.rule(strategy.st, 'chain', 'TakeProfit')

addPosLimit(
  portfolio=portfolio.st,
  symbol='GBPUSD',
  timestamp=initDate,
  maxpos=.orderqty)

### objective function

ess <- function(account.st, portfolio.st)
{
  require(robustbase, quietly=TRUE)
  require(PerformanceAnalytics, quietly=TRUE)

  portfolios.st <- ls(pos=.blotter, pattern=paste('portfolio', portfolio.st, '[0-9]*',sep='.'))
  pr <- PortfReturns(Account = account.st, Portfolios=portfolios.st)

  my.es <- ES(R=pr, clean='boudt')

  return(my.es)
}

my.obj.func <- function(x)
{
  # pick one of the following objective functions (uncomment)

  #return(max(x$tradeStats$Max.Drawdown) == x$tradeStats$Max.Drawdown)

  #return(max(x$tradeStats$Net.Trading.PL) == x$tradeStats$Net.Trading.PL)

  return(max(x$user.func$GBPUSD.DailyEndEq) == x$user.func$GBPUSD.DailyEndEq)
}

### walk.forward

r <- walk.forward(strategy.st,
                  paramset.label='WFA',
                  portfolio.st=portfolio.st,
                  account.st=account.st,
                  period='months',
                  k.training=3,
                  k.testing=1,
                  obj.func=my.obj.func,
                  obj.args=list(x=quote(result$apply.paramset)),
                  user.func=ess,
                  user.args=list('account.st'=account.st,
                                 'portfolio.st'=portfolio.st),
                  audit.prefix='wfa',
                  anchored=FALSE,
                  verbose=TRUE)

###############################################################################
# ERROR: Error in if (!all(i <= 0)) stop('only zeros may be mixed with negative subscripts') :
#        missing value where TRUE/FALSE needed

### analyse

pdf(paste('GBPUSD', .from, .to, 'pdf', sep='.'))
chart.Posn(portfolio.st)
dev.off()

ts <- tradeStats(portfolio.st)
save(ts, file=paste('GBPUSD', .from, .to, 'RData', sep='.'))

Followups:

Date: 2016-03-08 14:26
Sender: Kenny Liao
Also, there seems to be a similar error in luxor.4. I posted this question in full in stackoverflow: http://stackoverflow.com/questions/35843473/quantstrat-demo-luxor-4-error-attempt-to-select-less-than-one-elementCould you have a look at this as well? Thanks!

Date: 2016-03-08 12:11
Sender: Kenny Liao
Thanks Alex, your reply has been very helpful. I did what you suggested to overwrite and installed from updated source and it almost works perfectly. I didn't download the data you suggested in previous posts, could it be the reason(the data link you provided I cannot access from China)? Could you have a look? Thank youI tried twice: first, I tried to run luxor.8 directly; second time, I ran luxor.1, luxor.2, luxor.5, and finally luxor.8. I received similar error as following:

error calling combine function:
<simpleError in fun(result.1, result.2, result.3, result.4, result.5, result.6,
     result.7, result.8, result.9, result.10, result.11, result.12,
     result.13, result.14, result.15): attempt to select less than one element>
Error in walk.forward(strategy.st, paramset.label = 'WFA', portfolio.st = portfolio.st,  :   obj.func() returned empty result
In addition: There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In (function (x)  ... : discarding extra objective function result(s)
2: In ruleOrderProc(portfolio = portfolio, symbol = symbol,  ... :  ignoring order with quantity of zero

my sessionInfo() returns the following:

> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.11.3 (El Capitan)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] robustbase_0.92-5             doMC_1.3.4                    iterators_1.0.8               quantstrat_0.9.1709           foreach_1.4.3                 blotter_0.9.1695              PerformanceAnalytics_1.4.3541 [8] FinancialInstrument_1.2.0     quantmod_0.4-5                TTR_0.23-0                    xts_0.9-7                     zoo_1.7-12                   

loaded via a namespace (and not attached):
[1] lattice_0.20-33  codetools_0.2-14 grid_3.2.3       tools_3.2.3      DEoptimR_1.0-4   compiler_3.2.3 

Date: 2016-03-07 18:22
Sender: Alex B
Hello Kenny, the easiest solution at this point would be to download this SVN repository and simply overwrite the existing files with these: https://usercontent.irccloud-cdn.com/file/g1B2sUBZ/walk.forward.R and https://usercontent.irccloud-cdn.com/file/1R6dv311/luxor.8.walk.forward.R. Then install the quantstrat package from your updated sources. My patches have not been confirmed/approved yet (i.e. all I know is that they work on my computers). Also, I think you do not have to download all the www folder for this demo to work, if I remember correctly. (If you need more help, feel free to join the IRC channel #R-Finance at freenode network, but you might have to wait for your question to be answered depending on what time zone you live in.)

Date: 2016-03-07 09:45
Sender: Kenny Liao
Hi Alex B, thanks a lot for this fix and patch. I am new to R and quantstrat and encountered the same bug [#6258] you reported. I want to make use of your fix/patch to make luxor.8 work, but I don't know how exactly to put your patch into practice. could you kindly provide a step by step guide on how to make luxor.8 work using your patch? also, do I still need to download all the data in the 'www' folder in the svn repository? Thank you very much!Kenny

Date: 2016-02-18 00:05
Sender: Alex B
All the explicit errors should be fixed now. Please apply patches for ES() (in the attachment section to this 'thread') and patch [#6291] submitted in the 'Patches' section of the project.

Date: 2016-02-17 10:08
Sender: Alex B
I made the previous comment before I knew there was no bug tracker for the PerformanceAnalytics package. Please treat the ES()-related issue as a separate bug. To keep the proper record of history, I attached the patch for the ES() function to this thread (file 'ES_zero_return_column_input_bugfix-20160217-0326.diff'). PS: I will try to submit the rest of the fixes for walk.forward() later today or tomorrow.

Date: 2016-02-17 00:45
Sender: Alex B
I made several fixes in my copy of the code and it seems that I found the root of the bug: a failing user.func within apply.paramset() within walk.forward(). Some parameter settings do not produce any trades (can happen no matter how much data one uses). In such cases function ES (in this example) gets 'portfolio returns' argument with all values equal to zero. This zero-valued argument causes ES() to crash. Demonstration of the bug: get this data ES_test.RData (see the attachment, which contains object 'pr' - portfolio returns ) and run the following line 'ES(R=pr, clean='boudt')' (taken from the demo luxor.8....)I am submitting a patch for this particular bug now.At the same time, I still have to go back and reverse all the other changes in my local copy of the code and see whether those prior changes are still required after this bug is fixed. So let's not close this bug report for now ( [#6258] ).

Date: 2015-11-16 01:09
Sender: Alex B
I may have fixed the glitch. Seems like there must have been more data for the demo to work. Here's the patch: https://usercontent.irccloud-cdn.com/file/sEQaupmr/patch_luxor_demo8.diff For this fix to work, one must also create a folder /extdata_full/ and copy GBPUSD data from the repo folder 'www' up until & including the file dated '2003-02-27' (that's about 500 kilobytes in 82 files). Perhaps the number of files can still be reduced. (My hypothesis is that the glitch may have been caused by training period end dates which had no data, but that's just a guess.)

Date: 2015-11-15 01:20
Sender: Alex B
Brian, I downloaded all the data you uploaded to the 'www' folder in the svn repository, changed luxor.include.R to http://pastebin.com/8f3u3HQX (to include all the data). And here's what I get when running demo luxor.5.strategy.ordersets.R -- http://pastebin.com/62qH1Z2V The data skips a lot of dates and some files seem unusable. Am I doing anything wrong here or the data is indeed corrupted? PS if anyone needs to get this data quickly here's the code http://pastebin.com/NvY18tkR (feel free to include it in quantstrat)

Date: 2015-11-12 10:41
Sender: Alex B
Perhaps, as a temporary fix, the demo might include the code for downloading it from r-forge and run using that downloaded data (or part of it sufficient enough not to cause errors). Later, the data might simply be copied to Quandle (or somewhere where it could sit with a permanent web address) and the demo would download it from there.

Date: 2015-11-11 20:40
Sender: Brian Peterson
I can provide a little more context. Yes, the data originally used was many years longer, mirroring the data set from Tomasini. I recently put the entire data set back into the 'www' tree on R-Forge, so it would be possible for people to replicate exactly what we did at the time the scripts were written.

Date: 2015-11-11 19:28
Sender: Joshua Ulrich
I can replicate the 'missing value where TRUE/FALSE needed' error (after running luxor.5.strategy.ordersets.R). It seems to be because there isn't enough data for period='months' to work. After changing period='days', I get the 'obj.func() returned empty result' error. That was caused by robustbase not being installed on my system. After installing robustbase, I get an error, 'Error in runSum(x, n) : Invalid 'n'' because the code is trying to take a 40+ period SMA on only ~25 observations.I wondered if the data used in the demos was longer at some point, so I checked the commit history. The luxor.8 demo was added in r1466, and the GBPUSD data had not changed after that, so it could not have changed and caused a regression bug. So maybe the walk forward parameters were changed after r1466? No, they were set in r1464.Finally, I checked out r1467 and tried to run the luxor.8 demo (after running the luxor.5 demo first, and setting period='days'. I still got the 'invalid 'n'' error. Therefore, my current hypothesis is that this demo never worked as committed.

Date: 2015-11-10 17:01
Sender: Alex B
Here's what Guy Yollin suggested in this post: https://stat.ethz.ch/pipermail/r-sig-finance/2014q3/012721.htmlI quote: -begin quote-I'm pretty sure that you can get the luxor.8.walk.forward.R script to run successfully as follows:1. execute luxor.5.strategy.ordersets.R which saves the strategy2. modify the the period argument in the call to walk.forward() to be period='days'-end quote-In my case, I no longer see the error I reported in my bug report (above) ; however another error appears'Error in walk.forward(strategy.st, paramset.label = 'WFA', portfolio.st = portfolio.st, : obj.func() returned empty resultIn addition: There were 31 warnings (use warnings() to see them)'HTH

[R-Forge #5991] apply.paramset modifies the base portfolio environment

Submitted by: Offer Markovich
Assigned to: Nobody
R-Forge link

apply.paramset should use the strategy structure to text N different combination of parameters. To do that it is creating N different portfolio which are all starting as a clone of the base portfolio.
each portfolio is used with one combination.

The problem is that clone.protfolio [paramsets.R line 51] is not doing a proper job and as a results all the N portfolios and the base portfolio are all actually one portfolio which is being used.

At the end of the execution the base portfolio holds the data of the last combination used by apply.paramset.

This problem will not have any impact when using a parallel engine. but with a local (registerDoSEQ()) the base portfolio will be 'dirty'.

There is no impact on the result of apply.paramset since the execution is one by one and clone.portfolio is 'cleaning' the portfolio for each combination, BUT

  1. base portfolio will be dirty at the end
  2. output of walk.forward which is using apply.paramset will be completly wrong.

Below:

  1. Standalone script for testing (parallel can be turned off/on in lines 108:111)
  2. Output of parallel execution
  3. Output of local execution

Once reviewed and approved as a bug I can provide a patch

Offer

############ SCRIPT

require(quantstrat)
require(iterators)
Sys.setenv(TZ='UTC')

symbols=c('SPY')
currency('USD')
stock('SPY',currency='USD')
.from=as.Date('2014-06-01')
.to=Sys.Date()

suppressWarnings(getSymbols(symbols,adjust=T,from=.from,to=.to))
account.st='My Account'
portfolio.st='My Portfolio'
strategy.st='My Strategy'
initEq=100000

initDate=max(start(SPY))
rm.strat(strategy.st)
rm.strat(portfolio.st)
rm.strat(account.st)

initPortf(portfolio.st,symbols='SPY', initDate=initDate, currency='USD')
initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD',initEq=initEq)
initOrders(portfolio.st, initDate=initDate)

strategy(strategy.st, store=TRUE)

mktdata=SPY
.fast=10
.slow=20

add.indicator(strategy.st, name = 'EMA',arguments = list(x=quote(Cl(SPY)),n = .fast),label='spyFast')
add.indicator(strategy.st, name = 'EMA',arguments = list(x=quote(Cl(SPY)),n = .slow),label='spySlow')

mktdataInd=applyIndicators(strategy.st,mktdata=mktdata)

add.signal(strategy.st, name='sigCrossover',
arguments = list(
columns=c('spyFast','spySlow'),
relationship='gte'
),
label='long'
)

add.signal(strategy.st, name='sigCrossover',
arguments = list(
columns=c('spyFast','spySlow'),
relationship='lt'
),
label='short'
)

mktdataSig=applySignals(strategy.st,mktdata=mktdataInd)

add.rule(strategy.st, name='ruleSignal',
arguments=list(sigcol='short', sigval=TRUE,
orderside='short' ,
ordertype='market',
orderqty='all',prefer='Open',
TxnFees=.txnfees,
orderset='ocoshort',
replace=TRUE
),
type='exit',
label='CloseShort'
)
add.rule(strategy.st, name='ruleSignal',
arguments=list(sigcol='long' , sigval=TRUE,
orderside='short' ,
ordertype='market', prefer='Open',
#threshold=.threshold,
orderqty=-.orderqty,
replace=FALSE
),
type='enter',
label='Short'
)

out<-applyStrategy(getStrategy(strategy.st) , portfolios=portfolio.st)

.fast = seq(from=3,to=3,by=1)
.slow= seq(from=15,to=26,by=10)

add.distribution(strategy.st,
paramset.label = 'EMA',
component.type = 'indicator',
component.label = 'spyFast',
variable = list(n = .fast),
label = 'spyFast')

add.distribution(strategy.st,
paramset.label = 'EMA',
component.type = 'indicator',
component.label = 'spySlow',
variable = list(n = .slow),
label = 'spySlow')
add.distribution.constraint(strategy.st,
paramset.label = 'EMA',
distribution.label.1 = 'spyFast',
distribution.label.2 = 'spySlow',
operator = '<',
label = 'EMA')

library(doParallel)

cl<-makeCluster(3)

registerDoParallel(cl)

registerDoSEQ()
p=getPortfolio(portfolio.st)
print(p$symbols[['SPY']]$txn)

results <- apply.paramset(strategy.st,
paramset.label='EMA',
portfolio.st=portfolio.st,
account.st=account.st,
audit = NULL,
verbose=!TRUE,
nsamples=2,
#user.func=f,user.args=list(account.st=account.st)
)

p=getPortfolio(portfolio.st)
print(p$symbols[['SPY']]$txn)

########################## Parallel execution

before

       Txn.Qty Txn.Price Txn.Value Txn.Avg.Cost Pos.Qty Pos.Avg.Cost Gross.Txn.Realized.PL Txn.Fees

2014-06-02 0 0 0 0 0 0 0 0
Net.Txn.Realized.PL Con.Mult
2014-06-02 0 0

after

       Txn.Qty Txn.Price Txn.Value Txn.Avg.Cost Pos.Qty Pos.Avg.Cost Gross.Txn.Realized.PL Txn.Fees

2014-06-02 0 0 0 0 0 0 0 0
Net.Txn.Realized.PL Con.Mult
2014-06-02 0 0

########################### non parallel - wrong

before

       Txn.Qty Txn.Price Txn.Value Txn.Avg.Cost Pos.Qty Pos.Avg.Cost Gross.Txn.Realized.PL Txn.Fees

2014-06-02 0 0 0 0 0 0 0 0
Net.Txn.Realized.PL Con.Mult
2014-06-02 0 0
spyFast spySlow
1 3 15
[1] '2014-08-15 00:00:00 SPY -1 @ 195.55589173521'
[1] '2014-09-16 00:00:00 SPY 1 @ 197.685935041126'
[1] '2014-09-17 00:00:00 SPY -1 @ 199.835885293826'
[1] '2014-09-24 00:00:00 SPY 1 @ 198.04'
[1] '2014-09-25 00:00:00 SPY -1 @ 199.04'
[1] '2014-09-26 00:00:00 SPY 1 @ 196.7'
[1] '2014-10-23 00:00:00 SPY -1 @ 194.62'
spyFast spySlow
2 3 25
[1] '2014-08-15 00:00:00 SPY -1 @ 195.55589173521'
[1] '2014-09-26 00:00:00 SPY 1 @ 196.7'
[1] '2014-10-24 00:00:00 SPY -1 @ 195.25'

after

       Txn.Qty Txn.Price Txn.Value Txn.Avg.Cost Pos.Qty Pos.Avg.Cost Gross.Txn.Realized.PL Txn.Fees

2014-06-02 0 0.0000 0.0000 0.0000 0 0.0000 0.000000 0
2014-08-15 -1 195.5559 -195.5559 195.5559 -1 195.5559 0.000000 0
2014-09-26 1 196.7000 196.7000 196.7000 0 0.0000 -1.144108 0
2014-10-24 -1 195.2500 -195.2500 195.2500 -1 195.2500 0.000000 0
Net.Txn.Realized.PL Con.Mult
2014-06-02 0.000000 0
2014-08-15 0.000000 1
2014-09-26 -1.144108 1
2014-10-24 0.000000 1

note that the txn of the portfolio is exectly as the transaction of last combination

[R-Forge #5298] QuantStrat issue with adding more than one indicator

Submitted by: Ilya Kipnis
Assigned to: Nobody
R-Forge link

I'm doing this on Brian's server, so it may be an issue on my end, but...I added one more indicator to the Faber demo, and I'm attaching my 'broken' demo.

The difference is that I added another indicator, which is simply a copied and pasted version of the first, with the 10 changed to 15.

I'm not sure why this is happening...maybe I'm using some outdated technology, but, I just wanted to share this.

Here's my sessionInfo().

R version 3.0.2 (2013-09-25)
Platform: x86_64-pc-linux-gnu (64-bit)

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C LC_COLLATE=C
[5] LC_MONETARY=C LC_MESSAGES=C LC_PAPER=C LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=C LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] quantstrat_0.8.0 foreach_1.4.1 blotter_0.8.17
[4] PerformanceAnalytics_1.1.0 FinancialInstrument_1.1.9 quantmod_0.4-0
[7] Defaults_1.1-1 TTR_0.22-0.1 xts_0.9-7
[10] zoo_1.7-10

loaded via a namespace (and not attached):
[1] codetools_0.2-8 grid_3.0.2 iterators_1.0.6 lattice_0.20-24 tools_3.0.2

Followups:

Date: 2014-02-12 16:25
Sender: Seth Wayland
The attached file throws an error for me at the applyStrategy call on line 122. The error is:Error in colnames<-(*tmp*, value = c('XLB.Close.SMA.15', 'XLB.Close.SMA.10.SMA10.SMA.15' : length of 'dimnames' [2] not equal to array extentMy sessionInfo() output:R version 3.0.2 (2013-09-25)Platform: x86_64-apple-darwin10.8.0 (64-bit)locale:[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8attached base packages:[1] stats graphics grDevices utils datasets methods base other attached packages: [1] quantstrat_0.8.0 foreach_1.4.1 blotter_0.8.17 PerformanceAnalytics_1.1.2 [5] FinancialInstrument_1.1.9 quantmod_0.4-0 Defaults_1.1-1 TTR_0.22-0 [9] xts_0.9-7 zoo_1.7-10 loaded via a namespace (and not attached):[1] codetools_0.2-8 grid_3.0.2 iterators_1.0.6 lattice_0.20-24 tools_3.0.2

Date: 2014-01-22 23:35
Sender: Joshua Ulrich
The attached file works for me. You didn't say what is broken. My sessionInfo() output:R version 3.0.2 (2013-09-25) Platform: x86_64-pc-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages:[1] stats graphics grDevices utils datasets methods base other attached packages: [1] quantstrat_0.8.0 foreach_1.4.1 blotter_0.8.17 [4] PerformanceAnalytics_1.1.1 FinancialInstrument_1.1.9 quantmod_0.4-0 [7] Defaults_1.1-1 TTR_0.22-0.1 xts_0.9-7 [10] zoo_1.7-10 setwidth_1.0-3 colorout_1.0-1 loaded via a namespace (and not attached):[1] codetools_0.2-8 grid_3.0.2 iterators_1.0.6 lattice_0.20-24

[R-Forge #5795] iterators library is used in apply.paramset() but not included as a package dependency

Submitted by: Jonathan Owen
Assigned to: Joshua Ulrich
R-Forge link

The apply.paramset() method calls iter in the iterators library but quantstrat does not include a dependency on iterators. I'm currently seeing the message 'Error in eval(expr, envir, enclos) : could not find function 'iter' when running luxor.3.paramset.sma.R sequentially. I believe the parallel libraries, like doParallel, depend on iterators, so this message does not appear when run in parallel.

Rebuilding the package with the following updates appears to eliminate the error.

Add this to the roxygen comments for apply.paramset():
@importFrom iterators iter

And add this to NAMESPACE:
importFrom(iterators,iter)

Add this to DESCRIPTION:
Imports:
iterators

After these updates, the require commands at the top of paramsets.R may not be necessary.

Followups:

Date: 2015-11-14 23:45
Sender: Joshua Ulrich
Fixed in r1715 and r1716.

Date: 2014-07-09 12:38
Sender: Jonathan Owen
Related to this item, the reshape2 package is used in tradeGraphs() but also not included in the DESCRIPTION and NAMESPACE files. The quickest solution might be to add reshape2 to the Suggests list in the DESCRIPTION file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.