pymc-labs/pymc-marketing

Bayesian marketing toolbox in PyMC. Media Mix (MMM), customer lifetime value (CLV), buy-till-you-die (BTYD) models and more.

PyMC Labs

Connect with us:

pymc-marketing's Issues

Prior predictive checks MMM API

Prior predictive in #49

Publish paper about this package in JOSS

It would be good to maximise the benefits of writing this. So we could put together a paper at JOSS https://joss.theoj.org/

Publicise by writing blog posts

After #30, publicise by writing post at

• https://towardsdatascience.com/
• PyMC Labs

Marginalization over dichotomous variable

I know that this is ongoing work raised in issue 21 of AePPL, but is there a quick way to marginalize over a discrete parameter? Akin to the example provided in the issue linked above:

$$p(Y=y | X=0) * p(X=0) + p(Y=y | X=1) * p(X=1)$$

for some continuous $Y$ and dichotomous $X$. In this case, $X$ would be the variable indicating a customer churning or not and the difference between a contractual and non-contractual likelihood would be that one is the marginalized version of the other. I have not revisited the math in several weeks, but I'm fairly certain of this and the ability to marginalize likelihoods as such may provide better model building blocks rather than define a distribution class for each quadrant. Just an idea so far...

Add external regressors (both continuous and categorical variables)

Add external regressors (both continuous and categorical variables) in MMM API, see #49

Add `setup.py`

Add setup.py so that we can install the python package via python -m pip install -e .

MMM `adstock_max_lag` and `control_data` are model specific

If we intend our MMM class to be a suitable generic base for many media mix models, we should have as generic arguments to __init__ as possible. adstock_max_lag is completely specific to a particular way to apply a convolution operation on a vector, and control_data might be handled differently by different MMM subclasses (e.g. it could handle continuous controls differently than categorical controls). We should remove as much model specific arguments from the __init__ signature, and leave the rest as kwargs that are forwarded into _build_model

Add more MMM notebooks using a Facebook's Robyn dataset

In addition, to the simulated example we want to have an example with "real" data. It seems that Facebook's Robyn has some data sets which we can use.

`MMM` should rely on `axes.plot` as much as possible, and resort to seaborn only if really necessary

As far as I can tell, this plot could have been done using ax.plot instead of seaborn.lineplot. I think that for these simple situations, we should rely on matplotlib behavior instead of delegating to seaborn.

add black[jupyter] to the environment

Now we have notebooks, it would be useful to add black[jupyter] to the environment. I have no understanding of how to play with commit hooks, but maybe it's relevant to add in there too?

Add ROAS plot

Add a method to compute the return of ad spend for certain channels and plot it. The plot could follow the style of figure 3 from Jin et al 2017, which I'll copy down here just as a reference.

Add study case with MMM + CLV

Create some fake dataset with continuous non-contractual process and build a story around:

1. Using MMM to infer cost of acquisition across different channels
2. Using CLV to infer differential lifetime value of customers coming from different channels
3. Making business decision that takes the two sources of information into account
   a. Preferring to invest in a channel with higher CAC because of higher CLV
   b. Binary decision to not further invest in channel if CLV is lower than CAC

Requires: #24, #19, #39

We might use this case-study for the initial announcement of the package

Take the CLV grid to the next level

The idea is to follow up to #25, and extend the basic models in interesting ways, e.g., by adding hierarchical effects, time-varying effects... and so on. This should give us a more refined idea not only of what building blocks we need, but how flexible they should be.

This will, potentially, also be the biggest selling point of the package, as we will be doing things that are not really done out there (or at least not published in neat papers / packages), in large part by taking advantage of working with a fully-fledged PPL (PyMC!)

Unlike #25, these squares are not yet fixed, and any cool idea you have can be used.

• Continuous Non-contractual + Hierachical structure (#39) (up for grabs)
• Continuous Contractual + ??? (up for grabs)
• Discrete Non-contractual + ??? (up for grabs)
• Discrete Contractual + cohort / temporal effects (suggested to @drbenvincent), see #35

Possible the grid won't be over the 4 types of models, but perhaps over extensions:

• Cohort + Temporal effects on lifetime
• Cohort + Temporal effects on value
• Complex interactions between Lifetime and Value components
  • E.g., subscription fee affects churn-rate and value, @juanitorduz brought something that resonates with this
• THE NEXT BANG IN MARKETING MODELS ???

update environment + README to enable use of graphviz

Would be good to able to use pm.model_to_graphviz(model) in the Jupyter notebooks.

Gamma-Gamma Model of Monetary Value in PyMC

Similarly as in #18, we want to have the Gamma-Gamma Model of Monetary Value in PyMC and compare it against the results in lifetimes. See https://juanitorduz.github.io/gamma_gamma_pymc/

Implement pre-built shifted beta geometric `sBG` model

Related to #32

We should also add some useful summary stats / plots. If they are not specific to the sBG the better!

Implement MMM model fit

Implement MMM model fit in the base API, see #49

Add CLV Grid and description to README

See #25 and #26

Explore variants of the shifted Beta Geometric model

From: Fader, P. S., & Hardie, B. G. (2007). How to project customer retention. Journal of Interactive Marketing, 21(1), 76-90. pdf

They mention this other model derived from the beta-binomial, which is conceptually equivalent:

Their model is based on assumptions simi-
lar to those behind the sBG model: (a) Each person
responds to a direct-mail solicitation with constant
probability p, and (b) p varies across the population
according to a beta distribution. While BM base their
framework on the beta-binomial model, it could have
been derived as an sBG model (e.g., the mailing on
which the prospect responds to the offer is character-
ized by the shifted-geometric distribution). As such, it
is possible to identify clear relationships between
some of the results in this article [e.g., rt and S(t)] and
some quantities of interest in a list-falloff setting.

Then extensions with cohort covariates:

The BM framework was extended by Rao and Steckel
(1995) to incorporate (time-invariant) descriptor
variables such as age, income, and sex. This is accom-
plished using the beta-logistic model (Heckman &
Willis, 1977),

Incorporating the effects of time-
varying covariates (e.g., marketing-mix effects, sea-
sonality) is more complicated. The key is to bring in
all of these factors at the right level; that is, at the
level of the latent parameter of interest (in this case,
�) instead of just “jamming” different covariate effects
into a regression-like model (see Schweidel, Fader, &
Bradlow, 2006, for a discussion of how to do this in a
continuous-time contractual setting.)

And extensions with time effets:

Both the sBG model and its continuous-time analog
(i.e., the EG model) are based on the assumption that
the commonly observed phenomenon of increasing
retention rates is due entirely to heterogeneity;
individual-customer-level retention rates are assumed
to be constant. If we wish to allow for the possibility of
time dynamics at the level of the individual customer,
we can no longer characterize the duration of an indi-
vidual’s relationship with the firm using either the
shifted-geometric or exponential distributions, both of
which have the “memoryless” property (i.e., the proba-
bility of survival to s � t, given survival to t , is the
same as the initial probability of survival to s ). In a
continuous-time setting, we can accommodate this
effect by assuming that individual lifetimes can be
characterized by the Weibull distribution, which allows
for an individual’s risk of canceling a contract to
increase or decrease as the length of the relationship
with the firm increases. In a discrete-time contractual
setting, this leads to the beta-discrete-Weibull (BdW)
model (Fader & Hardie, 2006), which is a generaliza-
tion of the sBG model, while in a continuous-time con-
tractual setting, this leads to a generalization of the EG
model, the Weibull-gamma (WG) model (Hardie et al.,
1998; Morrison & Schmittlein, 1980).

Add `tanh` saturation function

Could be useful to add the 2-parameter saturation function we've got outlined in this blog post https://www.pymc-labs.io/blog-posts/reducing-customer-acquisition-costs-how-we-helped-optimizing-hellofreshs-marketing-budget/ We find this works well because the parameterisation has independent control of the initial slope and the saturation level.

Add type checking into pre-commit

We would have to add mypy as a dev dependency, a Makefile target and also add it as a pre-commit action

Vectorize the random method in `ContNonContract`

The rng_fn in ContNonContract uses a for loop, but it can surely be vectorized although it would require some clever math and perhaps some concepts from order statistics.

Improve scaling example mmm notebook

According to https://github.com/google/lightweight_mmm#preparing-the-data it is better to use a min-max scaler for the target variable instead of an usual standard scaler to keep the target positive.

Default choice of hyper priors on MMM parameters based on data where ever possible

The default prior parameter values for the MMM model are hard-coded at the moment. It is desirable to establish their values heuristically from the supplied data whenever possible.

Should `mmm` be a subpackage like `clv` instead of a module?

This would make it simpler to have multiple implementations of MMM models without having to resort to a huge monolithic script.

Sketch MMM user API

In this issue we want to discuss a potential API for the MMM model. For inspiration, take a look into Google's LightWeight MMM https://github.com/google/lightweight_mmm

Migrate MMM plotting tests to test_base

The tests of the plotting functions should be in the base class and operate on a mock inference data result, not on a real result from a fitted model.

Instantiate `pymc.Model` after data validation and preprocessing in MMM base class

The MMM base class instantiates an empty pymc.Model during its __init__. It does this before having gone through data preprocessing and validation. We should move the model instantiation later in the __init__, closer to the call to _build_model

Adstock transformation without `for` loop

We would like to write a more efficient implementation of the adstock transformations so that we do not use a for loop. An attempt with scan was (unsuccessfully) implemented in #15

Requirements:

• We should be able to add the l_max parameter to truncate the size of the effect.
• Be vectorised
• Should be faster than the current implementation.

New default sampler for `ContNonContract` when there are no observations

Currently, the intention is to primarily use ContNonContract to perform inference on observational data. However, without the observed= keyword, our samplers will misbehave as value = [t_x, x] for t_x being the time of the xth observation with x being an integer.

A moment method would be beneficial, but careful thought must be put into the sampler.

Continuous contractual notebook looks problematic

The parameters don't seem to be well recovered, suggesting we have a bug somewhere?

https://github.com/pymc-labs/pymc-marketing/blob/3228a4c01dd4411b121f4d9eed9dc3accb388820/notebooks/continuous-contractual.ipynb

CC @larryshamalama

MMM Example Notebook: Time Varying Coefficients

To prove the flexibility and potential of framework (namely, bayesian stats and pymc), we would like to have an example notebook to illustrate how to extend the base model (based on simulated data) introduced in #41 by allowing time varying coefficients via gaussian processes.

LTV computation

After having an implementation of #18 and #19 we would like to combine them to get an LTV estimation at user level following the approach in the lifetimes package. Se should be able to reproduce all the results of the documentation https://lifetimes.readthedocs.io/en/latest/Quickstart.html

Move distributions into CLV submodule

The ones implemented so far are CLV specific

Implement pre-built `BG/NBD`/`BetaGeoFitter` model

It would be good to add a BetaGeoFitter function that returns a ContNonContract with some default priors. A signature that resembles what is provided in the lifetimes package would be a good idea. Something along the lines of the following snipet.

def BetaGeoFitter(name, a, b, r, alpha, T, T0, *, observed, **kwargs):
    p = pm.Beta(f"{name}_beta", a, b, size=size, shape=shape)
    lam = pm.Gamma(f"{name}_gamma", r, 1/alpha, size=size, shape=shape)
    return ContNonContract(name, lam, p, T, T0, size=size, shape=shape, **kwargs)

We should also add some useful summary stats / plots. If they are not specific to the BG/NBD the better!

BG/NBD Model in PyMC (base)

We would like to have BG/NBD Model in PyMC. The results should match the estimates from the lifetimes package. See https://juanitorduz.github.io/bg_nbd_pymc/ For this first iteration we do not need external time-invariant regressors, but keep it mind we would like to add them in the future.

Decide how to handle `date_column` dtype and transformations

At the moment, the date_column is left untouched in the data. This might be ok, but it would be great to support datetime dtypes as inputs. Those can be easily handled in plots of time series, and they are more natural to reason about as opposed to floating point arrays referenced to some onset event and using some unknown resolution in some unknown time zone. The difficulty of supporting datetime dtypes is that we need to:

1. Ensure that the date_column is a date time compatible dtype
2. Convert it to a datetime if it isn't using some string format?
3. Add a transformation to go from datetimes to floating point arrays and back. This is necessary if we want to have time series with seasonal components or other mathematical dependencies on time.

Diagnostic plots (as for example the current notebook)

Diagnostic plots from model fit in #49

Import `RandomState` type hint from `pymc.util`

The MMM module implements the RandomState type hints. We could import it from pymc.util instead

Util function to generate Fourier modes

To model seasonality it would be useful to have a utility function to generate Fourier modes as described in https://docs.pymc.io/en/stable/pymc-examples/examples/time_series/Air_passengers-Prophet_with_Bayesian_workflow.html

Add study case with alternative to gamma-gamma model

If we don't summarize individual transaction values, there should be much more flexibility in how to model user latent "spend", with e.g, timeseries component, glm predictors, ....

Would be nice to add a study case of such, perhaps motivating new summary/plotting/prediction functionality of the library.

Should unit tests live outside of the package?

We are shipping our the tests folder inside of the pymmmc package. I don't think is is necessary or desirable. I propose that we move the tests folder to the root directory instead.

Current organization:
root/
└─> pymmmc/
└─> tests/

Proposed:
root/
├─> pymmmc/
└─> tests/

Figure out API for incorporating information from lift tests on MMM

@juanitorduz mentioned that in the literature this is usually incorporated into the prior information, but I think that in the HelloFresh project @lucianopaz found a way to incorporate such information via observed variables in the same MMM model?

Should prior/posterior predictive plots rely on arviz hdi or xarray symmetric quantiles?

The prior and posterior predictive plots of the MMM base class use arviz.hdi. This can lead to potentially confusing plots like these. Should we default to use xarray.quantile and plot symmetric quantiles around the median instead of the highest density interval?

Add delayed adstock function

Add delayed adstock function from Jin, Yuxue, et al. "Bayesian methods for media mix modeling with carryover and shape effects." (2017).

Add notebooks to fill the basic CLV grid

The idea is to write a notebook with pure PyMC model(s) for each of these CLV scenarios. We can start with the Lifetime part (not-value yet), but ideally we will include value as well by the end. This might be a constant with time-decay penalty in the simplest cases.

Hopefully this will give us a good picture of the building blocks that are necessary for a minimum viable package, and can also serve as the base documentation. Overtime we would replace the custom PyMC code with imports from the CLV sub-package.

• Continuous Non-contractual #16
• Continuous Contractual #36
• Discrete Non-contractual (up for grabs)
• Discrete Contractual #32

Rely on `xarray.DataArray.to_dataframe` more in MMM plotting.

This snippet creates a pandas DataFrame from an xarray DataArray. I think that it's better to rely on xarray builtin functionality to do this, specially if the underlying numpy.array dimensions are not aligned to the expected columns specification.

Wrap data into `pm.MutableData` containers for out-of-sample predictions.

Wrap data into pm.MutableData containers for out-of-sample predictions in the base model API. Make sure we can also extend coordinates.

Add long-term seasonality (via Fourier modes)

Add long-term seasonality (via Fourier modes) to base MMM API model, see #49

Initial MMM workflow example

Create a notebook where we show how to use the saturation and adstock transformations in a simple simulated example in a similar manner as in https://juanitorduz.github.io/pymc_mmm/ Let us also see how to scale the variables back-and-forth.