Memory concerns with large `l_max` values

[kenteross]

Memory concerns with large `l_max` values

[juanitorduz]

Hi @kenteross! Thanks for opening this issue! As the code is long and has text, can you share it in a GitHub gist or a Google Colab notebook, please 🙏 ?

[kenteross]

@wd60622 thanks for your suggestions! I tried reducing the l_max and importing the packages independently and it seems like the issue was coming from the l_max. It seems like the maximum threshold for an l_max is around 20 in this case. I'll stick with weekly data for now. Thanks for the help!

[kenteross]

@wd60622, yeah, that makes sense to model weekly data and reduce l_max so much. Thanks for the tips!

[kenteross]

Hi @juanitorduz! Thanks for the quick reply! Sorry, I didn't realize it was in text format. Here's a GitHub gist of my code!

[wd60622]

Hi @kenteross , I see that that the l_max is 50
Have you tried a smaller value? Does it cause issues with l_max = 5 or 10?
Weekly datasets tend to be common so 50 days = l_max of 7

[wd60622]

Are you also able to import the function from the package as well instead of copy and paste. I see the functions have changed and maybe there has been a bug fix. Module for reference

from pymc_marketing.mmm.transformers import delayed_adstock, geometric_adstock, logistic_saturation

[wd60622]

@wd60622 thanks for your suggestions! I tried reducing the l_max and importing the packages independently and it seems like the issue was coming from the l_max. It seems like the maximum threshold for an l_max is around 20 in this case. I'll stick with weekly data for now. Thanks for the help!

Glad that it helped.

And just to be clear, did the model start sampling? (progress bar, etc)?
And it only successful with which ordering?

1. adstock -> saturation
2. saturation -> adstock

If you just perform the adstock, does the large l_max cause issues?

[kenteross]

Yeah, thanks!

The model did start sampling even when it ran into an issue. It would just run my machine out of RAM, almost like it was caught in a loop.

When I reduce the l_max it's successful in either order, but with a large l_max it's only successful in the adstock -> saturation order.

If I remember correctly just performing adstock with a large l_max would result in an issue. I can test it again tomorrow if you like.

[wd60622]

If I remember correctly just performing adstock with a large l_max would result in an issue. I can test it again tomorrow if you like.

All good! Thank you for summarizing your findings!

My only suggestion is to model weekly data in order to reduce the l_max by factor of 7. Maybe @juanitorduz has some additional tips?