Giter Site home page Giter Site logo

thedmsgroup / mautic-eb Goto Github PK

View Code? Open in Web Editor NEW
104.0 35.0 27.0 1.11 MB

Mautic + Elastic Beanstalk = Scalable marketing automation

Home Page: https://www.mautic.org

License: GNU General Public License v3.0

Shell 77.34% PHP 22.66%
mautic marketing-automation php aws-elastic-beanstalk

mautic-eb's Introduction

Mautic EB Latest Stable Version Build Status

Deploy Mautic in a scalable Amazon Elastic Beanstalk cluster.

Mautic and AWS

The goal here is to make it simple and safe to scale Mautic up to millions of leads per week, while maintaining HIPAA & PCI compliance. Other helpful services such as CloudFlare and Newrelic are supported, but optional.

Requirements

  1. AWS EB environment running PHP 7.1 with environment variables described below
  2. AWS RDS MySQL instance (Aurora recommended, preferably encrypted)
  3. AWS EFS Volume (used for shared media/spool/etc)
  4. Add the AmazonEC2ReadOnlyAccess policy to the aws-elasticbeanstalk-ec2-role (for the cron script to run on leading instance only)
  5. We recommend having the Elastic Beanstalk CLI installed locally.

Elastic Beanstalk Environment Variables

Required:

APP_URL               - The default full URL for your site.
DB_HOST               - RDS Host.
DB_HOST_RO            - Optional: RDS Host for read-only slave cluster.
DB_USER               - RDS User.
DB_PASSWD             - RDS Password.
DB_NAME               - RDS Database name.
DB_PORT               - RDS port.
SECRET_KEY            - Hash key for encryption.
MAILER_FROM_NAME      - Default "from" email name.
MAILER_FROM_EMAIL     - Default "from" email address.
EFS_DNS_NAME          - The full DNS of the EFS mount.

Optional:

REDIS_HOST               - Set to the path of your Redis server/cluster to use Redis for session storage. Helps avoid CSRF errors when scaling. See https://github.com/phpredis/phpredis
MAUTIC_INSTALL           - Set to "1" to initialize mautic for the first/cold deployment only!
MAUTIC_WORKERS           - Number of concurrent campaign trigger workers to run on the leading instance.
                         - Instead of MAUTIC_WORKERS, we reccomend using the following 3 variables now to better control your timing.
MAUTIC_WORKERS_KICKOFF   - Number of concurrent campaign trigger workers running kickoff actions only. Overrides MAUTIC_WORKERS when present.
MAUTIC_WORKERS_SCHEDULED - Number of concurrent campaign trigger workers running scheduled actions only. Overrides MAUTIC_WORKERS when present.
MAUTIC_WORKERS_INACTIVE  - Number of concurrent campaign trigger workers running inactive actions only. Overrides MAUTIC_WORKERS when present.
MAUTIC_WORKERS_SCHEDULED_MID - Optionally sets a minimum contact ID for the scheduled worker group (for performance at large scale). 
MAUTIC_WORKERS_KICKOFF_MID   - Optionally sets a minimum contact ID for the kickoff worker group (for performance at large scale). 
MAUTIC_WORKERS_INACTIVE_MID  - Optionally sets a minimum contact ID for the inactive worker group (for performance at large scale). 
NR_APPNAME               - Newrelic application name.
NR_APPID                 - Newrelic application ID number for deployment notifications.
NR_APM_INSTALL_KEY       - NewRelic install key for Application Monitoring.
NR_INF_INSTALL_KEY       - NewRelic install key for Infrastructure.
NR_API_KEY               - NewRelic API key (optional, for Health plugin).
IP_RESTRICT              - Restrict all UI (/s/) access by IP address/range, must also select URI.
IP_RESTRICT_URI          - The URI to which 403 IP Restriction should be sent.

Travis CI Environment Variables

You will need to set these up if you wish to deploy your fork to an Elastic Beanstalk environment automatically from Travis.

AWS_ACCESS_KEY_ID             - From your IAM console (keep hidden).
AWS_SECRET_ACCESS_KEY         - From your IAM console (keep hidden).
AWS_REGION                    - The region to deploy to (us-east-1).
AWS_EB_APP                    - App name to deploy (mautic-eb).
AWS_EB_ENV                    - Elastic beanstalk environment name (mautic-eb-dev).
ELASTIC_BEANSTALK_LABEL       - Label name of the new version (optional).
ELASTIC_BEANSTALK_DESCRIPTION - Description of the new version (optional). Defaults to the last commit message.

Mautic Customizations

File structure in /mautic_eb will be copied (destructively) into the /mautic folder on composer install or update. For configuration files and other changes that cannot be plugin-based. Similarly you can use /mautic_custom for customizations that are brand-specific. Third party plugins that use the "mautic-plugin" installer will have their folders symlinked into the correct location.

  • mautic_eb/* --> mautic/*
  • mautic_custom/* --> mautic/*
  • plugins/* --> mautic/plugins/*

Custom dependencies can be included in a root composer.custom

Local setup

Pretty much the same as working with Mautic core:

  1. Clone & composer:
git clone https://github.com/TheDMSGroup/mautic-eb.git
cd mautic-eb
composer install
  1. Set up local host at http://mautic.loc that points to the .../mautic-eb/mautic sub-folder.
  2. Browse to http://mautic.loc and go through the standard setup.

Local setup for plugin/mautic-eb development

  1. Clone & composer:
git clone https://github.com/TheDMSGroup/mautic-eb.git
cd mautic-eb
bash ./scripts/build-local.sh
  1. Set up local host at http://mautic.loc that points to the .../mautic-eb/mautic sub-folder.
  2. Browse to http://mautic.loc and go through the standard setup.

Traditional deployment (from local)

Should you not wish to use Travis to deploy for you, you can do it manually from your local machine:

  • Make sure you've set up the Requirements above.
  • run bash ./scripts/build.sh
  • run eb init if you haven't already.
  • run eb deploy

Local tips and commands

  • Important: Do not run composer install from within the ./mautic folder, only in the root project folder.
  • composer cc to clear all Mautic/Symfony caches.
  • composer custom to update symlinks for all customizations to mautic core (from mautic_eb and mautic_custom).
  • composer less to compile LESS styles (should you need to extend or modify the core styles using SCSS).
  • composer assets to regenerate core CSS and JS files for deployment (should you need to modify core JS or SCSS).
  • composer test to run the full codeception suite.
  • The file composer.custom can be created for third-party dependencies/plugins/customizations

Additional Plugins / Customizations

Need to process a million new contacts every day? Need to add a custom integration every day, without writing code?

These are the kinds of requirements we have in Performance Marketing. With that in mind, we include optional plugins to augment this build. You can access them all at once by renaming composer.custom.dev to composer.custom and running composer install.

mautic-eb's People

Contributors

cykonetic avatar heathdutton avatar scottshipman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mautic-eb's Issues

Cloudflare no longer support mod_cloudflare. Need to switch to mod_remoteip?

What type of report is this:
Bug report

Description:

Getting this today in Elastic Beanstalk. It seems the perhaps Cloudflare has removed support for this mod and you can't download that file from their site anymore.

Application deployment failed at 2020-04-15T17:04:02Z with exit status 8 and error: command 20_cloudflare in .ebextensions/20_cloudflare.config failed.

--2020-04-15 17:00:57-- https://www.cloudflare.com/static/misc/mod_cloudflare/mod_cloudflare.c
Resolving www.cloudflare.com (www.cloudflare.com)... 104.17.210.9, ...
Connecting to www.cloudflare.com (www.cloudflare.com)|104.17.210.9|:443... connected.
HTTP request sent, awaiting response... 522
2020-04-15 17:04:02 ERROR 522: (no description).

You can see that mod_cloudflare.c has been deprecated here:
https://support.cloudflare.com/hc/en-us/articles/200170786-Restoring-original-visitor-IPs-Logging-visitor-IP-addresses-with-mod-cloudflare-
and here:
https://github.com/cloudflare/mod_cloudflare

Looks like we need to switch over to mod_remoteip.

I would love some direction on how we might go about doing that.

Problem composer install-path

What type of report is this:

Q A
Bug report? Y
Feature request?
Enhancement?

Description:

The installation of the repositoriy thedmsgroup/mautic-eb-custom into the custom installer path does not work for me. I used exactly the composer.custom that is delivered with your repo. There is nor error, but the directory mautic_custom is empty after deployment. I recognized this when trying it with my own customization repo for Mautic. After reading the docs everything seems to be OK, we need mnsami/composer-custom-directory-installer which is already required. I could not find any error. Do you have the composer.custom in use?

If a bug:

Q A
Mautic version 2.15.0
PHP version 7.2

Steps to reproduce:

  1. Rename composer.custom.dev to composer.custom
  2. composer update
  3. Directory mautic_custom is empty

Log errors:

No errors

Prevent https redirection

Question

Description:

When running on AWS with Cloudfront, I would like the EB Loadbalancer to only listen to http. It seems that the nodes automatically redirect http to https so that the health checks fail. Is there any option to prevent the redirection on the nodes (I did not found any config file, mayba Mautic is doing this?)

Deploy command 74_mautic_plugins fails due to non-existing tables

What type of report is this:

Q A
Bug report? x
Feature request?
Enhancement?

Description:

Would love to see some more detailed install/deploy instructions for us new to AWS EBS. I had to guess quite a bit, and I still can't get mautic-eb to install properly. First it failed because EC2 wasn't able to connect to EFS, then same with RDS. Setting up a new security group with some options solved this.

Now however, the command 74_mautic_plugins errors because the mautic tables have not been added in the database.

Disclaimer: first time ever using AWS EBS.

Thanks for creating this project!

If a bug:

Q A
Mautic version origin/dev
PHP version 7.1

Steps to reproduce:

  1. eb init php 7.1, etc.
  2. eb create + envvars as in documentation. -> fails due to no access to EFS.
  3. add security group to environment to give access to EFS and RDS.
  4. eb deploy

Log errors:

[2019-06-04T15:27:21.998Z] INFO  [347]   - [Application update app-2_14_0-399-g3cbd-190604_172253@4/AppDeployStage0/EbExtensionPostBuild/Infra-EmbeddedPostBuild/postbuild_5_remente_email_mautic/Command 74_mautic_plugins] : Activity execution failed, because: 
  
  [Doctrine\DBAL\Exception\TableNotFoundException]
  An exception occurred while executing 'SELECT p0_.id AS id_0, p0_.name AS name_1, p0_.description AS description_2, p0_.is_missing AS is_missing_3, p0_.bundle AS bundle_4, p0_.version AS version_5, p0_.author AS author_6 FROM plugins p0_ ORDER BY p0_.name ASC':
  
  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'mautic.plugins' doesn't exist
  
  
  Exception trace:
  () at /var/app/ondeck/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:53
  Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException() at /var/app/ondeck/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:128
  Doctrine\DBAL\DBALException::driverExceptionDuringQuery() at /var/app/ondeck/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php:855
  Doctrine\DBAL\Connection->executeQuery() at /var/app/ondeck/vendor/doctrine/orm/lib/Doctrine/ORM/Query/Exec/SingleSelectExecutor.php:50
  Doctrine\ORM\Query\Exec\SingleSelectExecutor->execute() at /var/app/ondeck/vendor/doctrine/orm/lib/Doctrine/ORM/Query.php:321
  Doctrine\ORM\Query->_doExecute() at /var/app/ondeck/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:962
  Doctrine\ORM\AbstractQuery->executeIgnoreQueryCache() at /var/app/ondeck/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:917
  Doctrine\ORM\AbstractQuery->execute() at /var/app/ondeck/vendor/doctrine/orm/lib/Doctrine/ORM/AbstractQuery.php:720
  Doctrine\ORM\AbstractQuery->getResult() at /var/app/ondeck/mautic/app/bundles/CoreBundle/Entity/CommonRepository.php:368
  Mautic\CoreBundle\Entity\CommonRepository->getEntities() at /var/app/ondeck/mautic/app/bundles/PluginBundle/Entity/PluginRepository.php:51
  Mautic\PluginBundle\Entity\PluginRepository->getEntities() at /var/app/ondeck/mautic/app/bundles/CoreBundle/Model/AbstractCommonModel.php:226
  Mautic\CoreBundle\Model\AbstractCommonModel->getEntities() at /var/app/ondeck/mautic/app/bundles/PluginBundle/Model/PluginModel.php:108
  Mautic\PluginBundle\Model\PluginModel->getInstalledPlugins() at /var/app/ondeck/mautic/app/bundles/PluginBundle/Facade/ReloadFacade.php:44
  Mautic\PluginBundle\Facade\ReloadFacade->reloadPlugins() at /var/app/ondeck/mautic/app/bundles/PluginBundle/Command/ReloadCommand.php:44
  Mautic\PluginBundle\Command\ReloadCommand->execute() at /var/app/ondeck/vendor/symfony/console/Command/Command.php:245
  Symfony\Component\Console\Command\Command->run() at /var/app/ondeck/vendor/symfony/console/Application.php:853
  Symfony\Component\Console\Application->doRunCommand() at /var/app/ondeck/vendor/symfony/console/Application.php:185
  Symfony\Component\Console\Application->doRun() at /var/app/ondeck/vendor/symfony/framework-bundle/Console/Application.php:84
  Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/app/ondeck/vendor/symfony/console/Application.php:117
  Symfony\Component\Console\Application->run() at /var/app/ondeck/mautic/app/console:41

Upgrade to 2.16

What type of report is this:
Enhancement

Description:

I'm running into many of the bugs and issues reports in 2.15 that are now supposedly fixed in 2.16. I'd like to make this upgrade asap and am not exactly sure where to start.

Log errors:

Here are some of the errors I'm running into on 2.15 that are fixed by the referenced issues:
mautic/mautic#7106
mautic/mautic#3655
mautic/mautic#7756

Mautic's settings different between nodes

Description:

When changing a parameter in Mautic's setting that isn't defined in parameters_local.php it won't be "synced" between cluster nodes. Shouldn't all settings been shared via EFS volume between the nodes?

Mautic-Version: 2.15.0
PHP: 7.2

Steps to reproduce:

  1. Change a Mautic setting (e.g. CORS or MessageQueue)
  2. Setting is only changed on current cluster node.

How to install 3rd Party Plugin

What type of report is this:

Q A
Bug report?
Feature request?
Enhancement?
Question? yes

Description:

I like to install this plugin: https://github.com/Logicify/mautic-advanced-templates-bundle
I tried with:
git submodule add https://github.com/Logicify/mautic-advanced-templates-bundle.git plugins/mautic-advanced-templates-bundle

I removed the folder "plugins" from .gitignore.

After deployment, the plugin exists in the appropriate folder. But when clicking on "Install Plugin" it does not appear.

Any better way to install this plugin?

If a bug:

Q A
Mautic version 2.15.0
PHP version 7.2

Steps to reproduce:

  1. comment out /plugins in .gitignore
  2. run "git submodule add https://github.com/Logicify/mautic-advanced-templates-bundle.git plugins/mautic-advanced-templates-bundle" from mautic-eb folder
  3. deploy, check, if folder is in mautic/plugins (symlink is OK)
  4. Go to settings->plugins->Install Plugin
  5. no plugin will be installed

Log errors:

Dropbox Dependency

What type of report is this:

Q A
Bug report? Y
Feature request?
Enhancement?

Description:

Receiving error when running install script:

Failed to download dropbox/dropbox-sdk from dist: The "https://api.github.com/repos/dropbox/dropbox-sdk-php/zipball/0f1a178ca9c0271bca6426dde8f5a2241578deae" file could not be downloaded (HTTP/1.1 404 Not Found)

If a bug:

Q A
Mautic version
PHP version

Steps to reproduce:

  1. Clone Library
  2. Run install script

Log errors:

Please check for related errors in the latest log file in [mautic root]/app/log/ and/or the web server's logs and post them here. Be sure to remove sensitive information if applicable.

bash ./scripts/build-local.sh

Pulling mautic-eb
Already up to date.

Cleaning up the build space.

Setting composer.lock and composer.custom files from the dev coppies.

Custom install.
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 159 installs, 0 updates, 0 removals

[RuntimeException]
Failed to clone https://github.com/dropbox/dropbox-sdk-php.git via https, s
sh protocols, aborting.

install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] []...


Applying selected patches to Mautic core.
./scripts/core-patches.sh: line 17: cd: ./mautic: No such file or directory

Creating initial local.php
cp: ./mautic/app/config/local.php: No such file or directory

Re-cloning all custom plugins
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
/Users/user/Non-Cloud-Documents/Sites/mautic-eb

Compiling Mautic JS/CSS assets.

rm -rf ./mautic/vendor ./mautic/bin
cd ./mautic ; ln -s ../vendor vendor
sh: line 0: cd: ./mautic: No such file or directory
ln: vendor/vendor: File exists
Script cd ./mautic ; ln -s ../vendor vendor handling the custom event returned with error code 1
rm -rf ./mautic/media/css/app.css ./mautic/media/css/libraries.css
cd ./mautic ; npm install ; npm install grunt-cli ; grunt less
sh: line 0: cd: ./mautic: No such file or directory
npm WARN saveError ENOENT: no such file or directory, open '/Users/user/Non-Cloud-Documents/Sites/mautic-eb/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/Users/user/Non-Cloud-Documents/Sites/mautic-eb/package.json'
npm WARN mautic-eb No description
npm WARN mautic-eb No repository field.
npm WARN mautic-eb No README data
npm WARN mautic-eb No license field.

up to date in 0.454s
found 0 vulnerabilities

npm WARN saveError ENOENT: no such file or directory, open '/Users/user/Non-Cloud-Documents/Sites/mautic-eb/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/Users/user/Non-Cloud-Documents/Sites/mautic-eb/package.json'
npm WARN mautic-eb No description
npm WARN mautic-eb No repository field.
npm WARN mautic-eb No README data
npm WARN mautic-eb No license field.

  • [email protected]
    added 150 packages from 121 contributors and audited 984 packages in 4.346s

1 package is looking for funding
run npm fund for details

found 0 vulnerabilities

sh: grunt: command not found
Script cd ./mautic ; npm install ; npm install grunt-cli ; grunt less handling the less event returned with error code 127
Script @less was called via assets
Updated 1 path from the index

Is this a WIP

Hello,

We just found this and it looks like a great job and we are looking at using it. Is it ready for a production environment and if not how can we help?

Thanks for your time

William

email preference center and unsubscibe link have localhost domian

What type of report is this:

Q A
Bug report? Question/Bug
Feature request?
Enhancement?

Description:

When sending emails, the {unsubscribe_url} gets replaced by http://localhost/email/unsubscribe/5c9a004a9d4e7568890675

APP_URL ist set properly: host.mydomain.com

In local.php I see:
'site_url' => 'host.mydomain.com',

I did not find any reference to localhost. On a single host Mautic installation the url is correct, so I assume it's related to and AWS EB setting ...

If a bug:

Q A
Mautic version 2.15.0
PHP version 7.2

Steps to reproduce:

  1. Sending email through a real campaign (no test email)
  2. Clicking on unsubscibe link
  3. Manual replacing localhost with real hostname works

Log errors:

Add configuration to deny access to composer files, .git etc.

What type of report is this:

Q A
Bug report?
Feature request?
Enhancement? Y

Description:

When deploying as described a lot of config files would be public (composer.lock, composer.custom, composer.json, .git etc.). I think we would need an additional apache configuration file to deny access to those files.

If a bug:

Q A
Mautic version 2.15.0
PHP version 7.2

Steps to reproduce:

  1. Deploy to Elastic Beanstalk
  2. Access http://your-domain.com/.git/config or /composer.json etc.

Log errors:

No errors

Tracking script inserts values in extended fields

What type of report is this:

Q A
Bug report? possible bug
Feature request?
Enhancement?

Description:

I‘ve upgraded to 2.15.1 and I am using extended fields. Inserting and updating values work perfect via api. But when using the mtc.js tracking script, every pageview inserts a number in each extend field (like 23, 24). This only happens with extended fields not with contact fields.

If a bug:

Q A
Mautic version 2.15.1
PHP version

Steps to reproduce:

  1. activate extended field plugin
  2. using tracking script on the website mtc.js
  3. Mautic inserts numbers in each extended field

Log errors:

Please check for related errors in the latest log file in [mautic root]/app/log/ and/or the web server's logs and post them here. Be sure to remove sensitive information if applicable.

`dev-custom` branch stability issue

Hi, again,

I'm trying to use the 'dev-custom' branch that relies on patches. But the patches in core-patches.sh fails at many occasions.
How does work the patching system ?
Are we applying patches (gist, diffs) that mutates on constant code base (composer) ? or are we applying constant patches on a code base ?

Why, core-patches.sh needs to be run manually after installation ? This manual part impedes autoscaling to run effectively, right ?

Should I avoid using this branch at all until stabilized ? 

Cronjobs stopped working

Q A
Question

Description:

My autoscaling currently has only one instance running. Unfortunatly the cronjobs stopped working. Although there are entries in the cron log, no results are created (e.g. no campaigns are sent, no segments built etc.). What can I do for further debbugging? What could be the reasons? I am not shure when exactly they stopped because I am still in a test run - maybe it was after rebuilding the environment.

If a bug:

Q A
Mautic version 2.15.0
PHP version 7.2

Log errors:

Mar 25 16:32:01 ip-172-31- CROND[32675]: (root) CMD (cron mautic:ledger:report:reprocess >/dev/null 2>&1)
Mar 25 16:33:01 ip-172-31- CROND[1384]: (root) CMD (if [ ! -d /tmp/imports ]; then sudo ln -sf /efs/mautic/imports /tmp ; fi ; console mautic:import --limit=1200 --quiet >/dev/null 2>&1)
Mar 25 16:33:01 ip-172-31- CROND[1386]: (root) CMD (cron mautic:broadcasts:send --quiet >/dev/null 2>&1)
Mar 25 16:33:01 ip-172-31- CROND[1385]: (root) CMD (console mautic:emails:send --quiet >/dev/null 2>&1)
Mar 25 16:33:01 ip-172-31- CROND[1387]: (root) CMD (/usr/sbin/awslogs-nanny > /dev/null 2>&1)
Mar 25 16:34:01 ip-172-31- CROND[2626]: (root) CMD (cron mautic:social:monitoring >/dev/null 2>&1)
Mar 25 16:34:01 ip-172-31- CROND[2627]: (root) CMD (console mautic:emails:send --quiet >/dev/null 2>&1)
Mar 25 16:34:01 ip-172-31- CROND[2629]: (root) CMD (if [ ! -d /tmp/imports ]; then sudo ln -sf /efs/mautic/imports /tmp ; fi ; console mautic:import --limit=1200 --quiet >/dev/null 2>&1)
Mar 25 16:34:01 ip-172-31- CROND[2628]: (root) CMD (cron mautic:ledger:report:reprocess >/dev/null 2>&1)
Mar 25 16:34:01 ip-172-31- CROND[2630]: (root) CMD (/usr/sbin/awslogs-nanny > /dev/null 2>&1)

Themes on efs?

What type of report is this:

Q A
Bug report?
Feature request?
Enhancement?
Question? Yes

Description:

Currently, themes are stored locally on the nodes. Should we link the themes directory to EFS so that themes files become available across all nodes? Otherwise we cannot use Mautic's GUI but have to deploy themes ...

If a bug:

Q A
Mautic version 2.15.0
PHP version 7.2

Steps to reproduce:

Log errors:

Have you experienced EFS timeouts during bulk emailing?

Hi team and @heathdutton,

This is not about your repository, but we have quite a similar setup for Mautic. We are using file spool and we are experiencing not only a horrible performance, but also intermittent timeouts especially with a larger queue (20K, 30K messages in spool directory). EFS already takes at least 10 seconds to list the directory, which I believe is causing 451 timeout in the SMTP protocol.

I hate both file and memory spools, how are you managing this? I am going to experiment with some Redis hack to force Swiftmailer to use Redis as spool, but first, I have 30K emails stuck in the queue at the moment...

Any insights are welcome.

declare BASEDIR

BASEDIR=$(dirname "$BASH_SOURCE")
cd $BASEDIR/../
BASEDIR=$( pwd )

Use this for a assigning script path:

declare BASEDIR
BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# for full symlink resolution
# BASEDIR="$( realpath -qs "$( cd "$( dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd )")"

And possibly this at the start of bash scripts.

# "unofficial" bash strict mode
# See: http://redsymbol.net/articles/unofficial-bash-strict-mode
set -o errexit  # Exit when simple command fails               'set -e'
set -o errtrace # Exit on error inside any functions or subshells.
set -o nounset  # Trigger error when expanding unset variables 'set -u'
set -o pipefail # Do not hide errors within pipes              'set -o pipefail'
# set -o xtrace # Display expanded command and arguments       'set -x'
IFS=$'\n\t'     # Split words on \n\t rather than spaces

.

What type of report is this:

Q A
Bug report?
Feature request?
Enhancement?

Description:

The more details the better...

If a bug:

Q A
Mautic version
PHP version

Steps to reproduce:

Log errors:

Please check for related errors in the latest log file in [mautic root]/app/log/ and/or the web server's logs and post them here. Be sure to remove sensitive information if applicable.

How to ensure custom configuration for my project ?

Hey,
I feel confuse with the configuration process, because there are a lot of files involved, both stored on EFS and local EC2 storage.
How can I extend the master behavior in my fork by extending the mautic_eb/app/config/parameters_local.php system ?
How can I run a reconfiguration from the ENV variable without rebuilding a new EC2 host, or pushing a new eb deploy ?

Question: Autoscaling compatibility ?

Hey, my company is looking forward reseting our poorly deployed instance of Mautic after many failure on AWS Elastic Beanstalk. I'd like to validate with you that you solution is compliant with autoscaling.

Fact is that our first attempt failed and inserted random nonsense in the database after we started multiple instance with cronjob capabilities. It ended with many id constraints issues.

Can you explain me how you achieve « make it simple and safe to scale » as our experience was rather catastrophic on Elastic Beanstalk. Is that mainly related to the common EFS instance that you added in the mix, and the forced usage of spool ?
How do you achieve cron to « self regulate » ? Is there some risks ?

In the meantime, I'll try to dig your code to understand by myself.

Thanks for this promising work, we're really looking forward using you solution.

[Documentation] Please mention that cron script requires particular EC2 permissions

What type of report is this:

Q A
Feature request?

Description:

aws-eb-cron.sh used for running cron tasks calls AWS CLI : aws ec2 describe-instances.
This command requires non default permission to be assigned to aws-elasticbeanstalk-ec2-role.
Read more here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/iam-instanceprofile.html

You can add AmazonEC2ReadOnlyAccess policto to aws-elasticbeanstalk-ec2-role for script to work.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.