Giter Site home page Giter Site logo

amp's Introduction

Download

sudo curl -LsS https://download.civicrm.org/amp/amp.phar -o /usr/local/bin/amp
sudo chmod +x /usr/local/bin/amp

Build and Test

The build and test processes are based on composer, phpunit, and box.

To facilitate local development and testing with multiple versions of php and mysqld (on Linux and macOS), the repo includes a few helpers: default.nix, ./scripts/run-tests.sh, and ./scripts/run-build.sh. These helper scripts require the nix package manager. Usage:

## Get the code
git clone https://github.com/amp-cli/amp
cd amp

## Start a shell with php+composer
nix-shell

## Run the test suites
./scripts/run-tests.sh

## Run the build, creating "dist/amp.phar"
./scripts/run-build.sh

About amp: Vision

amp is a tool to facilitate development of PHP web applications. The goal is to complement composer (and similar tools) by adding a (mostly) automated step to setup the database and webserver for newly downloaded code. For example, a developer checking out a project might say:

me@localhost:~/src$ composer create-project example/my-application --dev
me@localhost:~/src$ cd my-application
me@localhost:~/src/my-application$ ./bin/amp create --url=http://localhost:8003
URL: http://localhost:8003
Admin User: admin
Admin Password zFWx9D22

The my-application package depends on the amp package (using require-dev or suggest). The amp create step creates a new database in the local mysqld and a new virutal-host in the local httpd; then it writes out necessary credentials (eg the mysql username and password) to a config file.

Additional thoughts:

  • amp IS NOT a complete stack with bundled binaries (for PHP, MySQL, etc).
  • amp IS NOT a cluster-management tool for remote servers.
  • amp IS NOT a one-click installer.
  • amp IS NOT a system-administration suite.
  • amp is primarily AN INTERFACE to the local *AMP stack -- it aims to help application developers write their own install scripts.
  • amp aims to be PORTABLE -- to work with common PHP development environments such as Debian/Ubuntu, MAMP, XAMPP, or MacPorts.
  • amp is designed for DEVELOPMENT AND TESTING. If you need to automatically install copies of applications from source-code in a variety of environments (for integration-tests, test-fixtures, demos, bug-fixing, training, collaboration, etc), then amp can help.

About amp: Pre-Alpha Example

At time of writing, amp is in-development and doesn't fully meet its vision. In the third line, the developer shouldn't call amp create directly; rather, the author of my-application should include an install.sh script, and the downstream developer can run it:

me@localhost:~/src$ composer create-project example/my-application --dev
me@localhost:~/src$ cd my-application
me@localhost:~/src/my-application$ ./bin/amp config
me@localhost:~/src/my-application$ ./bin/install.sh
Login to the application:
 * URL: ${AMP_URL}
 * Username: admin
 * Password: default

The amp config command determines how to connect to MySQL and httpd. It may scan the local system for common configurations (Ubuntu vs MAMP vs MacPorts; Apache vs nginx), prompt the user for information, and retain the info (in ~/.amp) for future use.

The install.sh is mostly specific to the application, but it builds on amp to address the tedious bit about setting up mysqld and httpd. For example, one might say:

#!/bin/bash
set -e
APPDIR=`pwd`

## Create a new database and virtual-host
eval $(amp create --root="$APPDIR/web")
amp datadir "$APPDIR/log" "$APPDIR/cache"

## Load DB
cat $APPDIR/sql/install.sql | mysql -u$AMP_DB_USER -p$AMP_DB_PASS $AMP_DB_NAME

## Create config file
cat > $APPDIR/conf/my-application.ini <<MYCONFIG
[mysql]
username=${AMP_DB_USER}
password=${AMP_DB_PASS}
database=${AMP_DB_NAME}
hostname=${AMP_DB_HOST}
MYCONFIG

echo "Login to the application:"
echo " * URL: ${AMP_URL}"
echo " * Username: admin"
echo " * Password: default"

Backlog

See doc/backlog.md

FAQ

Q: Is amp stable? Should I rely on it right now?

A: Probably not. amp is pre-alpha. Interfaces and workflows are likely to change.

Q: How do I configure amp to work on my system?

A: Run `amp config

Q: How do I know if amp is working?

A: Run amp test

Q: How does amp assign a virtual hostname and port?

A: You can specify one by passing the --url option to create. If omitted, it will use localhost and assign an alternative port.

Q: How does amp name databases and database users?

A: The name is computed by taking the directory name (eg my-application) and appending some random characters. The directory name may be truncated to meet MySQL's size limits. The name is the same for the DB and user.

Q: Where does amp store its configuration data?

A: By default, ~/.amp. If you define the environment variable AMPHOME, it will store in the specified directory.

Q: I have five web apps installed. How does AMP distinguish them?

A: Each application should have its own directory (eg /home/me/src/my-application-1). By default, amp assumes that each directory corresponds to a single virtual-host and a single MySQL database. If you need an additional virtual-host and DB for that application, call create again with the --name argument. If you want an additional virtual-host XOR DB, specify --skip-db or --skip-url.

Q: How do I build a stand-alone PHAR executable for amp?

A: Install Box. Then, in the amp source dir, run "php -d phar.readonly=0 which box build"

Internal Architecture

amp uses components from Symfony 2 (eg Console, Config, and Dependency-Injection).

There are a few key services defined in the container:

  • db -- A service for creating and destroying MySQL DB's (based on DatabaseManagementInterface)
  • httpd -- A service for creating and destroying HTTP virtual-hosts (based on HttpdInterface)
  • perm -- A service for setting file permissions on data directories (based on PermissionInterface)
  • hosts -- A service for mapping hostnames to the local httpd (based on HostnameInterface)
  • instances -- A repository for CRUD'ing web-app instances (using the db and httpd services) which stores metadata in YAML (~/.app/instances.yml).

There may be competing implementations of db, httpd, hosts, and perm -- eg one implementation might connect to a remote mysqld while another launches a local mysqld on a ramdisk. These can be chosen at runtime by calling commands like:

## Set options interactively
amp config

## Set options individually
amp config:set --httpd_type=XXX
amp config:set --db_type=XXX
amp config:set --perm_type=XXX

## Set options en masse
amp config:set --httpd_type=XXX --db_type=XXX --perm_type=XXX

Parameters and services may be configured in amp's source-tree (app/defaults/services.yml) or in the local home directory (~/.amp/services.yml). Parameters entered through the CLI (amp config, amp config:set, etc) are stored in the local home directory (~/.amp/services.yml).

amp's People

Contributors

abdealiloko avatar dawnthorn avatar devarun avatar eileenmcnaughton avatar ejegg avatar mfb avatar michaelmcandrew avatar openbrian avatar seamuslee001 avatar tobiaslounsbury avatar totten avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

amp's Issues

Add ability to pass db user and password when creating (as we do with url etc)

Currently we have to pre-create a services.xml in order to pre-determine the db user name / password but it feels like civibuild should accept these as inputs. I understand the reasons for generating unique names but there are also good reasons for wanting consistent names in developer / ci environments at times

I believe the input is somewhere near here

https://github.com/amp-cli/amp/blob/master/src/Amp/Database/MySQL.php#L55

but I want to know there is not a reason NOT to add that option @totten

Explicitly identify common stacks/environments

There are several common stacks for doing local development -- e.g. Debian/Ubuntu, MAMP, XAMPP, MacPorts, etc.

At the start of the amp config interview, determine if we are in one of these environments. Subsequent suggestions/defaults can be tuned based on this.

(Note: This is not a new issue -- I just moved it from the README's backlog.)

nginx vhost view doesn't detect whether php is running on a port or a socket.

src/Amp/views/nginx-vhost.php assumes we are running php on port 9000 rather than on a socket. Is there any way that we can test for this and create an appropriate config? Maybe look for

listen = /var/run/php5-fpm.sock

in /etc/php5/fpm/pool.dfpm/pool.d/www.conf or similar?

If not trivial, we should potentially add this as a gotcha or similar in the documentation.

No random characters at the end of database name

Is it possible to omit the random characters of the database name created by amp? I have a few database installed with amp and it is hard to remember each random suffix to use in mysql workbench during development and in my case I have made sure that the prefix is unique, e.g. contains a project name

Issues running under Ubuntu with an encrypted home directory, need a way to move out of $HOME

I use an encrypted home directory which www-data cannot read.
I was able to run fine if I changed:
https://github.com/totten/amp/blob/master/src/Amp/Application.php#L31
to $appDir = '/my/permissionedfolder/amp'; EG: $appDir = '/var/www/vhosts/amp';

I also had to move civicrm-buildkit out of my home folder as well since the test includes the vendor autoload from there.
Perhaps you need a way for users to move out of the home folder if necessary like check for an environment variable such as $AMP_HOME and default to to the user home if it is not set?
Great utility by the way!

Clarity / documentation

Just noting that I found I was able to do

amp config:set --mysql_dsn=mysql:://root@database:3306

but

a) it's not in the read me and
b) it only appeared to work AFTER I'd run config::set with db_type

I think it makes sense to update the readme if that is the case

Also the 'pre-alpha' part of the readme is wrong IMHO given we use this daily

@totten

amp config failed to read :7979

I installed amp with buildkit and then I ran amp config and got the following services.yml :

parameters:
    version: 2
    mysql_dsn: 'mysql://root:<passwd>@localhost:3306'
    perm_type: worldWritable
    httpd_type: apache24
    db_type: mysql_dsn
services: {  }

This is a local mariadb installation and I am fairly certain the DSN is correct.
Now, when i run amp test I get:

ajk@ajk-ThinkPad-R60:~/buildkit$ amp test
Create test application
Create data directory: /home/ajk/.amp/canary/data/Av5ikz5lpujbK0fgNpOgMv5J1HOGj0aJ
AMP_URL='http://localhost:7979'
AMP_ROOT='/home/ajk/.amp/canary'
AMP_DB_DSN='mysql://canary_81ukt:[email protected]:3306/canary_81ukt?new_link=true'
AMP_DB_USER='canary_81ukt'
AMP_DB_PASS='gW9CB6OqPf8hUsix'
AMP_DB_HOST='127.0.0.1'
AMP_DB_PORT='3306'
AMP_DB_NAME='canary_81ukt'
AMP_DB_ARGS='-h 127.0.0.1 -u canary_81ukt -pgW9CB6OqPf8hUsix -P 3306 canary_81ukt'

Connect to test application
Expect response: "response-code-QY8FMePLvE"
PHP Warning:  file_get_contents(http://localhost:7979/index.php): failed to open stream: HTTP request failed! HTTP/1.0 500 Internal Server Error
 in phar:///home/ajk/buildkit/bin/amp/src/Amp/Command/TestCommand.php on line 153
PHP Stack trace:
PHP   1. {main}() /home/ajk/buildkit/bin/amp:0
PHP   2. require() /home/ajk/buildkit/bin/amp:10
PHP   3. Amp\Application::main() phar:///home/ajk/buildkit/bin/amp/bin/amp:18
PHP   4. Symfony\Component\Console\Application->run() phar:///home/ajk/buildkit/bin/amp/src/Amp/Application.php:49
PHP   5. Symfony\Component\Console\Application->doRun() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Application.php:124
PHP   6. Symfony\Component\Console\Application->doRunCommand() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Application.php:193
PHP   7. Symfony\Component\Console\Command\Command->run() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Application.php:889
PHP   8. Amp\Command\TestCommand->execute() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
PHP   9. Amp\Command\TestCommand->doPost() phar:///home/ajk/buildkit/bin/amp/src/Amp/Command/TestCommand.php:78
PHP  10. file_get_contents() phar:///home/ajk/buildkit/bin/amp/src/Amp/Command/TestCommand.php:153

Warning: file_get_contents(http://localhost:7979/index.php): failed to open stream: HTTP request failed! HTTP/1.0 500 Internal Server Error
 in phar:///home/ajk/buildkit/bin/amp/src/Amp/Command/TestCommand.php on line 153

Call Stack:
    0.0003     125200   1. {main}() /home/ajk/buildkit/bin/amp:0
    0.0227     240376   2. require('phar:///home/ajk/buildkit/bin/amp/bin/amp') /home/ajk/buildkit/bin/amp:10
    0.0336     488304   3. Amp\Application::main() phar:///home/ajk/buildkit/bin/amp/bin/amp:18
    0.2386    2502380   4. Symfony\Component\Console\Application->run() phar:///home/ajk/buildkit/bin/amp/src/Amp/Application.php:49
    0.2395    2505484   5. Symfony\Component\Console\Application->doRun() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Application.php:124
    0.2398    2505656   6. Symfony\Component\Console\Application->doRunCommand() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Application.php:193
    0.2399    2505200   7. Symfony\Component\Console\Command\Command->run() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Application.php:889
    0.2407    2506616   8. Amp\Command\TestCommand->execute() phar:///home/ajk/buildkit/bin/amp/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
    0.3187    2812180   9. Amp\Command\TestCommand->doPost() phar:///home/ajk/buildkit/bin/amp/src/Amp/Command/TestCommand.php:78
    0.3187    2813684  10. file_get_contents() phar:///home/ajk/buildkit/bin/amp/src/Amp/Command/TestCommand.php:153

Received incorrect response: ""
Tip: Try running "amp config" and/or restarting the webserver.
ajk@ajk-ThinkPad-R60:~/buildkit$ 

I am not sure what the issue is. The error message isn't very helpful.
I installed all the prerequisites using sudo apt-get install php5-cli php5-imap php5-ldap php5-curl php5-mysql php5-intl php5-gd php5-mcrypt php-apc apache2 libapache2-mod-php5 (as was written in the installer script for buildkit)

amp config fails for invalid mysql config

I was trying amp config and while running tried the "MySQL based on existing configuration" option.

Reproduce with:

  1. Install amp (I had done it using buildkit)
  2. Run amp config and used mysql_dsn.
  3. Ran amp test and it said it was not able to connect to the mysql
  4. Ran amp config again and tried mysql_cfg
  5. Now any amp command (including amp -h) throws an error.

The error I get:

ajk@ajk-ThinkPad-R60:~/buildkit$ amp config
PHP Fatal error:  Uncaught exception 'RuntimeException' with message 'Failed to locate MySQL service via ~/.my.cnf' in phar:///home/ajk/buildkit/bin/amp/src/Amp/Database/MySQLCnfFactory.php:16
Stack trace:
#0 [internal function]: Amp\Database\MySQLCnfFactory::get()
#1 phar:///home/ajk/buildkit/bin/amp/vendor/symfony/dependency-injection/Symfony/Component/DependencyInjection/ContainerBuilder.php(951): call_user_func_array(Array, Array)
#2 phar:///home/ajk/buildkit/bin/amp/vendor/symfony/dependency-injection/Symfony/Component/DependencyInjection/ContainerBuilder.php(488): Symfony\Component\DependencyInjection\ContainerBuilder->createService(Object(Symfony\Component\DependencyInjection\Definition), 'db.mysql_mycnf')
#3 phar:///home/ajk/buildkit/bin/amp/vendor/symfony/dependency-injection/Symfony/Component/DependencyInjection/ContainerBuilder.php(472): Symfony\Component\DependencyInjection\ContainerBuilder->get(Object(Symfony\Component\DependencyInjection\Alias))
#4 phar:///home/ajk/buildkit/bin/amp/vendor/symfony/dependency-injec in phar:///home/ajk/buildkit/bin/amp/src/Amp/Database/MySQLCnfFactory.php on line 16

Fatal error: Uncaught exception 'RuntimeException' with message 'Failed to locate MySQL service via ~/.my.cnf' in phar:///home/ajk/buildkit/bin/amp/src/Amp/Database/MySQLCnfFactory.php on line 16

RuntimeException: Failed to locate MySQL service via ~/.my.cnf in phar:///home/ajk/buildkit/bin/amp/src/Amp/Database/MySQLCnfFactory.php on line 16

Call Stack:
    0.0003     125104   1. {main}() /home/ajk/buildkit/bin/amp:0
    0.0227     240268   2. require('phar:///home/ajk/buildkit/bin/amp/bin/amp') /home/ajk/buildkit/bin/amp:10
    0.0332     488232   3. Amp\Application::main() phar:///home/ajk/buildkit/bin/amp/bin/amp:18
    0.0333     488872   4. Amp\Application->__construct() phar:///home/ajk/buildkit/bin/amp/src/Amp/Application.php:46
    0.1894    1669624   5. Amp\Application->createCommands() phar:///home/ajk/buildkit/bin/amp/src/Amp/Application.php:62
    0.2059    1918680   6. Symfony\Component\DependencyInjection\ContainerBuilder->get() phar:///home/ajk/buildkit/bin/amp/src/Amp/Application.php:152

Nginx support

I was getting 500s on front page - the following change makes NginX behave better for me.

Also provides NginX with a deny rule for sites/default/files/civicrm

amp config - Better advice on file permissions

Key points:

  1. The best options are "linuxAcl" and "osxAcl"
  2. "linuxAcl" may require updating your fstab, remounting, and installing "acl"
  3. "worldWriteable" is prone to creating files that cannot be edited by one user or another

Connection must originate on localhost

When I run 'amp test' I receive the error in the title.

It seems that this is because the script is checking the REMOTE_ADDR for 'localhost'. Expect because the request is coming from the command line the REMOTE_ADDR is not being set?

Unless there is something else that might be causing the 403 response at this point.

Any help debugging this would be great. Thanks.

NginX: Restrict access to sensitive directories

Discussed in #21

  • NginX does not respect .htaccess
  • Solution (until we move sensitive directories from webroot) is to use nginx config to prevent access
  • Directory paths are specific to CMS (eg sites//files/civicrm/ versus wp-content/something)
  • But since the risk of collision is low and AMP is targeted at local use, IMO it's OK to block the lot.

Update README.md

The README.md file needs updating, a lot of the text it contains seems to refer to a 'pre-a;pha' release, and the examples do not seem current anymore.

Check for Apache mod_rewrite

If you try to use Drupal without mod_rewrite, the site will nominally display, but you won't be able to login or do anything substantive. We should either:

  • Auto-enable it (amp config)
  • Test for mod_rewrite and warn if disabled (amp test)

Apache access denied without +x for user home directory

After setting up amp in Arch Linux running xampp/lampp I ran amp test and got

Warning: file_get_contents(http://localhost:7979/index.php): failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden
in phar:///home/colemanw/buildkit/bin/amp/src/Amp/Command/TestCommand.php on line 163

After a lot of trial-and-error with various apache settings I finally discovered this fix:

chmod +x /home/colemanw

And that solved it so my built sites are now accessible in the web browser.

Suggestion:
Can we add something about this to the output of amp test on failure? Particularly if the error message contains "403" and we can detect that the +x bit is not set on the home directory.

Warning: file_get_contents(http://localhost:7979/index.php): failed to open stream: Connection refused

I get the above error after running amp config and then amp test. I'm fairly sure the configuration values for amp config are correct. Any suggestion of where I could look to fix this?

The complete output of amp test is this:

as@as76:~$ amp test
Create test application
Create data directory: /home/as/.amp/canary/data/KvCDB9TIuN1rHpQHm0IqbMRjGwxLOrWo
AMP_URL='http://localhost:7979'
AMP_ROOT='/home/as/.amp/canary'
AMP_DB_DSN='mysql://canary_22i7b:[email protected]:3306/canary_22i7b?new_link=true'
AMP_DB_USER='canary_22i7b'
AMP_DB_PASS='TzClEIZZgAeHRpGP'
AMP_DB_HOST='127.0.0.1'
AMP_DB_PORT='3306'
AMP_DB_NAME='canary_22i7b'
AMP_DB_ARGS='-h 127.0.0.1 -u canary_22i7b -pTzClEIZZgAeHRpGP -P 3306 canary_22i7b'

Connect to test application
Expect response: "response-code-eAh4Poer5l"
PHP Warning:  file_get_contents(http://localhost:7979/index.php): failed to open stream: Connection refused in /opt/buildkit/vendor/totten/amp/src/Amp/Command/TestCommand.php on line 153

Warning: file_get_contents(http://localhost:7979/index.php): failed to open stream: Connection refused in /opt/buildkit/vendor/totten/amp/src/Amp/Command/TestCommand.php on line 153
Received incorrect response: ""
Tip: Try running "amp config" and/or restarting the webserver.

amp config - Clarify MySQL permissions

When prompting user for administrative credentials, specify the user credentials required.

Some developers instinctively want to add a new user. Buildkit will do this automatically for each build, but they still want to create a new user for buildkit per-se. We should display instructions for grantin permissions to this user, eg

GRANT ALL ON *.* to '$user'@'localhost' IDENTIFIED BY '$pass' WITH GRANT OPTION

Make apache vhost config 12 factor friendly

Hey there,

https://12factor.net/logs stats that you should treat logs as streams.

In the Docker apache image this is achieved by symlinking the apache log files to /dev/stderr and /dev/stdout` see this comment for more details.

By default amp defines its own log files for each site, which overrides the Docker config.

I know that amp isn't primarily aimed at Docker, and that https://12factor.net/logs is not the final word on app design, but I wanted to draw your attention to this in case you had any thoughts / it was helpful in thinking about how amp is designed.

It's not a blocker and I have got round it in the CiviCRM Docker image with a custom apache vhost template michaelmcandrew/civicrm-buildkit-docker@65a0971 (pending this PR).

amp config - Better advice on httpd

  1. Check for evidence of apache or nginx
  2. Check for evidence of apache 2.4+ (eg apachectl -v / apache2ctl -v)
  3. For Apache, check that mod_rewrite is enabled (eg apachectl -M)
  4. For Apache, check that htaccess support is enabled

Is app/defaults/services.yml distributed?

Is it correct to assume app/default/services is distributed? In other words if Amp is packaged, would it be distributed to every installation of Amp? I ask because a simple config command would not put ram_disk_type in my .amp repo, but Application.php assumes it's available.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.