opalstack/opalstack-python

Opalstack Python API Library

opalstack-python's Issues

clone_wp_site.py: Database config not updated properly

clone_wp_site.py does not update the cloned wp-config to point to the dst database. And so the dst app unintentionally reads and writes to the src database.

This means that when the script updates the siteurl and home options through the wp-cli tool, it modifies the src database, even though it's intended for the dst app.

opalstack-python/examples/clone_wp_site.py

Lines 159 to 161 in 13642f1

	log.info(f'Updating site url')
	sshrunner.run_passbased_ssh(f'/home/{DST_OSUSER_NAME}/bin/wp --path={dst_app_path} option set siteurl {site_url}')
	sshrunner.run_passbased_ssh(f'/home/{DST_OSUSER_NAME}/bin/wp --path={dst_app_path} option set home {site_url}')

So you end up with the src app redirecting to the dst domain, which is serving the dst app, but is still using the src database.

API endpoint methods should return response objects

Issue:

Endpoint methods currently return the JSON result from the API as a python object.

This means all other information regarding the response is lost, such as the status code. It also means the return type is unknown as it can sometimes be a list and other times a dict.

This leads to the issue that the only way to determine if a response was a 400 or 200 is by inspecting the returned result.

Proposed solution:

Return response objects with either the content attribute decoded and replaced in place or with another attribute result with the decoded JSON. the decoded json content is available with response.json()

I am happy to make a PR for this but it's a backwards incompatible change!

Perhaps there is a way to make it backwards compatible by returning an object of a custom class that behaves as the list/dict but with extra attributes for the response object. But this sounds complicated =]

clone_wp_site.py: App config not set

An app created by clone_wp_site.py will have an "empty" config.

Some consequences:

• The app is on PHP 7.3.33, which causes Wordpress to display a warning.
• "Manage Site URL" is missing.

I expected the script to clone the config from the src app, or use defaults that matched dashboard-created apps.

1. Would you consider updating clone_wp_site.py to show how to clone the src config?
2. Is it practical for the create_one function, or the installer scripts themselves, to set defaults when values are not provided?

Thanks,
Chad

For reference, this is the current app creation code.

opalstack-python/examples/clone_wp_site.py

Lines 108 to 115 in 13642f1

	# Create app
	log.info(f'Creating application {DST_APP_NAME}')
	created_app = opalapi.apps.create_one({
	'name': DST_APP_NAME,
	'osuser': dst_osuser['id'],
	'type': 'APA',
	'installer_url': 'https://raw.githubusercontent.com/opalstack/installers/master/core/wordpress/install.sh'
	})

400 and 404 responses should not raise exceptions when they are expected

Issue:

In my application I need to be able to handle when the API returns with a 400 status code for a "create" endpoint so I can display the returned form errors to the user.

Currently when a 400 is returned a RuntimeError is raised with a message along the lines of Unexpected status_code: 400, result: {"non_field_errors": ...} this means the result is swallowed along with the status code.

In this case I don't think any exception should be raised because we can expect a 400 as a valid response for the API.

The same is true for when a 404 is returned for a "delete" or "update" endpoint.

Proposed solution:

Update the appropriate calls to http_post_result throughout and add the appropriate status in the ensure_status kwarg.

I can make a PR for this but it's not as simple as this. There are considerations at least for when wait=True in ApiModelManager.create and I expect there will be more. If I can get a test API key so I can run the tests I'll gladly proceed.

clone_wp_site.py: "No such file or directory" error

Hi, I'm getting an error using clone_wp_site.py to clone an app locally.

Traceback (most recent call last):
  ...
  File "clone_mysite.py", line 156, in main
    mariatool_remote.import_remote_db(sshrunner, sql_filepath_remote)
  ...
RuntimeError: ... /home/myuser/mysite.sql: No such file or directory

Full log

[myuser@opal1 ~]$ python3 clone_mysite.py 
2022-09-29 15:21:00 : INFO  : Retrieving webserver information for opal1.opalstack.com
2022-09-29 15:21:01 : INFO  : Retrieving existing osuser myuser
2022-09-29 15:21:02 : INFO  : Creating domain test.mysite.com
2022-09-29 15:21:13 : INFO  : Creating application mysite
2022-09-29 15:21:35 : INFO  : Creating site mysite
2022-09-29 15:21:41 : INFO  : Retrieving database configurations
2022-09-29 15:21:45 : INFO  : Exporting database
2022-09-29 15:21:46 : INFO  : Copying files
2022-09-29 15:21:49 : INFO  : Copying database content
2022-09-29 15:21:49 : INFO  : Importing database
Traceback (most recent call last):
  File "clone_mysite.py", line 170, in <module>
    main(args)
  File "clone_mysite.py", line 156, in main
    mariatool_remote.import_remote_db(sshrunner, sql_filepath_remote)
  File "/usr/local/lib/python3.6/site-packages/opalstack/util.py", line 361, in import_remote_db
    sshrunner.run_ssh(cmd)
  File "/usr/local/lib/python3.6/site-packages/opalstack/util.py", line 259, in run_ssh
    return self.run_passbased_ssh(remote_cmd, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/opalstack/util.py", line 144, in run_passbased_ssh
    return self.run_via_sshpass(prelude + [remote_cmd], *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/opalstack/util.py", line 133, in run_via_sshpass
    return run(prelude + cmd, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/opalstack/util.py", line 23, in run
    raise RuntimeError(f'Command "{cmd}" exited with status {p.returncode}. Stderr: {stderr}')
RuntimeError: Command "['sshpass', '-f', 'myuser.sshpass', '/usr/bin/ssh', '-q', '-o', 'PasswordAuthentication=yes', '-o', 'PubkeyAuthentication=no', '-o', 'StrictHostKeyChecking=no', '[[email protected]](mailto:[email protected])', 'mysql --defaults-extra-file=/home/myuser/mysite.sqlpasswd -u mysite_12345678 mysite_12345678 < /home/myuser/mysite.sql']" exited with status 1. Stderr: bash: /home/myuser/mysite.sql: No such file or directory

When src and dst are the same server, line 152 copies the exported sql file to itself, line 153 deletes it, and then line 156 fails when it tries to import the missing file.

opalstack-python/examples/clone_wp_site.py

Lines 151 to 157 in 13642f1

	log.info(f'Copying database content')
	sshrunner.run_passbased_scp(sql_filepath_local, f'{userhost}:')
	os.remove(sql_filepath_local)
	log.info(f'Importing database')
	mariatool_remote.import_remote_db(sshrunner, sql_filepath_remote)
	sshrunner.run_passbased_ssh(f'rm {sql_filepath_remote}')