mozilla-services/sync-tps-setup

Setup scripts to run https://developer.mozilla.org/en-US/docs/Mozilla/Projects/TPS_Tests on an Ubuntu 14.04 LTS host

sync-tps-setup's Issues

Consolidate run files into one file and parametrize

The two run files provided are identical, with the exception of the test environment (stage|prod) being executed. This information would be available to any Jenkins job as $TEST_ENV and would just need to be passed into the Dockerfile.

Optimize base docker image for tests

If we're running in headless mode, we can probably dispense with full-on Ubuntu in favor of a smaller base image

Add config to also run TPS locally with the buffered engine on STAGE

add this additional config to also run TPS locally with the buffered engine (STAGE only)
https://bugzilla.mozilla.org/show_bug.cgi?id=1433827#c3

Handling of secrets with GPG should be changed to sops or equivalent

Using my individual GPG key to encrypt the secrets in the config files is not a great way to do this.

Using sops would add another dependency, but I'm not sure of any other ways that get the balance of security vs. convenience right.

Parameterize STAGE/PROD configs and remove virtualenv

If we're passing in STAGE and PROD configs already, we don't need to setup (and maintain) separate venvs for both.

Investigate errors from "src/libcore/result.rs:906"

via the magnificent @rpappalax:

panicked at 'called `Result::unwrap()` on an `Err` value 
"No such file or directory"
/checkout/src/libcore/result.rs:906:4

"result.rs" sounds like maybe Rust? Not sure if we need full Firefox source, versus just the bits and pieces we're currently extracting from the ZIP file. But now armed with a starter error message above, it should hopefully be easier to Google and track down possible causes/solutions.

Create an nightly build Docker image of mozilla-central (source)

The sync TPS tests require a the latest nightly source code to run. In order to fully dockerize the tests, we'll need to provide a pre-built base docker image with this updated source code on it.

Move SYNC_TPS_EMAIL_RECIPIENT to credentials

we could leave this as an ENV var, but our team is restricted to adding creds. this works just as well.

Pass in json config as ENV var at runtime (not at build time)

Add --no-cache to Jenkinsfile

old images and configs are being cached on Jenkins. bad.

Migrate config files to LastPass and remove *.asc files

TPS Prod succeeds once, then fails thereafter

https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/sync.prod/19/consoleText succeeds.
Then jobs 20-22 fail.

I suspect this line:

sync-tps-setup/Jenkinsfile

Line 50 in 4b9d442

sh 'gzip tps.log'

needs a sh 'rm -f tps.log.gz' before it, because the output in
https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/sync.prod/21/consoleText

indicates gzip: tps.log.gz already exists; not overwritten

TEST_ENV missing when email notification on FAIL

true for both STAGE and PROD. fix Jenkinsfile

Dockerize sync TPS test(s)

For tests to run in our CI system, ideally we'd like to have them fully dockerized so our Jenkins slave hosts don't require special pre-installations / configurations. Though it is possible to setup custom Jenkins slaves, setting up such a host would be a one-off that would need to be provided by Ops if linked into our QAJenkins.

I'd recommend instead converting the Makefile provided here into a Dockerfile that installs the test environment and also allows execution of the tests. Separate docker images will eventually be created by our team, including one that has a daily update of mozilla-central source code that could either be mounted as a separate volume or used a base image to the sync-tps-setup test Dockerfile.

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please reach out to [email protected].

(Message COC001)