mozilla-services/autopush-integration-tests

INACTIVE - Deployment verification and integration tests for Mozilla autopush server

autopush-integration-tests's Issues

Update Jenkinsfile to new format

see:
https://github.com/Kinto/kinto-integration-tests/blob/master/Jenkinsfile

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that matches requests==2.14.2,>=2.20.0
Tried: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.0, 2.0.1, 2.0.1, 2.1.0, 2.1.0, 2.2.0, 2.2.0, 2.2.1, 2.2.1, 2.3.0, 2.3.0, 2.4.0, 2.4.0, 2.4.1, 2.4.1, 2.4.2, 2.4.2, 2.4.3, 2.4.3, 2.5.0, 2.5.0, 2.5.1, 2.5.1, 2.5.2, 2.5.2, 2.5.3, 2.5.3, 2.6.0, 2.6.0, 2.6.1, 2.6.1, 2.6.2, 2.6.2, 2.7.0, 2.7.0, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 2.10.0, 2.10.0, 2.11.0, 2.11.0, 2.11.1, 2.11.1, 2.12.0, 2.12.0, 2.12.1, 2.12.1, 2.12.2, 2.12.2, 2.12.3, 2.12.3, 2.12.4, 2.12.4, 2.12.5, 2.12.5, 2.13.0, 2.13.0, 2.14.0, 2.14.0, 2.14.1, 2.14.1, 2.14.2, 2.14.2, 2.15.1, 2.15.1, 2.16.0, 2.16.0, 2.16.1, 2.16.1, 2.16.2, 2.16.2, 2.16.3, 2.16.3, 2.16.4, 2.16.4, 2.16.5, 2.16.5, 2.17.0, 2.17.0, 2.17.1, 2.17.1, 2.17.2, 2.17.2, 2.17.3, 2.17.3, 2.18.0, 2.18.0, 2.18.1, 2.18.1, 2.18.2, 2.18.2, 2.18.3, 2.18.3, 2.18.4, 2.18.4, 2.19.0, 2.19.0, 2.19.1, 2.19.1, 2.20.0, 2.20.0
There are incompatible versions in the resolved dependencies.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Add travis

sudo false
language: python
python: 3.5
env:
  - TOXENV-flake8
install:
  - pip install tox
script:
  - tox

Migrate service-workers test

mozilla-services/services-test#89
which is here:
https://github.com/kitcambridge/services-test/tree/kit/autopush-integration

This test uses MarionetteJS to launch the browser. MarionetteJS should be replaced by MarionettePython -or- even just Selenium (since we aren't making use of the browser chrome afaik).

Once test is migrated, pls delete PR from services-test repo

Add tox and tox.ini

Add a URL check test as test template

Add Android VAPID test

This could either run headless in a Docker container or on a real device via Sauce labs.
see:
https://bugzilla.mozilla.org/show_bug.cgi?id=1363079#c5

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please reach out to [email protected].

(Message COC001)

Add a LICENSE file

This Mozilla repository has been identified as lacking a LICENSE.md file. This repository does have licensing information in the README.md file. To make it easier for users (and scanning tools) to find licensing information please add a LICENSE.md file with that information to the root directory of the project.

Mozilla staff can access more information in our Software Licensing Runbook – search for “Licensing Runbook” in Confluence to find it.

If you have any questions you can contact Daniel Nazer who can be reached at dnazer on Mozilla email or Slack.

READMELIC-2023-01

Travis CI free usage ends Dec 3; mozilla repos should switch to other CI platforms

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

Requirements updates

Fix deps per pyup

Remove deprecated config files

see if application.json,dev.json,stage.json,prod.json

remove FXA vars in prefs.ini

NOTE:
also, once hosts are passed dynamically, (see: https://bugzilla.mozilla.org/show_bug.cgi?id=1407019), we can also remove prefs.ini

Add bash script for loadtest

This will be replaced by proper pytest

Add a DNS propagation polling "stage" to the Jenksfile

On STAGE, DNS propagates almost immediately so when Ops triggers tests, they can start to run without incident. On PROD, however, DNS propagation can take much longer. We need a polling mechanism in place to prevent tests from running prematurely. Add an initial stage to the Jenkinsfile for this purpose may be a good way to accomplish this.

Loop the URL_checker test could be used for this purpose:
https://github.com/rpappalax/autopush-integration-tests/blob/master/tests/url_checker.py

Add topics test scripts

we'll eventually replace these with proper pytests

Create service-workers (DOM API) test

There's already a test for testing service workers (DOM API) created by @kitcambridge, but written in MarionetteJS. Given that MarionetteJS is no longer supported, we'll want to convert this test to Marionnette Python or Selenium + fox-puppet.

Add URL test

Add iOS sync + FxA + push test

talk to @davehunt about pytest plugin

Add e2e-test

This will require fox-puppet. Eventually we may want to require a feature to look at browser console for further verification.
see:
https://bugzilla.mozilla.org/show_bug.cgi?id=1363079#c4

Migrate prefs.ini from mozilla-services/services-test

see:
https://github.com/mozilla-services/services-test/blob/master/autopush/prefs.ini

Add topics test

see:
https://bugzilla.mozilla.org/show_bug.cgi?id=1363079#c3

Modify basic and config scripts to handle dev deploys

Fix intermittent fail on IP check (test_sentry_check)

Dependabot can't resolve your Python dependency files

Dependabot can't resolve your Python dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

Could not find a version that matches requests==2.14.2,>=2.20.0
Tried: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.0, 2.0.1, 2.0.1, 2.1.0, 2.1.0, 2.2.0, 2.2.0, 2.2.1, 2.2.1, 2.3.0, 2.3.0, 2.4.0, 2.4.0, 2.4.1, 2.4.1, 2.4.2, 2.4.2, 2.4.3, 2.4.3, 2.5.0, 2.5.0, 2.5.1, 2.5.1, 2.5.2, 2.5.2, 2.5.3, 2.5.3, 2.6.0, 2.6.0, 2.6.1, 2.6.1, 2.6.2, 2.6.2, 2.7.0, 2.7.0, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 2.10.0, 2.10.0, 2.11.0, 2.11.0, 2.11.1, 2.11.1, 2.12.0, 2.12.0, 2.12.1, 2.12.1, 2.12.2, 2.12.2, 2.12.3, 2.12.3, 2.12.4, 2.12.4, 2.12.5, 2.12.5, 2.13.0, 2.13.0, 2.14.0, 2.14.0, 2.14.1, 2.14.1, 2.14.2, 2.14.2, 2.15.1, 2.15.1, 2.16.0, 2.16.0, 2.16.1, 2.16.1, 2.16.2, 2.16.2, 2.16.3, 2.16.3, 2.16.4, 2.16.4, 2.16.5, 2.16.5, 2.17.0, 2.17.0, 2.17.1, 2.17.1, 2.17.2, 2.17.2, 2.17.3, 2.17.3, 2.18.0, 2.18.0, 2.18.1, 2.18.1, 2.18.2, 2.18.2, 2.18.3, 2.18.3, 2.18.4, 2.18.4, 2.19.0, 2.19.0, 2.19.1, 2.19.1, 2.20.0, 2.20.0
There are incompatible versions in the resolved dependencies.

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

You can mention @dependabot in the comments below to contact the Dependabot team.

Move github call to test_url_checks

It shouldn't have session scope

Add sentry test

SETUP
add sentry token to Jenkins
add raven to requirements

TEST

1. verify matching IP
2. verify time window
3. resolve issue

see:

https://bugzilla.mozilla.org/show_bug.cgi?id=1363079#c2

autopush (master) job failing to run, on qa-preprod

@rpappalax are you still able to help with this? (or @karlht know whom might be?)

none of the autopush jobs on qa-preprod are passing; master fails with:

[Pipeline] { (Test)
[Pipeline] sh
06:15:24 [autopush_master-LY4IHD6FM7NR5WWOESTTUB2LW5QKTV2WTWASHQZN74WA2ZKDUMHQ] Running shell script
06:15:24 + /home/jenkins/workspace/autopush_master-LY4IHD6FM7NR5WWOESTTUB2LW5QKTV2WTWASHQZN74WA2ZKDUMHQ/run
06:15:24 /home/jenkins/workspace/autopush_master-LY4IHD6FM7NR5WWOESTTUB2LW5QKTV2WTWASHQZN74WA2ZKDUMHQ@tmp/durable-80498f79/script.sh: line 2: /home/jenkins/workspace/autopush_master-LY4IHD6FM7NR5WWOESTTUB2LW5QKTV2WTWASHQZN74WA2ZKDUMHQ/run: No such file or directory
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] mail
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

Could not update commit status, please check if your scan credentials belong to a member of the organization or a collaborator of the repository and repo:status scope is selected

ERROR: script returned exit code 1
Finished: FAILURE

https://qa-preprod-master.fxtest.jenkins.stage.mozaws.net/job/autopush/job/master/

Modify URL check to only query latest tag

Currently the URL check test queries all tags on github in order to have the latest 3 available. This was created in order to be able to create useful Release Notes and diff links into the Bugzilla deployment ticket:
https://bugzilla.mozilla.org/show_bug.cgi?id=1365770#c0
Since this step is unnecessary when no Bugzilla deploy ticket is created, it should be simplified to only grabbing the latest tag.

Add Jenkinsfile

Add (non-scaled) API testing

We will run tests using https://github.com/mozilla-services/ap-loadtester
with run forever set to false (so the APIs get hit 1 time).
This test can only look for Tracebacks and/or Sentry errors at this time since ap-loadtester does not thru a local error if the API has an incorrect response.

For these we only need the mult-scenario called "api-test":
https://github.com/mozilla-services/ap-loadtester/blob/master/aplt/scenarios.py#L261-L284

see:
https://bugzilla.mozilla.org/show_bug.cgi?id=1363079#c6