gocd/functional-tests

Smoke, acceptance and regression tests for GoCD

Pre-Requisites

• JDK 8 (verified on 1.8.0_172-b11)
• Gradle (verified to work on 4.6)
• Maven (verified to work on 3.5.4)
• Node 6.x (7.x will not work)
• Gauge 1.0.0
• Mercurial
• Ant
• Git
• Firefox <= 45.x (verified on 45.9.0esr; the version of webdriver used only supports up to 45)
• If running specs related to other SCM repos like TFS, SVN, Perforce ensure they are installed on the machine executing the specs

Setup

• Need the following repos cloned as sibling directories to this repo:
  • go.cd
  • go-plugins
• $ cd functional-tests
• $ gauge install
• $ gauge update --all (in case the plugins were already installed before the previous command, update them to the latest anyway)
• If you are using nvm, verify that Node 6.x is activated
• Modfiy the paths to firefox.exe or firefox-bin in src/test/java/twist.properties (for both the sahi.browserLocation and webdriver.firefox.bin properties) to reflect the paths on your machine

Clean/setup

• Build GoCD master repo and go-plugins repo from their respective folders

# This will prepare the functional tests related artifacts in the functional tests folder
$ cd functional-tests
$ rake clean setup

Running tests

$ GO_VERSION=X.x.x gauge run specs/AdminTaskListing.spec

Contributing

We encourage you to contribute to GoCD. For information on contributing to this project, please see our contributor's guide. A lot of useful information like links to user documentation, design documentation, mailing lists etc. can be found in the resources section.

License

Copyright 2018 ThoughtWorks, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

functional-tests's Issues

TFS test cases uses common TFS repo which cause conflict when 2 TFS related tests run in parallel

https://go05.thoughtworks.com/go/tab/build/detail/regression-postgresql/175/twist/1/firefox-split-runInstance-6

This job failed because the TFSMaterial test case was trying to change and push a file to TFS repo which had an updated copy of another file as shown in the log as below,

--OUTPUT ---
A newer version of $/for_twist_tests_1/AnotherHelloWorld.txt exists on the server.

No files checked in.
--- ERROR ---
ERROR: A resolvable conflict was flagged by the server: No files checked in: resolve the conflicts and try again.

---

Functional tests should be changed to use these scripts, and not their own.

gocd/gocd#1474

Write functional tests for Templates API

Functional test for clicking settings link on stage detail, job detail, pipeline history page.

See title

Fix functional test - 6 - Flaky - PackagePipelineScheduleApi.scn

@arvindsv commented on Wed Sep 23 2015

Happened here. Has happened before as well.

It seems to be expecting a label of 2, and the label is 1. Context (all the artifacts from that job) can be found here, just in case they get removed by Go.

---

@arvindsv commented on Sat Jun 04 2016

Closing. Haven't seen it happen.

AddNewPipeline.spec failing at config validation of `exec` task

AddNewPipeline.spec is failing only on the addons build environment, and not on build.go.cd. Refer to the regression pipeline on addons build environment(regression pipeline instance 709, run 1, Runinstance 4 of Firefox split job).

The application functionality is not broken as verified manually. Also the spec passed on build.go.cd environment. Looks like a test fix

Sample issue to check the template

Issue Type

Please specify the issue type to help us categorize the issue, mention any of the below type,

• Bug Report
• Feature proposal
• Feature enhancement
• Documentation Report

Sample issue to check the template

Summary

Provide a brief summary of the issue

Environment

Describe the environment in which you encountered this issue. Details of the environment GoCD is running in can be fetched from the GoCD support api - http:///api/go/support

Basic environment details

Below 3 info we recommend to be provided for us to quickly access the issue

Go Version:

JAVA Version:

OS:

Additional Environment Details

More environment details captured from the support API or other sources can be shared here

Steps to Reproduce

Provide steps to reproduce - How you encountered this issue?

Expected Results

From the steps mentioned above what did you expect to happen?

Actual Results

What actually happened?

Log snippets

If you could find any information/exceptions from the logs do provide it here. Do mask any confidential information which you don't want to be shared

Code snippets/Screenshots

if you would like to share any code/screenshots please add it here. Do mask any confidential information which you don't want to be shared

Any other info

Provide any other information you would like to share to explain this issue more better

Functional tests for all supported versions of TFS server

We support TFS server 2012, 2013 and 2015 for materials. The automated tests run against TFS server 2012 alone. Need the same set of tests to run for 2013 and 2015 too.
[And 2017 too when we officially start supporting it]