tolstislon/testrail-api

Python wrapper of the TestRail API

Testrail Api

This is a Python wrapper of the TestRail API according to the official documentation

Install

Install using pip with

pip install testrail-api

Support environment variables

TESTRAIL_URL=https://example.testrail.com/
TESTRAIL_EMAIL=[email protected]
TESTRAIL_PASSWORD=password

Example

from datetime import datetime

from testrail_api import TestRailAPI

api = TestRailAPI("https://example.testrail.com/", "[email protected]", "password")

# if use environment variables
# api = TestRailAPI()


new_milestone = api.milestones.add_milestone(
    project_id=1,
    name="New milestone",
    start_on=datetime.now()
)

my_test_run = api.runs.add_run(
    project_id=1,
    suite_id=2,
    name="My test run",
    include_all=True,
    milestone_id=new_milestone["id"]
)

result = api.results.add_result_for_case(
    run_id=my_test_run["id"],
    case_id=5,
    status_id=1,
    comment="Pass",
    version="1"
)
attach = "screenshots/attach.jpg"
api.attachments.add_attachment_to_result(result["id"], attach)

api.runs.close_run(my_test_run["id"])
api.milestones.update_milestone(new_milestone["id"], is_completed=True)

Custom response handler

from datetime import datetime
import simplejson

from testrail_api import TestRailAPI


def my_handler(response):
    if response.ok:
        return simplejson.loads(response.text)
    return 'Error'


api = TestRailAPI("https://example.testrail.com/", "[email protected]", "password", response_handler=my_handler)
new_milestone = api.milestones.add_milestone(
    project_id=1,
    name="New milestone",
    start_on=datetime.now()
)

Contributing

Contributions are very welcome.

Getting started

• python 3.11
• pipenv 2022.12.19+

1. Clone the repository

   git clone https://github.com/tolstislon/testrail-api.git
   cd testrail-api

2. Install dev dependencies

   pipenv install --dev
   pipenv shell

3. Run the black

   pipenv run black

4. Run the flake8

   pipenv run flake8

5. Run the tests

   pipenv run tests

testrail-api's Issues

Add support for Variables API methods

Documentation here: https://support.gurock.com/hc/en-us/articles/7077979742868-Variables

Adaptation is needed for "get_users" as it changed on TR side

Seems to be a bug in TR documentation
https://support.testrail.com/hc/en-us/articles/7077978310292-Users#getusers

Current output is a dictionary and has the same format as all methods that support bulk request (get_cases, get_tests etc.) and split by 250 users in output

The update_cases method appears incorrect/out-of-date

According to https://www.gurock.com/testrail/docs/api/reference/cases/#updatecases, the format of the request should be:

POST index.php?/api/v2/update_cases/{suite_id}

Meanwhile, the current implementation of this method uses the following code:

        params = {"suite_id": suite_id} if suite_id else {}
        return self._session.request(
            METHODS.POST,
            "update_case/{}".format(project_id),
            params=params,
            json=kwargs,
        )

This indicates that update_case is used as the path of the request, not update_cases. According to the documentation, it looks like project_id is not expected as the input here, and I imagine this can cause some unexpected results if done on a project whose ID matches the target suite. Furthermore, since case_ids is a required field for this method, it should be probably be declared and passed through explicitly.

Add ability to pass project_id when requesting list of users

Before TestRail 6.4 it was possible to retrieve list of users without administrative privileges, but from 6.4 it's not possible to retrieve users without administrative access or without specifying project_id in get_users.

Expected Result:
Successfully obtain list of users.

Actual Result:
403 Forbidden
testrail_api._exception.StatusCodeError: (403, 'Forbidden', 'https://testrail.com/index.php?/api/v2/get_users', b'{"error":"Access Denied. You are not a TestRail administrator. Field:project_id is a required field."}')

Runs _session.runs.add_run() for POST index.php?/api/v2/add_run/{project_id} not working as expected

Error message:

[2023-07-29 16:58:07,930][ERROR][_session]: Code: 400, reason: Bad Request url: https://hp-testrail.external.hp.com/index.php?/api/v2/add_run/466, content: b'{"error":"Field :suite_id is a required field."}'

The issue is the suite_id is not required parameter based on https://support.testrail.com/hc/en-us/articles/7077874763156-Runs#addrun, parameter. And the suite_id is provided in the payload. See the payload information next.

The payload is:
project id: 466
suite id: 65398
[2023-07-29 16:58:07,720][INFO ][tr_helper]: Create test run: TR Unit test 2023-07-29_16_58_07.715584-07_00 with payload {"suite_id": 65398, "name": "TR Unit test 2023-07-29_16_58_07.715584-07_00", "description": "Test run TR Unit test 2023-07-29_16_58_07.715584-07_00 with defined testcases", "include_all": false, "case_ids": [35076905, 35076906, 35076907, 35076908, 35076909, 35076910, 35076911, 35076912, 35076913, 35076914, 35076915, 35076916, 35076917, 35092638, 35092639, 35092640, 35092641, 35092642]}

The response is:
[2023-07-29 16:58:07,930][DEBUG][_session]: Response header: {'Date': 'Sat, 29 Jul 2023 23:58:07 GMT', 'Set-Cookie': 'tr_rememberme=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0, notificationbar=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/index, TS0104937c=01cc51d60f65ceaac8ccb9113c7ed2af29392e494590650e88f91f2816c687d5d132fa55b50c7630762e56a721ee5fd8b5fe216e01ac7687e66780f86fa40eaf200a3288354f4ee1b49b8a875c83435ec1f23f71ab; Path=/, TS01dcc28e=01cc51d60f0e31948020e5072e2a097b000496617890650e88f91f2816c687d5d132fa55b50c7630762e56a721ee5fd8b5fe216e01ac7687e66780f86fa40eaf200a3288356735f90794a3000489e519ab9613244003875726c359c8555bee4dc23636d04a; path=/index, TS98d9ab12029=08bf25d8b7ab2800d8d1f2bc61bed64729797a338741fd022c197bd5e265ad183568f1e636ead9949d74b04b7841d7a5; Max-Age=30; Path=/', 'Content-Length': '48', 'Connection': 'close', 'Content-Type': 'application/json; charset=utf-8', 'P3P': 'CP="{}"'}

FYI, the post request of add_run with the same endpoint and payload documented above works in the POSTMAN session.

Add support for Roles API methods

Documentation here: https://support.gurock.com/hc/en-us/articles/7077853258772-Roles

Issue with Plans.add_plan_entry

I use single repo style TestRail instance so suite_id is not needed.
When I use your API:

test_plan_response = api_client.plans.add_plan_entry(
                project_id=3,
                plan_id=None,
                suite_id=None,
                name="TEST",
                runs=runs,
)```

Here is the result:

File "/opt/homebrew/lib/python3.10/site-packages/testrail_api/_category.py", line 1059, in add_plan_entry
return self.s.post(

File "/opt/homebrew/lib/python3.10/site-packages/testrail_api/_session.py", line 171, in post
return self.request(

File "/opt/homebrew/lib/python3.10/site-packages/testrail_api/_session.py", line 215, in request
return response if raw else self.__response_handler(response)

File "/Users/steven/pytest-reporting/pytest_testrail/pytest_testrail/api.py", line 54, in response_handler
raise requests.HTTPError(f"Unexpected status code. {details}")

requests.exceptions.HTTPError: Unexpected status code. STATUS CODE RECEIVED: 400 - EXPECTED STATUSES: [200, 201]
REQUEST: <PreparedRequest [POST]> URL: '/index.php?/api/v2/add_plan_entry/None'

I shouldn't need to supply either plan_id or suite_id. Notice the None 
`'/index.php?/api/v2/add_plan_entry/None'`
The None should be project_id should be `3` since the TestRail API wants the project-id

Support for new bulk_api endpoints with pagination

The testrail API changed on cloud for the latest release with the formal move to pagination by default: https://discuss.gurock.com/t/bulk-api-endpoint-changes-coming-soon/19537

Is there a plan to update to support this , e.g. get_cases no longer returns a list of cases but a paginated object with a 'cases' key

Get Test Case Statuses

See https://www.gurock.com/testrail/docs/api/reference/statuses

Add the following function to class Statuses(_MetaCategory) to return a list of available test case statuses.

    def get_case_statuses(self) -> List[dict]:
        """
        Returns a list of available test case statuses.

        :return: response
        """
        return self.s.get(endpoint="get_case_statuses")

is_completed in _category taking an int instead of bool, pytestrail.py takes bool instead of int

I am having trouble using the get_runs(projectid=pid, is_completed=False), I believe False is a bool

I get:
testrail_api._exception.StatusCodeError: (400, 'Bad Request', 'https://testrail.link/index.php?/api/v2/get_runs/31&is_completed=False', b'{"error":"Field :is_completed is not a valid boolean."}')

_category line: 1148 get_runs
Line 1161 :key is_completed int

pytestail.py line: 339
Line 347 :type is_completed: bool
Line 356 :param is_completed: takes true/false

Sorry if i am missing something simple.

Latest version with logger bug

File "/builds/qa/portal-front-tests/venv/lib/python3.7/site-packages/testrail_plugin/testrail_plugin.py", line 60, in _report_overall_results self.reporter.report_overall_results(self.execution_timestamp) File "/builds/qa/portal-front-tests/venv/lib/python3.7/site-packages/cached_property.py", line 35, in __get value = obj.dict[self.func.name] = self.func(obj) File "/builds/qa/portal-front-tests/venv/lib/python3.7/site-packages/testrail_plugin/testrail_plugin.py", line 39, in reporter return CaseReporter(api_client=self.api_client, run_id=self.run_id) File "/builds/qa/portal-front-tests/venv/lib/python3.7/site-packages/cached_property.py", line 35, in get value = obj.dict[self.func.name] = self.func(obj) File "/builds/qa/portal-front-tests/venv/lib/python3.7/site-packages/testrail_plugin/testrail_plugin.py", line 24, in api_client self.testrail_config.password) File "/builds/qa/portal-front-tests/venv/lib/python3.7/site-packages/testrail_api/_session.py", line 54, in init log.info( AttributeError: 'NoneType' object has no attribute 'info'

Why did you need to add NullHandler?

Custom fields of type 'Multi-select' don't appear in case metadata.

It looks like my custom Multi-select fields aren't showing up in the api.cases.get_cases(project_id=project['id'], suite_id=suite['id']) dictionary.

I verified that this appears to be a Multi-select issue by creating a new Multi-select variable and verifying that it doesn't show up in the output, and then creating a new Checkbox item and verifying that it does show up.

Steps to reproduce are:

Create a custom item of type Multi-select and query a test case that it is an attribute of using:

import pprint;
project_id = <project_id>
suite_id = <suite_id>

for case in api.cases.get_cases(project_id=project_id, suite_id=suite_id):
    pprint.pprint(case)

Correct return type from Configurations API

Hello,
I love this library! Much better than Gurock's half-baked implementation.

I noticed that the return types for configuration mutation events (add_config, add_config_group, update_config_group, etc) all have a return type of None.

Gurock's documentation doesn't appear to mention this, but the response body does contain the modified entity in the response. This is especially important for scenarios such as "get_or_insert".

Are you opposed to changing these types, even though it doesn't strictly match Gurock's docs? I'd be happy to open a PR for this.

request: full response object in status code exception

I was wondering if it be ok to include the full response object in the exception along with the args currently there? The response has the headers, and request, which can make logging failures easier. It also might be nice to include some attributes on the exception.

class StatusCodeError(TestRailAPIError):
    def __init__(self, *args, **kwargs):
        super().__init__(*args)
        self.status_code = args[0]
        self.reason = args[1]
        self.url = args[2]
        self.content = args[3]
        self.response = kwargs.get('response')

existing args access still works, but you'd have attributes to hand onto.

>>> e = StatusCodeError(500, "bad", "url", "content", response={'a':1})
>>> e.args[0]
500
>>> e.status_code
500
>>> e.response
{'a': 1}
>>> raise e
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StatusCodeError: (500, 'bad', 'url', 'content')

Remove Python 3.7

Python 3.7 unsupported version. End-of-life: 2023-06-27 https://devguide.python.org/versions/#unsupported-versions

[update required] update_cases does not accept project_id now in latest APIs, it should be removed

Version: 1.11.1
Problem statement: When you try to update the fields for multiple tests using update_cases method, it does send the valid payload.
Current implementation is def update_cases( self, project_id: int, suite_id: Optional[int] = None, **kwargs )

But it should no longer use project_id as its no longer required as per new documentation of TestRail API.https://support.gurock.com/hc/en-us/articles/7077292642580-Cases#updatecases

I can see that the code present on github is using the latest format
def update_cases(self, case_ids: List[int], suite_id: int, **kwargs) -> dict:

testrail-api/testrail_api/_category.py

Line 504 in 97b32e7

def update_cases(self, case_ids: List[int], suite_id: int, **kwargs) -> dict:

But the zip archive still uses the previous format and thus the request does not success and we get error. When I try to install the pip package, it downloads the older code.

Maybe there is no new build after this change or the last one failed not sure. But you can check in the archives, it still uses the previous version code and is not latest.

Can you please rebuild the project with the code present on current repository.

Thanks!

PYTEST_CURRENT_TEST KeyError during request

There is a race condition in which pytest unsets PYTEST_CURRENT_TEST while urllib is iterating over the environment dictionary. This causes a KeyError to be left unhandled.

Documentation error in .add_attachment_to_result()

Hi there, Thanks for for this library, its a god send.

but I've noticed a small error in the example and possibly in the doc string.

The example states the following:

result = api.results.add_result_for_case(
    run_id=my_test_run["id"], 
    case_id=5, 
    status_id=1, 
    comment="Pass", 
    version="1"
)
attach = "attach.jpg"
api.attachments.add_attachment_to_result(result[0]["id"], attach)

But the type returned from api.results.add_result_for_case() is a dictionary, not a list containing a dictionary. Therefore result[0]["id"] produces a key error on the [0]

This seems to extend to the doc strings as pycharm also complains about the returned type, despite the code running without issue.

Safe adding/removing test cases to test runs

The original testrail api and this package are desgined to add or remove test cases from test runs through update call, which requires to send an array of test case ids including the ones that DO NOT need to be changed plus/minus the ones to be added or removed.
This desgin makes update of test run relatively unsafe: assume we have a legacy test run with many results of different versions on it and we are adding a few new test cases to that run once in a while. In this situation, if the user MISTANKENLY sends the update call w/o including the original test cases in the run, the entire run will be removed and only a few to-be-added test cases will remain in that.
Suggestions:
Adding two functions, besides test run update, for adding and removing test cases, where we can concatenate the arrays of test case ids before sending them.

Add support for Datasets API methods

Documentation here: https://support.gurock.com/hc/en-us/articles/7077300491540-Datasets

Add support for Groups API methods

Documentation here: https://support.gurock.com/hc/en-us/articles/7077338821012-Groups

update_run_in_plan_entry doesn't work

Accroding to https://support.gurock.com/hc/en-us/articles/7077711537684-Plans#updateruninplanentry URL is

POST index.php?/api/v2/update_run_in_plan_entry/{run_id}

but in the python code it's

return self.s.post(
   endpoint=f"update_run_in_plan_entry/{plan_id}/{run_id}", json=kwargs
)

As result, currently error like Field :run_id is not a valid test run.

setup via requirements.txt

Hi,

Is there some known issue related to installing testrail-api via requirements.txt ?
I have a requirements.txt file, executed by pip3 install -r requirements.txt which includes a few packages, and in addition: testrail_api==1.5.0
All the packages are collected and installed, except of this one. I'm getting the following error:

 Could not find a version that satisfies the requirement testrail_api==1.5.0 (from -r requirements.txt (line 10)) (from versions: )
No matching distribution found for testrail_api==1.5.0 (from -r requirements.txt (line 10))

Already tried all the variations of testrail_api, testrail-api, 1.5, 1.5.0, etc.
Any ideas?

Thanks!

Add support for BDD methods

Documentation here: https://support.gurock.com/hc/en-us/articles/7832161593620-BDDs

feature request: consider custom exceptions for api retry

I've been hitting connection errors that I'm fairly certain are coming from testrail. I'd really like to just use the retry mechanism already available by passing an iter of exceptions to the constructor, or maybe set a property - doesn't matter to me.

Happy to create a PR if it sounds useful/worth it.

add warnings.warn about limited write operations using HTTP only

This is a suggested feature: add a warnings.warn message if the user connects to a Testrail server using HTTP (and not HTTPS).

The Testrail server returns a misleading error when using HTTP-only and attempting an API write operation like add_case.
For example, an HTTP POST to /index.php?/api/v2/add_case/1234 returns the confusing

404 File Not Found
...
{"error":"Unsupported HTTP method \"get\" for this action."}

However, when using HTTPS, the same add_case API request succeed. The same problem occurs with add_section and similar APIs.

This wasted a lot of my time since the error message hint is not adequate. If a string with leading http:// is passed then printing a warning warnings.warn("Using HTTP and not HTTPS may cause writeable API requests to return 404 errors") might save other users some grief.