Linaro JIRA Update Tool
License: MIT License
Jipdate is a tool used at Linaro that makes it possible in one go to:
For all instructions (install, how to use, examples etc), please visit https://jipdate.readthedocs.io.
$ ./jipstatus.py --days 7 --html index.html
Robert Foss:
* Past
* [QLT-1325] ov8856: Build failure for arm - was done
* Issue localized to qcomlt branch
* Fixed issue in ov8856 tracking branch
* [QLT-1314] camss: VFE HW version log is spamming the log
* Submitted v1
[https://lkml.org/lkml/2021/7/26/121|https://lkml.org/lkml/2021/7/26/121|smart-link]
* [QLT-1240] sm8350: Display support
* Migrated to Jonathans dispcc driver
* [QLT-1148] drm-misc subsystem maintainership
* Applied some patches
* Reviewed it6505 driver
* Ongoing
* [QLT-1240] sm8350: Display support
* [QLT-1090] sm8350: Crypto
$ grep "QLT-1325" index.html
<li>[<a href="https://projects.linaro.org/browse/QLT-1325">QLT-1325</a>] ov8856: Build failure for arm - was done</li>
$ grep "QLT-1325" index.html
<li>[<a href="https://linaro.atlassian.net/browse/QLT-1325">QLT-1325</a>] ov8856: Build failure for arm - was done</li>
If you get the password or username wrong then JIRA will give you a 401 Unauthorized response, but jipdate doesn't handle it very gracefully:
$ ./jipdate.py -t -u peter.maydell -q -f /tmp/status.txt
Username ([email protected]): peter.maydell
Username not found in config.yml, want to store it? (y/n) n
Password:
WARNING:root:Got recoverable error from GET https://dev-projects.linaro.org/rest/api/2/serverInfo, will retry [1/3] in 19.6700271926s. Err: 401
WARNING:root:Got recoverable error from GET https://dev-projects.linaro.org/rest/api/2/serverInfo, will retry [2/3] in 33.6201041765s. Err: 401
WARNING:root:Got recoverable error from GET https://dev-projects.linaro.org/rest/api/2/serverInfo, will retry [3/3] in 53.8470725899s. Err: 401
Traceback (most recent call last):
File "./jipdate.py", line 602, in <module>
main(sys.argv)
File "./jipdate.py", line 575, in main
jira, username = get_jira_instance(g_args.t)
File "./jipdate.py", line 445, in get_jira_instance
return (JIRA(g_server, basic_auth=credentials), username)
File "/home/petmay01/.local/lib/python2.7/site-packages/jira/client.py", line 317, in __init__
si = self.server_info()
File "/home/petmay01/.local/lib/python2.7/site-packages/jira/client.py", line 1771, in server_info
j = self._get_json('serverInfo')
File "/home/petmay01/.local/lib/python2.7/site-packages/jira/client.py", line 2172, in _get_json
r = self._session.get(url, params=params)
File "/home/petmay01/.local/lib/python2.7/site-packages/jira/resilientsession.py", line 150, in get
return self.__verb('GET', url, **kwargs)
File "/home/petmay01/.local/lib/python2.7/site-packages/jira/resilientsession.py", line 146, in __verb
raise_on_error(response, verb=verb, **kwargs)
File "/home/petmay01/.local/lib/python2.7/site-packages/jira/resilientsession.py", line 56, in raise_on_error
r.status_code, error, r.url, request=request, response=r, **kwargs)
jira.exceptions.JIRAError: JiraError HTTP 401 url: https://dev-projects.linaro.org/rest/api/2/serverInfo
text:
<html>
<head>
<title>Unauthorized (401)</title>
...followed by pages and pages of raw HTML/Javascript.
It would be nice to recognize this error code and give the user a helpful error message like "Username or password not recognized".
From Linus W:
$ ./jipdate.py -q
/home/linus/jipdate/cfg.py:103: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
yml_config = yaml.load(yml)While I'm always eager to see magic involving cookies, in this case I'd just like Magic to have its cookie and let jipdate work:
$ ./jipdate.py -q -e
Password:
Exception ignored in: <bound method Magic.del of <magic.Magic object at 0x7f2733a1fd30>>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/magic/init.py", line 125, in del
if self.cookie and magic_close:
AttributeError: 'Magic' object has no attribute 'cookie'
After stepping up to Python3, we get an exception with an unknown parameter, like this there is an authentication failure.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./jipdate.py", line 494, in <module>
main(sys.argv)
File "./jipdate.py", line 466, in main
jira, username = jiralogin.get_jira_instance(cfg.args.t)
File "/home/jbech/devel/jipdate/jiralogin.py", line 107, in get_jira_instance
except (JIRAError, e):
NameError: name 'e' is not definedWe should take care of the exception in a nicer way. I.e., look at what is really available when getting a JIRAException.
The e should be removed, but doing that leads to another error (if CAPTCHA is about to be shown)
Traceback (most recent call last):
File "./jipdate.py", line 494, in <module>
main(sys.argv)
File "./jipdate.py", line 466, in main
jira, username = jiralogin.get_jira_instance(cfg.args.t)
File "/home/jbech/devel/jipdate/jiralogin.py", line 108, in get_jira_instance
if e.text.find('CAPTCHA_CHALLENGE') != -1:
NameError: name 'e' is not definedSome users of Jipdate would like to be able to log the time they spend on tickets via Jipdate. This could be added as yet another thing that we look for when parsing the query or user submitted file.
I have a wrong name to use with jipdate, it seems:
$ ./jipdate.py -q
Password:
Traceback (most recent call last):
File "./jipdate.py", line 458, in <module>
main(sys.argv)
File "./jipdate.py", line 442, in main
filename = get_jira_issues(jira, username)
File "./jipdate.py", line 213, in get_jira_issues
f.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 62: ordinal not in range(128)
Issue might lie in my last name: Díaz. It can also be any other thing, though, as I do have a tendency to throw in some accents left and right in my cards.
Both the textual and html output of jipstatus handle the raw textual contents of issue comments as pure text.
$ ./jipstatus.py --days 7 --html index.html
Robert Foss:
* Past
* [QLT-1314] camss: VFE HW version log is spamming the log
* Submitted v1
[https://lkml.org/lkml/2021/7/26/121|https://lkml.org/lkml/2021/7/26/121|smart-link]
[...]
$ grep "lkml" index.html
<li>Submitted v1<br><br>[https://lkml.org/lkml/2021/7/26/121|https://lkml.org/lkml/2021/7/26/121|smart-link] </li>
$ ./jipstatus.py --days 7 --html index.html
Robert Foss:
* Past
* [QLT-1314] camss: VFE HW version log is spamming the log
* Submitted v1
https://lkml.org/lkml/2021/7/26/121
[...]
$ grep "lkml" index.html
<li>Submitted v1<br><br><a href=""https://lkml.org/lkml/2021/7/26/121>https://lkml.org/lkml/2021/7/26/121</a></li>
jipdate seems to prompt for a username even if I specified it with the -u option:
$ ./jipdate.py -t -u peter.maydell -q -f /tmp/status.txt
Username ([email protected]):
./jipdate.py doesnt have an option for bugs and even with the option --all doesn't return the bugs
I configured jipdate with the new url/token stuff to access linaro.atlassian.net, and it does correctly seem to read from there, but it still prints a message saying it is going to update projects.linaro.org:
$ EDITOR=/bin/true [email protected] ~/linaro/jipdate/jipdate.py -v -f ~/jipdate.txt
[DEBUG] initiate_config():104 Using config file: /home/petmay01/.config/jipdate/.jipdate.yml
[DEBUG] get_jira_instance():114 Accessing https://linaro.atlassian.net with [email protected] using token based authentication
[DEBUG] _new_conn():975 Starting new HTTPS connection (1): linaro.atlassian.net:443
[DEBUG] _make_request():461 https://linaro.atlassian.net:443 "GET /rest/api/2/serverInfo HTTP/1.1" 200 None
[DEBUG] _make_request():461 https://linaro.atlassian.net:443 "GET /rest/api/2/field HTTP/1.1" 200 None
[DEBUG] _make_request():461 https://linaro.atlassian.net:443 "GET /rest/api/2/resolution HTTP/1.1" 200 None
[DEBUG] _make_request():461 https://linaro.atlassian.net:443 "GET /rest/api/2/issue/UM-2 HTTP/1.1" 200 None
[DEBUG] _make_request():461 https://linaro.atlassian.net:443 "GET /rest/api/2/issue/QEMU-406 HTTP/1.1" 200 None
These JIRA cards will be updated as follows:
[snip]
Server to update: https://projects.linaro.org
Are you sure you want to update Jira with the information above? [y/n]
(I haven't tried typing 'y' to see if it really will update projects.linaro.org or if it will actually update the atlassian server...)
We get an exception like this when running Jipdate as follows. It's been seen by two different users (Viresh and myself). I have logged in/out and I created a brand new token, but issue still persists.
$ ./jipdate.py -q --all
Traceback (most recent call last):
File "/home/jyx/devel/jipdate/./jipdate.py", line 543, in <module>
main(sys.argv)
File "/home/jyx/devel/jipdate/./jipdate.py", line 540, in main
parse_status_file(jira, filename, issues)
File "/home/jyx/devel/jipdate/./jipdate.py", line 290, in parse_status_file
resolution_map = dict([(t.name.title(), t.id) for t in jira.resolutions()])
File "/home/jyx/devel/pip-envs/jipdate/lib/python3.10/site-packages/jira/client.py", line 3003, in resolutions
r_json = self._get_json("resolution")
File "/home/jyx/devel/pip-envs/jipdate/lib/python3.10/site-packages/jira/client.py", line 3853, in _get_json
r = self._session.get(url, params=params)
File "/home/jyx/devel/pip-envs/jipdate/lib/python3.10/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/home/jyx/devel/pip-envs/jipdate/lib/python3.10/site-packages/jira/resilientsession.py", line 246, in request
elif raise_on_error(response, **processed_kwargs):
File "/home/jyx/devel/pip-envs/jipdate/lib/python3.10/site-packages/jira/resilientsession.py", line 71, in raise_on_error
raise JIRAError(
jira.exceptions.JIRAError: JiraError HTTP 406 url: https://linaro.atlassian.net/rest/api/2/resolution
+ response header
If you give jipdate -f file.txt a file which includes a section for a JIRA issue with no new comment text to add, jipdate will try to add an empty comment to the JIRA issue, provoking an HTTP 400 error from the server ("text: Comment body can not be empty!").
In more detail:
I typically have two or so JIRA tasks on the go, so I have a template file that looks like this:
# For doing automated JIRA updates weekly
# EDITOR=/bin/true [email protected] ~/linaro/jipdate/jipdate.py -f ~/jipdate.txt
[UM-2]
# Header: QEMU Upstream Maintainership
# Type: Epic
+ Stuff I did on this issue
[QEMU-406]
# Header: QEMU support for MVE
# Type: Epic
+ Stuff I did on this other issue
and every week I update the bullet-point list of stuff before running the jipdate command
EDITOR=/bin/true [email protected] ~/linaro/jipdate/jipdate.py -f ~/jipdate.txt
Occasionally I only work on one of my 2 active JIRA tasks during the week. If I do the "obvious" thing and just leave that part blank, so that there is a "[FOO-9434]" tag line but then no non-comment lines after it, or only blank lines after it, then jipdate will try to update the JIRA issue with an empty comment, and the server doesn't like it:
$ EDITOR=/bin/true [email protected] ~/linaro/jipdate/jipdate.py -f ~/jipdate.txt
These JIRA cards will be updated as follows:
[UM-2]
+ Worked through my code-review backlog
+ Noticed that we never got round to making our emulated GICv3 support having redistributors in more than one contiguous region; this prevents using more than 123 CPUs with the virt board. Sent out a patchset which adds the necessary handling.
+ Generally trying to tie off loose ends pre-holiday :-)
[QEMU-406]
Server to update: https://linaro.atlassian.net
Are you sure you want to update Jira with the information above? [y/n] y
Traceback (most recent call last):
File "/home/petmay01/linaro/jipdate/jipdate.py", line 535, in main
issues
UnboundLocalError: local variable 'issues' referenced before assignment
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/petmay01/linaro/jipdate/jipdate.py", line 543, in <module>
main(sys.argv)
File "/home/petmay01/linaro/jipdate/jipdate.py", line 538, in main
parse_status_file(jira, filename, None)
File "/home/petmay01/linaro/jipdate/jipdate.py", line 408, in parse_status_file
update_jira(jira, issue, comment, transition)
File "/home/petmay01/linaro/jipdate/jipdate.py", line 160, in update_jira
jira.add_comment(i, c)
File "/home/petmay01/.local/lib/python3.6/site-packages/jira/client.py", line 106, in wrapper
result = func(*arg_list, **kwargs)
File "/home/petmay01/.local/lib/python3.6/site-packages/jira/client.py", line 1645, in add_comment
r = self._session.post(url, data=json.dumps(data))
File "/home/petmay01/.local/lib/python3.6/site-packages/jira/resilientsession.py", line 175, in post
return self.__verb("POST", url, **kwargs)
File "/home/petmay01/.local/lib/python3.6/site-packages/jira/resilientsession.py", line 168, in __verb
raise_on_error(response, verb=verb, **kwargs)
File "/home/petmay01/.local/lib/python3.6/site-packages/jira/resilientsession.py", line 54, in raise_on_error
r.status_code, error, r.url, request=request, response=r, **kwargs
jira.exceptions.JIRAError: JiraError HTTP 400 url: https://linaro.atlassian.net/rest/api/2/issue/QEMU-406/comment
text: Comment body can not be empty!
response headers = {'Server': 'AtlassianProxy/1.19.3.1', 'cache-control': 'no-cache, no-store, no-transform', 'Content-Type': 'application/json;charset=UTF-8', 'Strict-Transport-Security': 'max-age=315360000; includeSubDomains; preload', 'Date': 'Fri, 01 Oct 2021 16:00:37 GMT', 'ATL-TraceId': 'XXX', 'x-arequestid': 'XXX', 'x-aaccountid': 'XXX', 'X-XSS-Protection': '1; mode=block', 'Transfer-Encoding': 'chunked', 'timing-allow-origin': '*', 'x-envoy-upstream-service-time': '86', 'X-Content-Type-Options': 'nosniff', 'Connection': 'close', 'Expect-CT': 'report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/global-proxy", enforce, max-age=86400'}
response text = {"errorMessages":[],"errors":{"comment":"Comment body can not be empty!"}}
(I've XXXed out things like accountid just in case they are sensitive data.)
It would be nice if jipdate noticed that there was no actual content for the issue, and didn't try to update it.
(The workaround is simple: temporarily comment out the [FOO-4343] tag for the issue that wouldn't have any content.)
I have installed jipdate on MacOSX (Ventura 13.5.1) using the instructions
I am following the video tutorial and when I do:
➜ jipdate -q -eI get this error:
Traceback (most recent call last):
File "/opt/homebrew/lib/python3.11/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 11 column 1 (char 10)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/homebrew/bin/jipdate", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/jipdate/jipdate.py", line 626, in main
jira, username = jiralogin.get_jira_instance(cfg.args.t)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/jipdate/jiralogin.py", line 119, in get_jira_instance
j = JIRA(url, basic_auth=(username, password)), username
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/jira/client.py", line 580, in __init__
si = self.server_info()
^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/jira/client.py", line 3120, in server_info
j = self._get_json("serverInfo")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/jira/client.py", line 3858, in _get_json
raise e
File "/opt/homebrew/lib/python3.11/site-packages/jira/client.py", line 3855, in _get_json
r_json = json_loads(r)
^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/jira/utils/__init__.py", line 76, in json_loads
return resp.json()
^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 11 column 1 (char 10)I have just joined and I don't have JIRA objects yet on my name. I'm wondering whether that could be the cause?
There doesn't seem to be any way to tell jipdate "do this, but instead of actually changing anything on the JIRA server just tell the user what would have been done". This is a really useful thing to have when you're trying out a new tool, because it means you can test whether you have the command line syntax right and so on without worrying about whether you're going to mess up the server at all. The usual syntax for this is "--dry-run". (You could use -n for the short option, by analogy with git.)
Hi Joakim,
It looks like commit 9a21e26 does the opposite of its commit message and is now less permissive than before
commit c770fda has already enable more permissive identifier. It enables [everything is allowed] but you have now restricted it to [XYZ-123 : This is my update]
You should revert 9a21e26
Vincent
jipdate writes the config.yml to your current working directory (wherever that happens to be) rather than looking in a well defined place for it (eg user's home directory, same directory as the jipdate script). This is unexpected (to me, anyway) and means that effectively you have to run jipdate from a well defined directory if you don't want it to forget your user settings and drop config.yml files everywhere.
TCWG CI uses jipdate master branch to add comments to jira about regressions detected in various components. Jipdate started to fail this morning with:
UnboundLocalError: local variable 'issues' referenced before assignment
Full log is at https://ci.linaro.org/view/tcwg_kernel-gnu/job/tcwg_kernel-bisect-llvm-master-arm-next-allmodconfig/73/console .
The problem occurred between April 3 and April 8, this range only has 4 commits from @ndechesne .
@ndechesne , would you please investigate?
To reproduce the problem this should be enough:
cat > jira-body.txt <<EOF
[GNU-587]
Test comment
EOF
export JIRA_USERNAME=$USER
export JIRA_PASSWORD=$PASSWORD
export EDITOR=cat
echo y | jipdate.py -f jira-body.txt
e104462:xenial:jipdate$ EDITOR=/bin/true [email protected] ~/linaro/jipdate/jipdate.py -f ~/jipdate.txt
No file provided and not in query mode
usage: jipdate.py [-h] [-e] [-f FILE] [-l] [-p] [-q] [-s] [-t] [-u USER] [-v]
[-x] [--all] [--dry-run]
[etc]
This worked with commit 741de62, but has broken somewhere between there and commit 044808e.
I tried to do a bisect, but in commit 87ebeb2 it can't even print the error message and instead you get a python backtrace:
e104462:xenial:jipdate$ EDITOR=/bin/true [email protected] ~/linaro/jipdate/jipdate.py -f ~/jipdate.txt
Traceback (most recent call last):
File "/home/petmay01/linaro/jipdate/jipdate.py", line 499, in <module>
main(sys.argv)
File "/home/petmay01/linaro/jipdate/jipdate.py", line 467, in main
eprint("No file provided and not in query mode\n")
File "/home/petmay01/linaro/jipdate/helper.py", line 7, in eprint
print(*args, file=sys.stderr, **kwargs)
NameError: global name 'sys' is not defined
I am trying to query the status of the stories assigned to another user. So based on the readme information I am running the following command
$> python jipdate.py -q -u [email protected] -v
Using config file: $HOME/.config/jipdate/.jipdate.yml
No user name found in JIRA_USERNAME environment variable
Forgot to export JIRA_PASSWORD?
Password:
WARNING:root:Got recoverable error from GET https://projects.linaro.org/rest/api/2/serverInfo, will retry [1/3] in 12.2820254916s. Err: 401
Captcha verification has been triggered by JIRA - please go to JIRA using your web browser, log out of JIRA, log back in entering the captcha; after that is done, please re-run the script
This captcha verification is triggered every time.
If I try to run the command a second time without going through the captcha verification I get
$> python jipdate.py -q -u [email protected] -v
Using config file: $HOME/.config/jipdate/.jipdate.yml
No user name found in JIRA_USERNAME environment variable
Forgot to export JIRA_PASSWORD?
Password:
Captcha verification has been triggered by JIRA - please go to JIRA using your web browser, log out of JIRA, log back in entering the captcha; after that is done, please re-run the script
(so everything the same without the WARNING).
In both cases there is no status output.
Can jipdate be improved to support cloning of a card like what jira's "Clone" functionality provides? I.e.,
jipclone.py --template EXISTING_CARD --summary "New Summary"
Jipdate supports creating new jira cards, but that requires providing information about many relevant aspects of a card -- card type, parent/epic link, status, etc. In certain cases (e.g., CI) it is more convenient to clone a card from a template.
Consider https://linaro.atlassian.net/browse/GNU-834 -- this card hosts sub-task cards for problems detected by Linaro TCWG CI. We want to have CI create these sub-task cards automatically, rather than human copy-pasting information from CI reports. Hard-coding data for sub-task cards in our CI scripts would be sub-optimal, since several fields change regularly -- e.g., parent card, assignee.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.