satelliteqe/greentea

Update me

greentea's Issues

ISE 500 - tests.html?search=...

Traceback (most recent call last):
  File "/data/greentea/web/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/data/greentea/web/env/lib/python2.7/site-packages/django/utils/decorators.py", line 110, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/data/greentea/web/env/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "./apps/core/views/tests.py", line 97, in dispatch
    return super(TestsListView, self).dispatch(request, *args, **kwargs)
  File "/data/greentea/web/env/lib/python2.7/site-packages/django/views/generic/base.py", line 89, in dispatch
    return handler(request, *args, **kwargs)
  File "./apps/core/views/tests.py", line 134, in get
    return super(self.__class__, self).get(request, *args, **kwargs)
  File "/data/greentea/web/env/lib/python2.7/site-packages/django/views/generic/base.py", line 158, in get
    context = self.get_context_data(**kwargs)
  File "./apps/core/views/tests.py", line 238, in get_context_data
    test_ids=test_ids, periodschedules=periodschedules)
  File "./apps/core/views/tests.py", line 290, in prepare_matrix
    'owner__name': i.test.owner.name,
AttributeError: 'NoneType' object has no attribute 'name'

rewrite kerberos-authentication integration

more information is on the page http://www.adelton.com/docs/apache/optional-kerberos-authentication

local variable 'distro_name' referenced before assignment

After freshly building docker image and starting the container with my beaker credentials, I've tried to schedule my first job. The documentation is sketchy about the initial setup steps so I just went to http://localhost/import/ and copied in my beaker job XML. I've hit Queue and got error page

local variable 'distro_name' referenced before assignment

Request Method:     POST
Request URL:    http://localhost/import/
Django Version:     1.8.6
Exception Type:     UnboundLocalError
Exception Value:    

local variable 'distro_name' referenced before assignment

Exception Location:     /data/GreenTea/apps/core/utils/beaker_import.py in recipe, line 103
Python Executable:  /data/GreenTea/env/bin/python
Python Version:     2.7.10
Python Path:    

['/data/GreenTea/env/lib/python2.7/site-packages/git/ext/gitdb',
 '/data/GreenTea',
 '/data/GreenTea/env/lib64/python27.zip',
 '/data/GreenTea/env/lib64/python2.7',
 '/data/GreenTea/env/lib64/python2.7/plat-linux2',
 '/data/GreenTea/env/lib64/python2.7/lib-tk',
 '/data/GreenTea/env/lib64/python2.7/lib-old',
 '/data/GreenTea/env/lib64/python2.7/lib-dynload',
 '/usr/lib64/python2.7',
 '/usr/lib/python2.7',
 '/data/GreenTea/env/lib/python2.7/site-packages',
 '/data/GreenTea/env/lib/python2.7/site-packages/gitdb/ext/smmap']

Server time:    Sun, 8 Nov 2015 13:16:00 +0100

unable to go to earlier pages in search pager

When I'm on:

http://.../?search=command%20not%20found&page=2

both arrows on pager links to ...&page=3.

Comment for test

When a test fail with a lot of running task then it is rather lengthy select all run on test page. I would like to select one test and comment that.

'ks_appends' subtree is being stripped from job template XML

When we import a job template whih contain <ks_appends> section to do some custom Kickstart setup, e.g.:

<ks_appends>
        <ks_append><![CDATA[
%post
dnf config-manager -y --set-enabled updates-testing
dnf update -y --best
%end
         ]]></ks_append>
      </ks_appends>

the whole section gets stripped from the job template upon import. This prevents us to run jobs which use custom host configuration, e.g enabling updates-testing on Fedora as can be seen from example.

Task in Taskomatic desn't save output from commands

when you run some asynchronous task, for example "check repo," you don't see log output in webui.

../admin/taskomatic/task/

Functionality to diff phases or testouts of two test runs

It would be nice to have a possibility to diff two (or to display side by side multiple):

• test journal (phases, asserts, times, results)
• TESTOUT.log
• sanitized TESTOUT.log (might be configurable, e.g. timestamps, hostnames, ... sanitized)

history of changes in tests

The test's changes was shown on page of tests. But currently It looks that this feature doesn't work. The commits are shown on dashboard, probably the problem is only on test page.

sql injection in searching by jobs

go to link: /jobs.html?search=Customer's database... and then you received ISE

syntax error at or near "s"
LINE 6: ...) LIKE lower('%Customer's database...

proposal: when waiving, duplicate waiver to Beaker comments

When we waive it GreenTea, maybe it would be good idea to duplicate waiver text to Beaker. Looks like you can comment on recipe set and on tasks, not on recipe. Would this make sense?

Role for group task templates seem ignored

We rely on having group task templates and then running them in different roles. Except, the Role set on the /admin/core/recipetemplate/167/ for the Group task template seems not applied when the job get scheduled.

Looking at the code, somewhere around https://github.com/SatelliteQE/GreenTea/blob/master/apps/core/utils/beaker.py#L340 the role from the taskG should get copied to the role of taskT in the similar way params are updated.

when showing score for most problematic tests, take waivers into account

Now, when I waive all the problems of the specific test, I still see it in top of the list without indication that it was waived

Document how container with persistent data should be run

Checking the https://github.com/SatelliteQE/GreenTea/wiki I fail to figure out how the GreenTea container should be started so that the data is stored persistently, somewhere. Please add documentation describing that use-case.

missing template id in url for chart

/jobs.html?tag=proxy?back=10&format_output=txt

ProgrammingError at /jobs.html

syntax error at or near ")"
LINE 6: AND "core_job"."template_id" IN ( )

pager on tests.html page does not preserve search terms

When I'm on:

http://.../tests.html?search=sync

pager on the page have links to .../tests.html?page=3& (so search term is missing there).

can not select tasks (with that "eye" icon) on tests.html page

I can not select tasks (with that "eye" icon) on tests.html page. Because of this, I can not waive on tasks results.

hard to select job with "eye" icon on jobs.html

Sometimes it is not possible to select job, sometimes it takes too much time (waiting for job info to be downloaded?), sometimes just another "Preview" is pinned in details bar.

Notification for events

Show notification that will be inform all people about some new change or status.

Problem with getting/parsing information from GIT repository

The command "checkrepo" does not work quite correctly.

Link like http://<...>/jobs.html#R:2716986 do not pin given recipe in details bar

In the past it worked, and I was able to go directly to waiving form and waive the recipe.

ISE /accounts/login - kerberos is not set

Show page with information about setup kerberos when kerberos is not set.

Build passed, starting it passed, I get to the main dashboard but on
http://localhost/accounts/login I get

AttributeError at /accounts/login

'NoneType' object has no attribute 'split'

Request Method:     GET
Request URL:    http://ibm-dx360m4-01.rhts.eng.bos.redhat.com/accounts/login
Django Version:     1.5.12
Exception Type:     AttributeError
Exception Value:    

'NoneType' object has no attribute 'split'

Exception Location:     /data/GreenTea/apps/kerberos/views.py in dispatch, line 15
Python Executable:  /data/GreenTea/env/bin/python
Python Version:     2.7.10
Python Path:    

['/data/GreenTea/env/lib/python2.7/site-packages/git/ext/gitdb',
 '/data/GreenTea',
 '/data/GreenTea/env/lib64/python27.zip',
 '/data/GreenTea/env/lib64/python2.7',
 '/data/GreenTea/env/lib64/python2.7/plat-linux2',
 '/data/GreenTea/env/lib64/python2.7/lib-tk',
 '/data/GreenTea/env/lib64/python2.7/lib-old',
 '/data/GreenTea/env/lib64/python2.7/lib-dynload',
 '/usr/lib64/python2.7',
 '/usr/lib/python2.7',
 '/data/GreenTea/env/lib/python2.7/site-packages',
 '/data/GreenTea/env/lib/python2.7/site-packages/gitdb/ext/smmap']

and

Traceback (most recent call last):
  File "/data/GreenTea/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 113, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/data/GreenTea/env/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/data/GreenTea/apps/kerberos/views.py", line 15, in dispatch
    username, real = self.remote_user.split("@")
AttributeError: 'NoneType' object has no attribute 'split'

on container's output.

super simple: "..." link on jobs.html should point to ...#tasks

With new Beaker webUI, link https:///recipes/ leads (at least in my case) to "reservation" tab. Would be great if link could contain "...#tasks" anchor to directly show "tasks" tab.

Searching in output beaker logs

We save all logs file (testout.log, ..) to local storage and it will be great if we can search in this logs.

• configure elastic search and indexing files
• implemented webui page for calling elastic search

could graph on jobs page show number of waived tests as well?

Would it be possible for graph on jobs page to show number of waived tests as well?

lots of errors in info.log

[root@tttt ~]# grep ^Traceback /data/greentea/log/info.log | wc -l
1080
[root@tttt ~]# grep ^ProtocolError /data/greentea/log/info.log | wc -l
1080
[root@tttt ~]# grep "codec can't decode bytes in position" /data/greentea/log/info.log | wc -l
270

PUT http://0.0.0.0:9200/testout.log/log/576513 [status:N/A request:12.865s]
Traceback (most recent call last):
  File "/data/greentea/web/env/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 94, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/data/greentea/web/env/lib/python2.7/site-packages/urllib3/connectionpool.py", line 623, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/data/greentea/web/env/lib/python2.7/site-packages/urllib3/util/retry.py", line 232, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/data/greentea/web/env/lib/python2.7/site-packages/urllib3/connectionpool.py", line 578, in urlopen
    chunked=chunked)
  File "/data/greentea/web/env/lib/python2.7/site-packages/urllib3/connectionpool.py", line 362, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python2.7/httplib.py", line 979, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1013, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 975, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 835, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 811, in send
    self.sock.sendall(data)
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
ProtocolError: ('Connection aborted.', error(104, 'Connection reset by peer'))
indexing /.../TESTOUT.log: 'utf8' codec can't decode byte 0xe2 in position 37930: invalid continuation byte

Is that OK? Was investigating this because of higher load on the machine, not sure if that is related.

Git Group Test

The automatic checking of RunFor in MakeFiles.

Add possibility to embed external reports into "Reports" page

Please add way to embed (via iframe(?)) reports from external systems to "Reports" page so all data can be on one place.

traceback when test is remove from git

GitCommandError: 'git log --decorate=full --since=7.days --simplify-by-decoration --pretty=%H|%aN|%ae|%ai|%d --follow HEAD /data/tttt/git/yum/Regression/test-installation' returned with exit code 128
stderr: 'fatal: ambiguous argument /data/tttt/git/yum/Regression/test-installation': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions'

Command for scheduling doesn't work

python manage.py beaker schedule --schedule_id=291

Traceback (most recent call last):
File "manage.py", line 16, in
execute_from_command_line(sys.argv)
File "/home/pstudeni/RedHat/git/GreenTea/env/lib/python2.7/site-packages/django/core/management/init.py", line 354, in execute_from_command_line
utility.execute()
File "/home/pstudeni/RedHat/git/GreenTea/env/lib/python2.7/site-packages/django/core/management/init.py", line 346, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/pstudeni/RedHat/git/GreenTea/env/lib/python2.7/site-packages/django/core/management/base.py", line 393, in run_from_argv
handle_default_options(options)
File "/home/pstudeni/RedHat/git/GreenTea/env/lib/python2.7/site-packages/django/core/management/base.py", line 81, in handle_default_options
if options.settings:
AttributeError: Values instance has no attribute 'settings'

RestAPI framework

Rewrite code which is using API call

• use http://www.django-rest-framework.org/
• url: /api/v1/

Job template: tags are separated by spaces

Description for tags in job template shows: A comma-separated list of tags. but in fact tags are separated by spaces (Learned by hard way) Please fix description.

search url with symbol ' apostrophe - examples's

url: /jobs.html?search=sw%27s&format_output=txt

syntax error at or near "s"
LINE 6: ... lower("core_jobtemplate"."whiteboard") LIKE lower('%sw's%')

One hostname for more recipe in one job.

We need use recipeSet instead of recipe for generating xml with same hostname.

graph for statistic of jobs

create and insert svg graph to jobs page.

Add reports of statistic tests and run tasks

Add statistic about test to report page:

• number of unique run tests
• results of run tasks

negative values in "Report of product" in "Result runs" for "warning"

There are negative values in "Report of product" in "Result runs" for "warning". E.g.:

Name Run tests Configurations Result runs
Client 87 20 fail (7)    pass (84)    warning (-4)

Show external links in detail of run

it should be possible add external link to some test for url with documentation or result of performance test. Tests can contains from 0-n external links.

keyboard shortcut keys for open and close detail panel

I would like to press some key to open panel of detail in bottom of page and then some other key to close it.

For example:
close press "q"
open press "a"

I think that it will be great feature.

No graphs is shown

on the page jobs I don't see any graphs. The AJAX received empty data from ?tag=&format_output=txt

universal schedule beaker's jobs

Add new model to taskomatic that it descends all jobs of schedule run. For example I want schedule jobs for plan "daily" with schedule run ID 1.

`python manage.py beaker schedule --schedule-daily --label=(daily, nightly,...)

All jobs will contain link to schedule run. On webui data will descend by this ID. currently it uses date for group of recipes and tasks.

Diff page doen't compare correct test

Page /diffs.html should compare two jobs and shows different tests that are included in it. But it doesn't compare tests in group. we need implemented it.

can not select task with "eye" icon to waive it

Only "Review" tabs appears when I click at it. Discussed this with Martin and he said it is because the task does not have repository assigned.

`python manage.py migrate` failing

On a clean installation this is failing:

:: [  BEGIN   ] :: Running 'python manage.py migrate'
SystemCheckError: System check identified some issues:

ERRORS:
?: debug_toolbar.middleware.DebugToolbarMiddleware is missing from MIDDLEWARE_CLASSES.
    HINT: Add debug_toolbar.middleware.DebugToolbarMiddleware to MIDDLEWARE_CLASSES.
:: [   FAIL   ] :: Command 'python manage.py migrate' (Expected 0, got 1)

In default (developer) install there shouldn't be cache enabled

This should not be enabled by default:

CACHES = {
    'default': {
        #'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'gt_cache_table',
        #'LOCATION': 'unique-snowflake'
    }
}

otherwise I was getting:

Internal Server Error: /__debug__/render_panel/
Traceback (most recent call last):
  File ".../GreenTea/env/lib/python2.7/site-packages/django/core/handlers/base.py", line 87, in get_response
    response = middleware_method(request)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/middleware/cache.py", line 148, in process_request
    cache_key = get_cache_key(request, self.key_prefix, 'GET', cache=self.cache)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/utils/cache.py", line 223, in get_cache_key
    headerlist = cache.get(cache_key, None)
  File ".../GreenTea/env/lib/python2.7/site-packages/debug_toolbar/panels/cache.py", line 33, in wrapped
    value = method(self, *args, **kwargs)
  File ".../GreenTea/env/lib/python2.7/site-packages/debug_toolbar/panels/cache.py", line 75, in get
    return self.cache.get(*args, **kwargs)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/core/cache/backends/db.py", line 65, in get
    "WHERE cache_key = %%s" % table, [key])
  File ".../GreenTea/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File ".../GreenTea/env/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 485, in execute
    return Database.Cursor.execute(self, query, params)
OperationalError: no such table: gt_cache_table

and fixed it after Pavel's advice by:

$ ./manage.py createcachetable gt_cache_table
load plugins... ['./manage.py', 'createcachetable', 'gt_cache_table'] (temporary debug)
Cache table 'gt_cache_table' already exists.

when test does not have GIT repo specified, it can not be marked by "eye" icon

When test does not have GIT repo specified, it can not be marked by "eye" icon

Even with "Event finish" set to "return" in job template, job XML is generated with '<reservesys duration="286400"/>'

Missing systemd script

Write systemd script for restarting service (start/stop). When system is restarting, this service Green Tea is needed run manually

running recipe with only passed and waived tasks is rendered in green colour

Same recipe when it finishes is rendered in blue colour. I suggest it should be blue in both times (not to mask already waived tasks).

Beaker link for task result in details bar is invlid

Talking about link in details panel (on jobs.html) next to failed task name with "suitcase" icon.

It says https:///recipes/[object%20Object]#task40849743