addisonelliott/pyqt5ac

Python module to automatically compile UI and QRC files in PyQt5 to Python files

• PyQt5 Auto Compiler (pyqt5ac)
  • Enter pyqt5ac!
• Installing
• Getting Started
  • Running from Command Line
  • Running from Python Script
• Configuration Options
• Example
  • Option 1: YAML Config File (Recommended)
  • Option 2: JSON Config File (Deprecated)
  • Option 3: Python Script
  • Option 4: Command Line
  • Resulting File Structure
• Support
• License

PyQt5 Auto Compiler (pyqt5ac)

pyqt5ac is a Python package for automatically compiling Qt's UI and QRC files into Python files.

In PyQt5, Qt Designer is the application used to create a GUI using a drag-and-drop interface. This interface is stored in a .ui file and any resources such as images or icons are stored in a .qrc file.

These two filetypes must be compiled into Python files before they can be used in your Python program. There are a few ways to go about this currently:

1. Manually compile the files using the command line and pyuic5 for .ui files and pyrcc5 for .qrc files.
2. Compile the files each time the application is started up by calling pyuic5 and pyrcc5 within your Python script

The downside to the first method is that it can be a tedious endeavor to compile the files, especially when one is faced with a larger project with many of these files that need to be compiled. Although the second method eliminates the tediousness of compilation, these files are compiled every time you run your script, regardless of if anything has been changed. This can cause a hit in performance and take longer to startup your script.

Enter pyqt5ac!

pyqt5ac provides a command-line interface (CLI) that searches through your files and automatically compiles any .ui or .qrc files. In addition, pyqt5ac can be called from your Python script. In both instances, ui and resource files are only compiled if they have been updated.

Installing

pyqt5ac is currently available on PyPi. The simplest way to install alone is using pip at a command line

pip install pyqt5ac

which installs the latest release. To install the latest code from the repository (usually stable, but may have undocumented changes or bugs)

pip install git+https://github.com/addisonElliott/pyqt5ac.git

For developers, you can clone the pyqt5ac repository and run the setup.py file. Use the following commands to get a copy from GitHub and install all dependencies

git clone https://github.com/addisonElliott/pyqt5ac.git
cd pyqt5ac
pip install .[dev]

to install in 'develop' or 'editable' mode, where changes can be made to the local working code and Python will use the updated code.

Getting Started

Running from Command Line

If pyqt5ac is installed via pip, the command line interface can be called like any Unix based program in the terminal

pyqt5ac [OPTIONS] [IOPATHS]...

In the interface, the options have slightly different names so reference the help file of the interface for more information. The largest difference is that the ioPaths argument is instead a list of space delineated paths where the even items are the source file expression and the odd items are the destination file expression.

The help file of the interface can be run as

pyqt5ac --help

Running from Python Script

The following snippet of code below demonstrates how to call pyqt5ac from your Python script

import pyqt5ac

pyqt5ac.main(rccOptions='', uicOptions='--from-imports', force=False, initPackage=True, config='',
             ioPaths=[['gui/*.ui', 'generated/%%FILENAME%%_ui.py'],
                     ['resources/*.qrc', 'generated/%%FILENAME%%_rc.py']])

Configuration Options

All of the options that can be specified to pyqt5ac can also be placed in a configuration file (JSON or YAML). My recommendation is to use a configuration file to allow easy compilation of your software. For testing purposes, I would use the options in the command line interface to make get everything working and then transcribe that into a configuration file for repeated use.

Whether running via the command line or from a script, the arguments and options that can be given are the same. The valid options are:

• rccOptions - Additional options to pass to the resource compiler. See the man page of pyrcc5 for more information on options. An example of a valid option would be "-compress 1". Default is to pass no options.
• uicOptions - Additional options to pass to the UI compiler. See the man page of pyuic5 for more information on options. An example of a valid option would be '--from-imports'. Default is to pass no options.
• force - Specifies whether to force compile all of the files found. The default is false meaning only outdated files will be compiled.
• config - JSON or YAML configuration file that contains information about these parameters.
• ioPaths - This is a 2D list containing information about what source files to compile and where to place the source files. The first column is the source file global expression (meaning you can use wildcards, ** for recursive folder search, ? for options, etc to match filenames) and the second column is the destination file expression. The destination file expression recognizes 'special' variables that will be replaced with information from the source filename:
  • %%FILENAME%% - Filename of the source file without the extension
  • %%EXT%% - Extension excluding the period of the file (e.g. ui or qrc)
  • %%DIRNAME%% - Directory of the source file
• variables - custom variables that can be used in the definition of the paths in ioPaths. For example, to limit the search of files to a specific directory, one can define a variable BASEDIR and then use it as %%BASEDIR%%/gui/*.ui*
• init_package - If specified, an empty __init__.py file is also generated in every output directory if missing. Does not overwrite existing __init__.py. Default value is True.

Note that all relative paths are resolved from the configuration file location, if given through a config file, or from the current working directory otherwise.

Example

Take the following file structure as an example project where any UI and QRC files need to be compiled. Assume that pyuic5 and pyrcc5 are located in /usr/bin and that '--from-imports' is desired for the UIC compiler.

|-- gui
|   |-- mainWindow.ui
|   |-- addDataDialog.ui
|   `-- saveDataDialog.ui
|-- resources
|   |-- images
|   |-- stylesheets
|   |-- app.qrc
|   `-- style.qrc
|-- modules
|   |-- welcome
|   |   |-- module.ui
|   |   `-- resources
|   |       |-- images
|   |       `-- module.qrc
|   `-- dataProbe
|       |-- module.ui
|       `-- resources
|           |-- images
|           `-- module.qrc

The sections below demonstrate how to setup pyqt5ac to compile the necessary files given the file structure above.

Option 1: YAML Config File (Recommended)

ioPaths:
  -
    - "gui/*.ui"
    - "generated/%%FILENAME%%_ui.py"
  -
    - "resources/*.qrc"
    - "generated/%%FILENAME_%%%%EXT%%.py"
  -
    - "modules/*/*.ui"
    - "%%DIRNAME%%/generated/%%FILENAME%%_ui.py"
  -
    - "modules/*/resources/*.qrc"
    - "%%DIRNAME%%/generated/%%FILENAME%%_rc.py"

uic_options: --from-imports
init_package: True
force: False

Now run pyqt5ac from the command line or Python script using your config file:

pyqt5ac --config config.yml

or

import pyqt5ac

pyqt5ac.main(config='config.yml')

Option 2: JSON Config File (Deprecated)

{
  "ioPaths": [
    ["gui/*.ui", "generated/%%FILENAME%%_ui.py"],
    ["resources/*.qrc", "generated/%%FILENAME%%_rc.py"],
    ["modules/*/*.ui", "%%DIRNAME%%/generated/%%FILENAME%%_ui.py"],
    ["modules/*/resources/*.qrc", "%%DIRNAME%%/generated/%%FILENAME%%_rc.py"]
  ],
  "rcc_options": "",
  "uic_options": "--from-imports",
  "init_package": true,
  "force": false
}

Now run pyqt5ac from the command line or Python script using your config file:

pyqt5ac --config config.yml

or

import pyqt5ac

pyqt5ac.main(config='config.yml')

Option 3: Python Script

import pyqt5ac

pyqt5ac.main(uicOptions='--from-imports', force=False, initPackage=True, ioPaths=[
        ['gui/*.ui', 'generated/%%FILENAME%%_ui.py'],
        ['resources/*.qrc', 'generated/%%FILENAME%%_rc.py'],
        ['modules/*/*.ui', '%%DIRNAME%%/generated/%%FILENAME%%_ui.py'],
        ['modules/*/resources/*.qrc', '%%DIRNAME%%/generated/%%FILENAME%%_rc.py']
    ])

Option 4: Command Line

pyqt5ac --uic_options "--from-imports" gui/*.ui generated/%%FILENAME%%_ui.py resources/*.qrc generated/%%FILENAME%%_rc.py modules/*/*.ui %%DIRNAME%%/generated/%%FILENAME%%_ui.py modules/*/resources/*.qrc %%DIRNAME%%/generated/%%FILENAME%%_rc.py

Resulting File Structure

|-- gui
|   |-- mainWindow.ui
|   |-- addDataDialog.ui
|   `-- saveDataDialog.ui
|-- resources
|   |-- images
|   |-- stylesheets
|   |-- app.qrc
|   `-- style.qrc
|-- generated
|   |-- __init__.py_
|   |-- mainWindow_ui.py
|   |-- addDataDialog_ui.py
|   |-- saveDataDialog_ui.py
|   |-- app_rc.py
|   `-- style_rc.py
|-- modules
|   |-- welcome
|   |   |-- module.ui
|   |   |-- resources
|   |   |   |-- images
|   |   |   `-- module.qrc
|   |   `-- generated
|   |       |-- module_ui.py
|   |       `-- module_rc.py
|   `-- dataProbe
|       |-- module.ui
|       |-- resources
|       |   |-- images
|       |   `-- module.qrc
|       `-- generated
|           |-- module_ui.py
|           `-- module_rc.py

Support

Issues and pull requests are encouraged!

Bugs can be submitted through the issue tracker.

Pull requests are welcome too!

License

pyqt5ac has an MIT-based license.

pyqt5ac's Issues

Custom variables not supported from CLI

It seems like I forgot to add support for the new custom variables feature from the CLI interface. Shall I fix this for consistency, or you plan to deprecate the CLI support and therefore this is not very relevant?
I have no need for this feature but I understand why it might be better to have it.

Deprecate json usage

If a json configuration is loaded then raise a warning that json support will be removed in 2.0.0.

Paths are not relative to the configuration file

I just stumbled upon this behavior. My main script contains this line:

pyqt5ac.main(config=os.path.join(os.path.dirname(os.path.realpath(__file__)), 'pyqt5ac.yml'))

and my pyqt5ac.yml is the following:

variables:
  BASEDIR : "be_bi_pyqt_template"
ioPaths:
  -
    - "%%BASEDIR%%/resources/*.ui"
    - "%%BASEDIR%%/resources/generated/ui_%%FILENAME%%.py"
  -
    - "%%BASEDIR%%/resources/*/*.qrc"
    - "%%BASEDIR%%/resources/generated/%%FILENAME%%_rc.py"
uic_options: --from-imports
force: False

The two files are siblings.

When I run main from the same folder that contains the two files, I get:

Skipping example_widget, up to date

When I run main from any other folder, I get:

No items found in be_bi_pyqt_template/resources/*.ui
No items found in be_bi_pyqt_template/resources/*/*.qrc

I am looking into this issue now, as it is rather critical for me. Expect a PR soon.

Set up CI on GitLab

Continuous testing and QA.

Update deprecated yaml loading

The current way we load the yamls is already deprecated:

pyqt5ac.py:133: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
configData = yaml.load(fh)

Introduce dependency scopes in setup.py

Modern Python modules tend to offer a setup.py that separates dependencies by usage. For example, they provide:

• core install (only necessary packages)
• test install (the packages you need to run the tests properly)
• doc install (the packages you need to build documentation)
• dev install (extra packages for development)
• aggregates install tags like all (install dependencies from all use-cases)
  and the possibility to define extra tags in case of need (like ci for extra packages needed on CI only, etc.)

I understand this package has not a lot of dependencies, but I believe it's a good practice to categorize them that way.

In addition, I'm not entirely sure what the requirements.txt is being used for. Currently installing this package with pip install -e . works just fine.

Ensure eventually generated folders contain `__init__.py` files

Folders into Python packages are always required to contain a __init__.py file, or otherwise they might not be recognized as submodules and imports can fail.

Currently pyqt5ac creates new folders if required, but it does not ensure the presence of such __init__.py and therefore the generated files might fail to import.

In #3 I proposed a patch that will make sure that at least the folder containing the files includes as well an __init__.py. I haven't tested it over more nested setups, hopefully it's a rare use-case, but in any case it seems rather simple to extend at need.
it also comes with a flag called --no-init to disable such check at need.

Introduce tests for CLI

Currently the click CLI is not tested

Set up webhook to GitLab mirror repository

Set up a webhook to the corresponding GitLab repository: https://gitlab.com/zkovari/pyqt5ac

Steps to do so:

• Create a Personal access token on GitLab. Ref: https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html. Scope should be API.
• Add a webhook to the GitHub repository to trigger the pipeline on each push. Reference: point 5 in https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html#connect-manually

The webhook setup should point to
https://gitlab.com/api/v4/projects/17788428/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN>, replacing with your GitLab personal access token. Content type is application/json. Event is push event.

Expected outcome: a push event on GitHub would automatically trigger a pipeline on GitLab.

"Bad file descriptor error" from 'click\_winconsole.py'

Hi.
I'm trying to use pyqt5ac in a rather straightforward way ( I think), using your example for python script.
I added this to my main script:

pyqt5ac.main(rccOptions='', uicOptions='--from-imports', force=False, initPackage=True, config='', ioPaths=[['gui/*.ui', 'gui/%%FILENAME%%_ui.py'], ['icons/*.qrc', 'icons/%%FILENAME%%_rc.py']])

when running it, I get this error:

from the click package.

Oddly, one of the .ui files (camerawindow.ui) does manage to compile and a camerawindow_ui.py is generated in the gui directory.

How can I make it work?

Thanks,
Ido

Define custom variables

Currently I see that three variables can be used to specify the name and location of the files to convert: FILENAME, EXT and DIRNAME.

Is there any way to define extra variables? A practical example:

config.yml

variables:
- BASEDIR: my_project_name
ioPaths:
  -
    - "%%BASEDIR%%/gui/*.ui"
    - "%%BASEDIR%%/generated/%%FILENAME%%_ui.py"
uic_options: --from-imports
force: False

That would be allow me to put the configuration file in the root of the project, so it doesn't get packaged when I release the code.

shlex.split hangs if rcc_options is defined by empty in YAML config

In the _buildCommand method, the shlex.split hangs on reading standard input if options happen to be None.

The options arguments can be None if one defines the rcc_options key in the YAML configuration file, as such:

# Invoke with pyqt5ac -c pyqt5.yml

# Additional options to pass to the resource compiler.
# See: https://manpages.ubuntu.com/manpages/hirsute/man1/pyrcc5.1.html
rcc_options:

Configurable logging levels

Very minor feature request.

What about adding something like a --quiet flag to suppress output if no files are compiled? I don't see the point of the Skipping [this file], up to date apart from debugging purposes. Optimally we could have properly configurable logging levels.

Write functional tests

Set up auto publication to Pypi repository

Set up a Travis pipeline that can publish the package to Pypi repository.

Release could be triggered based on a tag.

Fail execution if file generation fails

Currently if any generation fails (e.g. because a Qt file is invalid) then the script continues running. To raise awareness of the problem, rather let's fail the script's execution in such case.

Also introduce a new lenient argument which can disable this new strict behaviour. By default it's False, and if given, the script doesn't terminate in case a generation fails.

Set up codeclimate integration

Add the repository on https://codeclimate.com/dashboard.

After that, under Repo Settings -> Plugins, select the following:

• Fixme
• Git.legal
• Markdownlint
• Pep8
• Radon
• SonarPython

After the setup, it might require some time to run the analysis. And probably it will be only triggered on the next commit.

Tests seems to be not independent

In #3 I added a check on the variables dictionary. Running the tests before introducing modifications showed no issue. However, after introducing that simple check over the input, it seems like the tests have an issue and they are not independent.

How to reproduce:

1. Get the code from PR #3
2. pip install pytest pytest-random-order (btw, this was a guess and it wouldn't be necessary if test dependencies were specified in setup.py, see #21)
3. python -m pytest --> 7 tests fail: variables apparently contains the default keys already.
4. python -m pytest --random-order --> 8 tests fail: same reason.
5. python -m pytest tests/test_pyqt5ac.py::test_resource_generation --> passes
6. python -m pytest tests/test_pyqt5ac.py::<any other failed test> --> passes (I tested them all one by one)

I recommend having a look at pytest documentation in any case. Or, if you're using any other framework for testing and this is a consequence of a wrong guess, please document it somewhere and I'll retry.

More robust check for regeneration [WIP]

Currently, pyqt5ac verifies whether a file needs to be regenerated only basing on the last saved date recorded on the file system.

However, a generated file might become invalid for multiple other reasons:

• IDE's refactoring (or not refactoring properly) imports in the generated files and not in the *.ui / *.qrc
• Other files, which are being imported into the current one, changing name/path (like *.qrc's resources)
• Users editing the files
• Probably more unexpected ones

I would recommend to plan a strategy to detect such changes and how to deal with them.

Typing

As a suggestion, it would be nice to introduce typing, at least in function signatures.

Let me know if you need help on this, but it should be a really small thing.

Auto-detect configuration files

This is a feature request. If we can agree on a default name for the .yml configuration files, pyqt5ac could be modified to look for it at startup if no more specific instructions are given.

So, instead of typing something like

pyqt5ac --config pyqt5ac.yml

one could simply type

pyqt5ac

and, if pyqt5ac.yml exists in the folder, it would load it.

I can take on the implementation of this feature if you wish, but let's agree on the details first.