aiidateam/qe-tools

A set of useful tools for Quantum ESPRESSO

qe-tools

This repository contains a set of useful tools for Quantum ESPRESSO, in particular

• a parser for pw.x input files
• a parser for cp.x input files

The code is developed and maintained by the AiiDA Team and is licensed under the MIT license (see LICENSE file).

qe-tools's Issues

Add (minimal) documentation build

Should at least contain a short project description, and the reference.

Please provide a few more details on the purpose of this repo

I'm sure this sounds stupid, but to me the purpose of this repo isn't clear.
Is there a reason why we cannot add the functionality provided here in the aiida-quantumespresso plugin?
I see that the qeinputparser uses aiida, so it' doesn't seem to be about avoiding to install aiida...
Which brings me back to my question ;-)

Drop Python 3.5 compatibility, add Python 3.9 to test matrix

Python 3.5 is EOL.

dictionary problem with k_points

In the "k_points" dictionary and for automatic method.
MPgrid data is stored in "points" instead of what is specified in comment ("mesh")

Missing 'valid_ibrav' values in get_cell_from_parameters

The get_cell_from_parameters function (link) defines the valid ibrav values as list(range(15)) + [-3, -5, -9, -12].

Comparing this to the pw.x input description, this is missing the values -13 and 91. The -13 value could be a bit problematic to implement, because the input description states:

 IMPORTANT NOTICE: until QE v.6.4.1, axis for ibrav=-13 had a
 different definition: v1(old) = v2(now), v2(old) = -v1(now)

Check and update docstrings

The docstrings, in particular of PwInputFile and CpInputFile, need to be checked and potentially updated. This is probably easier once #44 is done, so we can directly check the doc output.

An issue that definitely needs solving is that they have a docstring both at the class level and for the __init__. In the documentation, only one of the two will show.

It'd be nice to convert the docstrings to numpy-style while we're at it, see https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html

Update or pin pytest-cases

The pytest-cases library has released a 2.x version, with an improved but backwards-incompatible interface.

We should either update to use the new interface (preferred) or pin 1.x (easier fix).

remove aiida dependencies?

qe-tools/qe_tools/parsers/pwinputparser.py

Line 188 in 67df37b

from aiida.orm.data.array.kpoints import KpointsData

qe-tools/qe_tools/parsers/qeinputparser.py

Line 235 in 67df37b

from aiida.orm.data.structure import StructureData, Kind, Site

Move CI from Travis to GHA

Wrong alat used if ibrav != 0

Here:
https://github.com/aiidateam/qe-tools/blob/develop/qe_tools/parsers/qeinputparser.py#L1107

we are assuming that alat is np.linalg.norm(cell[0]).
This is true only for ibrav = 0. For the other cases, we should use the value already stored above in alat.

Here is an example input file that gives the wrong result:

scf.in.txt

Problem with floats without a dot specified

If a float value is specified as 1e-06 instead of 1.e-06, i.e. without a dot, the code crashes saying that it does not support algebraic operations. But I think this is a valid way of specifying 10^{-6}.

ibrav -13 is incorrect

See the following QE mailing list thread: https://www.mail-archive.com/[email protected]/msg37950.html

Since we followed the documentation, this is bound to be wrong here. I'm not 100% sure if it's the "new" version that is incorrect or the "old" one.

Constructing cell from parameters

Dear all,

I think I found a small typo/error in the methods related to the ibrav parameters.

In the PW Input documentation one finds that the first lattice vector of the reconstructed cell for an orthorhombic base-centered structure (ibrav=-9) has the following shape:

v1 = (a/2, -b/2, 0)

Therefore, I think the following code needs to be adapted so that the minus is in the second component of the first lattice vector and not in the second lattice vector.

qe-tools/qe_tools/parsers/_input_base.py

Lines 984 to 988 in 5fe555c

	elif ibrav == -9:
	# -9 as 9, alternate description
	# v1 = (a/2,-b/2,0), v2 = (a/2,-b/2,0), v3 = (0,0,c)
	cell = np.array([[0.5 * alat, 0.5 * b, 0.], [0.5 * alat, -0.5 * b, 0.],
	[0., 0., c]])

AttributeError when calling get_structuredata

I get an AttributeError when calling get_structuredata (asking for an AiiDA instance).

/home/kahle/git/qe-tools/qe_tools/parsers/qeinputparser.py in get_structuredata(self)
256 # instance and set the cell
257 structuredata = StructureData()
--> 258 structuredata._set_attr('cell', structure_dict['cell'].tolist())
259
260 ################# KINDS ##########################

AttributeError: 'list' object has no attribute 'tolist'

Reason for this error was that we changed the return value of get_structure_from_qeinput from a numpy array to a list. This can be trivially fixed.

Slight inconsistencies in physical constants

See original issues on aiida-core and aiida-quantumespresso

Add README as long_description in setup.py

Currently a description is missing on PyPI, see: https://pypi.org/project/qe-tools/2.0.0rc1/

We can re-use the README.md for this, same as in aiida-core: https://github.com/aiidateam/aiida-core/blob/develop/setup.py#L55

Structure of Qe/Pw/CpInputFile

Both PwInputFile and CpInputFile have the same code for parsing namelists, atomic_positions, cell_parameters and atomic_species into attributes in their __init__.

As far as I can tell, QeInputFile is just a base class and not supposed to be used directly. But then, it has get_structure_from_qeinput - which uses all these attributes that aren't even defined for that class.

I would suggest to move the following code from both PwInputFile and CpInputFile into the base class:

        # Parse the namelists.
        self.namelists = parse_namelists(self.input_txt)
        # Parse the ATOMIC_POSITIONS card.
        self.atomic_positions = parse_atomic_positions(self.input_txt)
        # Parse the CELL_PARAMETERS card.
        self.cell_parameters = parse_cell_parameters(self.input_txt)
        # Parse the ATOMIC_SPECIES card.
        self.atomic_species = parse_atomic_species(self.input_txt)

The PwInputFile can add its k_points, while CpInputFile is just an empty wrapper around QeInputFile.

The question is what to do with the structure: Can it ever be that the structure is not present in an input file, or can we just add

    self.structure = parse_structure(
            txt=self.input_txt,
            namelists=self.namelists,
            atomic_positions=self.atomic_positions,
            atomic_species=self.atomic_species,
            cell_parameters=self.cell_parameters,
            qe_version=self._qe_version)
    )

to the end of the QeInputFile __init__?

Problem when encountering comments

In the following file:

&control
    pseudo_dir  = 'pseudo/'
    calculation = 'scf',
    prefix = 'Si_exc1',
 /
 &system
    ibrav = 0
    ! ibrav =  -3, 
    celldm(1) = 20.385647759,	
    nat =  1,
    ntyp = 1,
    ecutwfc = 30
 /
 &electrons
 /

ATOMIC_SPECIES
 Si 28.086  Si.pbe-n-rrkjus_psl.1.0.0.UPF

ATOMIC_POSITIONS {bohr}
Si    0. 0. 0.

CELL_PARAMETERS {alat}
-1.0 1.0 1.0
 1.0 -1.0 1.0
 1.0 1.0 -1.0

K_POINTS {automatic}
  6 6 6 1 1 1

if I run

import os, qe_tools

s = qe_tools.PwInputFile(os.path.abspath(FILENAME))

print("With comment:")
for v in s.get_structure_from_qeinput()['cell']:
    print(v)

I get:

With comment:
[-5.393810088203304, 5.393810088203304, 5.393810088203304]
[5.393810088203304, -5.393810088203304, 5.393810088203304]
[5.393810088203304, 5.393810088203304, -5.393810088203304]

If I instead remove the comments, I get:

Without comment:
[-10.787620176406609, 10.787620176406609, 10.787620176406609]
[10.787620176406609, -10.787620176406609, 10.787620176406609]
[10.787620176406609, 10.787620176406609, -10.787620176406609]

I think the reason is that it is not ignoring the commented line (indeed, from the QE docs, we have for ibrav=-3:

          cubic I (bcc), more symmetric axis:
      v1 = (a/2)(-1,1,1), v2 = (a/2)(1,-1,1),  v3 = (a/2)(1,1,-1)

from which the factor 1/2 between the two cases comes.

Exponentially-slow regex

As discussed in e.g. these links [1,2] there are some regex that, with the default python (or perl, ...) implementation, can become exponentially slow with some inputs.

To reproduce, one can e.g. run the following, where cell_parameters_block_re is the one defined in qeinputparser.py:

cell_string = """CELL_PARAMETERS angstrom
1.000000000 0.00000000 0.000000000
0.000000000 1.00000000 0.000000000
0.000000000 0.00000000 1.000000000"""
print cell_parameters_block_re.match(cell_string) # does not end after >60seconds
print cell_parameters_block_re.match(cell_string + "\n") # ends instantaneously

Already using a Thompson NFA implementation improves the situation: e.g. I tried this from this BitBucket repo with pip install regex and did a import regex as re as a drop-in replacement for re when generating cell_parameters_block_re can alleviate the problem, bringing the time down to ~3sec.
This can be done, adding a dependency, still it's not perfect and probably our regex can be improved.

At the moment, I suggest to always append a newline at the end of the input for safety, and if we see that this is not enough, we open this issue again and replace re with regex (and/or improve the regex).

[1] https://stackoverflow.com/questions/25982466/why-does-this-take-so-long-to-match-is-it-a-bug
[2] https://swtch.com/~rsc/regexp/regexp1.html

Mentioning @lekah

Parsing PW input file

Hello

I tried to parse an input file:

from qe_tools import PwInputFile

with open('aiida.in', 'r') as in_f:
    parsed_file = PwInputFile(in_f.read())
parsed_file.get_structure_from_qeinput()

and got the error that CELL_PARAMETERS is a nonetype.

The solution was proposed that my CELL_PARAMETERS values did not contain 0.0 instead of a 0.

Worked:

CELL_PARAMETERS angstrom
16.1762 0.0 0.0000000000
-8.0881 14.009 0.0000000000
0.0 0.0 18.3843

Failed:

CELL_PARAMETERS angstrom
16.1762 0 0.0000000000
-8.0881 14.009 0.0000000000
0 0 18.3843

When set qetools as the parser for qeinputgenerator, the generated input file for pwscf always use this setting: calculation = 'scf'.

Hi,
I noted the wonderful qeinputgenerator tool belong to the materialscloud project. For this tool, when I set qetools as the parser for qeinputgenerator, the generated input file for pwscf always use this setting: calculation = 'scf'.

As we all know, for a first-principle study on a system, the very first step should be a (vc-)relax calculation. So, I'm confused why qetools always use the calculation = 'scf' for the generated pwscf input file.

Regards
HY

Empty line after ATOMIC_SPECIES makes the parser fail

The attached file input.txt cannot be parsed by the current version of the library; removing the empty line after ATOMIC_SPECIES works, instead.
Probably the easiest solution (probably also to similar other issues in other cards and namelists) would be to remove empty lines before parsing?

Clearly define public vs. private interface

Ahead of the 2.0.0 release, I think we should clearly define public vs. private interface:

• All modules should have __all__ defined
• Everything that's private should have an underscore somewhere in its import name, or be excluded from the __all__ in the module where it is defined.
• As a simple check, ipython should autocomplete only the public parts when c.IPCompleter.limit_to__all__ = True is set.
• The public interface should probably be used (directly) in some test

I can do the changes to the code, but the important question is: which parts should be in the public interface? We have:

• qe_tools : public
  • parsers : public
    • cpinputparser: ?
      • CpInputFile : public, directly under parsers
    • pwinputparser: ?
      • PwInputFile : public, directly under parsers
      • parse_k_points: ?
    • qeinputparser: ?
      • QeInputFile : public, directly under parsers
      • get_cell_from_parameters: ?
      • get_structure_from_qeinput: ?
      • parse_atomic_positions: ?
      • parse_atomic_species: ?
      • parse_cell_parameters: ?
      • parse_namelists: ?
      • str2val: ?
      • strip_comment: ?
  • generators : public
    • cell_conversion : ?
      • get_parameters_from_cell : public (here, or directly under generators)?
  • utils: ?
    • _qe_version: private
      • parse_version: private
    • exceptions: ?
  • constants: ?

In aiida-quantumespresso, the following are used:

• qe_tools.constants
• From qe_tools.parsers, CpInputFile and PwInputFile are used in code, their "real" location (e.g. qe_tools.parsers.cpinputparser.CpInputFile is used in docstrings
• qe_tools.utils.exceptions.ParsingError is used in testing

@giovannipizzi @sphuber @lekah what's your opinion? If needed, now would also be a good time to move/rename things.

Drop py2 support?

Since the develop version of aiida-quantumespresso has just dropped py2 support, does it make sense to do the same here?

The reason I'm asking is because the interface of the cell -> parameters function I'm working on would benefit from keyword-only arguments.

Distribution requires MANIFEST.in file to include setup.json

As it stands the setup.json file will not be included and the distribution will be broken as the setup.py will not be able to load the file

Windows newlines are not dealt with properly

When using open() it's all good because python converts \r\n to \n but instead if reading with open(fname, 'rb') or just passing a string containing windows newlines, the regexes fail.

I'm already fixing this.