smbyc/cloudmasking

CloudMasking is a Qgis plugin for make the masking of clouds, cloud shadow, cirrus, aerosols, ice/snow and water for Landsat (4, 5, 7, 8, 9) products using different process such as Fmask, Blue Band, Cloud QA and QA Band.

CloudMasking

CloudMasking is a Qgis plugin for make the masking of clouds, cloud shadow, cirrus, aerosols, ice/snow and water for Landsat (4, 5, 7, 8, 9) products using different process and filters such as Fmask, Blue Band, Cloud QA, Aerosol and Pixel QA.

Read more in: https://smbyc.github.io/CloudMasking

Motivation

There are several ways for make cloud masking automatically, such as apply the default fmask band or cloud filter using a fixed pixel values in the QA bands. But these "by default" bands and values are not always good, and their efficiency varies in different regions and depends to a great extent on the type of terrain, vegetation or environmental conditions. The purpose of the plugin is to make cloud masking very personalized and configurable to perform the best possible masking by combining and using of various bands and filters.

If you want masking in a bulk process I recommend use python-fmask (but it is only fmask)

Installation

The plugin can be installed using the QGIS Plugin Manager.

the latest version of this plugin does not support old versions of Qgis

Source code

The official version control system repository of the plugin: https://github.com/SMByC/CloudMasking

The home plugin in plugins.qgis.org: https://plugins.qgis.org/plugins/CloudMasking/

Issue Tracker

Issues, ideas and enhancements: https://github.com/SMByC/CloudMasking/issues

About us

CloudMasking was developing, designed and implemented by the Group of Forest and Carbon Monitoring System (SMByC), operated by the Institute of Hydrology, Meteorology and Environmental Studies (IDEAM) - Colombia.

Author and developer: Xavier Corredor Ll.
Theoretical support, tester and product verification: SMByC-PDI group

Contact

Xavier Corredor Ll.: xcorredorl (a) ideam.gov.co
SMByC: smbyc (a) ideam.gov.co

License

CloudMasking is a free/libre software and is licensed under the GNU General Public License.

cloudmasking's Issues

Plugin causing issue with raster properties function

Hi! I've recently had issues with accessing the raster properties and determined that it was a 3rd party plugin that was interfering with the core function. After removing one by one, Cloud masking seems to be the issue. Can you please check?

Couldn't load plugin

I have a problem loading the Plugin, I am using Ubuntu Xfce 4.12 and I get the following error:

Couldn't load plugin 'CloudMasking' due to an error when calling its classFactory() method 

ImportError: dynamic module does not define module export function (PyInit__fillminima) 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 334, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/__init__.py", line 102, in classFactory
    from .cloud_masking import CloudMasking
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 792, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/cloud_masking.py", line 40, in 
    from CloudMasking.core import cloud_filters, color_stack
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 792, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/core/cloud_filters.py", line 44, in 
    from fmask import fmask, landsatTOA, landsatangles, config, saturationcheck
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 792, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/fmask.py", line 70, in 
    from . import fillminima
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 792, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/fillminima.py", line 35, in 
    from CloudMasking.libs.fmask import _fillminima
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 792, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ImportError: dynamic module does not define module export function (PyInit__fillminima)

Python version: 3.6.9 (default, Jan 26 2021, 15:33:00) [GCC 8.4.0]
QGIS version: 3.16.3-Hannover Hannover, 94ac9f21b8

Python Path:
/usr/share/qgis/python
/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python
/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/usr/share/qgis/python/plugins
/usr/lib/python36.zip
/usr/lib/python3.6
/usr/lib/python3.6/lib-dynload
/usr/local/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python
/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vrtbuilderplugin
/home/itadmin/.local/share/QGIS/QGIS3/profiles/default/python/plugins/vrtbuilderplugin/vrtbuilder

Please help

error cloud mask/generate the cloud mask / in only area of interest

Error en la opción:
generate the cloud mask / in only area of interest

Versión de Qgis 3.10.14

detalles del error:

Traceback (most recent call last):
File "C:/Users/Usuario/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\utils.py", line 40, in wrapper
return func(*args, **kwargs)
File "C:/Users/Usuario/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\utils.py", line 85, in wrapper
obj_returned = func(*args, **kwargs)
File "C:/Users/Usuario/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\cloud_masking.py", line 470, in process_mask
self.masking_result.do_blue_band(int(self.dockwidget.doubleSpinBox_BB.value()))
File "C:/Users/Usuario/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\cloud_filters.py", line 417, in do_blue_band
self.blue_band_for_process = self.clip(self.blue_band_file, self.blue_band_clip_file)
File "C:/Users/Usuario/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\cloud_filters.py", line 136, in clip
tmp_memory_file.unlink()
File "C:\PROGRA1\QGIS31.10\apps\Python37\lib\pathlib.py", line 1266, in unlink
self._accessor.unlink(self)
PermissionError: [WinError 32] El proceso no tiene acceso al archivo porque está siendo utilizado por otro proceso: 'C:\Users\Usuario\AppData\Local\Temp\memory_layer_aoi.gpkg'

Can't load plugin - classFactory()

I get this error when trying to install the plugin:

Couldn't load plugin 'CloudMasking' due to an error when calling its classFactory() method

AttributeError: module 'CloudMasking' has no attribute 'classFactory'
Traceback (most recent call last):
File "C:/PROGRA1/QGIS31.16/apps/qgis-ltr/./python\qgis\utils.py", line 334, in _startPlugin
plugins[packageName] = package.classFactory(iface)
AttributeError: module 'CloudMasking' has no attribute 'classFactory'

Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]
QGIS version: 3.16.4-Hannover Hannover, 654e76b3cb

Python Path:
C:\Users\apuzz\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\planet_explorer\extlibs
C:/PROGRA1/QGIS31.16/apps/qgis-ltr/./python
C:/Users/apuzz/AppData/Roaming/QGIS/QGIS3\profiles\default/python
C:/Users/apuzz/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins
C:/PROGRA1/QGIS31.16/apps/qgis-ltr/./python/plugins
C:\PROGRA1\QGIS31.16\apps\Python37
C:\PROGRA1\QGIS31.16\apps\Python37\Scripts
C:\Program Files\QGIS 3.16\bin\python37.zip
C:\PROGRA1\QGIS31.16\apps\Python37\DLLs
C:\PROGRA1\QGIS31.16\apps\Python37\lib
C:\Program Files\QGIS 3.16\bin
C:\Users\apuzz\AppData\Roaming\Python\Python37\site-packages
C:\PROGRA1\QGIS31.16\apps\Python37\lib\site-packages
C:\PROGRA1\QGIS31.16\apps\Python37\lib\site-packages\win32
C:\PROGRA1\QGIS31.16\apps\Python37\lib\site-packages\win32\lib
C:\PROGRA1\QGIS31.16\apps\Python37\lib\site-packages\Pythonwin
C:/Users/apuzz/AppData/Roaming/QGIS/QGIS3\profiles\default/python
C:\Users\apuzz\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\ee_plugin\extlibs_windows
C:\Users\apuzz\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\HCMGIS/forms
C:\Users\apuzz\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\SentinelHub\external\oauthlib-3.1.0-py2.py3-none-any.whl
C:\Users\apuzz\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\SentinelHub\external\requests_oauthlib-1.3.0-py2.py3-none-any.whl
C:/Users/apuzz/Desktop/intro-rs-data

I saw the other issues about this but couldn't figure out why it is happening, I'm using Python 64 bits. Any clues?

FILE_NAME_BAND_SR_4'

The cloud masking plugin v. 21.7.15 does not work in QGIS version 3.22.0. I present the error text.
Traceback (most recent call last): File "/home/taras/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/core/utils.py", line 40, in wrapper return func(*args, **kwargs) File "/home/taras/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/core/utils.py", line 85, in wrapper obj_returned = func(*args, **kwargs) File "/home/taras/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/cloud_masking.py", line 327, in load_stack self.color_stack_scene = color_stack.ColorStack(self.dockwidget.mtl_path, File "/home/taras/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/core/color_stack.py", line 61, in __init__ self.color_bands = [ File "/home/taras/.local/share/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/core/color_stack.py", line 62, in <listcomp> os.path.join(self.input_dir, self.mtl_file['FILE_NAME_BAND_SR_' + str(N)]) KeyError: 'FILE_NAME_BAND_SR_4'

CloudMasking Error

Issue Cloud masking

Hi, I had an issue for running Cloud Masking. I don't KNow if it is because I replaced the QGIS 3.10.14 for the 3.22.13 and I unistalled the previous QGIS versions. Following, the note that appears on screen, thanks.

Traceback (most recent call last):
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\utils.py", line 40, in wrapper
return func(*args, **kwargs)
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\utils.py", line 85, in wrapper
obj_returned = func(*args, kwargs)
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\cloud_masking.py", line 807, in process_mask
self.masking_result.do_qaband_c2(self.dockwidget.qabandc2_file, checked_items, qaband_svalues)
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\core\cloud_filters.py", line 890, in do_qaband_c2
gdal_calc.Calc(calc="1(numpy.all([{nfv}], axis=0)) + 10(numpy.any([{fv}], axis=0))".format(fv=filter_values, nfv=not_filter_values),
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\libs\gdal_calc.py", line 399, in Calc
doit(opts, None)
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\libs\gdal_calc.py", line 131, in doit
opts.format = GetOutputDriverFor(opts.outF)
File "P:\Users/D_Leon/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\libs\gdal_calc.py", line 110, in GetOutputDriverFor
print("Several drivers matching %s extension. Using %s" % (ext, drv_list[0]))
UnboundLocalError: local variable 'ext' referenced before assignment

Couldn't load plugin 'CloudMasking' due to an error when calling its classFactory() method

Hi,
I have QGis 3.12.3 installed from the 32-bit OSGeo4W and this error message appears. I am working with SCP 6.4.7.
How can I overcome this problem?

Couldn't load plugin 'CloudMasking' due to an error when calling its classFactory() method

ImportError: DLL load failed: Impossível localizar o módulo especificado. [Unable to find the specified module.]
Traceback (most recent call last):
File "C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking_init_.py", line 32, in fmask_libs
from CloudMasking.libs.fmask import _fillminima, _valueindexes
ImportError: cannot import name 'fillminima' from 'CloudMasking.libs.fmask' (C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking\libs\fmask_init.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:/OSGeo4W/apps/qgis/./python\qgis\utils.py", line 334, in startPlugin
plugins[packageName] = package.classFactory(iface)
File "C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking_init.py", line 100, in classFactory
fmask_libs()
File "C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\CloudMasking_init_.py", line 65, in fmask_libs
from CloudMasking.libs.fmask.win32_py36 import _fillminima, _valueindexes
File "C:/OSGeo4W/apps/qgis/./python\qgis\utils.py", line 743, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
ImportError: DLL load failed: Impossível localizar o módulo especificado.

Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)]
QGIS version: 3.12.3-București București, 8234261527

Python Path:
C:/OSGeo4W/apps/qgis/./python
C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python
C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins
C:/OSGeo4W/apps/qgis/./python/plugins
C:\OSGeo4W\apps\Python37
C:\OSGeo4W\apps\Python37\Scripts
C:\OSGeo4W\bin\python37.zip
C:\OSGeo4W\apps\Python37\DLLs
C:\OSGeo4W\apps\Python37\lib
C:\OSGeo4W\bin
C:\Users\Toni\AppData\Roaming\Python\Python37\site-packages
C:\Users\Toni\AppData\Roaming\Python\Python37\site-packages\win32
C:\Users\Toni\AppData\Roaming\Python\Python37\site-packages\win32\lib
C:\Users\Toni\AppData\Roaming\Python\Python37\site-packages\Pythonwin
C:\OSGeo4W\apps\Python37\lib\site-packages
C:\OSGeo4W\apps\Python37\lib\site-packages\win32
C:\OSGeo4W\apps\Python37\lib\site-packages\win32\lib
C:\OSGeo4W\apps\Python37\lib\site-packages\Pythonwin
C:/Users/Toni/AppData/Roaming/QGIS/QGIS3\profiles\default/python

Cloud mask can't be exported

Pretty straightforward; I run the process in plugin, click Export button, choose path and file name. When I click Save, I get "Error during saving the combined mask file". I'm running the following versions:

QGIS version
3.34.1-Prizren
QGIS code revision
133927424d9
Qt version
5.15.2
Python version
3.9.5
GDAL/OGR version
3.3.2
PROJ version
8.1.1
EPSG Registry database version
v10.028 (2021-07-07)
GEOS version
3.9.1-CAPI-1.14.2
SQLite version
3.35.2
PDAL version
2.3.0
PostgreSQL client version
unknown
SpatiaLite version
5.0.1
QWT version
6.1.6
QScintilla2 version
2.11.5
OS version
macOS 12.7

Active Python plugins
quick_map_services
0.19.29
pointsamplingtool
0.5.3
processing
2.12.99
grassprovider
2.12.99
db_manager
0.1.20
MetaSearch
0.3.6
CloudMasking
23.3.30

ClouMasking in python

Hello, i would like know if there's a tutorial for te cloud masking funcionalities in the terminal of the pyqgis.

AOI error

Create more than AOI (multiples Qgis sessions opened)

Cloud Masking Not working

'NoneType' object has no attribute 'name i need help

Masking error

Hello, I'm masking my satellite image, and when it comes out, not everything is masked (in this case, the sea). I'm using MacOS Big Sur Version 11.1, and QGIS version 3.16.3. I have attached an example of a satellite image that failed to mask.

Unable to install the Cloud Masking plugin

Hello, I can't install the Cloud Masking plugin. I'm learning to use QGIS, and I want to use the plugin to apply masks to satellite imagery. Right now I use Macbook Pro with MacOS Catalina version 10.15.5. I have reported the bug to QGIS, but they say that this is an issue with the plugin, not the QGIS itself. Error details are listed below when I installed the plugin into QGIS.

Couldn't load plugin 'CloudMasking' due to an error when calling its classFactory() method

ImportError: dlopen(/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/_fillminima.so, 2): no suitable image found.  Did find: /Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/_fillminima.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 /Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/_fillminima.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 
Traceback (most recent call last):
  File "/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 334, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/__init__.py", line 98, in classFactory
    from .cloud_masking import CloudMasking
  File "/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 743, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/cloud_masking.py", line 40, in 
    from CloudMasking.core import cloud_filters, color_stack
  File "/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 743, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/core/cloud_filters.py", line 44, in 
    from fmask import fmask, landsatTOA, landsatangles, config, saturationcheck
  File "/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 743, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/fmask.py", line 70, in 
    from . import fillminima
  File "/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 743, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/fillminima.py", line 35, in 
    from CloudMasking.libs.fmask import _fillminima
  File "/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/qgis/utils.py", line 743, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
ImportError: dlopen(/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/_fillminima.so, 2): no suitable image found.  Did find:
	/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/_fillminima.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00
	/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/CloudMasking/libs/fmask/_fillminima.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00

Python version: 3.7.3 (default, Mar 27 2019, 09:23:15) [Clang 10.0.1 (clang-1001.0.46.3)]
QGIS version: 3.12.3-București București, 8234261527

Python Path:
/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/networks
/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/contour
/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python
/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python
/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins
/Applications/QGIS3.12.app/Contents/MacOS/../Resources/python/plugins
/Applications/QGIS3.12.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python3.7/site-packages/geos
/Applications/QGIS3.12.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python3.7
/Applications/QGIS3.12.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python37.zip
/Applications/QGIS3.12.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python3.7/site-packages
/Applications/QGIS3.12.app/Contents/Resources/python
/Applications/QGIS3.12.app/Contents/Frameworks/Python.framework/Versions/Current/lib/python3.7/lib-dynload
/Users/luthfanahmad/Library/Application Support/QGIS/QGIS3/profiles/default/python
/Users/luthfanahmad/Downloads

Install Error on 3.16 QGIS

I had this plugin running about a week ago. I did an update to my QGIS, realized that the plugin did not work with 3.20 and reverted back to the long term version, 3.16. I am still getting the same error.

Couldn't load plugin 'CloudMasking' due to an error when calling its classFactory() method

The file it keeps getting confused with (I have tried on two separate systems now) is this one:
/CloudMasking/libs/fmask/_fillminima.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00

There seems to be an import error. I have tried installing via ZIP file and through the plugin library in QGIS.