Giter Site home page Giter Site logo

map2loop-2-legacy's Introduction

Map2Loop 2.0

Generate 3D geological model inputs from geographical maps — a high-level implementation and extension of the original map2loop code developed by Prof. Mark Jessell at UWA. To see an example interactive model built with map2loop and LoopStructural, follow this link:

3D Model from the Hamersley region, Western Australia

Install

You will need some flavour of conda (a python package manager, see here), as well as Python ≥ 3.6

Run

To just use map2loop, issue the following

conda install -c conda-forge -c loop3d map2loop -y

Documentation

If you can call it that, is available here

Development

If you want to tinker yourself/contribute, clone the source code with

git clone https://github.com/Loop3D/map2loop-2.git

Or get the source + example notebooks with

git clone https://github.com/Loop3D/map2loop-2.git
git clone --single-branch --branch yohan https://github.com/Loop3D/map2loop2-notebooks

Navigate into map2loop-2, and issue the following to install map2loop and its dependencies. Note: The 'develop' flag makes your source changes take effect on saving, so you only need to run this once

conda install -c loop3d --file dependencies.txt
python setup.py install

Building with Docker

Fair warning, we recommend conda to almost everyone. With great software development power comes great environment setup inconvenience. You'll need to download and install the docker containerisation software, and the docker and docker-compose CLI.

Development

  1. Clone this repo and navigate inside as per above

  2. Run the following and click on the Jupyter server forwarded link to access and edit the notebooks

    docker-compose up --build
  3. To hop into a bash shell in a running container, open a terminal and issue

    docker ps

    Find the container name or ID and then run

    docker exec -it <container_NAMEorID> bash
    # Probably -> docker exec -it  map2loop-2_dev_1 bash

Usage

Our notebooks cover use cases in more detail, but here is an example of processing Loop's South Australia remote geospatial data in just 20 lines of Python.

First, lets import map2loop and define a bounding box. You can use GIS software to find one or use Loop's Graphical User Interface for the best experience and complete toolset. Remember what projection your coordinates are in!

from map2loop.project import Project

bbox_3d = {
    'minx': 250805.1529856466,
    'miny': 6405084.328058686,
    'maxx': 336682.921539395,
    'maxy': 6458336.085975628,
    'base': -3200,
    'top': 1200
}

sa example

Then, specify: the state, directory for the output, the bounding box and projection from above - and hit go! That's it.

proj = Project(loopdata_state="SA")

proj.update_config(out_dir='sa-remote',
                   overwrite="true",
                   bbox_3d=bbox_3d,
                   proj_crs={'init': 'EPSG:28354'},
                #    drift_prefix=['T', 'Q', 'water', 'void'],
                #    quiet='no-figures',
                   )
proj.run()

This is a minimal example and a small part of Loop.

Our documentation and other resources outline how to extend map2loop and port to the LoopStructural modelling engine. We are working to incorporate geophysical tools and best provide visualisation and workflow consolidation in the GUI.

Loop is led by Laurent Ailleres (Monash University) with a team of Work Package leaders from:

  • Monash University: Roy Thomson, Lachlan Grose and Robin Armit
  • University of Western Australia: Mark Jessell, Jeremie Giraud, Mark Lindsay and Guillaume Pirot
  • Geological Survey of Canada: Boyan Brodaric and Eric de Kemp

Known Issues and FAQs

  • Developing with docker on Windows means you won't have GPU passthrough and can’t use a discrete graphics card in the container even if you have one.
  • If Jupyter links require a token or password, it may mean port 8888 is already in use. To fix, either make docker map to another port on the host ie -p 8889:8888 or stop any other instances on 8888.
  • Sometimes the submodules misbehave. Ensure you specify the recurse-submodules flag with an 's' as opposed to recurse-submodule, and double-check your .git directory is clean, see #41

Links

https://loop3d.github.io/

https://github.com/Loop3D/LoopStructural

map2loop-2-legacy's People

Contributors

markjessell avatar yohanderose avatar roythomsonmonash avatar lachlangrose avatar github-actions[bot] avatar t4mmi avatar markdlindsay avatar richardscottoz avatar

Stargazers

321YY avatar  avatar  avatar Zhangbo avatar  avatar  avatar Rian Dutch avatar  avatar Yichuan avatar  avatar Rodrigo Brust avatar Gabriel avatar  avatar Santosh avatar  avatar Erdong avatar Jonathan Davidson avatar  avatar PaulineCollon avatar Denis Anikiev avatar Andrea Balza avatar  avatar  avatar  avatar Ashton Krajnovich avatar

Watchers

James Cloos avatar  avatar  avatar Dominic Brown avatar  avatar  avatar  avatar

map2loop-2-legacy's Issues

[BUG] structures all have 90 degree dips

Describe your issue

after running map2loop, all the structures in orientation_clean.csv have a dip of 90 and the models thus have very steep surfaces

Minimal reproducing code example

from map2loop.project import Project 
from map2loop.m2l_enums import VerboseLevel 
import shutil
import os

proj = Project(
                geology_filename='../inputs/geology_coded.shp',
                fault_filename='../inputs/faults.shp',
                fold_filename='../inputs/faults.shp',
                structure_filename='../inputs/bedding.shp',
                mindep_filename='http://13.211.217.129:8080/geoserver/loop/wfs?service=WFS&version=1.0.0&request=GetFeature&typeName=loop:null_mindeps&bbox={BBOX_STR}&srs=EPSG:28350&outputFormat=shape-zip',
                dtm_filename='../inputs/Moulouya_dtm_small.tif',
                metadata_filename='./data.json',
                overwrite='false',
                verbose_level=VerboseLevel.ALL,
                project_path='../m2l_outputs',
                working_projection='epsg:26191',
                )
 
proj.update_config(
                    out_dir='../m2l_outputs',
                    bbox_3d={'minx': 710718, 'miny': 229811, 'maxx': 739675, 'maxy': 251774, 'base': -2900, 'top': 2100},
                    run_flags={'aus': True, 'close_dip': -999.0, 'contact_decimate': 5, 'contact_dip': -999.0, 'contact_orientation_decimate': 5, 'deposits': 'Fe,Cu,Au,NONE', 'dist_buffer': 10.0, 'dtb': '', 'fat_step': 750.0, 'fault_decimate': 5, 'fault_dip': 90.0, 'fold_decimate': 5, 'interpolation_scheme': 'scipy_rbf', 'interpolation_spacing': 500.0, 'intrusion_mode': 0, 'max_thickness_allowed': 10000.0, 'min_fault_length': 2000.0, 'misorientation': 30.0, 'null_scheme': 'null_scheme', 'orientation_decimate': 5, 'pluton_dip': 45.0, 'pluton_form': 'domes', 'thickness_buffer': 5000.0, 'use_fat': False, 'use_interpolations': False, 'fault_orientation_clusters': 2, 'fault_length_clusters': 2, 'use_roi_clip': False, 'roi_clip_path': '','ignore_codes':['q']},
                    proj_crs='epsg:26191',
                    clut_path='',
                )
if(not os.path.isdir(proj.config.project_path+'/MINERAL_DEPOSIT')):
    os.mkdir(proj.config.project_path+'/MINERAL_DEPOSIT')
if(not os.path.isdir(proj.config.project_path+'/FAULT')):
    os.mkdir(proj.config.project_path+'/FAULT')
if(not os.path.isdir(proj.config.project_path+'/GEOLOGY')):
    os.mkdir(proj.config.project_path+'/GEOLOGY')
if(not os.path.isdir(proj.config.project_path+'/STRUCTURE')):
    os.mkdir(proj.config.project_path+'/STRUCTURE')
if(not os.path.isdir(proj.config.project_path+'/FOLD')):
    os.mkdir(proj.config.project_path+'/FOLD')
proj.workflow['fold_axial_traces']=False
#proj.workflow['contact_dips'] = True
proj.run()

Error message

n/a

[BUG]

Describe your issue

When I run Example 2 - Draw Your Own Area from map2loop2-notebooks, I get an error!

Minimal reproducing code example

import os
from map2loop.project import Project
from map2loop.m2l_enums import VerboseLevel
from datetime import datetime
import time
import warnings


t0 = time.time()
warnings.filterwarnings('ignore')
nowtime=datetime.now().isoformat(timespec='minutes')   
model_name=nowtime.replace(":","-").replace("T","-")
proj = Project(
                loopdata_state="WA",
                overwrite="true",
                verbose_level=VerboseLevel.NONE,
                project_path=model_name,
                working_projection="EPSG:28350",
                )

proj.update_config(
                    out_dir=model_name,
                    bbox_3d={
                         "minx": mbbox.total_bounds[0], #500000,
                         "miny": mbbox.total_bounds[1], #7490000,
                         "maxx": mbbox.total_bounds[2], #545000,
                         "maxy": mbbox.total_bounds[3], #7520000,
                         "base": -4800,
                         "top": 1200,
                    },
                    #loopFilename='test.loop3d',
                    run_flags={                        
                        'aus': True,
                        'close_dip': -999,
                        'contact_decimate': 5,
                        'contact_dip': -999,
                        'contact_orientation_decimate': 5,
                        'deposits': "Fe,Cu,Au,NONE",
                        'dist_buffer': 10,
                        'dtb': '',
                        'fat_step': 750,
                        'fault_decimate': 5,
                        'fault_dip': -999,
                        'fold_decimate': 5,
                        'interpolation_scheme': 'scipy_rbf',
                        'interpolation_spacing': 500,
                        'intrusion_mode': 0,
                        'max_thickness_allowed': 10000,
                        'min_fault_length': 5000,
                        'misorientation': 30,
                        'null_scheme': 'null',
                        'orientation_decimate': 0,
                        'pluton_dip': 45,
                        'pluton_form': 'domes',
                        'thickness_buffer': 5000,
                        'use_fat': False,
                        'use_interpolations': False,
                        'fault_orientation_clusters':2,
                        'fault_length_clusters':2,
                        'map2graph':True,
                        'granular_map2graph':True
                    },
                  )
proj.config.c_l['intrusive']='intrusive'
#proj.config.c_l['intrusive']='mafic intrusive'
proj.workflow['fold_axial_traces']=False
proj.run()
t1 = time.time()
print("m2l",(t1-t0)/60.0,"minutes")

Error message

ValueError                                Traceback (most recent call last)
Cell In[8], line 21
     12 model_name=nowtime.replace(":","-").replace("T","-")
     13 proj = Project(
     14                 loopdata_state="WA",
     15                 overwrite="true",
   (...)
     18                 working_projection="EPSG:28350",
     19                 )
---> 21 proj.update_config(
     22                     out_dir=model_name,
     23                     bbox_3d={
     24                          "minx": mbbox.total_bounds[0], #500000,
     25                          "miny": mbbox.total_bounds[1], #7490000,
     26                          "maxx": mbbox.total_bounds[2], #545000,
     27                          "maxy": mbbox.total_bounds[3], #7520000,
     28                          "base": -4800,
     29                          "top": 1200,
     30                     },
     31                     #loopFilename='test.loop3d',
     32                     run_flags={                        
     33                         'aus': True,
     34                         'close_dip': -999,
     35                         'contact_decimate': 5,
     36                         'contact_dip': -999,
     37                         'contact_orientation_decimate': 5,
     38                         'deposits': "Fe,Cu,Au,NONE",
     39                         'dist_buffer': 10,
     40                         'dtb': '',
     41                         'fat_step': 750,
     42                         'fault_decimate': 5,
     43                         'fault_dip': -999,
     44                         'fold_decimate': 5,
     45                         'interpolation_scheme': 'scipy_rbf',
     46                         'interpolation_spacing': 500,
     47                         'intrusion_mode': 0,
     48                         'max_thickness_allowed': 10000,
     49                         'min_fault_length': 5000,
     50                         'misorientation': 30,
     51                         'null_scheme': 'null',
     52                         'orientation_decimate': 0,
     53                         'pluton_dip': 45,
     54                         'pluton_form': 'domes',
     55                         'thickness_buffer': 5000,
     56                         'use_fat': False,
     57                         'use_interpolations': False,
     58                         'fault_orientation_clusters':2,
     59                         'fault_length_clusters':2,
     60                         'map2graph':True,
     61                         'granular_map2graph':True
     62                     },
     63                   )
     64 proj.config.c_l['intrusive']='intrusive'
     65 #proj.config.c_l['intrusive']='mafic intrusive'

File <@beartype(map2loop.project.Project.update_config) at 0x1700519cd30>:48, in update_config(__beartype_func, __beartype_conf, __beartype_get_violation, *args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\project.py:435, in Project.update_config(self, overwrite, bbox_3d, dtm_crs, step_out, clut_path, run_flags, **kwargs)
    432     if self.state in ["WA", "NSW", "VIC", "SA", "QLD", "ACT", "TAS"]:
    433         clut_path = clut_paths[self.state]
--> 435 self.config.update(
    436     self.project_path,
    437     bbox_3d,
    438     polygon,
    439     step_out,
    440     dtm_crs,
    441     self.map_data.working_projection,
    442     self.map_data.get_filename(Datatype.METADATA),
    443     clut_path=clut_path,
    444     run_flags=run_flags,
    445 )
    447 self.map_data.load_all_map_data(self.config)

File <@beartype(map2loop.config.Config.update) at 0x170046715e0>:66, in update(__beartype_func, __beartype_conf, __beartype_get_violation, *args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\config.py:162, in Config.update(self, project_path, bbox_3d, polygon, step_out, dtm_crs, project_crs, metadata_filename, clut_path, run_flags)
    159 self.project_crs = project_crs
    161 self.read_metadata(metadata_filename)
--> 162 self.create_cmap()

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\config.py:204, in Config.create_cmap(self)
    197 def create_cmap(self):
    198     # Make colours consistent from map to model
    199     formations = sorted(
    200         [
    201             formation.replace(" ", "_").replace("-", "_")
    202             for formation in list(
    203                 set(
--> 204                     self.map_data.get_map_data(Datatype.GEOLOGY)[
    205                         "UNIT_NAME"
    206                     ].to_numpy()
    207                 )
    208             )
    209         ]
    210     )
    211     temp_colours = [""] * len(formations)
    212     self.colour_dict = dict(zip(formations, temp_colours))

File <@beartype(map2loop.mapdata.MapData.get_map_data) at 0x1700502ee50>:31, in get_map_data(__beartype_func, __beartype_conf, __beartype_get_violation, __beartype_object_1580561287408, *args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\mapdata.py:366, in MapData.get_map_data(self, datatype)
    363 @beartype.beartype
    364 def get_map_data(self, datatype: Datatype):
    365     if self.data_states[datatype] != Datastate.COMPLETE:
--> 366         self.load_map_data(datatype)
    367     return self.data[datatype]

File <@beartype(map2loop.mapdata.MapData.load_map_data) at 0x1700502e700>:31, in load_map_data(__beartype_func, __beartype_conf, __beartype_get_violation, __beartype_object_1580561287408, *args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\mapdata.py:196, in MapData.load_map_data(self, datatype)
    193     self.data_states[datatype] = Datastate.CLIPPED
    194 if self.data_states[datatype] == Datastate.CLIPPED:
    195     # Convert column names using codes_and_labels dictionary
--> 196     self.check_map(datatype)
    197     self.data_states[datatype] = Datastate.CONVERTED
    198 if self.data_states[datatype] == Datastate.CONVERTED:

File <@beartype(map2loop.mapdata.MapData.check_map) at 0x1700502e940>:31, in check_map(__beartype_func, __beartype_conf, __beartype_get_violation, __beartype_object_1580561287408, *args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\mapdata.py:256, in MapData.check_map(self, datatype)
    254 _errors = []
    255 if datatype == Datatype.GEOLOGY:
--> 256     self.data[Datatype.GEOLOGY] = m2l_map_checker.check_geology_map(
    257         self.data[Datatype.GEOLOGY],
    258         self.config.c_l,
    259         self.config.run_flags["ignore_codes"],
    260         _warnings,
    261         _errors,
    262         True,
    263         self.config.verbose_level,
    264     )
    265 if datatype == Datatype.STRUCTURE:
    266     self.data[Datatype.STRUCTURE] = m2l_map_checker.check_structure_map(
    267         self.data[Datatype.STRUCTURE],
    268         self.config.c_l,
   (...)
    271         self.config.verbose_level,
    272     )

File D:\Anaconda3\envs\test39\lib\site-packages\map2loop\m2l_map_checker.py:454, in check_geology_map(geology, c_l, ignore_codes, m2l_warnings, m2l_errors, explode_intrusives, verbose_level)
    450 elif len(geology[~geology[c_l["o"]].isnull()] != len(geology)):
    451     vals = geology[geology[c_l["o"]].astype(str).str.isnumeric()][
    452         c_l["o"]
    453     ].astype(int)
--> 454     next_index = int(np.nanmax(vals)) + 1
    455     for ind, layer in geology.iterrows():
    456         if pd.isna(layer[c_l["o"]]):

File <__array_function__ internals>:180, in nanmax(*args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\numpy\lib\nanfunctions.py:483, in nanmax(a, axis, out, keepdims, initial, where)
    480 else:
    481     # Slow, but safe for subclasses of ndarray
    482     a, mask = _replace_nan(a, -np.inf)
--> 483     res = np.amax(a, axis=axis, out=out, **kwargs)
    484     if mask is None:
    485         return res

File <__array_function__ internals>:180, in amax(*args, **kwargs)

File D:\Anaconda3\envs\test39\lib\site-packages\numpy\core\fromnumeric.py:2793, in amax(a, axis, out, keepdims, initial, where)
   2677 @array_function_dispatch(_amax_dispatcher)
   2678 def amax(a, axis=None, out=None, keepdims=np._NoValue, initial=np._NoValue,
   2679          where=np._NoValue):
   2680     """
   2681     Return the maximum of an array or maximum along an axis.
   2682 
   (...)
   2791     5
   2792     """
-> 2793     return _wrapreduction(a, np.maximum, 'max', axis, None, out,
   2794                           keepdims=keepdims, initial=initial, where=where)

File D:\Anaconda3\envs\test39\lib\site-packages\numpy\core\fromnumeric.py:86, in _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs)
     83         else:
     84             return reduction(axis=axis, out=out, **passkwargs)
---> 86 return ufunc.reduce(obj, axis, dtype, out, **passkwargs)

ValueError: zero-size array to reduction operation maximum which has no identity

to do (mwj)

  1. Calc secondary age relationships for faults (A cuts B, B cuts C so A cuts C)
  2. Exclude intrusions from supergroups
  3. Look for thickness variations across faults
  4. Record all data reoved by m2l
  5. Group Faults by orientation
  6. True fault offset with Z
  7. Add estimated fault offset using strat offset for 0 & 1 offsets... (and add 50% of unit for others to be equivalent)
  8. Calc true fault offset with fold limbs
  9. Contact dip estimate with large Z variation
  10. Save contact normals with polarity
  11. Create geol/fault/oc group layes for all states

[Question]

Example 2 - Draw Your Own Area——Error
proj.update_config()

--> 447 self.map_data.load_all_map_data(self.config)
NameError: map2loop error: Could not access DTM server after 10 attempts
What can I do about this error?

[BUG] Thickness result seems to have changed between 1.3.5 and 1.3.6

Describe your issue

Running a notebook under colab using map2loop 1.3.6@6IAS (branch to fix error in main branch on line 451 of m2l_map_checker.py):

https://colab.research.google.com/github/Loop3D/6IAS/blob/main/map2loop/1a_Building_a_model_from_local_sources.ipynb

gives me only 16 formation thickness estimates, and each one is positionally duplicated with a different thickness estimate.

running v 1.3.5 not in colab gives me 286 points also with duplicates, and I assume the issue with the number of points is the version, not the fact it is colab?

So there are two issues, the duplicates, which is a long-standing weirdness as far as I can tell, and the different number of measuremetns for the same input data between 1.3.5 & 1.3.6

Minimal reproducing code example

see colab link above

Error message

No response

Fault topology Bug [BUG]

Describe your issue

The output from map2model of fault-strat topology as

unit-fault-intersection.txt

is not completely carried over to

group-fault-relationships.csv

Minimal reproducing code example

unit-fault-intersection.txt:
0, jc, {1}
1, jm1, {0, 1, 3}
2, jm2, {0, 1, 3}
3, l2, {0}
4, l3, {2, 3}
5, l4, {1, 2}

group-fault-relationships.csv:
group	Fault_0	Fault_2	Fault_1	Fault_3
all	1	1	1	0

So Fault_3 is forgotten

Error message

No response

conda install fails

following instructions in readme, I'm in Windows 10, Anaconda3,
run
conda install -c conda-forge -c loop3d map2loop -y
get 'packageNotFoundError' for map2loop...

loop project file update does not work

Trying to run proj.config.update_projectfile() using m2l2 does not work.

File ./..loop3d already exists
File ./..loop3d already exists

ValueError Traceback (most recent call last)
in
----> 1 proj.config.update_projectfile()

~/map2loop/map2loop/config.py in update_projectfile(self)
982 self.output_path,
983 self.bbox_3d,
--> 984 self.proj_crs)
985
986 print("PROJECTFILE FOUND AT", self.loop_projectfile)

~/map2loop/map2loop/m2l_export.py in export_to_projectfile(loopFilename, tmp_path, output_path, bbox, proj_crs)
1790 "faultLog",
1791 data=faultEvents,
-> 1792 verbose=False)
1793 if resp["errorFlag"]:
1794 print(resp["errorString"])

/opt/conda/lib/python3.7/site-packages/LoopProjectFile/LoopProjectFile.py in Set(filename, element, **kwargs)
200 elif element == "stratigraphicLog": response = ExtractedInformation.SetStratigraphicLog(root, **kwargs)
201 elif element == "stratigraphicLogAppend": response = ExtractedInformation.SetStratigraphicLog(root, append=True, **kwargs)
--> 202 elif element == "faultLog": response = ExtractedInformation.SetFaultLog(root, **kwargs)
203 elif element == "faultLogAppend": response = ExtractedInformation.SetFaultLog(root, append=True, **kwargs)
204 elif element == "foldLog": response = ExtractedInformation.SetFoldLog(root, **kwargs)

/opt/conda/lib/python3.7/site-packages/LoopProjectFile/ExtractedInformation.py in SetFaultLog(root, data, append, verbose)
106
107 def SetFaultLog(root, data, append=False, verbose=False):
--> 108 return SetEventLog(root, data, 'faultEventIndex', 'faultEvents', append, verbose)
109
110 def SetFoldLog(root, data, append=False, verbose=False):

/opt/conda/lib/python3.7/site-packages/LoopProjectFile/ExtractedInformation.py in SetEventLog(root, data, indexName, variableName, append, verbose)
97 index = elGroup.dimensions[indexName].size
98 for i in data:
---> 99 eventLocation[index] = i
100 index += 1
101 else:

src/netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.setitem()

src/netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable._put()

ValueError: structures must have the same size

Create an abstract class for interpolation

I want to be able to use different interpolation algorithms for interpolating the orientation cosines and thicknesses - in theory using a discrete approach should be a lot better when we have a lot of data...

So can we create a base interpolation class that is used for the interpolation throughout map2loop and then create child classes that manage how each specific interpolator needs to be setup? Basically the same thing I have done with LoopStructural using the GeologicalInterpolator and the different daughter classes for discrete inteprolation approaches?

error with read only shapefile access

This happened when trying to load data from the remote sources for western australia.


CPLE_AppDefinedError Traceback (most recent call last)
fiona/_shim.pyx in fiona._shim.gdal_open_vector()

fiona/_err.pyx in fiona._err.exc_wrap_pointer()

CPLE_AppDefinedError: Only read-only mode is supported for /vsicurl

During handling of the above exception, another exception occurred:

DriverError Traceback (most recent call last)
fiona/ogrext.pyx in fiona.ogrext.WritingSession.start()

fiona/_shim.pyx in fiona._shim.gdal_open_vector()

DriverError: Only read-only mode is supported for /vsicurl

During handling of the above exception, another exception occurred:

CPLE_AppDefinedError Traceback (most recent call last)
fiona/_shim.pyx in fiona._shim.gdal_create()

fiona/_err.pyx in fiona._err.exc_wrap_pointer()

CPLE_AppDefinedError: /vsicurl/http://geo.loop-gis.org/geoserver/loop/wfs?service=WFS&version=1.1.0&request=GetFeature&typeName=linear_500k&bbox=500253.49057027657,7484158.223645598,509618.3857942696,7495253.0617621&srs=EPSG:28350 is not a directory.

During handling of the above exception, another exception occurred:

DriverError Traceback (most recent call last)
in
20 proj_crs={'init': 'EPSG:28350'},
21 # quiet=True
---> 22 overwrite='true',
23 )

~/map2loop-2/map2loop/project.py in update_config(self, out_dir, overwrite, loopFilename, bbox_3d, dtm_crs, proj_crs, step_out, quiet, **kwargs)
259 )
260
--> 261 self.config.preprocess()
262
263 def read_metadata(self, filename):

~/map2loop-2/map2loop/config.py in preprocess(self)
171 geology.to_file(self.geology_file)
172 faults.to_file(self.fault_file)
--> 173 folds.to_file(self.fold_file)
174 if len(mindeps) > 0:
175 mindeps.to_file(self.mindep_file)

/opt/conda/lib/python3.7/site-packages/geopandas/geodataframe.py in to_file(self, filename, driver, schema, index, **kwargs)
1084 from geopandas.io.file import _to_file
1085
-> 1086 _to_file(self, filename, driver, schema, index, **kwargs)
1087
1088 def set_crs(self, crs=None, epsg=None, inplace=False, allow_override=False):

/opt/conda/lib/python3.7/site-packages/geopandas/io/file.py in _to_file(df, filename, driver, schema, index, mode, crs, **kwargs)
326 crs_wkt = crs.to_wkt("WKT1_GDAL")
327 with fiona.open(
--> 328 filename, mode=mode, driver=driver, crs_wkt=crs_wkt, schema=schema, **kwargs
329 ) as colxn:
330 colxn.writerecords(df.iterfeatures())

/opt/conda/lib/python3.7/site-packages/fiona/env.py in wrapper(*args, **kwargs)
398 def wrapper(*args, **kwargs):
399 if local._env:
--> 400 return f(*args, **kwargs)
401 else:
402 if isinstance(args[0], str):

/opt/conda/lib/python3.7/site-packages/fiona/init.py in open(fp, mode, driver, schema, crs, encoding, layer, vfs, enabled_drivers, crs_wkt, **kwargs)
272 c = Collection(path, mode, crs=crs, driver=driver, schema=this_schema,
273 encoding=encoding, layer=layer, enabled_drivers=enabled_drivers, crs_wkt=crs_wkt,
--> 274 **kwargs)
275 else:
276 raise ValueError(

/opt/conda/lib/python3.7/site-packages/fiona/collection.py in init(self, path, mode, driver, schema, crs, encoding, layer, vsi, archive, enabled_drivers, crs_wkt, ignore_fields, ignore_geometry, **kwargs)
165 elif self.mode in ('a', 'w'):
166 self.session = WritingSession()
--> 167 self.session.start(self, **kwargs)
168 except IOError:
169 self.session = None

fiona/ogrext.pyx in fiona.ogrext.WritingSession.start()

fiona/ogrext.pyx in fiona.ogrext.WritingSession.start()

fiona/_shim.pyx in fiona._shim.gdal_create()

DriverError: /vsicurl/http://geo.loop-gis.org/geoserver/loop/wfs?service=WFS&version=1.1.0&request=GetFeature&typeName=linear_500k&bbox=500253.49057027657,7484158.223645598,509618.3857942696,7495253.0617621&srs=EPSG:28350 is not a directory.

Bedding point automation?

Anyone tinkering here?

One of the geos expressed interest and I said can automatically make project models.... but only if we have this part.

local variable 'project_crs' referenced before assignment

I tried to run the proj.run() function using my own data and then got this error

local variable 'project_crs' referenced before assignment

I saw that I could use working_projection instead, but I got the same error for project_crs.

the details of the error are below:

`---------------------------------------------------------------------------
UnboundLocalError Traceback (most recent call last)
Input In [2], in <cell line: 14>()
2 from map2loop.project import Project
4 proj = Project(
5 geology_file='C:/Users/rcha0044/LoopPhD/ArcGIS files/Flinders Ranges/Shapefiles/Odla_wirra_area_east_by_minage_GDA94_2.shp',
6 fault_file='',
(...)
11 metadata='./oodla_wirra_east_config.hjson'
12 )
---> 14 proj.update_config(
15 out_dir='oodla_wirra_east',
16 overwrite='true',
17 bbox_3d={
18 'minx': 3.27330661e+05 ,
19 'miny': 6.32155212e+06 ,
20 'maxx': 3.38277181e+05 ,
21 'maxy': 6.36982397e+06 ,
22 'base': 0,
23 'top': 1},
24 run_flags={'local' : True,
25 'aus': True,
26 'close_dip': -999,
27 'contact_decimate': 5,
28 'contact_dip': -999,
29 'contact_orientation_decimate': 5,
30 'deposits': "NONE",
31 'dist_buffer': 10,
32 'dtb': '',
33 'fat_step': 750,
34 'fault_decimate': 5,
35 'fault_dip': 90,
36 'fold_decimate': 5,
37 'interpolation_scheme': 'scipy_rbf',
38 'interpolation_spacing': 500,
39 'intrusion_mode': 0,
40 'max_thickness_allowed': 10000,
41 'min_fault_length': 5000,
42 'misorientation': 30,
43 'null_scheme': 'null',
44 'orientation_decimate': 0,
45 'pluton_dip': 45,
46 'pluton_form': 'domes',
47 'thickness_buffer': 5000,
48 'use_fat': False,
49 'use_interpolations': False,
50 'fault_orientation_clusters':False,
51 'fault_length_clusters':False,
52 'use_roi_clip':False,
53 'roi_clip_path':'',
54 },
55 proj_crs={'init': 'epsg:28354'},
56 clut_path= '',
57 quiet='None'
58 )
60 proj.run()

File C:\Anaconda3\envs\map2loop3\lib\site-packages\map2loop-1.2.8-py3.9.egg\map2loop\m2l_utils.py:1482, in timer_decorator..clean(*args, **kwargs)
1481 def clean(*args, **kwargs):
-> 1482 return func(*args, **kwargs)

File <@beartype(map2loop.project.Project.update_config) at 0x2b74b183040>:47, in update_config(__beartype_func, __beartype_raise_exception, *args, **kwargs)

File C:\Anaconda3\envs\map2loop3\lib\site-packages\map2loop-1.2.8-py3.9.egg\map2loop\project.py:380, in Project.update_config(self, overwrite, bbox_3d, dtm_crs, step_out, clut_path, model_engine, run_flags, **kwargs)
377 bbox_3d = {'minx':0,'maxx':1000,',miny':0,'maxy':1000,'base':-10000,'top':1200}
379 if self.map_data.working_projection == None:
--> 380 if project_crs in kwargs:
381 warnings.warn("project_crs in update_config is deprecated and will be removed in later versions of map2loop. Please use the working_projection parameter in Project instead", DeprecationWarning)
382 project_crs = kwargs['proj_crs']

UnboundLocalError: local variable 'project_crs' referenced before assignment`

add back in access to fold orientation data

It looks like the function m2l_geometry.save_fold_axial_traces_orientations() is not currently accessed in project.py.

When we reconnect it the parameter ‘syn’ in the hjson config file defines the text that the the field ‘t’ contains to show it is a syncline.

[FEATURE] table links to structural vocabulary

Is your feature request related to a problem? Please describe.
One of the datasets we don't currently store in an accessible manner are the relationships that the 3D structures will use. These need to be interpreted from existing reports, concepts and explicit 3D models.
There are early vocabularies (see GKM) that use rdf triples to describe relationships for things like faults that LoopStructural needs.

Describe the solution you'd like
a plugin for QGIS that generates an sqlite database (or a text table) from the GKM or other vocabulary and allows us to:
Assign available & permitted relationships (validated against vocabularies) for objects as rdf triples for subject-predicant-object as well as any attributes that we may need to add. for example a faults option that allows selecting of crosscutting or abutting or any other relationship or attribute defined for a fault in the appropriate vocabulary.

Describe alternatives you've considered
users create the database or table manually in a specified format to capture relevant information as per each user.

Use threading to speed up map2loop

Would it be possible to use threading to do multiple tasks at the same time? I noticed that there wasn't a huge speedup using more cores with map2loop.

Unable to access dtm as it does not get created consistently

 10%|███████▎                                                                 | 10/100 [00:07<01:05,  1.38it/s]
Done
Fetching DTM...  bbox:117.04431041708905 117.62994886664823 -22.79826965843374 -22.393797034787504
Attempt: 0   1
 10%|███████▎                                                                 | 10/100 [00:18<02:44,  1.83s/it]
dtm geotif saved as ../scratch/dtm/dtm.tif
Done.
---------------------------------------------------------------------------
CPLE_OpenFailedError                      Traceback (most recent call last)
rasterio\_base.pyx in rasterio._base.DatasetBase.__init__()
rasterio\_shim.pyx in rasterio._shim.open_dataset()
rasterio\_err.pyx in rasterio._err.exc_wrap_pointer()
CPLE_OpenFailedError: '../scratch/dtm/dtm.tif' not recognized as a supported file format.
During handling of the above exception, another exception occurred:
RasterioIOError                           Traceback (most recent call last)
<ipython-input-6-2bb76722df30> in <module>
     26                     quiet=False
     27                   )
---> 28 proj.run()
~\AppData\Roaming\Python\Python37\site-packages\map2loop-1.0.8-py3.7.egg\map2loop\project.py in run(self)
    248             pbar.update(10)
    249 
--> 250             self.config.load_dtm()
    251             pbar.update(10)
    252 
~\AppData\Roaming\Python\Python37\site-packages\map2loop-1.0.8-py3.7.egg\map2loop\config.py in load_dtm(self)
    319 
    320         geom_rp = m2l_utils.reproject_dtm(
--> 321             self.dtm_file, self.dtm_reproj_file, self.dtm_crs, self.proj_crs)
    322         self.dtm = rasterio.open(self.dtm_reproj_file)
    323         if not self.quiet:
~\AppData\Roaming\Python\Python37\site-packages\map2loop-1.0.8-py3.7.egg\map2loop\m2l_utils.py in reproject_dtm(path_in, path_out, src_crs, dst_crs)
    273 
    274 def reproject_dtm(path_in, path_out, src_crs, dst_crs):
--> 275     with rasterio.open(path_in) as src:
    276         transform, width, height = calculate_default_transform(
    277             src.crs, dst_crs, src.width, src.height, *src.bounds)
~\Anaconda3\envs\m2l2\lib\site-packages\rasterio\env.py in wrapper(*args, **kwds)
    443 
    444         with env_ctor(session=session):
--> 445             return f(*args, **kwds)
    446 
    447     return wrapper
~\Anaconda3\envs\m2l2\lib\site-packages\rasterio\__init__.py in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, **kwargs)
    217         # None.
    218         if mode == 'r':
--> 219             s = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
    220         elif mode == 'r+':
    221             s = get_writer_for_path(path)(path, mode, driver=driver, sharing=sharing, **kwargs)
rasterio\_base.pyx in rasterio._base.DatasetBase.__init__()
RasterioIOError: '../scratch/dtm/dtm.tif' not recognized as a supported file format.

Convert map2loop code to object oriented

All of the original m2l code should be converted into object-oriented code - off the top of my head i think it would be sensible to have a data server class. I think you could do most of this inside a data server class:
self.config.export_csv() self.config.load_dtm() self.config.join_features() self.config.calc_depth_grid(dtb)
You can then have an export class/option that will extract things from the data server and put it in the right format either a csv or as a pandas dataframe.

I've made a diagram of how I see this working, obviously we need to discuss the specifics.

path991

Hamersley notebook not json error

When executing the Hamersley map processing notebook, the following error appears.

image

FYI both "0 Test" and the "...leaflet..." example work.

install fail because of readme.MD problem

Installing from code of map2loop currently fails with:

(loop) D:\Dropbox\1_Jupyter_notebooks\map2loop-2>python setup.py develop
Traceback (most recent call last):
File "setup.py", line 39, in
long_description = fh.read()
File "D:\Anaconda\envs\loop\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 644: character maps to

Workaround: commenting out lines 37-39 of setup.py fixes the problem with no adverse side effects.

[BUG] iso characters

Describe your issue

If CODE field entries have accents, e.g. "Amphibolites_et_métagabbros" then networkx fails

probably true for GROUP entries as well?

see https://stackoverflow.com/questions/61789659/networkx-impossible-to-read-my-gml-file-input-is-not-ascii-encoded

Minimal reproducing code example

use accents in a field that will be used as CODE

Error message

File "/home/mark/map2loop-2_latest/map2loop-2/map2loop/topology.py", line 39, in __init__
    self.graph = nx.read_gml(config.strat_graph_filename, label="id")
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/utils/decorators.py", line 766, in func
    return argmap._lazy_compile(__wrapper)(*args, **kwargs)
  File "<class 'networkx.utils.decorators.argmap'> compilation 5", line 5, in argmap_read_gml_1
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 195, in read_gml
    G = parse_gml_lines(filter_lines(path), label, destringizer)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 438, in parse_gml_lines
    graph = parse_graph()
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 427, in parse_graph
    curr_token, dct = parse_kv(next(tokens))
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 373, in parse_kv
    curr_token, value = parse_dict(curr_token)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 421, in parse_dict
    curr_token, dct = parse_kv(curr_token)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 373, in parse_kv
    curr_token, value = parse_dict(curr_token)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 421, in parse_dict
    curr_token, dct = parse_kv(curr_token)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 358, in parse_kv
    curr_token = next(tokens)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 314, in tokenize
    for line in lines:
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/networkx/readwrite/gml.py", line 188, in filter_lines
    raise NetworkXError("input is not ASCII-encoded") from err

Unexpected Error when pulling submodules (notebooks)

Hi,

Thanks for refactoring of the repo, we have trouble pulling submodule (map2loop2-notebooks).
It throws an unexpected error without log/traceback.

Nothing blocking but might require a very little patch.

this is more of a feedback than an issue...

[Question]

Hello.
I created a new virtual environment.I'm stuck here in the picture below while installing map2loop, what should I do to fix this.
Thanks for your help.
image

map checker with no mineral deposits

on line 439 of m2l_map_checker.py throws error if no mineral deposits available with len(mindeps) as mindeps is NoneType

workaround add

mindeps=0

after line 352

m2l_warnings.append('no mindeps for analysis')

[BUG] Only a few contact orientations are calculated

Describe your issue

If I set 'contact_decimate': 0, and 'contact_orientation_decimate': 0 I expect the same number of contact orientations as contacts, but I get 1658 contacts in contacts_clean.csv but only 14 contact orientations in contact_orientations.csv

Minimal reproducing code example

Try with the default dataset included in the notebooks

Error message

No error, but surprising behaviour...

Remove plotting from calculation code

Can the plotting be removed from the calculation code and then additional functions be added in, to be able to create the plots.

You could have a flag that allows these plots to be called during the workflow.

Add ability to configure map_checker

Can we have the ability to control behaviour of map_checker (to begin with min_fault_length) but on the long term control various other map_checker and Ranee map simplification codes

[BUG] exported shapefiles don't have filenames and structures are polylines

Describe your issue

proj.save_mapdata_to_shapefiles(proj.config.project_path)

produces a series of directories(e.g. GEOLOGY/.shp) which contain a shapefile called ".shp.shp"

In addition the STRUCTURES shapefile is a polyline, but should be oints, and is empty

Minimal reproducing code example

proj.run()
proj.save_mapdata_to_shapefiles(proj.config.project_path)

Error message

n/a

[BUG] field renaming issues

Describe your issue

A couple of errors arising now that field names are reset in map2loop

  1. map2loop fails if I use the same column for "min" and "max" in the json dictionary. Often I hard define the stratigraphic order for maps because they are not provided by the original map. Creating a single column of ages then makes sense, but now fails becasue the column name is changed to MIN_AGE for the first case and therefore is no longer available for the seond usage.

  2. Similarly map2loop fails if one of the original columns was called CODE as this will cause a clash when the column that is defined by "c" in the json file attempts to be renamed. This will happen if ANY of the previous field names happen to coincide with those defined in the json file.

Minimal reproducing code example

1) set min and max to the same field

2) set any field name to a pre-defined internal name

Error message

1)   File "/home/mark/map2loop-2_latest/map2loop-2/map2loop/mapdata.py", line 317, in check_map
    raise NameError(
NameError: ERROR: data does not contain column SO_ORDER

2)   File "/home/mark/map2loop-2_latest/map2loop-2/map2loop/mapdata.py", line 470, in export_wkt_format_files
    sub_geol = geology[columns]
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/geopandas/geodataframe.py", line 1415, in __getitem__
    result = super().__getitem__(key)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/pandas/core/frame.py", line 3767, in __getitem__
    indexer = self.columns._get_indexer_strict(key, "columns")[1]
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5876, in _get_indexer_strict
    self._raise_if_missing(keyarr, indexer, axis_name)
  File "/home/mark/anaconda3/envs/m2l2-py39/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5938, in _raise_if_missing
    raise KeyError(f"{not_found} not in index")
KeyError: "['CODE'] not in index"

warnings when incorrect parameters are given

import os
import hjson
from map2loop.project import Project
proj = Project( 
                 loopdata_state = "WA",)
)                

proj.update_config(
                    out_dir='./gmdpaper-faultslip',
                    bbox_3d=bbox_3d,
                    proj_crs={'init': 'EPSG:28350'},
                    overwrite='true',
                  )

If overwrite is set to a boolean type, True it does not work. It should at least tell the user its the wrong type, considering the logical type for this parameter is a boolean

[BUG] ignore_codes not taken into account in save_basal_contacts()

Describe your issue

I use the run_flag ignore_codes to ignore surficial geology, but as far as I can tell the basal contacts which are used to extract contact polylines for later use by map2loop don't reflect the run_flag ignore_codes.

As a result contacts between surficial polygons and real geology polygons are kept, resulting in 3D model weirdness.

Minimal reproducing code example

same code and data maroc dataset I sent wrt issue #100

Error message

No response

[BUG] typo in m2l_geometry

Describe your issue

crashes in m2l_geometry:

line 273

agroup = apoint["GROUP"]

should be (I think)

agroup = apoly["GROUP"]

Minimal reproducing code example

changing to             agroup = apoly["GROUP"]

seems to fix it...

Error message

Traceback (most recent call last):
  File "Run_test.py", line 39, in <module>
    proj.run()
  File "/home/mark/anaconda3/envs/loop/lib/python3.8/site-packages/map2loop/m2l_utils.py", line 1400, in clean
    return func(*args, **kwargs)
  File "/home/mark/anaconda3/envs/loop/lib/python3.8/site-packages/map2loop/project.py", line 563, in run
    self.map_data.export_orientations(self.workflow)
  File "<@beartype(map2loop.mapdata.MapData.export_orientations) at 0x7f5609422280>", line 31, in export_orientations
  File "/home/mark/anaconda3/envs/loop/lib/python3.8/site-packages/map2loop/mapdata.py", line 674, in export_orientations
    m2l_geometry.create_orientations(self.config, self, workflow)
  File "<@beartype(map2loop.m2l_geometry.create_orientations) at 0x7f56091f6ee0>", line 71, in create_orientations
  File "/home/mark/anaconda3/envs/loop/lib/python3.8/site-packages/map2loop/m2l_geometry.py", line 273, in create_orientations
    agroup = apoint["GROUP"]
UnboundLocalError: local variable 'apoint' referenced before assignment

Overwriting the output dir is unpredictable

  • On some computers shutil.rmtree takes longer than the sleep time allocated
  • It might be useful to have an 'inplace' option which would allow users to have map2loop output files open in other software while rerunning map2loop

rejig run parameters

Can we have all run parameters defined in the dictionary, rather than having some as arguments and some in the dictionary.

I guess only required parameters should remain as parameters (such as bbox, proj_crs etc) that will cause code to fail if not set...?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.