equinor/xtgeo

XTGeo Python class library for subsurface Surfaces, Cubes, Wells, Grids, Points, etc

Introduction

XTGeo is a LGPL licensed Python library with C backend to support manipulation of (oil industry) subsurface reservoir modelling. Typical users are geoscientist and reservoir engineers working with reservoir modelling, in relation with RMS. XTGeo is developed in Equinor.

Detailed documentation for XTGeo at Read the Docs

Feature summary

• Python 3.8+ support
• Focus on high speed, using numpy and pandas with C backend
• Regular surfaces, i.e. 2D maps with regular sampling and rotation
• 3D grids (corner-point), supporting several formats such as RMS and Eclipse
• Support of seismic cubes, using segyio as backend for SEGY format
• Support of well data, line and polygons (still somewhat immature)
• Operations between the data types listed above; e.g. slice a surface with a seismic cube
• Optional integration with ROXAR API python for several data types (see note later)
• Linux is main development platform, but Windows and MacOS (64 bit) are supported and PYPI wheels for all three platforms are provided.

Installation

For Linux, Windows and MacOS 64bit, PYPI installation is enabled:

pip install xtgeo

For detailed installation instructions (implies C compiling), see the documentation.

Getting started

import xtgeo

# create an instance of a surface, read from file
mysurf = xtgeo.surface_from_file("myfile.gri")  # Irap binary as default

print(f"Mean is {mysurf.values.mean()}")

# change date so all values less than 2000 becomes 2000
# The values attribute gives the Numpy array

mysurface.values[mysurface.values < 2000] = 2000

# export the modified surface:
mysurface.to_file("newfile.gri")

Note on RMS Roxar API integration

The following applies to the part of the XTGeo API that is connected to Roxar API (RMS):

RMS is neither an open source software nor a free software and any use of it needs a software license agreement in place.

xtgeo's Issues

Defect in _gridprop_roxapi.py

Hei, there is a defect in the file _gridprop_roxapi.py

the call rprop.set_values() is missing the realization argument if the property is non-shared

Getting a shared property from RMS

While trying to export a Gaussian Random Field to disk, from RMS (11.0.1), I run into the following error:

import xtgeo

grf = project.grid_models['GridModelFine'].properties['GRF1']
print(grf)
other = xtgeo.gridproperty_from_roxar(project, 'GridModelFine', 'GRF1', project.current_realisation)

print(other)

Property(name='GRF1', type=_roxar.GridPropertyType.continuous, data_type=<class 'numpy.float32'>, shared=False)
Traceback (most recent call last):
  Python script, line 5
  File "/private/snis/.roxar/rms-11/python/lib/python3.6/site-packages/xtgeo/grid3d/grid_property.py", line 96, in gridproperty_from_roxar
    obj.from_roxar(project, gname, pname, realisation=realisation)
  File "/private/snis/.roxar/rms-11/python/lib/python3.6/site-packages/xtgeo/grid3d/grid_property.py", line 610, in from_roxar
    self, projectname, gname, pname, realisation
  File "/private/snis/.roxar/rms-11/python/lib/python3.6/site-packages/xtgeo/grid3d/_gridprop_roxapi.py", line 28, in import_prop_roxapi
    _get_gridprop_data(self, rox, gname, pname, realisation)
  File "/private/snis/.roxar/rms-11/python/lib/python3.6/site-packages/xtgeo/grid3d/_gridprop_roxapi.py", line 51, in _get_gridprop_data
    _convert_to_xtgeo_prop(self, rox, pname, roxgrid, roxprop)
  File "/private/snis/.roxar/rms-11/python/lib/python3.6/site-packages/xtgeo/grid3d/_gridprop_roxapi.py", line 67, in _convert_to_xtgeo_prop
    pvalues = roxprop.get_values()
ValueError: Realisation must be specified for non-shared data.

The script is run as a Python job in RMS on RHEL 6.10.

When resampling a well via rescale, it should be possible to restrict interval

e.g. well.rescale(delta=0.5, tvd = (1000, 2000)) to limit depth

Support for ROXAPI grid (corner point) export

Should be able to write XTGeo Grid to ROXAPI, using the new feature in RMS 11.1

RegularSurface fails when running multiple threads.

import logging
import concurrent.futures
from xtgeo import RegularSurface, Surfaces
import io
logger = logging.getLogger(__name__)

def get_files_as_regularsurfaces_thread(number_of_thread):
    surfs = []

    with concurrent.futures.ThreadPoolExecutor(max_workers=number_of_thread) as executor:

        futures = {executor.submit(get_file_as_regularsurface, i): i for i in range(0, number_of_thread)}

        for future in concurrent.futures.as_completed(futures):
            try:
                surf = future.result()
            except Exception as exc:
                logging.error(f'Error: {exc}')
            else:
                surfs.append(surf)

        regular_surfaces = Surfaces(surfs)
        return regular_surfaces



def get_file_as_regularsurface(i):
    logging.info(f'{i} start')

    file = 'files/1.gri'
    with open(file, "rb") as fin:
        stream = io.BytesIO(fin.read())

    rf = RegularSurface(file)
    logging.info(f'{i} end')
    return rf

if __name__ == '__main__':
    logging.basicConfig(level=logging.INFO)
    get_files_as_regularsurfaces_thread(20)

Gives error: Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

Enable initialization of RegularSurface object with byte array

For usage between an application and a blob storage, the concept of loading a file based on a file name breaks down. It would be beneficial to have the option to feed it a byte array, and initialize the object from that, instead of parsing the file based on a file name.

Fast loading of RegularSurface irap binary

Be able to read header only for fast loading of metadata. In general for all supported formats, but prioritize Irap binary format

Intersect wells/polygons with surface

Hi! I'm not sure if it's the best place to ask the question, if not, kindly point me elsewhere.
I'm using rms+xtgeo for some reservoir engineering and data-science problems.
Basically, I have a clean rms project without any grids/wells/horizons. I need to be able to make surfaces from points and read values from those surfaces at points given by x, y coordinates.

I didn't have any problems reading values from an existing surface at given x, y coordinates with xtgeo. The problem is I have to make my points and surfaces elsewhere, which is not what I want.

I want to get my points as intersections of wells' paths with stratigraphic surfaces (which I have loaded as regular surfaces). Is it possible to do using xtgeo? Since I don't need wells in my project per se, I could just load them as polygons (or true wells, doesn't really matter).

Also, is it possible to interpolate points to produce surfaces with xtgeo, without using other rms workflow-jobs (i.e. using pure python)? Ideally, my code should work even without rms, although at the moment it's not critical. Thank you!

Need flexibility on LOGS when getting in Roxar

The well_from_roxar method has a lognames key which is a list of well logs to be loaded. This will now fail with ValueError if a log is missing. Need this be more flexible

Points print fails

When trying to print a Point object, I get an error, although points are loaded successfully
I'm using Python 3.7.5, xtgeo 2.2.0

import pandas as pd
import numpy as np
import xtgeo
pfile = 'file.csv'
xp = xtgeo.points_from_file(pfile, fformat='xyz')
print(xp) # this fails
print(xp.dataframe) # this works fine

TypeError Traceback (most recent call last)
in
5 # xp = Points().from_file(f, fformat='xyz')
6 xp = xtgeo.points_from_file(pfile, fformat='xyz')
----> 7 print(xp) # this fails
8 print(xp.dataframe) # this works
9

TypeError: str returned non-string (type NoneType)

Direct usage of Python logging in C extensions

Simplify logging by common usage

Support for binary GRDECL formats

RMS 11.1 can output binary so-called binary GRDECL format. XTGeo should support read/write of this format.

Support for well distance maps

Compute horizontal distance from wells and make into a map.

Support folders in RMS when loading seismic cube

Cube window enhancements for Auto4D

Asymmetric window
Currently a symmetric window is defined above and below horizon. Window mode = Around horizon (symmetric)

Require additional option for an asymmetric window around horizon. Window mode = "Around horizon (asymmetric)"

This should be specified in the GUI in the same way as NRMS
•Vertical offset (ms) (default=0)
•Window above (ms) (default=10)
•Window below (ms) (default=10)

Extract attribute within a fixed window (T1 to T2)
Need a Fixed Window option to extract attribute between two user-specified Times (T1 to T2) or Depths (Z1 to Z2).

Scipy method in surface fill() is not in RMS scipy version?

RMS uses older scipy versions, or limited scipy versions(?)

Need to check on scipy versions for using fill()?

The scipy method is scipy.ndimage.distance_transform_edt which should be present in scipy 0.17,
while roxar API 11 says scipy 1.0... More investigations required.

Cannot load surface to RMS (from Python job inside RMS)

Hi!
I cannot load a surface from file to IRAP RMS (v11.1.0).
Trying to run this code

import os, sys
add_path = 'C:\\Program Files\\ROXAR\\RMS 11.1.0\\windows-amd64-vc_14_0-release\\bin\\Library\\bin'
if add_path not in os.environ['PATH']:
    os.environ['PATH'] = add_path + ';' + os.environ['PATH']

from xtgeo.surface import RegularSurface
map_path = r"d:\OWC.irap"
mysurf = RegularSurface(map_path, fformat='irap_ascii')
print('Mean is {}\n'.format(mysurf.values.mean()))
mysurf.to_roxar(project, 'TopReek', 'DS_whatever')

results in the following output:

Mean is 2101.7959529059276

Traceback (most recent call last):
  Python script, line 11
  File "C:\Program Files\ROXAR\RMS 11.1.0\windows-amd64-vc_14_0-release\bin\lib\site-packages\xtgeo\surface\regular_surface.py", line 928, in to_roxar
    self, project, name, category, stype, realisation
  File "C:\Program Files\ROXAR\RMS 11.1.0\windows-amd64-vc_14_0-release\bin\lib\site-packages\xtgeo\surface\_regsurf_roxapi.py", line 84, in export_horizon_roxapi
    rox = RoxUtils(project, readonly=False)
  File "C:\Program Files\ROXAR\RMS 11.1.0\windows-amd64-vc_14_0-release\bin\lib\site-packages\xtgeo\roxutils\roxutils.py", line 58, in __init__
    self._version = roxar.__version__
NameError: name 'roxar' is not defined

Note that the surface was read in ok, its mean was calculated and printed.
There are following lines in the source code where name roxar is defined:

try:
    import roxar
    import _roxar
except ImportError:
    pass

There are no files with name "roxar" in the xtgeo library directory. Where is this file supposed to be? Is my version of RMS not supported? What version of RMS is supported? Couldn't find this info in the docs. Thanks.

Write Well() back to RMS

Currently XTGeo can read from RMS ROXAPI and make them to Pandas. An enhancement would to to be able to write data back from Pandas to RMS ROXAPI

Snap points to surfaces

Surface IJXYZ must be more robust

Sometimes IJ XYZ files has a header and a footer, and I J may be Floats. E.g. @File_Version: 4
@Coordinate_Type_is: 3
@Export_Type_is: 1
@Number_of_Projects 1
....
#End_of_Horizon_ASCII_Header_
326.0 1160.0 536881.62764 6729209.96816 -1.00000
326.0 1161.0 536891.04756 6729218.18492 -1.00000
326.0 1162.0 536900.46749 6729226.40169 -1.00000
...
1686.0 2552.0 533231.87949 6759864.31445 -1.00000
1687.0 2550.0 533200.71443 6759862.01077 -0.79232
EOD

Enhance documentation, be ready for ReadTheDocs

• Consider change README.rst to README.md, must ensure Sphinx works
• Document install alternatives
• Continuously update the CHANGELOG
• More badges in README
• Test RTD uploads

When exporting a property, and fformat=... is mispelled, nothing is exported

E.g.

x.to_file("myfile.x", fformat="grdellclclcl") 

exports nothing

Python 3.8 wheels

🐍 Python 3.8 released Oct. 14 📦📆

Add Python 3.8 to xtgeo CI matrix and built wheels? 🎉

Import of ASCII GRDECL properties for Grid3D/GridProperty

Missing option

Let setup.py install swig

So that pip install git+https://github.com/equinor/xtgeo works

The Surfaces collection class should support different statistical calculations

It should be possible to calculate a range of statistics using this class.

This could be done by expanding the statistics() function to include more calculations,
or perhaps add a generic apply() function where the user can provide the statistic function to apply to the surface array.

When exporting a grid to `grdecl`, the given name is not stored

The name that is given in the generated .grdecl file is unknown, even though a name is explicitly given.

Example code:

import xtgeo
import numpy as np

nx, ny, nz = (24, 34, 40)

values = np.zeros((nx, ny, nz))

model = xtgeo.GridProperty(
    ncol=nx, nrow=ny, nlay=nz,
    values=values, 
)
model.to_file('./a_filed.grdecl', fformat='grdecl', name='GRF1')

Run on Windows

The goal is to make an PYPI for Windows 64 bits, for two reasons

• Different compilers strengtens code
• General question on using a WIndows install

Current status is that is compiles, but some C routines are much slower. Need to investigate

Additional attributes for Auto4D, cube averages

MaxPos (maximum positive)
•MaxNeg (maximum negative)
•MaxAbs (maximum absolute)
•SumPos (sum of positive values)
•SumNeg (sum of negative values)
•SumAbs (sum of absolute values)
•MeanAbs (arithmetic mean of absolute values)
•MeanPos (arithmetic mean of positive values)
•MeanNeg (arithmetic mean of negative values)

Support for grid faults, establish a class GridFault, GridFaults

Estiblish a class GridFault and GridFaults. The GridFault class should deal with one named fault, and be able to compute applied multipliers, throws, etc. GridFaults should be a collection of GridFault instances.

Investigate read from OW DB directly (wells)

Is it possible to read data from OW wells?

The benefit could be that RMS well master could import directly!

Need to determine if polygons import from RMS has ID column

If Points are imported as mistake instead of Polygons, the import goes well, but methods later will crash.

Need to verify that ID column is present at import

2.1 Development

• XSections of grid3D
• Extract surface from 3D grid
• New methods for surfaces etc

Make cube slicing a window more efficient

Attributes must be a list, and return a number of attribute surfaces

Surface unrotate does not cope with yflip -1

to_dataframe() method for RegularSurface

A Pandas dataframe view of the RegularSurface, with one "depth" value pr. row. Columns should be x and y coordinates, and possibly also integer indices (i,j) if meaningful.

get_randomline() on Grid raises ValueError if zmin and zmax are not specified

To reproduce:

import xtgeo

grid = xtgeo.Grid('xtgeo-testdata/3dgrids/reek/reek_geo_grid.roff')
prop = xtgeo.GridProperty('xtgeo-testdata/3dgrids/reek/reek_geo_hcpv.roff')
fencespec = xtgeo.Polygons('xtgeo-testdata/polygons/reek/1/mypoly.pol').get_fence()

#This raises a ValueError: negative dimensions are not allowed
grid.get_randomline(fencespec, prop)

#Works when specifying zmin, zmax 
geom = grid.get_geometrics(return_dict=True)
grid.get_randomline(fencespec, prop, zmin=geom['zmin'], zmax=geom['zmax'])

Expected:
According to docs zmin and zmax should be calculated automatically if not specified.

Actual:
Raises value error if zmin, zmax are not specified.

Why fix:
Using grid.get_geometrics() is quite slow. It would be useful if zmin, zmax could be calculated faster automatically.
It is also a bug.

Porting to Max OSX

At least version 10.14

XSections from 3D grids

This may be pretty difficult to get fast, but the idea is to sample maps from the 3D grid in order to locate I and J for the 3D grid

EGRID format not supported by ecl library

It seems like ERT's ecl.ecl library cannot read EGRID from XTGeo (just crashes). It appears it is related to Fortran block sizes of binary records.

When exporting gridprops to grdecl, undef values becomes very large

>>> import xtgeo
>>> g = xtgeo.Grid("../../xtgeo-testdata/3dgrids/etc/TEST_SP.EGRID", fformat="egrid")
>>> p = xtgeo.GridProperty("../../xtgeo-testdata/3dgrids/etc/TEST_SP.INIT", fformat="init", name="PORO", grid=g)
>>> p.to_file("xxx", fformat="grdecl")
>>> 
$ more xxx
PORO    
        0.1000        0.1100 999999994495727286427992885035008.0000        0.1300        0.1400 999999994495727286427992885035008.0000
        0.1600        0.1700        0.1800        0.1900        0.2000 999999994495727286427992885035008.0000
        0.2200 999999994495727286427992885035008.0000        0.2400
/

When sampling discrete prop to well, it should be discrete log

well.get_gridproperties(facies, ...) should distinguish between continuous and discrete

Add six to list of dependecies

six is an explicit dependency of xtgeo and should hence be added to the list of dependencies.

Support jsonwelllogformat

Wells should have an export option to this JSON format:

https://jsonwelllogformat.org/

Create a rectangular shoebox grid from simple spec

Adjust or snap 3D grid to surfaces

Add the possibility to "snap" a grid to a surface and/or adjust by difference map; this was present in the old Perl version of XTGeo.

Add copy() methods for XYZ data

Handle Eclipse output with dual porosity model

Dual porosity ECL output should be handled. Examples in xtgeo-testdata 3dgrids/etc/COARSE_SCALE* files.

Support for RMS clipboard i/o for surfaces

Add examples on IPL to Python in RMS

Docs: Add examples on python with XTGeo instead of IPL