ezwelty/glimpse

glimpse: Glacier Image Particle Sequencer

glimpse's Issues

Issue with equality types (isinstance more-so for inheritance?)

There is a quirk here: https://github.com/ezwelty/glimpse/blob/rewrite/glimpse/optimize.py#L1340

I resolved it with this:
`
if type(control) is Points:
xyz.append(control.xyz)
elif type(control) is Lines:
xyz.extend(control.xyzs)
'

TypeError: match_keypoints() got an unexpected keyword argument 'as_type'

Error:

`
                File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2597, in build_matches
                    matches = pool.map(
                  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 719, in map
                    rt = [realreduce(realfunc(i)) for i in sequence]
                  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 719, in <listcomp>
                    rt = [realreduce(realfunc(i)) for i in sequence]
                  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 711, in realfunc
                    if star: return func(*i)
                  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2570, in process
                    result = match_keypoints(
                TypeError: match_keypoints() got an unexpected keyword argument 'as_type'
                
`

Optimize focal length and principal point independent of image size

The current optimization scheme allows optimizing the focal length in pixels (Camera.f). However, this only works if all images have the same pixel dimensions, since:

f = focal_length (mm) x image_size (pixels) / sensor_size (mm)

Same with the principal point offset (Camera.c):

c = principal_point_offset (mm) x image_size (pixels) / sensor_size (mm)

Determine horizon coordinates from DEM

Given a DEM and a viewing position, return the 3-D coordinates of the horizon as a polyline. We will need this for line-based camera calibration.

Read and write Camera to and from file

Since optimized camera models are difficult and time-consuming to produce, it makes sense to be able to write them to file (and read them back in). I would suggest a simple JSON so that they're text files readable by both humans and machines.

Sphinx error: master file glimpse/docs/contents.rst not found

Fixed issue according to this thread: https://stackoverflow.com/questions/56336234/build-fail-sphinx-error-contents-rst-not-found

added master_doc = "index" here: https://github.com/ezwelty/glimpse/blob/master/docs/conf.py#L53

Provide access to original (pre-normalized) particle weights

It would be better to return pre-normalized particle weights from the tracker, since we could then directly merge the particle populations of forward, backward, and repeat runs before computing the weighted population mean and standard deviation. Currently, we are limited to merging the weighted population mean and standard deviation of each run.

Improve writing of SVG files

• Clarify that SVG child elements are drawn in order, and thus from bottom to top layer as viewed in a vector editor like Adobe Illustrator
• Add <circle> element creator:

import xml.etree.ElementTree as ET
ET.Element('circle', cx=x, cy=y, **kwargs)

• Stringify non-string element attributes
• Add helper for scaling an :

SCALE = 0.25
glimpse.svg.svg(
  glimpse.svg.image(width=width, height=height, transform=f'matrix({SCALE} 0 0 {SCALE} 0 0)'),
  ET.Element('circle', cx=uv * SCALE, cy=uv * SCALE),
  width=str(width * SCALE),
  height=str(height * SCALE)
)

Stabilize Levenberg–Marquardt optimization

Optimization often fails because the Levenberg–Marquardt algorithm (scipy.optimize.root) steps are currently equal for all variables, leading it into non-sensical parts of the parameter space.

I will first try to compute scale factors based on the pixel displacements that result from modulating each variable by a certain equal.

It may be further necessary to provide extreme variable bounds using something like lmfit.

Confusing terminology around image pre-processing steps

The sub-images used for tracking are pre-processed in part with a median filter. In argument names and docstrings, this is referred to as both a 'low-pass' filter (which it is) and 'high-pass' filter (which it is not).

glimpse/src/glimpse/track/tracker.py

Line 31 in 7112a76

highpass (dict): Median high-pass filter

"RecursionError: maximum recursion depth exceeded"

`
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2046, in build_matches
      self.matcher.convert_matches(RotationMatchesXYZ)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2661, in convert_matches
      _ = pool.map(
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 719, in map
      rt = [realreduce(realfunc(i)) for i in sequence]
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 719, in <listcomp>
      rt = [realreduce(realfunc(i)) for i in sequence]
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 711, in realfunc
      if star: return func(*i)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2652, in process
      m = m.to_type(mtype)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 652, in to_type
      return mtype(cams=self.cams, uvs=self.uvs, weights=self.weights)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 965, in __init__
      super().__init__(cams=cams, uvs=uvs, xys=xys, weights=weights)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 869, in __init__
      self.uvs, self.xyz = self._initialize_uvs_xys(uvs, xys)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 975, in __getattribute__
      if name in self._EXCLUDED:
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 975, in __getattribute__
      if name in self._EXCLUDED:
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 975, in __getattribute__
      if name in self._EXCLUDED:
    [Previous line repeated 984 more times]
  

`

error: OpenCV(4.2.0) /io/opencv_contrib/modules/xfeatures2d/src/sift.cpp:1210: error: (-213:The function/feature is not implemented)

"error: OpenCV(4.2.0) /io/opencv_contrib/modules/xfeatures2d/src/sift.cpp:1210: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'"

There seems to be a dependency issue with the latest version of opencv2. I tried various versions of pip/conda uninstall and installs, but to no avail.

Related Stack Overflow Posts: https://stackoverflow.com/questions/52669705/attributeerror-module-cv2-cv2-has-no-attribute-xfeatures2d-opencv-3-4-3/52685991#52685991

Fragile parallel processing

Many methods in glimpse benefit from placing large arrays into shared memory to share across parallel processes. We currently accomplish this task using sharedmem, which has worked brilliantly but has some downsides:

• Ease of use over multiprocessing.Pool comes at the cost of not supporting Windows. However, this comment by sharedmem's author suggests that this can be addressed by installing Windows Subsystem for Linux (WSL, https://docs.microsoft.com/en-us/windows/wsl/install-win10).
• Not actively maintained, which could lead to trouble down the road. For example, MapReduce no longer seems to work on Python 3.8 (see rainwoodman/sharedmem#24), requiring the slower MapReduceByThread to be used (glimpse.config.backend = sharedmem.MapReduceByThread).

Possible alternatives to consider:

• multiprocessing, which as of Python 3.8 implements SharedMemory (https://docs.python.org/3/library/multiprocessing.shared_memory.html).
• joblib, especially once/if SharedMemory is supported (joblib/joblib#1094, joblib/joblib#915).
• ray, based on the tutorial here: https://towardsdatascience.com/10x-faster-parallel-python-without-python-multiprocessing-e5017c93cce1.

Automatically filter out low-visibility images

I've explored a few simple approaches on a test sample of 890 images (AK10b 20170512 service) which I manually classified as either good or bad. Good images included those with good visibility on the glacier surface, but poor visibility elsewhere.

Results are given alongside the classification results from the highest-performing threshold in percent of images selected as good.

@douglas-brinkerhoff

• Which do you think is more important – minimizing false negatives or false positives?
• Would we also want to filter out images with good visibility over the glacier, but poor visibility elsewhere, since these cannot be corrected for camera motion? Since it would be very hard to know whether or not there are sufficient patches of good visibility, I think it would probably be best to detect this failure mode during the alignment stage (e.g. too few SIFT keypoint matches above a threshold, no RANSAC solution within tolerance, etc).

Filesize

A very fast approach which uses the existing image file size, relying on the fact that JPEG compression more aggressively compresses low-contrast images.

False negatives: 19% | False positives: 0%

Filesize (local)

Similar to above. Non-glacier pixels are replaced with 0 and the image is recompressed as a JPEG.

False negatives: 13% | False positives: 0%

Contrast (local)

Compute the RMS contrast over glacier pixels.

False negatives: 46% | False positives: 0%

Entropy (local)

Compute Shannon's entropy over glacier pixels.

False negatives: 7% | False positives: 0.1%
False negatives: 43% | False positives: 0%

Gradients (local)

Compute the mean of gradient magnitudes over glacier pixels.

False negatives: 3% | False positives: 1%
False negatives: 20% | False positives: 0%

" AttributeError: 'Image' object has no attribute 'anchor' "

  `
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2017, in __init__
      is_anchor = [img.anchor for img in self.observer.images]
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2017, in <listcomp>
      is_anchor = [img.anchor for img in self.observer.images]
  
  `

Write guide to time-lapse photo management

Outline of possible topics:

• Track which camera, took which image, from where
  • Notes in the field: camera and lens make, model, serial number, camera position
  • Assign unique codes to camera positions ('station') versus camera systems ('camera')
• Directory structure and file names (e.g. {station code}/{service date}/{camera directory}/{camera filename})
  • Group images taken with the same camera and settings
  • Ensure chronological sorting
  • Robust to frame counter reset (e.g. 1023 to 1000) or recycling (e.g. 9999 to 0001)
  • Independent of image capture time, since this may need to be revised in the future (e.g. wrong timezone)
• Check the integrity of an existing image archive using image filenames and metadata
  • Check for monotonic increasing filenames and capture times
  • Find discontinuities in camera, lens, and image attributes

IndexError: Box extends beyond grid bounds

Trace:

`Traceback (most recent call last):
  File "trackhelheim.py", line 95, in <module>
    tracks = tracker.track(motion_models=motion_models, parallel=True)
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/track/tracker.py", line 364, in track
    results = pool.map(
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 770, in map
    raise pg.get_exception()
sharedmem.sharedmem.WorkerException: Box extends beyond grid bounds
Traceback (most recent call last):
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 295, in _workerMain
    self.main(self, *self.args)
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 631, in _main
    r = realfunc(work)
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 711, in realfunc
    if star: return func(*i)
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/track/tracker.py", line 348, in process
    raise e
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/track/tracker.py", line 322, in process
    self.initialize_template(
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/track/tracker.py", line 526, in initialize_template
    box = self.observers[obs].tile_box(uv, size=tile_size, img=img)
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/track/observer.py", line 130, in tile_box
    return grid.snap_box(uv, size, centers=False, edges=True).astype(int)
  File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/raster.py", line 417, in snap_box
    raise IndexError("Box extends beyond grid bounds")
IndexError: Box extends beyond grid bounds
`

Code:

`path = glob.glob(join(DEM_DIR,"*.tif"))[0]
print("DEM PATH: {}".format(path))
dem = glimpse.Raster.open(path=path)

    dem.crop(zlim=(0, np.inf))
    dem.fill_crevasses(mask=~np.isnan(dem.array), fill=True)
    
    # ---- Prepare viewshed ----
    for obs in observers:
        dem.fill_circle(obs.images[0].cam.xyz, radius=50)
    viewshed = dem.copy()
    viewshed.array = np.ones(dem.shape, dtype=bool)
    for obs in observers:
        viewshed.array &= dem.viewshed(obs.images[0].cam.xyz)
    
    print("\n *****Viewshed Done**** \n")
    # ---- Run Tracker ----
    xy = []
    xy0 = np.array([ 533528.0,7361411.0])[np.newaxis,:]
    #xy.append(xy0)
    
    xy = [xy0] #qu+ np.vstack([xy for xy in itertools.product(range(-200, 200, 25), range(-200, 200, 25))])
    
    # Helheim vels around 0.8 m/day
    
    time_unit = datetime.timedelta(days=1)
    motion_models = [glimpse.CartesianMotion(
        xyi, time_unit=time_unit, 
        dem=dem, 
        dem_sigma=2.5, 
        n=5000, 
        xy_sigma=(0.5,0.5),
        vxyz_sigma=(.4, .3, 0.02),
        axyz=(0,0,0), 
        axyz_sigma=(.25, .25, .05)) for xyi in xy]
    
    tracker = glimpse.Tracker(observers=observers, viewshed=viewshed)
    print("\n****Tracking Now*****\n")
    tracks = tracker.track(motion_models=motion_models, parallel=True)`

From what I have seen of the code, it looks as though this call should be under the Try/except branch (https://github.com/ezwelty/glimpse/blob/master/src/glimpse/track/tracker.py#L300). My best guess is that the projected camera coordinates called at (https://github.com/ezwelty/glimpse/blob/master/src/glimpse/track/observer.py#L130) are not in the camera image?

cam_bounds: IndexError: tuple index out of range

Code:
https://github.com/frigusgulo/wolverineglacier/blob/master/scripts/opt_viewdirs.py#L136

It looks as though self.group_indices is causing trouble. Is there any workaround?
Using the "group_indices" argument did not help.

**
IndexError Traceback (most recent call last)
in
----> 1 glimpse.optimize.Cameras(cams = cams, controls=np.array(tounge_matcher.matches.data),cam_params=dict(viewdir=True),group_indices=len(cams),group_params=dict(viewdir=True) )

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in init(self, cams, controls, cam_params, group_indices, group_params, weights, scales, sparsity)
871 # Build lmfit parameters
872 # params, cam_masks, group_masks, cam_breaks, group_breaks for set_cameras()
--> 873 self.update_params()
874 # Test for errors
875 self._test()

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in update_params(self)
1189 for group, idx in enumerate(self.group_indices):
1190 bounds = np.column_stack((
-> 1191 np.column_stack([cam_bounds[i][:, 0] for i in idx]).max(axis=1),
1192 np.column_stack([cam_bounds[i][:, 1] for i in idx]).min(axis=1)))
1193 mask, bounds = self.class.parse_params(self.group_params[group], default_bounds=bounds)

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in (.0)
1189 for group, idx in enumerate(self.group_indices):
1190 bounds = np.column_stack((
-> 1191 np.column_stack([cam_bounds[i][:, 0] for i in idx]).max(axis=1),
1192 np.column_stack([cam_bounds[i][:, 1] for i in idx]).min(axis=1)))
1193 mask, bounds = self.class.parse_params(self.group_params[group], default_bounds=bounds)

IndexError: tuple index out of range

**

Test relative camera motion on real image pairs

Currently, our method for recovering relative camera motion has only been tested on a synthetic image generated by rotating the camera and projecting the image into that camera. A more realistic test is to use two images from the same camera separated by visible camera motion. The quality of the solution can be assessed by projecting one image into the other and visually checking for any unaccounted for motion.

Error in optimize.Cameras.fit()

"ValueError: zero-size array to reduction operation maximum which has no identity"
Getting this error when trying to calculate the camera parameters with 4 image to world correspondence points.

Choose and apply a license

For a list of options, see https://opensource.org/licenses (and https://tldrlegal.com). I am personally open to a more permissive license like MIT or New BSD.

Add support for `pathlib.Path` path arguments

Many functions accept a path as argument, type-annotated as str. Many of these functions may not work when called with a pathlib.Path.

• Update type annotations to Union[str, pathlib.Path]
• Update functions to support pathlib.Path (list not exhaustive):
  • glimpse.exif.Exif(path)

AttributeError: 'Matches' object has no attribute 'as_type'

Error:

 '
   File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py",
line 2567, in process
     match = match.as_type(as_type)
 AttributeError: 'Matches' object has no attribute 'as_type'
 
 '

Code:

`

			matcher.build_keypoints(
				clear_images=True,
    			overwrite=True,
    			masks=mask)

			matcher.build_matches(
    			maxdt=MAXDT, 
    			seq=MATCH_SEQ,
    			path=TOUNGE_MATCHES_PATH,
		        max_ratio=0.75,
		        weights=True,
		        as_type=glimpse.optimize.RotationMatchesXY
		        )

`

Translate camera calibration routines to python

Calibrating cameras with control lines (e.g., horizons, coastlines) adds a great deal of complexity, so I'll start with control points, then add more complexity. All these features exist in the MATLAB code.

• Optimize camera with control points
• Orient camera with points matching an oriented camera
• Filter point correspondences with RANSAC
• Optimize multiple cameras simultaneously
• Optimize camera with control lines

AttributeError: 'NoneType' object has no attribute 'cams'

I am getting an inexplicable error in my code:
https://github.com/frigusgulo/wolverineglacier/blob/master/scripts/opt_viewdirs.py#L128
Despite checking the documents, it would appear that the list of cameras is not jiving with the class instantiation.
'''
AttributeError Traceback (most recent call last)
~/Research/wolverine/wolverineglacier/scripts/opt_viewdirs.py in
125
126
--> 127 Cameras = glimpse.optimize.Cameras([image.cam for image in tounge_images],[tounge_matcher.matches],cam_params=dict(viewdir=True) )
128 new_params = Cameras.fit()
129 Cameras.set_cameras(new_params)

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in init(self, cams, controls, cam_params, group_indices, group_params, weights, scales, sparsity)
856 # Core attributes
857 self.cams = cams
--> 858 controls = self.class.prune_controls(controls, cams=self.cams)
859 self.controls = controls
860 ncams = len(self.cams)

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in prune_controls(controls, cams)
939 list: Control which reference the cameras in cams
940 """
--> 941 return [control for control in controls
942 if len(set(cams) & set(control.cams)) > 0]
943

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in (.0)
940 """
941 return [control for control in controls
--> 942 if len(set(cams) & set(control.cams)) > 0]
943
944 @staticmethod

AttributeError: 'NoneType' object has no attribute 'cams'

``

Camera dict not saving certain parameters

`params= dict(
sensorsz=(35.9, 24),
xyz=(393506.713, 6695855.64, 961.3370),
viewdir=(6.55401513e+01, -1.95787507e+01, 7.90589866e+00),
p=(1.42603100e-01,4.12163305e-01),
k=(-4.78577912e-01, 1.08772049e+00, -1.27452460e+00,6.78479918e-01, 2.39852156e-02, -1.229e-01),
imgsz=(7360, 4912),
fmm=28
)

camera = glimpse.Camera(**params)

camera.as_dict()

{'xyz': [393506.713, 6695855.64, 961.337],
'viewdir': [65.5401513, -19.5787507, 7.90589866],
'imgsz': [7360.0, 4912.0],
'f': [5740.389972144847, 5730.666666666667],
'c': [0.0, 0.0],
'k': [-0.478577912,
1.08772049,
-1.2745246,
0.678479918,
0.0239852156,
-0.1229],
'p': [0.1426031, 0.412163305]}`

Camera object initialization not defaulting to integer values

When instantiating a Camera object manually (as opposed to from a .JSON), there is difficulty with the object when passed to an Image instantiation. It appears that by default, Camera.imgsz is set as a float which causes issues when instantiating an Observer object.

TypeError: in method 'Driver_Create', argument 3 of type 'int'

There is an issue with returning a boolean raster from a polygon:

Context: Taking 2d projection of a point cloud and trying to produce a viewshed via convex hull

code:

scan_hull = list(ConvexHull(points[:,:2].simplices.astype(np.float32))
scan_hull = [tuple(point) for point in scan_hull]
viewshed.array &= dem.rasterize_polygons([scan_hull])

Error:

return _gdal.Driver_Create(self, *args, **kwargs)
TypeError: in method 'Driver_Create', argument 3 of type 'int'

Function Def Typo: "def rasterize_poygons"

https://github.com/ezwelty/glimpse/blob/master/src/glimpse/raster.py#L1119

matcher.convert_matches(glimpse.optimize.RotationmatchesXY) issue

Code: https://github.com/frigusgulo/wolverineglacier/blob/master/scripts/viewdirs.py#L80

`IndexError Traceback (most recent call last)
in
----> 1 viewopt.run()

~/Research/wolverine/wolverineglacier/scripts/viewdirs.py in run(self)
90 Cameras = None
91 fit = None
---> 92 subSet = []
93 def run(self):
94 self.modelRef()

~/Research/wolverine/wolverineglacier/scripts/viewdirs.py in iterMatch(self, setSize)
78 )
79
---> 80 print("\nFitting\n")
81 fit = Cameras.fit(ftol=1)
82 print("\nFitting Complete\n")

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in fit(self, index, cam_params, group_params, full, method, nan_policy, reduce_fcn, **kwargs)
1383 self.update_params()
1384 result = lmfit.minimize(params=self.params, fcn=self.residuals, kws=dict(index=index), iter_cb=callback,
-> 1385 method=method, nan_policy=nan_policy, reduce_fcn=reduce_fcn, **kwargs)
1386 sys.stdout.write('\n')
1387 if iterations:

~/anaconda3/lib/python3.7/site-packages/lmfit-0.9.11-py3.7.egg/lmfit/minimizer.py in minimize(fcn, params, method, args, kws, scale_covar, iter_cb, reduce_fcn, **fit_kws)
2137 iter_cb=iter_cb, scale_covar=scale_covar,
2138 reduce_fcn=reduce_fcn, **fit_kws)
-> 2139 return fitter.minimize(method=method)

~/anaconda3/lib/python3.7/site-packages/lmfit-0.9.11-py3.7.egg/lmfit/minimizer.py in minimize(self, method, params, **kws)
1809 val.lower().startswith(user_method)):
1810 kwargs['method'] = val
-> 1811 return function(**kwargs)
1812
1813

~/anaconda3/lib/python3.7/site-packages/lmfit-0.9.11-py3.7.egg/lmfit/minimizer.py in least_squares(self, params, **kws)
1280 bounds=(lower_bounds, upper_bounds),
1281 kwargs=dict(apply_bounds_transformation=False),
-> 1282 **kws)
1283 except AbortFitException:
1284 pass

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_lsq/least_squares.py in least_squares(fun, x0, jac, bounds, method, ftol, xtol, gtol, x_scale, loss, f_scale, diff_step, tr_solver, tr_options, jac_sparsity, max_nfev, verbose, args, kwargs)
905 result = trf(fun_wrapped, jac_wrapped, x0, f0, J0, lb, ub, ftol, xtol,
906 gtol, max_nfev, x_scale, loss_function, tr_solver,
--> 907 tr_options.copy(), verbose)
908
909 elif method == 'dogbox':

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_lsq/trf.py in trf(fun, jac, x0, f0, J0, lb, ub, ftol, xtol, gtol, max_nfev, x_scale, loss_function, tr_solver, tr_options, verbose)
122 return trf_no_bounds(
123 fun, jac, x0, f0, J0, ftol, xtol, gtol, max_nfev, x_scale,
--> 124 loss_function, tr_solver, tr_options, verbose)
125 else:
126 return trf_bounds(

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_lsq/trf.py in trf_no_bounds(fun, jac, x0, f0, J0, ftol, xtol, gtol, max_nfev, x_scale, loss_function, tr_solver, tr_options, verbose)
542 cost = cost_new
543
--> 544 J = jac(x, f)
545 njev += 1
546

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_lsq/least_squares.py in jac_wrapped(x, f)
863 J = approx_derivative(fun, x, rel_step=diff_step, method=jac,
864 f0=f, bounds=bounds, args=args,
--> 865 kwargs=kwargs, sparsity=jac_sparsity)
866 if J.ndim != 2: # J is guaranteed not sparse.
867 J = np.atleast_2d(J)

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_numdiff.py in approx_derivative(fun, x0, method, rel_step, f0, bounds, sparsity, as_linear_operator, args, kwargs)
398 return _sparse_difference(fun_wrapped, x0, f0, h,
399 use_one_sided, structure,
--> 400 groups, method)
401
402

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_numdiff.py in _sparse_difference(fun, x0, f0, h, use_one_sided, structure, groups, method)
553 row_indices.append(i)
554 col_indices.append(j)
--> 555 fractions.append(df[i] / dx[j])
556
557 row_indices = np.hstack(row_indices)

IndexError: index 6740134 is out of bounds for axis 0 with size 6740134

`

I am sure there was something missed in the .fit() method call. Do you have any clue what could lead to this error?
Also, any critique on my approach here for generalizing the viewdir fitting process would be welcome.

AttributeError: 'function' object has no attribute 'iter'

`
   File "viewdirs.py", line 61, in buildmatches
      keypoints.build_matches()
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2574, in build_matches
      matches = pool.map(
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 719, in map
      rt = [realreduce(realfunc(i)) for i in sequence]
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 719, in <listcomp>
      rt = [realreduce(realfunc(i)) for i in sequence]
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 711, in realfunc
      if star: return func(*i)
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/glimpse/optimize.py", line 2570, in process
      with pool.ordered:
    File "/home/dunbar/anaconda3/envs/glimpse/lib/python3.8/site-packages/sharedmem/sharedmem.py", line 475, in __enter__
      while self.counter.value != self.tls.iter:
  AttributeError: 'function' object has no attribute 'iter'
  
`

Improve Exif tag support and performance

We currently use piexif for reading image metadata. This works fine, but faster (and more actively maintained) libraries may be faster and/or be able to parse manufacturer-specific maker notes.

Alternatives to consider include:

• GDAL, since we are using it already (https://gdal.org/drivers/raster/jpeg.html#exif-and-gps-tags)
• pyexiv2 (https://github.com/LeoHsiao1/pyexiv2)
• exif-py (https://github.com/ianare/exif-py)
• PyExifTool (https://github.com/smarnach/pyexiftool) or another Python wrapper of exiftool

Todo

• Add support for Exif(path: pathlib.Path) (only Exif(path: str) is currently supported)
• Add properties to access additional tags useful for camera system forensics
  • Camera serial number: BodySerialNumber, CameraSerialNumber, maker notes (e.g. Canon.SerialNumber, Nikon3.SerialNumber)
  • Lens serial number: LensSerialNumber
  • Lens make: LensMake
  • Lens model: LensModel
  • Lens info (focal range, aperture range, etc): LensSpecification, LensInfo, maker notes (e.g. Canon.LensModel, Canon.LensInfo)
  • Shutter count: maker notes (e.g. Nikon3.ImageCount)
  • Frame number: file name (e.g. 'IMG_1234.JPG'), maker notes (e.g. Canon.FileNumber, NikonFi.FileNumber, NikonFi.DirectoryNumber)
  • GPS coordinates: GPSLatitudeRef, GPSLatitude, GPSLongitudeRef, GPSLongitude, GPSAltitudeRef, GPSAltitude, GPSMapDatum
  • GPS datetime: GPSDateStamp, GPSTimeStamp

"AttributeError: 'Lines' object has no attribute 'xyz'"

line 1340, in camera_scales xyz.append(control.xyz) AttributeError: 'Lines' object has no attribute 'xyz'

https://github.com/ezwelty/glimpse/blob/rewrite/glimpse/optimize.py#L1340

I think this should be:

xyz.append(control.xyzs)

Write image to file with EXIF metadata

When a python array is saved as an image, by default it won't contain any of the metadata stored in the original file. One option is to evoke Exiftool, but I'd prefer to avoid dependencies external to python.

Within python, PIL and piexif together support writing JPG EXIF, but I cannot find an equivalent for TIFF, which might be desirable if we want to avoid lossy compression (at the expense of much larger file size).

Speed tests

Source: 3872 x 2592 x 3 JPG: 2.6 MB

0.48 s – Read image (JPG)
0.64 s – Read image (PNG)
0.08 s – Read image (TIF)

0.68 s – Write image (JPG: 2.3 MB, quality=95)
7.93 s – Write image (PNG: 10.3 MB)
0.19 s – Write image (TIF: 30.1 MB)

14.23 s – Project image (without distortion)
43.17 s – Project image (with distortion) - additional time due to undistort of target grid, which needs to be done only once per image sequence

ValueError: zero-size array to reduction operation maximum which has no identity

sensorx,sensory = 35.9,24
cam_coords = [393506.713,6695855.64 ,961.3370] #Easting, Northing, Elevation
cam_viewdir = [72.08866164934302, -24.53,0.35382] # An intial guess
cliff_camera = glimpse.Camera(xyz=cam_coords,viewdir=cam_viewdir,sensorsz (sensorx,sensory))
cliff_imgpts = np.array([[5193, 549],[3101, 642],[3680, 2456],[6153.0, 2297.0]])
cliff_worldpts = np.array([[408245.86,6695847.03,1560 ],[416067.22,6707259.97,988],[393506.713, 6695855.641, 961.337],[394569.509, 6695550.678, 621.075]])
cliff_points = glimpse.optimize.Points(cliff_camera,cliff_imgpts,cliff_worldpts)

cliff_cam_params = glimpse.optimize.Cameras([cliff_camera],[cliff_points])
cliff_cam_params = cliff_cam_params.fit()

Traceback:

---

ValueError Traceback (most recent call last)
in
1 cliff_cam_params = glimpse.optimize.Cameras([cliff_camera],[cliff_points])
----> 2 cliff_cam_params = cliff_cam_params.fit()

~/anaconda3/lib/python3.7/site-packages/glimpse/optimize.py in fit(self, index, cam_params, group_params, full, method, nan_policy, reduce_fcn, **kwargs)
1383 self.update_params()
1384 result = lmfit.minimize(params=self.params, fcn=self.residuals, kws=dict(index=index), iter_cb=callback,
-> 1385 method=method, nan_policy=nan_policy, reduce_fcn=reduce_fcn, **kwargs)
1386 sys.stdout.write('\n')
1387 if iterations:

~/anaconda3/lib/python3.7/site-packages/lmfit-0.9.11-py3.7.egg/lmfit/minimizer.py in minimize(fcn, params, method, args, kws, scale_covar, iter_cb, reduce_fcn, **fit_kws)
2137 iter_cb=iter_cb, scale_covar=scale_covar,
2138 reduce_fcn=reduce_fcn, **fit_kws)
-> 2139 return fitter.minimize(method=method)

~/anaconda3/lib/python3.7/site-packages/lmfit-0.9.11-py3.7.egg/lmfit/minimizer.py in minimize(self, method, params, **kws)
1809 val.lower().startswith(user_method)):
1810 kwargs['method'] = val
-> 1811 return function(**kwargs)
1812
1813

~/anaconda3/lib/python3.7/site-packages/lmfit-0.9.11-py3.7.egg/lmfit/minimizer.py in least_squares(self, params, **kws)
1280 bounds=(lower_bounds, upper_bounds),
1281 kwargs=dict(apply_bounds_transformation=False),
-> 1282 **kws)
1283 except AbortFitException:
1284 pass

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_lsq/least_squares.py in least_squares(fun, x0, jac, bounds, method, ftol, xtol, gtol, x_scale, loss, f_scale, diff_step, tr_solver, tr_options, jac_sparsity, max_nfev, verbose, args, kwargs)
869 return J
870
--> 871 J0 = jac_wrapped(x0, f0)
872
873 if J0 is not None:

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_lsq/least_squares.py in jac_wrapped(x, f)
863 J = approx_derivative(fun, x, rel_step=diff_step, method=jac,
864 f0=f, bounds=bounds, args=args,
--> 865 kwargs=kwargs, sparsity=jac_sparsity)
866 if J.ndim != 2: # J is guaranteed not sparse.
867 J = np.atleast_2d(J)

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_numdiff.py in approx_derivative(fun, x0, method, rel_step, f0, bounds, sparsity, as_linear_operator, args, kwargs)
398 return _sparse_difference(fun_wrapped, x0, f0, h,
399 use_one_sided, structure,
--> 400 groups, method)
401
402

~/anaconda3/lib/python3.7/site-packages/scipy/optimize/_numdiff.py in _sparse_difference(fun, x0, f0, h, use_one_sided, structure, groups, method)
490 fractions = []
491
--> 492 n_groups = np.max(groups) + 1
493 for group in range(n_groups):
494 # Perturb variables which are in the same group simultaneously.

~/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py in amax(a, axis, out, keepdims, initial)
2503 """
2504 return _wrapreduction(a, np.maximum, 'max', axis, None, out, keepdims=keepdims,
-> 2505 initial=initial)
2506
2507

~/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py in _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs)
84 return reduction(axis=axis, out=out, **passkwargs)
85
---> 86 return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
87
88

ValueError: zero-size array to reduction operation maximum which has no identity