xmba15/coarse_to_fine_image_registration

Coarse-to-Fine High Resolution Image Alignment/Registration

📝 image_registration

Toy application to register high resolution images

---

🎉 TODO

---

• Coarse image alignment by SuperGlue
• Fine image alignment by Thin Plate Spline Transformation

🎛 Dependencies

---

• pytorch c++ API. Follow the installation instruction in HERE

  • If libtorch is installed into /opt/libtorch, we need to add /opt/libtorch/lib into paths where system searches for shared libraries:

  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/libtorch/lib

• torch_cpp:

    git clone https://github.com/xmba15/torch_cpp.git
    cd torch_cpp
    make default && sudo make install

• other depedencies:

sudo apt-get install -y --no-install-recommends \
    libopencv-dev

🔨 How to Build

---

# build library
make default -j`nproc`

# build examples
make apps -j`nproc`

🏃 How to Run (after building examples)

---

• Download model weights

mkdir -p .tmp
wget -P .tmp https://github.com/xmba15/torch_cpp/releases/download/0.0.1/superpoint_model.pt
wget -P .tmp https://github.com/xmba15/torch_cpp/releases/download/0.0.1/superglue_model.pt

• Application for test data
  • Multispectral Data from MicaSense: MSI(5 bands: Blue, Green, Red, Red-edge, NIR) high resolution images from MicaSense Sensors. The following app aligns all the 5 bands together.

./build/examples/super_glue_matcher_micasense_app .tmp/superpoint_model.pt .tmp/superglue_model.pt

• Prokudin-Gorskii Collection: Black and White images from Prokudin-Gorskii collection, taken by Miethe-Bermpohl camera. One work will consist of three images (blue, green, red) over a span of 2-6 seconds. One sample image can be obtained from the following link:

wget https://tile.loc.gov/storage-services/master/pnp/prok/00500/00564a.tif

./build/examples/super_glue_matcher_app .tmp/superpoint_model.pt .tmp/superglue_model.pt

💎 References

---

• Adaptive Registration of Very Large Images

• Micasense Sensor Structure

• Approximation Methods for Thin Plate Spline Mappings and Principal Warps