justingosses/geovec-playground

playground for exploring geoVec pre-trained glove model of geoscience embeddings

geoVec-playground

playground for exploring geoVec pre-trained glove model of geoscience embeddings

Pay attention to .gitignore file as otherwise your git can be clogged with files that are too big!

Use this notebook, not the other one: Exploration of GeoVec Word Embeddings & Parsed English Only Version.ipynb

Theoretically should front-end only with identical notebook as run from local machine but there's some bug I haven't figured out yet!!!! and the size of the embedding file requires git large file service, which has now stopped working on any repositories for me even though I have data left before the cap, fun.

For now the quickest way to play with the geovec embedding in the projector is to use this github pages page (clone of the google embedding projector page) and load the vecs.tsv file and metadata1.tsv file into it from this repository. https://justingosses.github.io/embedding-projector-standalone/

For now, second quickest way to see the embedding is to clone this repo and run from source folder python3 -m http.server http://0.0.0.0:8000/embedding-projector-standalone/embedding-projector-standalone-master/

Alternatively, you can load the notebook and run through it, specifically this one: https://github.com/JustinGOSSES/geoVec-playground/blob/master/Exploration%20of%20GeoVec%20Word%20Embeddings%20%26%20Parsed%20English%20Only%20Version.ipynb

This repo is just messing around with this original word embedded work by these authors:

paper: https://soil.copernicus.org/articles/5/177/2019/soil-5-177-2019.html article of interest: https://towardsdatascience.com/deep-learning-and-soil-science-part-1-8c0669b18097 researchgate: https://www.researchgate.net/publication/341342446_3D_lithological_mapping_of_borehole_descriptions_using_word_embeddings phd thesis: https://ses.library.usyd.edu.au/bitstream/handle/2123/22081/Padarian_J_thesis.pdf?sequence=1&isAllowed=y

Data file link: https://osf.io/4uyeq/wiki/home/

See gloVe-test example for the basic idea.

CITATION¶ @misc{padarian2019geovec, title={GeoVec}, url={https://osf.io/4uyeq}, DOI={10.17605/OSF.IO/4UYEQ}, publisher={OSF}, author={Padarian, José and Fuentes, Ignacio}, year={2019} }z

Article @misc{padarian2019word, title={Word embeddings for application in geosciences: development, evaluation and examples of soil-related concepts}, url={https://doi.org/10.5194/soil-2018-44} DOI={10.5194/soil-2018-44}, publisher={Copernicus GmbH}, author={Padarian, José and Fuentes, Ignacio}, year={2019}, journal={SOIL Discuss} }

Data file of the pretrained emb

Windows 10 Notes for anyone doing further investigation: some sage advice for glove installation pip install glove==1.0.0 (after trying builds and installation in visual studio developer prompts etc etc)

geovec-playground's Issues

No log directory?

writeEmbeddingWithWordsAndVectorsToFilesForEmbeddingProjector(new_vector_embedding_english_only2)

---

FileNotFoundError Traceback (most recent call last)
in
----> 1 writeEmbeddingWithWordsAndVectorsToFilesForEmbeddingProjector(new_vector_embedding_english_only2)

in writeEmbeddingWithWordsAndVectorsToFilesForEmbeddingProjector(embedding)
4 for key in keys_in_array:
5 vectors_only.append(embedding[key])
----> 6 makeVectorTSV(vectors_only)
7 makeMetaTSV(keys_in_array)
8 print("done")

in makeVectorTSV(embeddingVectorsOnly)
1 def makeVectorTSV(embeddingVectorsOnly):
----> 2 out_v = io.open('log/vecs.tsv', 'w', encoding='utf-8')
3 for n,weights in enumerate(embeddingVectorsOnly):
4 vec = embeddingVectorsOnly[n]
5 out_v.write('\t'.join([str(x) for x in vec]) + "\n")

FileNotFoundError: [Errno 2] No such file or directory: 'log/vecs.tsv'