agarsev/quevedo

Tool for managing datasets of images with compositional semantics, part of VisSE project.

Hi!

My name is Antonio Fernando García Sevilla (a.k.a. Antonio F. G. Sevilla, AFGS, agarsev...), and I’m a computational linguist and AI engineer. I earned my PhD at Universidad Complutense de Madrid with the great people of NIL and GRIFFOS, working on Sign Language processing. I've also been at University of Malta, Charles University in Prague, and Universidad Autónoma de Madrid.

If you want to see my CV, click here! You can also check out my website or ORCiD profile.

Recent projects:

• VISSE "Visualizing SignWriting": tools and research for improving the usability of SignWriting (a sign language writing system).
• Signario LSE A parametric dictionary ("Signary") of Spanish Sign Langugae (LSE).

Some other links:

• 🔑 RSA Public Key
• 📚 Scientific publications
• 💿 Apps

quevedo's Issues

Cancelling navigation breaks it

Where: web-app annotation editor

Steps to reproduce:

• Make some changes
• Try to navigate away with arrow buttons
• Click cancel on browser prompt
• Navigation is now broken (doesn't work, doesn't throw an error)

Modifying a transcription meaning doesn't update it in the listing

Steps to reproduce:

1. Go to webapp and edit a transcription
2. Change the meaning and save
3. Go to listing

The heading for the translation should be the new meaning.

Allow train/test split to take sets into account

Add a command line option to the train/test split command that restricts
operation based on subdirectory. For example, if we want to train only with
subdir 1, and test on subdir 2, we could do:

$ quevedo split --dirs=subdir1 100

$ quevedo split --dirs=subdir2 0

Order automatically generated symbols by position

In the web interface, when detecting symbols automatically with the trained neural net, order them according to position (eg left-to-right and top-to-bottom) so that they are easier to quickly understand by users. (via JM)

Tagger undo/redo

Add undo/redo functionality in the web interface for the tagger (annotation
editor), and then auto-save instead of requiring clicking the button.

Maybe can be done with a reducer state hook or a store + action.

Highlight symbol being edited

When editing the tags for a symbol, visually highlight the corresponding bounding box (via JM)

Upload transcriptions in the webapp

Allow uploading transcriptions directly in the web interface. This makes it
easier for informants to add transcriptions.

List only trained experiments in the web app

In the web app annotation editor, only list experiments that have been trained
as available for auto detection of symbols.

Experiment class should probably have a method is_trained, code is in info
command.

Directory structure for real transcriptions

Allow organizing real transcriptions in directories. This can help organize data
sources, and directories can be marked train/test/no use so all transcriptions
within are so used.

Internationalization

Support different languages for the web interface. Start with English and
Spanish, more can be accepted as external contributions.

Add bash completion

Add bash completion to the CLI as per https://click.palletsprojects.com/en/7.x/bashcomplete/

Improve experiment support

Improve UX of experiments.

• Move experiment configuration into info.yaml instead of its own file.
• Create run config and command that runs all steps for an experiment.
• Save/improve reports of completed experiments so can be run headless.

Improve portability of datasets

Right now, datasets are not directly portable because some paths are hardcoded
into configuration, and some things are not deterministic.

• Sort the obj.names file so it is deterministic.
• Add a prepare command that must be run after moving the dataset, that
  regenerates paths. There is some overlap with pre_train that must be
  resolved/rewritten.

Add preprocessing

Add a configurable preprocessing module that applies common image enhancement to data, for example increasing contrast, reducing noise, whatever.

This should be enabled/disabled in config. Maybe there can be a flag in add_images that decides whether to process added images. This preprocessing should be used when predicting new instances (also when testing, but images there have already been added and therefore processed).