mapillary/mapillary_sls

Mapillary Street-level Sequences Dataset

Mapillary-plugin for JOSM

A plugin for showing Mapillary images inside the OpenStreetMap-Editor JOSM.

You can download images from mapillary.com.

For instructions on setting up your local copy of this repository, see INSTALL.md.

Contributing

Our contribution guidelines can be found in CONTRIBUTING.md.

• The source code locations
  • GitHub
  • JOSM GitLab Group
• Issues are managed in JOSM Trac
  • Report a New Ticket
• Translations are done on JOSM's Launchpad and were previously done on transifex.

• Maintainers:
  • Taylor Smock [email protected] (active, @smocktaylor)
  • nokutu (inactive)
  • floscher (inactive)
• License: GPL v2 or later
• Feel free to contact me for any bug or suggestion.

mapillary_sls's Issues

Reproducing the results

Hello, are you planning to release the entire code used to train the networks on MSLS? Specifically, I'm referring to Tab.3 in the main paper and Tab.1 in the supplementary (which uses the "Positive Mining Strategy")

Challenge: The maximum number of submissions this day have been reached.

Hiya,
Unfortunately I receive the error "The maximum number of submissions this day have been reached." although I have not made a submission today yet. My last (and only) submission was at 08/18/2020 20:31:21; now the server time is 08/19/2020, 9:43:00, which is both a new day and more than 24 hours since the last submission.

It would be great if you could look into this.

Many thanks, Tobias

[Challenge] Ignoring predictions although they are in test cities + panorama images

Hiya,
Two more questions re challenge evaluation.

1. We receive the following message for ~60 query images (full list below): "Ignoring predictions for B55p4NnNAh1-Y8sOBha3NA at line 42. It is not in the selected cities". I checked and "B55p4NnNAh1-Y8sOBha3NA" is contained in the athens query images. Is the test set on the server diverging from the test set given out?
2. We also get "Some of your predictions are not in the database for the selected task [...]. This is probably because they are panorama images. They will be ignored in evaluation" -> Indeed I checked and the images are panoramas. Are the queries associated to these predictions completely removed from the evaluation process or what does "ignore" mean?

Ignoring predictions for B55p4NnNAh1-Y8sOBha3NA at line 42. It is not in the selected cities
Ignoring predictions for Qj0DPkWJUZV8tfPjKDAIhQ at line 183. It is not in the selected cities
Ignoring predictions for P-cjCsztzXwCRb2VwYONXQ at line 198. It is not in the selected cities
Ignoring predictions for 46Y2uYU_qHCbWRxjAbuZVw at line 394. It is not in the selected cities
Ignoring predictions for AC00dsAUt3p4-PlppbcCNA at line 558. It is not in the selected cities
Ignoring predictions for nLeBb_hgrQLilzWX_hn1Jw at line 678. It is not in the selected cities
Ignoring predictions for N6crkK6U_vTEMP9O2ZY9Qg at line 679. It is not in the selected cities
Ignoring predictions for 4km_uFuxPA-eIPTUSqrZIQ at line 685. It is not in the selected cities
Ignoring predictions for 74NTOmTmmnfkRJ2qlcLPCg at line 687. It is not in the selected cities
Ignoring predictions for QCLiybp5TS7HneAJfGMycg at line 689. It is not in the selected cities
Ignoring predictions for xqoli-E3rRgo-Hb81l7rT2 at line 690. It is not in the selected cities
Ignoring predictions for A1nsEM4NVTDSfAranGb3e_ at line 693. It is not in the selected cities
Ignoring predictions for YTkY36z6aysIGjf-XXw3QL at line 696. It is not in the selected cities
Ignoring predictions for VRTBJkkyq8LZw30lgJOYLQ at line 697. It is not in the selected cities
Ignoring predictions for aw9nP0itM_5E3e18z4l1Lw at line 698. It is not in the selected cities
Ignoring predictions for 5JzPKfo7VGQJgiAo47jM3g at line 699. It is not in the selected cities
Ignoring predictions for 9K4o7qUD6liUTi5tq9JCIw at line 700. It is not in the selected cities
Ignoring predictions for CQIf3WY6eFKA7nwR2xEL9Q at line 701. It is not in the selected cities
Ignoring predictions for RJ94yTIEtpOq5S3oMqYlUQ at line 703. It is not in the selected cities
Ignoring predictions for 3muZuCNoYLPq85AsK26Big at line 704. It is not in the selected cities
Ignoring predictions for Ml8AB7ONMrXyAf9OLBZJ-Q at line 706. It is not in the selected cities
Ignoring predictions for Hm9ah7EXyLYQwYqe_-SDmw at line 707. It is not in the selected cities
Ignoring predictions for 04jec_ilMrb8QfDp80G4dw at line 710. It is not in the selected cities
Ignoring predictions for Hv4AyQZ5G2FsyNyDaUhznQ at line 711. It is not in the selected cities
Ignoring predictions for OXdJ03v_cpf_sOvxpr99Bw at line 714. It is not in the selected cities
Ignoring predictions for d64mHbDDlhx4U3xxV3Nthg at line 715. It is not in the selected cities
Ignoring predictions for 0nBECBLVAgxy9rosfZP-Dg at line 717. It is not in the selected cities
Ignoring predictions for 9DTBAM0SeMxAMOn5433b2Q at line 718. It is not in the selected cities
Ignoring predictions for N2pAaPu47ryJWflWjhCpyQ at line 719. It is not in the selected cities
Ignoring predictions for u4pgo8LWD5nbLSO71YgrWA at line 720. It is not in the selected cities
Ignoring predictions for UCbwF47OVq7yMuzj2U0vgw at line 722. It is not in the selected cities
Ignoring predictions for pPQUNtR3yNp2ssnMdkqiIw at line 725. It is not in the selected cities
Ignoring predictions for liqAuW1u_vNa1ue9Td2Ltw at line 726. It is not in the selected cities
Ignoring predictions for as5UiJTaMGybf1PW7GS66g at line 727. It is not in the selected cities
Ignoring predictions for JOqViqeZ_Jsms84Ov5p40A at line 728. It is not in the selected cities
Ignoring predictions for fFYh5sPLc0cTBKubjL5Nnw at line 730. It is not in the selected cities
Ignoring predictions for Jdo5SyHJkeKfIvKRrL3u7g at line 732. It is not in the selected cities
Ignoring predictions for t8ajBUgabicusgtfZATv7Q at line 733. It is not in the selected cities
Ignoring predictions for 2IZYu_QoPDlHMM1NeKROuA at line 734. It is not in the selected cities
Ignoring predictions for 7UncugdfpZcmE0I5kaAnrQ at line 735. It is not in the selected cities
Ignoring predictions for wTJeqUgQNu3ncKFcQYoSOQ at line 736. It is not in the selected cities
Ignoring predictions for PNIu0dsY5GQLTJt2UdecTg at line 737. It is not in the selected cities
Ignoring predictions for ATu5gw5brpT0WuGPndRIQQ at line 738. It is not in the selected cities
Ignoring predictions for P9_JBSwBkYpJphg67IviPw at line 740. It is not in the selected cities
Ignoring predictions for KXxXtr0OiUcC8lxrztxQXA at line 741. It is not in the selected cities
Ignoring predictions for TRrVZQ-gI5SpuVngPWqWoA at line 747. It is not in the selected cities
Ignoring predictions for IrVCAePVHes4VoFteA5Epg at line 748. It is not in the selected cities
Ignoring predictions for CuS2xp-s4WNh9gwfPPP59w at line 749. It is not in the selected cities
Ignoring predictions for Y8TcQHaVeIQtA5bhoaAAwA at line 751. It is not in the selected cities
Ignoring predictions for dKmqPdm_908QQdVkTJ0yFQ at line 755. It is not in the selected cities
Ignoring predictions for NCS7bxeaz99oBV_RaMqKOQ at line 758. It is not in the selected cities
Ignoring predictions for HiATUbk6BWLVnHuYN2gBtQ at line 765. It is not in the selected cities
Ignoring predictions for y5XDC6bVZX0VWUEinJVEfM at line 767. It is not in the selected cities
Ignoring predictions for 0g_q2e68f_0RtBnb2EWIAJ at line 768. It is not in the selected cities
Ignoring predictions for 566X0ZIVNfbechJqbQ-4Ag at line 770. It is not in the selected cities
Ignoring predictions for uFVUTv-mgQdI-AaHx_pINA at line 773. It is not in the selected cities
Ignoring predictions for rhB2az2WAH6TsZi2ykZV88 at line 774. It is not in the selected cities
Ignoring predictions for oyl8jyAL10V91bGpFwIjQw at line 777. It is not in the selected cities
Ignoring predictions for 5OfWYXU5PyeEd4CNhvbTDg at line 778. It is not in the selected cities
Ignoring predictions for Dk0klQdCgLmGj3aN6_YE0P at line 780. It is not in the selected cities
Ignoring predictions for 0Wo8KBGc9Gn-rCmb2I2SYO at line 783. It is not in the selected cities
Ignoring predictions for kus39RdNaE_b00jhSuHedQ at line 784. It is not in the selected cities
Ignoring predictions for qX6ZYHxaAYSgV7tZst-lKQ at line 785. It is not in the selected cities
Ignoring predictions for l2V0e3Jg1sJOF8eUjjCgfw at line 786. It is not in the selected cities
Ignoring predictions for DY-Nsorqppy3EgLs2dWSRw at line 788. It is not in the selected cities
Ignoring predictions for oksKmhQaall1gj7exJNm_c at line 789. It is not in the selected cities
Ignoring predictions for InfVOLHXeJsr1LNK4TJuLw at line 1163. It is not in the selected cities

how you generate the image key?

I want to build a dataset with your dataset structure, what should I do to organize the image data? Thanks.

About validation code

I run evaluate.py to load MSLS val (sf, cph) for evaluate predictions. The number of database images loaded are 18871. But only 747 query images are loaded.
This is obviously lower than the number of actual images of queries in val set. I wonder if this is correct.

Challenge: BadZipfile

Hiya, I get the following error when trying to submit to the challenge:

Traceback (most recent call last):
  File "/worker/worker.py", line 313, in run
    bundles = get_bundle(root_dir, 'run', bundle_url)
  File "/worker/worker.py", line 173, in get_bundle
    metadata[k] = get_bundle(bundle_path, k, v)
  File "/worker/worker.py", line 173, in get_bundle
    metadata[k] = get_bundle(bundle_path, k, v)
  File "/worker/worker.py", line 132, in get_bundle
    with ZipFile(bundle_file.file, 'r') as z:
  File "/usr/local/lib/python2.7/zipfile.py", line 770, in __init__
    self._RealGetContents()
  File "/usr/local/lib/python2.7/zipfile.py", line 811, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
BadZipfile: File is not a zip file

The zip file contains only the predictions.csv file. I can open the zipfile alright in a Python 2 environment on my local machine. I tried several ways to compress the zip file, too. I attach one of the zip files that didn't work.

It would be great if you could let me know how to get around this issue.

Best, Tobias
predictions.zip

The meaning of 'assert subtask in ('all', 's2w', 'w2s', 'o2n', 'n2o', 'd2n', 'n2d')' in msls.py

Dear authors,
thank you very much for you extraordinary contributions. Could you please tell me the meaning of the code marked with red box in the picture below? (msls.py)
To be exact, what do 'all', 's2w', 'w2s', 'o2n', 'n2o', 'd2n' and 'n2d' stand for respectively?
Thank you!

Sorry, failed to upload file.

hi, I want to upload my zip to CodaLab,but I got this result "Sorry, failed to upload file."

can you tell me how to fix it?
thank you very much

Dataset download failed

Hello, I created and logged in to my account. I'm having trouble downloading: Error submitting request, please try again
Sincerely hope to get your help.

No csv files found in data downloaded

Hi，I have download all the zip files from https://www.mapillary.com/dataset/places.
After unziped, I found no csv files (there expect to be seq_info.csv, subtask_index.csv etc.)

Please check if the zip files are good to use.

datasets download

Hello~ When I try to download the mapillary dataset, it always informs “Error submitting request, please try again”. Maybe the requested full name I input is wrong? Or anything else?

Dataset Download

Thank you for this nice work.
I am trying to download the dataset following the standard instruction. All the links are working well except the second zip file: "msls_images_vol_2.zip ". Could you please check and verify this link?

Can you introduce the structure of the downloaded data?

From this URL【https://www.mapillary.com/dataset/places】 I can download the following data:

msls_checksums.md5
msls_images_vol_1.zip
msls_images_vol_2.zip
msls_images_vol_3.zip
msls_images_vol_4.zip
msls_images_vol_5.zip
msls_images_vol_6.zip
msls_metadata.zip
msls_patch_v1.1.zip

But I don't know which pictures are included in these data, can you tell me?

How do we calculate performances on the test set ?

Sorry to ask this, but is there any way we can calculate performances on the test set ? I don't see any "postprocessed.csv" in the test directories.
Is there a website where I can submit the results ? There arepeople talking about a challenge in the other discussions.

Thanks

Sample dataset link doesn't work

Hello,
I tested your demo and noticed that your link to the sample dataset does not work (https://static.mapillary.com/MSLS_samples.zip).

Information about framerate

Hi, thanks for the contribution!

Is it possible to calculate the time difference between two frames?
The raw.csv file contains a column captured_at, but it is only a date, not capture time with millisecond accuracy.
Other than that, I could not find any information about framerate / fps / time difference in your repository, in the sample dataset or on the mapillary sls site.

Thanks, Fabian

fail downloading msls_images_vol_[1|2|3|4|5].zip

Hi, I am trying to download the MSLS dataset from https://www.mapillary.com/dataset/places, and I successfully downloaded

• msls_checksums.md5
• msls_images_vol_6.zip
• msls_metadata.zip
• msls_patch_v1.1.zip

while I failed downloading

• msls_images_vol_[1|2|3|4|5].zip

for example, specifically, after clicking msls_images_vol_1.zip, the broswer would bring me to a new page, then just stucked there:

(I am using Firefox with incognito tabs)

Trained weights available?

Hi! First of all, amazing work, congrats! I would like to know if your already trained models from the paper "Mapillary Street-Level Sequences: A Dataset for Lifelong Place Recognition" are available for the public. I would like to initialize my own model with yours without having to download and train in the whole dataset.

Thanks in advance!

can't download the Mapillary dataset

I have tried many times and tried different VPN, but I can't open the url https://www.mapillary.com/dataset/places.
Where can I download the Mapillary Street-level Sequences dataset?
Can u help me? thanks a lot!

msls_images_vol_1.zip download fail

All files can be downloaded normally, but only the "msls_images_vol_1.zip" file failed to download, and the web page reported an error 400. I saw this problem in other documents before. Does this one need to wait for a period of time?

can't download MSLS_samples.zip

the link does not work, can you please fix it?

amsterdam/database/images

When I finished downloading all the files, I found that amsterdam/database/images is missing. is this file not there itself? If it exists, please let me know in which zip file.
I look forward to receiving your reply!

Can not find the images of phoenix's database

Hi,
I can not find the images of the phoenix's database after I downloaded the MSLS dataset and unzipped it

Downloading the dataset from CLI

Hey,
Is there any easy way to download the dataset via cli in a remote server?
Thanks.

About model evaluation code

When evaluating the model with validation dataset and test dataset, why is Recall@N calculated excluding pano data among the query images?

Images number of MSLS val

Hello,thanks for your codes.
I run evaluate.py to load MSLS val (sf, cph) for evaluate predictions. The number of database images loaded are 18871. But only 747 query images are loaded. This is obviously lower than the number of actual images of queries in val set. I wonder if this is correct.

Data missing

Hi, I found that there are some images that are empty, such as: "_Bkgic7JtCaaMGrgw6kukQ.jpg" at Melbourne's database dir, "yOCOTojshHsfJd0g5B7jPP.jpg" at Melbourne's query dir, and "Ek08yvVCR1tXrD4ZSx59CA.jpg" at Berlin's query dir.
Could you please reload these images? Thanks very much.