Comments (6)
Do you have documentation as to which photos from each synset appeared in your training and test sets?
from deepdetect.
@kmatzen I don't have documentation about that.
But it should be pretty easy to get the list of synsets from the model class list. I might be able to provide a list of the exact synsets in most cases if this would prove something important for some of the model users.
It's a bit more tricky for the files I believe. The list of files is likely to be stored in the training db, and from there, there may be a way to get back to the photo without providing the file itself.
It may not be of great direct help at this stage, but I've fixed a script to grab what publicly online from the full Imagenet (~85%): https://github.com/beniz/imagenet_downloader
I've actually trained from that, which may make the matching to the full Imagenet tarball more tricky even. FTR, I've had access to the full Imagenet dump sometimes in the past but they have recently resetted the credentials as it seems.
from deepdetect.
In the information about the website, you've listed that you split the dataset into training and testing.
I understand you cannot share the images due to licensing. But presuming that we're looking at the same imagenet dataset, how did you decide the split? If for example, you simply chose the first 4500 to be training, and a remaining 500 to be testing, then we could do the same.
Otherwise we run the risk of using the training data in our tests, which would obviously be problematic. Is there some log somewhere of what files you used?
from deepdetect.
@roblkw226 good point, unfortunately the splits are random from a shuffled dataset (and no seed).
I guess you would like to get an appreciation of the accuracy of one or more models, is that correct ? We could think of a few ways to help you do that.
from deepdetect.
Yes, that's right. We'd like to understand the accuracy, before and after some fine-tuning adjustments we have in mind. We'd appreciate any help in that area.
from deepdetect.
OK, there are several ways to get or approximate the accuracy:
- Finetune the model of interest on your own dataset. Finetuning is the primary reason behind sharing the training net, the solver file and the
model.json
file that contains the training calls. Having a training set of your own will allow you to control the value of any adjustment you decide to take. - Another method I've used in the past when the test set is not available: call on Google image for the categories of interest and build your test set from there. Intersection with Imagenet and any of the training images is unlikely (sometimes it may be safer to not use the exact full length Imagenet-like class names though). Typically such a tool can easily be built based on https://github.com/asciimoo/searx
- We could run a little experiment in which I would share the testing set for one of the models and see how useful it can be in practice. If your interest mostly lies in the datasets and less in the pre-trained models and network description files, I guess we could again think of some ways around that.
If you need more lively interaction, ping me on gitter, https://gitter.im/beniz/deepdetect
from deepdetect.
Related Issues (20)
- Inconsistent predictons using refinedet model HOT 12
- Memory leak on constant /predict requests HOT 8
- Refinedet Tensorrt prediction fails HOT 7
- Memory leak on compressed predict requests with oatpp HOT 7
- Different prediction with tensorrt on refinedet model for the version v0.18.0 HOT 3
- getting error while training, .solverstate HOT 23
- Chain predictions swapped between images HOT 2
- Simsearch query segfault when using IVF indexes, but not default/flat index HOT 6
- On object detect training call, missing either test or train list causes a segfault
- dd_client not find in this path anyone help HOT 2
- How do I do a face recognition using this? HOT 2
- DeepDetect full rewrite in Pure Java
- 'OCR' object has no attribute 'histogram_equalization' HOT 13
- "best: -1" in predict behaves differently in torch models HOT 2
- Torch v1.12 requires libcupti* but nvidia/cuda:11.6.0-cudnn8-runtime-ubuntu20.04 doesn't include it
- Race condition / pthread error when predicting
- I have error build xgboost HOT 1
- Using `true` or `false` instead of `1` or `0` for query params for status or labels returns a internal server error HOT 1
- Question about hosting the docker image HOT 4
- Graphics problem with tsne algorithm HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deepdetect.