Steps for creating a custom dataset

[deepjavalibrary]

Steps for creating a custom dataset

[thhart]

Hi contributors thanks for the suggestions added here. However I have some criticism and I have to say the documentation is very incomplete. Maybe it is worth to put some more effort in this whole topic.

1. SageMaker is a very high level labeling tool which does not answer how to integrate a dataset into DJL.
2. PikachuDetection is a good basic example which unfortunately only contains 1 label, further it is not clear how the Pikachu is woven into a record dataset. From the index structure I can see it is a relative bounding box structure. But still it is unclear how multiple labels can be converted to a NDArray record structure.
3. Finally your link to the own dataset creation only addresses a different AI problem and not image labeling for object detection not to speak of image segmentation.

[frankfliu]

@ffazil
You can try AWS SageMaker ground truth: https://aws.amazon.com/sagemaker/groundtruth/

[lanking520]

Hi @ffazil, in order to train a basic classification or object detection model, you need to label each image.

About custom dataset, you can try some tools to generate labels for you. As an example:
SageMaker Ground Truth or Mechanical turk.

[zachgk]

@ffazil Also, you are not required to follow the same formats as PikachuDetection dataset. You can just create your own dataset class extending RandomAccessDataset and then format your data in whatever way works best for you.

[stu1130]

@ffazil Also, you are not required to follow the same formats as PikachuDetection dataset. You can just create your own dataset class extending RandomAccessDataset and then format your data in whatever way works best for you.

You can follow the instruction here if you want to extend RandomAccessDataset dataset.
https://github.com/awslabs/djl/blob/master/docs/development/how_to_use_dataset.md#how-to-create-your-own-dataset

[ffazil]

Thank you all.