aws-samples/amazon-sagemaker-ground-truth-task-uis

Example task UIs for Amazon SageMaker Ground Truth

Amazon SageMaker Ground Truth Sample Task UIs

This repository contains sample task UIs for Amazon SageMaker Ground Truth. It contains subdirectories by use case (audio, images, text, video, other).

For additional task UI resources, please refer to the official documentation.

Amazon SageMaker Ground Truth helps you build highly accurate training datasets for machine learning quickly. SageMaker Ground Truth offers easy access to public and private human labelers and provides them with built-in workflows and interfaces for common labeling tasks.

License Summary

This sample code is made available under the MIT-0 license. See the LICENSE file.

amazon-sagemaker-ground-truth-task-uis's Issues

Sample Point Cloud Task UIs?

Looks like point clouds in Sagemaker ground truth are still new but it would be nice to see some sample UIs to better learn how to customize tasks.

One specific thing I'd like to customize is the ability to color the point cloud based on point attributes. For example, if the format is binary/xyzirgb then I'd like to be able to toggle between intensity (i) and color (rgb).

Limitations with <crowd-keypoint> HTML Element in Sagemaker Groundtruth

Issue Description

I am currently working with Sagemaker groundtruth for keypoint annotation, and I have encountered an issue with the HTML element.

When reviewing the documentation at sms-ui-template-crowd-keypoint, it appears that the base HTML element was not designed with the most popular keypoint annotation task (e.g. keypoints-2018) in mind. Instead of defining it as a list of objects with a list of different types of keypoints (e.g. multiple instances of humans with a "left_hand", "right_hand", "left_knee", etc.), it is defined as a simple list of keypoints.

The second major issue I have encountered is that I have been testing the usage of the HTML element, and I have found that it is not possible to move or edit the keypoints if they are preloaded/populated with the initial-value="[]" list.

Solution

Is there any other HTML/JavaScript example/class created to address this issue, such as "CrowdKeypoint extends HTMLElement" that you can provide access?. This would allow to support the most popular keypoint annotation tasks and enable editing/moving of preloaded keypoints.

Thank you for your attention to this matter.

Order of the source objects in manifest file is not the same as seen in the worker portal

Is there a way that the order from the jsonl manifest file is the same way the labeler sees it in the worker portal, it seems that the objects are randomized each time a job is created, so if I specify in the manifest:

{"source-ref": "s3://mybucket/image-1.jpg"}
{"source-ref": "s3://mybucket/image-2.jpg"}
{"source-ref": "s3://mybucket/image-3.jpg"}
...

I expect that the worker to view and label the images in that same order.
How can I achieve this?

Thank you

bug not possible to label text with multiple paragraphs

See issue here https://stackoverflow.com/questions/67701971/how-to-label-a-text-with-multiple-paragraphs-in-aws-ground-truth

images/bounding-box-custom-labels.liquid.html Delete and Backspace keys don't work for input control

https://github.com/aws-samples/amazon-sagemaker-ground-truth-task-uis/blob/master/images/bounding-box-custom-labels.liquid.html

customLabel input does not process Delete and Backpace key events, so it is painful editing text in this control. It looks like <crowd-bounding-box> control captures these events

Google Chrome Version 79.0.3945.130 (Official Build) (64-bit)

A CSS that include a background image causes 403

In my custom bounding box template, I have a CSS class like the below and it fails with 403:

.someclass {
    background: url({{ task.input.taskObject | grant_read_access }}) 0 0;
}

The same image URL when referenced from an image tag, it is loading the image without an issue.
When looking at the generated image URL. The one from the image tag looks like this:

https://s3.us-west-2.amazonaws.com/dsml-us-ml-aws2-historic/hongyan/images/p0000155ejmd_001CF9FE49DF3D51AECFB3F03C71852D.jpg?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKX%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJIMEYCIQDFxOfd2AaaWP5WMogka%2FhRaoVSSDn7irGuE3ZQJaRFXgIhAN3CXs4Ucno48knZsZ3rcq2uho61KbWhjNIgHVpUisd0Kt8CCN7%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQARoMOTk1MzgzOTIzMjM4Igyl37R5C82gpqaQE20qswLYbrW0O6KlzN3pG0x6uNRDefy68wtq30FlCweHh%2BcbKp8WLiyAxRMehBSANSAOzVsJ%2B5zXc3i2UfKy4FmHvh4HdLS6YwjppmqCdrxDzekVrjdRqtfLUH9MhcYYcp7Om5ugwQCFH4iqepFtbF8TEwc2ABwwonlCgZVeIP5gWh%2FpEYwfqIwdBZKjKQMZBbYUX2lwECuZoly7vO%2Fx5iAFolK3zCf%2F1X8eyfzYhInB6tGDNhQ%2F4O0TxPhZdigI4RWPIbbwaGQcfUdD%2F1lYRwJeGId9602s5prmyYiw0euDnVXdzInz4kQZP8JtxJXhv6kKGGBTDtrd%2BpewTgb3yJSbDgtrHLps6DtcrkzLGhvZ8s1JUBOc5M0g4DYC3To9hazU5yN0XUqDSuYKoMu83jYXao5KI0OBMMqKhpAGOo4BSOcgS3UmFnpDvlkleodDS0%2BFyLTVKTFRFV3V0kjaRKtPPXbaYuuUIospIrQO7wUPkqm3qoNIOuOuQIK6Pe7G9ztkuzWO7u0Uo6Ecqmf8aQhmPhkC%2Fx%2BGoLQ04V8ZJXZxkgZawLpyWPzClyCnMspIP5lD5zIF6ARtzCTDMHw%2FAL7KqSWpfYPtSAYGEgoj9Q%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220207T204706Z&X-Amz-SignedHeaders=host&X-Amz-Expires=956&X-Amz-Credential=ASIA6PQMBKITCFKKXANR%2F20220207%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=e6c348623e442032a5d2656d7b22a1bed90b40cc7bfa35b9b1a8615fda1e2c58

The one from the CSS looks like this:

https://s3.us-west-2.amazonaws.com/dsml-us-ml-aws2-historic/hongyan/images/p0000155ejmd_001CF9FE49DF3D51AECFB3F03C71852D.jpg?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEKX%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJIMEYCIQDFxOfd2AaaWP5WMogka%2FhRaoVSSDn7irGuE3ZQJaRFXgIhAN3CXs4Ucno48knZsZ3rcq2uho61KbWhjNIgHVpUisd0Kt8CCN7%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQARoMOTk1MzgzOTIzMjM4Igyl37R5C82gpqaQE20qswLYbrW0O6KlzN3pG0x6uNRDefy68wtq30FlCweHh%2BcbKp8WLiyAxRMehBSANSAOzVsJ%2B5zXc3i2UfKy4FmHvh4HdLS6YwjppmqCdrxDzekVrjdRqtfLUH9MhcYYcp7Om5ugwQCFH4iqepFtbF8TEwc2ABwwonlCgZVeIP5gWh%2FpEYwfqIwdBZKjKQMZBbYUX2lwECuZoly7vO%2Fx5iAFolK3zCf%2F1X8eyfzYhInB6tGDNhQ%2F4O0TxPhZdigI4RWPIbbwaGQcfUdD%2F1lYRwJeGId9602s5prmyYiw0euDnVXdzInz4kQZP8JtxJXhv6kKGGBTDtrd%2BpewTgb3yJSbDgtrHLps6DtcrkzLGhvZ8s1JUBOc5M0g4DYC3To9hazU5yN0XUqDSuYKoMu83jYXao5KI0OBMMqKhpAGOo4BSOcgS3UmFnpDvlkleodDS0%2BFyLTVKTFRFV3V0kjaRKtPPXbaYuuUIospIrQO7wUPkqm3qoNIOuOuQIK6Pe7G9ztkuzWO7u0Uo6Ecqmf8aQhmPhkC%2Fx%2BGoLQ04V8ZJXZxkgZawLpyWPzClyCnMspIP5lD5zIF6ARtzCTDMHw%2FAL7KqSWpfYPtSAYGEgoj9Q%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220207T204706Z&X-Amz-SignedHeaders=host&X-Amz-Expires=956&X-Amz-Credential=ASIA6PQMBKITCFKKXANR%2F20220207%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=e6c348623e442032a5d2656d7b22a1bed90b40cc7bfa35b9b1a8615fda1e2c58

As it is shown that the bad one has the "&amp;" NOT decoded.

adding video to short instructions

Is it possible to add a video for the good/bad example in the short instructions always coming from an s3 location?
If so, how?

Thank you

Add more than 10 labels for image

I see that while creating a labeling there is limit of adding up-to 10 labels. How to add more than 10 labels for image?
Let's say I want my images to have 20 labels for semantics. How do I go ahead? Do I have to create multiple jobs?
First job - first 10 labels
Second job - remaining 10 labels?

Countdown for workers - How many images left?

I created a custom template for image classification. I would like to show to the human workers the number of images (tasks) that need to be labeled within the current labeling job. How can I do that? Is there a liquid component I can use?
Thank you

Validation exception while running the create_labeling_job api?

Hello,

I have been trying to run create_labeling_api for the last two days, but I could not able to do it. I tried with different options, but no success. Can I get any help with this problem?
Here is the code:

## Set up Create Labeling Request

labelAttributeName = job_name + "-ref"

if task_type == "3DPointCloudObjectDetection":
    labelAttributeName = job_name

print(job_name)

ground_truth_request = {
        "InputConfig" : {
          "DataSource": {
            "S3DataSource": {
              "ManifestS3Uri": '{}'.format(manifest_s3_uri_map[task_type]),
            }
          },
        },
        "OutputConfig" : {
          "S3OutputPath": f's3://{BUCKET}/{EXP_NAME}/output/',
        },
        "HumanTaskConfig" : human_task_config,
        "LabelingJobName": job_name,
        "RoleArn": role, 
        "LabelAttributeName": labelAttributeName,
        "LabelCategoryConfigS3Uri": label_category_file_s3_uri_map[task_type]
    }

print(json.dumps(ground_truth_request, indent=4, sort_keys=True))

Ouput of the above code:

Point-Cloud-OD
{
    "HumanTaskConfig": {
        "AnnotationConsolidationConfig": {
            "AnnotationConsolidationLambdaArn": "arn:aws:lambda:eu-central-1:326177286064:function:ACS-3DPointCloudObjectDetection"
        },
        "MaxConcurrentTaskCount": 200,
        "NumberOfHumanWorkersPerDataObject": 1,
        "PreHumanTaskLambdaArn": "arn:aws:lambda:eu-central-1:326177286064:function:PRE-3DPointCloudObjectDetection",
        "TaskAvailabilityLifetimeInSeconds": 18000,
        "TaskDescription": "point-cloud-labelling",
        "TaskKeywords": [
            "lidar",
            "pointcloud"
        ],
        "TaskTimeLimitInSeconds": 360,
        "TaskTitle": "Detection-labelling-for-point-cloud",
        "UiConfig": {
            "HumanTaskUiArn": "arn:aws:sagemaker:eu-central-1:394669845002:human-task-ui/3DPointCloudObjectDetection"
        },
        "WorkteamArn": "arn:aws:sagemaker:eu-central-1:326177286064:workteam/private-crowd/suri"
    },
    "InputConfig": {
        "DataSource": {
            "S3DataSource": {
                "ManifestS3Uri": "s3://sample-point-cloud/artifacts/gt-point-cloud-demos/manifests/SingleFrame-manifest.json"
            }
        }
    },
    "LabelAttributeName": "Point-Cloud-OD",
    "LabelCategoryConfigS3Uri": "s3://sample-point-cloud/artifacts/gt-point-cloud-demos/label-category-config/label-category.json",
    "LabelingJobName": "Point-Cloud-OD",
    "OutputConfig": {
        "S3OutputPath": "s3://sample-point-cloud/validation/output/"
    },
    "RoleArn": "arn:aws:iam::326177286064:role/service-role/AmazonSageMaker-ExecutionRole-20210115T142880"
}

sagemaker_client.create_labeling_job(**ground_truth_request)

Error:
ClientError                               Traceback (most recent call last)
<ipython-input-220-84378bff2ae5> in <module>
----> 1 sagemaker_client.create_labeling_job(**ground_truth_request)
      2 print(f'Labeling Job Name: {job_name}')

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    355                     "%s() only accepts keyword arguments." % py_operation_name)
    356             # The "self" in this scope is referring to the BaseClient.
--> 357             return self._make_api_call(operation_name, kwargs)
    358 
    359         _api_call.__name__ = str(py_operation_name)

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    674             error_code = parsed_response.get("Error", {}).get("Code")
    675             error_class = self.exceptions.from_code(error_code)
--> 676             raise error_class(parsed_response, operation_name)
    677         else:
    678             return parsed_response

ClientError: An error occurred (ValidationException) when calling the CreateLabelingJob operation: HumanTaskUiArn is not supported for the Custom task type. You must provide an S3 URI identifying the location of a worker task template (UiTemplateS3Uri) for this task type. Remove the HumanTaskUiArn from your request, provide an UiTemplateS3Uri, and try your request again.

Thanks in advance

lambda templates to go with liquid templates

Are there pre- and post-processing lambda templates to go with the liquid templates here? In particular I am interested in lambda templates for image-classification-multiselect.liquid.html

bug with default ground truth labeling interface.

Unsure if this is the place to log it - but for me when inserting a link on the Instructions menu, the link when looking at the active labeling interface doesn't do anything when clicking it. The only way to open the link is by right clicking the blue link text, and then selecting open in new tab / window.

The issue seems to affect header 1, header 2, and bold text with links as well.