Incompatibility with Openpose Pose Recognition

[space-nuko]

Incompatibility with Openpose Pose Recognition

[coreyryanhanson]

@mancat49 I suspect there won't be a way to do it from images. Being able to reverse the rasterization to the pose coordinates probably would be too big of an operation for a lightweight node like this one.

On the other hand, I could imagine this plugin getting that information directly from ComfyUI. comfyui_controlnet_aux (The currently best custom node for using the ControlNet preprocessors) posted a few ways of extracting the json data from a recent pose extraction.

The first way, I'd imagine would end up in this plugin itself at some point and be a huge improvement. If I was familiar enough with NodeJS, I'd make a pull request myself.

I did test a quick and dirty workaround based on the For API users: Python code.

To use it, you'd want to save this code in a .py file and run it with python after running the DWPreprocessor (an improved OpenPose extractor) node and filling in the value for the corresponding prompt_id inside the single quotes.

import json, urllib.request

server_address = "127.0.0.1:8188"
prompt_id = '' #Too lazy to POST /queue
output_file = "output/openpose.json"

def get_history(prompt_id):
    with urllib.request.urlopen("http://{}/history/{}".format(server_address, prompt_id)) as response:
        return json.loads(response.read())

history = get_history(prompt_id)[prompt_id]
for o in history['outputs']:
    for node_id in history['outputs']:
        node_output = history['outputs'][node_id]
        if 'openpose_json' in node_output:
            print(json.loads(node_output['openpose_json'][0]))
            with open(output_file, "w") as f:
                pose_info = json.loads(node_output['openpose_json'][0])
                if not pose_info.get("width"):
                    pose_info["width"] = pose_info.get("canvas_width")
                if not pose_info.get("height"):
                    pose_info["height"] = pose_info.get("canvas_height")
                if not pose_info.get("keypoints"):
                    people = pose_info.get("people")
                    if not people or len(people) == 0:
                        raise RunTimeError("No people detected in openpose data")
                    keypoints = people[0].get("pose_keypoints_2d")
                    pose_info["keypoints"] = [[[keypoints[i], keypoints[i+1]] for i in range(0, len(keypoints), 3)]]
                json.dump(pose_info, f)
            print("Saved to", output_file)

With that, you'd at least be able to use the extracted poses as starting points. It would be missing a few extra details like finger positions. Those aren't supported in this custom node. But it would update every part of the pose that is currently supported in this repo.