Tasks

Keypoint Detection

Keypoint detection is the task of identifying meaningful distinctive points or features in an image.

Inputs
Keypoint Detection Model
Output

About Keypoint Detection

Task Variants

Pose Estimation

Pose estimation is the process of determining the position and orientation of an object or a camera in a 3D space. It is a fundamental task in computer vision and is widely used in various applications such as robotics, augmented reality, and 3D reconstruction.

Use Cases for Keypoint Detection

Facial Landmark Estimation

Keypoint detection models can be used to estimate the position of facial landmarks. Facial landmarks are points on the face such as the corners of the mouth, the outer corners of the eyes, and the tip of the nose. These landmarks can be used for a variety of applications, such as facial expression recognition, 3D face reconstruction, and cinematic animation.

Fitness Tracking

Keypoint detection models can be used to track the movement of the human body, e.g. position of the joints in a 3D space. This can be used for a variety of applications, such as fitness tracking, sports analysis or virtual reality applications.

Inference Code

Below you can find an example of how to use a keypoint detection model and how to visualize the results.

from transformers import AutoImageProcessor, SuperPointForKeypointDetection
import torch
import matplotlib.pyplot as plt
from PIL import Image
import requests

url_image = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url_image_1, stream=True).raw)

# initialize the model and processor
processor = AutoImageProcessor.from_pretrained("magic-leap-community/superpoint")
model = SuperPointForKeypointDetection.from_pretrained("magic-leap-community/superpoint")

# infer
inputs = processor(image, return_tensors="pt").to(model.device, model.dtype)
outputs = model(**inputs)

# postprocess
image_sizes = [(image.size[1], image.size[0])]
outputs = processor.post_process_keypoint_detection(model_outputs, image_sizes)
keypoints = outputs[0]["keypoints"].detach().numpy()
scores = outputs[0]["scores"].detach().numpy()
image_width, image_height = image.size

# plot
plt.axis('off')
plt.imshow(image)
plt.scatter(
    keypoints[:, 0],
    keypoints[:, 1],
    s=scores * 100,
    c='cyan',
    alpha=0.4
)
plt.show()

Compatible libraries

Keypoint Detection demo

No example widget is defined for this task.

Note Contribute by proposing a widget for this task !

Models for Keypoint Detection
Browse Models (26)
Datasets for Keypoint Detection
Browse Datasets (1)
Spaces using Keypoint Detection

Note An application that detects hand keypoints in real-time.

Note An application to try a universal keypoint detection model.

Metrics for Keypoint Detection

No example metric is defined for this task.

Note Contribute by proposing a metric for this task !