Object Detection and Segmentation using Meta's Detectron2

Hey guys, In this article, we are going to see how to use Meta's Detectron2 for object detection and segmentation and explore its various applications. First of all, Let's see what is detectron2?

DETECTRON2

Detectron is a PyTorch-based modular object detection library developed by Facebook Researchers. It is Facebook AI Research's next-generation library that provides state-of-the-art detection and segmentation algorithms.

Detectron2 is a ground-up rewrite of Detectron that started with maskrcnn-benchmark. The platform is now implemented in PyTorch. With a new, more modular design, Detectron2 is flexible and extensible, and able to provide fast training on single or multiple GPU servers.

Detectron2 includes high-quality implementations of state-of-the-art object detection algorithms, including DensePose, panoptic feature pyramid networks, and numerous variants of the pioneering Mask R-CNN model family also developed by Facebook AI Research (FAIR). Its extensible design makes it easy to implement cutting-edge research projects without having to fork the entire codebase.

The interesting part of detectron2 is it supports a range of tasks related to object detection. Like the original Detectron, it supports object detection with boxes and instance segmentation masks, as well as human pose prediction. Beyond that, Detectron2 adds support for semantic segmentation and panoptic segmentation, a task that combines both semantic and instance segmentation.

sounds interesting? let's get into CODE 👨‍💻-

Lets us take a simple application of detecting different objects for a given image.

I use google collab and I recommend It for its easy setups. It's your wish to use any environment anyway.

so, create a new notebook

Install pyyaml to start with,

!pip install pyyaml==5.1

we need torch, torchvision, torchaudio to be installed, copy the below code and run it on the cell.

!pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Import the torch and check whether it got installed properly.

import torch
print(torch.cuda.is_available())

if you get True as output -> proceed

now install detectron2. copy the below and run it on the cell.

!pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.7/index.html

import required dependencies from detectron2

import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()

from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog

import cv2
from google.colab.patches import cv2_imshow

import an image locally on the notebook which is to get detected

image = cv2.imread('./det5.jpeg') // import path of your image
cv2_imshow(image)

cfg = get_cfg()

// detection type from model_zoo

cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_R_50_FPN_1x.yaml"))

setting up the model weights and threshold test value

cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/faster_rcnn_R_50_FPN_1x.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.6

predictor = DefaultPredictor(cfg)
outputs=predictor(image)

output using the detectron2's builtin visualizer

viz = Visualizer(image[:,:,::-1]),MetadataCatalog.get(cfg.DATASETS.TRAIN[0])
output = viz[0].draw_instance_predictions(outputs['instances'].to('cpu'))
cv2_imshow(output.get_image()[:,:,::-1])

we can also mask the objects using different detection types. for that, I encourage you to explore model_zoo .we can have Instance Segmentation Baselines with Mask R-CNN, COCO Panoptic Segmentation Baselines with Panoptic FPN, COCO Person Keypoint Detection Baselines with Keypoint R-CNN, etc...

There are various types of detection and segmentation types on model_zoo which are developed and also developing.

As you saw, image detection and segmentation become so easy to implement. Thanks to FAIR. If you are new to detectron, I encourage you to give it a try. It is open-source and very easy to use compared to the traditional way which would take n lines of code to achieve a simple segmentation and masking.

we can also enable a live video cam and mask objects in live, which is very interesting and easy to do. I encourage you to try it by exploring the documentation of detectron2.

OUTRO

It is just a simple application of detectron2 to detect different objects in an image or in a scene. The creative applications are enormous, as detectron is in its early stage, I request you to explore it and come up with new ideas.

related sources:

Thank you for reading 😊

HAPPY LEARNING

-JHA