Annolid on Detectron2 Tutorial 1 : Introduction to Detectron2#
Welcome to Annolid on detectron2! This is modified from the official colab tutorial of detectron2. Here, we will go through some basics usage of detectron2, including the following:
Run inference on images or videos, with an existing detectron2 model
Train a detectron2 model on a new dataset
You can make a copy of this tutorial by “File -> Open in playground mode” and play with it yourself. DO NOT request access to this tutorial.
Install detectron2#
# Is running in colab or in jupyter-notebook
try:
import google.colab
IN_COLAB = True
except:
IN_COLAB = False
# install dependencies:
!pip install pyyaml==5.3
import torch, torchvision
TORCH_VERSION = ".".join(torch.__version__.split(".")[:2])
CUDA_VERSION = torch.__version__.split("+")[-1]
print("torch: ", TORCH_VERSION, "; cuda: ", CUDA_VERSION)
# Install detectron2 that matches the above pytorch version
# See https://detectron2.readthedocs.io/tutorials/install.html for instructions
!pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/$CUDA_VERSION/torch$TORCH_VERSION/index.html
# If there is not yet a detectron2 release that matches the given torch + CUDA version, you need to install a different pytorch.
# exit(0) # After installation, you may need to "restart runtime" in Colab. This line can also restart runtime
Requirement already satisfied: pyyaml==5.3 in /home/jeremy/anaconda3/lib/python3.9/site-packages (5.3)
torch: 1.10 ; cuda: cu102
Looking in links: https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.10/index.html
Requirement already satisfied: detectron2 in /home/jeremy/anaconda3/lib/python3.9/site-packages (0.6+cu102)
Requirement already satisfied: pycocotools>=2.0.2 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (2.0.3)
Requirement already satisfied: Pillow>=7.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (8.4.0)
Requirement already satisfied: tqdm>4.29.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (4.62.3)
Requirement already satisfied: omegaconf>=2.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (2.1.1)
Requirement already satisfied: hydra-core>=1.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (1.1.1)
Requirement already satisfied: iopath<0.1.10,>=0.1.7 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (0.1.9)
Requirement already satisfied: black==21.4b2 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (21.4b2)
Requirement already satisfied: termcolor>=1.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (1.1.0)
Requirement already satisfied: tensorboard in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (2.7.0)
Requirement already satisfied: future in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (0.18.2)
Requirement already satisfied: tabulate in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (0.8.9)
Requirement already satisfied: pydot in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (1.4.2)
Requirement already satisfied: yacs>=0.1.8 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (0.1.8)
Requirement already satisfied: matplotlib in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (3.4.3)
Requirement already satisfied: cloudpickle in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (2.0.0)
Requirement already satisfied: fvcore<0.1.6,>=0.1.5 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from detectron2) (0.1.5.post20211023)
Requirement already satisfied: click>=7.1.2 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from black==21.4b2->detectron2) (8.0.3)
Requirement already satisfied: toml>=0.10.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from black==21.4b2->detectron2) (0.10.2)
Requirement already satisfied: regex>=2020.1.8 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from black==21.4b2->detectron2) (2021.8.3)
Requirement already satisfied: pathspec<1,>=0.8.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from black==21.4b2->detectron2) (0.9.0)
Requirement already satisfied: mypy-extensions>=0.4.3 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from black==21.4b2->detectron2) (0.4.3)
Requirement already satisfied: appdirs in /home/jeremy/anaconda3/lib/python3.9/site-packages (from black==21.4b2->detectron2) (1.4.4)
Requirement already satisfied: numpy in /home/jeremy/anaconda3/lib/python3.9/site-packages (from fvcore<0.1.6,>=0.1.5->detectron2) (1.20.3)
Requirement already satisfied: pyyaml>=5.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from fvcore<0.1.6,>=0.1.5->detectron2) (5.3)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from hydra-core>=1.1->detectron2) (4.8)
Requirement already satisfied: portalocker in /home/jeremy/anaconda3/lib/python3.9/site-packages (from iopath<0.1.10,>=0.1.7->detectron2) (2.3.2)
Requirement already satisfied: setuptools>=18.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from pycocotools>=2.0.2->detectron2) (58.0.4)
Requirement already satisfied: cython>=0.27.3 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from pycocotools>=2.0.2->detectron2) (0.29.24)
Requirement already satisfied: python-dateutil>=2.7 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from matplotlib->detectron2) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from matplotlib->detectron2) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from matplotlib->detectron2) (1.3.1)
Requirement already satisfied: pyparsing>=2.2.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from matplotlib->detectron2) (3.0.4)
Requirement already satisfied: six in /home/jeremy/anaconda3/lib/python3.9/site-packages (from cycler>=0.10->matplotlib->detectron2) (1.16.0)
Requirement already satisfied: werkzeug>=0.11.15 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (2.0.2)
Requirement already satisfied: absl-py>=0.4 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (1.0.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (0.6.1)
Requirement already satisfied: requests<3,>=2.21.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (2.26.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (0.4.6)
Requirement already satisfied: protobuf>=3.6.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (3.19.1)
Requirement already satisfied: grpcio>=1.24.3 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (1.43.0)
Requirement already satisfied: wheel>=0.26 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (0.37.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (2.3.3)
Requirement already satisfied: markdown>=2.6.8 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (3.3.6)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from tensorboard->detectron2) (1.8.1)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2) (4.2.4)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2) (1.3.0)
Requirement already satisfied: importlib-metadata>=4.4 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard->detectron2) (4.8.1)
Requirement already satisfied: zipp>=0.5 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard->detectron2) (3.6.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->detectron2) (0.4.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard->detectron2) (2021.10.8)
Requirement already satisfied: oauthlib>=3.0.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2) (3.1.1)
# import some common libraries
import json
import os
import cv2
import random
import glob
import numpy as np
if IN_COLAB:
from google.colab.patches import cv2_imshow
import matplotlib.pyplot as plt
%matplotlib inline
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()
# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog
# is there a gpu
if torch.cuda.is_available():
GPU = True
print('gpu available')
else:
GPU = False
print('no gpu')
gpu available
Upload a labeled dataset.#
The following code is expecting the dataset in the COCO format to be in a .zip file. For example: sample_dataset.zip \
!pip install gdown
!gdown --id 1fUXCLnoJ5SwXg54mj0NBKGzidsV8ALVR
Requirement already satisfied: gdown in /home/jeremy/anaconda3/lib/python3.9/site-packages (4.2.0)
Requirement already satisfied: filelock in /home/jeremy/anaconda3/lib/python3.9/site-packages (from gdown) (3.3.1)
Requirement already satisfied: six in /home/jeremy/anaconda3/lib/python3.9/site-packages (from gdown) (1.16.0)
Requirement already satisfied: tqdm in /home/jeremy/anaconda3/lib/python3.9/site-packages (from gdown) (4.62.3)
Requirement already satisfied: beautifulsoup4 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from gdown) (4.10.0)
Requirement already satisfied: requests[socks] in /home/jeremy/anaconda3/lib/python3.9/site-packages (from gdown) (2.26.0)
Requirement already satisfied: soupsieve>1.2 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from beautifulsoup4->gdown) (2.2.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests[socks]->gdown) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests[socks]->gdown) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests[socks]->gdown) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests[socks]->gdown) (2021.10.8)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /home/jeremy/anaconda3/lib/python3.9/site-packages (from requests[socks]->gdown) (1.7.1)
Downloading...
From: https://drive.google.com/uc?id=1fUXCLnoJ5SwXg54mj0NBKGzidsV8ALVR
To: /home/jeremy/Documents/annolid/book/tutorials/novelctrlk6_8_coco_dataset.zip
100%|██████████████████████████████████████| 10.3M/10.3M [00:00<00:00, 96.4MB/s]
if IN_COLAB:
dataset = '/content/novelctrlk6_8_coco_dataset.zip'
else:
dataset = 'novelctrlk6_8_coco_dataset.zip'
Note1: If you want to use your own dataset instead of the demo one, please uncomment and edit the following code.#
Note2: please make sure there is no white space in your file path if you encounter file not found issues.#
# if IN_COLAB:
# from google.colab import files
# uploaded = files.upload()
# else:
# from ipywidgets import FileUpload
# from IPython.display import display
# !jupyter nbextension enable --py widgetsnbextension
# uploaded = FileUpload()
# display(uploaded)
# if IN_COLAB:
# dataset = list(uploaded.keys())[0]
# else:
# dataset = list(uploaded.value.keys())[0]
If your dataset has the same name as the file you uploaded, you do not need to manually input the name (just run the next cells). Otherwise, you need to replace DATASET_NAME and DATASET_DIR with your own strings like DATASET_NAME = "NameOfMyDataset" and DATASETDIR="NameOfMyDatasetDirectory". To do that, uncomment the commented out cell below and replace the strings with the appropriate names
# DATASET_NAME = 'NameOfMyDataset'
# DATASET_DIR = 'NameOfMyDatasetDirectory'
if IN_COLAB:
!unzip $dataset -d /content/
else:
!unzip -o $dataset -d .
Archive: novelctrlk6_8_coco_dataset.zip
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00001416_41.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004233_81.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004515_22.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000636_6.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006297_11.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006818_79.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006056_25.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006094_12.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004340_96.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000557_50.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000979_94.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00005018_19.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00001257_80.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004335_26.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004804_39.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006396_43.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006993_65.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000865_92.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003114_35.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00001918_2.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004935_56.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006959_78.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00005216_14.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003092_22.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00001528_44.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003208_12.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004966_35.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004517_47.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004769_82.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004767_87.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000378_62.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006563_31.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00001476_48.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004555_67.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003070_62.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00005007_38.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003342_41.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00002267_91.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00002248_99.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000702_46.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003237_15.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00002412_96.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00001322_73.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00002328_97.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004312_85.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004443_46.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003783_81.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000779_3.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00002654_1.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004685_90.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004098_59.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006538_99.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006832_71.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00007024_26.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00005772_51.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00006330_76.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00000871_10.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00003034_64.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00004539_89.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/JPEGImages/00005788_49.jpg
inflating: ./novelctrlk6_8_coco_dataset/valid/annotations.json
inflating: ./novelctrlk6_8_coco_dataset/.DS_Store
inflating: ./__MACOSX/novelctrlk6_8_coco_dataset/._.DS_Store
inflating: ./novelctrlk6_8_coco_dataset/data.yaml
inflating: ./novelctrlk6_8_coco_dataset/train/.DS_Store
inflating: ./__MACOSX/novelctrlk6_8_coco_dataset/train/._.DS_Store
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001466_29.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006804_57.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00007044_50.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006232_55.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000759_28.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003680_32.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004441_88.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005090_94.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002476_27.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000965_42.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001652_55.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002000_18.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002333_0.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005752_39.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006670_0.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005763_18.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004469_75.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005566_43.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004336_74.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006574_88.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001922_1.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001795_5.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005755_34.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003576_54.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002647_78.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004295_47.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003667_56.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006131_85.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000311_77.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000879_40.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002991_7.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000757_5.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004628_68.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004348_33.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003269_66.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003224_49.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002628_21.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001838_25.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002825_40.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002247_29.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003100_30.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001922_57.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001320_92.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006299_63.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000952_60.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000842_70.jpg
inflating: ./__MACOSX/novelctrlk6_8_coco_dataset/train/JPEGImages/._00000842_70.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002295_32.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004117_59.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003976_44.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004032_20.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006212_69.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003134_31.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006169_98.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006830_36.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004570_24.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000735_23.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003905_42.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003012_37.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005456_95.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002456_9.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002433_90.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006715_97.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005597_15.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006319_53.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005025_83.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000530_64.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002362_63.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002311_72.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005068_37.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006353_11.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005056_91.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000970_34.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002577_24.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006601_16.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000652_65.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002639_95.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005953_8.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006000_27.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005700_89.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004672_51.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004062_19.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005246_28.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005154_53.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005584_93.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003834_80.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004711_6.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006038_20.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006373_2.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006470_84.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003956_16.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002625_79.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006270_45.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006521_86.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001999_45.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004081_4.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00000776_14.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002546_58.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001181_61.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002226_87.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001632_33.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006218_71.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00005400_9.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001299_86.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00002067_75.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006844_69.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001892_72.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001153_93.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001389_82.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001439_52.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006367_98.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004018_67.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004535_48.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006361_13.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003167_10.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00006832_66.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00004956_38.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00001980_61.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/JPEGImages/00003364_13.jpg
inflating: ./novelctrlk6_8_coco_dataset/train/annotations.json
DATASET_NAME = DATASET_DIR = f"{dataset.replace('.zip','')}"
Run a pre-trained detectron2 model#
First, we check a random selected image from our training dataset:
# select and display one random image from the training set
img_file = random.choice(glob.glob(f"{DATASET_DIR}/train/JPEGImages/*.*"))
im = cv2.imread(img_file)
if IN_COLAB:
cv2_imshow(im)
else:
plt.imshow(im)
Then, we create a Detectron2 config and a detectron2 DefaultPredictor to run inference on this image.
cfg = get_cfg()
if GPU:
pass
else:
cfg.MODEL.DEVICE='cpu'
# add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.1 # set threshold for this model
# Find a model from Detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
predictor = DefaultPredictor(cfg)
outputs = predictor(im)
/home/jeremy/anaconda3/envs/annolid-env/lib/python3.7/site-packages/detectron2/structures/image_list.py:88: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
max_size = (max_size + (stride - 1)) // stride * stride
/home/jeremy/anaconda3/envs/annolid-env/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:2157.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
# look at the outputs. See https://detectron2.readthedocs.io/tutorials/models.html#model-output-format for specification
print(outputs["instances"].pred_classes)
print(outputs["instances"].pred_boxes)
tensor([ 6, 74, 74, 65, 74, 68], device='cuda:0')
Boxes(tensor([[ 100.5904, 16.4716, 1262.0560, 977.0276],
[ 504.0177, 541.5104, 551.6971, 589.2094],
[ 898.6787, 518.7311, 947.8806, 568.1455],
[ 19.0260, 140.1515, 163.2243, 981.9127],
[ 788.5672, 356.4314, 832.5961, 404.7740],
[ 121.0352, 0.0000, 1254.3234, 988.9520]], device='cuda:0'))
outputs['instances'].pred_masks
tensor([[[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False]],
[[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False]],
[[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False]],
[[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False]],
[[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False]],
[[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False],
[False, False, False, ..., False, False, False]]], device='cuda:0')
MetadataCatalog.get(cfg.DATASETS.TRAIN[0])
namespace(name='coco_2017_train',
json_file='datasets/coco/annotations/instances_train2017.json',
image_root='datasets/coco/train2017',
evaluator_type='coco',
thing_dataset_id_to_contiguous_id={1: 0,
2: 1,
3: 2,
4: 3,
5: 4,
6: 5,
7: 6,
8: 7,
9: 8,
10: 9,
11: 10,
13: 11,
14: 12,
15: 13,
16: 14,
17: 15,
18: 16,
19: 17,
20: 18,
21: 19,
22: 20,
23: 21,
24: 22,
25: 23,
27: 24,
28: 25,
31: 26,
32: 27,
33: 28,
34: 29,
35: 30,
36: 31,
37: 32,
38: 33,
39: 34,
40: 35,
41: 36,
42: 37,
43: 38,
44: 39,
46: 40,
47: 41,
48: 42,
49: 43,
50: 44,
51: 45,
52: 46,
53: 47,
54: 48,
55: 49,
56: 50,
57: 51,
58: 52,
59: 53,
60: 54,
61: 55,
62: 56,
63: 57,
64: 58,
65: 59,
67: 60,
70: 61,
72: 62,
73: 63,
74: 64,
75: 65,
76: 66,
77: 67,
78: 68,
79: 69,
80: 70,
81: 71,
82: 72,
84: 73,
85: 74,
86: 75,
87: 76,
88: 77,
89: 78,
90: 79},
thing_classes=['person',
'bicycle',
'car',
'motorcycle',
'airplane',
'bus',
'train',
'truck',
'boat',
'traffic light',
'fire hydrant',
'stop sign',
'parking meter',
'bench',
'bird',
'cat',
'dog',
'horse',
'sheep',
'cow',
'elephant',
'bear',
'zebra',
'giraffe',
'backpack',
'umbrella',
'handbag',
'tie',
'suitcase',
'frisbee',
'skis',
'snowboard',
'sports ball',
'kite',
'baseball bat',
'baseball glove',
'skateboard',
'surfboard',
'tennis racket',
'bottle',
'wine glass',
'cup',
'fork',
'knife',
'spoon',
'bowl',
'banana',
'apple',
'sandwich',
'orange',
'broccoli',
'carrot',
'hot dog',
'pizza',
'donut',
'cake',
'chair',
'couch',
'potted plant',
'bed',
'dining table',
'toilet',
'tv',
'laptop',
'mouse',
'remote',
'keyboard',
'cell phone',
'microwave',
'oven',
'toaster',
'sink',
'refrigerator',
'book',
'clock',
'vase',
'scissors',
'teddy bear',
'hair drier',
'toothbrush'],
thing_colors=[[220, 20, 60],
[119, 11, 32],
[0, 0, 142],
[0, 0, 230],
[106, 0, 228],
[0, 60, 100],
[0, 80, 100],
[0, 0, 70],
[0, 0, 192],
[250, 170, 30],
[100, 170, 30],
[220, 220, 0],
[175, 116, 175],
[250, 0, 30],
[165, 42, 42],
[255, 77, 255],
[0, 226, 252],
[182, 182, 255],
[0, 82, 0],
[120, 166, 157],
[110, 76, 0],
[174, 57, 255],
[199, 100, 0],
[72, 0, 118],
[255, 179, 240],
[0, 125, 92],
[209, 0, 151],
[188, 208, 182],
[0, 220, 176],
[255, 99, 164],
[92, 0, 73],
[133, 129, 255],
[78, 180, 255],
[0, 228, 0],
[174, 255, 243],
[45, 89, 255],
[134, 134, 103],
[145, 148, 174],
[255, 208, 186],
[197, 226, 255],
[171, 134, 1],
[109, 63, 54],
[207, 138, 255],
[151, 0, 95],
[9, 80, 61],
[84, 105, 51],
[74, 65, 105],
[166, 196, 102],
[208, 195, 210],
[255, 109, 65],
[0, 143, 149],
[179, 0, 194],
[209, 99, 106],
[5, 121, 0],
[227, 255, 205],
[147, 186, 208],
[153, 69, 1],
[3, 95, 161],
[163, 255, 0],
[119, 0, 170],
[0, 182, 199],
[0, 165, 120],
[183, 130, 88],
[95, 32, 0],
[130, 114, 135],
[110, 129, 133],
[166, 74, 118],
[219, 142, 185],
[79, 210, 114],
[178, 90, 62],
[65, 70, 15],
[127, 167, 115],
[59, 105, 106],
[142, 108, 45],
[196, 172, 0],
[95, 54, 80],
[128, 76, 255],
[201, 57, 1],
[246, 0, 122],
[191, 162, 208]])
# We can use `Visualizer` to draw the predictions on the image.
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
if IN_COLAB:
cv2_imshow(out.get_image()[:, :, ::-1])
else:
plt.imshow(out.get_image()[:, :, ::-1])
As we can see, the network doesn’t detect what we want. That is expected as we have not fine-tuned the network with our custom dataset. We are going to do that in the next steps.