Skip to content
/ tide Public
forked from dbolya/tide

A General Toolbox for Identifying Object Detection Errors

License

Notifications You must be signed in to change notification settings

wtiandong/tide

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A General Toolbox for Identifying Object Detection Errors

████████╗██╗██████╗ ███████╗
╚══██╔══╝██║██╔══██╗██╔════╝
   ██║   ██║██║  ██║█████╗  
   ██║   ██║██║  ██║██╔══╝  
   ██║   ██║██████╔╝███████╗
   ╚═╝   ╚═╝╚═════╝ ╚══════╝

An easy-to-use, general toolbox to compute and evaluate the effect of object detection and instance segmentation on overall performance. This is the code for our paper: TIDE: A General Toolbox for Identifying Object Detection Errors (ArXiv) [ECCV2020 Spotlight].

Check out our ECCV 2020 short video for an explanation of what TIDE can do:

TIDE Introduction

Installation

TIDE is available as a python package for python 3.6+ as tidecv. To install, simply install it with pip:

pip3 install tidecv

The current version is v1.0.1 (changelog).

Usage

TIDE is meant as a drop-in replacement for the COCO Evaluation toolkit, and getting started is easy:

from tidecv import TIDE, datasets

tide = TIDE()
tide.evaluate(datasets.COCO(), datasets.COCOResult('path/to/your/results/file'), mode=TIDE.BOX) # Use TIDE.MASK for masks
tide.summarize()  # Summarize the results as tables in the console
tide.plot()       # Show a summary figure. Specify a folder and it'll output a png to that folder.

This prints evaluation summary tables to the console:

-- mask_rcnn_bbox --

bbox AP @ 50: 61.80

                         Main Errors
=============================================================
  Type      Cls      Loc     Both     Dupe      Bkg     Miss
-------------------------------------------------------------
   dAP     3.40     6.65     1.18     0.19     3.96     7.53
=============================================================

        Special Error
=============================
  Type   FalsePos   FalseNeg
-----------------------------
   dAP      16.28      15.57
=============================

And a summary plot for your model's errors:

A summary plot

Jupyter Notebook

Check out the example notebook for more details.

Datasets

The currently supported datasets are COCO, LVIS, Pascal, and Cityscapes. More details and documentation on how to write your own database drivers coming soon!

Citation

If you use TIDE in your project, please cite

@inproceedings{tide-eccv2020,
  author    = {Daniel Bolya and Sean Foley and James Hays and Judy Hoffman},
  title     = {TIDE: A General Toolbox for Identifying Object Detection Errors},
  booktitle = {ECCV},
  year      = {2020},
}

Contact

For questions about our paper or code, make an issue in this github or contact Daniel Bolya. Note that I may not respond to emails, so github issues are your best bet.

About

A General Toolbox for Identifying Object Detection Errors

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%