Visual Tracking with Online Multiple Instance Learning Boris Babenko1, Ming-Hsuan Yang2, Serge Belongie1 1. University of California, San Diego 2. University of California, Merced Tracking вЂў Problem: track arbitrary object in video given location in first frame вЂў Typical Tracking System: o Appearance Model [Ross et al. вЂ�07] вЂў Color histograms, filter banks, subspaces, etc o Motion/Dynamic Model o Optimization/Search вЂў Greedy local search, particle filter, etc Tracking вЂў Problem: track arbitrary object in video given location in first frame вЂў Typical Tracking System: o Appearance Model [Ross et al. вЂ�07] вЂў Color histograms, filter banks, subspaces, etc o Motion/Dynamic Model o Optimization/Search вЂў Greedy local search, particle filter, etc Tracking by Detection вЂў Recent tracking work o Focus on appearance model o Borrow techniques from obj. detection вЂў Slide a discriminative classifier around image o Adaptive appearance model [Collins et al. вЂ�05, Grabner et al. вЂ™06, Ross et al. вЂ�08] Tracking by Detection вЂў First frame is labeled Tracking by Detection вЂў First frame is labeled Classifier Online classifier (i.e. Online AdaBoost) Tracking by Detection вЂў Grab one positive patch, and some negative patch, and train/update the model. Classifier Tracking by Detection вЂў Get next frame Classifier Tracking by Detection вЂў Evaluate classifier in some search window Classifier Classifier Tracking by Detection вЂў Evaluate classifier in some search window X old location Classifier Classifier Tracking by Detection вЂў Find max response XX old location new location Classifier Classifier Tracking by Detection вЂў RepeatвЂ¦ Classifier Classifier Problems with Adaptive Appearance Models вЂў What if classifier is a bit off? o Tracker starts to drift вЂў How to choose training examples? How to Get Training Examples Classifier Classifier MIL Classifier Multiple Instance Learning (MIL) вЂў Ambiguity in training data вЂў Instead of instance/label pairs, get bag of instances/label pairs вЂў Bag is positive if one or more of itвЂ™s members is positive [Keeler вЂ�90, Dietterich et al. вЂ�97] Object Detection вЂў Problem: o Labeling with rectangles is inherently ambiguous o Labeling is sloppy [Viola et al. вЂ�05] MIL for Object Detection вЂў Solution: o Take all of these patches, put into positive bag o At least one patch in bag is вЂњcorrectвЂќ [Viola et al. вЂ�05] Multiple Instance Learning (MIL) вЂў Supervised Learning Training Input вЂў MIL Training Input Multiple Instance Learning (MIL) вЂў Positive bag contains at least one positive instance вЂў Goal: learning instance classifier вЂў Classifier is same format as standard learning How to Get Training Examples Classifier Classifier MIL Classifier How to Get Training Examples Classifier Classifier MIL Classifier Online MILBoost вЂў Need an online MIL algorithm вЂў Combine ideas from MILBoost and Online Boosting [Oza et al. вЂ�01, Viola et al. вЂ™05, Grabner et al. вЂ�06] Boosting вЂў Train classifier of the form: where is a weak classifier вЂў Can make binary predictions using [Freund et al. вЂ�97] MILBoost вЂў Objective to maximize: Log likelihood of bags: where (as in LogitBoost) (Noisy-OR) [Viola et al. вЂ™05, Friedman et al. вЂ�00] MILBoost вЂў Train weak classifier in a greedy fashion вЂў For batch MILBoost can optimize using functional gradient descent. вЂў We need an online versionвЂ¦ Online MILBoost вЂў At all times, keep a pool of classifier candidates [Grabner et al. вЂ�06] weak Updating Online MILBoost вЂў At time t get more training data o Update all candidate classifiers o Pick best K in a greedy fashion Online MILBoost Frame t Get data (bags) Update all classifiers in pool Greedily add best K to strong classifier Frame t+1 MILTrack вЂў MILTrack = o Online MILBoost + o Stumps for weak classifiers + o Randomized Haar features + [Dollar et al. вЂ�07] o Simple motion model + greedy local search Experiments вЂў Compare MILTrack to: o OAB1 = Online AdaBoost w/ 1 pos. per frame o OAB5 = Online AdaBoost w/ 45 pos. per frame o SemiBoost = Online Semi-supervised Boosting o FragTrack = Static appearance model вЂў All params were FIXED вЂў 8 videos, labeled every 5 frames by hand (available on the web) [Grabner вЂ�06, Adam вЂ�06, Grabner вЂ™08] OAB1 OAB5 Classifier Classifier MILTrack MIL Classifier VideosвЂ¦ Results вЂў Ground truth: labeled every 5 frames Best Second Best Conclusions вЂў Proposed Online MILBoost algorithm вЂў Using MIL to train an appearance model results in more robust tracking вЂў Data and code on my website Thanks! вЂў Special thanks to: o Kristin Branson, Piotr DollГЎr, David Ross вЂў Supported by: o NSF CAREER Grant #0448615, NSF IGERT Grant DGE-0333451, and ONR MURI Grant #N00014-081-0638, Honda Research Institute USA.