Machine Perception Primitive:
An implementation of a GentleBoost cascaded classifier for full image object search.
More...
#include <GentleBoostCascadedClassifier.h>
Public Member Functions | |
GentleBoostCascadedClassifier () | |
Constructor. | |
~GentleBoostCascadedClassifier () | |
Destructor. | |
void | setSearchParams (int useFast=0, cv::Size minSize=cv::Size(0, 0), cv::Size maxSize=cv::Size(0, 0), double scaleInc=1.2, double stepWidth=1, int scaleStepWidth=1) |
Set parameters affecting how GentleBoostCascadedClassifier searches through images for objects. | |
void | searchImage (const cv::Mat &gray_image, std::vector< SearchResult > &keptPatches, int NMSRadius=0, double thresh=-INFINITY) |
Search through and image for objects that this classifier was trained to detect. Results are passed back via the SearchResult vector keptPatches. | |
void | setCurrentImage (const cv::Mat &gray_image) |
Set the current image to search, without actually searching. This allows you to search each scale (size) separately and manually, using searchCurrentImageAtScale(). | |
void | searchCurrentImageAtScale (std::vector< SearchResult > &keptPatches, int scale, int NMSRadius=0, double thresh=-INFINITY) |
Search through and image for objects at a single scale (size). | |
int | getNumScales () |
Query the number of scales available for searching for objects. See PatchList for further discussion. | |
cv::Size | getSizeOfScale (int scale) |
Query the size of object searched for at a given scale. See PatchList for further discussion. | |
void | searchPatches (const std::vector< ImagePatch * > &patches, const cv::Mat &labels, cv::Mat &survived, cv::Mat &featureSum, cv::Mat &posterior, cv::Mat &predictions, std::vector< cv::Mat > &featureOutputs, double &perf, cv::Mat &weights) |
Apply the GentleBoostCascadedClasssifier to a classify collection of image patches. | |
void | setTrainingParams (double maxPosRejects=0.001, double desiredNegRejects=1) |
Set parameters used to determine the rejection threshold for each cascade step. | |
virtual PerformanceMetrics | trainOneRound (int patience=1, int boostRounds=0) |
Train boosted classifier for one round by searching for one good feature, and adding it. | |
virtual Feature * | getGoodFeature (int patience=1, std::string featureType="HaarFeature") |
Look for a feature that would improve classifier performance. | |
virtual void | addFeature (Feature *nextFeature) |
Add a feature to the classifier. | |
virtual void | setTrainingSet (const std::vector< ImagePatch * > &trainingPatches, const cv::Mat &trainingLabels) |
Set ImagePatch data and labels used for training the classifier. | |
virtual void | setTestingSet (const std::vector< ImagePatch * > &testingPatches, const cv::Mat &testingLabels) |
Set ImagePatch data and labels used for evaluating the classifier. | |
void | setBGTrainingFromImageDataset (const std::string &datasetFileName) |
Specify a large collection of images known not have the object you are learning a detector for. These will serve as a source of training data. Only a portion of these will be used for training at a time. Call setHardNegativeExamplesFromBGImages() to swap out patches currently used for training ones for harder ones from the background image pool. | |
void | setHardNegativeTrainingExamplesFromBGImages () |
Replace the rejected negative example patches in the current training set with patches in taken from background images that haven't yet been rejected. The background image pool is specified with setBGTrainingImagesFromImageDataset(). | |
void | setHardNegativeTrainingExamplesFromPatches () |
Replace the rejected negative example patches in the current training set with patches in taken from a pool of background patches yet been rejected. The negative patch pool is specified with setBGTrainingPatches(). | |
bool | exhaustedAllNegPatches () |
Returns "yes" if all known negative patches have been rejected. This is a good time to stop training. | |
void | addFeatureBoosted (Feature *nextFeature, int boostRounds) |
Add a feature to the classifier by applying multiple rounds of boosting. | |
virtual void | getProbabilityMap (cv::Mat &dest, PatchList *patches) |
Evaluate every pixel in an image for its probability that an object is located there. | |
void | getPerformanceMeasures (Feature *candidate, double &chiSq, int &posRejects, int &negRejects) |
Compute perfomance measures for this classifier, depending on the Feature* passed in. This is primarily used for finding the best feature to add to the classifier, using addFeature(). | |
void | getPerformanceMeasures (const Feature *candidate, PerformanceMetrics &perf) |
Compute perfomance measures for this classifier, depending on the Feature* passed in. This is primarily used for finding the best feature to add to the classifier, using addFeature(). | |
int | getNumNegPatches () |
Total number of negative patches currently being used to train the classifier. | |
int | getNumPosPatches () |
Total number of positive patches currently being used to train the classifier. | |
void | getImageHeaderForNegPatch (cv::Mat &dst, int patchNum) |
Get an IplImage for visualizing the negative patches in the classifier - This must be released using cvReleaseImageHeader, and not cvReleaseImage, or you will destroy the data used for training. | |
void | getImageHeaderForPosPatch (cv::Mat &dst, int patchNum) |
Get an IplImage for visualizing the positive patches in the classifier - This must be released using cvReleaseImageHeader, and not cvReleaseImage, or you will destroy the data used for training. | |
void | sharePatchListWithClassifier (GentleBoostCascadedClassifier *otherClassifier) |
Share the patch list of another GentleBoostCascadedClassifier, to increase efficiency of multiple classifiers processing the same image. | |
Friends | |
std::ostream & | operator<< (std::ostream &ofs, GentleBoostCascadedClassifier *booster) |
Write to a file. | |
std::istream & | operator>> (std::istream &ifs, GentleBoostCascadedClassifier *&booster) |
Read from a file. |
Machine Perception Primitive:
An implementation of a GentleBoost cascaded classifier for full image object search.
The GentleBoost approach is described in Fasel's "Learning Real-Time Object Detectors: Probabilistic Generative Approaches", 2006 (see Related Publications).
GentleBoostCascadedClassifier is designed for full frame object detection.
void GentleBoostCascadedClassifier::addFeature | ( | Feature * | nextFeature ) | [virtual] |
Add a feature to the classifier.
This will create a FeatureRegressor and compute its tuning curve based on the current weighting of the training examples, and then reweight the training examples according to the GentleBoost algorithm.
nextFeature | The feature to add. |
Reimplemented from GentleBoostClassifier.
void GentleBoostCascadedClassifier::addFeatureBoosted | ( | Feature * | nextFeature, |
int | boostRounds | ||
) |
Add a feature to the classifier by applying multiple rounds of boosting.
This allows a single feature to have more discriminative power, with the danger of overfitting.
The classification result is equivalent to calling addFeature() multiple times on the same feature, but leads to greater computational efficiency of the overall classifier.
nextFeature | The feature to add. |
boostRounds | Number of times to (effectively) call addFeature on this feature. |
virtual Feature* GentleBoostCascadedClassifier::getGoodFeature | ( | int | patience = 1 , |
std::string | featureType = "HaarFeature" |
||
) | [virtual] |
Look for a feature that would improve classifier performance.
patience | How long do you want to wait to find a good feature? |
featureType | Type of feature to look for. |
void GentleBoostCascadedClassifier::getImageHeaderForNegPatch | ( | cv::Mat & | dst, |
int | patchNum | ||
) |
Get an IplImage for visualizing the negative patches in the classifier - This must be released using cvReleaseImageHeader, and not cvReleaseImage, or you will destroy the data used for training.
This is useful for, for example, seeing what negative examples are currently confusing the classifier.
patchNum | Must be between 0 and getNumNegPatches() |
void GentleBoostCascadedClassifier::getImageHeaderForPosPatch | ( | cv::Mat & | dst, |
int | patchNum | ||
) |
Get an IplImage for visualizing the positive patches in the classifier - This must be released using cvReleaseImageHeader, and not cvReleaseImage, or you will destroy the data used for training.
This is useful for, for example, seeing what positive examples are currently confusing the classifier.
patchNum | Must be between 0 and getNumPosPatches() |
int GentleBoostCascadedClassifier::getNumScales | ( | ) |
Query the number of scales available for searching for objects. See PatchList for further discussion.
void GentleBoostCascadedClassifier::getPerformanceMeasures | ( | Feature * | candidate, |
double & | chiSq, | ||
int & | posRejects, | ||
int & | negRejects | ||
) |
Compute perfomance measures for this classifier, depending on the Feature* passed in. This is primarily used for finding the best feature to add to the classifier, using addFeature().
getPerformanceMeasures has three modes depending on the candidate feature:
candidate | Can be NULL, one of the features in the classifier, or a candidate feature to add to the classifier. |
chiSq | The computed error measure (lower is better). |
posRejects | The number of positive examples rejected by adding this feature (lower is better). |
negRejects | The number of negative examples rejected by adding this feature (higher is better). |
void GentleBoostCascadedClassifier::getPerformanceMeasures | ( | const Feature * | candidate, |
PerformanceMetrics & | perf | ||
) |
Compute perfomance measures for this classifier, depending on the Feature* passed in. This is primarily used for finding the best feature to add to the classifier, using addFeature().
getPerformanceMeasures has three modes depending on the candidate feature:
candidate | Can be NULL, one of the features in the classifier, or a candidate feature to add to the classifier. |
perf | Metrics of performance. See PerformanceMetrics documentation for further reference. |
virtual void GentleBoostCascadedClassifier::getProbabilityMap | ( | cv::Mat & | dest, |
PatchList * | patches | ||
) | [virtual] |
Evaluate every pixel in an image for its probability that an object is located there.
patches | A PatchList that has had setImage() called, and also resetListToScale(). Only the current search scale is evaluated. |
Size GentleBoostCascadedClassifier::getSizeOfScale | ( | int | scale ) |
Query the size of object searched for at a given scale. See PatchList for further discussion.
scale | Search scale. |
void GentleBoostCascadedClassifier::searchCurrentImageAtScale | ( | std::vector< SearchResult > & | keptPatches, |
int | scale, | ||
int | NMSRadius = 0 , |
||
double | thresh = -INFINITY |
||
) |
Search through and image for objects at a single scale (size).
keptPatches | Results of the search are recorded in this vector. This is faster than returning a vector. |
scale | Scale to search at. Must be between 0 and getNumScales()-1. |
NMSRadius | Suppress object detection results if there is a another SearchResult with higher value nearby (within radius NMSRadius pixels). |
thresh | Suppress object detection results with a value lower than thresh. Using -INFINITY returns all results. Using thresh=0 returns only SearchResult records that the cascade has at least 50% confidence in. |
void GentleBoostCascadedClassifier::searchImage | ( | const cv::Mat & | gray_image, |
std::vector< SearchResult > & | keptPatches, | ||
int | NMSRadius = 0 , |
||
double | thresh = -INFINITY |
||
) |
Search through and image for objects that this classifier was trained to detect. Results are passed back via the SearchResult vector keptPatches.
gray_image | Image or Frame to search. Must be single-channel, with depth IPL_DEPTH_8U. |
keptPatches | Results of the search are recorded in this vector. This is faster than returning a vector. |
NMSRadius | Suppress object detection results if there is a another SearchResult with higher value nearby (within radius NMSRadius pixels). |
thresh | Suppress object detection results with a value lower than thresh. Using -INFINITY returns all results. Using thresh=0 returns only SearchResult records that the cascade has at least 50% confidence in. |
void GentleBoostCascadedClassifier::searchPatches | ( | const std::vector< ImagePatch * > & | patches, |
const cv::Mat & | labels, | ||
cv::Mat & | survived, | ||
cv::Mat & | featureSum, | ||
cv::Mat & | posterior, | ||
cv::Mat & | predictions, | ||
std::vector< cv::Mat > & | featureOutputs, | ||
double & | perf, | ||
cv::Mat & | weights | ||
) |
Apply the GentleBoostCascadedClasssifier to a classify collection of image patches.
This classification requires a lot of scratch memory, which you can provide. The behavior of this function depends on which provided pointers are NULL. For example, perf and weights are only computed if labels is not NULL, whereas featureSum, posterior, and predictions are always computed. If any matrices that are required are NULL or incorrectly sized, they will be (re)allocated, and the caller of the function is responsible for managing that memory.
patches | A list of patches to classify. |
labels | Optional labels for the patches, to evaluate classifier performance. |
survived | Indicates whether or not a patch was rejected before classification finished. If rejected, it is automatically given the label prediction -1, regardless of the posterior. |
featureSum | The accumulated feature output that is used to predict the label of the patch. |
posterior | The probability estimate that the patch was generated by the trained class. |
predictions | Binary (+1/-1) classification labels applied by the classifier. |
featureOutputs | The output of each individual FeatureRegressor applied to each patch. |
perf | Chi-Squared error in predicting labels (only set if labels are provided). |
weights | Weights based on boosting that can be used for training the next feature. |
void GentleBoostCascadedClassifier::setBGTrainingFromImageDataset | ( | const std::string & | datasetFileName ) |
Specify a large collection of images known not have the object you are learning a detector for. These will serve as a source of training data. Only a portion of these will be used for training at a time. Call setHardNegativeExamplesFromBGImages() to swap out patches currently used for training ones for harder ones from the background image pool.
datasetFileName | Path to a file containing the paths to images (one per line) known to not contain the detector target. |
void GentleBoostCascadedClassifier::setCurrentImage | ( | const cv::Mat & | gray_image ) |
Set the current image to search, without actually searching. This allows you to search each scale (size) separately and manually, using searchCurrentImageAtScale().
gray_image | Image or Frame to search. Must be single-channel, with depth IPL_DEPTH_8U. |
void GentleBoostCascadedClassifier::setSearchParams | ( | int | useFast = 0 , |
cv::Size | minSize = cv::Size(0, 0) , |
||
cv::Size | maxSize = cv::Size(0, 0) , |
||
double | scaleInc = 1.2 , |
||
double | stepWidth = 1 , |
||
int | scaleStepWidth = 1 |
||
) |
Set parameters affecting how GentleBoostCascadedClassifier searches through images for objects.
These are all parameters that are passed on to a PatchList object. See PatchList documentation for further details.
useFast | Use "FastPatchList" rather than "PatchList" for searching. This improves speed, but can significantly hamper detector performance. Using useFast=0 is highly recommended. |
minSize | Don't search for objects smaller than minSize. If minSize is (0,0) then baseObjectSize is used. |
maxSize | Don't search for objects larger than maxSize. If maxSize is (0,0), then the upper bound on the object size is the smallest dimension of the image size. |
scaleInc | Relative size of the object in the next scale, compared to the current one. scaleInc must be greater than 1. |
stepWidth | Search interval between subsequent patches. A stepWidth of 1 means slide the object evaluation window over one pixel. 2 means only evaluate patches at every other pixel. 1.5 means search two and skip the third, etc. |
scaleStepWidth | Only applies if useFast=1. A boolean (0 or non-zero) value, indicating whether the stepWidth should scale up with the patch size. Setting 1 gives the same relative coverage to all scales, setting to 0 gives relatively finer coverage to larger scales, and searching will take considerably more time. |
virtual void GentleBoostCascadedClassifier::setTestingSet | ( | const std::vector< ImagePatch * > & | testingPatches, |
const cv::Mat & | testingLabels | ||
) | [virtual] |
Set ImagePatch data and labels used for evaluating the classifier.
testingPatches | A collection of positive and negative examples that the classifier's discrimination will be evaluated on. |
testingLabels | Binary (+1/-1) indicating the label of each patch. This should be a matrix of type CV_64FC1 with size numPatches x 1. |
void GentleBoostCascadedClassifier::setTrainingParams | ( | double | maxPosRejects = 0.001 , |
double | desiredNegRejects = 1 |
||
) |
Set parameters used to determine the rejection threshold for each cascade step.
The threshold is chosen as soon as more than a fraction maxPosRejects of the remaining positive patches have been rejected, or when a fraction desiredNegRejects of the remaining negative patches have been rejected.
maxPosRejects | Max fraction of remaining positive patches rejected per training round. |
desiredNegRejects | Desired fraction of remaining negative patches rejected per training round. |
virtual void GentleBoostCascadedClassifier::setTrainingSet | ( | const std::vector< ImagePatch * > & | trainingPatches, |
const cv::Mat & | trainingLabels | ||
) | [virtual] |
Set ImagePatch data and labels used for training the classifier.
trainingPatches | A collection of positive and negative examples that the classifier should learn to discriminate. |
trainingLabels | Binary (+1/-1) indicating the label of each patch. This should be a matrix of type CV_64FC1 with size numPatches x 1. |
void GentleBoostCascadedClassifier::sharePatchListWithClassifier | ( | GentleBoostCascadedClassifier * | otherClassifier ) |
Share the patch list of another GentleBoostCascadedClassifier, to increase efficiency of multiple classifiers processing the same image.
By sharing a patch list, calling "setImage" on one classifier will set it for all of them. Thus different classifiers can share processing on the same image data.
However, be careful with this. It cannot be undone (within the lifetime of a single object -- reloading the classifier from disk will undo it).
otherClassifier | Classifier to share a PatchList with. |
PerformanceMetrics GentleBoostCascadedClassifier::trainOneRound | ( | int | patience = 1 , |
int | boostRounds = 0 |
||
) | [virtual] |
Train boosted classifier for one round by searching for one good feature, and adding it.
patience | How long do you want to wait to find a good feature? |
boostRounds | Train for multiple rounds of boosting on one feature: makes the classifier more discriminative but more prone to overfit. |