# Image Features

## Color

• RGB：将RGB空间进行归一化，即可使得颜色信息独立于光强。
• YIQ
• YUV
• HSI

# Density Extimation

## Histograms

The bins of the histogram are defined as the intervals [x0 +mh, x0 +(m + 1)h), for m positive and negative integers, x0 is the origin and h the bin width.

Deawbacks：

• In procedures like cluster analysis and nonparametric discriminant analysis, using histogram
results in inefficient use of the data.
• The histogram is not continuous so trouble arises when derivatives are required.
• Choice of origin may have an effect in the interpretation.
• Representing bivariate or trivariate data by histogram is difficult.

## Naive Estimator

If the random variable X has density f, then:

Thus, the naive estimator is written as:

We define a weight function as follows:

Then f(x) becomes

Drawbacks:

• $\hat f$is not continuous but has jumps at the points Xi±h and has zero derivative everywhere else.
• The following example shows the “stepwise” nature of the esitmate.

## Kernel Estimator

$h$为平滑因子。类似高斯分布函数里的$\sigma$。

## References

• “Density estimation for statistics and data analysis”, B.W. Silverman, 1998, London: Chapman & Hall/CRC.

# Segmentation

## Challenges in Scene Modeling

Illumination changes:

• Gradual change in illumination as might occur in outdoor scenes due to the change in the relative location of the sun during the day.
• Sudden change in illumination as might occur in an indoor environment by switching the lights on or off, or in an outdoor environment, e.g. a change between cloudy and sunny conditions.
• Shadows cast on the background by objects in the background itself (e.g., buildings and trees) or by moving foreground objects.

Motion changes:

• Global image motion due to small camera displacements. Small camera displacements are common in outdoor situations due to wind load or other sources of motion, which causes global motion in the images.
• Motion in parts of the background. For example, tree branches moving with the wind, or rippling water.

Structural changes:

• These are changes introduced to the background, including any change in the geometry or the appearance of the background of the scene introduced by targets. Such changes typically occur when something relatively permanent is introduced into the scene background (BK object removed or parking car.

## References

• Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: Principles and practice of background maintenance. In: IEEE International Conference on Computer Vision (1999)
• Wern, C.R., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of human body. IEEE Trans. Pattern Anal. Mach. Intell. (1997)
• Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (1999)
• Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and foreground modeling using non-parametric kernel density estimation for visual surveillance. Proc. IEEE 90(7), 1151–1163 (2002)

# Mean Shift

Clustering

Smoothing

Segmentation

Mean Shift运动跟踪：

## References

• Mean-Shift算法
• [综] meanshift算法
• Dorin Comaniciu, Peter Meer: Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5): 603-619 (2002).
• Dorin Comaniciu, Peter Meer: Distribution Free Decomposition of Multivariate Data. Pattern Anal. Appl. 2(1): 22-30 (1999).

# Visual Tracking

## Kalman Filter

Cons：

• Kalman filtering is inadequate because it is based on the unimodal Gaussian distribution assumption, and it can’t represent simultaneous alternative hypotheses.
• It works relatively poorly in clutter which causes the density to be multi-modal and therefore non- Gaussian.

Kalman filter is based on the single Gauss model, and different components have different effects on the Gauss distribution, as follows:

• The deterministic component causes the density function to drift bodily.
• The random component of the dynamical model leads to spreading—increasing uncertainty.
• The effect of an external observation is to superimpose a reactive effect on the diffusion.

## Particle Filter

The CONDENSATION Algorithm

At the top of the diagram, the output from time-step t -1 is the weighted sample-set. The aim is to maintain, at successive time-steps, sample sets of fixed size N.

• The first operation is to sample N times from the set , choosing a given element with probability. Some elements, especially those with high weights, may be chosen several times, leading to identical copies of elements in the new set. Others with relatively low weights may not be chosen at all.
• Each element chosen from the new set is now subjected to the predictive steps.First, an element undergoes drift and, since this is deterministic, identical elements in the new set undergo the same drift.
• The second predictive step, diffusion, is random and identical elements now split because each undergoes its own independent motion step. At this stage, the sample set for the new time-step has been generated but, as yet, without its weights;
• Finally, the observation step is applied, generating weights from the observation density.

Algorithm:

## Color-based Particle Filter

Color histograms have many advantages for tracking non-rigid objects as they are robust to partial occlusion, are rotation and scale invariant and are calculated efficiently.

A target is tracked with a particle filter by comparing its histogram with the histograms of the sample positions using the Bhattacharyya distance.

Bhattacharyya distance：在统计学中，Bhattacharyya距离（以下称巴氏距离）测量的是两个离散或连续概率分布的相似性。计算方式和Bhattacharyya系数关系很密切。

Algorithm:

## Kernel-based Particle Filter

A PF does not perform well when the dynamic system has a very small system noise or if the observation noise has very small variance. In these cases, the particle set quickly collapses to one single point in the state space.

The standard PF often fails to produce a particle set that captures the “irregular” motion, leading to gradually drifting estimates and ultimate loss of target.

KPF estimates the gradient of the kernel density and moves particles toward the modes of the posterior, leading to a more effective allocation of particles.

The gradient estimation and particle allocation is implemented by the mean shift algorithm.

## A Boosted Particle Filter

The problem of tracking a varying number of non- rigid objects has two major difficulties:

• First, the observation models and target distributions can be highly non-linear and non- Gaussian.
• Second, the presence of a large, varying number of objects creates complex interactions with overlap and ambiguities.

An effective way is to combine mixture particle filters and Adaboost. The crucial issues in mixture particle filters are the choice of the proposal distribution and the treatment of objects leaving and entering the scene.

The mixture particle filter is ideally suited to multi-target tracking as it assigns a mixture component to each player. The proposal distribution can be constructed by using a mixture model that incorporates information from the dynamic models of each player and the detection hypotheses generated by Adaboost.

Methods：

• Most multi-target tracking assumed a fixed number of objects.
• BraMBLe has an automatic object detection system that relies on modeling a fixed background.
• The authors will relax the assumption of a fixed background where the background changes.
• Particle filters may perform poorly when the posterior is multimodal for multiple targets. Vermaak et al introduce a mixture particle filter (MPF), where each component is modelled with an individual particle filter. BPF is based on MPF.
• The authors adopt a multi-color observation model based on Hue-Saturation-Value (HSV) color histograms.

The boosted particle filter introduces two important extensions of the MPF:

• First, it uses Adaboost to construct the proposal distribution. It incorporates the recent observations in proposal distributions (through the Adaboost detections), and outperforms naive transition prior proposals considerably.
• Second, Adaboost provides a mechanism for obtaining and maintaining the mixture representation. It allows us to detect objects leaving and entering the scene efficiently.

## References

• M. Isard and A. Blake. Condensation–conditional density propagation for visual tracking. Int. J. Computer Vision, 29(1):5– 28, 1998.
• S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for on-line non-linear/non-Gaussian Bayesian tracking,” IEEE Transactions on Signal Processing, vol. 50, pp. 174–188, Feb. 2002.
• K. Nummiaroa, E. Koller-Meierb, L. V. Gool, “An adaptive color- based particle filter”, Image and Vision Computing 21 (2003) 99– 110.
• C.Chang, and R. Ansari, “Kernel Particle Filter for Visual Tracking”, IEEE SIGNAL PROCESSING LETTERS, VOL. 12, NO. 3, pp242-245, 2005.
• K. Okuma, et al., “A Boosted Particle Filter: Multitarget Detection and Tracking”, ECCV 2004 (2004), pp. 28-39.
• 基于粒子滤波器的目标跟踪算法及实现