Image Features


  • RGB:将RGB空间进行归一化,即可使得颜色信息独立于光强。
  • YIQ
  • YUV
  • HSI



Optical Flow




Object Tracking Approches

Point Tracking

Kalman Filter

Kernel Filter

Object detection

Background Subtraction


From a single Image(Machine Learning)

Density Extimation


The bins of the histogram are defined as the intervals [x0 +mh, x0 +(m + 1)h), for m positive and negative integers, x0 is the origin and h the bin width.
\hat f(x) = \frac{number of X_i in the same bin as x}{nh}


  • In procedures like cluster analysis and nonparametric discriminant analysis, using histogram
    results in inefficient use of the data.
  • The histogram is not continuous so trouble arises when derivatives are required.
  • Choice of origin may have an effect in the interpretation.
  • Representing bivariate or trivariate data by histogram is difficult.

Naive Estimator

If the random variable X has density f, then:
f(x) = \lim_{h\to0}\frac{1}{2h}P(x-h<X<x+h)
Thus, the naive estimator is written as:
\hat f(x) = \frac{[number of X_i^{‘}s in (x-h, x+h)]}{2hn}
We define a weight function as follows:
\omega(x) =
\frac{1}{2} & if & -1<x<1 \
0 & if & otherwise
Then f(x) becomes
\hat{f}(x) = \frac{1}{nh}\sum_{i=1}^{n}\omega(\frac{x-x_i)}{h})

  • $\hat f$is not continuous but has jumps at the points Xi±h and has zero derivative everywhere else.
  • The following example shows the “stepwise” nature of the esitmate.

Kernel Estimator



  • “Density estimation for statistics and data analysis”, B.W. Silverman, 1998, London: Chapman & Hall/CRC.


Pixel-based Figure-Ground Segmentation

Challenges in Scene Modeling

Illumination changes:

  • Gradual change in illumination as might occur in outdoor scenes due to the change in the relative location of the sun during the day.
  • Sudden change in illumination as might occur in an indoor environment by switching the lights on or off, or in an outdoor environment, e.g. a change between cloudy and sunny conditions.
  • Shadows cast on the background by objects in the background itself (e.g., buildings and trees) or by moving foreground objects.

Motion changes:

  • Global image motion due to small camera displacements. Small camera displacements are common in outdoor situations due to wind load or other sources of motion, which causes global motion in the images.
  • Motion in parts of the background. For example, tree branches moving with the wind, or rippling water.

Structural changes:

  • These are changes introduced to the background, including any change in the geometry or the appearance of the background of the scene introduced by targets. Such changes typically occur when something relatively permanent is introduced into the scene background (BK object removed or parking car.

Parametric Background Models

实验中利用高斯混合模型(Gaussian mixture model, GMM)方法来训练背景模型。GMM方法假设图像中的每一个像素点的强度都服从混合高斯分布(MoG)并对其建模,然后使用on-line方法来逐步更新背景模型。

对于一个视频序列,令$X_t$表示某个像素点$(x_0,y_0)$在时间t时的强度值。那么在时间t时,像素点$(x_0,y_0 )$的历史信息为:$X_1,… ,X_t={V(x_0,y_0,i):i≤t}$ 。这个历史信息可以用k个高斯分布的混合来建模:
P(X_t )=∑{i=1}^{K}ω(i,t) N(X_t ┤| μ(i,t),Σ(i,t))
其中,$N(X_t | μ{it},Σ{i,t})= \frac {1}{(2π)^{D/2}}\frac{1}{|Σ_{i,t}| ^{1/2}} exp⁡(-\frac{1}{2}(X_t-μ{i,t} )^T Σ{i,t}^{-1} (X_t-μ_{i,t}))$

在t=0时,我们利用上述MoG公式对图像中的每一个像素点进行建模,并对k个权重$ω_{i,t}$和k个高斯分布$N(X_t | μ{it},Σ{i,t})$进行相同的初始化。在t=1,2, … ,T时,我们可以利用新检测到的像素点$X_t$对权重和高斯分布的参数进行更新。

权重$ω{i,t}$更新公式为:$ω{i,t}=(1- α) ω_{i,t-1}+αM(i,t)$。其中当新像素点X_t服从第i个高斯分布时,M(i,t)=1,否则M(i,t)=0。

μt=(1-p) μ(t-1)+px_t

σt^2=(1-p) σ(t-1)^2+p(x_t-μt )^T (x_t-u_t)

当利用训练集更新好高斯分布的参数后,对每个像素点的k个高斯分布按照$ωj/σ_j^2$降序排序,并选择前B个高斯分布作为对应像素点的背景模型。其中$B=arg min_b⁡(∑_{j=1}^bω_j>T)$,T为全局阈值。对新像素点$X_t$,如果与B个高斯分布中的任何一个之间的标准差超过2.5,即被检测为前景。


Nonparametric Background Models

Moving Shadow Suppression


  • Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: Principles and practice of background maintenance. In: IEEE International Conference on Computer Vision (1999)
  • Wern, C.R., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: Real-time tracking of human body. IEEE Trans. Pattern Anal. Mach. Intell. (1997)
  • Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (1999)
  • Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and foreground modeling using non-parametric kernel density estimation for visual surveillance. Proc. IEEE 90(7), 1151–1163 (2002)

Mean Shift

Feature Space Analysis

Mean Shift Approach

Proof for Mean Shift Convergence

Pros and Cons

The Bandwidth Selection







Mean Shift运动跟踪:



  • Mean-Shift算法
  • [综] meanshift算法
  • Dorin Comaniciu, Peter Meer: Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5): 603-619 (2002).
  • Dorin Comaniciu, Peter Meer: Distribution Free Decomposition of Multivariate Data. Pattern Anal. Appl. 2(1): 22-30 (1999).

Visual Tracking

Kalman Filter


  • Kalman filtering is inadequate because it is based on the unimodal Gaussian distribution assumption, and it can’t represent simultaneous alternative hypotheses.
  • It works relatively poorly in clutter which causes the density to be multi-modal and therefore non- Gaussian.

Kalman filter is based on the single Gauss model, and different components have different effects on the Gauss distribution, as follows:

  • The deterministic component causes the density function to drift bodily.
  • The random component of the dynamical model leads to spreading—increasing uncertainty.
  • The effect of an external observation is to superimpose a reactive effect on the diffusion.


Particle Filter



At the top of the diagram, the output from time-step t -1 is the weighted sample-set. The aim is to maintain, at successive time-steps, sample sets of fixed size N.

  • The first operation is to sample N times from the set , choosing a given element with probability. Some elements, especially those with high weights, may be chosen several times, leading to identical copies of elements in the new set. Others with relatively low weights may not be chosen at all.
  • Each element chosen from the new set is now subjected to the predictive steps.First, an element undergoes drift and, since this is deterministic, identical elements in the new set undergo the same drift.
  • The second predictive step, diffusion, is random and identical elements now split because each undergoes its own independent motion step. At this stage, the sample set for the new time-step has been generated but, as yet, without its weights;
  • Finally, the observation step is applied, generating weights from the observation density.




Color-based Particle Filter

Color histograms have many advantages for tracking non-rigid objects as they are robust to partial occlusion, are rotation and scale invariant and are calculated efficiently.

A target is tracked with a particle filter by comparing its histogram with the histograms of the sample positions using the Bhattacharyya distance.

Bhattacharyya distance:在统计学中,Bhattacharyya距离(以下称巴氏距离)测量的是两个离散或连续概率分布的相似性。计算方式和Bhattacharyya系数关系很密切。



Kernel-based Particle Filter

A PF does not perform well when the dynamic system has a very small system noise or if the observation noise has very small variance. In these cases, the particle set quickly collapses to one single point in the state space.

The standard PF often fails to produce a particle set that captures the “irregular” motion, leading to gradually drifting estimates and ultimate loss of target.

KPF estimates the gradient of the kernel density and moves particles toward the modes of the posterior, leading to a more effective allocation of particles.

The gradient estimation and particle allocation is implemented by the mean shift algorithm.

A Boosted Particle Filter

The problem of tracking a varying number of non- rigid objects has two major difficulties:

  • First, the observation models and target distributions can be highly non-linear and non- Gaussian.
  • Second, the presence of a large, varying number of objects creates complex interactions with overlap and ambiguities.

    Mixture particle filters and Adaboost:

An effective way is to combine mixture particle filters and Adaboost. The crucial issues in mixture particle filters are the choice of the proposal distribution and the treatment of objects leaving and entering the scene.

The mixture particle filter is ideally suited to multi-target tracking as it assigns a mixture component to each player. The proposal distribution can be constructed by using a mixture model that incorporates information from the dynamic models of each player and the detection hypotheses generated by Adaboost.


  • Most multi-target tracking assumed a fixed number of objects.
  • BraMBLe has an automatic object detection system that relies on modeling a fixed background.
  • The authors will relax the assumption of a fixed background where the background changes.
  • Particle filters may perform poorly when the posterior is multimodal for multiple targets. Vermaak et al introduce a mixture particle filter (MPF), where each component is modelled with an individual particle filter. BPF is based on MPF.
  • The authors adopt a multi-color observation model based on Hue-Saturation-Value (HSV) color histograms.

The boosted particle filter introduces two important extensions of the MPF:

  • First, it uses Adaboost to construct the proposal distribution. It incorporates the recent observations in proposal distributions (through the Adaboost detections), and outperforms naive transition prior proposals considerably.
  • Second, Adaboost provides a mechanism for obtaining and maintaining the mixture representation. It allows us to detect objects leaving and entering the scene efficiently.


  • M. Isard and A. Blake. Condensation–conditional density propagation for visual tracking. Int. J. Computer Vision, 29(1):5– 28, 1998.
  • S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for on-line non-linear/non-Gaussian Bayesian tracking,” IEEE Transactions on Signal Processing, vol. 50, pp. 174–188, Feb. 2002.
  • K. Nummiaroa, E. Koller-Meierb, L. V. Gool, “An adaptive color- based particle filter”, Image and Vision Computing 21 (2003) 99– 110.
  • C.Chang, and R. Ansari, “Kernel Particle Filter for Visual Tracking”, IEEE SIGNAL PROCESSING LETTERS, VOL. 12, NO. 3, pp242-245, 2005.
  • K. Okuma, et al., “A Boosted Particle Filter: Multitarget Detection and Tracking”, ECCV 2004 (2004), pp. 28-39.
  • 基于粒子滤波器的目标跟踪算法及实现

Multiple Cues for Tracking

Ensemble Tracking

Face Detection



Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now