There are various sources of correlation affecting the state estimation and fusion process in a distributed architecture. Failing to consider the cross-correlation leads to overconfident results and even divergence of the fusion algorithm [ 9 , 24 ]. Nonetheless, due to double counting and the unavailability of internal parameters, it is very difficult to exactly estimate the cross-correlation in a vast distributed sensor network. In some applications, such as in map building, weather forecasting etc.

### Image and Sensor Signal Processing (Volume One), 2nd Edition

Maintaining and taking care of cross-correlation is expensive, and it scales quadratically with the number of updates [ 23 ]. Therefore, various suboptimal strategies have been devised to provide a fused solution from multiple data sources without the need of an actual cross-correlation. The analysis of fusion under unknown correlation is carried out according to the categorization of Figure 6. Double counting happens when the same data follows different or cyclic paths to reach the fusion node [ 9 , 13 ].

An effective way to avoid the data incest issue is to keep the record of estimate updates. References [ 27 , 28 ] propose a method to remove the correlation by explicitly eliminating double counting. The idea is to resolve remote measurements from state estimates of other sensor nodes, store them and use them to update its own state estimate. This way the double counted data is removed before the data is fused. This method assumes a specific network topology to avoid the correlation due to double counting.

In References [ 76 , 77 ], a more general solution using graph theoretic algorithms is proposed, which is viable for arbitrary network topologies with variable time delays. However, this is neither scalable nor practical for a large network of sensors [ 78 ]. Another approach for decorrelation is measurement reconstruction [ 25 , 26 , 79 ], where the system noise is artificially adjusted by reconstructing the measurements so that correlation between the sequence of measurements is removed.

The remote measurements are reconstructed at the fusion node based on the local sensor estimates.

This method is further developed for tracking in clutter [ 80 ], Out-of-sequence filtering [ 81 ] and non-Gaussian distributions with Gaussian mixture models [ 82 ]. However, internal information like Kalman gain, association weights and sensor model information etc. The decorrelation methods result in a compromised fusion performance due to their dependency on empirical knowledge and special analysis for a particular real system. Furthermore, with an increase in the number of sensors, these methods become highly inefficient and impractical. Although an exact cross-correlation between local estimates in a distributed architecture is difficult to obtain, the properties of the joint covariance matrix put some restriction on the possible cross-correlation.

Furthermore, certain applications may provide prior knowledge and constraints on the degree of correlation such that we may infer whether the local estimates are strongly or weakly correlated. In fact, the estimates provided by multiple sensors are neither independent nor exactly dependent, meaning that the cross-correlation is not completely unknown. Thus, the information and knowledge regarding unknown cross-correlations can be exploited to improve the accuracy of the fused solution under unknown correlation.

The joint covariance matrix P is positive semidefinite if and only if there is a contraction matrix C such that [ 83 ],. In the case of scalar-valued estimates, the cross-correlation can be computed as,. Based on the correlation model 17 an analytic analysis of the BC formula is carried out to give an exact solution for fusion under unknown correlation [ 29 ]. A closed-form equation for scalar-valued fusion and an approximate solution for vector valued fusion based on a uniformly distributed correlation coefficient is proposed in Reference [ 30 ].

Based on bounded correlations, a general method was proposed as the Bounded Covariance Inflation BCInf [ 85 ] with upper and lower bounds on cross-correlation. The method exploits the available information regarding known independence in the sensor network. An approximate correlation model is adopted for two data sources in high dimensions as [ 32 ],. It is illustrated in Reference [ 32 ] that the proposed model ensures the positive semi definiteness of the joint covariance matrix P and agrees with the Canonical Correlation Analysis of multivariate correlation [ 87 ].

Based on the correlation model 18 , a track association and fusion is carried out in the Maximum Likelihood sense in Reference [ 31 ]. In Reference [ 29 ], the correlation model 18 was used in BC formula to analytically estimate the maximum bounds of the unknown correlation in track-to-track fusion. The multisensor estimation problem with the assumption of norm-bounded cross-correlation is studied in [ 88 ], where the worst-case fused MSE is minimized for all feasible cross-covariances.

To utilize some prior information of the cross-covariance, a formulation named allowance of cross-covariance is proposed in Reference [ 89 ]. Based on the proposed model an optimal fusion method in the sense of minimizing the worst-case fused MSE by semidefinite programming SDP is derived. Yet, the number of correlation coefficients increases with the number of sensors and the closed-form solution for even scalar-valued estimates becomes difficult.

For instance, in the case of three data sources in R 1 the joint covariance matrix can be written as,. Three correlation coefficients can now be noted to represent the dependency among the three data sources and optimizing any function of P in terms of correlation coefficients becomes a daunting task. In general, it is difficult to interpret cross-correlation for more than two data sources in high dimensions. It should also be noted that the general correlation analysis techniques like canonical correlation analysis CCA [ 87 ] and multivariate linear regression MLA [ 90 ] have limited use in connection with the cross-correlation among multiple data sources.

Since these techniques assess the correlation property when given a vast set of data points. The joint covariance matrix of the multiple data sources, on the other hand, is a block covariance matrix that represents the relationships among the individual states of the sensor and among different sensors. The two data sources are assumed to be correlated with cross covariance matrix P From 7 and 8 , we can observe that the underlying fused covariance and mean of the two data sources is dependent on the unknown cross-covariance P The possible cross covariances between the data sources are bounded [ 14 , 33 , 34 , 35 ], which in turn, restricts the possible outcomes of the fused covariance to a bounded set.

As shown in Figure 7 , for different choices of cross-covariance P 12 , the fused covariance P f will lie inside the intersection of the individual data sources. The Ellipsoidal Methods EM attempt to provide a fused estimate by approximating the intersection region of the individual ellipsoids. The EM are analyzed one by one here. The gray area represents all possibilities of a fused covariance. Covariance Intersection Method CI [ 35 ] was proposed by Julier and Uhlman for fusion under unknown correlation in a decentralized network. The algorithm is given by [ 14 , 35 ],. The CI method obtains a consistent fused result without computing the cross-correlation.

Since, for any possible cross-correlation the fused result lies inside the intersection region of the individual ellipsoids, CI method provides a consistent solution by enclosing the region of the intersection of individual ellipsoids, as depicted in Figure 8. Two estimates at the origin, i.

## CEVA- Leading Licensor of Signal Processing IP

Since its inception, the CI method has received much attention, and some improvements have been made to enhance the capabilities of the methodology itself while others have focused on its applications in various fields [ 2 , 33 , 34 , 49 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , , , , , , , ]. For example, the CI method is generalized as a split CI method [ ] to fuse independent as well as dependent information with an unknown degree of correlation. In Reference [ 97 ], the CI method is examined with a Chernoff fusion rule, and it is noted that the CI method is suitable for fusing any distributions, and is not limited to Gaussian density function.

Meanwhile, CI is used for a non-linear estimation in [ ], where the distributions are represented as pseudo-Gaussian densities, while a closed form optimization of CI for low-dimensional matrices was proposed in Reference [ ]. In References [ , ], the CI method is studied for track-to-track fusion with memory and without memory. Furthermore, a comparative analysis of CI with different optimal fusion rules is presented in Reference [ 98 ]. The CI method is applied in many applications, namely, localization [ , , ], target tracking [ , ], simultaneous localization and mapping SLAM [ 1 , 2 ], image integration [ 99 ], NASA MARS rover [ ] and spacecraft state estimation [ , ].

Although state-of-the-art CI method has its own disadvantages including: 1 requirement of a nonlinear iterative optimization and 2 it overestimates the intersection region of individual covariances, resulting in a degradation of the estimation performance. For the sake of computational efficiency, approaches to directly compute the weights based on the determinants of individual covariances have been proposed [ 91 , 92 ] at the expense of further performance degradation without taking the relative orientation of individual covariances into account.

Different optimization criteria for weight computation based on information theory [ 93 , 94 ] as well as set theory [ 95 ] have been proposed for computational efficiency. To avoid the computational cost of the CI method for more than two sensors, a sequential covariance intersection SCI [ 96 ] is presented. The SCI method reduces the multidimensional non-linear optimization problem of CI into many one-dimensional non-linear functions by sequentially applying the CI method of two sensors to n sensors. A proof that CI method results in a minimum consistent covariance bound for two sensors is given in Reference [ ].

## Multi-sensor image fusion and its applications

Recently an Inverse Covariance Intersection ICI [ ] method based on the common information of two sensors was proposed, which results in a tighter estimate than with the CI method. Contrary to the CI method which yields a minimum overestimation of the intersection region of individual covariances, the Maximum Ellipsoidal Methods ME , that is, LE [ 36 ], IEA [ 37 , 38 ] and EI [ 39 ] sought a maximum ellipsoid inside the intersection region of individual covariance ellipsoids as shown in Figure 9.

Since the fused covariance for any possible choice of cross-correlation lies inside the intersection of individual ellipsoids, the ME methods attempt to obtain a maximum ellipsoid inside the region of the intersection. Although aiming for a common objective, the ME methods follow different approaches from each other, thus resulting in subtle differences in the computation of the fused mean and covariance. The ME methods are analyzed one by one below. To avoid an overestimation of the CI, the Largest Ellipsoid Method [ 36 ] provides the largest ellipsoid inside the intersection of two individual ellipsoids by manipulating their orientation.

The two covariances are transformed by a transformation matrix T r as,.

This scaling operation transform the ellipsoid P 1 into a sphere with all eigenvalues of P 1 s r being equal. Similarly, the second ellipsoid is transformed as,. The intersection of the two ellipsoids P 1 s r and P 2 s r in the transformed space is computed as,. The corresponding largest ellipsoid is transformed back to original space by an inverse transformation as,. The fused mean x L E of the two data sources is calculated using the simple convex equation of KF,.

inalacical.tk Although, the LE method for fused covariance results in the largest ellipsoid inside the intersection of the individual ellipsoids, the computation of the fused mean is incorrect. Because calculation of the fused mean is based on the independence assumption of KF and does not consider the cross-correlation, which may lead to inconsistent results. To ensure the consistency and optimality in multisensor data fusion, the fused covariance, as well as the correct calculation of the fused mean, is important.

To fill the gap in the LE Method, an Internal Ellipsoidal Approximation Method IEA [ 37 , 38 , ] was proposed which provides an internal approximation of the region of intersection of the individual ellipsoids. The fused mean and covariance of the algorithm are written as,.