Mobile manipulation robots have great potential for roles in support of rescuers on disaster-response missions. Robots can operate in places too dangerous for humans and therefore can assist in accomplishing hazardous tasks while their human operators work at a safe distance. We developed a disaster-response system that consists of the highly flexible Centauro robot and suitable control interfaces, including an immersive telepresence suit and support-operator controls offering different levels of autonomy.

This article presents a computationally efficient approach to the triangulation of 3D points from their projections in two views. The homogenous coordinates of a 3D point is given as a multi-linear mapping on its homogeneous image coordinates, a computation of low computational complexity. The multi-linear mapping is a tensor, and an element of a projective space, that can be computed directly from the camera matrices and some parameters. These parameters imply that the tensor is not unique: for a given camera pair the subspace K of triangulation tensors is six-dimensional. The triangulation tensor is 3D projective covariant and satisfies a set of internal constraints. Reconstruction of 3D points using the proposed tensor is studied for the non-ideal case, when the image coordinates are perturbed by noise and the epipolar constraint exactly is not satisfied exactly. A particular tensor of K is then the optimal choice for a simple reduction of 3D errors, and we present a computationally efficient approach for determining this tensor. This approach implies that normalizing image coordinate transformations are important for obtaining small 3D errors.

In addition to computing the tensor from the cameras, we also investigate how it can be further optimized relative to error measures in the 3D and 2D spaces. This optimization is evaluated for sets of real 3D + 2D + 2D data by comparing the reconstruction to some of the triangulation methods found in the literature, in particular the so-called optimal method that minimizes 2D L₂ errors. The general conclusion is that, depending on the choice of error measure and the optimization implementation, it is possible to find a tensor that produces smaller 3D errors (both L₁ and L₂) but slightly larger 2D errors than the optimal method does. Alternatively, we may find a tensor that gives approximately comparable results to the optimal method in terms of both 3D and 2D errors. This means that the proposed tensor based method of triangulation is both computationally efficient and can be calibrated to produce small reconstruction or reprojection errors for a given data set.

This paper describes a method for vision-based unmanned aerial vehicle (UAV) motion estimation from multiple planar homographies. The paper also describes the determination of the relative displacement between different UAVs employing techniques for blob feature extraction and matching. It then presents and shows experimental results of the application of the proposed technique to multi-UAV detection of forest fires.  

This Chapter presents a vision-based method for unmanned aerial vehicle (UAV) motion estimation that uses as input an image motion field obtained from matches of point-like features. The Chapter enhances visionbased techniques developed for single UAV localization and demonstrates how they can be modified to deal with the problem of multi-UAV relative position estimation. The proposed approach is built upon the assumption that if different UAVs identify, using their cameras, common objects in a scene, the relative pose displacement between the UAVs can be computed from these correspondences under certain assumptions. However, although point-like features are suitable for local UAV motion estimation, finding matches between images collected using different cameras is a difficult task that may be overcome using blob features. Results justify the proposed approach.

We present Lambda Twist; a novel P3P solver which is accurate, fast and robust. Current state-of-the-art P3P solvers find all roots to a quartic and discard geometrically invalid and duplicate solutions in a post-processing step. Instead of solving a quartic, the proposed P3P solver exploits the underlying elliptic equations which can be solved by a fast and numerically accurate diagonalization. This diagonalization requires a single real root of a cubic which is then used to find the, up to four, P3P solutions. Unlike the direct quartic solvers our method never computes geometrically invalid or duplicate solutions.

Extensive evaluation on synthetic data shows that the new solver has better numerical accuracy and is faster compared to the state-of-the-art P3P implementations. Implementation and benchmark are available on github.

An open issue in multiple view geometry and structure from motion, applied to real life scenarios, is the sparsity of the matched key-points and of the reconstructed point cloud. We present an approach that can significantly improve the density of measured displacement vectors in a sparse matching or tracking setting, exploiting the partial information of the motion field provided by linear oriented image patches (edgels). Our approach assumes that the epipolar geometry of an image pair already has been computed, either in an earlier feature-based matching step, or by a robustified differential tracker. We exploit key-points of a lower order, edgels, which cannot provide a unique 2D matching, but can be employed if a constraint on the motion is already given. We present a method to extract edgels, which can be effectively tracked given a known camera motion scenario, and show how a constrained version of the Lucas-Kanade tracking procedure can efficiently exploit epipolar geometry to reduce the classical KLT optimization to a 1D search problem. The potential of the proposed methods is shown by experiments performed on real driving sequences.

We introduce a simple and efficient procedure for the segmentation of rigidly moving objects, imaged under an affine camera model. For this purpose we revisit the theory of "linear combination of views" (LCV), proposed by Ullman and Basri [20], which states that the set of 2d views of an object undergoing 3d rigid transformations, is embedded in a low-dimensional linear subspace that is spanned by a small number of basis views. Our work shows, that one may use this theory for motion segmentation, and cluster the trajectories of 3d objects using only two 2d basis views. We therefore propose a practical motion segmentation method, built around LCV, that is very simple to implement and use, and in addition is very fast, meaning it is well suited for real-time SfM and tracking applications. We have experimented on real image sequences, where we show good segmentation results, comparable to the state-of-the-art in literature. If we also consider computational complexity, our proposed method is one of the best performers in combined speed and accuracy. © 2011. The copyright of this document resides with its authors.

We propose a method for segmenting an arbitrary number of moving objects using the geometry of 6 points in 2D images to infer motion consistency. This geometry allows us to determine whether or not observations of 6 points over several frames are consistent with a rigid 3D motion. The matching between observations of the 6 points and an estimated model of their configuration in 3D space, is quantified in terms of a geometric error derived from distances between the points and 6 corresponding lines in the image. This leads to a simple motion inconsistency score, based on the geometric errors of 6points that in the ideal case should be zero when the motion of the points can be explained by a rigid 3D motion. Initial point clusters are determined in the spatial domain and merged in motion trajectory domain based on this score. Each point is then assigned to the cluster, which gives the lowest score.Our algorithm has been tested with real image sequences from the Hopkins155 database with very good results, competing withthe state of the art methods, particularly for degenerate motion sequences. In contrast to the motion segmentation methods basedon multi-body factorization, that assume an affine camera model, the proposed method allows the mapping from 3D space to the 2D image to be fully projective.

The paper describes a minimal set of 18 parameters that can representany trifocal tensor consistent with the internal constraints.  9parameters describe three orthogonal matrices and 9 parameters describe10 elements of a sparse tensor T' with 17 elements in well-defined positions equal to zero.  Any valid trifocal tensor isthen given as some specific T' transformed by the orthogonalmatrices in the respective image domain.  The paper also describes asimple approach for estimating the three orthogonal matrices in thecase of a general 3 x 3 x 3 tensor, i.e., when the internalconstraints are not satisfied.  This can be used to accomplish a leastsquares approximation of a general tensor to a tensor that satisfies the internal constraints.  This type of constraint enforcement, inturn, can be used to obtain an improved estimate of the trifocal tensorbased on the normalized linear algorithm, with the constraintenforcement as a final step.  This makes the algorithm more similar tothe corresponding algorithm for estimation of the fundamental matrix. An experiment on synthetic data shows that the constraint enforcementof the trifocal tensor produces a significantly better result thanwithout enforcement, expressed by the positions of the epipoles, giventhat the constraint enforcement is made in normalized image coordinates.

Point-of-interest detection is a way of reducing the amount of data that needs to be processed in a certain application and is widely used in 2D image analysis. In 2D image analysis, point-of-interest detection is usually related to extraction of local descriptors for object recognition, classification, registration or pose estimation. In analysis of range data however, some local descriptors have been published in the last decade or so, but most of them do not mention any kind of point-of-interest detection. We here show how to use an extended Harris detector on range data and discuss variants of the Harris measure. All described variants of the Harris detector for 3D should also be usable in medical image analysis, but we focus on the range data case. We do present a performance evaluation of the described variants of the Harris detector on range data.

A single-view matching constraint is described which represents a necessary condition which 6 points in an image must satisfy if they are the images of 6 known 3D points under an arbitrary projective transformation. Similar to the well-known matching constrains for two or more view, represented by fundamental matrices or trifocal tensors, single-view matching constrains are represented by tensors and when multiplied with the homogeneous image coordinates the result vanishes when the condition is satisfied. More precisely, they are represented by 6-th order tensors on ℝ³ which can be computed in a simple manner from the camera projection matrix and the 6 3D points. The single-view matching constraints can be used for finding correspondences between detected 2D feature points and known 3D points, e.g., on an object, which are observed from arbitrary views. Consequently, this type of constraint can be said to be a representation of 3D shape (in the form of a point set) which is invariant to projective transformations when projected onto a 2D image.

This paper describes a novel compact representation of local features called the tensor doublet. The representation generates a four dimensional feature vector which is significantly less complex than other approaches, such as Lowe's 128 dimensional feature vector. Despite its low dimensionality, we demonstrate here that the tensor doublet can be used for pose estimation, where the system is trained for an object and evaluated on images with cluttered background and occlusion.

 The paper describes a method for extracting point features from an image, corresponding to corners and crossings of lines. The method is based on a fourth order tensor representation which can describe the parameters of a local pair of line segments. By considering the rank of the tensor, it is possible to find points which correspond to corners, crossings or junctions. These points can then be further analyzed to provide detailed information about the configuration of the segments. The proposed method is intended for features which can be used for estimation of position and pose of 3D objects, e.g., for the purpose of grasping.

 This paper presents a novel representation for 3D shapes in terms of planar surface patches and their boundaries. The representation is based on a tensor formalism similar to the usual orientation tensor but extends this concept by using projective spaces and a fourth order tensor, even though the practical computations can be made in normal matrix algebra. This paper also discusses the possibility of estimating the proposed representation from motion field which are generated by a calibrated camera moving in the scene. One method based on 3D spatio-temporal orientation tensors is presented and results from this method are included.

This paper presents and overview of the basic and applied research carried out by the Computer Vision Laboratory, Linköping University, in the WITAS UAV Project. This work includes customizing and redesigning vision methods to fit the particular needs and restrictions imposed by the UAV platform, e.g., for low-level vision, motion estimation, navigation, and tracking. It also includes a new learning structure for association of perception-action activations, and a runtime system for implementation and execution of vision algorithms. The paper contains also a brief introduction to the WITAS UAV Project.

One important problem within the WITAS project is detection of moving objects in aerial images. This paper presents an original method to estimate the displacement between two frames, based on multiscale local polynomial expansions of the images. When the displacement field has been computed, a plane + parallax approach is used to separate moving objects from the camera egomotion.

The WITAS Unmanned Aerial Vehicle Project is a long term basic research project located at Linköping University (LIU), Sweden. The project is multi-disciplinary in nature and involves cooperation with different departments at LIU, and a number of other universities in Europe, the USA, and South America. In addition to academic cooperation, the project involves collaboration with a number of private companies supplying products and expertise related to simulation tools and models, and the hardware and sensory platforms used for actual flight experimentation with the UAV. Currently, the project is in its second phase with an intended duration from 2000-2003.

This paper will begin with a brief overview of the project, but will focus primarily on the computer vision related issues associated with interpreting the operational environment which consists of traffic and road networks and vehicular patterns associated with these networks.

WITAS will be engaged in goal-directed basic research in the area of intelligent autonomous vehicles and other autonomous systems. In this paper an overview of the project is given together with a presentation of our research interests in the project. The current status of our part in the project is also given.

The tensor representation has proven a successful tool as a mean to describe local multi-dimensional orientation. In this respect, the tensor representation is a map from the local orientation to a second order tensor. This paper investigates how variations of the orientation are mapped to variation of the tensor, thereby giving an explicit equivariance relation. The results may be used in order to design tensor based algorithms for extraction of image features defined in terms of local variations of the orientation, \eg multi-dimensional curvature or circular symmetries. It is assumed that the variation of the local orientation can be described in terms of an orthogonal transformation group. Under this assumption a corresponding orthogonal transformation group, acting on the tensor, is constructed. Several correspondences between the two groups are demonstrated.

This thesis presents a signal representation in terms of operators. The signal is assumed to be an element of a vector space and subject to transformations of operators. The operators form continuous groups, so-called Lie groups. The representation can be used for signals in general, in particular if spatial relations are undefinied and it does not require a basis of the signal space to be useful.

Special attention is given to orthogonal operator groups which are generated by anti-Hermitian operators by means of the exponential mapping. It is shown that the eigensystem of the group generator is strongly related to properties of the corresponding operator group. For one-parameter orthogonal operator groups, a phase concept is introduced. This phase can for instance be used to distinguish between spatially even and odd signals and, therefore, corresponds to the usual phase for multi-dimensional signals.

Given one operator group that represents the variation of the signal and one operator group that represents the variation of a corresponding feature descriptor, an equivariant mapping maps the signal to the descriptor such that the two operator groups correspond. Suficient conditions are derived for a general mapping to be equivariant with respect to a pair of operator groups. These conditions are expressed in terms of the generators of the two operator groups. As a special case, second order homo-geneous mappings are considered, and examples of how second order mappings can be used to obtain different types of feature descriptors are presented, in particular for operator groups that are homomorphic to rotations in two and three dimensions, respectively. A generalization of directed quadrature lters is made. All feature extraction algorithms that are presented are discussed in terms of phase invariance.

Simple procedures that estimate group generators which correspond to one-parameter groups are derived and tested on an example. The resulting generator is evaluated by using its eigensystem in implementations of two feature extraction algorithms. It is shown that the resulting feature descriptor has good accuracy with respect to the corresponding feature value, even in the presence of signal noise.

This book contains material for an introductory course on homogeneous representations for geometry in 2 and 3 dimensions, camera projections, representations of 3D rotations, epipolar geometry, and estimation of various type of geometric objects. Based on these results, a set of applications are presented.  It also contains a toolbox of general results that are useful for the presented material.  The book is intended for undergraduate studies at advanced level in master programs, or in PhD-courses at introductory level.

This document is an addendum to the main text in A local geometry-based descriptor for 3D data applied to object pose estimation by Fredrik Viksten and Klas Nordberg. This addendum gives proofs for propositions stated in the main document. This addendum also details how to extract information from the fourth order tensor refered to as S₂₂ in the main document.

This report describes a fourth order tensor defined on projective spaces which can be used for the representation of medium-level features, e.g., one or more oriented segments. The tensor has one part which describes what type of local structures are present in a region, and one part which describes where they are located. This information can be used, e.g., to represent multiple orientations, corners, and line-endings. The tensor can be defined for arbitrary signal dimension, but the presentation focuses on the properties of the fourth order tensor for the case of 2D and 3D image data. A method for estimating the proposed tensor representation by means of simple computations directly from the structure tensor is presented. Given a simple matrix representation of the tensor, it can be shown that there is a direct correspondence between the number of oriented segments and the rank of the matrix provided that the number of segments is three or less. The \publication also presents techniques for extracting information about the oriented segments which the tensor represent. Finally, it shown that a small set of coefficients can be computed from the proposed tensor which are invariant to changes of the coordinate system.

This report defines the rank complement of a diagonalizable matrix (i.e. a matrix which can be brought to a diagonal form by means of a change of basis) as the interchange of the range and the null space. Given a diagonalizable matrix A there is in general no unique matrix Ac which has a range equal to the null space of A and a null space equal to the range of A, only matrices of full rank have a unique rank complement; the zero matrix. Consequently, the rank complement operation is not a distinct operation, but rather a characterization of any operation which makes an interchange of the range and the null space. One particular rank complement operation is introduced here, which eventually leads to an implementation of rank complement operations in terms of polynomials in A. The main result is that for each possible rank r of A there is a polynomial in A which evaluates to a matrix Ac which is a rank complement of A. The report provides explicit expressions for matrix polynomials which compute a rank complement of a symmetric matrix. These results are then generalized to the case of diagonalizable matrices. Finally, a Matlab function is described that implements a rank complement operation based on the results derived.

A robust, fast and general method for estimation of object properties is proposed. It is based on a representation of theses properties in terms of channels. Each channel represents a particular value of a property, resembling the activity of biological neurons. Furthermore, each processing unit, corresponding to an artificial neuron, is a linear perceptron which operates on outer products of input data. This implies a more complex space of invariances than in the case of first order characteristic without abandoning linear theory. In general, the specific function of each processing unit has to to be learned and a fast and simple learning rule is presented. The channel representation, the processing structure and the learning rule has been tested on stereo image data showing a cube with various 3D positions and orientations. The system was able to learn a channel representation for the horizontal position, the depth, and the orientation of the cube, each property invariant to the other two.

The topic of this report is signal representation in the context of hierarchical image processing. An overview of hierarchical processing systems is included as well as a presentation of various approaches to signal representation, feature representation and feature extraction. It is claimed that image hierarchies based on feature extraction, so called feature hierarchies, demand a signal representation other than the standard spatial or linear representation used today. A new representation, the operator representation is developed. It is based on an interpretation of features in terms of signal transformations. This representation has no references to any spatial ordering of the signal element and also gives an explicit representation of signal features. Using the operator representation, a generalization of the standard phase concept in image processing is introduced. Based on the operator representation, two algorithms for extraction of feature values are presented. Both have the capability of generating phase invariant feature descriptors. It is claimed that the operator representation in conjunction with some appropriate feature extraction algorithm is well suited as a general framework for defining multi level feature hierarchies. The report contains an appendical chapter containing the mathematical details necessary to comprehend the presentation.