The major goal of the COSPAL project is to develop an artificial cognitive system architecture, with the ability to autonomously extend its capabilities. Exploratory learning is one strategy that allows an extension of competences as provided by the environment of the system. Whereas classical learning methods aim at best for a parametric generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class, and to apply generalization on a conceptual level, resulting in new models. Incremental or online learning is a crucial requirement to perform exploratory learning. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning, and in this paper we focus on the organization of cognitive systems for efficient operation. Learning is used over the entire system. It is organized in the form of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail. We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user (teacher) and system is a major difference to classical robotics systems, where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems. We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module. We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

This paper presents a method that combines shadow detection and a 3D box model including shadow simulation, for estimation of size and position of vehicles. We define a similarity measure between a simulated image of a 3D box, including the box shadow, and a captured image that is classified into background/foreground/shadow. The similarity Measure is used in all optimization procedure to find the optimal box state. It is shown in a number of experiments and examples how the combination shadow detection/simulation improves the estimation compared to just using detection or simulation, especially when the shadow detection or the simulation is inaccurate. We also describe a tracking system that utilizes the estimated 3D boxes, including highlight detection, spatial window instead of a time based window for predicting heading, and refined box size estimates by weighting accumulated estimates depending oil view. Finally, we show example results.

This paper brings together a novel information representation model for use in signal processing and computer vision problems, with a particular algorithmic development of the Landweber iterative algorithm. The information representation model allows a representation of multiple values for a variable as well as an expression for confidence. Both properties are important for effective computation using multi-level models, where a choice between models will be implementable as part of the optimization process. It is shown that in this way the algorithm can deal with a class of high-dimensional, sparse, and constrained least-squares problems, which arise in various computer vision learning tasks, such as object recognition and object pose estimation. While the algorithm has been applied to the solution of such problems, it has so far been used heuristically. In this paper we describe the properties and some of the peculiarities of the channel representation and optimization, and put them on firm mathematical ground. We consider the optimization a convexly constrained weighted least-squares problem and propose for its solution a projected Landweber method which employs oblique projections onto the closed convex constraint set. We formulate the problem, present the algorithm and work out its convergence properties, including a rate-of-convergence result. The results are put in perspective with currently available projected Landweber methods. An application to supervised learning is described, and the method is evaluated in an experiment involving function approximation, as well as application to transient signals. © 2006 Elsevier Ltd. All rights reserved.

 A method for unrestricted recognition of 3-D objects has been developed. By unrestricted, we imply that the recognition shall be done independently of object position, scale, orientation and pose, against a structured background. It shall not assume any preceding segmentation and allow a reasonable degree of occlusion. The method uses a hierarchy of triplet feature invariants, which are at each level defined by a learning procedure. In the feed-back learning procedure, percepts are mapped upon system states corresponding to manipulation parameters of the object. The method uses a learning architecture employing channel information representation. The paper contains a discussion of how objects can be represented. A structure is proposed to deal with object and contextual properties in a transparent manner.

This paper deals with a new framework in analyzing the formal mathematical correspondence between quantum mechanics and time-frequency representations of a signal. It is also shown that joint time-frequency distributions have a close link with Heisenberg uncertainty relations if the observables are taken as fuzzy entities. This result contradicts the arguments of Cohen [IEEE Proc. 77(7):941 (1989)] regarding the time-frequency distributions and the uncertainty relation. It is postulated that these mechanisms will be of crucial importance in highly fragmented computation structures, such as neural networks, as they may exhibit a strong mutual interaction between data and operator.

A framework for computer-aided analysis of mammograms is described. General computer vision algorithms are combined with application specific procedures in a hierarchical fashion. The system is under development and is currently limited to detection of a few types of suspicious areas. The image features are extracted by using feature extraction methods where wavelet techniques are utilized. A low-pass pyramid representation of the image is convolved with a number of quadrature filters. The filter outputs are combined according to simple local Fourier domain models into parameters describing the local neighborhood with respect to the model. This produces estimates for each pixel describing local size, orientation, Fourier phase, and shape with confidence measures associated to each parameter. Tentative object descriptions are then extracted from the pixel-based features by application-specific procedures with knowledge of relevant structures in mammograms. The orientation, relative brightness and shape of the object are obtained by selection of the pixel feature estimates which best describe the object. The list of object descriptions is examined by procedures, where each procedure corresponds to a specific type of suspicious area, e.g. clusters of microcalcifications.

A new predictive coder, based on an estimation method which adapts to line and edge features in images, is described. Quantization of the prediction error is performed by a two-level adaptive scheme: an adaptive transform coder, and a threshold coding in both transform and spatial domains. Control information, which determines the behavior of the predictor, is quantized using a simple variable rate technique. The results are improved by pre- and post-filtering using a related noncausal form of the estimator. Acceptable images have been produced in this way at bit rates of less than 0.5 bit/pixel.

In a special radiographic process, ectomography, an image of a slice is produced by simple summation of a set of specially filtered component images, of which each represents one of at least 60 different projections of the object. After being digitized, they are stored, filtered, and summed in a computer. Images representing any slice of any thickness in the object may be produced from the same set of component images. All details within the slice are pictured correctly while details outside are almost completely eliminated.

Radiographic technology has advanced considerably during the last years with the advent of reconstruction techniques allowing visualization of slices through the body. In spite of the advantage of computed tomography compared to conventional radiographic methods, there are still some shortcomings with the method If a different section of the body is desired, another recording has to be made, the width of the dice reconstructed is fixed, and a full 1800 view angle is required.

The advent of new stains for chromosomes has increased the possibility of implementing useful automated chromosome analysis. The case with which chromosomes can now be recognized makes it possible to perform detailed statistical analysis of the chromosomes of an individual. This paper describes methods for assembling chromosome information from several cells in such a way that accidental variations due to preparation, etc. can be eliminated and an undistorted set of characteristics of the chromosome complement can be established. This set of characteristics can then be compared with various references, and statements can be made concerning the relationships between variations in the chromosome complement and genetic traits. These same methods can be employed in multiple-cell karyotyping to circumvent the classical problem of touching and overlapping chromosomes. The methods also allow one to achieve very reliable descriptions of the chromosome complement. The importance of appropriate descriptors of the chromosomes is illustrated.

Signal Processing for Computer Vision is a unique and thorough treatment of the signal processing aspects of filters and operators for low-level computer vision.

Computer vision has progressed considerably over recent years. From methods only applicable to simple images, it has developed to deal with increasingly complex scenes, volumes and time sequences. A substantial part of this book deals with the problem of designing models that can be used for several purposes within computer vision. These partial models have some general properties of invariance generation and generality in model generation.

Signal Processing for Computer Vision is the first book to give a unified treatment of representation and filtering of higher order data, such as vectors and tensors in multidimensional space. Included is a systematic organisation for the implementation of complex models in a hierarchical modular structure and novel material on adaptive filtering using tensor data representation.

Signal Processing for Computer Vision is intended for final year undergraduate and graduate students as well as engineers and researchers in the field of computer vision and image processing.

This volume is a post-event proceedings volume and contains selected papers based on the presentations given, and the lively discussions that ensued, during a seminar held in Dagstuhl Castle, Germany, in October 2003. Co-sponsored by ECVision, the cognitive vision network of excellence, it was organized to further strengthen cooperation between research groups from different countries working in the field of cognitive vision systems.

Most of the processing in vision today uses spatially invariant operations. This gives efficient and compact computing structures, with the conventional convenient separation between data and operations. This also goes well with conventional Cartesian representation of data. Currently, there is a trend towards context dependent processing in various forms. This implies that operations will no longer be spatially invariant, but vary over the image dependent upon the image content. There are many ways in which such a contextual control can be implemented. Mechanisms can be added for the modification of operator behavior within the conventional computing structure. This has been done e.g. for the implementation of adaptive filtering. In order to obtain sufficient flexibility and power in the computing structure, it is necessary to go further than that. To achieve sufficiently good adaptivity, it is necessary to ensure that sufficiently complex control strategies can be represented. It is becoming increasingly apparent that this can not be achieved through prescription or program specification of rules. The reason being that these rules will be dauntingly complex and can not be be dealt with in sufficient detail. At the same time that we require the implementation of a spatially variant processing, this implies the requirement for a spatially variant information representation. Otherwise a sufficiently effective and flexible contextual control can not be implemented. This paper outlines a new structure for effective space variant processing. It utilises a new type of localized information representation, which can be viewed as outputs from band pass filters such as wavelets. A unique and important feature is that convex regions can be built up from a single layer of associating nodes. The specification of operations is made through learning or action controlled association.

This chapter is not original, but presents methods for linear classification in the tradition of N. J. Nilsson as well as R. O. Duda and P. E. Hart. Part of the motivation for including this well-known material is to allow the vision structure to be brought to a logical conclusion in which feature properties are combined to form responses or class statements. Another motivation developed here is to display the similarity in structure between convolution operations and linear discriminant functions. This brings all operations for feature extraction and classification to the use of a common component, linear discriminants. This is also illustrated in the form of perceptrons, which allows a transition to the modern theory of neural networks.

This chapter givesan overview of important biological vision mechanisms. Although agreat deal is known about neural processing of visual information,most essential questions about biological vision remain as yetunanswered. Nonetheless, the knowledge available has already provideduseful guidance to the organization of effective machine visionsystems.

A framework for computer-aided analysis of mammograms is described. General computer vision algorithms are combined with application specific procedures in a hierarchical fashion. The system is under development and is currently limited to detection of a few types of suspicious areas. The image features are extracted by using feature extraction methods where wavelet techniques are utilized. A low-pass pyramid representation of the image is convolved with a number of quadrature filters. The filter outputs are combined according to simple local Fourier domain models into parameters describing the local neighborhood with respect to the model. This produces estimates for each pixel describing local size, orientation, Fourier phase, and shape with confidence measures associated to each parameter. Tentative object descriptions are then extracted from the pixel-based features by application-specific procedures with knowledge of relevant structures in mammograms. The orientation, relative brightness and shape of the object are obtained by selection of the pixel feature estimates which best describe the object. The list of object descriptions is examined by procedures, where each procedure corresponds to a specific type of suspicious area, e.g. clusters of microcalcifications.

This paper explores the possibility to use a single low-resolution FIR camera for detection of pedestrians in the near zone in front of a vehicle. A low resolution sensor reduces the cost of the system, as well as the amount of data that needs to be processed in each frame.

We present a system that makes use of hot-spots and image positions of a near constant bearing to detect potential pedestrians. These detections provide seeds for an energy minimization algorithm that fits a pedestrian model to the detection. Since false alarms are hard to tolerate, the pedestrian model is then tracked, and the distance-to-collision (DTC) is measured by integrating size change measurements at sub-pixel accuracy, and the car velocity. The system should only engage braking for detections on a collision course, with a reliably measured DTC.

Preliminary experiments on a number of recorded near collision sequences indicate that our method may be useful for ranges up to about 10m using an 80x60 sensor, and somewhat more using a 160x120 sensor. We also analyze the robustness of the evaluated algorithm with respect to dead pixels, a potential problem for low-resolution sensors.

This paper describes a novel compact representation of local features called the tensor doublet. The representation generates a four dimensional feature vector which is significantly less complex than other approaches, such as Lowe's 128 dimensional feature vector. Despite its low dimensionality, we demonstrate here that the tensor doublet can be used for pose estimation, where the system is trained for an object and evaluated on images with cluttered background and occlusion.

In this paper we address one of the standard problems of image processing and computer vision: The detection of points of interest (POI). We propose two new approaches for improving the detection results. First, we define an energy tensor which can be considered as a phase invariant extension of the structure tensor. Second, we use the channel representation for robustly clustering the POI information from the first step resulting in sub-pixel accuracy for the localisation of POI. We compare our method to several related approaches on a theoretical level and show a brief experimental comparison to the Harris detector.

Channel smoothing is an alternative to diffusion filtering for robust estimation of image features. Its main advantages are speed, stability with respect to parameter changes, and a simple implementation. However, channel smoothing becomes instable in certain situations, typically for elongated, periodic patterns like for instance fingerprints. As for the diffusion filtering an anisotropic extension is required in these cases. In this paper we introduce a new method for anisotropic channel smoothing which is comparable to coherence enhancing diffusion, but faster and easier to implement. Anisotropic channel smoothing implements an orientation adaptive non-linear filtering scheme as a special case of adaptive channel filtering. The smoothing algorithm is applied to several fingerprint images and the results are compared to those of coherence enhancing diffusion.

This article describes an essential step towards what is called a view centered representation of the low-level structure in an image. Instead of representing low-level structure (lines and edges) in one compact feature map, we will separate structural information into several feature maps, each signifying features at a characteristic phase, in a specific scale. By characteristic phase we mean the phases 0, pi, and +/-pi/2, corresponding to bright, and dark lines, and edges between different intensity levels, or colours. A lateral inhibition mechanism selects the strongest feature within each local region of scale represented. The scale representation is limited to maps one octave apart, but can be interpolated to provide a continous representation. The resultant image representation is sparse, and thus well suited for further processing, such as pattern detection.

Perceptual experiments indicate that corners and curvature are very important features in the process of recognition. This paper presents a new method to detect rotational symmetries, which describes complex curvature such as corners, circles, star, and spiral patterns. It works in two steps: 1) it extracts local orientation from a gray-scale or color image; and 2) it applies normalized convolution on the orientation image with rotational symmetry filters as basis functions. These symmetries can serve as feature points at a high abstraction level for use in hierarchical matching structures for 3D estimation, object recognition, image database retrieval, etc

There is no indication that it will ever be possible to find some simple trick that miraculously solves most problems in vision. It turns out that the processing system must be able to implement a model structure, the complexity of which is directly related to the structural complexity of the problem under consideration in the external world. It has become increasingly apparent that Vision cannot be treated in isolation from the response generation, because a very high degree of integration is required between different levels of percepts and corresponding response primitives. The response to be produced at a given instance is as much dependent upon the state of the system, as the percepts impinging upon the system. In addition, it has become apparent that many classical aspects of perception, such as geometry, probably do not belong to the percept domain of a Vision system, but to the response domain. This article will focus on what are considered crucial problems in Vision for robotics for the future, rather than on the classical solutions today. It will discuss hierarchical architectures for combination of percept and response primitives. It will discuss the concept of combined percept–response invariances as important structural elements for Vision. It will be maintained that learning is essential to obtain the necessary flexibility and adaptivity. In consequence, it will be argued that invariances for the purpose of Vision are not abstractly geometrical, but derived from the percept–response interaction with the environment. The issue of information representation becomes extremely important in distributed structures of the types foreseen, where uncertainty of information has to be stated for update of models and associated data. The question of object representation is central to the paper. Equivalence is established between the representations of response, geometry and time. Finally an integrated percept–response structure is proposed for flexible response control.

This paper introduces a signal processing strategy for depth segmentation and scene reconstruction that incorporates occlusion as a natural component. The work aims to maximize the use of connectivity in the temporal domain as much as possible under the condition that the scene is static and that the camera motion is known. An object behind the foreground is reconstructed using the fact that different parts of the object have been seen in different images in the sequence. One of the main ideas in this paper is the use of a spatio- temporal certainty volume c(x) with the same dimension as the input spatio- temporal volume s(x), and then use c(x) as a 'blackboard' for rejecting already segmented image structures. The segmentation starts with searching for image structures in the foreground, eliminate their occluding influence, and then proceed. Normalized convolution, which is a Weighted Least Mean Square technique for filtering data with varying spatial reliability, is used for all filtering. High spatial resolution near object borders is achieved and only neighboring structures with similar depth supports each other.

The tensor representation has proven a successful tool as a mean to describe local multi-dimensional orientation. In this respect, the tensor representation is a map from the local orientation to a second order tensor. This paper investigates how variations of the orientation are mapped to variation of the tensor, thereby giving an explicit equivariance relation. The results may be used in order to design tensor based algorithms for extraction of image features defined in terms of local variations of the orientation, \eg multi-dimensional curvature or circular symmetries. It is assumed that the variation of the local orientation can be described in terms of an orthogonal transformation group. Under this assumption a corresponding orthogonal transformation group, acting on the tensor, is constructed. Several correspondences between the two groups are demonstrated.

The related problems of enhancing and restoring noisy images have received a considerable amount of attention in recent years. Restoration methods have generally been based on minimum mean-squared error operations, such as Wiener filtering or recursive filtering. The rather vague title of enhancement has been given to a wide variety of more or less ad-hoc methods, such as median filtering, which have nonetheless been found useful. In mast cases, however, the aim is the same: an improvement of the subjective quality of the image.

One major goal of the COSPAL project is to develop an artificial cognitive system architecture with the capability of exploratory learning. Exploratory learning is a strategy that allows to apply generalization on a conceptual level, resulting in an extension of competences. Whereas classical learning methods aim at best possible generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class. Incremental or online learning is an inherent requirement to perform exploratory learning.

Exploratory learning requires new theoretic tools and new algorithms. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning and in this paper we focus on its algorithmic aspect. Learning is performed in terms of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail.

We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user (’teacher’) and system is a major difference to most existing systems where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems.

We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module.We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

This report brings together a novel approach to some computer vision problems and a particular algorithmic development of the Landweber iterative algorithm. The algorithm solves a class of high-dimensional, sparse, and constrained least-squares problems, which arise in various computer vision learning tasks, such as object recognition and object pose estimation. The algorithm has recently been applied to these problems, but it has been used rather heuristically. In this report we describe the method and put it on firm mathematical ground. We consider a convexly constrained weighted least-squares problem and propose for its solution a projected Landweber method which employs oblique projections onto the closed convex constraint set. We formulate the problem, present the algorithm and work out its convergence properties, including a rate-of-convergence result. The results are put in perspective of currently available projected Landweber methods. The application to supervised learning is described, and the method is evaluated in a function approximation experiment.

A new architecture for learning systems has been developed. A number of particular design features in combination result in a very high performance and excellent robustness. The architecture uses a monopolar channel information representation. The channel representation implies a partially overlapping mapping of signals into a higher-dimensional space, such that a flexible but continuous restructuring mapping can be made. The high-dimensional mapping introduces locality in the information representation, which is directly available in wavelets or filter outputs. Single level maps using this representation can produce closed decision regions, thereby eliminating the need for costly back-propagation. The monopolar property implies that data only utilizes one polarity, say positive values, in addition to zero, allowing zero to represent no information. This leads to an efficient sparse representation.

The processing mode of the architecture is association where the mapping of feature inputs onto desired state outputs is learned from a representative training set. The sparse monopolar representation together with locality, using individual learning rates, allows a fast optimization, as the system exhibits linear complexity. Mapping into multiple channels gives a strategy to use confidence statements in data, leading to a low sensitivity to noise in features. The result is an architecture allowing systems with a complexity of some hundred thousand features described by some hundred thousand samples to be trained in typically less than an hour. Experiments that demonstrate functionality and noise immunity are presented. The architecture has been applied to the design of hyper complex operations for view centered object recognition in robot vision.

This report introduces a signal processing strategy for depth segmentation and scene reconstruction that incorporates occlusion as a natural component. The work aims to maximize the use of connectivity in the temporal domain as much as possible under the condition that the scene is static and that the camera motion is known. An object behind the foreground is reconstructed using the fact that different parts of the object have been seen in different images in the sequence. One of the main ideas in the reported work is the use of a spatiotemporal certainty volume c(x) with the same dimension as the input spatiotemporal volume s(x), and then use c(x) as a 'blackboard' for rejecting already segmented image structures. The segmentation starts with searching for image structures in the foreground, eliminates their occluding influence, and then proceeds. Normalized convolution, which is a Weighted Least Mean Square technique for filtering data with varying spatial reliability, is used for all filtering. High spatial resolution near object borders is achieved and only neighboring structures with similar depth supports each other.

This paper presents an algorithm for estimation of local curvature from gradients of a tensor field that represents local orientation. The algorithm is based on an operator representation of the orientation tensor, which means that change of local orientation corresponds to a rotation of the eigenvectors of the tensor. The resulting curvature descriptor is a vector that points in the direction of the image in which the local orientation rotates anti-clockwise and the norm of the vector is the inverse of the radius of curvature. Two coefficients are defined that relate the change of local orientation with either curves or radial patterns.

A definition of central symmetry for local neighborhoods of 2-D images is given. A complete ON-set of centrally symmetric basis functions is proposed. The local neighborhoods are expanded in this basis. The behavior of coefficient spectrum obtained by this expansion is proposed to be the foundation of central symmetry parameters of the neighbqrhoods. Specifically examination of two such behaviors are proposed: Point concentration and line concentration of the energy spectrum. Moreover, the study of these types of behaviors of the spectrum are shown to be possible to do in the spatial domain.

In recent years, considerable research effort has been devoted to the development of useful descriptors for image quality. The attempts have been hampered by i n complete understanding of the operation of the human visual system. This has made it difficult to relate physical measures and perceptual traits.

A new model for determination of image quality is proposed. Its main feature is that it tries to invoke image content into consideration. The model builds upon a theory of image linearization, which means that the information in an image can well enough be represented using linear segments or structures within local spatial regions and frequency ranges. This implies a l so a suggestion that information in an image has to do with one- dimensional correlations. This gives a possibility to separate image content from noise in images, and measure them both.

Also a hypothesis is proposed that the visual system of humans does in fact perform such a linearization.