Publications by Gösta H. Granlund

The major goal of the COSPAL project is to develop an artificial cognitive system architecture, with the ability to autonomously extend its capabilities. Exploratory learning is one strategy that allows an extension of competences as provided by the environment of the system. Whereas classical learning methods aim at best for a parametric generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class, and to apply generalization on a conceptual level, resulting in new models. Incremental or online learning is a crucial requirement to perform exploratory learning. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning, and in this paper we focus on the organization of cognitive systems for efficient operation. Learning is used over the entire system. It is organized in the form of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail. We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user (teacher) and system is a major difference to classical robotics systems, where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems. We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module. We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

The purpose of cognitive systems is to produce a response to appropriate percepts. The response may be a direct physical action which may change the state of the system. It may be delayed in the form of a reconfiguration of internal models in relation to the interpreted context of the system. Or it may be to generate in a subsequent step a generalized symbolic representation which will allow its intentions of actions to be communicated to some other system. As important as the percepts, is the dependence upon context. A fundamental property of cognitive vision systems is that they shall be extendable. This requires that systems both acquire and store information about the environment autonomously on their own terms. The distributed organization foreseen for processing and for memory to allow learning, implies that later acquired information has to be stored in relation to earlier. The semantic character of the information which this requires, implies a storage with respect to similarity, and the availability of a metric. The paper discusses organization aspects of such systems, and proposes an architecture for cognitive systems. In this architecture, which consists of two major parts, the first part of the system, step by step performs a mapping from percepts onto actions or states. The central mechanism is the perception-action feedback cycle. In a learning phase, action precedes perception. The reason for this reversed causal direction is that action space is much less complex than percept space. It is easy to distinguish which percepts change as a function of an action or a state change. Percepts shall be mapped directly onto states or responses or functions involving these. The second part of the architecture deals with more invariant representations, or symbolic representations, which are derived mainly from system states and action states. Through active exploration of the environment, i.e. using perception-action learning, a system builds up concept spaces, defining the phenomena it can deal with. Information can subsequently be acquired by the system within these concept spaces without interaction, by extrapolation using passive observation or communication such as language. This structure has been implemented for the learning of object properties and view parameters in a fairly unrestricted setting, to be used for subsequent recognition purposes.

A method for unrestricted recognition of three-dimensional objects was developed. By unrestricted, we imply that the recognition will be done independently of object position, scale, orientation, and pose against a structured background. It does not assume any preceding segmentation or allow a reasonable degree of occlusion. The method uses a hierarchy of triplet feature invariants, which are at each level defined by a learning procedure. In the feedback learning procedure, percepts are mapped on system states corresponding to manipulation parameters of the object. The method uses a learning architecture with channel information representation. This article discusses how objects can be represented. We propose a structure to deal with object and contextual properties in a transparent manner.

Perceptual experiments indicate that corners and curvature are very important features in the process of recognition. This paper presents a new method to efficiently detect rotational symmetries, which describe complex curvature such as corners, circles, star- and spiral patterns. The method is designed to give selective and sparse responses. It works in three steps, first extract local orientation from a gray-scale or color image, second correlate the orientation image with rotational symmetry filters and third let the filter responses inhibit each other in order to get more selective responses, The correlations can be made efficient by separating the 2D-filters into a small number of 1D-filters. These symmetries can serve as feature points at a high abstraction level for use in hierarchical matching structures for 3D-estimation, object recognition, etc.

There is no indication that it will ever be possible to find some simple trick that miraculously solves most problems in vision. It turns out that the processing system must be able to implement a model structure, the complexity of which is directly related to the structural complexity of the problem under consideration in the external world. It has become increasingly apparent that Vision cannot be treated in isolation from the response generation, because a very high degree of integration is required between different levels of percepts and corresponding response primitives. The response to be produced at a given instance is as much dependent upon the state of the system, as the percepts impinging upon the system. In addition, it has become apparent that many classical aspects of perception, such as geometry, probably do not belong to the percept domain of a Vision system, but to the response domain. This article will focus on what are considered crucial problems in Vision for robotics for the future, rather than on the classical solutions today. It will discuss hierarchical architectures for combination of percept and response primitives. It will discuss the concept of combined percept-response invariances as important structural elements for Vision. It will be maintained that learning is essential to obtain the necessary flexibility and adaptivity. In consequence, it will be argued that invariances for the purpose of Vision are not abstractly geometrical, but derived from the percept-response interaction with the environment. The issue of information representation becomes extremely important in distributed structures of the types foreseen, where uncertainty of information has to be stated for update of models and associated data. The question of object representation is central to the paper. Equivalence is established between the representations of response, geometry and time. Finally an integrated percept-response structure is proposed for flexible response control.

In this paper, we discuss certain issues regarding robot vision. The main theme will be the importance of the choice of information representation. We will see the implications at different parts of a robot vision structure. We deal with aspects of pre-attentive versus attentive vision, control mechanisms for low level focus of attention, and representation of motion as the orientation of hyperplanes in multdimensional time-space. Issues of scale will be touched upon, and finally, a depth-from stereo algorithm based on guadrature filter phase is presented.

The problem of detection of orientation in finite dimensional Euclidean spaces is solved in the least squares sense. In particular, the theory is developed for the case when such orientation computations are necessary at all local neighborhoods of the n-dimensional Euclidean space. Detection of orientation is shown to correspond to fitting an axis or a plane to the Fourier transform of an n-dimensional structure. The solution of this problem is related to the solution of a well-known matrix eigenvalue problem. Moreover, it is shown that the necessary computations can be performed in the spatial domain without actually doing a Fourier transformation. Along with the orientation estimate, a certainty measure, based on the error of the fit, is proposed. Two applications in image analysis are considered: texture segmentation and optical flow. An implementation for 2-D (texture features) as well as 3-D (optical flow) is presented. In the case of 2-D, the method exploits the properties of the complex number field to by-pass the eigenvalue analysis, improving the speed and the numerical stability of the method. The theory is verified by experiments which confirm accurate orientation estimates and reliable certainty measures in the presence of noise. The comparative results indicate that the proposed theory produces algorithms computing robust texture features as well as optical flow. The computations are highly parallelizable and can be used in realtime image analysis since they utilize only elementary functions in a closed form (up to dimension 4) and Cartesian separable convolutions.

A new form of image estimator, which takes account of linear features, is derived using a signal equivalent formulation. The estimator is shown to be a nonstationary linear combination of three stationary estimators. The relation of the estimator to human visual physiology is discussed. A method for estimating the nonstationary control information is described and shown to be effective when the estimation is made from noisy data. A suboptimal approach which is computationally less demanding is presented and used in the restoration of a variety of images corrupted by additive white noise. The results show that the method can improve the quality of noisy images even when the signal-to-noise ratio is very low.

The mathematical basis is described of a new radiographic method by which an arbitrarily thick layer of the patient may be reconstructed. The reconstruction is performed from at least 60 images of the volume under examination. Each of these images, which have to be in digital form, is subjected to a special filtration process of its spatial frequencies. The combination of all the images will form the resulting image of the layer--the ectomogram. The method has been analysed and tested in experiments simulated with a computer.

The problem of finding a general, parallel, and hierarchical operator for picture processing is considered. An operator is defined which at different levels can detect and describe structure as opposed to uniformity within local regions, whatever structure and uniformity may imply at a particular level. The operator performs a mapping from one complex field to another. The important characteristic of this approach is the use of complex fields which allows a global-to-local feedback. In the transformation process the image is simplified. A Fourier implementation of the operator is described and a new transform is defined. The operators become increasingly global on higher levels in order to include adjacent high-level features. A hierarchical structure of such transformations gives a sequential description of structure over increasingly larger regions of the image. The processed information at different levels can be used as input to a classifier. Examples are given of processing results.

The advent of new stains for chromosomes has increased the possibilities that useful automated chromosome analysis can be implemented. The search for appropriate descriptors to use in this process is an important task. Data compression using integrated intensity and density profiles has already shown itself to be valuable. A method is proposed in this paper to describe these profiles as a sum of distribution functions. Every distribution function can be described by a triplet stating peak height, position, and width and it appears that these parameters are directly related to physical processes. The importance of such parameters in statistical chromosome analysis is emphasized. A classification experiment is described in which 240 chromosomes 1 to 22, X and Y have been classified with an accuracy of 96%.

Signal Processing for Computer Vision is a unique and thorough treatment of the signal processing aspects of filters and operators for low-level computer vision.

Computer vision has progressed considerably over recent years. From methods only applicable to simple images, it has developed to deal with increasingly complex scenes, volumes and time sequences. A substantial part of this book deals with the problem of designing models that can be used for several purposes within computer vision. These partial models have some general properties of invariance generation and generality in model generation.

Signal Processing for Computer Vision is the first book to give a unified treatment of representation and filtering of higher order data, such as vectors and tensors in multidimensional space. Included is a systematic organisation for the implementation of complex models in a hierarchical modular structure and novel material on adaptive filtering using tensor data representation.

Signal Processing for Computer Vision is intended for final year undergraduate and graduate students as well as engineers and researchers in the field of computer vision and image processing.

This chapter gives an introductory treatment of operations andrepresentations for low-level features in multi-dimensional spaces. Animportant issue is how to combine contributions from several filtersto provide robust statements in accordance with certain low-levelmodels. This chapter gives an introduction to the problems ofunambiguous mappings in multi-dimensional spaces.

This chapter establishes the motivation anduse of hierarchical operation structures to provide a systematicorganization for the implementation of complicated models. The chaptergives an intuitive treatment of most aspects that are considered inthe later chapters.

This paper explores the possibility to use a single low-resolution FIR camera for detection of pedestrians in the near zone in front of a vehicle. A low resolution sensor reduces the cost of the system, as well as the amount of data that needs to be processed in each frame.

We present a system that makes use of hot-spots and image positions of a near constant bearing to detect potential pedestrians. These detections provide seeds for an energy minimization algorithm that fits a pedestrian model to the detection. Since false alarms are hard to tolerate, the pedestrian model is then tracked, and the distance-to-collision (DTC) is measured by integrating size change measurements at sub-pixel accuracy, and the car velocity. The system should only engage braking for detections on a collision course, with a reliably measured DTC.

Preliminary experiments on a number of recorded near collision sequences indicate that our method may be useful for ranges up to about 10m using an 80x60 sensor, and somewhat more using a 160x120 sensor. We also analyze the robustness of the evaluated algorithm with respect to dead pixels, a potential problem for low-resolution sensors.

The purpose of cognitive systems is to produce a response to appropriate percepts. The response may be a direct physical action which may change the state of the system. It may be delayed in the form of a reconfiguration of internal models in relation to the interpreted context of the system. Or it may be to generate in a subsequent step a generalized symbolic representation which will allow its intentions of actions to be communicated to some other system. As important as the percepts, is the dependence upon context. A fundamental property of cognitive vision systems is that they shall be extendable. This requires that systems both acquire and store information about the environment autonomously on their own terms. The distributed organization foreseen for processing and for memory to allow learning, implies that later acquired information has to be stored in relation to earlier. The semantic character of the information which this requires, implies a storage with respect to similarity, and the availability of a metric. The paper discusses organization aspects of such systems, and proposes an architecture for cognitive systems. In this architecture, which consists of two major parts, the first part of the system, step by step performs a mapping from percepts onto actions or states. The central mechanism is the perception-action feedback cycle. In a learning phase, action precedes perception. The reason for this reversed causal direction is that action space is much less complex than percept space. It is easy to distinguish which percepts change as a function of an action or a state change. Percepts shall be mapped directly onto states or responses or functions involving these. The second part of the architecture deals with more invariant representations, of symbolic representations, which are derived mainly from system states and action states. Through active exploration of the environment, e.g. using perceptionaction learning, a system builds up concept spaces, defining the phenomena it can deal with. Information can subsequently be acquired by the system within these concept spaces without interaction, by extrapolation using passive observation or communication such as language. This structure has been implemented for the learning of object properties and view parameters in a fairly unrestricted setting, to be used for subsequent recognition purposes.

 To program a robot to solve a simple shape-sorter puzzle is trivial. To devise a Cognitive System Architecture, which allows the system to find out by itself how to go about a solution, is less than trivial. The development of such an architecture is one of the aims of the COSPAL project, leading to new techniques in vision based Artificial Cognitive Systems, which allow the development of robust systems for real dynamic environments. The systems developed under the project itself remain however in simplified scenarios, likewise the shape-sorter problem described in the present paper. The key property of the described system is its robustness. Since we apply association strategies of local features, the system behaves robustly under a wide range of distortions, as occlusion, colour and intensity changes. The segmentation step which is applied in many systems known from literature is replaced with local associations and view-based hypothesis validation. The hypotheses used in our system are based on the anticipated state of the visual percepts. This state replaces explicit modeling of shapes. The current state is chosen by a voting system and verified against the true visual percepts. The anticipated state is obtained from the association to the manipulator actions, where reinforcement learning replaces the explicit calculation of actions. These three differences to classical schemes allow the design of a much more generic and flexible system with a high level of robustness. On the technical side, the channel representation of information and associative learning in terms of the channel learning architecture are essential ingredients for the system. It is the properties of locality, smoothness, and non-negativity which make these techniques suitable for this kind of application. The paper gives brief descriptions of how different system parts have been implemented and show some examples from our tests.

This paper presents a method for detection of homogeneous regions in grey-scale images, representing them as blobs. In order to be fast, and not to favour one scale over others, the method uses a scale pyramid. In contrast to most multi-scale methods this one is non-linear, since it employs robust estimation rather than averaging to move through scale-space. This has the advantage that adjacent and partially overlapping clusters only affect each other's shape, not each other's values. It even allows blobs within blobs, to provide a pyramid blob structure of the image.

This paper presents and overview of the basic and applied research carried out by the Computer Vision Laboratory, Linköping University, in the WITAS UAV Project. This work includes customizing and redesigning vision methods to fit the particular needs and restrictions imposed by the UAV platform, e.g., for low-level vision, motion estimation, navigation, and tracking. It also includes a new learning structure for association of perception-action activations, and a runtime system for implementation and execution of vision algorithms. The paper contains also a brief introduction to the WITAS UAV Project.

The purpose of this paper is to provide a broad overview of the WITAS Unmanned Aerial Vehicle Project. The WITAS UAV project is an ambitious, long-term basic research project with the goal of developing technologies and functionalities necessary for the successful deployment of a fully autonomous UAV operating over diverse geographical terrain containing road and traffic networks. Theproject is multi-disciplinary in nature, requiring many different research competences, and covering a broad spectrum of basic research issues, many of which relate to current topics in artificial intelligence. A number of topics considered are knowledge representation issues, active vision systems and their integration with deliberative/reactive architectures, helicopter modeling and control, ground operator dialogue systems, actual physical platforms, and a number of simulation techniques.

Most of the processing in vision today uses spatially invariant operations. This gives efficient and compact computing structures, with the conventional convenient separation between data and operations. This also goes well with conventional Cartesian representation of data. Currently, there is a trend towards context dependent processing in various forms. This implies that operations will no longer be spatially invariant, but vary over the image dependent upon the image content. There are many ways in which such a contextual control can be implemented. Mechanisms can be added for the modification of operator behavior within the conventional computing structure. This has been done e.g. for the implementation of adaptive filtering. In order to obtain sufficient flexibility and power in the computing structure, it is necessary to go further than that. To achieve sufficiently good adaptivity, it is necessary to ensure that sufficiently complex control strategies can be represented. It is becoming increasingly apparent that this can not be achieved through prescription or program specification of rules. The reason being that these rules will be dauntingly complex and can not be be dealt with in sufficient detail. At the same time that we require the implementation of a spatially variant processing, this implies the requirement for a spatially variant information representation. Otherwise a sufficiently effective and flexible contextual control can not be implemented. This paper outlines a new structure for effective space variant processing. It utilises a new type of localized information representation, which can be viewed as outputs from band pass filters such as wavelets. A unique and important feature is that convex regions can be built up from a single layer of associating nodes. The specification of operations is made through learning or action controlled association.

This paper introduces a novel algorithm for extracting the optical flow obtained from a translating camera in a static scene. Occlusion between objects is incorporated as a natural component in a scene reconstruction strategy by first evaluate and reconstruct the foreground and then exclude its influence on the partly occluded objects behind.

WITAS will be engaged in goal-directed basic research in the area of intelligent autonomous vehicles and other autonomous systems. In this paper an overview of the project is given together with a presentation of our research interests in the project. The current status of our part in the project is also given.

It has become increasingly apparent that perception cannot be treated in isolation from the response generation, firstly because a very high degree of integration is required between different levels of percepts and corresponding response primitives. Secondly, it turns out that the response to be produced at a given instance is as much dependent upon the state of the system, as the percepts impinging upon the system. The state of the system is in consequence the combination of the responses produced and the percepts associated with these responses. Thirdly, it has become apparent that many classical aspects of perception, such as geometry, probably do not belong to the percept domain of a Vision system, but to the response domain. There are not yet solutions available to all of these problems. In consequence, this overview will focus on what are considered crucial problems for the future, rather than on the solutions available today. It will discuss hierarchical architectures for combination of percept and response primitives, and the concept of combined percept-response invariances as important structural elements for Vision. It will be maintained that learning is essential to obtain the necessary exibility and adaptivity. In consequence, it will be argued that invariances for the purpose of vision are not geometrical but derived from the percept-response interaction with the environment. The issue of information representation becomes extremely important in distributed structures of the types foreseen, where uncertainty of information has to be stated for update of models and associated data.

A methodology for spatio-temporal filtering of image sequences is under development at Computer Vision Laboratory, Linköping University. In later years scale analysis has been found to be a necessary tool in image analysis of stationary images. It is our belief that a combination of spatio-temporal filtering and scale analysis is required to get satisfactory results on image sequences. A growing need and the availability of more powerful computers are the most important reasons for this development. The objectives and proposed methods are discussed in relation to known properties of mammal vision.

The problem of optimal detection of orientation in arbitrary neighborhoods is solved in the least squares sense. It is shown that this corresponds to fitting an axis in the Fourier domain of the n-dimensional neighborhood, the solution of which is a well known solution of a matrix eigenvalue problem. The eigenvalues are the variance or inertia with respect to the axes given by their respective eigen vectors. The orientation is taken as the axis given by the least eigenvalue. Moreover it is shown that the necessary computations can be pursued in the spatial domain without doing a Fourier transformation. An implementation for 2-D is presented. Two certainty measures are given corresponding to the orientation estimate. These are the relative or the absolute distances between the two eigenvalues, revealing whether the fitted axis is much better than an axis orthogonal to it. The result of the implementation is verified by experiments which confirm an accurate orientation estimation and reliable certainty measure in the presence of additive noise at high level as well as low levels.

[No abstract available]

One major goal of the COSPAL project is to develop an artificial cognitive system architecture with the capability of exploratory learning. Exploratory learning is a strategy that allows to apply generalization on a conceptual level, resulting in an extension of competences. Whereas classical learning methods aim at best possible generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class. Incremental or online learning is an inherent requirement to perform exploratory learning.

Exploratory learning requires new theoretic tools and new algorithms. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning and in this paper we focus on its algorithmic aspect. Learning is performed in terms of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail.

We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user (’teacher’) and system is a major difference to most existing systems where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems.

We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module.We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

This report brings together a novel approach to some computer vision problems and a particular algorithmic development of the Landweber iterative algorithm. The algorithm solves a class of high-dimensional, sparse, and constrained least-squares problems, which arise in various computer vision learning tasks, such as object recognition and object pose estimation. The algorithm has recently been applied to these problems, but it has been used rather heuristically. In this report we describe the method and put it on firm mathematical ground. We consider a convexly constrained weighted least-squares problem and propose for its solution a projected Landweber method which employs oblique projections onto the closed convex constraint set. We formulate the problem, present the algorithm and work out its convergence properties, including a rate-of-convergence result. The results are put in perspective of currently available projected Landweber methods. The application to supervised learning is described, and the method is evaluated in a function approximation experiment.

A new architecture for learning systems has been developed. A number of particular design features in combination result in a very high performance and excellent robustness. The architecture uses a monopolar channel information representation. The channel representation implies a partially overlapping mapping of signals into a higher-dimensional space, such that a flexible but continuous restructuring mapping can be made. The high-dimensional mapping introduces locality in the information representation, which is directly available in wavelets or filter outputs. Single level maps using this representation can produce closed decision regions, thereby eliminating the need for costly back-propagation. The monopolar property implies that data only utilizes one polarity, say positive values, in addition to zero, allowing zero to represent no information. This leads to an efficient sparse representation.

The processing mode of the architecture is association where the mapping of feature inputs onto desired state outputs is learned from a representative training set. The sparse monopolar representation together with locality, using individual learning rates, allows a fast optimization, as the system exhibits linear complexity. Mapping into multiple channels gives a strategy to use confidence statements in data, leading to a low sensitivity to noise in features. The result is an architecture allowing systems with a complexity of some hundred thousand features described by some hundred thousand samples to be trained in typically less than an hour. Experiments that demonstrate functionality and noise immunity are presented. The architecture has been applied to the design of hyper complex operations for view centered object recognition in robot vision.

This report introduces a signal processing strategy for depth segmentation and scene reconstruction that incorporates occlusion as a natural component. The work aims to maximize the use of connectivity in the temporal domain as much as possible under the condition that the scene is static and that the camera motion is known. An object behind the foreground is reconstructed using the fact that different parts of the object have been seen in different images in the sequence. One of the main ideas in the reported work is the use of a spatiotemporal certainty volume c(x) with the same dimension as the input spatiotemporal volume s(x), and then use c(x) as a 'blackboard' for rejecting already segmented image structures. The segmentation starts with searching for image structures in the foreground, eliminates their occluding influence, and then proceeds. Normalized convolution, which is a Weighted Least Mean Square technique for filtering data with varying spatial reliability, is used for all filtering. High spatial resolution near object borders is achieved and only neighboring structures with similar depth supports each other.

This paper presents an algorithm for estimation of local curvature from gradients of a tensor field that represents local orientation. The algorithm is based on an operator representation of the orientation tensor, which means that change of local orientation corresponds to a rotation of the eigenvectors of the tensor. The resulting curvature descriptor is a vector that points in the direction of the image in which the local orientation rotates anti-clockwise and the norm of the vector is the inverse of the radius of curvature. Two coefficients are defined that relate the change of local orientation with either curves or radial patterns.