Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

13
3
12 1
Reset filters

Settings
Now showing 1 - 10 of 13
  • Publication
    Handling Missing Observations with an RNN-based Prediction-Update Cycle
    ( 2021)
    Becker, Stefan
    ;
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    ;
    Morris, Brendan T.
    In tasks such as tracking, time-series data inevitably carry missing observations. While traditional tracking approaches can handle missing observations, recurrent neural networks (RNNs) are designed to receive input data in every step. Furthermore, current solutions for RNNs, like omitting the missing data or data imputation, are not sufficient to account for the resulting increased uncertainty. Towards this end, this paper introduces an RNN-based approach that provides a full temporal filtering cycle for motion state estimation. The Kalman filter inspired approach enables to deal with missing observations and outliers. For providing a full temporal filtering cycle, a basic RNN is extended to take observations and the associated belief about its accuracy into account for updating the current state. An RNN prediction model, which generates a parametrized distribution to capture the predicted states, is combined with an RNN update model, which relies on the prediction model output and the current observation. By providing the model with masking information, binary-encoded missing events, the model can overcome limitations of standard techniques for dealing with missing input values. The model abilities are demonstrated on synthetic data reflecting prototypical pedestrian tracking scenarios.
  • Publication
    Generating Synthetic Training Data for Deep Learning-Based UAV Trajectory Prediction
    ( 2021)
    Becker, Stefan
    ;
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    ;
    Morris, Brendan T.
    Deep learning-based models, such as recurrent neural networks (RNNs), have been applied to various sequence learning tasks with great success. Following this, these models are increasingly replacing classic approaches in object tracking applications for motion prediction. On the one hand, these models can capture complex object dynamics with less modeling required, but on the other hand, they depend on a large amount of training data for parameter tuning. Towards this end, we present an approach for generating synthetic trajectory data of unmanned-aerial-vehicles (UAVs) in image space. Since UAVs, or rather quadrotors are dynamical systems, they can not follow arbitrary trajectories. With the prerequisite that UAV trajectories fulfill a smoothness criterion corresponding to a minimal change of higher-order motion, methods for planning aggressive quadrotors flights can be utilized to generate optimal trajectories through a sequence of 3D waypoints. By projecting these maneuver trajectories, which are suitable for controlling quadrotors, to image space, a versatile trajectory data set is realized. To demonstrate the applicability of the synthetic trajectory data, we show that an RNN-based prediction model solely trained on the generated data can outperform classic reference models on a real-world UAV tracking dataset. The evaluation is done on the publicly available ANTI-UAV dataset.
  • Publication
    Introducing Probabilistic Bézier Curves for N-Step Sequence Prediction
    ( 2020)
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    Representations of sequential data are commonly based on the assumption that observed sequences are realizations of an unknown underlying stochastic process, where the learning problem includes determination of the model parameters. In this context, a model must be able to capture the multi-modal nature of the data, without blurring between single modes. This paper proposes probabilistic Bezier curves (N-Curves) as a basis for effectively modeling continuous-time stochastic processes. The model is based on Mixture Density Networks (MDN) and Bezier curves with Gaussian random variables as control points. Key advantages of the model include the ability of generating smooth multi-mode predictions in a single inference step which reduces the need for Monte Carlo simulation. This property is in line with recent attempts to address the problem of quantifying uncertainty as a regression problem. Essential properties of the proposed approach are illustrated by several toy examples and the task of multi-step sequence prediction. As an initial proof of concept, the model performance is compared to an LSTM-MDN model and recurrent Gaussian processes on two real world use-cases, trajectory prediction and motion capture sequence prediction.
  • Publication
    Modeling continuous-time stochastic processes using N-Curve mixtures
    ( 2019)
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    Representations of sequential data are commonly based on the assumption that observed sequences are realizations of an unknown underlying stochastic process, where the learning problem includes determination of the model parameters. In this context the model must be able to capture the multi-modal nature of the data, without blurring between modes. This property is essential for applications like trajectory prediction or human motion modeling. Towards this end, a neural network model for continuous-time stochastic processes usable for sequence prediction is proposed. The model is based on Mixture Density Networks using Bézier curves with Gaussian random variables as control points (abbrev.: N-Curves). Key advantages of the model include the ability of generating smooth multi-mode predictions in a single inference step which reduces the need for Monte Carlo simulation, as required in many multi-step prediction models, based on state-of-the-art neural networks. Essential properties of the proposed approach are illustrated by several toy examples and the task of multi-step sequence prediction. Further, the model performance is evaluated on two real world use-cases, i.e. human trajectory prediction and human motion modeling, outperforming different state-of-the-art models.
  • Publication
    An RNN-Based IMM Filter Surrogate
    ( 2019)
    Becker, Stefan
    ;
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    The problem of varying dynamics of tracked objects, such as pedestrians, is traditionally tackled with approaches like the Interacting Multiple Model (IMM) filter using a Bayesian formulation. By following the current trend towards using deep neural networks, in this paper an RNN-based IMM filter surrogate is presented. Similar to an IMM filter solution, the presented RNN-based model assigns a probability value to a performed dynamic and, based on them, puts out a multi-modal distribution over future pedestrian trajectories. The evaluation is done on synthetic data, reflecting prototypical pedestrian maneuvers.
  • Publication
    Particle-based Pedestrian Path Prediction using LSTM-MDL Models
    ( 2018)
    Hug, Ronny
    ;
    Becker, Stefan
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    Recurrent neural networks are able to learn complex long-term relationships from sequential data and output a probability density function over the state space. Therefore, recurrent models are a natural choice to address path prediction tasks, where a trained model is used to generate future expectations from past observations. When applied to security applications, like predicting pedestrian paths for risk assessment, a point-wise greedy evaluation of the output pdf is not feasible, since the environment often allows multiple choices. Therefore, a robust risk assessment has to take all options into account, even if they are overall not very likely. Towards this end, a combination of particle filtering strategies and a LSTM-MDL model is proposed to address a multimodal path prediction task. The capabilities and viability of the proposed approach are evaluated on several synthetic test conditions, yielding the counter-intuitive result that the simplest approach performs best. Further, the feasibility of the proposed approach is illustrated on several real world scenes.
  • Publication
    Interactive concepts for shaping generative models of spatial behavior
    ( 2017)
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    A technique widely used in video based situation assessment, and especially in anomaly detection, is the analysis of spatial behavior in terms of motion profiles recorded along trajectories. An intuitive assessment metric is the deviation from normal behavior, where generative models are a natural choice for capturing the underlying statistics. Applying such outlier methods in open world scenarios has the drawback that long observation times are required, in order to fully determine the model, while underdetermined models are very prone to generate non-intuitive or wrong results. In order to address this problem, the usage of interactive concepts for supporting the learning process and refining learned models is proposed. Thereby, the method keeps track of automatically integrated observations and stochastic priors generated by examples provided by the user. Examples can be given in terms of individual labeled samples, or in terms of complex pdfs. The feasibility of the proposed approach is illustrated on the BIWI Walking Pedestrians dataset, using partitioned Gaussian mixture models as the generative model.
  • Publication
    Supporting generative models of spatial behavior by user interaction
    ( 2017)
    Hug, Ronny
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    The analysis of spatial behavior in terms of motion profiles recorded along trajectories is a widely used technique in video analysis. Inherent to this approach is the problem to assign a meaningful score to observations. This score builds the basis for classification, ranking, or to generate user feedback. Score assignment can be done in terms of deviations from normal behavior, where normality is determined by learning a generative model. A general drawback is that the unsupervised learning process often assigns non-intuitive scores. In order to address this problem this paper proposes the usage of interactive concepts, which support the learning process. Interaction thereby strongly utilizes the generative models capabilities to synthesize samples, to give insight into the underlying representation. Initial results are shown on a trajectory rating task, illustrating the feasibility of the proposed approach.
  • Publication
    On the benefit of state separation for tracking in image space with an interacting multiple model filter
    ( 2016)
    Becker, Stefan
    ;
    Kieritz, Hilke
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    When tracking an object, it is reasonable to assume that the dynamic model can change over time. In practical applications, Interacting Multiple Model (IMM) filter are a popular choice for considering such varying system characteristics. The motion of the object is often modeled using position, velocity, and acceleration. It seems obvious that different image space dimensions can be considered in one overall system state vector. In this paper, the fallacy of simply extending the state vector in case of tracking an object solely in image space is demonstrated. Thereby, we show how under such conditions the effectiveness of an IMM filter can be improved by separating particular states. The proposed approach is evaluated on the VOT 2014 dataset.
  • Publication
    Detection of infrastructure manipulation with knowledge-based video surveillance
    ( 2016)
    Münch, David
    ;
    Hilsenbeck, Barbara
    ;
    Kieritz, Hilke
    ;
    Becker, Stefan
    ;
    Grosselfinger, Ann-Kristin
    ;
    Hübner, Wolfgang
    ;
    Arens, Michael
    We are living in a world dependent on sophisticated technical infrastructure. Malicious manipulation of such critical infrastructure poses an enormous threat for all its users. Thus, running a critical infrastructure needs special attention to log the planned maintenance or to detect suspicious events. Towards this end, we present a knowledge-based surveillance approach capable of logging visual observable events in such an environment. The video surveillance modules are based on appearance-based person detection, which further is used to modulate the outcome of generic processing steps such as change detection or skin detection. A relation between the expected scene behavior and the underlying basic video surveillance modules is established. It will be shown that the combination already provides sufficient expressiveness to describe various everyday situations in indoor video surveillance. The whole approach is qualitatively and quantitatively evaluated on a prototypical scenario in a server room.