Publications

2024

1. ICLR

   Calibrating Bayesian UNet++ for Sub-Seasonal Forecasting

   B. Asan, A. Akgül, A. Unal, and 2 more authors

   In Tackling Climate Change with Machine Learning at ICLR 2024 2024

   Abs Bib HTML

   Seasonal forecasting is a crucial task when it comes to detecting the extreme heat and colds that occur due to climate change. Confidence in the predictions should be reliable since a small increase in the temperatures in a year has a big impact on the world. Calibration of the neural networks provides a way to ensure our confidence in the predictions. However, calibrating regression models is an under-researched topic, especially in forecasters. We calibrate a UNet++ based architecture, which was shown to outperform physics-based models in temperature anomalies. We show that with a slight trade-off between prediction error and calibration error, it is possible to get more reliable and sharper forecasts. We believe that calibration should be an important part of safety-critical machine learning applications such as weather forecasters.

   @inproceedings{asan2024calibrating,
     title = {Calibrating Bayesian UNet++ for Sub-Seasonal Forecasting},
     author = {Asan, B. and Akgül, A. and Unal, A. and Kandemir, M. and Unal, G.},
     booktitle = {Tackling Climate Change with Machine Learning at ICLR 2024},
     year = {2024},
   }

2. L4DC

   Continual Learning of Multi-modal Dynamics with External Memory

   A. Akgül, G. Unal, and M. Kandemir

   In Proceedings of The 6th Annual Learning for Dynamics and Control Conference 2024

   Abs Bib HTML

   We study the problem of fitting a model to a dynamical environment when new modes of behavior emerge sequentially. The learning model is aware when a new mode appears, but it does not have access to the true modes of individual training sequences. We devise a novel continual learning method that maintains a descriptor of the mode of an encountered sequence in a neural episodic memory. We employ a Dirichlet Process prior on the attention weights of the memory to foster efficient storage of the mode descriptors. Our method performs continual learning by transferring knowledge across tasks by retrieving the descriptors of similar modes of past tasks to the mode of a current sequence and feeding this descriptor into its transition kernel as control input. We observe the continual learning performance of our method to compare favorably to the mainstream parameter transfer approach.

   @inproceedings{akgul2024cddp,
     title = {Continual Learning of Multi-modal Dynamics with External Memory},
     author = {Akgül, A. and Unal, G. and Kandemir, M.},
     booktitle = {Proceedings of The 6th Annual Learning for Dynamics and Control Conference},
     year = {2024},
   }

3. arXiv

   Probabilistic Actor-Critic: Learning to Explore with PAC-Bayes Uncertainty

   B. Tasdighi, N. Werge, Y. Wu, and 1 more author

   2024

   Abs Bib HTML Code

   We introduce Probabilistic Actor-Critic (PAC), a novel reinforcement learning algorithm with improved continuous control performance thanks to its ability to mitigate the exploration-exploitation trade-off. PAC achieves this by seamlessly integrating stochastic policies and critics, creating a dynamic synergy between the estimation of critic uncertainty and actor training. The key contribution of our PAC algorithm is that it explicitly models and infers epistemic uncertainty in the critic through Probably Approximately Correct-Bayesian (PAC-Bayes) analysis. This incorporation of critic uncertainty enables PAC to adapt its exploration strategy as it learns, guiding the actor’s decision-making process. PAC compares favorably against fixed or pre-scheduled exploration schemes of the prior art. The synergy between stochastic policies and critics, guided by PAC-Bayes analysis, represents a fundamental step towards a more adaptive and effective exploration strategy in deep reinforcement learning. We report empirical evaluations demonstrating PAC’s enhanced stability and improved performance over the state of the art in diverse continuous control problems.

   @article{tasdighi2024probabilistic,
     title = {Probabilistic Actor-Critic: Learning to Explore with PAC-Bayes Uncertainty},
     author = {Tasdighi, B. and Werge, N. and Wu, Y. and Kandemir, M.},
     booktitle = {arXiv Preprint},
     year = {2024},
   }

4. arXiv

   Latent variable model for high-dimensional point process with structured missingness

   M. Sinelnikov, M. Haussmann, and H. Lähdesmäki

   2024

   Abs Bib HTML

   Longitudinal data are important in numerous fields, such as healthcare, sociology and seismology, but real-world datasets present notable challenges for practitioners because they can be high-dimensional, contain structured missingness patterns, and measurement time points can be governed by an unknown stochastic process. While various solutions have been suggested, the majority of them have been designed to account for only one of these challenges. In this work, we propose a flexible and efficient latent-variable model that is capable of addressing all these limitations. Our approach utilizes Gaussian processes to capture temporal correlations between samples and their associated missingness masks as well as to model the underlying point process. We construct our model as a variational autoencoder together with deep neural network parameterised encoder and decoder models, and develop a scalable amortised variational inference approach for efficient model training. We demonstrate competitive performance using both simulated and real datasets.

   @article{sinelnikov2024latent,
     title = {Latent variable model for high-dimensional point process with structured missingness},
     author = {Sinelnikov, M. and Haussmann, M. and L\"{a}hdesm\"{a}ki, H.},
     booktitle = {arXiv Preprint},
     year = {2024},
   }

2023

1. arXiv

   PAC-Bayesian Soft Actor-Critic Learning

   B. Tasdighi, A. Akgül, K.K. Brink, and 1 more author

   In arXiv Preprint 2023

   Abs Bib HTML

   Actor-critic algorithms address the dual goals of reinforcement learning, policy evaluation and improvement, via two separate function approximators. The practicality of this approach comes at the expense of training instability, caused mainly by the destructive effect of the approximation errors of the critic on the actor. We tackle this bottleneck by employing an existing Probably Approximately Correct (PAC) Bayesian bound for the first time as the critic training objective of the Soft Actor-Critic (SAC) algorithm. We further demonstrate that the online learning performance improves significantly when a stochastic actor explores multiple futures by critic-guided random search. We observe our resulting algorithm to compare favorably to the state of the art on multiple classical control and locomotion tasks in both sample efficiency and asymptotic performance.

   @inproceedings{bahareh2022pac4sac,
     title = {PAC-Bayesian Soft Actor-Critic Learning},
     author = {Tasdighi, B. and Akgül, A. and Brink, K.K. and Kandemir, M.},
     booktitle = {arXiv Preprint},
     year = {2023},
   }

2. arXiv

   Demystifying the Myths and Legends of Nonconvex Convergence of SGD

   A. Dutta, E.H. Bergou, S. Boucherouite, and 3 more authors

   2023

   Abs Bib HTML

   Stochastic gradient descent (SGD) and its variants are the main workhorses for solving large-scale optimization problems with nonconvex objective functions. Although the convergence of SGDs in the (strongly) convex case is well-understood, their convergence for nonconvex functions stands on weak mathematical foundations. Most existing studies on the nonconvex convergence of SGD show the complexity results based on either the minimum of the expected gradient norm or the functional sub-optimality gap (for functions with extra structural property) by searching the entire range of iterates. Hence the last iterations of SGDs do not necessarily maintain the same complexity guarantee. This paper shows that \em an ε-stationary point exists in the final iterates of SGDs, given a large enough total iteration budget, T, not just anywhere in the entire range of iterates — a much stronger result than the existing one. Additionally, our analyses allow us to measure the \emphdensity of the ε-stationary points in the final iterates of SGD, and we recover the classical O(\frac1\sqrtT) asymptotic rate under various existing assumptions on the objective function and the bounds on the stochastic gradient. As a result of our analyses, we addressed certain myths and legends related to the nonconvex convergence of SGD and posed some thought-provoking questions that could set new directions for research.

   @article{dutta2023demystifying,
     title = {Demystifying the Myths and Legends of Nonconvex Convergence of SGD},
     author = {Dutta, A. and Bergou, E.H. and Boucherouite, S. and Werge, N. and Kandemir, M. and Li, X.},
     booktitle = {arXiv Preprint},
     year = {2023},
   }

3. arXiv

   On Adaptive Stochastic Optimization for Streaming Data: A Newton’s Method with O(dN) Operations

   A. Godichon-Baggioni, and N. Werge

   2023

   Abs Bib HTML

   Stochastic optimization methods encounter new challenges in the realm of streaming, characterized by a continuous flow of large, high-dimensional data. While first-order methods, like stochastic gradient descent, are the natural choice, they often struggle with ill-conditioned problems. In contrast, second-order methods, such as Newton’s methods, offer a potential solution, but their computational demands render them impractical. This paper introduces adaptive stochastic optimization methods that bridge the gap between addressing ill-conditioned problems while functioning in a streaming context. Notably, we present an adaptive inversion-free Newton’s method with a computational complexity matching that of first-order methods, \mathcalO(dN), where d represents the number of dimensions/features, and N the number of data. Theoretical analysis confirms their asymptotic efficiency, and empirical evidence demonstrates their effectiveness, especially in scenarios involving complex covariance structures and challenging initializations. In particular, our adaptive Newton’s methods outperform existing methods, while maintaining favorable computational efficiency.

   @article{godichon2023adaptive,
     title = {On Adaptive Stochastic Optimization for Streaming Data: A {N}ewton's Method with {O}(dN) Operations},
     author = {Godichon-Baggioni, A. and Werge, N.},
     booktitle = {arXiv Preprint},
     year = {2023},
   }

4. arXiv

   BOF-UCB: A Bayesian-Optimistic Frequentist Algorithm for Non-Stationary Contextual Bandits

   N. Werge, A. Akgül, and M. Kandemir

   arXiv Preprint 2023

   Abs Bib HTML

   We propose a novel Bayesian-Optimistic Frequentist Upper Confidence Bound (BOF-UCB) algorithm for stochastic contextual linear bandits in non-stationary environments. This unique combination of Bayesian and frequentist principles enhances adaptability and performance in dynamic settings. The BOF-UCB algorithm utilizes sequential Bayesian updates to infer the posterior distribution of the unknown regression parameter, and subsequently employs a frequentist approach to compute the Upper Confidence Bound (UCB) by maximizing the expected reward over the posterior distribution. We provide theoretical guarantees of BOF-UCB’s performance and demonstrate its effectiveness in balancing exploration and exploitation on synthetic datasets and classical control tasks in a reinforcement learning setting. Our results show that BOF-UCB outperforms existing methods, making it a promising solution for sequential decision-making in non-stationary environments.

   @article{werge2023bof,
     title = {BOF-UCB: A Bayesian-Optimistic Frequentist Algorithm for Non-Stationary Contextual Bandits},
     author = {Werge, N. and Akgül, A. and Kandemir, M.},
     journal = {arXiv Preprint},
     year = {2023},
   }

5. arXiv

   If there is no underfitting, there is no Cold Posterior Effect

   Y. Zhang, Y. Wu, L.A. Ortega, and 1 more author

   2023

   Abs Bib HTML

   The cold posterior effect (CPE) \citepWRVS+20 in Bayesian deep learning shows that, for posteriors with a temperature T<1, the resulting posterior predictive could have better performances than the Bayesian posterior (T=1). As the Bayesian posterior is known to be optimal under perfect model specification, many recent works have studied the presence of CPE as a model misspecification problem, arising from the prior and/or from the likelihood function. In this work, we provide a more nuanced understanding of the CPE as we show that \emphmisspecification leads to CPE only when the resulting Bayesian posterior underfits. In fact, we theoretically show that if there is no underfitting, there is no CPE.

   @article{zhang2023if,
     title = {If there is no underfitting, there is no Cold Posterior Effect},
     author = {Zhang, Y. and Wu, Y. and Ortega, L.A. and Masegosa, A.R.},
     booktitle = {arXiv Preprint},
     year = {2023},
   }

6. arXiv

   EdVAE: Mitigating Codebook Collapse with Evidential Discrete Variational Autoencoders

   G. Baykal, M. Kandemir, and G. Unal

   In arXiv Preprint 2023

   Abs Bib HTML

   Codebook collapse is a common problem in training deep generative models with discrete representation spaces like Vector Quantized Variational Autoencoders (VQ-VAEs). We observe that the same problem arises for the alternatively designed discrete variational autoencoders (dVAEs) whose encoder directly learns a distribution over the codebook embeddings to represent the data. We hypothesize that using the softmax function to obtain a probability distribution causes the codebook collapse by assigning overconfident probabilities to the best matching codebook elements. In this paper, we propose a novel way to incorporate evidential deep learning (EDL) instead of softmax to combat the codebook collapse problem of dVAE. We evidentially monitor the significance of attaining the probability distribution over the codebook embeddings, in contrast to softmax usage. Our experiments using various datasets show that our model, called EdVAE, mitigates codebook collapse while improving the reconstruction performance, and enhances the codebook usage compared to dVAE and VQ-VAE based models.

   @inproceedings{baykal2023edvae,
     author = {Baykal, G. and Kandemir, M. and Unal, G.},
     booktitle = {arXiv Preprint},
     year = {2023},
   }

7. NeurIPS

   Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures

   H. Flynn, D. Reeb, M. Kandemir, and 1 more author

   In Neural Information Processing Systems 2023

   Abs Bib HTML

   We present improved algorithms with worst-case regret guarantees for the stochastic linear bandit problem. The widely used "optimism in the face of uncertainty" principle reduces a stochastic bandit problem to the construction of a confidence sequence for the unknown reward function. The performance of the resulting bandit algorithm depends on the size of the confidence sequence, with smaller confidence sets yielding better empirical performance and stronger regret guarantees. In this work, we use a novel tail bound for adaptive martingale mixtures to construct confidence sequences which are suitable for stochastic bandits. These confidence sequences allow for efficient action selection via convex programming. We prove that a linear bandit algorithm based on our confidence sequences is guaranteed to achieve competitive worst-case regret. We show that our confidence sequences are tighter than competitors, both empirically and theoretically. Finally, we demonstrate that our tighter confidence sequences give improved performance in several hyperparameter tuning tasks.

   @inproceedings{flynn2023improved,
     author = {Flynn, H. and Reeb, D. and Kandemir, M. and Peters, J.},
     year = {2023},
     title = {Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures},
     booktitle = {Neural Information Processing Systems},
   }

8. ACML

   Estimation of Counterfactual Interventions under Uncertainties

   J. Weilbach, S. Gerwinn, M. Kandemir, and 1 more author

   In Asian Conference on Machine Learning 2023

   Abs Bib HTML

   Counterfactual analysis is intuitively performed by humans on a daily basis eg. "What should I have done differently to get the loan approved?". Such counterfactual questions also steer the formulation of scientific hypotheses. More formally it provides insights about potential improvements of a system by inferring the effects of hypothetical interventions into a past observation of the system’s behaviour which plays a prominent role in a variety of industrial applications. Due to the hypothetical nature of such analysis, counterfactual distributions are inherently ambiguous. This ambiguity is particularly challenging in continuous settings in which a continuum of explanations exist for the same observation. In this paper, we address this problem by following a hierarchical Bayesian approach which explicitly models such uncertainty. In particular, we derive counterfactual distributions for a Bayesian Warped Gaussian Process thereby allowing for non-Gaussian distributions and non-additive noise. We illustrate the properties our approach on a synthetic and on a semi-synthetic example and show its performance when used within an algorithmic recourse downstream task.

   @inproceedings{weilbach2023estimation,
     title = {Estimation of Counterfactual Interventions under Uncertainties},
     author = {Weilbach, J. and Gerwinn, S. and Kandemir, M. and Fraenzle, M.},
     booktitle = {Asian Conference on Machine Learning},
     year = {2023},
   }

9. T-PAMI

   PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison

   H. Flynn, D. Reeb, M. Kandemir, and 1 more author

   IEEE Transactions on Pattern Analysis and Machine Intelligence 2023

   Abs Bib HTML

   PAC-Bayes has recently re-emerged as an effective theory with which one can derive principled learning algorithms with tight performance guarantees. However, applications of PAC-Bayes to bandit problems are relatively rare, which is a great misfortune. Many decision-making problems in healthcare, finance and natural sciences can be modelled as bandit problems. In many of these applications, principled algorithms with strong performance guarantees would be very much appreciated. This survey provides an overview of PAC-Bayes performance bounds for bandit problems and an experimental comparison of these bounds. Our experimental comparison has revealed that available PAC-Bayes upper bounds on the cumulative regret are loose, whereas available PAC-Bayes lower bounds on the expected reward can be surprisingly tight. We found that an offline contextual bandit algorithm that learns a policy by optimising a PAC-Bayes bound was able to learn randomised neural network polices with competitive expected reward and non-vacuous performance guarantees.

   @article{flyn2022pacbayes,
     title = {PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison},
     author = {Flynn, H. and Reeb, D. and Kandemir, M. and Peters, J.},
     journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
     year = {2023},
   }

10. MDPI

    ALReg: Registration of 3D Point Clouds Using Active Learning

    Y.H. Sahin, O. Karabacak, M. Kandemir, and 1 more author

    MDPI Applied Sciences 2023

    Abs Bib HTML

    After the success of deep learning in point cloud segmentation and classification tasks, it has also been adopted as common practice in point cloud registration applications. State-of-the-art point cloud registration methods generally deal with this problem as a regression task to find the underlying rotation and translation between two point clouds. However, given two point clouds, the transformation between them could be calculated using only definitive point subsets from each cloud. Furthermore, training time is still a major problem among the current registration networks, whereas using a selective approach to define the informative point subsets can lead to reduced network training times. To that end, we developed ALReg, an active learning procedure to select a limited subset of point clouds to train the network. Each of the point clouds in the training set is divided into superpoints (small pieces of each cloud) and the training process is started with a small amount of them. By actively selecting new superpoints and including them in the training process, only a prescribed amount of data is used, hence the time needed to converge drastically decreases. We used DeepBBS, FMR, and DCP methods as our baselines to prove our proposed ALReg method. We trained DeepBBS and DCP on the ModelNet40 dataset and FMR on the 7Scenes dataset. Using 25% of the training data for ModelNet and 4% for the 7Scenes, better or similar accuracy scores are obtained in less than 20% of their original training times. The trained models are also tested on the 3DMatch dataset and better results are obtained than the original FMR training procedure.

    @article{sahin2023alreg,
      title = {ALReg: Registration of 3D Point Clouds Using Active Learning},
      author = {Sahin, Y.H. and Karabacak, O. and Kandemir, M. and Unal, G.},
      journal = {MDPI Applied Sciences},
      year = {2023},
      url = {https://www.mdpi.com/2076-3417/13/13/7422},
    }

11. TMLR

    Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems

    A. Look, B. Rakitsch, M. Kandemir, and 1 more author

    Transactions on Machine Learning Research 2023

    Abs Bib HTML

    Graph neural networks are often used to model interacting dynamical systems since they gracefully scale to systems with a varying and high number of agents. While there has been much progress made for deterministic interacting systems, modeling is much more challenging for stochastic systems in which one is interested in obtaining a predictive distribution over future trajectories. Existing methods are either computationally slow since they rely on Monte Carlo sampling or make simplifying assumptions such that the predictive distribution is unimodal. In this work, we present a deep state-space model which employs graph neural networks in order to model the underlying interacting dynamical system. The predictive distribution is multimodal and has the form of a Gaussian mixture model, where the moments of the Gaussian components can be computed via deterministic moment matching rules. Our moment matching scheme can be exploited for sample-free inference leading to more efficient and stable training compared to Monte Carlo alternatives. Furthermore, we propose structured approximations to the covariance matrices of the Gaussian components in order to scale up to systems with many agents. We benchmark our novel framework on two challenging autonomous driving datasets. Both confirm the benefits of our method compared to state-of-the-art methods. We further demonstrate the usefulness of our individual contributions in a carefully designed ablation study and provide a detailed empirical runtime analysis of our proposed covariance approximations.

    @article{look2023cheap,
      title = {Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems},
      author = {Look, A. and Rakitsch, B. and Kandemir, M. and Peters, J.},
      journal = {Transactions on Machine Learning Research},
      year = {2023},
      url = {https://openreview.net/forum?id=dqgdBy4Uv5&noteId=xKtcWgwdxX},
    }

12. TMLR

    Meta Continual Learning on Graphs with Experience Replay

    A. Unal, A. Akgül, M. Kandemir, and 1 more author

    2023

    Abs Bib HTML

    Continual learning is a machine learning approach where the challenge is that a constructed learning model executes incoming tasks while maintaining its performance over the earlier tasks. In order to address this issue, we devise a technique that combines two uniquely important concepts in machine learning, namely "replay buffer" and "meta learning", aiming to exploit the best of two worlds. In this method, the model weights are initially computed by using the current task dataset. Next, the dataset of the current task is merged with the stored samples from the earlier tasks and the model weights are updated using the combined dataset. This aids in preventing the model weights converging to the optimal parameters of the current task and enables the preservation of information from earlier tasks. We choose to adapt our technique to graph data structure and the task of node classification on graphs. We introduce MetaCLGraph, which outperforms the baseline methods over various graph datasets including Citeseer, Corafull, Arxiv, and Reddit. This method illustrates the potential of combining replay buffer and meta learning in the field of continual learning on graphs.

    @article{unal2023meta,
      title = {Meta Continual Learning on Graphs with Experience Replay},
      author = {Unal, A. and Akg{\"u}l, A. and Kandemir, M. and Unal, G.},
      booktitle = {Transactions on Machine Learning Research},
      year = {2023},
      url = {https://openreview.net/forum?id=8tnrh56P5W},
    }

2022

1. NeurIPS

   Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs

   C. Yildiz, M. Kandemir, and B. Rakitsch

   In Neural Information Processing Systems 2022

   Abs Bib HTML

   We study for the first time uncertainty-aware modeling of continuous-time dynamics of interacting objects. We introduce a new model that decomposes independent dynamics of single objects accurately from their interactions. By employing latent Gaussian process ordinary differential equations, our model infers both independent dynamics and their interactions with reliable uncertainty estimates. In our formulation, each object is represented as a graph node and interactions are modeled by accumulating the messages coming from neighboring objects. We show that efficient inference of such a complex network of variables is possible with modern variational sparse Gaussian process inference techniques. We empirically demonstrate that our model improves the reliability of long-term predictions over neural network based alternatives and it successfully handles missing dynamic or static information. Furthermore, we observe that only our model can successfully encapsulate independent dynamics and interaction information in distinct functions and show the benefit from this disentanglement in extrapolation scenarios..

   @inproceedings{yildiz2022learning,
     title = {Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs},
     author = {Yildiz, C. and Kandemir, M. and Rakitsch, B.},
     booktitle = {Neural Information Processing Systems},
     year = {2022},
   }

2. ICLR

   Evidential Turing Processes

   M. Kandemir, A. Akgül, M. Haussmann, and 1 more author

   In International Conference on Learning Representations 2022

   Abs Bib HTML Code

   A probabilistic classifier with reliable predictive uncertainties i) fits successfully to the target domain data, ii) provides calibrated class probabilities in difficult regions of the target domain (e.g. class overlap), and iii) accurately identifies queries coming out of the target domain and reject them. We introduce an original combination of Evidential Deep Learning, Neural Processes, and Neural Turing Machines capable of providing all three essential properties mentioned above for total uncertainty quantification. We observe our method on three image classification benchmarks to consistently improve the in-domain uncertainty quantification, out-of-domain detection, and robustness against input perturbations with one single model. Our unified solution delivers an implementation-friendly and computationally efficient recipe for safety clearance and provides intellectual economy to an investigation of algorithmic roots of epistemic awareness in deep neural nets.

   @inproceedings{kandemir2022evidential,
     title = {Evidential Turing Processes },
     author = {Kandemir, M. and Akgül, A. and Haussmann, M. and Unal, G.},
     booktitle = {International Conference on Learning Representations},
     year = {2022},
     url = {https://openreview.net/forum?id=84NMXTHYe-},
   }

3. T-PAMI

   A Deterministic Approximation to Neural SDEs

   A. Look, M. Kandemir, B. Rakitsch, and 1 more author

   IEEE Transactions on Pattern Analysis and Machine Intelligence 2022

   Abs Bib HTML

   Neural Stochastic Differential Equations (NSDEs) model the drift and diffusion functions of a stochastic process as neural networks. While NSDEs are known to make accurate predictions, their uncertainty quantification properties haven been remained unexplored so far. We report the empirical finding that obtaining well-calibrated uncertainty estimations from NSDEs is computationally prohibitive. As a remedy, we develop a computationally affordable deterministic scheme which accurately approximates the transition kernel, when dynamics is governed by a NSDE. Our method introduces a bidimensional moment matching algorithm: vertical along the neural net layers and horizontal along the time direction, which benefits from an original combination of effective approximations. Our deterministic approximation of the transition kernel is applicable to both training and prediction. We observe in multiple experiments that the uncertainty calibration quality of our method can be matched by Monte Carlo sampling only after introducing high computation cost. Thanks to the numerical stability of deterministic training, our method also provides improvement in prediction accuracy.

   @article{look2022adeterministic,
     title = {A Deterministic Approximation to Neural SDEs},
     author = {Look, A. and Kandemir, M. and Rakitsch, B. and Peters, J.},
     journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
     year = {2022},
     publisher = {IEEE},
   }

4. L4DC

   Traversing Time with Multi-Resolution Gaussian Process State-Space Models

   K. Longi, J. Lindinger, O. Duennbier, and 3 more authors

   In Proceedings of The 4th Annual Learning for Dynamics and Control Conference 2022

   Abs Bib HTML

   Gaussian Process state-space models capture complex temporal dependencies in a principled manner by placing a Gaussian Process prior on the transition function. These models have a natural interpretation as discretized stochastic differential equations, but inference for long sequences with fast and slow transitions is difficult. Fast transitions need tight discretizations whereas slow transitions require backpropagating the gradients over long subtrajectories. We propose a novel Gaussian process state-space architecture composed of multiple components, each trained on a different resolution, to model effects on different timescales. The combined model allows traversing time on adaptive scales, providing efficient inference for arbitrarily long sequences with complex dynamics. We benchmark our novel method on semi-synthetic data and on an engine modeling task. In both experiments, our approach compares favorably against its state-of-the-art alternatives that operate on a single time-scale only.

   @inproceedings{lungi2022gpssm,
     title = {Traversing Time with Multi-Resolution Gaussian Process State-Space Models},
     author = {Longi, K. and Lindinger, J. and Duennbier, O. and Kandemir, M. and Klami, A. and Rakitsch, B},
     booktitle = {Proceedings of The 4th Annual Learning for Dynamics and Control Conference},
     year = {2022},
   }

5. DMKD

   PAC-Bayesian lifelong learning for multi-armed bandits

   H. Flynn, D. Reeb, M. Kandemir, and 1 more author

   Data Mining and Knowledge Discovery 2022

   Abs Bib HTML

   We present a PAC-Bayesian analysis of lifelong learning. In the lifelong learning problem, a sequence of learning tasks is observed one-at-a-time, and the goal is to transfer information acquired from previous tasks to new learning tasks. We consider the case when each learning task is a multi-armed bandit problem. We derive lower bounds on the expected average reward that would be obtained if a given multi-armed bandit algorithm was run in a new task with a particular prior and for a set number of steps. We propose lifelong learning algorithms that use our new bounds as learning objectives. Our proposed algorithms are evaluated in several lifelong multi-armed bandit problems and are found to perform better than a baseline method that does not use generalisation bounds.

   @article{flynn2022pac,
     title = {PAC-Bayesian lifelong learning for multi-armed bandits},
     author = {Flynn, H. and Reeb, D. and Kandemir, M. and Peters, J.},
     journal = {Data Mining and Knowledge Discovery},
     year = {2022},
     publisher = {Springer},
     url = {https://link.springer.com/article/10.1007/s10618-022-00825-4},
   }