Other

Probabilistic Deep Learning

Showing of

Easy PixelCNN with tfprobability

PixelCNN is a deep learning architecture - or bundle of architectures - designed to generate highly realistic-looking images. To use it, no reverse-engineering of arXiv papers or search for reference implementations is required: TensorFlow Probability and its R wrapper, tfprobability, now include a PixelCNN distribution that can be used to train a straightforwardly-defined neural network in a parameterizable way

Sigrid Keydana

May 29, 2020

keras3 Machine Learning Ai Image Recognition Image Processing TensorFlow Keras Probabilistic Machine Learning Probabilistic Deep Learning Generative Models

blog post

Infinite surprise - the iridescent personality of Kullback-Leibler divergence

Infinite surprise - the iridescent personality of Kullback-Leibler divergence

Kullback-Leibler divergence is not just used to train variational autoencoders or Bayesian networks (and not just a hard-to-pronounce thing). It is a fundamental concept in information theory, put to use in a vast range of applications. Most interestingly, it’s not always about constraint, regularization or compression. Quite on the contrary, sometimes it is about novelty, discovery and surprise

Sigrid Keydana

Feb 19, 2020

Machine Learning Best Practices Ai Probabilistic Machine Learning Probabilistic Deep Learning Concepts

blog post

Gaussian Process Regression with tfprobability

Gaussian Process Regression with tfprobability

Continuing our tour of applications of TensorFlow Probability (TFP), after Bayesian Neural Networks, Hamiltonian Monte Carlo and State Space Models, here we show an example of Gaussian Process Regression. In fact, what we see is a rather “normal” Keras network, defined and trained in pretty much the usual way, with TFP’s Variational Gaussian Process layer pulling off all the magic

Sigrid Keydana

Dec 10, 2019

keras3 Machine Learning Ai Probabilistic Machine Learning Probabilistic Deep Learning TensorFlow Keras

blog post

Variational convnets with tfprobability

In a Bayesian neural network, layer weights are distributions, not tensors. Using tfprobability, the R wrapper to TensorFlow Probability, we can build regular Keras models that have probabilistic layers, and thus get uncertainty estimates “for free”. In this post, we show how to define, train and obtain predictions from a probabilistic convolutional neural network

Sigrid Keydana

Nov 13, 2019

keras3 Machine Learning Ai Probabilistic Machine Learning Probabilistic Deep Learning Time Series TensorFlow Keras

blog post

tfprobability 0.8 on CRAN: Now how can you use it?

tfprobability 0.8 on CRAN: Now how can you use it?

Part of the r-tensorflow ecosystem, tfprobability is an R wrapper to TensorFlow Probability, the Python probabilistic programming framework developed by Google. We take the occasion of tfprobability’s acceptance on CRAN to give a high-level introduction, highlighting interesting use cases and applications

Sigrid Keydana

Nov 7, 2019

keras3 Machine Learning Ai Probabilistic Machine Learning Probabilistic Deep Learning Packages Releases TensorFlow Keras

blog post

Dynamic linear models with tfprobability

Dynamic linear models with tfprobability

Previous posts featuring tfprobability - the R interface to TensorFlow Probability - have focused on enhancements to deep neural networks (e.g., introducing Bayesian uncertainty estimates) and fitting hierarchical models with Hamiltonian Monte Carlo. This time, we show how to fit time series using dynamic linear models (DLMs), yielding posterior predictive forecasts as well as the smoothed and filtered estimates from the Kálmán filter

Sigrid Keydana

Jun 24, 2019

tfprobability tensorflow Machine Learning Ai Probabilistic Machine Learning Probabilistic Deep Learning Time Series

blog post

Adding uncertainty estimates to Keras models with tfprobability

Adding uncertainty estimates to Keras models with tfprobability

As of today, there is no mainstream road to obtaining uncertainty estimates from neural networks. All that can be said is that, normally, approaches tend to be Bayesian in spirit, involving some way of putting a prior over model weights. This holds true as well for the method presented in this post: We show how to use tfprobability, the R interface to TensorFlow Probability, to add uncertainty estimates to a Keras model in an elegant and conceptually plausible way

Sigrid Keydana

Jun 5, 2019

keras3 Machine Learning Best Practices Ai Probabilistic Machine Learning Probabilistic Deep Learning TensorFlow Keras Concepts

blog post

Experimenting with autoregressive flows in TensorFlow Probability

Experimenting with autoregressive flows in TensorFlow Probability

Continuing from the recent introduction to bijectors in TensorFlow Probability (TFP), this post brings autoregressivity to the table. Using TFP through the new R package tfprobability, we look at the implementation of masked autoregressive flows (MAF) and put them to use on two different datasets

Sigrid Keydana

Apr 24, 2019

keras3 Machine Learning Ai Probabilistic Machine Learning Probabilistic Deep Learning Generative Models TensorFlow Keras

blog post

Getting into the flow: Bijectors in TensorFlow Probability

Getting into the flow: Bijectors in TensorFlow Probability

Normalizing flows are one of the lesser known, yet fascinating and successful architectures in unsupervised deep learning. In this post we provide a basic introduction to flows using tfprobability, an R wrapper to TensorFlow Probability. Upcoming posts will build on this, using more complex flows on more complex data

Sigrid Keydana

Apr 5, 2019

keras3 Machine Learning Best Practices Ai Probabilistic Machine Learning Probabilistic Deep Learning TensorFlow Keras Concepts Generative Models

blog post

Discrete Representation Learning with VQ-VAE and TensorFlow Probability

Discrete Representation Learning with VQ-VAE and TensorFlow Probability

Mostly when thinking of Variational Autoencoders (VAEs), we picture the prior as an isotropic Gaussian. But this is by no means a necessity. The Vector Quantised Variational Autoencoder (VQ-VAE) described in van den Oord et al’s “Neural Discrete Representation Learning” features a discrete latent space that allows to learn impressively concise latent representations. In this post, we combine elements of Keras, TensorFlow, and TensorFlow Probability to see if we can generate convincing letters resembling those in Kuzushiji-MNIST

Sigrid Keydana

Jan 24, 2019

keras3 Machine Learning Ai TensorFlow Keras Probabilistic Machine Learning Probabilistic Deep Learning Generative Models

blog post

Getting started with TensorFlow Probability from R

Getting started with TensorFlow Probability from R

TensorFlow Probability offers a vast range of functionality ranging from distributions over probabilistic network layers to probabilistic inference. It works seamlessly with core TensorFlow and (TensorFlow) Keras. In this post, we provide a short introduction to the distributions layer and then, use it for sampling and calculating probabilities in a Variational Autoencoder

Sigrid Keydana

Jan 8, 2019

keras3 Machine Learning Ai TensorFlow Keras Probabilistic Machine Learning Probabilistic Deep Learning Generative Models

blog post

You sure? A Bayesian approach to obtaining uncertainty estimates from neural networks

You sure? A Bayesian approach to obtaining uncertainty estimates from neural networks

In deep learning, there is no obvious way of obtaining uncertainty estimates. In 2016, Gal and Ghahramani proposed a method that is both theoretically grounded and practical: use dropout at test time. In this post, we introduce a refined version of this method (Gal et al. 2017) that has the network itself learn how uncertain it is

Sigrid Keydana

Nov 12, 2018

keras3 Machine Learning Ai Image Recognition Image Processing Probabilistic Machine Learning Probabilistic Deep Learning TensorFlow Keras