Promises

Barret Schloerke || Maximize computing resources using future_promise() || RStudio

00:00 Introduction 01:45 Setting up a multisession using the future package 02:05 Simulation using two workers 04:14 Simulation using 10 workers 05:20 What happens when we run out of workers? 05:35 How Shiny handles future processes like promises 07:16 Introduction to future_promise() 07:45 Demo of the promises package 09:21 Setting the number of workers 10:40 Demo of processing without future_promise() 14:11 Wrapping a slow calculation in a future() 14:53 Demo of processing using Plumber 16:25 Considerations on the number of cores to use 17:21 What happens if we run out of workers? 19:44 Decrease in execution times using future_promise()

In an ideal situation, the number of available future workers (future::nbrOfFreeWorkers()) is always more than the number of future::future() jobs. However, if a future job is attempted when the number of free workers is 0, then future will block the current R session until one becomes available.

The advantage of using future_promise() over future::future() is that even if there aren’t future workers available, the future is scheduled to be done when workers become available via promises. In other words, future_promise() ensures the main R thread isn’t blocked when a future job is requested and can’t immediately perform the work (i.e., the number of jobs exceeds the number of workers).