Many years ago, Jouni and I wrote an unpublished paper called Fully Bayesian Computing:
A fully Bayesian computing environment calls for the possibility of defining vector and array objects that may contain both random and deterministic quantities, and syntax rules that allow treating these objects much like any variables or numeric arrays. Working within the statistical package R, we introduce a new object-oriented framework based on a new random variable data type that is implicitly represented by simulations.
A few years later we published an updated version of the paper under the title, Manipulating and summarizing posterior simulations using random variable objects. Here’s the background:
In the Bayesian paradigm, unknown quantities have probability distributions and are thus random variables. Observed values are just realizations of random variables, and constants may be thought of as random variables with point mass distributions. In mathematical notation, we deal with objects that are random variables, but in practice these objects are approximated by vectors of numbers, that is, simulations. Consequently, when programming for manipulating simulations of unknown quantities, we must write code to manipulate arrays of numeric constants.
And here’s the challenge:
Arrays of simulations are cumbersome objects to work with. Functions that work with vectors will not in general work with matrices, so special versions of the functions need to be written to accommodate matrices of simulations as arguments. For example, a scalar-valued random variable becomes a vector of simulations, and a random vector becomes a matrix of simulations. . . . The usual approach now is to use loops and vector-matrix computations to manipulate the matrix of posterior simulations. This approach is general but awkward and far from transparent . . .
As part of this effort, Jouni wrote a package in R called “rv” that allows direct manipulation of random variable objects, so for example if alpha and beta are random variables, each a vector of length J, and gamma is a scalar, then you can compute something like delta = (alpha + gamma)/beta, which would be a vector of length J. Internally, alpha, beta, and delta would be stored as S x K arrays, and gamma would be stored as an array of length S, where S is the number of simulation draws in your simulation frame, but the user would be able to work with these without having to keep track of that indexing; it would all be implicit.
The rv package was just a demonstration, but then more recently Paul Bürkner, Jonah Gabry, Matthew Kay, and Aki Vehtari implemented the idea in the “posterior” package in R. They call it rvar: The Random Variable Datatype. I still haven’t integrated rvar() into my Bayesian workflow, but I’d like to do it.
The point I want to make here is that this random variable datatype moves us closer to what I’d call “truly probabilistic programming.”
Here’s what I mean by that.
Stan (and Bugs, PyMC, etc.) are called “probabilistic programming languages,” characterized by Wikipedia as “based on the declarative specification of probabilistic models, for which inference is performed automatically.” Fair enough. In Stan, the variables defined in the parameters block, transformed parameters block, model block, and generated parameters block represent random variables with a joint density that is specified by the Stan program.
But, although this is called a probabilistic program as the term is used, it doesn’t fully feel like probabilistic programming in the sense of allowing me to grab random variables and work with them directly. I can do all that within the Stan program in the generated quantities block, but at a certain remove. Stan and other probabilistic programming languages are set up to run in batch mode: you specify a model and then the inference engine runs and spits out posterior simulations (or some other distributional summary).
To me, a truly probabilistic programming environment would allow me to interactively work with random variables, and to work with them as random variables, not as arrays of simulation draws. I want to be able to multiply two matrices without having to loop through the simulations. Part of this is about clean code, but more generally it’s about workflow. Any mathematical or graphical operations we can do on numbers, we should be able to do with random variables, with the simulations all hiding under the hood. Kind of by analogy to how we can do floating-point math on the computer without (usually) having to worry about how all these digits are being stored.
This will require tools such as rvar() and also some experience with the workflow. The payoff is . . . you could call it “better postprocessing of Bayesian simulations,” but I think that would understate the potential benefits of being able to work directl with uncertainty without that extra array dimension getting in your face at every step.

