This vignette demonstrates how to access most of data stored in a stanfit object. A stanfit object (an object of class "stanfit"
) contains the output derived from fitting a Stan model using Markov chain Monte Carlo or one of Stan’s variational approximations (meanfield or full-rank). Throughout the document we’ll use the stanfit object obtained from fitting the Eight Schools example model:
Warning: There were 9 divergent transitions after warmup. Increasing adapt_delta above 0.8 may help. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
Warning: Examine the pairs() plot to diagnose sampling problems
[1] "stanfit"
attr(,"package")
[1] "rstan"
There are several functions that can be used to access the draws from the posterior distribution stored in a stanfit object. These are extract
, as.matrix
, as.data.frame
, and as.array
, each of which returns the draws in a different format.
The extract
function (with its default arguments) returns a list with named components corresponding to the model parameters.
[1] "mu" "tau" "eta" "theta" "lp__"
In this model the parameters mu
and tau
are scalars and theta
is a vector with eight elements. This means that the draws for mu
and tau
will be vectors (with length equal to the number of post-warmup iterations times the number of chains) and the draws for theta
will be a matrix, with each column corresponding to one of the eight components:
[1] 5.694755 6.452653 6.847224 8.903306 -2.952371 -1.603703
[1] 12.052607 11.087726 1.641105 5.426428 11.355279 20.811877
iterations [,1] [,2] [,3] [,4] [,5] [,6]
[1,] 14.9090095 -3.06832443 -8.620168 4.708655 11.4515639 1.07188
[2,] 21.3211832 12.88066818 6.659683 1.202132 0.8229435 21.27804
[3,] 8.2556800 7.51413388 8.609569 5.220954 6.5325973 5.05246
[4,] 8.5752749 14.87720071 12.870250 9.245411 -3.8228270 10.80066
[5,] 15.2055224 0.09275822 -18.262625 3.216336 -2.9307794 -14.95347
[6,] -0.1360786 18.17648719 -2.362693 7.789155 -1.6366367 13.85463
iterations [,7] [,8]
[1,] 19.835612 13.209071
[2,] 18.014502 11.470222
[3,] 7.832636 6.022004
[4,] 13.140196 8.155220
[5,] 3.028193 -3.264669
[6,] 20.217160 16.980962
The as.matrix
, as.data.frame
, and as.array
functions can also be used to retrieve the posterior draws from a stanfit object:
[1] "mu" "tau" "eta[1]" "eta[2]" "eta[3]" "eta[4]"
[7] "eta[5]" "eta[6]" "eta[7]" "eta[8]" "theta[1]" "theta[2]"
[13] "theta[3]" "theta[4]" "theta[5]" "theta[6]" "theta[7]" "theta[8]"
[19] "lp__"
[1] "mu" "tau" "eta[1]" "eta[2]" "eta[3]" "eta[4]"
[7] "eta[5]" "eta[6]" "eta[7]" "eta[8]" "theta[1]" "theta[2]"
[13] "theta[3]" "theta[4]" "theta[5]" "theta[6]" "theta[7]" "theta[8]"
[19] "lp__"
$iterations
NULL
$chains
[1] "chain:1" "chain:2" "chain:3" "chain:4"
$parameters
[1] "mu" "tau" "eta[1]" "eta[2]" "eta[3]" "eta[4]"
[7] "eta[5]" "eta[6]" "eta[7]" "eta[8]" "theta[1]" "theta[2]"
[13] "theta[3]" "theta[4]" "theta[5]" "theta[6]" "theta[7]" "theta[8]"
[19] "lp__"
The as.matrix
and as.data.frame
methods essentially return the same thing except in matrix and data frame form, respectively. The as.array
method returns the draws from each chain separately and so has an additional dimension:
[1] 4000 19
[1] 4000 19
[1] 1000 4 19
By default all of the functions for retrieving the posterior draws return the draws for all parameters (and generated quantities). The optional argument pars
(a character vector) can be used if only a subset of the parameters is desired, for example:
parameters
iterations mu theta[1]
[1,] 4.1725580 4.020880
[2,] 12.0268782 13.647565
[3,] 1.1814306 1.302227
[4,] -2.0887965 5.861635
[5,] 1.4794296 8.658329
[6,] 0.9507218 3.702802
Summary statistics are obtained using the summary
function. The object returned is a list with two components:
[1] "summary" "c_summary"
In fit_summary$summary
all chains are merged whereas fit_summary$c_summary
contains summaries for each chain individually. Typically we want the summary for all chains merged, which is what we’ll focus on here.
The summary is a matrix with rows corresponding to parameters and columns to the various summary quantities. These include the posterior mean, the posterior standard deviation, and various quantiles computed from the draws. The probs
argument can be used to specify which quantiles to compute and pars
can be used to specify a subset of parameters to include in the summary.
For models fit using MCMC, also included in the summary are the Monte Carlo standard error (se_mean
), the effective sample size (n_eff
), and the R-hat statistic (Rhat
).
mean se_mean sd 2.5% 25%
mu 8.15177782 0.16858038 5.2685829 -1.8514752 4.7817786
tau 6.70279195 0.18465417 5.5713959 0.3485009 2.6007487
eta[1] 0.37443066 0.01457896 0.9220543 -1.4871548 -0.2388864
eta[2] -0.02216593 0.01598622 0.8993298 -1.8333619 -0.6086056
eta[3] -0.18919218 0.01433966 0.9069197 -1.9707087 -0.7792353
eta[4] -0.04024442 0.01390925 0.8796983 -1.7697027 -0.6148987
eta[5] -0.37472740 0.01399903 0.8853763 -2.0563406 -0.9747813
eta[6] -0.22710926 0.01369881 0.8663885 -1.9287004 -0.8141525
eta[7] 0.31342296 0.01559546 0.8890755 -1.4680378 -0.2459586
eta[8] 0.05357563 0.01429246 0.9039343 -1.7745936 -0.5532847
theta[1] 11.46791468 0.18027685 8.2568570 -1.5614333 5.8682250
theta[2] 7.93366075 0.09939962 6.2865837 -4.6494106 4.1526104
theta[3] 6.25556582 0.14091000 7.7254973 -11.4342916 2.2930675
theta[4] 7.70693364 0.10496028 6.6382710 -5.7249868 3.6112703
theta[5] 5.10737764 0.10108455 6.3931482 -8.8003526 1.3849675
theta[6] 6.17945953 0.10788734 6.8233943 -8.0298559 2.0583146
theta[7] 10.72578621 0.10960942 6.9323083 -1.4177650 6.1333457
theta[8] 8.70872644 0.17634943 7.8513292 -5.8934066 4.0180731
lp__ -39.43629272 0.07476220 2.5376437 -45.0652524 -41.0178537
50% 75% 97.5% n_eff Rhat
mu 7.981444644 11.4370762 18.669592 976.7278 1.0033494
tau 5.366905256 9.1369370 21.341089 910.3530 1.0036477
eta[1] 0.408442788 0.9935699 2.139521 4000.0000 1.0005635
eta[2] -0.003070435 0.5716305 1.762240 3164.8008 0.9996797
eta[3] -0.202227052 0.4181758 1.577805 4000.0000 1.0004539
eta[4] -0.050170812 0.5296182 1.749207 4000.0000 1.0001248
eta[5] -0.393897607 0.2097872 1.418852 4000.0000 1.0010889
eta[6] -0.238675934 0.3355586 1.496226 4000.0000 1.0009867
eta[7] 0.333160370 0.8816249 2.042002 3249.9813 0.9996716
eta[8] 0.072793976 0.6468558 1.810305 4000.0000 0.9996117
theta[1] 10.303783645 15.6528181 31.603109 2097.7301 1.0018469
theta[2] 7.806813918 11.7918802 21.118962 4000.0000 0.9997625
theta[3] 6.783178666 10.8876873 20.786863 3005.8637 0.9997555
theta[4] 7.743598569 11.6889290 21.598172 4000.0000 0.9995479
theta[5] 5.484240702 9.4468291 16.632160 4000.0000 0.9995460
theta[6] 6.473025664 10.5029694 18.940775 4000.0000 1.0006208
theta[7] 10.070931619 14.8001383 26.477181 4000.0000 0.9997880
theta[8] 8.211781728 13.0045356 26.436258 1982.1579 1.0000548
lp__ -39.244564224 -37.6087564 -35.155461 1152.1186 1.0037937
If, for example, we wanted the only quantiles included to be 10% and 90%, and for only the parameters included to be mu
and tau
, we would specify that like this:
mu_tau_summary <- summary(fit, pars = c("mu", "tau"), probs = c(0.1, 0.9))$summary
print(mu_tau_summary)
mean se_mean sd 10% 90% n_eff Rhat
mu 8.151778 0.1685804 5.268583 1.762455 14.44875 976.7278 1.003349
tau 6.702792 0.1846542 5.571396 1.079909 13.98739 910.3530 1.003648
Since mu_tau_summary
is a matrix we can pull out columns using their names:
10% 90%
mu 1.762455 14.44875
tau 1.079909 13.98739
For models fit using MCMC the stanfit object will also contain the values of parameters used for the sampler. The get_sampler_params
function can be used to access this information.
The object returned by get_sampler_params
is a list with one component (a matrix) per chain. Each of the matrices has number of columns corresponding to the number of sampler parameters and the column names provide the parameter names. The optional argument inc_warmup (defaulting to TRUE
) indicates whether to include the warmup period.
sampler_params <- get_sampler_params(fit, inc_warmup = FALSE)
sampler_params_chain1 <- sampler_params[[1]]
colnames(sampler_params_chain1)
[1] "accept_stat__" "stepsize__" "treedepth__" "n_leapfrog__"
[5] "divergent__" "energy__"
To do things like calculate the average value of accept_stat__
for each chain (or the maximum value of treedepth__
for each chain if using the NUTS algorithm, etc.) the sapply
function is useful as it will apply the same function to each component of sampler_params
:
mean_accept_stat_by_chain <- sapply(sampler_params, function(x) mean(x[, "accept_stat__"]))
print(mean_accept_stat_by_chain)
[1] 0.7100091 0.9103609 0.8373795 0.8662187
max_treedepth_by_chain <- sapply(sampler_params, function(x) max(x[, "treedepth__"]))
print(max_treedepth_by_chain)
[1] 4 5 5 5
The Stan program itself is also stored in the stanfit object and can be accessed using get_stancode
:
The object code
is a single string and is not very intelligible when printed:
[1] "data {\n int<lower=0> J; // number of schools \n real y[J]; // estimated treatment effects\n real<lower=0> sigma[J]; // s.e. of effect estimates \n}\nparameters {\n real mu; \n real<lower=0> tau;\n vector[J] eta;\n}\ntransformed parameters {\n vector[J] theta;\n theta = mu + tau * eta;\n}\nmodel {\n target += normal_lpdf(eta | 0, 1);\n target += normal_lpdf(y | theta, sigma);\n}"
attr(,"model_name2")
[1] "schools"
A readable version can be printed using cat
:
data {
int<lower=0> J; // number of schools
real y[J]; // estimated treatment effects
real<lower=0> sigma[J]; // s.e. of effect estimates
}
parameters {
real mu;
real<lower=0> tau;
vector[J] eta;
}
transformed parameters {
vector[J] theta;
theta = mu + tau * eta;
}
model {
target += normal_lpdf(eta | 0, 1);
target += normal_lpdf(y | theta, sigma);
}
The get_inits
function returns initial values as a list with one component per chain. Each component is itself a (named) list containing the initial values for each parameter for the corresponding chain:
$mu
[1] 0.9252574
$tau
[1] 0.1526429
$eta
[1] 0.4727558 1.1829434 -1.3894450 1.6758501 -0.8801524 -0.7616199
[7] 1.6175855 -1.4346626
$theta
[1] 0.9974202 1.1058254 0.7131685 1.1810641 0.7909084 0.8090015 1.1721704
[8] 0.7062663
The get_seed
function returns the (P)RNG seed as an integer:
[1] 1172774004
The get_elapsed_time
function returns a matrix with the warmup and sampling times for each chain:
warmup sample
chain:1 0.032175 0.020417
chain:2 0.032019 0.034921
chain:3 0.032214 0.029322
chain:4 0.034210 0.032532