Hi,
I have 100s of FittedPumasModels I want to evaluate so I’m trying to squeeze all the performance that I can. One of the things I ran into was differences in performance in empirical_bayes with different LikelihoodApproximation, even though the docs say: The empirical bayes estimates will be obtained using the `LaplaceI` approximation regardless of the value of the `approx` argument except for `NaivePooled` where an empty `NamedTuple` is returned.
If I do
# results_df is a loaded DataFrame with previously trained models, params, pops, etc.
const m = results_df[1, :model]
const p = results_df[1, :params]
const d = results_df[1, :validation_pop]
@benchmark empirical_bayes(m, d, p, FOCE())
@benchmark empirical_bayes(m, d, p, LaplaceI())
the results in my case are
julia> @benchmark empirical_bayes(m, d, p, FOCE())
BenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
Single result which took 229.822 s (44.20% GC) to evaluate,
with a memory estimate of 299.70 GiB, over 941509319 allocations.
@benchmark empirical_bayes(m, d, p, LaplaceI())
julia> @benchmark empirical_bayes(m, d, p, LaplaceI())
BenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
Single result which took 359.108 s (61.62% GC) to evaluate,
with a memory estimate of 730.39 GiB, over 1321230405 allocations.
The estimated EBEs in the two cases match to float precision, but one takes close to 50% longer and I don’t quite understand why, if they’re both estimated using LaplaceI()?
Are there any other non-obvious things to try to speed up empirical_bayes? There’s diffeq_options, but not sure they’d make a big difference as the defaults are pretty good.