Deserialize fitted model in Quarto notebook

1

I’m trying to load a saved fit object (saved with serialize) in quarto notebook and use that for plotting etc.
It does run fine when interactively executing codes in REPL, but it fails when running Quarto: Preview. Can that be engine: julia issue?
Below is the model fit code

using Pumas
using DataFrames
using Serialization

model = @model begin
  @param begin
    CL ∈ RealDomain()
    Vc ∈ RealDomain()
    σ ∈ RealDomain()
  end
  @dynamics begin
    Central' = - (CL / Vc) * Central
  end
  @derived begin
    conc_model := @. Central / Vc
    dv ~ @. Normal(conc_model, abs(conc_model) * σ)
  end
end

params = (;
  CL=1.0,
  Vc=1.0,
  σ=0.02
)

data_df = DataFrame(
    id = [1, 1, 1, 1, 1, 1],
    time = [0.0, 1.0, 2.0, 4.0, 8.0, 12.0],
    amt = [100.0, 0.0, 0.0, 0.0, 0.0, 0.0],
    dv = [missing, 10.0, 8.0, 5.0, 2.0, 1.0],
    cmt = [1, 1, 1, 1, 1, 1],
    evid = [1, 0, 0, 0, 0, 0]
)

data = read_pumas(data_df, id=:id)

fpm = fit(model, data, params, Pumas.NaivePooled())
serialize("test.jls", fpm)
2

And here is the qmd file, it doesn’t get formatted correctly but essentially each chunk is surrounded by ```{julia} and ```

---
title: "Test"
engine: julia
format: html
---

```{julia}
#| label: setup

using Pumas
using PumasPlots
using Distributions
using Serialization

fpm = deserialize("test.jls")
subject_fits(predict(fpm; obstimes=0:0.5:15))
3

Forgot to add error message…

Error 1 of 1
@ /home/jrun/data/code/test/test2.qmd:16
UndefVarError: `Distribution` not defined

Stacktrace:
  [1] (::Serialization.__deserialized_types__.var"#8#13")(_pre::Returns{@NamedTuple{CL::Float64, Vc::Float64, ##A##::StaticArraysCore.SMatrix{1, 1, Float64, 1}, ##b##::StaticArraysCore.SVector{1, Int64}}}, _sol::Pumas.PKPDAnalyticalSolution{StaticArraysCore.SVector{1, Float64}, 2, Vector{StaticArraysCore.SVector{1, Float64}}, Vector{Float64}, Vector{StaticArraysCore.SVector{1, Float64}}, Vector{StaticArraysCore.SVector{1, Float64}}, Returns{@NamedTuple{##A##::StaticArraysCore.SMatrix{1, 1, Float64, 1}, ##b##::StaticArraysCore.SVector{1, Int64}}}, Pumas.AnalyticalPKPDProblem{StaticArraysCore.SVector{1, Float64}, Float64, false, Pumas.LinearODE, Vector{Pumas.Event{Float64, Float64, Float64}}, Vector{Float64}, Returns{@NamedTuple{##A##::StaticArraysCore.SMatrix{1, 1, Float64, 1}, ##b##::StaticArraysCore.SVector{1, Int64}}}}}, _obstimes::Vector{Float64}, _subject::Subject{@NamedTuple{dv::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{}}, DosageRegimen{Float64, Int64, Float64, Float64, Float64, Float64}, Vector{Float64}}, _param::@NamedTuple{CL::Float64, Vc::Float64, σ::Float64}, _random::@NamedTuple{})
    @ Main /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/dsl/model_macro.jl:1931
 [2] DerivedObj
    @ /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/dsl/model_macro.jl:1961 [inlined]
  [3] #_derived#326
    @ /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/models/model_api.jl:1232 [inlined]
  [4] _derived
    @ /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/models/model_api.jl:1180 [inlined]
...
 [48] #predict#1114
    @ /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/estimation/diagnostics.jl:138 [inlined]
 [49] predict
    @ /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/estimation/diagnostics.jl:128 [inlined]
 [50] predict(fpm::Pumas.FittedPumasModel{PumasModel{(CL = 1, Vc = 1, σ = 1), 0, (:Central,), (Symbol("##A##"), Symbol("##b##")), ParamSet{@NamedTuple{CL::RealDomain{TransformVariables.Infinity{false}, TransformVariables.Infinity{true}, Float64}, Vc::RealDomain{TransformVariables.Infinity{false}, TransformVariables.Infinity{true}, Float64}, σ::RealDomain{TransformVariables.Infinity{false}, TransformVariables.Infinity{true}, Float64}}}, Serialization.__deserialized_types__.var"#5#10", Serialization.__deserialized_types__.var"#6#11", Nothing, Serialization.__deserialized_types__.var"#7#12", Pumas.LinearODE, Pumas.DerivedObj{(:dv,), Serialization.__deserialized_types__.var"#8#13"}, Serialization.__deserialized_types__.var"#9#14", ModelingToolkit.ODESystem, Pumas.PumasModelOptions}, Vector{Subject{@NamedTuple{dv::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{}}, DosageRegimen{Float64, Int64, Float64, Float64, Float64, Float64}, Vector{Float64}}}, Optim.MultivariateOptimizationResults{Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Nothing, Float64, Optim.Flat}, Vector{Float64}, Float64, Float64, Vector{Optim.OptimizationState{Float64, Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Nothing, Float64, Optim.Flat}}}, Bool, @NamedTuple{f_limit_reached::Bool, g_limit_reached::Bool, h_limit_reached::Bool, time_limit::Bool, callback::Bool, f_increased::Bool}}, NaivePooled, Vector{Vector{Float64}}, @NamedTuple{optimize_fn::Pumas.DefaultOptimizeFN{Optim.BFGS{LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Nothing, Float64, Optim.Flat}, @NamedTuple{show_trace::Bool, store_trace::Bool, extended_trace::Bool, g_tol::Float64, allow_f_increases::Bool}}, constantcoef::Tuple{}, omegas::Tuple{}, ensemblealg::EnsembleThreads, diffeq_options::@NamedTuple{abstol::Float64, reltol::Float64, alg::CompositeAlgorithm{1, Tuple{Vern7{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, Rodas5P{1, true, LinearSolve.GenericLUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEq.DEFAULT_PRECS), Val{:forward}, true, nothing, typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!)}}, OrdinaryDiffEq.AutoSwitch{Vern7{typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!), Static.False}, Rodas5P{1, true, LinearSolve.GenericLUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEq.DEFAULT_PRECS), Val{:forward}, true, nothing, typeof(OrdinaryDiffEq.trivial_limiter!), typeof(OrdinaryDiffEq.trivial_limiter!)}, Rational{Int64}, Int64}}}}, ParamSet{@NamedTuple{CL::RealDomain{TransformVariables.Infinity{false}, TransformVariables.Infinity{true}, Float64}, Vc::RealDomain{TransformVariables.Infinity{false}, TransformVariables.Infinity{true}, Float64}, σ::RealDomain{TransformVariables.Infinity{false}, TransformVariables.Infinity{true}, Float64}}}, Pumas.OptimState{NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, Optim.BFGSState{Vector{Float64}, Matrix{Float64}, Float64, Vector{Float64}}}}; nsim::Nothing, obstimes::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, ensemblealg::EnsembleThreads)
    @ Pumas /build/_work/DeepPumasSystemImages/DeepPumasSystemImages/julia_depot/packages/Pumas/aZRyj/src/estimation/diagnostics.jl:180

ERROR: Internal julia server error

Stack trace:
    at writeJuliaCommand (file:///opt/quarto/bin/quarto.js:41721:19)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async executeJulia (file:///opt/quarto/bin/quarto.js:41615:22)
    at async Object.execute (file:///opt/quarto/bin/quarto.js:41352:20)
    at async renderExecute (file:///opt/quarto/bin/quarto.js:78033:27)
    at async renderFileInternal (file:///opt/quarto/bin/quarto.js:78201:43)
    at async renderFiles (file:///opt/quarto/bin/quarto.js:78069:17)
    at async render (file:///opt/quarto/bin/quarto.js:82929:21)
    at async renderForPreview (file:///opt/quarto/bin/quarto.js:83956:26)
    at async render (file:///opt/quarto/bin/quarto.js:83839:29)
4

Morning @yoshidk6. I assume you are using the most recent version of Pumas here, 2.6? If that is the case then you should be able to successfully deserialize that .jls file using the new QuartoTools package that is available in our latest release. That package provides a version of serialize and deserialize that will work correctly in a Quarto notebook.

Just doing

using QuartoTools

instead of

using Serialization

in your .qmd file should be sufficient.

Note that since both of those packages export the names serialize and deserialize if you using both of them in the same script then you will have to prefix the function names with the module name to disambiguate them.

The details regarding why this is needed: due to the evaluation environment that is used within a Quarto notebook when using engine: julia the code is not evaluated in the normal Main module (that the REPL uses), but instead in sandbox modules. The QuartoTools.deserialize function takes this into account and looks for module references in these sandbox modules instead of Main, which is what Serialization does.

2 Likes
5

Working perfect with QuartoTools, thank you!

1 Like