Differing results from pumas versions

When running an old analysis of drug y ( last run in September of 2021) I have gotten a different result for my Clearance values ( an about 5% difference in the typical value). I then plotted the old individual clearance values against the new individual clearance values and got a graph that was less tight than expected. Is this expected across these version changes/ is there a possible explanation? (Data has been confirmed to be the same and overall model did not change).
clearance question
Thank you in advance for your help!

Hello Brooke,

We generally don’t expect results to change from one release to the next but sometimes we introduce improvements to e.g. the accuracy of the derivatives of the objective function or introduce a more efficient ODE solver. In rare cases, we might also fix bugs that could affect results. The more fundamental problem is that nonlinear mixed effects models are fairly sensitive. This means that relatively minor changes to how things are calculated might cause noticeable changes to the estimates. It’s hard or impossible to say if one of the results are more correct than the other. Their difference might just be a reflection of the sensitivity of the model. In particular so when the model has many parameters and relatively few observations. It might be beneficial to compare predictions from the old run and the new run. Predictions tend to be less sensitive than the parameter estimates.

Andreas Noack

1 Like

I see, thank you very much.