EBEs output for graphing

[shli6161]

Hello.

I am using empirical_bayes() function for my analysis and I found out all EBEs are merged into one column. Is there a way to output EBEs using empirical_bayes() in a manner that can be used for graphing and further analyses? Right now, I have a report due so that needs this plot. Is there a way to work it around? Thank you very much~

[vijay]

thanks for reporting. There is a fix for this now which will be incorporated into the next release.

[Anitha]

Hi Vijay,

I have a similar situation for my empirical_bayes() function estimates. The data output is without column names (given below) is there a way around?

 :[0.003893017273809057, -0.0064450897057206896, -0.4832060380064258, -0.3182802749018907, -0.17328388537013614, 0.17692167320092497, -0.20657537748229318, 0.32688207001643593, 0.4408790170505515]
[0.0008039381680160061, 0.40692768174742994, -0.19052280418943815, -0.04766732218965573, 0.10899599268605104, -0.3042683918275428, -0.5320424018883979, 0.2769142699890851, 0.10287926660637328]
[-0.014222914360039892, -0.27825300175500967, 0.04650368905751066, 0.8223221110594434, -0.31471916195993854, -1.2005493073109694, -0.22623475195467443, -0.45139917214592845, -0.2074268722919592]
[0.008481939173097272, 0.2762267839609055, -0.17863133686883892, 0.020495341487036505, 0.027784765856280043, 0.36723239532160984, 0.8875372032632546, -0.18080334531270237, 0.8300731535547978]
[-0.019878243446784752, -0.4278111141523505, -1.0981370176624246, -1.1236560559804318, -0.6193907449016971, -0.19477408627077147, -0.7401290587533766, -0.4889376223961829, -0.07041045991056755]
[-0.003934857213489369, 0.24029421466658812, -0.5412611188854506, -0.7707892208716053, -0.2590068820661076, 0.33418668577238064, -0.4426825915767087, -0.46762064956943, 0.24663286486711308]
[-0.006213644293878277, -0.013908895159926663, -0.3622288107930953, 0.6266078495825546, -0.08367759402400056, -0.22683410345394112, -0.8381489423562987, -0.07684526559008711, -0.25530083570432544]
[-0.0019136687866613488, 0.05357869650828251, 0.18441980399155375, -0.1561105937869351, -0.05530150093480558, -0.5063107925444591, 0.45389260468026843, -0.02945563919613748, -0.12409815646964925]
[0.0033425568928669167, 0.132210340132933, -0.07537987414314398, -0.1470835903369388, 0.3243703957172484, -0.6390644043747173, 0.1847624460997283, 0.1188097927575027, 0.33825544128555646]
[0.006468301009316608, -0.07925703541188905, 0.43524516406166164, -0.2305205983467, 0.339387126527767, 0.3422595543117683, 0.32610516447211424, 0.26715353606482384, 0.39524399026831086]
[-0.0051132021911700215, -0.0762589223935508, -0.5450177725933981, -0.6310463719672519, -0.18205505097899422, -0.19719332949817878, -0.08775852121495444, -0.10764745026932888, -0.18516444694437953]
[-0.00820305073664453, -0.21126339328082763, -0.6527823574564875, -0.7328272311186826, -0.5163621544792658, 0.3689179439961686, -0.5170075347440016, 0.11472442421520221, -0.08696497844668538]
[0.00199495451503551, 0.18077340791812443, -0.24383160661799305, -0.16766064757794794, 0.14959320750796687, 0.2345061891298344, 0.014465620620134598, -0.05478403161328483, 0.07652797553617802]
[-0.018921945618245145, -0.11775921114414262, -0.655403338769185, -0.9591892416239679, -0.30053712316489545, -0.1859074744409664, -1.175865312628053, -0.6501855029649246, -0.6661733951878989]
[0.009455542070219657, 0.266818080489206, -0.5412087708318123, -0.06868055985260811, 0.121830412571343, 1.0735100578753805, 0.38787904545357543, 1.0459134326535127, 0.03324610090726631]

[vijay]

What version of Pumas are you on? You can do

]  st

[Anitha]

Vijay, I am using Pumas v0.10.0.

[pkofod]

It’s not clear to me that this is wrong, but it would be great with more context. When are you getting these vectors? Did you try using inspect which is the higher level function?

[Anitha]

With inspect here is the output in REPL:

Fitting was successful: true
Likehood approximations used for
 * Predictions:        Pumas.FOCEI()
 * Weighted residuals: Pumas.FOCEI()
 * Empirical bayes:    Pumas.FOCEI()

Below are the empirical Bayes estimates. The output does not come with column names? I have 9 omegas in my model and here I have 9 etas per row. Is there a way I can specify each column with the name to the corresponding omega. 

julia> empirical_bayes(result_two_cmtaddprop_pg):

64-element Array{NamedTuple{(:η,),Tuple{Array{Float64,1}}},1}:
 (η = [0.003893017273809057, -0.0064450897057206896, -0.4832060380064258, -0.3182802749018907, -0.17328388537013614, 0.17692167320092497, -0.20657537748229318, 0.32688207001643593, 0.4408790170505515],)
 (η = [0.0008039381680160061, 0.40692768174742994, -0.19052280418943815, -0.04766732218965573, 0.10899599268605104, -0.3042683918275428, -0.5320424018883979, 0.2769142699890851, 0.10287926660637328],)
 (η = [-0.014222914360039892, -0.27825300175500967, 0.04650368905751066, 0.8223221110594434, -0.31471916195993854, -1.2005493073109694, -0.22623475195467443, -0.45139917214592845, -0.2074268722919592],)
 (η = [0.008481939173097272, 0.2762267839609055, -0.17863133686883892, 0.020495341487036505, 0.027784765856280043, 0.36723239532160984, 0.8875372032632546, -0.18080334531270237, 0.8300731535547978],)
 ⋮

[andreasnoack]

You can construct a dataframe from the inspect output with DataFrame(inspect(result)). That dataframe will contain columns for each of the random effect variables. One small inconvenience is that the value of the EBE will be repeated in the dataframe for each of time index within a subject. Hence, you can do something like

by(DataFrame(inspect(result)), :id, first)

to just get the first row for each subject.

[vijay]

@Anitha as Andreas suggested you can get the EBE’s using the DataFrame construct on the inspect feature

julia> ebes = DataFrame(inspect(pkres);include_covariates=false)
Calculating: predictions, weighted residuals, empirical bayes. Done.
1210×13 DataFrame. Omitted printing of 3 columns
│ Row  │ id     │ time    │ evid  │ dv      │ dv_pred │ dv_ipred │ pred_approx │ dv_wres   │ dv_iwres  │ wres_approx │
│      │ String │ Float64 │ Int64 │ Float64 │ Float64 │ Float64  │ String      │ Float64   │ Float64   │ Pumas.FOCEI │
├──────┼────────┼─────────┼───────┼─────────┼─────────┼──────────┼─────────────┼───────────┼───────────┼─────────────┤
│ 1    │ 1      │ 0.0     │ 0     │ 1429.5  │ 1307.04 │ 1805.21  │ Pumas.FOCEI │ 0.19554   │ -1.0314   │ FOCEI()     │
│ 2    │ 1      │ 1.0     │ 0     │ 1444.87 │ 1290.6  │ 1734.4   │ Pumas.FOCEI │ 0.179012  │ -0.827269 │ FOCEI()     │
│ 3    │ 1      │ 2.0     │ 0     │ 1513.53 │ 1273.44 │ 1666.37  │ Pumas.FOCEI │ 0.366663  │ -0.454535 │ FOCEI()     │
│ 4    │ 1      │ 3.0     │ 0     │ 1732.73 │ 1255.64 │ 1601.01  │ Pumas.FOCEI │ 0.980394  │ 0.407719  │ FOCEI()     │
│ 5    │ 1      │ 4.0     │ 0     │ 1717.83 │ 1237.28 │ 1538.21  │ Pumas.FOCEI │ 0.891291  │ 0.578708  │ FOCEI()     │
⋮
│ 1205 │ 10     │ 115.0   │ 0     │ 533.524 │ 412.679 │ 422.046  │ Pumas.FOCEI │ 1.27836   │ 1.30898   │ FOCEI()     │
│ 1206 │ 10     │ 116.0   │ 0     │ 405.293 │ 383.228 │ 396.876  │ Pumas.FOCEI │ 0.0572661 │ 0.105098  │ FOCEI()     │
│ 1207 │ 10     │ 117.0   │ 0     │ 432.941 │ 355.821 │ 373.207  │ Pumas.FOCEI │ 0.732809  │ 0.793193  │ FOCEI()     │
│ 1208 │ 10     │ 118.0   │ 0     │ 345.416 │ 330.32  │ 350.949  │ Pumas.FOCEI │ -0.14893  │ -0.078139 │ FOCEI()     │
│ 1209 │ 10     │ 119.0   │ 0     │ 283.573 │ 306.596 │ 330.019  │ Pumas.FOCEI │ -0.753072 │ -0.697452 │ FOCEI()     │
│ 1210 │ 10     │ 120.0   │ 0     │ 266.422 │ 284.526 │ 310.338  │ Pumas.FOCEI │ -0.728372 │ -0.701284 │ FOCEI()     │

julia> names(ebes)
13-element Array{Symbol,1}:
 :id         
 :time       
 :evid       
 :dv         
 :dv_pred    
 :dv_ipred   
 :pred_approx
 :dv_wres    
 :dv_iwres   
 :wres_approx
 :η_1        
 :η_2        
 :ebes_approx

Notice the names of the random effects in the column names.

To show what Andreas suggested, you will get

julia> by(ebes, :id, first)
10×13 DataFrame. Omitted printing of 3 columns
│ Row │ id     │ time    │ evid  │ dv      │ dv_pred │ dv_ipred │ pred_approx │ dv_wres   │ dv_iwres  │ wres_approx │
│     │ String │ Float64 │ Int64 │ Float64 │ Float64 │ Float64  │ String      │ Float64   │ Float64   │ Pumas.FOCEI │
├─────┼────────┼─────────┼───────┼─────────┼─────────┼──────────┼─────────────┼───────────┼───────────┼─────────────┤
│ 1   │ 1      │ 0.0     │ 0     │ 1429.5  │ 1307.04 │ 1805.21  │ Pumas.FOCEI │ 0.19554   │ -1.0314   │ FOCEI()     │
│ 2   │ 2      │ 0.0     │ 0     │ 2416.01 │ 1194.58 │ 2119.11  │ Pumas.FOCEI │ 1.6614    │ 0.694318  │ FOCEI()     │
│ 3   │ 3      │ 0.0     │ 0     │ 1758.89 │ 1617.36 │ 1880.99  │ Pumas.FOCEI │ 0.216884  │ -0.321674 │ FOCEI()     │
│ 4   │ 4      │ 0.0     │ 0     │ 1358.8  │ 1348.31 │ 1447.04  │ Pumas.FOCEI │ 0.0208924 │ -0.302217 │ FOCEI()     │
│ 5   │ 5      │ 0.0     │ 0     │ 2555.35 │ 1575.0  │ 2465.69  │ Pumas.FOCEI │ 1.14604   │ 0.1802    │ FOCEI()     │
│ 6   │ 6      │ 0.0     │ 0     │ 943.624 │ 1320.41 │ 1149.22  │ Pumas.FOCEI │ -0.945041 │ -0.886571 │ FOCEI()     │
│ 7   │ 7      │ 0.0     │ 0     │ 1229.74 │ 1363.81 │ 1070.52  │ Pumas.FOCEI │ -0.360989 │ 0.737079  │ FOCEI()     │
│ 8   │ 8      │ 0.0     │ 0     │ 845.896 │ 1448.94 │ 1232.36  │ Pumas.FOCEI │ -1.41048  │ -1.55409  │ FOCEI()     │
│ 9   │ 9      │ 0.0     │ 0     │ 1084.21 │ 1227.76 │ 892.116  │ Pumas.FOCEI │ -0.46381  │ 1.06705   │ FOCEI()     │
│ 10  │ 10     │ 0.0     │ 0     │ 1179.46 │ 1357.36 │ 1047.86  │ Pumas.FOCEI │ -0.489355 │ 0.622375  │ FOCEI()     │

Can you try it out on your version and check if you are getting these and let us know?

Vijay

[Anitha]

Thanks Vijay, and Andreas.
I did the DataFrame for infer function and now i can see the etas in columns numbered.

η_1	η_2	η_3	η_4	η_5	η_6	η_7	η_8	η_9	ebes_approx
0.003893017	-0.00644509	-0.483206038	-0.318280275	-0.173283885	0.176921673	-0.206575377	0.32688207	0.440879017	Pumas.FOCEI
0.003893017	-0.00644509	-0.483206038	-0.318280275	-0.173283885	0.176921673	-0.206575377	0.32688207	0.440879017	Pumas.FOCEI

[vijay]

I am assuming you meant to say DataFrame for the inspect function?

[Anitha]

Sorry, yes it was for the inspect function.