Is this analysis sound?

How would you respond to this query, if somebody asked your opinion?
Author

Eric Rexstad

Published

May 8, 2025

Received this email

A former workshop participant recently wrote more than a year after attending this workshop. She had been working on her own data and was concluding her analysis. She sent this table (produced by the summarize_ds_models() function) asking

Is it fine for me to use the model with the smallest AIC for inference for this survey?

How would you respond?


Postscript

The email also included a table of stratum-specific estimates. I hide this table so as not to spoil the answer to the question above. If you are curious, simply click on the arrow on the top right.

The warning signs are more obvious in this output than from the model selection table. However, you are discouraged from looking at estimates produced by models until your model selection process in complete. This prevents subconscious bias to use models that produce results the analyst “likes.”