Is there a balance to be struck between simple hierarchical models and
statmodeling.stat.columbia.edu40 points by luu 7 days ago
40 points by luu 7 days ago
Full Title: Is there a balance to be struck between simple hierarchical models and more complex hierarchical models that augment the simple frameworks with more modeled interactions when analyzing real data?
Well, my impression is that the statistic paradigm itself limits the complexity of a model through it's basic aims and measures. Especially, a statistical model aims to be an unbiased predictor of a variable whereas machine learning/"AI" just aims for prediction and doesn't care about bias in the sense of statistics.
I think this is accurate but mostly because statistical modelling aims for interpretable parameters. That very strongly regularises complexity.
I think they have totally different goals typically. For example, let’s say we are doing a sampling procedure. How do you estimate the sampling error? I’m not aware of a machine learning technique that will help, but you can use Bayesian and MCMC techniques
"When working on your particular problem, start with simple comparisons and then fit more and more complicated models until you have what you want."
sounds algorithmic...
Yes and you can even build symbolic engines that do this for you. I think the real question we must ask ourselves as data scientists or statisticians or whatever is whether we believe these data models represent the space of data fully or by happenstance. And if by happenstance is it because the data doesn’t capture the underlying processes that produced the data or are they uncapturable in this way and function approximators like neural networks or gradient booster machines are better. And is that because those function approximators capture interactions between the driving processes that otherwise go unseen or is it because those processes have fractional dimensions that control their impact that are not captured by data models. This all is summed up well by Leo Breimans two cultures paper in my opinion. I have gone back and forth on which “culture” is the correct representation of how processes produce data. If you buy that only function approximators truly capture the complexity of whatever processes you are observing then you have to wonder why physics works so well. That’s because, at least in my opinion, from the statistical point of view physics has spent centuries developing equations that are linear combinations of variables that are essentially data models according to Leo. I hope this opinion generates discussion because I don’t know what the answer is or if it matters that there is one.
seems to me that one approach is fueled by data and the other is fueled by understanding. in the former, the observations form a view of behavior which is then modeled with high fidelity. in the latter, active inquiry, adversarial data collection and careful reasoning produce simpler models of hypothsized underlying processes that often prove to have nearly perfect generalization.
the interesting future is probably the one where the former produces new building blocks for the latter. (ie, the computer generates new simple and easy to understand constructs from which it explains previously not understood or well modeled phenomena.)