Hello!
Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.
Now…let’s dive into this week's interesting links.
Happy 100th Anniversary, Arizona (With an Asterisk?)
September, 2026 marks the 100th anniversary of Arizona’s admission into the federal birth and death registration areas. The last step needed for admission was a survey that, according to the Census Bureau, demonstrated that Arizona’s system of collecting vital statistics captured at least 90 percent of the state’s births and deaths (see Figure 1). But did the survey really demonstrate that level of completeness? Or should Arizona’s admission to the registration areas have come with an asterisk?…
Partial pooling as an answer to the reference class problem
What should be the correct reference class? If you want to know whether your income is high, should you compare yourself to all Americans, all Californians, all residents of San Jose, or just your neighborhood?… This question: which comparison group is “right” for assigning a probability to an individual case, is one of the oldest open problems in the foundations of probability. Venn identified it in The Logic of Chance (1866; 1876 second edition). Reichenbach gave it a name in 1949: the reference class problem. Hájek argued in 2007 that it afflicts every interpretation of probability, not just frequentism. Philosophers have been debating it for 150 years…An answer has been sitting in the statistics toolbox for decades.…The Great British Converter: From Bytes to Barleycorns: The Ultimate Imperial & Metric Converter
Nine unit converters, an archive of over two hundred historical measures, and the stories behind how Britain came to reckon the world.…
.
.
.
Analysis of variance--why it is more important than ever
Analysis of variance (ANOVA) is an extremely important method in exploratory and confirmatory data analysis. Unfortunately, in complex problems (e.g., split-plot designs), it is not always easy to set up an appropriate ANOVA. We propose a hierarchical analysis that automatically gives the correct ANOVA comparisons even in complex scenarios…Why is the p-value measured against 0.05? [Reddit]
This may have been asked before, but all I can find is explaining what it means. I want to know why we’ve chosen 0.05 and not 0.04 or 0.06 etc for a lot of disciplines….
Why the One-Sided P-Value Does not Actually Test the Null-Hypothesis
It has long been known that the frequentist one-sided p-value from Null-Hypothesis Statistical Testing can be interpreted as a poor person’s Bayesian posterior probability. Specifically, the one-sided p-value is usually a good approximation to the posterior probability that the effect is in the direction opposite to that stipulated by the alternative hypothesis (e.g., Casella & Berger, 1987; Jeffreys, 1939; Marsman & Wagenmakers, 2017, and references therein). The consequences of this are rather dramatic, so let’s make that concrete. Suppose we ask 100 people about their favorite orientation toilet paper, and 60 answer “over”.…Data dictionaries for humans, agents and jumping frogs
Apparently, you can rent a frog and enter a jumping competition. You might find yourself up against teams with years or even decades of experience selecting and preparing their frogs. Welcome to the Calaveras County Jumping Frog Jubilee.…In Astley et al. (2013), researchers studied bullfrog jumps at the event, comparing frogs rented by fairgoers with those entered by experienced teams…Their measurements provide the example for the Data Dict quickstart…It is certainly a memorable way to get into data documentation….How I changed teaching after AI managed to do all my homework assignments
For the last couple of years, I have mostly taught the course Machine Learning in Production, an upper-level course on building production-ready software around ML models with a heavy focus on MLOps, usually with 100 to 170 students. These days one common question when talking to other educators is how we have changed teaching in the age of generative AI and coding agents, so let me outline what we did.…What happens when you analyze college football like the CIA?
For the past 15 years, we’ve worked with people whose job is to make judgments about uncertain futures: intelligence and government analysts, foreign-policy researchers, investors, and corporate strategists. This year I decided to try an experiment. I took the methodology we’ve developed for that kind of work and applied it to something considerably less consequential: assessing the fortunes of the University of Illinois football team. To understand why, it helps to think about what an intelligence analyst actually does…Celebrating the rOpenSci Community
rOpenSci turns 15 and I’ve been a part of this community for a good chunk of that time. As I reflected on those years, a lot of great memories came to mind, and I realized some of them taught me some lessons that I would love to share.…Guesstimate - A spreadsheet for things that aren’t certain
Guesstimate is a tool for performing estimates using monte carlo experiments. It can be used similarly to excel, but provides the option of providing ranges and distributions as values instead of individual points. Other metrics can do mathematical operations on these cells/metrics. After each new input is added or changed, a set of 5000 samples is randomly generated from each input and goes through the specified operations to produce confidence intervals in the output.…The North American Bioconductor Conference 2026 (BioC2026) took place from August 10-12, 2026, at the Fred Hutchinson Cancer Center in Seattle, Washington…Across the three days, the programme featured three keynote speakers, 20 contributed speakers, and 17 sessions. The programme highlighted the breadth of biological research supported by Bioconductor, as well as the tools, infrastructure, and communities that enable open and reproducible computational biology.…
Measuring the common denominator in ecology
I’ve been an advisor for many different conservation and ecological monitoring programs over a period of two decades now. I’m finding my advice to the empiricists often boils down to one concept: Don’t forget to measure the denominator…Let me explain…Ecologists get excited about measuring the numerator. Its often the fun, interesting and biological part of observation. Its also where we have technological innovations that people get excited about - underwater robots with video, eDNA and big data from citizen scientists are leading examples. The denominator is just as important, but often harder and less interesting to measure. Let me give citizen science as an example.…What a p-value tells you, and what it does not
What a p-value means, checked by simulating the null in R, and why p = 0.03 in a grazing experiment is no promise that an exact repeat will be significant.…Twelve fenced plots and twelve grazed plots on an upland pasture, and after one season, the height of the tallest rowan seedling in each plot. The fenced plots grew taller seedlings on average, and t.test() returns p = 0.03. The question that follows is almost always the same: does that mean the effect is real, and would I find it again if I ran the experiment a second time?…
Safe / Not Safe - A local-first Postgres migration safety checker for developers
Schema migrations fail in boring but expensive ways: blocking indexes, table rewrites, validating constraints, transaction-wrapped DDL, and rollout-incompatible renames. Safe / Not Safe catches the common traps while keeping raw SQL local.…Fourier Analysis: Drawing Llamas with Circles
The Fourier transform is a method of transforming an input signal from the time domain to the frequency domain. It has a huge range of applications, for instance audio engineers can pick out individual undesired frequencies in a song with the Fourier transform and then get back the sound without that frequency using the inverse Fourier transform. We can make use of the Fourier transform in digital image processing for filtering images, like gaussian blurs and compressing images using the JPEG format. Lastly, as this article will go into: drawing!…
.
* Based on unique clicks.
** You can find last week's issue #670 here.
asgl: A Python Package for Penalized Quantile, Linear, and Logistic Regression
Exploding variance of means of exponentials: least-squares to the rescue
The Rock’s rise and fall—and the end of a certain type of movie stardom
.
Thank you for joining us this week! :)
Stay Data Science-y!
All our best,
Hannah & Sebastian