Hello!
Every week, we write this email to share links we think are worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.
Sponsor Message
Find your algorithm for success with an online data science degree from Drexel University. Gain essential skills in tool creation and development, data and text mining, trend identification, and data manipulation and summarization by using leading industry technology to apply to your career.
.
Now…let’s dive into this week's interesting links.
The Most Popular Song You’ve Never Heard
According to the music industry’s foremost record keeper, “Choosin’ Texas” is the most popular track of the last 70 years—a distinction that makes you question how Billboard (and its charts) came to such a conclusion. So today, we’ll examine the remarkable commercial longevity of “Choosin’ Texas,” deconstruct how Billboard compiles its marquee Hot 100 list, and explore why this song is uniquely predisposed to that longevity (as measured by Billboard)…
Plumbers, chains, and famous painters: The (updated) history of the pipe operator in R
Our story starts as many other R related stories… in stackoverflow. On January 17th, 2012, an anonymous user “user4” asked the following question: How can you implement F#’s forward pipe operator in R? The operator makes it possible to easily chain a sequence of calculations….Can you use autoregressive diffusion to generate market data?
This summer, a research intern, Kavish, built a diffusion model for market data. Diffusion and flow-matching models are powerful tools for representing sequential, multimodal data in regimes such as image, video, robotics, and audio. Taking inspiration from Autoregressive Image Generation without Vector Quantization, Kavish created an event-level generative model of market data using autoregressive diffusion…
.
.
.
2D Vehicles
One weekend in late August 1996, I wrote a 2D physics simulation which later became the basis of the vehicle system in the eventually popular game “Grand Theft Auto” aka “GTA”. It began as a little wireframe demo written in GFA BASIC on my Atari ST home computer (and was later ported to C for use in the actual game). I have recreated* it here in JavaScript for its 30th anniversary in a slightly prettier, “remastered” form, for anyone who is interested in seeing or playing it…Are .ipynb notebooks already outdated in the agentic era? [Reddit]
I am a data scientist who started working in the industry before the LLM revolution. Back then, Jupyter Notebooks were a perfect fit for the classical DS pipeline: EDA -> data prep -> fit -> eval -> tune -> save model artefact and notebook. Lately, I have been thinking a lot about how agentic development and LLMs are changing the way data scientists work. Especially in classical ML applications, where you still need to explore data, run experiments, check different hypotheses and decide what to do next based on the results. I mean, Claude/Codex can already write most of the code for us. But why do we still need to organise the whole workflow around code cells?…
From 35 seconds to half a second: 70× faster dashboards in plain Postgres
Pick “Last 12 months” on an analytics dashboard and Postgres reads every event of the year, once for every panel. At scale, ours took 35 seconds. Now it takes half a second on the same database, thanks to one table and one property of our numbers that most people forget to check….Visualize data on Nobel laureates per country
The Nobel laureates of 2020 were announced last week, and I thought it would be interesting to visualize the repartition of laureates per country, as there are several ways to do so. I’m going to use this dataset available on Kaggle, which contains information on the year, category, name of the laureate, country, city and date of birth and death, among other things. Notice that this dataset goes from 1901 to 2016 and therefore doesn’t contain the most recent laureates…A Bayesian Network to Explore Data Center Risks
This post presents a template for constructing Bayesian Network models to explore the risks that data centers pose to a community. It demonstrates a model contingent on several factors of practical concern such as the scale and location of the data center. A unique feature is the use of synthetic data generator which can be modified to suit a particular community. This enables sorting through risks in the presence of uncertain data The DAG contains both nodes that are thought to cause risks to the community as well as nodes that quantify various kinds of risk….Claude Dashboards: The Jupyter Notebooks of Agent Age
Claude Dashboards were recently announced, and it goes to show that Dashboard are indeed not dead - when Claude has now built their own services. The integrations is done into these Cloud Data Platform: Claude Dashboards connects to a data platform like BigQuery, Databricks or Snowflake, or a CRM tool like Salesforce, and is in beta on paid plans. This is following the trend of dbt Charts, MotherDuck Dives and other BI-as-Code first tools such as Rill Data or Evidence…
Brownian Motion
Building intuition for Brownian motion by deriving its marginal distribution from a discrete-time random walk.What Would Have Happened Otherwise? Causal Inference in Practice
Difference-in-differences, regression discontinuity, synthetic control, propensity score matching and double machine learning, explained through real case studies…Five months treating bugs like patients and coding agents like a medical team
We deployed a coding agent pipeline structured around a teaching hospital metaphor, using distinct roles like Triage Nurses, Fellows, Review Attendings, and Discharge Nurses, to enforce rigorous reviews, mandatory planning, and safety guardrails before merging code. Over a 5 month period, this agent pipeline successfully processed over a million lines of code with only 7 total reverts, proving that structured agentic workflows can safely build and maintain complex database migrations. By prioritizing meticulous record-keeping, strict test coverage, and planning reviews before writing code, the system minimized costly regressions and enabled features like the Migration Assistant, successfully balancing autonomous output with enterprise-grade software reliability…
World Action Models vs VLAs: Robotics Just Had Its GPT-2 Moment
This post is a math-first walkthrough of the entire World Action Model wave: the factorization that defines a WAM, the objective zoo (teacher forcing → diffusion forcing → shortcut forcing → context amortization → next forcing), the five distinct ways people are extracting actions from video models, the brutal systems engineering needed to run a 14B diffusion model in a 7 Hz control loop, the benchmark scoreboard, and the very real counterarguments from the JEPA camp…Validating data analyses written by coding agents
Claude Code (and many other agents) has been getting so good at ecological analysis that I find I’m increasingly tempted not to directly review the code it writes. In fact, it finds more mistakes in my code than I find in its code these days. So I need a way to check the validity of the code it writes. Here’s the strategy for this I’ve been developing. Its a work in progress…
How do you investigate a metric that suddenly changes when nothing obvious has changed? [Reddit]
One thing I’ve found interesting about analytics is that sometimes a metric changes significantly, but there isn’t an obvious explanation. Conversion drops, revenue looks different, or a dashboard suddenly shows a trend nobody expected. The tracking appears to be working, and there hasn’t been a major product or campaign change. At that point, the investigation can go in several directions: data freshness, tracking changes, attribution logic, filters, SQL transformations, or an actual change in user behavior. I’m curious how other analytics teams approach this. Do you have a standard checklist for investigating unexpected metric changes, or does it mostly depend on the analyst digging into the problem? What’s the first thing you usually check?…Dogs of New York
I recently updated nycdogs, a data package for R that I originally made for teaching purposes. It bundles up New York City’s dog licensing and dog bite datasets into an R package. The data is already provided by the city in a directly usable way (as CSV files), but packaging it up makes it easier to integrate with other packages and get things moving a little more quickly in class…
.
* Based on unique clicks.
** You can find last week's issue #672 here.
What does my project do? - Using specifications as a live project documentation
LorenzRegression: An Implementation of the Lorenz and Penalized Lorenz Regressions in R
Calculating tree regression Shap contributions in R is a mess [Reddit]
The problem is noise, not N: why increasing the sample size isn’t all that.
10 Years of the SRE book: Announcing the second edition of “Site Reliability Engineering”
.
Thank you for joining us this week! :)
Stay Data Science-y!
All our best,
Hannah & Sebastian