David Salazar – blog

Fat vs Thin: does LLN work?

Statistical estimation is based…

Statistical Rethinking: Week 1

Week 1 tries to go as deep as possible in the intuition and the mechanics of a very simple model. As always with McElreath, he goes on with both clarity and erudition.

Spurious PCA under Thick Tails

PCA is a dimensionality reduction technique. It seeks to project the data onto a lower dimensional hyperplane such that as much of the original data variance is preserved.…

Statistical Rethinking: Week 2

Week 2 has gotten us to start exploring linear regression from a bayesian perspective. I found it the most interesting to propagate uncertainty through the model.

Statistical Rethinking: Week 3

Week 3 gave the most interesting discussion of multiple regression. Why isn’t it enough with univariate regression? It allows us to disentagle two types of mistakes:

What does it mean to fatten the tails?

First, let’s define what we mean by fatter tails.

Statistical Rethinking: Week 4

This week was a…

Statistical Rethinking: Week 5 -> Interactions

Standard Deviation and Fat Tails

In this post, I’ll continue to explore with Monte-Carlo simulations the ideas in Nassim Taleb’s latest book: Statistical Consequences of Fat Tails.…

Statistical Rethinking Week 5 -> HMC samples

After a quick tour around interactions, this week was a quick introduction to MCMC samplers and how they are the engine that powers current Bayesian…

Understanding the tail exponent

Power Laws are ubiquitous to describe fat tails, a topic that I’ve been trying to wrap my…

Statistical Rethinking: Week 6

The week was a whirlwind tour of:

Correlation is not Correlation

To the usual phrase of correlation is not causation, Nassim Taleb often answers: correlation is not correlation. First, just like the mean and PCA, the sample correlation coefficient has persistent small sample effects when variables from Extremistan are involved. These topics are…

Statistical Rethinking: Week 7

This week paid off. All the hard work of understanding link functions, HMC flavored Monte-Carlo, and GLM allowed to study more complex models. To keep…

R-squared and fat tails

Simulating into understanding Multilevel Models

Pooling is the process and shrinkning is the result

Statistical Rethinking Week 8

This week was our first introduction to Multilevel models. Models where we…

Central Limit Theorem in Action

I have recently been exploring Nassim Taleb’s latest technical book: Statistical Consequences of Fat Tails. In it, we have seen how the Law of Large Numbers for different estimators simply does not work fast enough (in Extremistan) to be…

Understanding Pooling across Intercepts and Slopes

Statistical Rethinking is a fabulous course on Bayesian Statistics (and much more). By following simulations in the book, I recently tried to understand why pooling is the process and…

LLN for higher p Moments

I have recently been exploring Nassim Taleb’s latest technical book: Statistical Consequences of Fat Tails. In it, we have seen how the Law of Large Numbers for different estimators simply does not work fast enough (in Extremistan) to be…

Bayesian Instrumental Variable Regression

Statistical Rethinking is a fabulous course on Bayesian…

Statistical Rethinking: Week 9

Week 9 was all about fitting models with multivariate distributions in them. For example, a multivariate likelihood helps us use an instrumental variable to estimate the…

Statistical Rethinking Week 10

This is the final week of the best Statistics course out there. It showed the benefits of being ruthless with conditional…

Fisher Tippet Th: a “CLT” for the sample maxima

For fat-tailed random variables, the statistical properties are determined by a few observations in the tail. In Nassim Taleb’s words, “the tail wags the dog”. Therefore, it is vital to study the distribution of these few observations. A logical question to ask, then, is: is there a limiting…

How to not get fooled by the “Empirical Distribution”

With fat-tailed random variables, as Nassim Taleb says, the tail wags the dogs. That is, “the tails (the rare events) play a disproportionately…

When are GARCH (and friends) models warranted?

In this blogpost, I’ll answer the question, following Nassim Taleb’s latest technical book: Statistical Consequences of Fat Tails, when can we use GARCH (and firends) models? As an example, also following Taleb, I’ll check the resulting conditions with the…

Extreme Value Theory for Time Series

The Fisher-Tippet theorem (a type of CLT for the tail events) rests on the assumption that the observed…

Bayesian Data Analysis: Week 2

Bayesian Data Analysis (Gelman, Vehtari et. alter) is equals part a great introduction and THE reference for advanced Bayesian Statistics. Luckily, it’s …

Bayesian Data Analysis: Week 3 -> Fitting a Gaussian probability model

Bayesian Data Analysis (Gelman, Vehtari et. alter) is equals part a great introduction and THE reference for advanced Bayesian Statistics. Luckily, it’s …

Probability Calibration under fat-tails: useless

Probability calibration refers to…

Bayesian Data Analysis: Week 3-> Exercises

Bayesian Data Analysis (Gelman, Vehtari et. alter) is equals part a great introduction and THE reference for advanced Bayesian Statistics. Luckily, it’s …

Gini Index under Fat-Tails

I have recently been exploring Nassim Taleb’s latest technical book: Statistical Consequences of Fat Tails. In this blogpost, I’ll follow Taleb’s…

Bayesian Data Analysis: Week 4 -> Importance Sampling

Bayesian Data Analysis (Gelman, Vehtari et. alter) is equals part a great introduction and THE reference for advanced Bayesian Statistics. Luckily, it’s …

Bayesian Data Analysis: Week 5 -> Metropolis

Bayesian Data Analysis (Gelman, Vehtari et. alter) is equals part a great introduction and THE reference for advanced Bayesian Statistics. Luckily, it’s …

BDA Week 6: MCMC in High Dimensions, Hamiltonian Monte Carlo

Tail Risk of diseases in R

Pasquale Cirillo and Nassim Taleb published a short, interesting and important paper on the Tail Risk of contagious diseases. In short, the distribution of fatalities is strongly fat-tailed: thus rendering any forecast, whether is pointwise or a…

BDA week 7: LOO and its diagnostics

Once Stan’s implementation of HMC has run its magic, we finally have samples from the posterior distribution \(\pi (\theta | y))\). We can then run posterior predictive checks and hopefully our samples looks plausible under our posterior.…

BDA Week 8: Bayesian Decision Analysis

Many if not most statistical analyses are performed for the ultimate goal of decision making. Bayesian Statistics has the advantage of…

BDA Week 9: Large Sample Theory for the Posterior

As Richard McElreath says in his fantastic Statistics course, Frequentist statistics is more a framework to evaluate estimators than a framework for deriving them. Therefore, we can use frequentist tools to evaluate the…

Causality: Bayesian Networks and Probability Distributions

Stats people know that correlation coefficients do not imply causal effects. Yet, very often, partial correlation coefficients from regressions…

Causality: Invariance under Interventions

In the last post we saw how two causal models can yield the same testable implications and thus cannot be distinguished from data alone. That is, we cannot gain…

Causality: To adjust or not to adjust

In this blogpost, I’ll simulate data to show how conditioning on as many variables as possible is not a good idea. Sometimes, conditioning can lead to de-confound an effect; other times, however, conditioning on a variable can create unnecessary…

Causality: The front-door criterion

In a past blogpost, I’ve explore the backdoor criterion: a simple graphical algorithm, we can define which variables we must include in our analysis in order to cancel out all the information coming from different causal relationships than the one…

Causality: Testing Identifiability

We’ve defined causal effects as an interventional distribution and posit two identification strategies to estimate them: the back-door and the front-door criteria. However, we cannot always use these…

Causality: Counterfactuals - Clash of Worlds

We’ve seen how the language of causality require an exogenous intervention on the values of \(X\); so far…

Causality: Regret? Look at Effect of Treatment on the Treated

Regret about our actions stems from a counterfactual question: What if I had acted differently?. Therefore, to answer such question, we need a more elaborate language than the one we…

Causality: Probabilities of Causation

Questions of attribution are everywhere: i.e., did \(X=x\) cause \(Y=y\)? From…

Causality: Mediation Analysis

Kids are the prototypical question makers; they never stop asking questions. Just after you have answered a Why? question, they ask yet another Why? This is the problem of mediation…

Forecasting elections? Taleb says no

With US elections around the corner, news outlets are…

Introduction to Survival Analysis

Survival Analysis is fun, but not often taught in statistics courses. It’s just like regular statistics but with a few twists.

Notes from Simon Wood’s Generalized Additive Models

Simon Wood is a world expert on GAMs and creator of the fantastic mgcv package in R. His book on GAMs is a great introduction to the subject. I’ve been re-reading it and taking notes to be able to better teach this to my co-workers. This post is a summary of the…