{ "cells": [ { "cell_type": "markdown", "id": "329b70ac-fb96-46f9-93a5-a935043d835d", "metadata": {}, "source": [ "
\n", " Climate of the Earth system\n", "

Prof. Dr. Markus Meier
\n", " Leibniz Institute for Baltic Sea Research Warnemünde (IOW)
\n", " E-Mail: markus.meier@io-warnemuende.de

\n", "
" ] }, { "cell_type": "markdown", "id": "519b554f-e050-4dc0-9cf9-c8dd075a8033", "metadata": {}, "source": [ "# Probability density and distribution\n", "1. Probability density function and important parameters\n", "2. Different probability distributions\n", "\n", "## Probability density function and important parameters\n", "\n", "#### Probability density function\n", "\n", "- let $x$ be a continous (not discrete!) variable that takes values in $\\Omega$, for example temperature, probability density function $f_X(x)$ of an event X (i.e. T=10°C) is defined as a continous function on $\\mathbb{R}$ with the following three attributes:\n", "\n", "```{math}\n", ":label: pdf\n", "\\begin{align*}\n", "&1.~~~f_X(x) \\geq 0~~\\textrm{for all}~~ x \\epsilon \\Omega,\\\\\n", "&2.~~~\\int_{\\Omega}f_X(x)~dx = 1,\\\\\n", "&3.~~~P(X\\epsilon (a,b)) = \\int_a^bf_X(x)~dx ~~\\textrm{for all}~~ (a,b) \\subset \\Omega\n", "\\end{align*}\n", "```\n", "- Question: What is the unit of the pdf? \n", " Answer: [$f_X(x)$] = [x]$^{-1}$.\n", "- Question: What is the integral of the pdf?\n", " Answer: The cumulative distribution function.\n", "\n", "#### Cumulative distribution function\n", "\n", "- Cumulative distribution function for an event X is a montonously increasing, non-dimensional function F_X(x) on $\\mathbb{R}$ defined as:\n", "```{math}\n", ":label: cdf1\n", "F_X(x) = \\int_{-\\infty}^x f_(r)~dr,\n", "```\n", "- which is equivalent to:\n", "```{math}\n", ":label: cdf2\n", "\\begin{align*}\n", "& \\lim_{x \\to -\\infty} F_X(x) = 0\\\\\n", "& \\lim_{x \\to \\infty} F_X(x) = 1\\\\\n", "& \\frac{d}{dx}F_X(x) = f_X(x)\n", "\\end{align*}\n", "```\n", "- consequently the probability of the event $X$ to be inside the range of $(a,b)$ is:\n", "```{math}\n", "P(X \\epsilon (a,b)) = F_X(b) - F_X(a)\n", "```\n", "\n", "#### Expectation $\\varepsilon$\n", "- the expectation of a given pdf weighs is with $x$ in the integral:\n", "```{math}\n", "\\begin{align*}\n", "\\varepsilon(X) &= \\int_{\\Omega}xf_X(x)~dx\\\\\n", "\\varepsilon(g(X)) &= \\int_{\\Omega}g(x)f_X(x)~dx\n", "\\end{align*}\n", "```\n", "- two attributes of the expectation are:\n", "```{math}\n", "\\begin{align*}\n", "\\varepsilon(g_1(X) + g_2(X)) &= \\varepsilon(g_1(X)) + \\varepsilon(g_2(X))\\\\\n", "\\varepsilon(ag(X)+b) &= a\\varepsilon(g(X)) + b\n", "\\end{align*}\n", "```\n", "\n", "#### Central moments $\\mu$\n", "- k-th moment of a continous random variable X:\n", "```{math}\n", "\\mu^{(k)} = \\varepsilon(x^k) = \\int_{\\Omega}x^kf(x)~dx\n", "```\n", "- k-th central moment of a continous random variable X:\n", "```{math}\n", "\\mu^{'(k)} = \\int_{\\Omega}(x-\\mu)^kf(x)~dx\n", "```\n", "- example: anomalies with $\\mu$ mean seasonal cycle\n", "- mean $\\mu$: location parameter $\\mu = \\mu^{(1)}$\n", "- variance:\n", "```{math}\n", "Var(X) = \\mu^{'(2)} = \\int_{\\Omega}(x-\\mu)^2f(x)~dx\n", "```\n", "- standard deviation:\n", "```{math}\n", "\\sigma_X = \\sqrt{Var(X)}\n", "```\n", "- Chebyshev's inequality:\n", "```{math}\n", "P(|X-\\mu| \\geq \\lambda \\sigma) \\leq \\frac{1}{\\lambda^2}\n", "```\n", "\n", "#### Skewness $\\gamma_1$\n", "- is a measure of the asymmetry of a distribution: symmetric for $\\gamma_1=0$, scaled version of the third central moment, non-dimensional shape parameter\n", "```{math}\n", "\\gamma_1 = \\int_{\\Omega}\\left( \\frac{x-\\mu}{\\sigma} \\right)^3f_X(x)~dx\n", "```\n", "```{figure} figures/L10/L10_1_skewness.PNG\n", "---\n", "width: 30%\n", "---\n", "

Figure 1: Illustration of differently skewed (see legend) distributions. Note that all distributions are shifted and scaled to have a mean value of 0 and a standard deviation of 1.

\n", "\n", "```\n", "\n", "#### Kurtosis $\\gamma_2$\n", "- is a measure of the peakedness of a distribution: a normal distribution (will be explained later this lecture) has $\\gamma_2=0$, scaled and shifted version of the fourth central moment, non-dimensional shape parameter\n", "```{math}\n", "\\gamma_2 = \\int_{-\\infty}^{\\infty} \\left( \\frac{x-\\mu}{\\sigma} \\right)^4f_X(x)~dx -3\n", "```\n", "```{figure} figures/L10/L10_2_kurtosis.PNG\n", "---\n", "width: 70%\n", "---\n", "

Figure 2: Left: Illustration of distributions with different kurtosis (see legend), but with identic means and variance on a exp(-|x|) (dashed), a normal (gray solid) and a uniform (black solid) distribution. Right: As left, but the exp({-|x|}) distribution is modified to have a uniform (flat) distribution for near zero values (black solid line).

\n", "```\n", "\n", "#### Examples\n", "- summer sea level at Kieler Förde, $\\mu = 0.06$, $\\sigma=0.19$, $\\gamma_1=-0.6$, $\\gamma_2=4.07$\n", "```{figure} figures/L10/L10_3_example.PNG\n", "---\n", "width: 30%\n", "---\n", "

Figure 3: Probability densitiy function (red line) of daily Kieler Foerde gauge height in summer months [in meter].

\n", "```\n", "\n", "- probability densities of some measured variables\n", "```{figure} figures/L10/L10_4_example.PNG\n", "---\n", "width: 50%\n", "---\n", "

Figure 4: Some probability density functions of observed daily mean climate variables.

\n", "```\n", "\n", "#### P-quantiles\n", "- mean and variance are affected by the tail ends of the pdf (likelihood of extreme values), but p-quantiles $x_p$ are insensitive to extreme values.\n", "- p quantile of 0.3 means that 30% of the x values are below this threshold\n", "```{math}\n", "\\begin{align*}\n", "F_X(x_p) = p ~~~~~\\textrm{with}~~~~~ &P(X \\varepsilon (-\\infty,x_p)) = p,\\\\\n", "&P(X \\varepsilon (x_p, \\infty)) = 1-p\n", "\\end{align*}\n", "```\n", "- median m$_{50}$ is the 50%-quantile: half of the distribution lays above and the other half below m$_{50}$.\n", "```{math}\n", "F_X(m_{50}) = 0.5 ~~~\\rightarrow~~~ P(x\\leq m_{50}) = P(x\\geq m_{50}) = 0.5\n", "```\n", "- let's look at the p-quantiles of the log-normal distribution in Figure 5 to get an idea. note the difference of mean and median!\n", "\n", "```{figure} figures/L10/L10_5_lognormal.PNG\n", "---\n", "width: 30%\n", "---\n", "

Figure 5: The Log-Normal distribution and its cumulative distribution function (cdf), mean, median, 10%-quantile and the 90%-quantile.

\n", "```" ] }, { "cell_type": "markdown", "id": "5cf4b921-50b1-4c36-b9f1-ee32da0c0fac", "metadata": {}, "source": [ "## Different probability distributions\n", "#### Uniform distribution\n", "- symmetric and less peaked than the normal distribution:\n", "```{math}\n", "f_X(x) = \\cal U(a,b) = \\left\\{\n", "\\begin{array}{ll}\n", " 1/(b-a) ~~~\\textrm{for all}~ x\\epsilon [a,b]\\\\\n", " 0~~~~~~~~~~~~~~~~~ \\textrm{elsewhere}\\\\\n", "\\end{array} \\right.\n", "```\n", "- with the cumulative distribution function:\n", "```{math}\n", "F_X(x) = \\left\\{\n", "\\begin{array}{ll}\n", " 0~~~~~~~~~~~~~~~~~~~~~~~~~~ \\textrm{ for}~x \\leq a\\\\\n", " (x-a)/(b-a) ~~~\\textrm{for all}~ x\\epsilon [a,b]\\\\\n", " 1~~~~~~~~~~~~~~~~~~~~~~~~~~ \\textrm{ for}~x \\geq a\\\\ \n", "\\end{array} \\right.\n", "```\n", "```{figure} figures/L10/L10_6_uniform.PNG\n", "---\n", "width: 30%\n", "---\n", "

Figure 6: A uniform probability density function with the borders a=1 and b=3.

\n", "```\n", "- exercise: calculate $\\mu,Var,\\sigma,\\gamma_1,\\gamma_2$ of the uniform distribution $\\cal U(a,b)$ \n", " solutions: $\\mu(\\cal U(a,b))= \\frac{1}{2}(a+b)$, $Var(\\cal U(a,b))= \\frac{1}{12}(b-a)^2$, $\\sigma(\\cal U(a,b))= \\sqrt{\\frac{1}{12}}(b-a)$, $\\gamma_1(\\cal U(a,b))= 0$, $\\gamma_2(\\cal U(a,b))= -1.2$\n", "\n", "#### Normal (Gaussian) distribution\n", "- most physical quantities are nearly normal distributed\n", "```{math}\n", "f_{\\cal N}(x) = \\frac{1}{\\sqrt{2\\pi}\\sigma}e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}} ~~~\\textrm{with}~~~ X \\sim \\cal N(\\mu,\\sigma^2)\n", "```\n", "- no skewness or kurtosis: $\\gamma_1=\\gamma_2=0$\n", "- no analytical form of cdf, approximation:\n", "```{math}\n", ":label: gausscdf\n", "F_{\\cal N}(x) \\approx \\frac{1}{2} \\left( 1+sign\\left( \\frac{x-\\mu}{\\sigma} \\right) \\sqrt{1-e^{\\frac{-2}{\\pi} \\left(\\frac{x-\\mu}{\\sigma} \\right)^2 }} \\right)\n", "```\n", "```{figure} figures/L10/L10_7_gauss.PNG\n", "---\n", "width: 60%\n", "---\n", "

Figure 7: Normal probability density function and cumulative density function for a mean of 0 and standard deviation of 1. The blue line in the right panel is a numerical estimate of the cdf and the dashed red line the approximation of the cdf after eq. (19).

\n", "```\n", "- central limit theorem states: If $X_k,k=1,2,...$ is an infinite series of independent and identically distributed random variables with $\\varepsilon(X_k)=\\mu$ and $Var(X_k)=\\sigma^2$ then the average $\\frac{1}{n} \\sum^n_{k=1}X_k$ is asymptotically normal distributed. That is:\n", "```{math}\n", "\\lim_{n \\to \\infty} \\frac{\\frac{1}{n} \\sum^n_{k=1}(X_k-\\mu)}{\\frac{\\sigma}{\\sqrt{n}}} \\sim \\cal N(0,1)\n", "```\n", "- a larger sample size reduces the standard deviation as of:\n", "```{math}\n", "\\lim_{n \\to \\infty} \\frac{1}{n} \\sum^n_{k=1}(X_k-\\mu) \\sim \\cal N(0,\\frac{\\sigma^2}{n}) ~~\\Rightarrow ~~ \\sigma_{\\Sigma}= \\frac{\\sigma}{\\sqrt{n}}\n", "```\n", "```{figure} figures/L10/L10_8_clt1.PNG\n", "---\n", "width: 60%\n", "---\n", "

Figure 8: Illustration of the central limit theorem with a uniform distribution for n=1,2,5. The normal distribution is shown for comparison. Left: all pdfs are normalized (scaled) for better comparison of the shape, illustrating eq. (20). Right: pdfs are not normalized, illustrating eq. (21).

\n", "```\n", "```{figure} figures/L10/L10_9_clt2.PNG\n", "---\n", "width: 60%\n", "---\n", "

Figure 9: Illustration of the central limit theorem on the basis of 24hrs and monthly mean surface temperatures at 56°N/10°E in July/August. The normal distribution with identical mean and variance are shown for comparison (solid lines).

\n", "```\n", "\n", "#### Log-normal distribution\n", "- distribution of positive definite quantities such as rainfall, wind speed\n", "```{math}\n", "f_X(x) = \\frac{1}{\\sigma\\sqrt{2\\pi}} \\frac{1}{x} e^{\\frac{-(ln(x)-ln(\\theta))^2}{2\\sigma^2}} ~~\\textrm{for}~~ x>0\n", "```\n", "- with the median value $\\theta$ and\n", "```{math}\n", "ln(X) \\sim \\cal N(ln(\\theta),\\sigma)\n", "```\n", "```{figure} figures/L10/L10_10_lognormal.PNG\n", "---\n", "width: 30%\n", "---\n", "

Figure 10: The log-normal distribution for a median and standard deviation equal to 1.

\n", "```\n", "- exercise: derive a general for the k-th central moment of the distribution \n", " solution: $\\varepsilon(X^k) = \\theta^ke^{{k\\sigma}^2/2}$ \n", "\n", "#### $\\chi^2$-distribution\n", "- sum of k independent squared $\\cal N(0,1)$ random variables, k number of degrees of freedom, application for the pdfs of variance estimates:\n", "```{math}\n", "f_{\\chi}(x) = \\frac{x^{(k-2)/2}e^{-x/2}}{\\Gamma(k/2)2^{k/2}} ~~\\textrm{if}~~ x>0\n", "\n", "```\n", "- with\n", "```{math}\n", ":label: gamma\n", "\\Gamma(x) = \\int_0^{\\infty}e^{-t}t{x-1}dt ~\\textrm{for}~x>0\n", "```\n", "- it has handy attributes:\n", "```{math}\n", "\\begin{align*}\n", "\\varepsilon(X) &=k\\\\\n", "Var(X) &=2k\n", "\\end{align*}\n", "```\n", "```{figure} figures/L10/L10_11_chisquared.PNG\n", "---\n", "width: 60%\n", "---\n", "

Figure 11: The chi-squared distribution for different degrees of freedom k=1,2,10.

\n", "```\n", "\n", "#### Student's t-distribution\n", "- application for testing the significance of the differences in the means. be $t(k)$ a test variable with $k>0$, if A and B are independent random variables such that\n", "```{math}\n", "B \\sim \\chi^2(k) ~\\textrm{and} ~A \\sim \\cal {N(0,1)}\n", "```\n", "- the t-distribution can be written as:\n", "```{math}\n", "t(k) \\sim \\frac{A}{\\sqrt{B/k}}\n", "```\n", "- using the $\\Gamma$-function [](#gamma) the distribution can also be written as:\n", "```{math}\n", "F_{\\cal T}(t) = \\frac{\\Gamma((k+1)/2)}{\\sqrt{k\\pi}\\Gamma(k/2)} \\left( 1+\\frac{t^2}{k} \\right)^{\\frac{-(k+1)}{2}}\n", "```\n", "```{figure} figures/L10/L10_12_t.PNG\n", "---\n", "width: 60%\n", "---\n", "

Figure 12: The t-distribution and its cdf for different degrees of freedom k=1,2,10.

\n", "```\n", "- t-test?\n", "\n", "#### Fisher-F-distribution\n", "- application for testing the significance of the differences in the variance. for $\\chi^2$-distributed $K$ and $L$:\n", "```{math}\n", "K \\sim \\chi^2(k) ~\\textrm{and}~ L \\sim\\chi^2(l)\n", "```\n", "- the F-distribution is given by:\n", "```{math}\n", "{\\cal F}(k,l) = \\frac{K/k}{L/l}\n", "```\n", "- alternatively the probsbility density of the F-distribution is also given by:\n", "```{math}\n", "f_{\\cal F}(x) = \\frac{(k/l)^{k/2}\\Gamma((k+l)/2)}{\\Gamma(k/2)\\Gamma(l/2)} x^{(k-2)/2} \\left(1+\\frac{k}{l}x\\right)^{-(k+l)}\n", "```\n", "\n", "#### Summary of theoretical distributions\n", "```{figure} figures/L10/L10_13_summary.PNG\n", "---\n", "width: 70%\n", "---\n", "```\n", "\n", "#### Continous random vectors, multi-variate data\n", "- example: vectors **X** temperature and **Y** sea level pressure:\n", "```{math}\n", "\\mathbf{X}~\\textrm{and}~\\mathbf{Y} \\sim f_{\\mathbf{X},\\mathbf{Y}}(\\vec{x},\\vec{y})\n", "```\n", "```{figure} figures/L10/L10_14_correlation.PNG\n", "---\n", "width: 70%\n", "---\n", "

Figure 13: Three examples of 2-dimensional normal distributions for uncorrelated variables (left) and for correlated variables (middle and right). Note that the covariance is proportional to the correlation.

\n", "```" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.6" } }, "nbformat": 4, "nbformat_minor": 5 }