{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic pyerrors example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import pyerrors, as well as autograd wrapped numpy and matplotlib."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pyerrors as pe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.style.use('./base_style.mplstyle')\n",
    "plt.rc('text', usetex=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use numpy to generate some fake data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_sample1 = np.random.normal(2.0, 0.5, 1000)\n",
    "test_sample2 = np.random.normal(1.0, 0.1, 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From this we can construct `Obs`, which are the basic object of `pyerrors`. For each sample we give to the obs, we also have to specify an ensemble/replica name. In this example we assume that both datasets originate from the same gauge field ensemble labeled 'ens1'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs1 = pe.Obs([test_sample1], ['ens1'])\n",
    "obs2 = pe.Obs([test_sample2], ['ens1'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now combine these two observables into a third one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs3 = np.log(obs1 ** 2 / obs2 ** 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`pyerrors` overloads all basic math operations, the user can work with these `Obs` as if they were real numbers. The proper resampling is performed in the background via automatic differentiation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we are now interested in the error of obs3, we can use the `gamma_method` to compute it and then print the object to the notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Obs[1.387(19)]\n"
     ]
    }
   ],
   "source": [
    "obs3.gamma_method()\n",
    "print(obs3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With print level 1 we can take a look at the integrated autocorrelation time estimated by the automatic windowing procedure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result\t 1.38669742e+00 +/- 1.94840399e-02 +/- 9.74201997e-04 (1.405%)\n",
      "  t_int\t 5.01998002e-01 +/- 4.47213596e-02 S = 2.00\n"
     ]
    }
   ],
   "source": [
    "obs3.print(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected the random data from numpy exhibits no autocorrelation ($\\tau_\\text{int}\\,\\approx0.5$). It can still be interesting to have a look at the window size dependence of the integrated autocorrelation time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 14656 characters]\n",
      "text/plain": [
       "<Figure size 640x395.55 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "obs3.plot_tauint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This figure shows the windowsize dependence of the integrated autocorrelation time. The red vertical line signalizes the window chosen by the automatic windowing procedure with $S=2.0$.\n",
    "Choosing a larger windowsize would not significantly alter $\\tau_\\text{int}$, so everything seems to be correct here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Correlated data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now generate fake data with given covariance matrix and integrated autocorrelation times:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "cov = np.array([[0.5, -0.2], [-0.2, 0.3]]) # Covariance matrix\n",
    "tau = [4, 8] # Autocorrelation times\n",
    "c_obs1, c_obs2 = pe.misc.gen_correlated_data([2.8, 2.1], cov, 'ens1', tau)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and once again combine the two `Obs` to a new one with arbitrary math operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result\t 3.27194697e-01 +/- 1.79228480e+00 +/- 3.07835024e-01 (547.773%)\n",
      "  t_int\t 5.31748262e+00 +/- 1.57262234e+00 S = 2.00\n"
     ]
    }
   ],
   "source": [
    "c_obs3 = np.sin(c_obs1 / c_obs2 - 1)\n",
    "c_obs3.gamma_method()\n",
    "c_obs3.print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time we see a significant autocorrelation so it is worth to have a closer look at the normalized autocorrelation function (rho) and the integrated autocorrelation time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 23844 characters]\n",
      "text/plain": [
       "<Figure size 640x395.55 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 19352 characters]\n",
      "text/plain": [
       "<Figure size 640x395.55 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "c_obs3.plot_rho()\n",
    "c_obs3.plot_tauint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now redo the error analysis and alter the value of S or attach a tail to the autocorrelation function to take into account long range autocorrelations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result\t 3.27194697e-01 +/- 1.88231459e+00 +/- 2.01855751e-01 (575.289%)\n",
      "  t_int\t 5.86511391e+00 +/- 2.16269625e+00 tau_exp = 20.00,  N_sigma = 1\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 23892 characters]\n",
      "text/plain": [
       "<Figure size 640x395.55 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "c_obs3.gamma_method(tau_exp=20)\n",
    "c_obs3.print()\n",
    "c_obs3.plot_tauint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jackknife"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For comparison and as a crosscheck, we can do a jackknife binning analysis. We compare the result for different binsizes with the result from the gamma method. Besides the more robust approach of the gamma method, it can also be shown that the systematic error of the error decreases faster with $N$ in comparison to the binning approach (see hep-lat/0306017)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Binning analysis:\n",
      "Result:\t 3.27194697e-01 +/- 1.30323584e+00 +/- 1.74847436e-01 (398.306%)\n",
      "Result:\t 3.27194697e-01 +/- 1.42921199e+00 +/- 3.13124657e-01 (436.808%)\n",
      "Result:\t 3.27194697e-01 +/- 1.36761713e+00 +/- 4.28131883e-01 (417.983%)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 16408 characters]\n",
      "text/plain": [
       "<Figure size 640x395.55 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result from the automatic windowing procedure for comparison:\n",
      "Result\t 3.27194697e-01 +/- 1.78414777e+00 +/- 2.73504675e-01 (545.286%)\n",
      "  t_int\t 5.26930916e+00 +/- 1.36902941e+00 S = 1.50\n",
      "Result\t 3.27194697e-01 +/- 1.79228480e+00 +/- 3.07835024e-01 (547.773%)\n",
      "  t_int\t 5.31748262e+00 +/- 1.57262234e+00 S = 2.00\n",
      "Result\t 3.27194697e-01 +/- 1.67905409e+00 +/- 3.16358031e-01 (513.167%)\n",
      "  t_int\t 4.66682386e+00 +/- 1.53936903e+00 S = 3.00\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 19172 characters]\n",
      "text/plain": [
       "<Figure size 640x395.55 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pyerrors.jackknifing as jn\n",
    "jack1 = jn.generate_jack(c_obs1, max_binsize=50)\n",
    "jack2 = jn.generate_jack(c_obs2, max_binsize=50)\n",
    "jack3 = jn.derived_jack(lambda x: np.sin(x[0] / x[1] - 1), [jack1, jack2])\n",
    "\n",
    "print('Binning analysis:')\n",
    "jack3.print(binsize=10)\n",
    "jack3.print(binsize=25)\n",
    "jack3.print(binsize=50)\n",
    "\n",
    "jack3.plot_tauint()\n",
    "\n",
    "print('Result from the automatic windowing procedure for comparison:')\n",
    "c_obs3.gamma_method(S=1.5)\n",
    "c_obs3.print()\n",
    "c_obs3.gamma_method(S=2)\n",
    "c_obs3.print()\n",
    "c_obs3.gamma_method(S=3)\n",
    "c_obs3.print()\n",
    "\n",
    "c_obs3.gamma_method(S=2)\n",
    "c_obs3.plot_tauint()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this specific example the binned Jackknife procedure seems to underestimate the final error, the deduced intergrated autocorrelation time depends strongly on the chosen binsize. The automatic windowing procedure displayed for comparison gives more robust results for this example."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
