.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "generated/examples/misc/plot_beads_preprocessing.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_generated_examples_misc_plot_beads_preprocessing.py: Preprocessing for beads ----------------------- The Baseline Estimation And Denoising with Sparsity (:meth:`~.Baseline.beads`) algorithm is a robust method for both performing baseline subtraction and removing noise. One of the main drawbacks of the original algorithm is that it requires that both ends of the data to be at zero. This example will explore the consequences of this as well as a preprocessing step proposed by `Navarro-Huerta, J.A., et al. Assisted baseline subtraction in complex chromatograms using the BEADS algorithm. Journal of Chromatography A, 2017, 1507, 1-10` implemented in pybaselines that helps to address this issue. .. GENERATED FROM PYTHON SOURCE LINES 15-53 .. code-block:: Python from itertools import cycle import matplotlib.pyplot as plt import numpy as np from pybaselines import Baseline from pybaselines.misc import _parabola from pybaselines.utils import gaussian def make_data(x, baseline_type=0): """Creates the data and baseline where the endpoints are either zero or nonzero.""" signal = ( gaussian(x, 9, 100, 12) + gaussian(x, 6, 180, 5) + gaussian(x, 8, 350, 11) + gaussian(x, 15, 400, 18) + gaussian(x, 6, 550, 6) + gaussian(x, 13, 700, 8) + gaussian(x, 9, 800, 9) + gaussian(x, 9, 880, 7) ) if baseline_type == 0: # simple parabola that ends at 0 on both ends baseline = 2e-5 * (x - 500)**2 - 5 elif baseline_type == 1: # exponentially decaying baseline baseline = 10 - 10 * np.exp(-x / 600) else: # very complicated baseline # approximate a logistic baseline by integrating a Gaussian baseline = -np.cumsum(gaussian(x, 0.05, 400, 100)) + gaussian(x, 3, 800, 100) - 5 noise = np.random.default_rng(0).normal(0, 0.2, len(x)) y = signal + baseline + noise return y, baseline linestyles = cycle(('-', '--')) .. GENERATED FROM PYTHON SOURCE LINES 55-60 Three different baselines will be compared: (1) a simple polynomial that ends at zero on both ends; (2) an exponential baseline that is only zero on one end; (3) a complex baseline that is not near 0 on either end. Each plot shows the raw data, the parabola used to preprocess the data as proposed by Navarro-Huerta, J.A., et al. to get both ends of the data to zero, and the raw data after subtracting the parabola. .. GENERATED FROM PYTHON SOURCE LINES 60-79 .. code-block:: Python x = np.linspace(0, 1000, 1000) baseline_fitter = Baseline(x_data=x) figure, axes = plt.subplots(nrows=3) for i in range(3): y, baseline = make_data(x, baseline_type=i) parabola = _parabola(y) labels = ('raw data', 'fit parabola', 'processed data') if i == 0 else (None, None, None) axes[i].plot(y, label=labels[0]) axes[i].plot(parabola, '--', label=labels[1]) axes[i].plot(y - parabola, label=labels[2]) axes[i].set_xticks([]) axes[i].set_title(f'Baseline {i + 1}', y=0.95) figure.legend(ncol=3, loc='upper center') figure.tight_layout() figure.subplots_adjust(top=0.85) .. image-sg:: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_001.png :alt: Baseline 1, Baseline 2, Baseline 3 :srcset: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 80-84 For the polynomial baseline that is zero at both ends, the beads algorithm gives similar results for both the raw data and the parabola-subtracted data. This indicates that any preprocessing is not needed for such a baseline, but also shows that the preprocessing does not negatively influence the result. .. GENERATED FROM PYTHON SOURCE LINES 84-95 .. code-block:: Python y, baseline = make_data(x, baseline_type=0) plt.figure() plt.plot(y) for subtract_parabola in (False, True): fit_baseline = baseline_fitter.beads( y, lam_0=0.005, lam_1=0.01, lam_2=1, fit_parabola=subtract_parabola )[0] plt.plot(fit_baseline, ls=next(linestyles), label=f'parabola subtracted: {subtract_parabola}') plt.plot(baseline, ':', label='true baseline') plt.legend() .. image-sg:: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_002.png :alt: plot beads preprocessing :srcset: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 96-99 For the baseline that is zero only on the left side, the non-processed data now incorrectly fits the data on the right side due to the non-zero value. The parabola-subtracted data, however, fits the data well on both ends and gives the expected baseline. .. GENERATED FROM PYTHON SOURCE LINES 99-110 .. code-block:: Python y, baseline = make_data(x, baseline_type=1) plt.figure() plt.plot(y) for subtract_parabola in (False, True): fit_baseline = baseline_fitter.beads( y, lam_0=0.015, lam_1=0.1, lam_2=1, fit_parabola=subtract_parabola )[0] plt.plot(fit_baseline, ls=next(linestyles), label=f'parabola subtracted: {subtract_parabola}') plt.plot(baseline, ':', label='true baseline') plt.legend() .. image-sg:: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_003.png :alt: plot beads preprocessing :srcset: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 111-112 The parabola subtraction method works well even for more complicated baselines, as seen below. .. GENERATED FROM PYTHON SOURCE LINES 112-125 .. code-block:: Python y, baseline = make_data(x, baseline_type=2) plt.figure() plt.plot(y) for subtract_parabola in (False, True): fit_baseline = baseline_fitter.beads( y, lam_0=0.00006, lam_1=0.00008, lam_2=0.05, fit_parabola=subtract_parabola, tol=1e-3, freq_cutoff=0.04, asymmetry=3 )[0] plt.plot(fit_baseline, ls=next(linestyles), label=f'parabola subtracted: {subtract_parabola}') plt.plot(baseline, ':', label='true baseline') plt.legend() plt.show() .. image-sg:: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_004.png :alt: plot beads preprocessing :srcset: /generated/examples/misc/images/sphx_glr_plot_beads_preprocessing_004.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.954 seconds) .. _sphx_glr_download_generated_examples_misc_plot_beads_preprocessing.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_beads_preprocessing.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_beads_preprocessing.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_beads_preprocessing.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_