VIP - Variational Inversion Package with example implementations of Bayesian tomographic imaging




Bayesian inference has become an important methodology to solve inverse problems and to quantify uncertainties in their solutions. Variational inference is a method that provides probabilistic, Bayesian solutions efficiently by using optimisation. In this study we present a Python Variational Inversion Package (VIP), to solve inverse problems using variational inference methods. The package includes automatic differential variational inference (ADVI), Stein variational gradient descent (SVGD) and stochastic SVGD (sSVGD), and provides implementations of 2D travel time tomography and 2D full waveform inversion including test examples and solutions. Users can solve their own problems by supplying an appropriate forward function and a gradient calculation code. In addition, the package provides a scalable implementation which can be deployed easily on a desktop machine or using modern high performance computational facilities. The examples demonstrate that VIP is an efficient, scalable, extensible and user-friendly package, and can be used to solve a wide range of low or high dimensional inverse problems in practice.


Afanasiev, M., Boehm, C., van Driel, M., Krischer, L., Rietmann, M., May, D. A., Knepley, M. G., & Fichtner, A. (2019). Modular and flexible spectral-element waveform modelling in two and three dimensions. Geophysical Journal International, 216(3), 1675–1692. DOI:

Agata, R., Shiraishi, K., & Fujie, G. (2023). Bayesian seismic tomography based on velocity-space Stein variational gradient descent for physics-informed neural network. IEEE Transactions on Geoscience and Remote Sensing. DOI:

Ahmed, Z., Yunyue, L., & Arthur, C. (n.d.). Regularized seismic amplitude inversion via variational inference. Geophysical Prospecting, n/a(n/a).

Aki, K., & Lee, W. (1976). Determination of three-dimensional velocity anomalies under a seismic array using first P arrival times from local earthquakes: 1. A homogeneous initial model. Journal of Geophysical Research, 81(23), 4381–4399. DOI:

Andersen, K. E., Brooks, S. P., & Hansen, M. B. (2001). A Bayesian approach to crack detection in electrically conducting media. Inverse Problems, 17(1), 121. DOI:

Arnold, R., & Curtis, A. (2018). Interrogation theory. Geophysical Journal International, 214(3), 1830–1846. DOI:

Aster, R. C., Borchers, B., & Thurber, C. H. (2018). Parameter estimation and inverse problems. Elsevier. DOI:

Ba, J., Erdogdu, M. A., Ghassemi, M., Sun, S., Suzuki, T., Wu, D., & Zhang, T. (2022). Understanding the Variance Collapse of SVGD in High Dimensions. International Conference on Learning Representations.

Behnel, S., Bradshaw, R., Citro, C., Dalcin, L., Seljebotn, D. S., & Smith, K. (2010). Cython: The best of both worlds. Computing in Science & Engineering, 13(2), 31–39. DOI:

Bishop, C. M. (2006). Pattern recognition and machine learning. springer.

Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518), 859–877. DOI:

Bodin, T., & Sambridge, M. (2009). Seismic tomography with the reversible jump algorithm. Geophysical Journal International, 178(3), 1411–1436. DOI:

Brooks, S., Gelman, A., Jones, G., & Meng, X.-L. (2011). Handbook of Markov chain Monte Carlo. CRC press. DOI:

Chen, T., Fox, E., & Guestrin, C. (2014). Stochastic Gradient Hamiltonian Monte Carlo. In E. P. Xing & T. Jebara (Eds.), Proceedings of the 31st International Conference on Machine Learning (Vol. 32, Issue 2, pp. 1683–1691). PMLR.

Curtis, A., & Lomax, A. (2001). Prior information, sampling distributions, and the curse of dimensionality. Geophysics, 66(2), 372–378. DOI:

Dosso, S. E., Holland, C. W., & Sambridge, M. (2012). Parallel tempering for strongly nonlinear geoacoustic inversion. The Journal of the Acoustical Society of America, 132(5), 3030–3040. DOI:

Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987). Hybrid Monte Carlo. Physics Letters B, 195(2), 216–222. DOI:

Duchi, J., Hazan, E., & Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(Jul), 2121–2159.

Fichtner, A, Bunge, H.-P., & Igel, H. (2006). The adjoint method in seismology: I. Theory. Physics of the Earth and Planetary Interiors, 157(1–2), 86–104. DOI:

Fichtner, Andreas, Zunino, A., & Gebraad, L. (2018). Hamiltonian Monte Carlo solution of tomographic inverse problems. Geophysical Journal International, 216(2), 1344–1363. DOI:

Galetti, E., Curtis, A., Baptie, B., Jenkins, D., & Nicolson, H. (2017). Transdimensional Love-wave tomography of the British Isles and shear-velocity structure of the East Irish Sea Basin from ambient-noise interferometry. Geophysical Journal International, 208(1), 36–58. DOI:

Galetti, E., Curtis, A., Meles, G. A., & Baptie, B. (2015). Uncertainty loops in travel-time tomography from nonlinear wave physics. Physical Review Letters, 114(14), 148501. DOI:

Gallagher, K., Charvin, K., Nielsen, S., Sambridge, M., & Stephenson, J. (2009). Markov chain Monte Carlo (MCMC) sampling methods to determine optimal models, model resolution and model choice for Earth Science problems. Marine and Petroleum Geology, 26(4), 525–535. DOI:

Gallego, V., & Insua, D. R. (2018). Stochastic gradient MCMC with repulsive forces. ArXiv Preprint ArXiv:1812.00071.

Gebraad, L., Boehm, C., & Fichtner, A. (2020). Bayesian Elastic Full-Waveform Inversion Using Hamiltonian Monte Carlo. Journal of Geophysical Research: Solid Earth, 125(3), e2019JB018428. DOI:

Gong, C., Peng, J., & Liu, Q. (2019). Quantile Stein Variational Gradient Descent for Batch Bayesian Optimization. In K. Chaudhuri & R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning (Vol. 97, pp. 2347–2356). PMLR.

Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Byesian model determination. Biometrika, 711–732. DOI:

Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. DOI:

Hawkins, R., & Sambridge, M. (2015). Geophysical imaging using trans-dimensional trees. Geophysical Journal International, 203(2), 972–1000. DOI:

Hukushima, K., & Nemoto, K. (1996). Exchange Monte Carlo method and application to spin glass simulations. Journal of the Physical Society of Japan, 65(6), 1604–1608. DOI:

Izzatullah, M., Alkhalifah, T., Romero, J., Corrales, M., Luiken, N., & Ravasi, M. (2023). Plug-and-Play Stein variational gradient descent for Bayesian post-stack seismic inversion. 84th EAGE Annual Conference & Exhibition, 2023(1), 1–5. DOI:

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. ArXiv Preprint ArXiv:1412.6980.

Komatitsch, D., Tromp, J., Garg, R., Gharti, H. N., Nagaso, M., Oral, E., & et al. (2023). SPECFEM/specfem3d: SPECFEM3D v4.1.0 (v4.1.0) [Computer software]. Zenodo.

Kubrusly, C., & Gravier, J. (1973). Stochastic approximation algorithms and applications. 1973 IEEE Conference on Decision and Control Including the 12th Symposium on Adaptive Processes, 763–766. DOI:

Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., & Blei, D. M. (2017). Automatic differentiation variational inference. The Journal of Machine Learning Research, 18(14), 1–45.

Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1), 79–86. DOI:

Kumar, R., Kotsi, M., Siahkoohi, A., & Malcolm, A. (2021). Enabling uncertainty quantification for seismic data preprocessing using normalizing flows (NF)—An interpolation example. First International Meeting for Applied Geoscience & Energy, 1515–1519. DOI:

Li, X., Bürgi, P. M., Ma, W., Noh, H. Y., Wald, D. J., & Xu, S. (2023). DisasterNet: Causal Bayesian Networks with Normalizing Flows for Cascading Hazards Estimation from Satellite Imagery. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4391–4403. DOI:

Lions, J. L. (1971). Optimal control of systems governed by partial differential equations (Vol. 170). Springer. DOI:

Liu, Q., & Wang, D. (n.d.). In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in Neural Information Processing Systems.

Lomas, A., Luo, S., Irakarama, M., Johnston, R., Vyas, M., & Shen, X. (2023). 3D Probabilistic Full Waveform Inversion: Application to Gulf of Mexico Field Data. 84th EAGE Annual Conference & Exhibition, 2023(1), 1–5. DOI:

Ma, Y.-A., Chen, T., & Fox, E. (n.d.). Advances in Neural Information Processing Systems.

Malinverno, A. (2002). Parsimonious Byesian Markov chain Monte Carlo inversion in a nonlinear geophysical problem. Geophysical Journal International, 151(3), 675–688. DOI:

Malinverno, A., Leaney, S., & others. (2000). A Monte Carlo method to quantify uncertainty in the inversion of zero-offset VSP data. 2000 SEG Annual Meeting. DOI:

Martin, J., Wilcox, L. C., Burstedde, C., & Ghattas, O. (2012). A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion. SIAM Journal on Scientific Computing, 34(3), A1460–A1487. DOI:

McKean, S., Priest, J., Dettmer, J., Fradelizio, G., & Eaton, D. (2023). Separating Hydraulic Fracturing Microseismicity From Induced Seismicity by Bayesian Inference of Non-Linear Pressure Diffusivity. Geophysical Research Letters, 50(14), e2022GL102131. DOI:

Metropolis, N., & Ulam, S. (1949). The Monte Carlo method. Journal of the American Statistical Association, 44(247), 335–341. DOI:

Mosegaard, K., & Sambridge, M. (2002). Monte Carlo analysis of inverse problems. Inverse Problems, 18(3), R29. DOI:

Mosegaard, K., & Tarantola, A. (1995). Monte Carlo sampling of solutions to inverse problems. Journal of Geophysical Research: Solid Earth, 100(B7), 12431–12447. DOI:

Nicolson, H., Curtis, A., & Baptie, B. (2014). Rayleigh wave tomography of the British Isles from ambient seismic noise. Geophysical Journal International, 198(2), 637–655. DOI:

Nicolson, H., Curtis, A., Baptie, B., & Galetti, E. (2012). Seismic interferometry and ambient noise tomography in the British Isles. Proceedings of the Geologists’ Association, 123(1), 74–86. DOI:

O’Hagan, A., & Forster, J. J. (2004). Kendall’s advanced theory of statistics, volume 2B: Bayesian inference (Vol. 2). Arnold.

Oksendal, B. (2013). Stochastic differential equations: an introduction with applications. Springer Science & Business Media.

Orozco, R., Louboutin, M., Siahkoohi, A., Rizzuti, G., van Leeuwen, T., & Herrmann, F. (2023). Amortized Normalizing Flows for Transcranial Ultrasound with Uncertainty Quantification. ArXiv Preprint ArXiv:2303.03478.

Parisi, G. (1988). Statistical field theory. Addison-Wesley. DOI:

Pinder, T., Nemeth, C., & Leslie, D. (2020). Stein variational Gaussian processes. ArXiv Preprint ArXiv:2009.12141.

Plessix, R.-E. (2006). A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophysical Journal International, 167(2), 495–503. DOI:

Ramgraber, M., Weatherl, R., Blumensaat, F., & Schirmer, M. (2021). Non-Gaussian Parameter Inference for Hydrogeological Models Using Stein Variational Gradient Descent. Water Resources Research, 57(4), e2020WR029339. DOI:

Ramirez, A. L., Nitao, J. J., Hanley, W. G., Aines, R., Glaser, R. E., Sengupta, S. K., Dyer, K. M., Hickling, T. L., & Daily, W. D. (2005). Stochastic inversion of electrical resistivity changes using a Markov Chain Monte Carlo approach. Journal of Geophysical Research: Solid Earth, 110(B2). DOI:

Rawlinson, N. (2005). FMST: fast marching surface tomography package–Instructions. Research School of Earth Sciences, Australian National University, Canberra, 29, 47.

Rawlinson, N., & Sambridge, M. (2004). Multiple reflection and transmission phases in complex layered media using a multistage fast marching method. Geophysics, 69(5), 1338–1350. DOI:

Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 400–407. DOI:

Roberts, G. O., Tweedie, R. L., & others. (1996). Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli, 2(4), 341–363. DOI:

Rocklin, M., & others. (2015). Dask: Parallel computation with blocked algorithms and task scheduling. Proceedings of the 14th Python in Science Conference, 130, 136. DOI:

Rücker, C., Günther, T., & Wagner, F. M. (2017). pyGIMLi: An open-source library for modelling and inversion in geophysics. Computers & Geosciences, 109, 106–123. DOI:

Sambridge, M. (2013). A parallel tempering algorithm for probabilistic sampling and multimodal optimization. Geophysical Journal International, ggt342. DOI:

Sambridge, M., & Mosegaard, K. (2002). Monte Carlo methods in geophysical inverse problems. Reviews of Geophysics, 40(3), 3–1. DOI:

Sen, M. K., & Biswas, R. (2017). Transdimensional seismic inversion using the reversible jump Hamiltonian Monte Carlo algorithm. Geophysics, 82(3), R119–R134. DOI:

Shen, W., Ritzwoller, M. H., Schulte-Pelkum, V., & Lin, F.-C. (2012). Joint inversion of surface wave dispersion and receiver functions: a Byesian Monte-Carlo approach. Geophysical Journal International, 192(2), 807–836. DOI:

Siahkoohi, A., Orozco, R., Rizzuti, G., & Herrmann, F. J. (2022). Wave-equation-based inversion with amortized variational Bayesian inference. ArXiv Preprint ArXiv:2203.15881.

Siahkoohi, A., Rizzuti, G., & Herrmann, F. J. (2022). Deep Bayesian inference for seismic imaging with tasks. Geophysics, 87(5), S281–S302. DOI:

Siahkoohi, A., Rizzuti, G., Witte, P. A., & Herrmann, F. J. (2020). Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows. ArXiv Preprint ArXiv:2007.07985.

Smith, J. D., Ross, Z. E., Azizzadenesheli, K., & Muir, J. B. (2022). HypoSVI: Hypocentre inversion with Stein variational inference and physics informed neural networks. Geophysical Journal International, 228(1), 698–710. DOI:

Tarantola, A. (1984). Inversion of seismic reflection data in the acoustic approximation. Geophysics, 49(8), 1259–1266. DOI:

Tarantola, A. (1988). Theoretical background for the inversion of seismic waveforms, including elasticity and attenuation. In Scattering and Attenuations of Seismic Waves, Part I (pp. 365–399). Springer. DOI:

Tarantola, A. (2005). Inverse problem theory and methods for model parameter estimation (Vol. 89). SIAM. DOI:

Team, S. D., & others. (2016). Stan modeling language users guide and reference manual. Technical Report.

Tromp, J., Tape, C., & Liu, Q. (2005). Seismic tomography, adjoint methods, time reversal and banana-doughnut kernels. Geophysical Journal International, 160(1), 195–216. DOI:

Valentine, A. P., & Sambridge, M. (2020). Gaussian process models—I. A framework for probabilistic continuous inverse theory. Geophysical Journal International, 220(3), 1632–1647. DOI:

Wang, D., Tang, Z., Bajaj, C., & Liu, Q. (2019). Stein variational gradient descent with matrix-valued kernels. Advances in Neural Information Processing Systems, 7836–7846.

Wang, W., McMechan, G. A., & Ma, J. (2023). Re-weighted variational full waveform inversions. Geophysics, 88(4), 1–61. DOI:

Wathelet, M., Chatelain, J.-L., Cornou, C., Giulio, G. D., Guillier, B., Ohrnberger, M., & Savvaidis, A. (2020). Geopsy: A user-friendly open-source tool set for ambient vibration processing. Seismological Research Letters, 91(3), 1878–1889. DOI:

Welling, M., & Teh, Y. W. (2011). Bayesian learning via stochastic gradient Langevin dynamics. Proceedings of the 28th International Conference on Machine Learning (ICML-11), 681–688.

Zaroli, C. (2016). Global seismic tomography using Backus–Gilbert inversion. Geophysical Supplements to the Monthly Notices of the Royal Astronomical Society, 207(2), 876–888. DOI:

Zaroli, C. (2019). Seismic tomography using parameter-free Backus–Gilbert inversion. Geophysical Journal International, 218(1), 619–630. DOI:

Zaroli, C., Koelemeijer, P., & Lambotte, S. (2017). Toward seeing the Earth’s interior through unbiased tomographic lenses. Geophysical Research Letters, 44(22), 11–399. DOI:

Zeiler, M. D. (2012). Adadelta: an adaptive learning rate method. ArXiv Preprint ArXiv:1212.5701.

Zhang, Chendong, & Chen, T. (2022). Bayesian slip inversion with automatic differentiation variational inference. Geophysical Journal International, 229(1), 546–565. DOI:

Zhang, Cheng, Bütepage, J., Kjellström, H., & Mandt, S. (2018). Advances in variational inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8), 2008–2026. DOI:

Zhang, X., & Curtis, A. (2020a). Seismic tomography using variational inference methods. Journal of Geophysical Research: Solid Earth, 125(4), e2019JB018589. DOI:

Zhang, X., & Curtis, A. (2020b). Variational full-waveform inversion. Geophysical Journal International, 222(1), 406–411. DOI:

Zhang, X., & Curtis, A. (2021). Bayesian Full-waveform Inversion with Realistic Priors. Geophysics, 86(5), 1–20. DOI:

Zhang, X., & Curtis, A. (2022). Interrogating probabilistic inversion results for subsurface structural information. Geophysical Journal International, 229(2), 750–757. DOI:

Zhang, X., Curtis, A., Galetti, E., & de Ridder, S. (2018). 3-D Monte Carlo surface wave tomography. Geophysical Journal International, 215(3), 1644–1658. DOI:

Zhang, X., Lomas, A., Zhou, M., Zheng, Y., & Curtis, A. (2023). 3-D Bayesian variational full waveform inversion. Geophysical Journal International, 234(1), 546–561. DOI:

Zhang, Z., & Curtis, A. (2023). Variational Inversion Package.

Zhao, X., & Curtis, A. (2024). Bayesian inversion, uncertainty analysis and interrogation using boosting variational inference. Journal of Geophysical Research: Solid Earth, 129(1), e2023JB027789. DOI:

Zhao, X., Curtis, A., & Zhang, X. (2022a). Bayesian seismic tomography using normalizing flows. Geophysical Journal International, 228(1), 213–239. DOI:

Zhao, X., Curtis, A., & Zhang, X. (2022b). Interrogating Subsurface Structures using Probabilistic Tomography: an example assessing the volume of Irish Sea basins. Journal of Geophysical Research: Solid Earth, 127(4), e2022JB024098. DOI:

Zhao, Z., & Sen, M. K. (2019). A gradient based MCMC method for FWI and uncertainty analysis. In SEG Technical Program Expanded Abstracts 2019 (pp. 1465–1469). Society of Exploration Geophysicists. DOI:

Zhdanov, M. S. (2002). Geophysical inverse theory and regularization problems (Vol. 36). Elsevier. DOI:

Zunino, A., Gebraad, L., Ghirotto, A., & Fichtner, A. (2023). HMCLab: a framework for solving diverse geophysical inverse problems using the Hamiltonian Monte Carlo method. ArXiv Preprint ArXiv:2303.10047. DOI:

Additional Files



How to Cite

Zhang, X., & Curtis, A. (2024). VIP - Variational Inversion Package with example implementations of Bayesian tomographic imaging. Seismica, 3(1).



Reports (excl. Fast Reports)

Funding data