The Bayesian approach to inverse problems provides a principled and flexible methodology for estimating unknown parameters in partial differential equations (PDEs), making it an important frontier for statistical inference from the sparse, noise-corrupted data characteristic of physics-informed settings. Recent algorithmic advances and growing computational capacity have brought a new class of high-dimensional PDE inference problems into focus, with rich mathematical challenges and impactful applications. This talk will survey this emerging field and describe some of our recent and ongoing work in this domain. We consider some model PDE inference problems related to the measurement of fluid flows. We will then review some recent developments in Markov Chain Monte Carlo (MCMC) sampling. In particular, we will introduce several of our new algorithms, multiproposal preconditioned Crank-Nicolson (mpCN) - with local and global variants - as well as surrogate trajectory Hamiltonian methods. These methods are carefully tailored for resolving high-dimensional, strongly anisotropic posteriors with nonlinear correlation geometry while taking advantage of parallel computing architectures. These new MCMC algorithms have broad scope, but we are using our fluids model problems as a physically motivated challenge test bed. We describe how our new methods have been derived on the basis of an "involutive theory" that we developed. This theory provides a unified view of reversible, Metropolized MCMC while serving as an oracle to derive and validate new methods. We will also outline some rigorous bounds on mixing rates for mpCN that we derived on the basis of the so-called Weak Harris theory. These bounds demonstrate that our methods partially beat the "curse of dimensionality". This is joint work with Jeff Borggaard (Virginia Tech), Giulia Carigi (Indiana), Andrew Holbrook (UCLA), Justin Krometis (Virginia Tech), Cecilia Mondaini (Drexel), and Guillermina Senn (Indiana, NTNU).