Rayleigh-wave dispersion curve inversion is an important technique for estimating subsurface shear-wave velocity structures; however, the inversion process is highly nonlinear due to the presence of multiple parameters and multiple extrema. Traditional local optimization algorithms are highly dependent on the initial model and prone to converging to local optima, whereas conventional global optimization algorithms often suffer from slow convergence and limited inversion accuracy. To address these limitations, this study introduces the mantis shrimp optimization algorithm (MShOA) for Rayleigh-wave dispersion curve inversion. The algorithm simulates the visual perception and behavioral strategies of mantis shrimp and employs a polarization-type indicator mechanism to balance global exploration and local exploitation during optimization. The performance of MShOA is evaluated against three theoretical geological models, including a velocity-increasing model, a low-velocity interlayer model, and a high-velocity interlayer model. Both noise-free and noisy dispersion curves contaminated with 10% random noise are considered. In addition, multi-order dispersion curve inversion experiments and measured microtremor data from the Wyoming region of the United States are used to further assess the proposed method’s applicability. The inversion results demonstrate that MShOA can accurately retrieve shear-wave velocity and layer thickness parameters across diverse geological conditions. For the noise-free models, the maximum relative error is 2.32%, while for the noisy models it is 1.64%. Comparative experiments indicate that MShOA achieves lower inversion errors and smaller parameter standard deviations than particle swarm optimization, dung beetle optimization, and mantis shrimp algorithm under the same search conditions. The results from both theoretical and measured data indicate that MShOA provides stable inversion performance and effectively handles complex Rayleigh-wave dispersion curve inversion problems. The proposed method offers a feasible approach for high-resolution shallow subsurface characterization and surface-wave inversion applications.