According to classic stomatal optimization theory, plant stomata are regulated to maximize carbon assimilation for a given water loss. A key component of stomatal optimization models is marginal water-use efficiency (mWUE), the ratio of the change of transpiration to the change in carbon assimilation. Although the mWUE is often assumed to be constant, variability of mWUE under changing hydrologic conditions has been reported. However, there has yet to be a consensus on the patterns of mWUE variabilities and their relations with atmospheric aridity. We investigate the dynamics of mWUE in response to vapor pressure deficit (VPD) and aridity index using carbon and water fluxes from 115 eddy covariance towers available from the global database FLUXNET. We demonstrate a non-linear mWUE-VPD relationship at a sub-daily scale in general; mWUE varies substantially at both low and high VPD levels. However, mWUE remains relatively constant within the mid-range of VPD. Despite the highly non-linear relationship between mWUE and VPD, the relationship can be informed by the strong linear relationship between ecosystem-level inherent water-use efficiency (IWUE) and mWUE using the slope, m*. We further identify site-specific m* and its variability with changing site-level aridity across six vegetation types. We suggest accurately representing the relationship between IWUE and VPD using Michaelis–Menten or quadratic functions to ensure precise estimation of mWUE variability for individual sites.