A planet's climate can be strongly affected by its orbital eccentricity and obliquity. Here we use a one-dimensional energy balance model modified to include a simple runaway greenhouse (RGH) parameterization to explore the effects of these two parameters on the climate of Earth-like aqua planets - completely ocean-covered planets - orbiting F-, G-, K-, and M-dwarf stars. We find that the range of instellations for which planets exhibit habitable surface conditions throughout an orbit decreases with increasing eccentricity. However, the appearance of temporarily habitable conditions during an orbit creates an eccentric habitable zone (EHZ) that is sensitive to orbital eccentricity and obliquity, planetary latitude, and the spectral type of the host star. We find that the fraction of a planet's orbit over which it exhibits habitable surface conditions is larger on eccentric planets orbiting M-dwarf stars, due to the lower broadband planetary albedos of these planets. Planets with larger obliquities have smaller EHZs, but exhibit warmer climates if they do not enter a snowball state during their orbits. We also find no transient RGH state on planets at all eccentricities. Rather, planets spend their entire orbits either in an RGH or not. For G-dwarf planets receiving 100% of the modern solar constant and with eccentricities above 0.55, an entire Earth ocean inventory can be lost in 3.6 Gyr. M-dwarf planets, due to their larger incident X-ray and extreme ultraviolet flux, can become desiccated in only 690 Myr with eccentricities above 0.38. This work has important implications for eccentric planets that may exhibit surface habitability despite technically departing from the traditional habitable zone as they orbit their host stars.