Current gravitational wave (GW) experiments cover a large frequency range from nHz to kHz. Beyond that, the regime of high frequency GWs is both extremely challenging challenging, and highly motivated as a unique window to the very early Universe. In this talk I will discuss the possibility to use axion detectors to search for high frequency GWs, making use of the observation that GWs generate oscillating electromagnetic effects in the vicinity of external electric and magnetic fields. Remarkably, some existing axion experiments (in particular CAST) have the sensitivity to detect even single gravitons, and I will discuss the implications of this.