Control and manipulation of light is key to many information and communication technologies. Despite the many impressive achievements in this direction, many information-processing functionalities are still missing in all-optical devices. These missing functionalities are mostly related to the fact that photons in linear media do not mutually interact. Consequently, in linear media light cannot generally be controlled with light. Nonlinear media offer a solution to this problem, by mediating effective photon-photon interactions. However, in weakly nonlinear media extreme excitation densities are required, while highly nonlinear media are often difficult (if not currently impossible) to integrate with other existing technologies like cavities or resonators. In this thesis, we explore ways to overcome these issues. We present experiments with open-access tunable cavities which allow easy integration of materials inside, while storing light for a long time so that it may interact with the material of choice. We focus on two particular nonlinear media, where the nonlinear response can have different origins. In Chapters 2 and 3 we couple resonators to oils. Oils provide a strong albeit slow room-temperature thermo-optical nonlinearity, which enabled us to discover novel behavior. In Chapters 4 and 5 we explore semiconductor cavities in the strong light-matter coupling regime. Therein, since the relevant quasi-particles are cavity polaritons formed by strong exciton-photon coupling, optical nonlinearity due to polariton-polariton interactions can emerge.