Coral reef research has predominantly focused on the effect of temperature on the breakdown of coral-dinoflagellate symbioses. However, less is known about how increasing temperature affects the establishment of new coral-dinoflagellate associations. Inter-partner specificity and environment-dependent colonization are two constraints proposed to limit the acquisition of more heat tolerant symbionts. Here, we investigated the symbiotic dynamics of various photosymbionts in different host genotypes under “optimal” and elevated temperature conditions. To do this, we inoculated symbiont-free polyps of the sea anemone Exaiptasia pallida originating from Hawaii (H2), North Carolina (CC7), and the Red Sea (RS) with the same mixture of native symbiont strains (Breviolum minutum, Symbiodinium linucheae, S. microadriaticum, and a Breviolum type from the Red Sea) at 25 and 32 °C, and assessed their ITS2 composition, colonization rates, and PSII photochemical efficiency (Fv/Fm). Symbiont communities across thermal conditions differed significantly for all hosts, suggesting that temperature rather than partner specificity had a stronger effect on symbiosis establishment. Overall, we detected higher abundances of more heat resistant Symbiodiniaceae types in the 32 °C treatments. Our data further showed that PSII photophysiology under elevated temperature improved with thermal pre-exposure (i.e., higher Fv/Fm), yet, this effect depended on host genotype and was influenced by active feeding as photochemical efficiency dropped in response to food deprivation. These findings highlight the role of temperature and partner fidelity in the establishment and performance of symbiosis and demonstrate the importance of heterotrophy for symbiotic cnidarians to endure and recover from stress.