Effective responses to intracellular pathogens are characterized by T cell clones with a broad affinity range for their cognate peptide and diverse functional phenotypes. How T cell clones are selected throughout the response to retain a breadth of avidities remains unclear. Here, we demonstrate that direct sensing of the cytokine IFN-γ by CD8+ T cells coordinates avidity and differentiation during infection. IFN-γ promotes the expansion of low-avidity T cells, allowing them to overcome the selective advantage of high-avidity T cells, whilst reinforcing high-avidity T cell entry into the memory pool, thus reducing the average avidity of the primary response and increasing that of the memory response. IFN-γ in this context is mainly provided by virtual memory T cells, an antigen-inexperienced subset with memory features. Overall, we propose that IFN-γ and virtual memory T cells fulfil a critical immunoregulatory role by enabling the coordination of T cell avidity and fate.