The trajectories of acquired immunity to severe acute respiratory syndrome coronavirus 2 infection are not fully understood. We present a detailed longitudinal cohort study of UK healthcare workers prior to vaccination, presenting April-June 2020 with asymptomatic or symptomatic infection. Here we show a highly variable range of responses, some of which (T cell interferon-gamma ELISpot, N-specific antibody) wane over time, while others (spike-specific antibody, B cell memory ELISpot) are stable. We use integrative analysis and a machine-learning approach (SIMON - Sequential Iterative Modeling OverNight) to explore this heterogeneity. We identify a subgroup of participants with higher antibody responses and interferon-gamma ELISpot T cell responses, and a robust trajectory for longer term immunity associates with higher levels of neutralising antibodies against the infecting (Victoria) strain and also against variants B.1.1.7 (alpha) and B.1.351 (beta). These variable trajectories following early priming may define subsequent protection from severe disease from novel variants.