A persistent microbial seed bank is postulated to sustain the marine biosphere, and recent findings show that prokaryotic taxa present in the ocean's surface dominate prokaryotic communities throughout the water column. Yet, environmental conditions exert a tight control on the activity of prokaryotes, and drastic changes in these conditions are known to occur from the surface to deep waters. The simultaneous characterization of the total (DNA) and active (i.e. with potential for protein synthesis, RNA) free-living communities in 13 stations distributed across the tropical and subtropical global ocean allowed us to assess their change in structure and diversity along the water column. We observed that active communities were surprisingly more similar along the vertical gradient than total communities. Looking at the vertical connectivity of the active vs. the total communities, we found that taxa detected in the surface sometimes accounted for more than 75% of the active microbiome of bathypelagic waters (50% on average). These active taxa were generally rare in the surface, representing a small fraction of all the surface taxa. Our findings show that the drastic vertical change in environmental conditions leads to the inactivation and disappearance of a large proportion of surface taxa, but some surface-rare taxa remain active (or with potential for protein synthesis) and dominate the bathypelagic active microbiome.