In this research, the nonlinear evolution of jet-like spikes in the single-mode ablative Rayleigh-Taylor instability (ARTI) in the presence of preheating, is studied numerically. It is demonstrated that the preheating plays an essential role in the formation of jet-like spikes in the nonlinear ARTI. The evolution of jet-like spikes in the ARTI with preheating consists of three stages with distinctly different distinguishing features. In the early stage, the preheating contributes to significantly increase the density-gradient scale length and broaden the velocity profile of the ablation surface, where the former can reduce the linear growth of the ARTI and mitigate the growth of its harmonics. In the middle stage, the ablative Kelvin-Helmholtz instability is dramatically suppressed due to the ablation effects. In the late stage, the jet's length (i.e. bubble-spike amplitude) is further increased by the bubble acceleration in the highly nonlinear ARTI, resulting eventually in the formation of jet-like spikes.