Talc is an insulating layered material that is stable at ambient conditions and has high-quality basal cleavage, which is a major advantage for its use in van der Waals heterostructures. Here, we use near-field synchrotron infrared nanospectroscopy, Raman spectroscopy, and first-principles calculations to investigate the structural and vibrational properties of talc crystals, ranging from monolayer to bulk, in the 300-750 and <60 cm-1 spectral windows. We observe a symmetry crossover from mono to bilayer talc samples, attributed to the stacking of adjacent layers. The in-plane lattice parameters and frequencies of intralayer modes of talc display weak dependence on the number of layers, consistent with a weak interlayer interaction. On the other hand, the low-frequency (<60 cm-1) rigid-layer (interlayer) modes of talc are suitable to identify the number of layers in ultrathin talc samples, besides revealing strong in-plane and out-of-plane anisotropy in the interlayer force constants and related elastic stiffnesses of single crystals. The shear and breathing force constants of talc are found to be 66 and 28%, respectively, lower than those of graphite, making talc an excellent lubricant that can be easily exfoliated. Our results broaden the understanding of the structural and vibrational properties of talc at the nanoscale regime and serve as a guide for future ultrathin heterostructures applications.