mmWave communication requires accurate and continuous beam steering to overcome the severe propagation loss and user mobility. In this paper, we leverage a self-supervised deep learning approach to exploit sub-6 GHz channels and propose a novel method to predict beamforming vectors in the mmWave band for a single access point – user link. This complex channel-beam mapping is learned via data issued from the DeepMIMO dataset. We then compare our proposed method with existing supervised deep learning and classic reinforcement learning methods. Our simulations show that choosing an appropriate beam steering method depends on the target application and is a tradeoff between data rate and computational complexity. We also investigate tuning the size of our neural network depending on the number of transmit and receive antennas at the access point. Finally, we extend our method to the case of multiple links and introduce a federated learning (FL) approach to efficiently predict their mmWave beams by sharing only the weights of the locally trained neural networks (and not the local data). We investigate both synchronous and asynchronous FL methods. Our numerical simulations show the high potential of our approach, especially when the local available data is scarce or imperfect.