Laser diodes typically behave like damped oscillators: they are generally expected to only show damped relaxation oscillations toward a stable fixed point. In vertical-cavity surface-emitting lasers (VCSELs), the picture appears to be quite different as polarization dynamics can be experimentally observed including bifurcations to self- pulsation and even chaos. Physically, the circular geometry of VCSELs makes the polarization selection very weak and, thus, the additional degree of freedom can enable complex dynamical behavior in the laser diode. Here we report on a new dynamical behavior in a free-running VCSEL: we observe a bistability between two limit cycles oscillating around two distinct elliptical polarization states whose main axes are symmetrical with respect to the polarization at threshold. Although the existence of two symmetric elliptical polarizations and the associated limit cycles are predicted by the San Miguel, Feng and Moloney (SFM) model, the hysteresis cycle observed experimentally highlights the importance of asymmetry in the dynamics from the elliptically polarized states. We demonstrate that this behavior can be accurately reproduced in theory within the SFM framework when taking into account a small misalignment between the phase and amplitude anisotropies of the laser cavity. Our results bring new light into VCSEL polarization dynamics and provide a very good qualitative agreement with the bifurcation scenario predicted by the SFM model.