We report theoretically on a pure frequency-induced polarization bistability in a vertical-cavity surface-emitting laser (VCSEL) subject to orthogonal optical injection, i.e., the master laser light polarization is orthogonal to that of the slave VCSEL. As the frequency detuning is scanned from negative to positive values and for a fixed injected power, the VCSEL exhibits two successive and possibly bistable polarization switchings. The first switching (from the slave laser polarization to the injected light polarization) exhibits a bistable region whose width is maximum for a given value of the injected power. Such a dependency of hysteresis width on the injected power is similar to that recently found experimentally by Hong et al.[Electron. Lett. 36, 2019 (2000)]. The bistability accompanying the second switching (from the injected light polarization back to the slave laser free-running polarization) exhibits, however, significantly different features related to the occurrence of optical chaos. Interestingly, the width of the bistable region can be tuned over a large range not only by modifying the injection parameters but also by modifying the device parameters, in particular the VCSEL linewidth enhancement factor.