We demonstrate theoretically and experimentally that modulation instability leading to optical pattern formation can arise by using non conventional counterpropagating beams carrying an orbital angular momentum (optical vortices). Such a vortex beam is injected into a nonlinear single feedback system. We evidence different complex patterns with peculiar phase singularities and rotating dynamics. We prove that the dynamics is induced by the vortex angular momentum and the rotation velocity depends non linearly on both the vortex topological charge and the intensity of the input beam.