We demonstrate that modulation instability leading to optical pattern formation can arise by using nonconventional counterpropagating beams carrying an orbital angular momentum (optical vortices). Such a vortex beam is injected into a nonlinear single feedback system. We evidence different complex patterns with peculiar phase singularities and rotating dynamics. We prove that the dynamics is induced by the vortex angular momentum and the rotation velocity depends nonlinearly on both the vortex topological charge and the intensity of the input beam.