I have the same issue. Every time I start my bird this same pod does this initially but after a second it will start and fly normally. I've had this issue about 5 or 6 times where it will not fully start at all. After a couple of days I'll go out and try it again and it will start like it always does...twitch and then start. I'm worried about losing a motor in flight but it has never acted up once it starts spinning normally. I've even had it "twitch only" after flying and then switching batteries.
I'd check the bullet connectors and their accompanying solder joints. It sounds like you've got one that isn't quite making cold contact. Once it heats up something expands and makes firmer contact is me guess.