Nice work Rob!

As far as acting, which is the heart of character animation, I'd suggest think of specific  moments rather than simply moving him whenever he talks. For example, when he says "that's embarassing" how would you react when embarassed? How would a person's posture and movement change when feeling that emotion? Focus on making a really strong pose for each beat and each emotion, and then move him around to make him talk within that pose or emotion. Be careful not to move him too much when the off camera person is speaking, so it's clear he isn't the one talking, and if you have him start going into another motion and then freeze after the offscreen person starts talking again, it will feel more like he's being interrupted.

Having a different voice actor for the offscreen person probably isn't too important in a simple test like this, but when you put the time in for a serious production, having voice actors will boost believability and production value immensely. Many brickfilmers are generous in lending their voices to each other's productions, so scour the forums for the appropriate place to post your casting calls.