I can't believe this skit still makes me laugh. I think it's funniest at the beginning. By the end, it gets more complicated, harder to follow, and starts to just get annoying, like an SNL skit that kept going a little too long.
As for the animation, I find that animating dialog in an interesting way is extremely challenging. I'm curious: Did you try to animate it to match the original skit?
A couple suggestions:
1) Try to ease in an out of movements. To me, it looked like the movements were constant speeds rather than starting out slow, speeding up, and then slowing down as the motion ends, like real bodies move in real life.
2) With only the 1 over head light, their faces are always under the shadows of their hats. If you don't want to add another light, you can use white index cards angled at 45 degrees and set out of frame to reflect some of the over head light back up and at their faces.
Otherwise, nice job.