For now igonre the face, the problem I wanna adress is that hat. I'm having a lot of trouble trying to draw in. Any thoughts or suggestions?
Part of the problem is that the right side of the crown of the hat is blown out in your reference, making it appear narrower than it really is. Remember, there's a head under that hat, so rough in the shape of the skull, establish the ellipse where the brim meets it, and extend a cylinder upward.
I can't ignore the face because: coolest mustache evar!
Primary issue with the hat is the fact that the inside is too narrow for his head. Start with making it able to fit around his head, when fiddle with the perspective of the rim. It's going to be close to dead-on and slightly from below. You won't actually see a whole lot of the rim.
Is that Poirot?