Problem solved!
Turns out it was the speaker. Really didn't think it would be, but I was wrong. Removed the door panel in 10 minutes, replaced speaker, tested it, put the door panel back on, done. Fixed. Never sounded better.
Two things, first, nothing should be taken away from Focal! Problems happen. Still, hands down, the best speakers I have heard in any car. Second, Crutchfield is awesome! Called them on Saturday evening and explained the situation, they quickly sent out another set without hesitation. Got the shipment on Tuesday (today), replaced today, and they e-mailed me a return label so I don't get charged anything. Had a 1 year warranty on these, luckily the warranty was to end this November. Highly recommend Crutchfield!!