Drums are the key to the entire band, really. If they can't keep tempo or time, then you are going to suck. It doesn't matter how good the other members are.
I feel for you man. This is why I got out of that game, I can't deal with trying to figure out who's going to show up on acid, who's not going to show up at all, and if anyone is going to show up to hear us play!
Now, I only gig with my brothers, and you can't fire them!