You were definitely justified in doing that. I'm actually thinking of quitting my band right now partially because of that. This type of thing has happened to me in about 3 bands so far up until now and I quit 2 of those bands specifically because of it. It's something that really pisses me off that I can't take.
Looks like time to start looking for a new drummer.