I had to look up ableist. Apparently it means “discrimination in favor of able-bodied people.” So I’m discriminating if I prefer my roofer to be able-bodied. Damn straight.
The apology is silly. But so is your comparison.
What's silly about it? We discriminate in favor of able-bodied people all the time. We just don't regard it as discrimination in the bad sense. Come to think of it, we discriminate all the time, period. "Ableist" is a ridiculous word.