I hate driverless cars. That is the fact that needs to be explained. Not justified, but explained.
Driverless cars pose no threat to my job, my income, or my wealth. That's not it.
The insurance companies, or safety-nazis, might force us to use driverless cars. That would be a threat to my enjoyment of driving. But even if that threat were eliminated, so each of us always had the option to drive ourselves, I would still hate driverless cars. That's not it either.
What I hate is the very option to use a driverless car. Because being able to exercise that option, even if I did not have to exercise that option, would make "driving" something very different from what it is without that option. In much the same way that "hunting" means something very different today in an agricultural economy with supermarkets than it did for our ancestors. "Wouldn't it be cheaper and easier and safer to just buy some meat at the supermarket, or do you think it's fun to kill animals for sport?" "Why not just press the auto button, so you can sit back and relax and enjoy the scenery or read a book?"