Artificial Philosophy: ethics in computer intelligence

Such a scenario is not so far fetched.

Except that it is. This is not a discussion about self-driving cars because they don't work that way and are a long way off potentially working this way.

This is the age-old trolley problem (http://en.wikipedia.org/wiki/Trolley_problem), which was never really satisfying as a philosophical problem. You create contrived examples and try to transport that situation to a wider world.

The trolley problem seems to be popular for self-driving cars because it's easy to create horrible situations without engaging with the way the technology actually works. Moreover, you can make the decisions for the computer to fit the situation without having to think about how that'd apply generally.

Or at least, if you're going to do it, go the whole hog: http://www.mindspring.com/~mfpatton/Tissues.htm