Presentation: A Young Profession Coping With Ethical Debt

If you look around our industry the vast majority of people that are working today in our industry, writing code, making decisions that impact users, haven't had an intense ethics course in their life, they haven't taken an ethics course in high school, they haven't taken an ethics course in college. It doesn't mean that they don't know ethics, ethics are pretty innate in human beings. They're not surprises most of the time, but having a rigorous model for understanding whether or not what you're doing is ethical and what the consequences are of those actions is something that is learned. It's not something that is innate, and I can propose a rigorous model for asking yourself questions, not getting answers. There's a playbook for discussing ethics; there's a playbook for contemplating them; there's not a playbook for answering them. Ethics are very open ended. The vast majority of hard ethical questions have two or more non-ethical answers: you don’t choose from right and wrong ethical answers. The question is what your mental model is for making yourself answer those questions instead of just avoiding them and pretend they don't exist.

There's a YouTube video called Slaughter bots. The scariest part of that video is that there is not a single piece of technology that's discussed in isolation that does not already exist today capably. So the only thing in that thing that's a little unbelievable is the battery life on the drones. It's incredibly disturbing. And all of that is a long chain of ethical lapses in software development. I don't want to paint a doom and gloom thing. What I really want to do is to get people to incorporate ethical considerations into the work that they do. I think that software developers have come a long way and they try to think about security as they develop code now. I don't think they have done a great job of it, but I will say that the security community has through various mechanisms -fear, uncertainty, doubt, liability - all of these things have caused that conversation to come up more often than not during the cycles of software development. Is this secure or should we be concerned about that? Just those two simple questions are fantastic. Sometimes we don't have the right answers but at least software developers ask themselves that. They're not asking themselves, what are the ethical consequences of this? Who could get hurt by this? Who does this enable over another person? Who does this disadvantage or advantage? They're not asking those questions. My goal is to have that part of the natural sequence of developing software. Someone should be asking those questions. Every developer coming to your other question of who this is for, every line level developer that writes a line of code should have the capacity to incorporate those questions into what they do and it should go all the way up the stack to grand vision.

Sure. There are certain soap dispensers in airports that are automatic, that detect when your hand is under it unless you're African-American because it doesn't see your hand. One needs to hold a white piece of paper under the soap dispenser to get it to fire. The reason that's there is not because a group of engineers thought that would be really funny to disenfranchise African-American people and people with dark skin. That is a layered cake of a lack of ethical questioning. It's simply because people chose not to ask the questions, not because they asked them and were like "let's do evil." A lot of ethical concerns there. That system is a whole bunch of chip integrators and product designers, and some software writers. But they adopted software from other libraries and they adopted models. There's a tree of software development. A lot of people who wrote software for that didn't know what was going into that machine.

It's highly dependent on what they do. What type of software they work on. I think that the one thing that they can absolutely take away is a simple question of how could this harm someone. Honestly, asking the question is so valuable. Just asking it, some percentage of the time will result in a corrected behavior or consideration that wasn't otherwise there. That's what they can do. They can look at the code that they're writing and ask questions. I would argue that no one ever writes software without that software eventually interfacing with a human being. We are the ultimate consumers of software. We build software to make human lives better. It's not like we're going to make robot lives better. If we do it's because robots make human lives better. There's a chain there that always ends up with the human as the as the benefactor of technology. Just asking the question of how is my code going to be used by humans?