Agency and influence: moral communication for interactive robots

Abstract

As robots with social behaviors proliferate into a widening variety of contexts and roles, it is clear that we have a lot to learn about how humans expect (and prefer) these robots to act, how humans perceive different robot behaviors and judge or sanction robot misbehaviors, and how robots should fit into, shape, and be shaped by social structures and norms. This thesis presents several studies on human-robot interaction that focus on enabling robots to communicate effectively and appropriately through natural language in morally sensitive contexts.

We begin by examining the concept of social agency, and constructing a new theoretical understanding of social agency for robots. We discuss the implications of robots' potential ontological status as social agents, including the capacity for significant normative influence. We then examine this moral influence in the context of clarification dialogues, and show how a failure to perform moral reasoning when generating clarification requests can cause robots to generate utterances with unintended implied meanings that can weaken human application of moral norms. We then present and evaluate an algorithm that fixes this problem.

Next, we examine robot command rejections under the premise that robots should not follow immoral human commands. We present evidence that robot command rejections should be phrased with a degree of politeness proportional to the severity of the norm violation motivating the command rejection. Given the importance of gender in performing and perceiving politeness, we reexamine these results with specific attention to human gender and robot gender presentation.

We then present part of a cross-cultural study on how female presenting social robots might respond to gendered verbal abuse from humans without propagating harmful sexist stereotypes or damaging robot credibility. Our results highlight a couple of promising response styles.

Finally, we present the integration of a norm-aware task planner and a context recognition module into a robot cognitive architecture. This integration establishes the capacity for multi-step task planning under context-sensitive norms and lays the groundwork for generating more informative command rejections.