Comments have closed. View all of the finalist entries below.

Equity & Beneficence in Sociotech System

What is the compelling question or challenge?

For technology to promote a healthy society, it must benefit people equitably. How do we design for, evaluate, and monitor the distribution of effects, not just overall utility, in technical systems?

What do we know now about this Big Idea and what are the key research questions we need to address?

The next generation of large-scale computing systems must ensure an equitable distribution of systems’ benefits and impacts. Doing so requires equity and beneficence to be considered throughout the design, development, deployment, and monitoring of systems that affect the daily lives of members of society. This calls for a wide range of disciplines, including computer science and humanities. It starts with understanding our systems: Do outcomes from different systems equally serve all individuals, regardless of background or context? Do systems provide privacy and security to all users equitably? It also requires going deep into the needs and expectations of different systems to answer questions such as: Do systems meet the needs of all their users, or are the needs of a less visible population not recognized? How do we incorporate equity and beneficence into curriculum to train current and future workers and the next generation of researchers? What methods and tools are needed to investigate these questions?

Documenting system utility and impact is only the beginning. We must systematize that knowledge to predict the likely impacts of new technologies, enabling better ethical reasoning about the technologies we build. We need (i) modeling strategies and impact models that foster development of technologies that are resistant to negative effects or disparate impact, (ii) quality assurance techniques to detect such properties, (iii) the ability to monitor deployed systems to alert on undesirable social effects or disparate utility degradation, and (iv) the ability to identify when groups that we may not yet know to identify are being mistreated or ill-served. Doing so should be guided by the following directives:

Stakeholder participation. We need methods of engaging more and more diverse stakeholders across the product lifecycle. Participatory design provides a framework for involving users in the design process, yet, there are open challenges in scaling the idea so that relevant stakeholders can be involved in all stages of the lifecycle and in the assessment of complex systems.

Privacy and security. Systematically weaker privacy or security protections for vulnerable subsets of society will only exacerbate inequality. Research on privacy management highlights access control languages, different privacy settings in applications, and formal privacy policies, whereas existing tools and algorithms focus on enterprise/organizational privacy management. We need to include the main actor (the user), and provide a privacy framework with customizable policies since the definition of privacy varies from person to person based on their personality, cultural background, and users’ privacy policies are dynamic and change based on the information sharing context and environment.

Real-time monitoring. It is not enough to design and deploy systems with beneficence and equity in mind; we must monitor those systems to ensure they continue to provide equitable benefit as they are used and adapt to their environment and user input. Aggregating heterogeneous technologies and evaluating their effects on various groups requires analysis of distributed data in near real-time settings, with long-term archives allowing time-varying trends to be discovered and evaluated.

Formal methods. To guarantee a fair access and deployment of systems, we need to employ formal methods to express fairness and information adequacy requirements in a formal language (which involves developing a domain-specific language that expressive enough to capture fairness and adequacy, yet is verifiable), model complex systems and verify that the systems satisfy those requirements.

Effective accountability. The computing systems we are now seeing deployed may require new notions of accountability: how and to whom should they be held accountable, and who should be held accountable for their operation and effects?

Show More

Contact

National Science Foundation, 2415 Eisenhower Avenue, Alexandria VA 22314, USA

Tel: (703) 292-5111, FIRS: (800) 877-8339 | TDD: (800) 281-8749