CRAFT: Critiquing and Rethinking Accountability, Fairness and Transparency

The ACM FAccT conference has predominantly focused on Fairness, Accountability and Transparency. Its success has also attracted much critique and renewed attention to the limitations of achieving these goals in systems that implement statistical, machine learning, optimization, or autonomous computing techniques. A number of prominent studies acknowledge that addressing societal problems embedded in such computing systems may require more holistic approaches. In the spirit of reflection and response, we invite academics of all disciplines and people representing different communities of practice (including journalism, advocacy, activism, organizing, education, art, spirituality, public authorities) to to contribute to a program that will be embedded in the ACM FAccT conference, subject to its own review process*. This call invites contributions in the form of workshops, panels and other formats to:

1. creatively engage with and address critiques of the field of fairness, transparency, and accountability -- its gaps, omissions, and possibilities by taking a more holistic approach


2. highlight novel modes of interaction with questions of fairness, accountability, and transparency


3. invite an interdisciplinary and cross-practice group of organizers, researchers, activists, and artists to explore and inspire conversation and open future lines of research, collaboration, and practice


4. push beyond the epistemological and methodological boundaries of their practice

In addition to contributions that explore the problem space in greater depth and from broader perspectives, we particularly encourage proposals that explore solution spaces, indicate mechanisms for positive change, or open possibilities for a greater conversation around countering automated injustices. We value proposals focused on interaction among participants, and we will prioritize formats that allow participants to share and explore starting assumptions, prior experiences, or competing values and to foster community building, collaborative knowledge production, and future engagement.

Themes

Below we offer a set of themes, some of which build on last year’s CRAFT sessions and existing critiques, and others which point to new and emergent areas for research. Each theme is accompanied by a non-exhaustive list of questions that a proposal might seek to investigate, as a way of fleshing out the theme.

We think of these themes as “dimensions” rather than “categories”. We invite you to identify the primary theme to which your proposal connects. You are welcome to mention additional themes that are relevant, but please explicitly name the most prominent theme that informs your proposal.

Theme 1: Modeling and (Non-)Deployment

• How do we ensure that designers of algorithmic systems that aspire for fairness are cognizant of the “tussles” between entities that deploy these systems (e.g. through APIs) across different contexts?


• How do we build the right (economic/social/legal/political) incentives and mechanisms to deploy fair and just systems?


• What are ways to evaluate and monitor the downstream consequences of unfair systems? Are there forms of evaluation that enable those affected in engaging in the evaluation of the damage?


• What role can computer science play in developing processes for drawing red-lines that may be applied to current, legacy or future machine learning systems?

Theme 2: Values, Assumptions and Context

• How do we surface hidden values and assumptions embedded in current perspectives on fairness, accountability and transparency, and how do we provide new and alternate perspectives across systems, populations and regions?


• How can FAccT learn from critics who point out that systems affect people in situated ways and that system design requires that designers understand power structures, exploitation, marginalization and other theories of subjugation?

Theme 3: Generating Higher Order Critiques

• How do we turn the FAccT process inwards on our own community and reflect on how, cognizant of power differentials, we might improve research processes internally as well as interactions between and engagement with different disciplines?


• What are the methods and processes from different fields that may come to help in the exploration of inequitable outcomes and mechanisms and how to mitigate these?

Theme 4: Emerging Problems

• What are (a) unresolved questions in existing fields that need to be addressed, (b) desirable operating characteristics for FAccT frameworks in particular application areas that have yet to be explored, or (c) new boundaries of FAccT approaches that require rethinking current practices?

Theme 5: New Models of Engagement

• How can we build non-exploitative, non-predatory mechanisms for engaging with people with lived experience?


• What can we learn from other disciplines and traditions (academic, activist, or otherwise) on how to engage community partners and constructively participate in coalitioning work?

Guidelines for CRAFT Contributors

Submitting Proposals

The submission link will go live on December 7, 2020

Session Formats for Contributions

In the spirit of openness, we welcome contributions in a variety of different session formats. We realize that the online format this year may limit what is possible in this space, but also know that these boundaries can also be generative. These may include, but are not limited to the following:

• Lightning Talk or Rump Session


• Panel


• Poster and/or Demo Session


• Unconference


• Interactive Workshop


• Art Exhibit or Other Artistic Intervention

We welcome novel contributions which take advantage of the affordances of the online space. If you would like to propose another format or want to mix and match formats, please get in touch with us. In general, we welcome formats that differ from the conference program and avoid dominating the time with frontal presentations. We especially encourage proposals that bring together people from different backgrounds, be it people of different disciplinary, epistemological, institutional practices, as well as contributions that bring in and bridge different communities, movements, organizations, as well as positions.

CRAFT also has a small budget which can be used for purchasing materials, subscriptions to online tools and platforms, and other workshop expenses such as honoraria for speakers, presenters, or other contributors. If you are requesting funds, we ask that you provide a short budget and description as part of your proposal.

Roles and Responsibilities for Contributors

Because we are open to as many formats as possible, we recognize that proposals may contain people who play different roles. Please name these roles in your proposal. For example, some potential roles to mention are:

• Coordinator: The person(s) who are responsible for executing the proposal and contact person(s) for the CRAFT Co-chairs. Prior to the conference, the person(s) may serve as the ringleader for the proposal. During the conference, the coordinator(s) are expected to facilitate discussion, help maintain productive interaction, and encourage participation.


• Presenters: Those who are expected to present at the proposed session.


• Documenter: Because CRAFT is a newer space within the ACM FAccT program, we strongly suggest that you designate someone to help to document your contribution for posterity’s sake.

Important: Note that you can submit no more than two contributions to CRAFT as a coordinator.

Important: If you are proposing a workshop, panel, poster session or another group format, you will need enough time to either circulate your own call or wrangle people who will participate. Your proposal should include a timeline for distributing a call, as well as for completing your final program.

Proposal Requirements

Your initial proposal should include sufficient information to evaluate it equally among other proposals. We provide a template here so as to streamline the proposal process, as well as the selection process.

• Proposed Title: Let us know what your session will be called.


• Coordinator(s): Please indicate the name or names of session coordinators, including their titles, affiliations, and contact information.


• Presenter(s): If known, indicate the name or names of session presenters, including their titles and affiliations. (Note: The timeline below may be helpful.) If you don’t know these, please say so. Where possible, briefly describe what they will present on or contribute to your session.


• Documenter: If you intend to have someone document your session, please indicate their name, title, and affiliation here.


• Format: Indicate and describe the kind of format you will use for your session.


• Length: Choose the appropriate length for your session. The proposed events can be 1.5h, 3h or 6h in duration. Events that require 6 hours will be split over two days. Please provide a schedule of how the time will be spent, including any time allocated for breaks.


• Description: Indicate the problem and/or critique you are engaging and the learning (or otherwise) goals of your session in approximately 800-1000 words (e.g., the form of an extended abstract). What are you trying to achieve with your session? Where do you want participants to be by the end of your event? Please also make sure to indicate how you will know that your participants are effectively engaged with the problems/solutions, or critiques, reflections, visions that your session seeks to generate. Please note, the title, coordinators’ names, and a shorter description (max: 500 words) will be listed in the ACM FAccT 2021 program.


• Additional artifacts (optional): If you have additional materials which would support your proposal, such as a video example of the facilitation, a website, or written workbooks, you can also include that in your proposal.


• Primary Theme Your Session Fits Into: Tell us which theme (see above) your proposal fits into.


• Platform: The conference will take place online, but CRAFT sessions are not constrained to any particular platform. If you know what online platform would best accommodate your session, let us know; if you don’t, please describe the functionality your session would require. The conference will be able to provide some support in selecting and configuring an appropriate platform.


• Target Audience Size: Let us know how many people you anticipate coming to a session or how many participants you would ideally like to have.


• Documentation and Reporting Plans (optional): We would like to give the opportunity for CRAFT sessions to be properly documented. Indicate how you plan to capture and/or catalyze ideas that arise in your session (e.g., mention format and technical infrastructure for documentation during the event). We plan to compile artifacts for sessions that so choose (max 1000 words), and publish them on a microsite, linked to the FAccT website. Deadlines for reports to be included are May 1.
• Publicity Plans: If you plan to use social media to promote or document your session, please indicate a hashtag here.
• Other Needs: Include a note if you have any special requirements for your session that are not addressed elsewhere.

Evaluation Criteria for CRAFT Session Proposals

CRAFT uses a single-blind process for review, which means the Coordinator(s) do not know who the reviewers are, but the reviewers know who the Coordinator(s) are. Each proposal will be evaluated by two people. To encourage reflexivity and support a range of perspectives and practices in the CRAFT program, our selection committee will reflect reviewers coming from different backgrounds, experiences, and expertise on automated decision systems.

The review process will focus on four basic areas:

1. Organizational details: Is the proposal clear, sensible, and thorough?


2. Rigor and quality: The potential for the topic of the proposal to generate stimulating discussions and useful results.


3. Planning: The organizers’ ability to demonstrate in the proposal a well-organized process and plan for the workshop that is welcoming and fosters interactivity. Ensure there is a clear, and workable plan, especially in the virtual setting.


4. Contribution and Relevance: The overall balance of topics in the proposed program and relevance to the main conference theme. If multiple submissions are received on the same or similar topics, the organizers may be encouraged to merge them or differentiate them. Ideally, one or more of the organizers should have experience in organizing similar events.