CRAFT Call (Closed)
The ACM FAT* conference has predominantly focused on fairness, accountability, and transparency. The success of the field of fairness, accountability, and transparency has also attracted much critique and renewed attention to the limitations of achieving these goals in systems that implement statistical, machine learning, optimization, or autonomous computing techniques. A number of prominent studies acknowledge that addressing societal problems embedded in such computing systems may require more holistic approaches.
In the spirit of reflection and response, we invite academics of all disciplines and people representing different communities of practice (including journalism, advocacy, organizing, education, art, public authorities) to contribute to a program that will be subject to its own review process and that will be embedded in the ACM FAT* conference (the CRAFT call is very much inspired by the NeurIPS 2018 CRACT workshop.) This call invites contributions to that program in the form of workshops, panels, and other formats to:
- address critiques of the field of fairness, accountability, and transparency, such as its blind spots,omissions, or alternative possibilities that take a more holistic approach,
- open future lines of research, collaboration, and practice.
In addition to contributions that explore the problem space in greater depth and from broader perspectives, we particularly encourage proposals that explore solution spaces, indicate mechanisms for positive change, or open possibilities for a greater conversation around countering automated injustices. We value proposals focused on interaction among participants, and we look forward to formats that allow participants to explore starting assumptions, prior experiences, or competing values and to foster community building, shared knowledge production, and future engagement.
Note 1 (Financial support): a limited number of travel grants, about 10, will be made available for organizers of CRAFT sessions requiring financial support to attend the conference.
Note 2 (Tutorials track): in addition to the CRAFT program, ACM FAT* also solicits proposals on the Tutorials track. Please note that CRAFT emphasizes reflection and critique and differs in important and subtle ways from Tutorials. By offering contributors a space to address blind spots and omissions, offer alternative approaches, and open new possibilities for the field, CRAFT seeks contributions that challenge. By contrast, Tutorials primarily educate and inform. We expect that CRAFT will feature unconventional ideas in engaging and diverse formats. Tutorials, by contrast, might include explainers regarding existing tools, overviews of bodies of literature, or best practices.
Some proposals we receive under the CRAFT program may be more appropriate as proposals for Tutorials track, and vice-versa. In such cases, the Tutorial and CRAFT Co-chairs may transfer such proposals to the other program, in consultation with Coordinator(s). In the event that a CRAFT decision suggests your proposal for the Tutorials section of the ACM FAT* 2020 program, you will have one week to confirm whether you accept this choice.
Themes
Below we offer a set of themes, some of which are based on existing critiques, and others which point to new and emergent areas for research. Each theme is accompanied by a non-exhaustive list of questions that a proposal might seek to investigate, as a way of fleshing out the theme.
We think of these themes as “dimensions” rather than “categories”. We invite you to identify the primary theme to which your proposal connects. You are welcome to mention additional themes that are relevant, but please explicitly name the most prominent theme that informs your proposal.
Theme 1: Modeling and (Non-)Deployment
- How do we build the right (economic/social/legal/political) incentives and mechanisms to deploy and evaluate systems with fairness aspirations?
- What are ways to evaluate and monitor the downstream consequences of unfair systems? Are there forms of evaluation that enable those affected in engaging in the evaluation of the damage?
- Are there differences in how we should design algorithms addressing injustices and fairness when these will be used across different contexts (e.g. through APIs) rather than in a single application? How will fairness models and aspirations fare as such systems become ‘legacy’?
- What role can computer science play in developing processes for drawing red-lines that may be applied to current, legacy or future machine learning systems?
Theme 2: Values, Assumptions and Context
- How do we surface hidden values and assumptions embedded in current perspectives on fairness, accountability and transparency, and how do we provide new and alternate perspectives across systems, populations and regions?
- How can ACM FAT* learn from critics who point out that systems affect people in situated ways and that system design requires that designers understand power structures, exploitation, marginalization and other theories of subjugation?
Theme 3: Generating Higher Order Critiques
- How do we turn the FAT process inwards on our own community and reflect on how, cognizant of power differentials, we might improve research processes internally as well as interactions between and engagement with different disciplines?
- What are the methods and processes from different fields that may come to help in the exploration of inequitable outcomes and mechanisms and how to mitigate these?
Theme 4: Emerging Problems
- What are (a) unresolved questions in existing fields that need to be addressed, (b) desirable operating characteristics for ACM FAT* frameworks in particular application areas that have yet to be explored, or (c) new frontiers of FAT approaches that require rethinking current practices?
Guidelines for CRAFT Contributors
Session Formats for Contributions
In the spirit of openness, we welcome contributions in a variety of different session formats. These may include, but are not limited to the following:
- Interactive Workshop
- Panel
- Debate
- Unconference
- Art Exhibit or Other Artistic Intervention
- Site Visit
- Poster and/or Demo Session
- Lightning Talk or Rump Session
If you would like to propose another format or want to mix and match formats, please get in touch with us. In general, we welcome formats that differ from the conference program and avoid dominating the time with frontal presentations. We especially encourage proposals that bring together people from different backgrounds, be it people of different disciplinary, epistemological, institutional practices, as well as contributions that bring in and bridge different communities, movements, organizations, as well as positions.
Roles and Responsibilities for Contributors
Because we are open to as many formats as possible, we recognize that proposals may contain people who play different roles. Please name these roles in your proposal. For example, some potential roles to mention are:
- Coordinator: The person(s) who are responsible for executing the proposal and contact person(s) for the CRAFT Co-chairs. Prior to the conference, the person(s) may serve as the ringleader for the proposal. During the conference, the coordinator(s) are expected to facilitate discussion, help maintain productive interaction, and encourage participation. We especially encourage coordinators to ensure they have people on their team that can bridge across different communities of practice.
- Presenters: Those who are expected to present at the proposed session.
- Documenter: Because CRAFT is a new space within the ACM FAT* 2020 program, we strongly suggest that you mention someone who will help to document your contribution for posterity’s sake.
Important: Note that you can submit no more than two contributions to CRAFT as a coordinator.
Important: If you are proposing a workshop, panel, poster session or another group format, you will need enough time to either circulate your own call or wrangle people who will participate. Your proposal should include a timeline for distributing a call, as well as for completing your final program.
Proposal Requirements
Your initial proposal should include sufficient information to evaluate it equally among other proposals. We provide a template so as to streamline the proposal process, as well as the selection process.
- Proposed Title: Let us know what your session will be called. (Max. 25 words)
- Coordinator(s): Please indicate the name or names of session coordinators, including their titles, affiliations, and contact information.
- Presenter(s): If known, indicate the name or names of session presenters, including their titles and affiliations. (Note: The timeline below may be helpful.) If you don’t know these, please say so. Where possible, briefly describe what they will present on or contribute to your session.
- Documenter: If you intend to have someone document your session, please indicate their name, title, and affiliation here.
- Format: Indicate and describe the kind of format you will use for your session.
- Length: Choose the appropriate length for your session. The proposed events can be 1.5h, 3h or 6h in duration. Events that require 6 hours will be split over two days.
- Description: Indicate the problem and/or critique you are engaging and the learning (or otherwise) goals of your session in approximately 800-1000 words (e.g., the form of an extended abstract). What are you trying to achieve with your session? Where do you want participants to be by the end of your event? Please also make sure to indicate how you will know that your participants effectively engaged with the problems/solutions, or critiques, reflections, visions that your session seeks to generate. Please note, the title, coordinators’ names, and a shorter description (max: 500 words) will be listed in the ACM FAT* 2020 program.
- Primary Theme Your Session Fits Into: Tell us which theme (see above) your proposal fits into.
- On-site/Offsite Location: Let us know whether your session will take place within the ACM FAT* 2020 conference space, or whether you intend to run your session off-site. If you intend to do something offsite, please explain whether you have already confirmed another space and by when you expect to have a confirmation, in addition to a description of how the participants will reach the off-site location.
- Target Audience Size: Let us know how many people you anticipate coming to a session or how many participants you would ideally like to have. At the moment for on-site workshops we have two types of meeting rooms: MEDIUM meeting rooms for 50-60 people in a standard setting (seated, looking forward) or 20-25 people in a round-table setting, and a LARGE meeting room that fits 200-300 people
- Documentation and Reporting Plans: We would like to ensure that CRAFT sessions are properly documented. Indicate how you plan to capture and/or catalyze ideas that arise in your session (e.g., mention format and technical infrastructure for documentation during event). We plan to compile reports of each session (max 1000 words), with the potential of producing a special issue. Please indicate your interest in a special issue. Deadline for reports are March 2020.
- Publicity Plans: If you plan to use social media to promote or document your session, please indicate a hashtag here.
- Other Needs: Include a note if you have any special requirements for your session that are not addressed elsewhere. By default for on-site sessions, audio/video support including projector, a laptop, speakers, podium, podium mic, lapel mic, and handheld mic will be provided. CRAFT sessions will not be recorded by the ACM FAT* 2020 conference organization.
To submit a proposal, use the "New submission" option in fat2020-craft.hotcrp.com and use this template.
Pre-registration only requires that you "save a draft" of your proposal with an informative title and a brief abstract/summary.
Timeline
- CRAFT call for contributions announced: July 15th
- CRAFT submission link goes live: July 30th
- July-August -- Coordinators solicit participants for their sessions
- CRAFT proposal pre-registration: September 9th
- CRAFT proposal submission: September 12th
- CRAFT Co-chairs assign proposal titles to selection committee: September 9th-12th
- Selection committee submits reviews: September 24th
- Tutorial and CRAFT Co-chairs deliberate: September 25th-30th
- CRAFT notification date: October 3rd
- CRAFT session proposal description (500 words) due: December 5th
- CRAFT program published: December 12th
- ACM FAT* 2020 Conference registration opens: mid-December 2019
- CRAFT sessions: January 29th and/or January 30th, 2020
- CRAFT Coordinators submit final reports: March 2020
Reviewing Proposals and Making Recommendations
Keep in mind that we are looking for proposals that:
- Are accessible to people coming from multiple communities of practice (academic, industry, journalistic, artistic, activist, organizing, etc.)
- Go beyond a conventional academic approach of critiquing ACM FAT*
- Offer a vision that helps fulfill the aims of the CRAFT call
In general, reviewers will look favorably upon proposals that demonstrate that you have thought through the steps needed to achieve the goals of your session. So please be as detailed as possible. A list of reviewers will be made available shortly.
Evaluation Criteria for CRAFT Session Proposals
CRAFT uses a single-blind process for review, which means the Coordinator(s) do not know who the reviewers are, but the reviewers know who the Coordinator(s) are. Each proposal will be evaluated by two people: one who is assigned to a particular thematic area, and one person who is outside the thematic area. To encourage reflexivity and support a range of perspectives and practices in the CRAFT program, our selection committee will reflect reviewers coming from different backgrounds, experiences, and expertise on automated decision systems.
The review process will focus on five basic areas:
- Organizational details: Is the proposal clear, sensible, and thorough?
- Rigor and quality: The potential for the topic of the proposal to generate stimulating discussions and useful results.
- Planning: The coordinators’ ability to demonstrate a well-organized process and plan for a contribution that is welcoming and fosters interactivity.
- Contribution and relevance: Is the proposal about critique and reflection? Does it address blind spots, omissions, or dominant approaches of ACM FAT* work? Does it consider or offer alternative, holistic approaches? Does the proposal open new lines of inquiry, collaboration, or practice?
- Catering to the audience: Is there a clear and workable plan for facilitating a lively environment for discussion for all participants?
If multiple submissions are received on the same or similar topics, the organizers may be encouraged to merge them or differentiate them. Ideally, one or more of the organizers should have experience in organizing similar events.
Please note the CRAFT program is not a separate event from the ACM FAT* conference. All participants—whether CRAFT contributors, Tutorial presenters, Paper presenters, or other attendees—will benefit from participating in the entire conference program.
Roles and Responsibilities of CRAFT Co-chairs
As Co-Chairs of CRAFT, we will work with a selection committee (i.e., a set of reviewers) to guarantee a fair decision making process as well as a diligent process for participating in CRAFT. This includes due process in selection, timeliness, openness to critique and respect for others. Finally, we are responsible for making a final program in which all CRAFT activities are listed and made available to conference participants.
As Co-Chairs, we are committed to upholding the following throughout CRAFT:
- We respect the dignity, experiences, and perspectives of the communities impacted by the systems that are under discussion at CRAFT.
- We recognize the problematic effects of systemic optimization but also that not all populations and communities are impacted equally. In fact, some are likely to benefit economically, socially, or otherwise, while others are suffering from these systems.
- We respect the dignity, experiences, and perspectives of each CRAFT participant.
- We recognize multiple forms of evidence for the way socio-technical systems operate and for their effects.
- We are cognizant of the role power plays in shaping how we inhabit rooms, and speak up and shape urgencies or commit to challenge unjust existing power structures such as (but not limited to): sexism, racism, and transphobia.
Selection committee
- Nasma Ahmed, Digital Justice Lab
- J. Khadijah Abdurahman, WordToRI
- Ruha Benjamin, Princeton
- Bettina Berendt, KU Leuven
- Crofton Black, Bureau of Investigative Journalism
- Manuela Bojadzijev, Leuphana University
- Rosamunde Elise van Brakel, Vrije Universiteit, Brussels
- Ingrid Burrington, artist
- Rumman Choudhury, Accenture
- Wendy Chun, Simon Fraser
- Roderic Crooks, Univ. California, Irvine
- Francien Dechesne, University of Leiden
- Rebekah Delsol, Open Society Justice Initiative
- Lina Dencik, Cardiff University
- Desiree Fields, Berkeley
- Sorelle Friedler, Haverford College
- Ryan Gerety, Independent
- Brent Hecht, Northwestern/Microsoft
- Joris Van Hoboken, Univ. Amsterdam/Free University Brussels
- Anna Lauren Hoffman, U. Washington
- Lily Hu, Harvard
- Ben Hutchinson, Google
- Lilly Irani, Univ. California, San Diego
- Malavika Jayaram, Singapore Management University
- Seny Kamara, Brown
- Anne Kaun, Södertön University
- Marina Kogan, University of Utah
- Joshua Kroll, Berkeley
- Alana Lentin, Western Sydney University
- Manu Luksch, Artist and Filmmaker
- Smarika Lulz, Smar KY Inc/Humboldt University
- Jedrek Niklas, LSE
- Safiya Noble, Univ. California, Los Angeles
- Zara Rahman, The Engine Room
- Jara Rocha, independent
- Hannah Sassaman, Media Mobilizing Project
- Andrew Selbst, Data And Society/Univ. California, Los Angeles
- Matthias Spielkamp, Algorithm Watch
- Carmela Troncoso, EPFL
- Berk Ustun, Harvard
- Nisheeth Vishnoi, Yale
- Zeerak Wassem, Sheffield
- Christo Wilson, Northeastern University
CRAFT Co-Chairs
Please contact craft@fatconference.org for any questions.