Last Updated: October 2024

Reviewers are the key drivers of the conference peer review process. Reviewing is a major commitment from members of the communities represented by ACM FAccT, and the conference makes a commitment to ensure that Reviewers are provided with all the information they need clearly and in advance, are supported by an effective decision-making structure, and are recognized for their efforts. In turn, Reviewers are expected to be on time, be active throughout the reviewing process, use their best professional judgment, and support and mentor the work under review. This guidance provides information Reviewers need to do their work.

General Guidance

Papers will be evaluated based on scholarly criteria in their designated area, including

• Relevance to the conference topic and chosen area: These areas are broad and described in the CFP.
• Quality and clarity of the submission: correctness, depth of exposition (including contextualizing the approach, methodology, perspective, domain considered, etc.), evaluation of both strengths and limitations.
• Originality: new analyses, results, tasks, theories, methods, data, and/or perspectives.
• Potential for impact: potential to influence academic disciplines, public discourse, policy, or real-world systems.

More information on these criteria can be found in the section on the Review Form below.

Your review is important for two reasons:

1. It will help the Area Chairs (ACs) and Program Chairs (PCs) decide whether to accept a submission.
2. It will help authors by providing both clarity about the reasons for the decision and useful feedback about the strengths and weaknesses of their scholarship, which they can use later to strengthen their work.

Please aim to make your review as detailed and informative as possible—thorough, thoughtful feedback is essential to ensuring high-quality submissions are fairly assessed. As you prepare your reviews and meta-reviews for papers submitted to FAccT 2024, please make sure to carefully review the guidance below. Remember that you are not only assessing whether a paper is worth publishing, but that you are also providing feedback to the authors. Try to be helpful, constructive, and kind in your reviews.

Reviewing Process

The reviewing process proceeds linearly from submission to final decision over several months, as follows:

1. Abstract Submission. Authors register their submissions by the abstract deadline, typically a week before the full paper submission.
2. Final Submission. Full papers are submitted. The Program Chairs perform an initial assessment for any major issues and may desk reject papers that are out of scope for the conference or in clear violation of the applicable policies.
3. Bidding. Reviewers bid on submissions to review. You will receive an email communicating the bidding window. You should bid on as many papers as possible in your area of expertise to enable the best flexibility in the matching. Avoid bidding on papers that are too far outside of your area of expertise for you to assess.
4. Reviewer Assignment. Based on Reviewer bidding and the submission’s area designations, ACs and PCs will assign submissions to Reviewers. Each paper will be assigned to several Reviewers (typically three).
5. Initial Reviewing Period. Reviewers are given several weeks to provide detailed assessments and feedback to the authors. The review form, available below, provides guidance on how to assess each paper.
6. Initial Discussion Period. After submitting their reviews, Reviewers engage in an initial discussion of the paper, facilitated by the AC. This discussion period allows Reviewers to integrate their diverse expertise and address potential disagreements or inconsistencies between their reviews. This collaborative discussion helps ACs identify key issues and write a preliminary AC summary to guide authors during the rebuttal stage. You can see more about this in the AC Guide.
7. Rebuttal. Reviews and the preliminary AC summary are shared with the authors who are given a short period to write a rebuttal in response. The rebuttal is meant to clarify misunderstandings or errors in the review, not to debate substantive disagreements or introduce new data.
8. Post-rebuttal Discussion and Review Updating. The AC and Reviewers assigned to the paper discuss the response provided by the authors. Reviewers should update their reviews to comment on the extent to which their concerns were addressed. They may or may not reach consensus on a recommendation for the paper. In most circumstances, new concerns should not be raised at this stage.
9. Meta-reviews and Final Recommendations. A final recommendation is made by the AC that incorporates the Reviewers’ assessments, the rebuttal, and the discussion. The reasoning behind the recommendation is described in a meta-review.
10. Final Calibration. Program Chairs review all the meta-reviews and recommendations to ensure a consistent standard in decision-making has been made, discuss selected papers with ACs, and then make final decisions.
11. Notification. Authors are notified. Accepted papers move to finalizing camera-ready papers.

FAccT does not allow Reviewers to delegate their reviews to external reviewers. If you feel you are not personally equipped to review a paper, please contact the Program Chairs as soon as possible so that the paper can be assigned to a different Reviewer if necessary. For concerns and questions, please contact the PCs at program-chairs@facctconference.org

Writing your review

The overall scholarly criteria listed above are intended to guide your assessment of the contributions, potential impact, and how appropriately the work was conducted. See the Review Form for more details on assessment criteria.

Good research does not require complicated methods so that comments about the methods being too simple should be avoided. A paper should also not be penalized for not using the latest ML/AI models, especially when the research contributions or the problem the authors try to illustrate do not depend on models being used for experimental purposes.

You can view the full Review Form with detailed guidance for completing each field.

Raising Ethical Concerns

Reviewers are expected to hold papers to a high standard of ethical practice. The ACM Code of Ethics and Professional Conduct provides the basis upon which ethical considerations are made. The code of ethics comprises seven principles: Contribute to society and to human well-being, acknowledging that all people are stakeholders in computing; Avoid harm; Be honest and trustworthy; Be fair and take action not to discriminate; Respect the work required to produce new ideas, inventions, creative works, and computing artifacts; Respect privacy; Honor confidentiality. This general set of principles allows reviewers to make them specific for the papers they review.

For FAccT, we ask that Reviewers use their best judgment to assess whether a paper might need further consideration in areas listed above, and specifically: improper treatment of human subjects or data annotators, including issues to do with fair pay; use of unethically collected datasets, datasets that have been retracted, or datasets not allowed to be used in the research context; potential negative downstream harms that require additional scrutiny or mitigation; environmental concerns; rights concerns; privacy concerns; safety or security concerns; legal concerns, including around copyright; dangerous materials and practices; warfare; discrimination, bias, and fairness issues; deception or harassment; and other topics of concern.

Authors may have addressed concerns directly in their paper (for instance, in an ethical impacts statement at the end of the paper), so when there are concerns, reviewers should check these statements and see if they are satisfied. Reviewers are not expected to assess and provide evidence on these concerns, but to assess whether there are ethical concerns that warrant further expert review and outside decision-making on the suitability of the paper for inclusion in the proceedings. Reviewers are given a check box to flag any concerning submissions to the ACs and PCs, and the ability to leave comments with further details on ethical issues in a dedicated field in the Reviewer Form.

Any flagged papers are triaged as follows: flagged papers are studied by the Program Chairs using all available information to confirm the concern. If the concern is confirmed, then additional Reviewers in the areas of concern are assigned to provide further information. Program Chairs will use this to guide the final decision making, and provide these additional reviews to authors to support their work. Discussion with authors in such cases are handled sensitively in line with the best practices recommended by the Committee on Publication Ethics (COPE) which FAccT uses. Also see the guide on the use of the code of ethics.

If you have evidence of plagiarism or other severe research integrity issues, please email the Program Chairs directly as soon as possible.

Frequently Asked Questions

Can I use LLMs to write reviews?

Reviewers should refer to the April 2023 ACM Policy on Authorship and use of large language models (LLMs), and the associated SIGCHI blog post, which also apply to reviews and meta-reviews.

When using third-party or software (local or cloud) to check fluency, formatting, grammar, spelling, punctuation, and language translation, Reviewers should maintain the confidentiality of the paper submissions they review at all times including by not uploading the submissions or text fragments to any third-party website, in accordance with the ACM Peer Review Policy.

What if a submission does not cite recent work?

Is not uncommon for different groups to be working on the same idea (referred to as contemporaneous work), and for these to appear on preprint repositories. For FAccT, contemporaneous work are papers released 4 months prior to the conference deadline. Hence, lack of awareness, citations, or comparisons to such works is not a basis for rejection of a paper. Similarly, it is not reasonable to expect papers to engage with work published close to or after the FAccT submission deadline.

Should I comment on an author’s command of English?

You should avoid commenting on the author's command of English, and instead focus on whether the paper is clear and readable. If a review contains problematic statements that the Reviewer is unable or unwilling to address, the Program Chairs (PC) and Area Chairs (ACs) reserve the right to make necessary edits to ensure the review meets the conference's standards.

What should I do if I have a concern about potential plagiarism or other severe research integrity issues?

In such cases, please contact the Program Chairs immediately at program-chairs@facctconference.org.