Timeline of the alleged ISCA 2019 peer-review incident

TL;DR

See these two links on the tragic case of Huixiang Chen:

• story of his death due to alleged peer-review irregularities at ISCA'19
• evidence found on his hard-drive that showed there was possibly some sort of breach in the double-blind peer-review at ISCA'19

A student committed suicide due to alleged peer-review fraud at ISCA 2019. ACM SIGARCH/ IEEE TCCA ran an "investigation" run by a team with ISCA conflicts-of-interest and found that no fraud occurred. Further evidence was found that showed the student had access to the full un-redacted comments for all papers from ISCA2019 and from HPCA. This is a clear violation of double-blind review. After much feet dragging ACM SIGARCH re-opened the investigation, but it took them over a year and a half to find any wrongdoing. In the meantime ISCA 2020 and ISCA 2021 were unwisely allowed to continue. After two years the ACM finally released the name of the primary perpretrator in ISCA'19 and ASPLOS'17 fraud, but the various collaborators have seemingly escaped justice. Minimal (if any) changes have been made to the peer review process that would prevent this from happening again.

---

Full Timeline

The lead up to ISCA'19 and the horrific aftermath

• 7 December 2018
  • ISCA 2019 paper deadline
• 22 February 2019
  • ISCA 2019 rebuttals due (responses to comments raised by anonymous reviewers. ISCA is double blind so authors are not supposed to be able to see identities of reviewers, or the internal discussion of the papers).
• 15 March 2019
  • ISCA 2019 Author Notification (told if paper got in or not)
• 22 May 2019
  • ISCA 2019 camera ready deadline (when the final version of the paper to appear in proceedings is due)
• 13 June 2019
  • University of Florida student Huixiang Chen committed suicide.
  • For more details see this article
  • Allegedly there were irregularities with the peer review and problems with the paper methodology. The student wanted to withdraw the paper but allegedly his advisor would not allow it.
• 15 June 2019
  • The advisor allegedly threatened legal action against those reporting on the incident.
  • For more details see: this article
• 26 June 2019
  • Despite the suicide, Chen's paper "3D-based Video Understanding Acceleration by Leveraging Temporal Locality and Activation Sparsity" was still presented at ISCA.
  • It still appears in the proceedings, it can be viewed at the ACM here: 3D-based video recognition acceleration by leveraging temporal locality
  • The paper claims support from millions of dollars of US government NSF grants, including grants 1900713, 1822989, 1822459, 1527535, 1423090, and 1320100. Taxpayer money supported those behind these alleged events.
• 3 July 2019
  • The University of Florida starts an investigation after requests from the Graduate Assistants United group.
  • Article
• 10 July 2019
  • A memorial service for Huixiang Chen is held at the University of Florida.
  • Article 1
  • Article 2

The Failed First Investigation

• ?? June 2019
  • The ACM SIGARCH and IEEE TCCA chairs post an initial statement saying they will let the University of Florida investigate and that they will follow their respective processes.
• 5 August 2019
  • ACM SIGARCH chair makes a statement that they are continuing to investigate.
  • Link: ACM SIGARCH follow-up statement about violations of code of ethics
• 26 September 2019
  • A bug report is filed by SIGARCH against the HotCRP conference submission system to add extra logging. Before this change it was possible for downloads of paper data without being logged (this data could be used to break the peer-review process).
  • Apparently someone at SIGARCH was aware of this issue, but officially SIGARCH claimed to be unaware of any sort of ISCA'19 data breach until the end of January 2020.
  • twitter discussion and the relevant git commit
• 26 November 2019
  • Paper deadline for ISCA 2020
• 1 December 2019
  • Josep Torrellas of UIUC announces both on twitter and on the SIGARCH website the outcome of the "investigation".
  • Link: Outcome of investigation on ISCA 2019 paper
  • Twitter Link: Beginning of Torrellas Statement (the threading is broken)
  • This announcement conveniently happened just after the ISCA 2020 deadline so it was too late for submitters to know the outcome of the investigation before submitting.
  • The final conclusion in the announcement: "The committee has determined that there was no evidence of misconduct as part of the paper review process."
  • To quote him: "The committee which conducted the investigation included members from the IEEE Computer Society TCCA and ACM SIGARCH, as well as members of the ISCA Program Committee."
  • This committee seemingly had a large number of conflicts of interest. Many of the people involved (ACM policy is the members of this committee are kept secret) apparently have deep ties to ISCA and it would reflect poorly on their careers if any ill news came up about the conference.
  • 66% of ISCA'19 papers had PC members as authors, which might have affected their objectivity as they reviewed the case.
  • Torrellas himself was a member of the ISCA'19 PC as well as had three papers accepted there, so he had obvious conflicts of interest in investigating whether there were irregularities with the peer review process.
• 4 December 2019
  • The various ACM SIGARCH and IEEE TCCA office-holders are contacted about the issue via e-mail. No response is received.
  • The ACM president (Cherri Pancake) sounded concerned, but due to obscure procedural rules within the ACM she is not allowed to get involved in any sort of investigation.
  • A first attempt is made to contact the ACM Ethics board (upon the recommendation of the ACM President) but no response is received.
• 9 December 2019
  • An ethics complaint is made with IEEE about how the investigation was run (EthicsPoint Complaint #50).
  • The lawyer who runs the IEEE code-of-ethics enforcement says he called the people involved and he thought they made a convincing argument that they didn't have any conflicts of interest because "several" of the members were not past ISCA PC members. Case closed.
• 11 December 2019
  • Various attempts to get a response from the IEEE failed at all levels (from president down to local regional representatives). A senior IEEE member at a local University had the position that the only way to change things is to join up and rise through the ranks for a few years and maybe then someone might care.
  • The IEEE president-elect responded but she was only willing to discuss off-the-record via phone call.
• 12 December 2019
  • An attempt is made to contact individually each program committee member from ISCA2019 and make sure they were aware of the alleged irregularities at ISCA2019. The hope is maybe they will press internally to do something about the issue.
  • Of the large number of people contacted (ISCA'19 PC was pretty large) only two e-mail responses were received. One inquired if there was any evidence. The other said the first author of a paper has no right to withdraw an article due to methodology concerns and that if an advisor says publish, they should just publish without complaint.
  • A few more responses came in where the PC members only wanted to discuss off-the-record via phone. This was declined.
  • One phone message was left by a prominent member of SIGARCH with a checkered past saying they had been planning to eventually bring the topic up for discussion at the ISCA business meeting (meaning, in 6 months, June 2020).
  • A reason for the PC members' reluctance turns up later, when it turns out that 66% of ISCA'19 papers have PC members as authors (the number for ISCA'20 is 48%).
• 3 January 2020
  • After a month the ACM ethics board finally responded (their reason for their delay was they were overwhelmed with other reports of ethical misconduct?)
  • Their response was that "conflicts of interest" are not something mentioned in the official ACM Code of Ethics and thus they were not going to do anything.
  • As an aside, reporting an ethics problem to the ACM was nearly impossible. You have to search the website for contact info, then sort of guess who to e-mail. Then you get no confirmation and you might have to bug them again to get a response. The IEEE, for all their flaws, has an issue-tracker system that works much better for this.
• 24 January 2020
  • The ACM director of publications reported by e-mail that the makeup of the investigation board was being kept secret as part of a general policy (to avoid retribution/harassment). He personally vouched for the team and said he was sure if they had been able to find any evidence of misconduct they would have done something about it.
  • Possibly related to that, the same day the SIGARCH and TCCA chairs (Babak Falsafi and Josep Torrellas) posted an announcement meant to clarify how the investigation was run Information about the ISCA'19 investigation it is made clear that a lot of insiders were deeply involved in the investigation.

The Release of More Evidence and start of Second Investigation

• 28 January 2020
  • An article is posted Evidence Puts Doubts on the IEEE/ACM's Investigation that allegedly shows there was peer-review compromise at both ISCA'19 and possibly HPCA'19.
  • Full reviews from the conferences, with full unredacted names, were found on Chen's hard-drive. With double-blind peer review, no one should have access to the full unredacted reviews, especially not someone in the process of submitting papers.
• 29 January 2020
  • Eddie Kohler, author of the HotCRP submission software used by ISCA, confirmed that there was a breach and that the full paper review info was downloaded. However at the time of the breach there was no logging to tell who obtained the files. It is also unclear how early they had access to these files.
  • This is documented in a twitter thread.
  • confirmation that the audit logs were not enabled for ISCA'19
  • Back in September, after the time of the breach (but before the breach was made public), code was added to HotCRP to enable proper logging of downloads like this. In what might be a co-incidence, the request to add this extra logging support was a request from members of SIGARCH. twitter discussion and the relevant git commit
• 29 January 2020
  • The top-level HPCA'19 organizers are notified about the issue, but they deferred to let Torrellas/TCCA handle things.
  • The top-level HPCA'20 organizers are notified. The PC Chair of HPCA'20 responds that any peer-review issue was with HPCA'19 and can't possibly have any impact on HPCA'20.
• 8 April 2020
  • The "Joint Investigation Committee" is announced. More or less a re-opening of the investigation being run by a bunch of lawyers.
  • Joint Investigative Committee (JIC) Update
• 14 May 2020
  • The various candidates for the upcoming ACM election were contacted to see their take on the ISCA issue. The few that replied were unaware of the issue.
• 29 May 2020
  • Despite the alleged issues with peer review in the past, ISCA 2020 is happening as usual (or as usual as it can in the lockdown year of 2020).
• 1 June 2020
  • Margaret Martonosi of Princeton and head of the NSF CISE directorate gave a keynote speech at ISCA'20. She also is author of 3 papers published there.
  • In December she had given encouragement to everyone who had contacted her that she cared deeply about the ISCA'19 issues and that we should keep investigating. Apparently she did not care enough to withdraw her papers or refuse the keynote.
  • She was also a founding member of SIGARCH/SIGMICRO CARES
• 1 June 2020
  • José F. Martínez of Cornell University (General Chair of ISCA'20, on Board of Directors of SIGARCH) is of the opinion that actions at ISCA19 should have no bearing at ISCA20, as if somehow there isn't a huge overlap between the people involved in the peer review process of both conferences.
  • Twitter link
• 2 June 2020
  • An attempt is made to contact all first and last authors of papers in the remaining two days of ISCA, to encourage them to cancel their presentation out of solidarity with Huixiang Chen.
  • Not a single response was received.
• 4 June 2020
  • SIGARCH wants everyone to be patient with their investigation that they started in February 2020.
  • An update from Joint Investigative Committee (JIC)
  • What they failed to mention is the investigation should have started in June 2019 but Josep Torrellas wasted 6 months dragging his feet on a conflict-of-interest riddled (he was a PC at ISCA'19 and had 3 papers in) "investigation" that found nothing wrong. The JIC is the second investigation after the first one turned out to be a farce.
  • ISCA'20 should really have been cancelled until all of these issues were resolved.
• 8 June 2020
  • T. N. Vijaykumar posts an article about peer-review issues at ACM/IEEE computer architecture conferences.
  • Link: Potential Organized Fraud in ACM/IEEE Computer Architecture Conferences
  • Apparently the SIGARCH blog refused to publish the article. We have a page where we discuss the many issues with the SIGARCH blog
  • The article appears to have been posted on 12 May 2020 but it wasn't widely noticed until it hit the top article on Hackernews on 8 June 2020.
• 15 June 2020
  • Emily Reisbaum the, lawyer in charge of the Joint Investigative Committee (JIC) investigating ISCA'19, sent us an e-mail saying "that there are allegations - even troubling ones - about the integrity of one aspect of the ISCA 2019 conference, does not warrant punishing all of those doing good work by cancelling ISCA 2020".
  • There are a lot of assumptions in that statement, especially from someone supposedly running an unbiased investigation.
  • Reisbaum seems to be of the same opinion as Martínez, that alleged academic fraud at ISCA'19 cannot have any possibly bearing on ISCA'20, as if there isn't significant overlap in the communities behind both conferences.
  • In addition, Reisbaum asked us to refrain from calling for the cancellation of ISCA'20. A bit odd, as it's a bit late for that. Also we feel we had every right to have called for ISCA'20 to be cancelled until the ISCA'19 allegations had been fully and properly investigated.
  • Also, why is the supposedly unbiased JIC investigator sending out e-mails defending the honor of those she's supposedly investigating? We have not received one single response from anyone we e-mailed asking to show solidarity with the dead student. Apparently instead the ISCA authors complained about us to the JIC, compelling them to try to silence us. The JIC is supposed to be investigating the ISCA authors, not sending e-mails on their behalf.
  • If you are one of the ISCA'20 authors complaining to the JIC about us: we don't think asking to cancel ISCA'20 in solidarity with the dead student is the equivalent of some sort of harsh group punishment. Send your work somewhere else instead. MICRO for example. Unless you think MICRO has peer-review issues (in which case, contact the JIC). Or maybe you think your paper would get preferential treatment at ISCA, and if that's the case then you should probably also report that to the JIC.
  • We do not want a rushed investigation, we just feel it was irresponsible to hold ISCA'20 before the issues at ISCA'19 were properly investigated. If you're going to be mad at anyone over calls for ISCA'20 to be cancelled, it's ACM SIGARCH, IEEE TCCA, and the chairs of ISCA'20 who pushed things through in spite of protests.
• 23 June 2020
  • SIGARCH posted a message Remembering Huixiang Chen through our Actions
  • The timeline they post conveniently leaves out the failed first investigation that found "no evidence of improper action [found] during the peer review process". A thing to note here is that Josep Torrellas (who lead the first investigation) was the PhD advisor of José F. Martínez who is a member of the SIGARCH Executive Committee.
  • The "Actions" they took did not involve cancelling or delaying ISCA'20 until a full investigation could be concluded. This is probably because that would have affected the publication count of too many important people (ISCA'20 was 48% papers by the the Program Committee, meaning the people with final decision oversight on which papers got in somehow made the conference nearly half papers that they wrote themselves).
  • Many of the "actions" they claim to have taken seem to be them saying yes, ISCA might have problems, but we also found a bunch of stuff going on at other conferences too.
  • The article is signed by the SIGARCH executive committee. Please let them know how you feel about their decisions.
• 26 June 2020
  • We have called for the members of the SIGARCH executive committee to resign their positions due to their lack of handling of this whole incident.
  • José F. Martínez (Board of Directors of SIGARCH, General Co-Chair of ISCA'20 and ISCA'21 and Associate Dean of Engineering at Cornell University) posted a rebuttal in this (twitter thread) which was hard to read as he blocked us immediately after posting it so we were unable to respond.
    He says that "the overwhelming majority of our research community" does not agree that ISCA 2020 should have been cancelled until the investigation finished, and that by contacting the various presenters and PC members to make sure they were aware of the ongoing investigation we were engaging in harassment.
• 29 June 2020
  • Since the initial incident that sparked this whole thing was allegedly a student so upset about poor methodology at ISCA that he committed suicide, we have started a page where we look at the methodology found in past ISCA papers.
• 8 July 2020
  • An article is published at Nature Index: Probe into leaked papers submitted to leading engineering conference
  • The author investigated some interesting things, including the current situation of the advisor in Florida.
  • Odd that Jose Renau seems to be one of the few people willing to speak up on the issue. For those of you who think we have some sort of anti-UIUC bent, we might as well point out that Renau got his PhD under Torrellas at UIUC too.
• 6 November 2020
  • It looks like they are planning to push through having ISCA2021 despite the investigation still being underway.
  • Any bets they are withholding the outcome of the investigation until November 25th (the day after the papers are due) just like last year?
• 12 November 2020
  • Another sad, but not surprising article from T. N. Vijaykumar where anonymous e-mails allege more collusion fraud at ASPLOS and with NSF reviews.
  • Sad because while the general feeling is this is indeed a widespread problem, none of the profs involved in running the various conferences are willing to do the right thing and do something about it.
  • The even sadder part is most computer architecture researchers know who that bad actors are. Get them talking off the record at a conference, and they'll tell you which cliques play peer-review games, which high-profile computer architecture researcher was kicked out of multiple universities for having affairs with students, and which computer architects are known for harassing women at conferences. For whatever reason though they all feel powerless to do anything, and fear for their career if they speak up. Which means they are cowards at best and complicit at worst.

The Release of the JIC Report

• 8 February 2021
  • The JIC finally released the results of their investigation
  • It took them 1.5 years and multiple stumbles to possibly confirm the events at ISCA'19 that initiated all of this.
  • While investigating they also found evidence of of misconduct at other high-profile Computer Architecture conferences (HPCA'19, ASPLOS'17).
  • It's not even 100% clear that the people punished or the paper involved was the one that led to Huixiang Chen's death as apparently the names will not be released by the ACM.
  • As of 8 February 2021 the paper involved in Huixiang Chen's death is still available for download on the ACM website by the 9th it was finally marked as being retracted, retroactive to 26 January 2021
  • This was all too little, too late, and it raises questions about what other incidents might be happening where the people involved were a bit better at hiding things.
  • Also remember that various higher-ups in the ACM and IEEE (the people running the recent ISCA conferences, as well at the IEEE TCCA and ACM SIGARCH board members) dragged their feet against a second investigation and did their best to dismiss the allegations.
  • Jonathan Aldrich reports: "Apparently it has been longstanding ACM policy to keep names/institutions of those disciplined confidential. The reason given is to avoid creating 3rd party conflicts or having external groups imposing additional penalties.
  • By keeping names a secret It's unclear how the ACM is going to ensure those punished do not serve as reviewers or external reviewers, and also they provide no protection to other organizations such as IEEE or USENIX from accidentally using them as reviewers. Also, these reviewers will possibly still be able to compete for grants and faculty positions with the ACM hiding their findings.
  • For all we know those guilty remain high-level members of the ACM as the findings makes no mention of them being expelled. Apparently a different branch of the ACM (ACM COPE) would handle the non-publication related issues and that probably takes time, but the ACM is definitely not being transparent here.
  • Boycot ISCA! Cancel your ACM and IEEE Membership!
  • Don't belong to an organization that provides privacy protections to those found guilty of academic fraud!
• 9 February 2021
  • The paper still has not been retracted by the IEEE.
  • It's a bit unclear if/when the IEEE will act on things. Some of the co-authors on the paper withdrawn by the ACM are fairly high-ranking IEEE members.
  • The authors on the retracted paper also have a few more papers that got published after the ISCA'19 paper. Is the 15-year publication ban retro-active, or do they get to keep those?
• 14 February 2021
  • After a brief burst of interest on this issue, it's now all been forgotten again. Swept under the rug, as I'm sure most IEEE/ACM members prefer.
  • At some point as many as 30 people were talking about it on twitter, amazing (sarcasm). But due to twitter's idiotic threading most people didn't see each other's comments.
  • Samira Khan posted a few comments that might be relevant.
    link1
    link2
    
• 15 February 2021
  • Everyone is excitedly posting their ASPLOS papers, which are hard to take seriously, as the JIC report claims there has been peer-review issues at ASPLOS recently too. Were the peer-review miscreants involved with the most recent ASPLOS reviews? We can't know unless ACM ever releases names. Should this mean that all ASPLOS publications should live under a cloud of suspicion? Probably, but none of the authors seem to care as long as they get to put another publication next to their name.
  • It's about time for the ISCA'21 PC meeting. Will any of the members be willing to show some honor and boycott things?
  • The PC Chair for ISCA'21 is Lizy K. John, who was the PhD advisor for the author of the retracted ISCA'19 paper, and thus grand-advisor to Huixiang Chen. Will she allow ISCA'21 to continue?
• 16 February 2021
  • Still think ISCA should be cancelled until there's a major change of leadership in both SIGARCH and TCCA.
  • Sent reminder e-mails to much of the ISCA'21 Program Committee, just so they can't pull the whole "oh, I never heard about any of this" excuse they sometimes use to avoid responsibility. As usual, no responses.
  • Really, I think most of them probably feel like the finished investigation is some sort of validation that it was a few bad apples, rather than the tip of the iceberg of the toxic atmosphere present at the conference.
  • Why do we care? Why don't we just let them be to sit around pretending they are relevant and accepting each others papers into their conference? Many of us did try that, just letting them fade into irrelevance. And after the good people left, a student died in a horrifying manner. And no one at the ACM, IEEE, SIGARCH, or the ISCA PC seems to care, or at least they don't care enough to put their publication count into jeopardy. So the least we can do is try out best to keep any other students being dragged into this quagmire.
• 22 February 2021
  • SIGARCH posted a new blog post signed by the SIGARCH board [Babak Falsafi (Chair), Natalie Enright Jerger (Vice Chair), Karin Strauss (Treasurer), Sarita Adve (Past Chair), and Board of Directors members Joel Emer, Boris Grot, Martha Kim and José F. Martínez.]
  • They call out the ACM for a lot of the ACM's nonsense. This letter is much different in tone than the usual ones from the SIGARCH board, making us suspect it was written by someone different than normal.
  • Before we go rah-rah-rah behind them, remember this is the group that was in charge when Torrellas ran the first failed investigation that delayed this whole thing by months, and they refused to do anything about that situation.
• 1 March 2021
  • There was finally some news on the University of Florida side of this whole incident: Article at WUFT
  • The revelations give a pretty grim outlook on what it's like being a grad student in academia today, especially if you have the kind of advisor who thinks publishing in ISCA is a good idea.
  • And despite all that has happened, the University appears to still be dragging its feet on doing anything useful about the whole situation.
• 3 March 2021
  • Inside Higher Ed ran an article
• 16 March 2021
  • Retraction Watch published an article
  • It's always laughable seeing how the ACM director of publications continues to be useless.
  • Also, it's frustrating how all of the chairs of ASPLOS, ISCA, etc, like to claim how powerless they are to do anything about peer review fraud. It almost makes you wonder why they bother pretending the peer review process is anything but a sham.
• 25 March 2021
  • The IEEE finally retracted the paper too
• 15 April 2021
  • Dalmeet Singh Chawla at Nature Index published another article on this: Research misconduct findings, 15-year publishing ban in graduate student suicide case
• 26 April 2021
  • It is reported that IEEE Fellow Tao Li (Huixiang's advisor) resigned from U of Florida effective 15 April. (Gainesville Sun) (WUFT)
  • By resigning, he possibly escapes any further investigation.
• 28 April 2021
  • U of Florida releases to GAU the results of their investigation, which found Huixiang's advisor had "insufficient evidence" of a poor work environment. A separate report INV-21-07 found the advisor erased a large amount of potentially relevant evidence before the investigators had a chance to investigate.
• 1 August 2021
  • The ACM has revoked Tao Li's membership
  • It was nice of the ACM to name at least one name and also to give a lot more details, the peer-review fraud was a lot more involved than you might think. It also raises issues about how much went on that this very superficial investigation missed.
  • The ISCA people will probably consider the issue settled now, an isolated incident. If you read the article though there were various collaborators (possibly even in the peer committee) in the peer-review fraud both at ASPLOS'17 and ISCA'19 and we have not been told what their fate is. They're possibly the people who refused to co-operate mentioend in the JIC report.
  • It is unclear if Tao Li is still an IEEE Fellow.
• 1 August 2024
  • The NSF Office of the Inspector General (OIG) sent out an e-mail saying that their investigation into this matter has been closed.

---

Back to the cancel ISCA page