Skip to main content

Fairness, accountability and transparency in Machine Learning? Jo Bates reports back from ACM FAT* in Atlanta, USA


A couple of weeks ago I travelled to Atlanta, USA to attend ACM FAT* - an interdisciplinary conference that addresses issues of Fairness, Accountability and Transparency in Machine Learning. Officially, I was there on the hunt for potential papers and authors to invite to submit their work to Online Information Review. However, the FAT* field is also closely related to my research interests around the politics of data and algorithms, and my teaching on the Information School’s MSc Data Science. I was keen to check out what was happening in the FAT* community, and feed my findings back into my teaching and into two new projects I am working on in this field: CYCAT & supervising a new PhD student – Ruth Beresford – whose research will investigate algorithmic bias in collaboration with the Department for Work and Pensions.

I was privileged to hear a number of great papers – the best of which engaged critically with issues of social context and justice. My two favourite papers which I highly recommend for anyone interested in these topics were:

Fairness and Abstraction in Sociotechnical Systems (Selbst et al, 2019) was a great paper that spoke to a sense of unease I have felt recently about the increasing amount of technical work attempting to solve the ‘algorithmic bias problem’. The paper – written by an interdisciplinary team of social and computer scientists - not only speaks to such concerns in an engaging and insightful way, but also offers a strong analytical framework that illuminates five “traps” that such technically-driven work often falls into:
  •          Framing: The authors begin by critiquing the ‘algorithmic framing’ common in data science. In such an abstraction, the focus of the data/computer scientist is simply on evaluating, for example, whether the model has high accuracy. They point out that such a framing is ineffective for addressing issues of bias and fairness. Expanding this algorithmic framing to a ‘data frame’ which also involves interrogating directly the data inputs and outputs for issues of bias and fairness, can address some of these issues, but also has its limitations. Instead they advocate data scientists adopting a socio-technical framing which explicitly recognises that any ML model is part of a socio-technical system – and we need to move all the decisions made by humans and human institutions into the abstraction boundary. I couldn’t agree more!
  •         Formalism: This is an important ‘trap’ for computer scientists and mathematicians to be aware of. It relates to the failure to account for the complexity of social concepts such as fairness, bias etc. The meaning of such concepts is contextual and contestable – they cannot be reduced to mathematical formalisms!
  •         The Ripple Effect: This ‘trap’ points to the lack of awareness that when technical solutions are embedded into existing social systems, they can impact upon the behaviours and values of those in the social system – often in unexpected ways.
  •         Portability: This ‘trap’ relates to the problems inherent in repurposing algorithmic solutions from one social context to another – and the inevitable problems of inaccuracy, misleading results, and potential for harm.
  •         Solutionism: And, finally - the simple observation that technologists often fail to recognise that the best solution to a problem may not involve any technology!

Putting some of these ideas into practice was my second favourite paper – which also won the prize for best technical and interdisciplinary paper - Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments (Green and Chen, 2019).

With a focus on risk assessments being used in the US criminal justice system, the authors argue that given risk assessment tools do not actually make decisions, but are used to inform judges’ decisions, it is important to understand how people actually interpret and use the outputs of these tools. Their study is based on an experiment involving Amazon Mechanical Turk workers – rather than actual judges – however, their findings are concerning. They found their participants under-performed the risk assessment tool even when presented with the prediction of the tool; they were unable to effectively evaluate the accuracy of their own decisions or those of the tool; and, most concerning they exhibited biased interaction with the tool’s prediction whereby use of risk assessments in decision making led to participants making higher risk predictions for black defendants and lower risk predictions for white defendants. Clearly, these findings need examining ‘in the wild’, but they are concerning, and evidence the importance of a socio-technical framing as called for by Selbst et al.

While there were some excellent papers presented at FAT*, there were also a good few that fell into some of the ‘traps’ of abstraction identified in Selbst et al’s paper – and this resulted in some interesting commentary about the nature and direction of the field. For example, important questions were raised by Stanford PhD student Pratyusha Ria Kalluri, who drew upon Catherine D'Ignazio and Lauren Klein’s forthcoming book Data Feminism, to question the language of fairness, accountability and transparency, and how it relates to notions of justice. Her comments received a lot of support from attendees and online:


Pratyusha’s observations reflect many of my own – and others in the Critical Data Studies space - concerns about what it means to work across disciplinary boundaries in this field, and the politics of engaging in such work with people who may have very different agendas, assumptions, and understandings about what is at stake. It can sometimes be difficult to know how best to navigate these tensions – but it feels like 2019 could be an important moment for shaping the direction of the field.

Comments

Popular posts from this blog

Joint PhD presentation between Sheffield and Makerere, Uganda, delivered by Liliana Sepulveda Garcia

Last week saw the first presentation in a series of joint talks between the Information School's Health Informatics and Information Systems Research Groups in Sheffield. and Makerere University in Kampala, Uganda.The talks aim to promote research collaboration and knowledge sharing.

Dr Laura Sbaffi and Dr Efpraxia Zamani are organising this series and chairing the Sheffield presentations, and Prof Josephine Nabukenya will be chairing the presentations from Makerere.

In this first session, PhD student Liliana Sepulveda offered the audience a great overview of her PhD research on "An experiential study of the human-technology relationship between informal caregivers of people with dementia and assistive technologies".


There will be similar virtual meetings every month and the next schedule one is for Tuesday 11th June, when a PhD student from Makerere will be presenting their work. More details will be forthcoming.

You can view the recording of the session here: https://di…

PhD student Gianmarco Ghiandoni presents at UK-QSAR conference

Gianmarco Ghiandoni, PhD student in our Chemoinformatics research group, recently attended and presented at the UK-QSAR conference in Cambridge.

Gianmarco attended the conference and presented a part of his PhD project, which involves the development of "Reaction Class Recommender Systems in de novo Drug Design".

'These algorithms are machine learning models that have recently acquired great importance due to their effectiveness in product recommendation', Gianmarco said. 'In particular, companies such as Amazon, Netflix, Spotify, etc., have built their reputations and businesses on the top of these models. At Sheffield, we have decided to apply these methods in order to produce suggestions for decision making in automated molecular design. The results from their application indicate that recommender systems can improve the synthetic accessibility of the designed molecules whilst reducing the computational requirements.'


New Article: Services for Student Well-Being in Academic Libraries: Three Challenges

Services for Student Well-Being in Academic Libraries: Three Challenges
Our Director of Research and Senior Lecturer, Dr Andrew Cox, has published a new article alongside Dr Liz Brewster at Lancaster University.

There has been a wave of interest in UK academic libraries in developing services to support student well-being. This paper identifies three fundamental and interrelated issues that need to be addressed to make such initiatives effective and sustainable. Firstly, well-being has to be defined and the impacts of
interventions must be measured in appropriate ways. Secondly, there is a need to identify the true nature of the underlying social problem around well-being. Thirdly, relevant approaches to the issue need to be located within the professional knowledge base of librarianship.
To read the article, click here.