Is your tech discriminatory?
Employers are increasingly reliant on technology to assist with all kinds of functions – from strengthening security to streamlining recruitment processes.
Most organisations will consider the basic data protection implications of incorporating technology into employee processes, but there are other legal risks that may be relevant but less obvious.
Indirect tech discrimination
There have been a number of claims in the press relating to facial recognition systems that allegedly indirectly discriminate on the ground of race. Uber in particular has faced claims. It has been reported that its drivers are required to log in by providing a photograph of their face and that if the photograph taken does not match the photograph on file, they risk having their account terminated. The Independent Workers Union of Great Britain (IWGB) and App Drivers and Couriers Union (ADCU) have both contended that Uber’s facial recognition system generates more inaccurate results when used by black and minority ethnic workers, leaving them more vulnerable to losing their jobs with Uber.
Automated decision-making (ADM)
UK GDPR gave data subjects the right not to be subject to a decision based solely on automated processing, including profiling, if the decision had legal effects or was otherwise significant in its impact save in very limited circumstances, namely where the data subject has explicitly consented, when the decision is necessary for the performance of a contract with the data subject or where it is required by law. .
However, the law has since been relaxed by the Data (Use and Access) Act 2025 which retains the existing stringent rules for ADM involving special category data but for all other ADM, this can now be used (subject to compliance with the usual data protection principles) provided the data subject has been given information about the decision made about them, are able to make representations and challenge the decision and can obtain human intervention about the decision.
There have been a number of claims in the press relating to facial recognition systems that allegedly indirectly discriminate on the ground of race.
It’s an area of data protection that has remained relatively under-the-radar, and many organisations routinely declare that ADM is not taking place – without stopping to ask what ADM really means and when or where it might be used. With increased integration of AI into recruitment processes, this is one area where employers may need to revisit the default position of declaring ADM is not applicable.
The ICO is currently updating its guidance on the use of ADM. Employers would certainly benefit from guidance clearly delineating between decisions made solely by automatic means without human intervention and automated processes which merely assist human decision-making – with examples and case studies.
Watch this space
These developments indicate an important direction of travel. As technology becomes increasingly integrated into decision-making, employers need to recognise that their systems – as well as individual members of staff – could be a source of litigation risk.
Contact our employment and data protection team for further legal support.
