List items
Items from the current list are shown below.
Blog
11 Oct 2020 : The danger of a non-transparent AI Register #
The cities of Helsinki and Amsterdam recently announced the launch of their local government AI Register (Helsinki) and Algorithm Register (Amsterdam). This is certainly forward-looking and with positive aims, but actually looking through the registers, I was surprised and a little perturbed by how vague the entries are.
If the purpose of the registers is to promote accountability then it concerns me that the current implementation only provides the veneer of transparency. If government is claiming to provide transparency when it's not, however well-intentioned, this can lead to more harm than good.
Here's the feedback I sent to the city administrations and to the company running the registers. I'm not really expecting any results, but writing out my concerns was extremely therapeutic, albeit also quite time consuming. I recommend it as a satisfying activity if you have the time to spare.
If the purpose of the registers is to promote accountability then it concerns me that the current implementation only provides the veneer of transparency. If government is claiming to provide transparency when it's not, however well-intentioned, this can lead to more harm than good.
Here's the feedback I sent to the city administrations and to the company running the registers. I'm not really expecting any results, but writing out my concerns was extremely therapeutic, albeit also quite time consuming. I recommend it as a satisfying activity if you have the time to spare.
With the recent establishment of your AI/algorithm registers, it’s great that you're taking the transparency of automated processes seriously. I hesitate therefore to criticise the schemes which are clearly well-intentioned and a step in the right direction, but it concerns me — based on the data currently available in the registers — that in their current form they may do more harm than good.
My three main concerns are the following.
1. Confusion between AI and algorithms. These two things are not the same, and conflating the two degrades public understanding of the issues involved. Algorithms cover a very broad set of concepts that includes every piece of software in use today. AI (or more specifically Machine Learning) is a much narrower concept. Machine learning involves applying an algorithm to a dataset, in order to produce a separate algorithm that can then be used as the basis for decision-making (or some other task). The resulting algorithms are much more opaque, their biases much harder to understand, and the datasets much more important for providing that understanding. Right now the register seems to include a mixture of both machine learning and traditional algorithms, but without any clarity over which is which. For each of the entries it should be made clear whether machine learning is involved, and if so what type.
2. Providing the algorithms. The entries in the database provide only a very high-level overview of the algorithms being used. Frankly, these are of no real use without more detail and the code for the algorithms needs to be made available. I’m very aware that commercial sensitivity is often used as an argument for why this can't be done, but as someone who works for a company developing open source software, I’m also aware that keeping algorithms and datasets private isn’t the only way to run a commercial or public service. If the register is to have real benefit, Helsinki and Amsterdam cities need to apply pressure to companies to make their algorithms available, or else give preference to those companies that will. Otherwise the register will end up being no more than a list of names of companies supplying software to local government.
3. Providing the datasets. If the algorithms are machine learning algorithms, then the full datasets used for training need to be made available (or a recent snapshot in the case of dynamic learning). Consideration must be given to privacy, and this is a real challenge, but the good news is that there’s a wealth of existing good practice in this area, especially coming from universities with their growing culture of open data for validating research, supported and encouraged by EU funding requirements.
To reiterate, I applaud the idea behind the registers, but I’d also encourage you to go further in order to allow them to be the real tools of accountability that the public needs, and that I think you're aiming for.
I was pleased to complete the survey on your site, where I also entered these comments. When you reply please let me know if you prefer me not to make our correspondence public (I’ll assume that it’s okay for me to do so unless you state otherwise).
Thanks for your efforts with the registers, which I wish you every success with, and for taking the time to read this message.
My three main concerns are the following.
1. Confusion between AI and algorithms. These two things are not the same, and conflating the two degrades public understanding of the issues involved. Algorithms cover a very broad set of concepts that includes every piece of software in use today. AI (or more specifically Machine Learning) is a much narrower concept. Machine learning involves applying an algorithm to a dataset, in order to produce a separate algorithm that can then be used as the basis for decision-making (or some other task). The resulting algorithms are much more opaque, their biases much harder to understand, and the datasets much more important for providing that understanding. Right now the register seems to include a mixture of both machine learning and traditional algorithms, but without any clarity over which is which. For each of the entries it should be made clear whether machine learning is involved, and if so what type.
2. Providing the algorithms. The entries in the database provide only a very high-level overview of the algorithms being used. Frankly, these are of no real use without more detail and the code for the algorithms needs to be made available. I’m very aware that commercial sensitivity is often used as an argument for why this can't be done, but as someone who works for a company developing open source software, I’m also aware that keeping algorithms and datasets private isn’t the only way to run a commercial or public service. If the register is to have real benefit, Helsinki and Amsterdam cities need to apply pressure to companies to make their algorithms available, or else give preference to those companies that will. Otherwise the register will end up being no more than a list of names of companies supplying software to local government.
3. Providing the datasets. If the algorithms are machine learning algorithms, then the full datasets used for training need to be made available (or a recent snapshot in the case of dynamic learning). Consideration must be given to privacy, and this is a real challenge, but the good news is that there’s a wealth of existing good practice in this area, especially coming from universities with their growing culture of open data for validating research, supported and encouraged by EU funding requirements.
To reiterate, I applaud the idea behind the registers, but I’d also encourage you to go further in order to allow them to be the real tools of accountability that the public needs, and that I think you're aiming for.
I was pleased to complete the survey on your site, where I also entered these comments. When you reply please let me know if you prefer me not to make our correspondence public (I’ll assume that it’s okay for me to do so unless you state otherwise).
Thanks for your efforts with the registers, which I wish you every success with, and for taking the time to read this message.
Comments
Uncover Disqus comments