Artificial Intelligence in Basic and Clinical Neuroscience: Opportunities and Ethical Challenges

Shared agency and autonomy in human-AI interaction

In the context of very close human-AI interaction, for example in a closed-loop brain-computer interface in an epilepsy patient, the degree to which the underlying AI system is granted decision-making capacity and—conversely—how much the human subject is kept in the loop in these interactions may lead to new hybrid forms of human-machine or human-AI actions.

Imagine, for example, Maria, a 45 year old woman with severe motor paralysis of the upper and lower limbs who has been implanted with a closed-loop electrode system that allows her to use her brain activity to operate a service robot that can reach, grasp and bring her objects.

Now, any particular action sequence by Maria, say for example fetching a cup and drinking tea, is only realizable by decoding her brain activity and having the robot perform the required tasks. Suppose further that the robot itself also has some degree of autonomy in terms of how it realizes this goal, for example it may have the capacity to freely roam the room and grasp the cup any way that is optimal for realizing the set goal. In such a scenario, it would be reasonable to consider the human-robot interaction necessary for realizing Maria’s goals as requiring a form of shared agency (and autonomy) between Maria and the service robot.

This may seem perfectly fine for all instances in which the interaction works as intended by Maria and her goals a fully realized without significant deviations from the robot. But what happens in cases of unintended yet substantial failures—what if, for example, the robot spills the hot tea and injures a third person or Maria herself? Such interactions gone awry lead to questions of responsibility and accountability in human-AI interaction that are difficult to navigate both ethically and legally.

Accountability, responsibility and the question of trust

Without having the space to provide a detailed and philosophically grounded conceptual analysis here, I urge the reader to consider the difference—ethically and legally—of the concepts of accountability and responsibility. Both denote the ascription to an individual (or active claim by an individual) of some kind of causal agency in a particular action or sequence of actions; for example: “Margret was responsible for writing the letter to the president.” or “The policeman is accountable for explaining his use of his service weapon.”.

In cases of very close, shared or even hybrid actions that are performed in concert by a human and an AI system, however, one might encounter a gap in our ability to unequivocally ascribe responsibility and/or accountability to particular actions. This “accountability gap” (Kellmeyer et al., 2016) may arise in many situations in which decision-making capacity is relegated to an AI system—e. g. a deep-learning-based brain implant or a self-driving car—whose internal learning dynamics and decision-making processes we cannot sufficiently infer: the so-called “black box” aspect of AI (Castelvecchi, 2016). In the ethical and legal domain, we do not yet have effective and resilient norms to ascribe (let alone adjudicate) responsibility in cases of system failures for such black box systems.

Therefore, the topic of → interpretability of machine learning algorithms, particularly ANNs for deep learning, is not only of great interest for computer scientist and engineers, but also an indispensable prerequisite to be able to provide a reasonable ethical understanding and precise legal instruments to adjudicate future cases of liable human-AI interactions.

Intrusive AI and the protection of brain data, mental privacy and personal identity

Today, our methods for observing brain activity, mainly EEG, functional MRI and related methods have inherent limit in the ways in which they can measure the temporal, spatial and frequency-related characteristics of brain signals. This limits the amount and quality of information that we can extract from these signals with our current analytical methods, yet we already see how emerging machine learning methods, specifically deep learning, improve our information extraction capabilities substantially (Akkus et al., 2017; Milletari et al., 2017; Schirrmeister et al., 2017a).

If this progress in data analysis will be complemented by substantial improvement in our measurement methods, for example with intracortical microelectrode grids that measure EEG directly from the cortical surface or, as yet unproven methods such as “neural dust” (a system of intracortical nanoparticles and ultrasound) (Neely et al., 2018), we can expect substantial further progress in the types and amounts of information that can be extracted from neurotechnological measurements.

The practical limits on the amount and specificity of information that can be extracted from brain signals at the individual level—now and in the near future—mean that scenarios involving “reading” the “mind” or “thoughts” will remain elusive for the time being. This will not deter scientists in public (or private-public) research institution nor researchers in technology companies (that have invested substantially in their own neuroscience and neurotechnology research in recent years (Kellmeyer, 2018; Strickland, 2017; Regalado, 2017; Clark, 2017), however, to use brain data as an interesting class of personal data in multimodal deep learning analysis frameworks. In this scenario, we do not yet know whether: a) the combination of many different classes of data (e. g. user behavior, geolocation data, data from devices, brain data etc.) allows for hitherto unprecedented inferences on an individual’s first-person subjective (i. e. “mental”) experience and/or her personal identity (Kreitmair et al., 2017); or, b) the aggregation of such multimodal data from an unprecedented number of individuals (e. g. in a large-scale “experiment” on an internet platform in which a company outfits thousands of users with a consumer neurotechnology device, e. g. a dry-cap EEG, for measuring and uploading brain data to their company servers) would allow to identify particular groups of individuals based on social, biological or other markers—which would raise concerns regarding the protection of group privacy (Ienca et al., 2018; Taylor et al., 2017).

In the neuroethics community and other research fields, these concerns have precipitated a discussion around whether brain data should be treated as a special class of data that needs extra protection in data protection guidelines and regulations (such as genetic data e. g.) (Kellmeyer, 2018), perhaps even “neurorights” that refer to basic human rights (Ienca and Andorno, 2017), or whether in fact the question of what actually constitutes biomedical or health-related data becomes increasingly meaningless as AI methods can make health-related inferences on many different types of data (and their combination and aggregation) that would have previously not been considered to be special health-related data (e. g. your movement patterns from your mobile phone, or your user behavior on the web).

Bias in human and artificial intelligence in interaction

The propensity to take “mental shortcuts” (also known as a → heuristic) for judgement is an inherent feature of human cognition and serves important purposes in everyday life decision-making. If these heuristics, however, produce systematic skews in our decision-making, they are called biases which, if accumulated over time, can produce substantial distortions of knowledge and behavior both at the individual and societal level. These individual and societal biases are also an important driver of creating and maintaining social injustices, e. g. rooted in prejudice, stereotyping, discrimination and other negative social attributions.

The data streams that power current large-scale AI systems, e. g. in translation engines, navigation systems or computer vision (e. g. face recognition technology) today are based on human-derived knowledge structures (ontologies) and most artificial neural networks for deep learning are trained with data that require the input of human experts (e. g. for selecting and labeling the data). Therefore, any bias that is engrained at the level of data selection, structuring, labeling and so forth, may be reproduced, inflated and disseminated by an AI system that is trained on these biased data. Many examples in recent years show, how this can lead to a perpetuation of social injustices and discriminations that are based on human biases, e. g. with respect to ethnicity, gender and other social markers (Knight, 2017; Baeza-Yates, 2016).

Now, there is no easy fix for this deeply entrenched and interlocked problem of human biases and their spillover effects into AI bias. For one, human cognitive biases are almost impossible to contain effectively at the individual level, i. e. most behavioral training methods for so-called de-biasing have failed to show substantial let alone sustainable effects in reducing cognitive biases in humans (Smith and Slack, 2015; Croskerry et al., 2013a; Croskerry et al., 2013b). Furthermore, there is also no straightforward way computationally to effectively de-bias AI systems, both in terms of reducing the technical aspect of bias in algorithms (see → bias) nor the human-derived biased data structures and ontologies (Krywko, 2017; Geman et al., 1992). On the bright side, many computer scientists and data scientists have now recognized the problem and are actively working on potential ways to mitigate the problem (Courtland, 2018).

Meanwhile, however, it is important for researchers in neuroscience and clinicians to be aware (and to raise critical awareness) that automated AI systems may contain biases in their decision-making procedures.

The problem of “perpetual ethics”, governance of AI systems and fair access

In the legal, political and regulatory sphere, these developments raise questions on whether existing regulatory and legal procedures suffice for ensuring the responsible research and effective governance of AI across all sectors in society, while preserving the innovation dynamics of the beneficial applications of this emerging technology (Voeneky and Neuman, 2018; Kaebnick et al., 2016)[42, 43]. Raising awareness and promoting a participatory societal discourse on the ethical issues around AI is commendable and necessary first step for achieving a more inclusive process of deliberation and technology governance.

At the same time, however, the inherent complexity of human-AI interaction and the many stakeholders in AI could also produce a potential problem of “perpetual ethics”—an infinite loop of inter- and transdisciplinary debate without a mechanism and route for democratically legitimized and evidence-based sociopolitical adaptations in the form of laws, rules and regulations.

We can already see how the big five technology companies are all too eager to participate in (some might say usurp) the ethical discourse around AI and neurotechnology (Murgia and Shrikanth, 2019; Hoffmann, 2017). A democratically grounded process of multistakeholder deliberation on the ethics of AI and neurotechnology, however, requires equal and fair access to the debate in the public sphere, rather than the oligopolization of ethical discourse by academia, experts and big companies. Importantly, researchers of all career levels involved in AI-related disciplines (whether from a developmental, computer science perspective or in applied areas such as medicine) can actively participate in exerting counterpressure to this domination of the ethical discourse by private companies by engaging in science communication and public outreach at their institutions.

Furthermore, apart from this bottom-up process of a participatory discourse in societies on equal terms, the transnational nature of technology governance also requires the involvement of supranational bodies (such as the EU) and international organizations (such as the UNESCO) in developing effective and adaptive instruments of governance (i. e. laws and regulations) that preserve the right and freedom to science while making sure that AI is used to nurturing human well-being and flourishing rather than feeding the revenue stream of big technology companies.