For a Human-Centered AI

Artificial intelligence and health data

April 5, 2024

World Health Day will be celebrated on Sunday, April 7. One of the issues debated in recent years is the importance of diagnosis: according to a 2016 University of Baltimore analysis, in the United States alone, medical errors cause more than 250 thousand deaths a year out of a total of about 2.6 million deaths.

Fortunately, in the face of this real social emergency, AI has come to the aid of health care workers, that appears to be almost twice as accurate as doctors in making diagnoses. However, this brings with it a number of issues, mainly related to data privacy. Solutions are also moving toward the use of synthetic data, which offers an optimal combination for future healthcare and research. But from what perspective? We talked about this with the director of Fondazione Bruno Kessler‘s Center for Digital Health and Wellbeing, Stefano Forti.

The use of AI raises an important question that experts have been wondering about for some time: will the physician be replaced by these systems (e.g., Machine Learning or Virtual and Augmented Reality) in the coming years?

“The answer has to be contextualized,” said Stefano Forti, who is also a contact person for FBK on the Executive Committee of the “TrentinoSalute4.0” center for digital health development. “It depends on the type of tool and the intended use,” he added.

Explain further, please.

“Regarding the area of diagnostic and prognostic models trained on data, to date we cannot disregard the fact that the final decision always rests with the physician. The main reason is that these tools ‘see’ only one part of the problem. Algorithms do a single task well (e.g., say whether an x-ray image is positive or negative), but they cannot count on a range of other contextual data (such as the patient’s medical history), which only the health care provider possesses. Radiologists were the first to support the development and deployment of AI and at the same time to fear being replaced by those algorithms.  For the past few years, diagnostic algorithms that replicate the diagnostic accuracy of the best radiologists have been available. The useful way to use them is as decision support for the radiologist, who, at the time of diagnosis, can take into account the “algorithm’s suggestion,” but puts it in context with respect to the patient’s history. As can be seen in recent years, radiologists have not lost jobs, far from it. There is a shortage of radiologists in the national health care system. So integrating AI into health care can not only improve outcomes, but it also promotes more efficient and patient-centered diagnosis and treatment, and, let me be clear, by supporting human expertise and not replacing it.”

Another important issue on national discussion tables is the relationship between AI and privacy. What are the risks and what are the opportunities?

“The AI Act, a regulatory document from the European Union, is a very good reference point. The general directions now will have to be specifically applied based on the different areas.  In the health sector there are many issues to be addressed. We know that algorithms need data to be trained. But the availability of data in healthcare is unfortunately still a problem to be solved. The development of AI models cannot be separated from the informed consent that the patient must provide to allow the use of information about him or her. Any study whose purpose is to validate clinical efficacy and risk must include consultation with the ethics committee of the health care institution that makes its patient data available to researchers. That is why to date the focus on privacy and consent from patients is rightly very high.”

In this context, citizen consent is a key issue.

“Patients must be informed for what purpose their data is being used. But actually there is nothing new: the same principle in health care already applies when undertaking the various clinical trials (for example, on the approval of new drugs).”

There is also much discussion about the responsibility of physicians when supported by Artificial Intelligence.

“Beyond the need to certify these tools as medical devices, their use in clinical practice is still in its infancy.  Bringing an AI diagnostic-predictive algorithm ‘into production’ in clinical practice implies a systems approach in which various stakeholders must necessarily be involved, starting with clinicians and including the various institutional levels that deal with socio-economic-organizational and privacy aspects.

We are talking about digital and AI solutions that are not aimed at doctors, but directly involve patients. In what area are they most useful?

“The potential is great, especially in the area of prevention.  We are already seeing a lot of development of appropriately designed ‘virtual’ assistants (chatbots) that can perform a variety of tasks in an automated way. Particularly in behavioral medicine, they are designed to provide patients with skills and competencies that enable them to better manage their health and take care of themselves.    This is what is called patientempowerment . Through these systems, health care providers could entrust virtual assistants with some educational and psycho-educational tasks and thus gain more time to devote to other more critical aspects of their patients’ care. The study of these digital behavioral interventions is part of a growing area of science and innovation called ‘digital therapies.'”

Speaking of apps and digital therapies, is the Italian patient-user ready for this new challenge?

“Let’s look around: who doesn’t own a smartphone nowadays? Who doesn’t use apps to manage every aspect of their lives, from bank account management, to entertainment or, as in the case of the provincial TreC+ platform, for health? So the answer is yes, all the more so if within the design and development process of these digital interventions, aspects of usability and user experience have been taken into account. Then there is a part of the population that owns a smartphone, but refuses to use any kind of app and only uses it to make phone calls. With those people, we need to activate awareness programs and courses on the importance of using these tools for health purposes. If the patient finds the app useful then they will use it. Also, if it is the doctor who proposes the use of new technologies to their patients, they will have more confidence and start using them.”

And are physicians ready?

The most important aspect is to make healthcare personnel understand the potential of AI and digital therapies, highlighting the benefits they can derive for their work. This educational component should be offered as early as within medical and health professions degree programs, but it should also be part of the training of  managers working in various capacities within health care organizations, starting with general managers and medical directors. The key theme is that these new technologies have a strong impact at the organizational level in that they cannot be just an ‘addition’ to current processes, but necessarily lead to the need to rethink new models of health service delivery.”

Citizens, therefore, become more and more managers of themselves and increasingly aware of the management of their own health. This is one of the missions of TreC+, a digital platform through which citizens can manage their data and network with the health care system. How does the platform develop and how will the role of the citizen user/patient evolve?  

“We believe that the issue of prevention (primary, secondary or tertiary) is of paramount importance. To date, the health care system is very much in the doldrums with respect to these issues.  Providers focus on treatment, and with dwindling resources, the opportunities to proactively put in place prevention actions, especially in mental wellness, which has a lot of hidden cases, are very few today.  In Trentino, the platform “TreC +” (which stands for Cartella Clinica del Cittadino, Citizen’s Medical Records) has been active for several years, through which Autonomous Province of Trento residents can take advantage of a wide set of online health services. Users can access one’s Electronic Health File (Fascicolo Sanitario Elettronico (FSE) to consult information and documents concerning one’s health, such as reports, drug prescriptions and specialist referrals. They can schedule exams and visits, and through auhtorization, also manage all functions for loved ones they provide care for, such as an elderly parent or minor children. In addition, telemedicine services will be provided in the coming years through the TreC+ platform.”

What have you been working on in recent months?

“We have been working on the “third layer” for the TreC+ platform, which concerns virtual assistants in the area of mental wellness and promotion of healthy lifestyles. In particular, we are studying solutions that target “at-risk” individuals (e.g., overweight adults or children), with interventions that motivate and teach the patient how to become a more responsible and educated “manager” of their health.”

How many citizens are using the TreC+ platform today?

“More than 270 thousand users in the autonomous province of Trento, representing more than 50 percent of the Province population. In the cell phone and tablet app versions, we count more than 170 thousand users.”


The author/s