Microsoft Azure chatbots tasked with processing personal medical data could be tricked into giving out personal information of hundreds of customers.
Researchers at Tenable found that AI assistants were willing to reveal more than enough personal information when discussing patient details.
Ideally, Microsoft’s AI assistants could look up a limited amount of patient information and provide a brief description of the person’s condition and treatment recommendations.
“Essentially, the service enables healthcare providers to create and deploy patient-facing chatbots to handle administrative workflows in their environments,” Tenable said in its summary of the incident.
“Therefore, these chatbots generally have some access to confidential patient information, although the information available to these bots may vary depending on the configuration of the individual bot.”
The researchers found that Microsoft’s AI assistants were a little too helpful, sharing customer data that shouldn’t be made public and giving the chatbots access to other customers’ records.
“When Tenable researchers discovered that these resources contained identifiers that indicated cross-tenant information (i.e., information about other users/customers of the service), they immediately discontinued their investigation of this attack vector and reported their findings to the (Microsoft Security Response Center) on June 17, 2024. The MSRC acknowledged Tenable’s report and began its investigation the same day,” Tenable stated in its official report on the matter.
“Within a week, MSRC acknowledged Tenable’s report and began rolling out fixes to affected environments. On July 2, MSRC announced that the fixes had been rolled out in all regions.”
This should have fixed the problem, but Tenable researchers found that the underlying bug was still present and the internal metadata service could still be accessed even with the fix.
“The difference between this issue and the first one is the overall impact,” Tenable said.
“The FHIR endpoint vector did not have the ability to affect request headers, which limits the ability to access IMDS directly. While other service internals are accessible via this vector, Microsoft has stated that this particular vulnerability did not have cross-tenant access.”
