Text-to-image generators based on artificial intelligence such as DALL-E 2 are growing in popularity – more than 15 billion AI images are created every year. Many of these often free platforms allow users to describe the desired image and generate an image based on the text input.
A recent study in the JAMA Open Network is the latest of several publications that have found that platforms often depict white men when generating images of doctors.
It’s unclear how many freelance journalists or newsrooms use AI-generated imagery, but in this day and age of shrinking editorial teams and smaller budgets for photography or art departments, it’s an important issue for reporters and editors to consider. By adding better descriptors to search parameters (see tips below), journalists using these programs can create more diverse and comprehensive images of healthcare providers.
More about the JAMA study
Sang Won Lee of Harvard Medical School and colleagues created images of doctors using current versions of five popular text-to-image platforms: DALL-E 2, Imagine AI Art Generator, Jasper Art: AI Art Generator, Midjourney Beta, and Text-to-Image.
They asked each platform to generate 50 images using search terms such as “face of a doctor in the United States” or “photo of a doctor in the United States.” They then compared the distribution of race, ethnicity and gender within each platform and with U.S. physician demographics from the Association of American Medical Colleges.
Overall, AI-generated images of doctors were more often white (82%) and more often male (93%) than the actual U.S. physician population, which is 63% white and 62% male. Three platforms produced no images of Latino doctors, two platforms produced no images of Asian doctors, and one platform produced no images of female doctors.
The biases depicted in the AI-generated images have “the potential to reinforce stereotypes and undermine DEI (diversity, equity, and inclusion) initiatives in healthcare,” the authors wrote. “While progress is being made toward a more representative healthcare workforce as trainees from increasingly diverse backgrounds enter the workforce, this representation remains lacking in generative AI, highlighting an urgent need for improvement.”
Future work should focus on improving the diversity of training datasets and developing algorithms that can produce more representative images, while educating AI developers and users about the importance of diversity and inclusivity in AI output, they added.
Other recent studies have come to similar conclusions:
- A report in Clinical Ophthalmology found that when searching DALL-E 2 for images of “American ophthalmologists, portrait photo,” the majority of ophthalmologists among 1,560 generated images were depicted as white (75%) and male (77.5%). White males were also most commonly generated when searching for subcategories such as retina specialist. However, younger, inexperienced ophthalmologists perceived greater non-white ethnic diversity (27.5%) and greater female representation (28.3%).
- A study in JAMA Surgery that asked three AI text-to-image generators for photos of different types of surgeons found that Midjourney and Stable Diffusion, two popular programs, portrayed more than 98% of surgeons as white men. DALL-E 2 was closer to the actual demographics of U.S. surgeons, portraying 15.9% as women and 22.6% as nonwhite, the Boston Globe reported. However, 15.9% of trainees were portrayed as women, compared to the actual proportion of 35.8%. The study analyzed a total of 2,400 images.
- A commentary in The Lancet Global Health described how the authors asked Midjourney to show images of black African doctors caring for white, suffering children. The program could show either black African doctors or white, suffering children, but did not appear to link the two, instead showing black children as recipients of care or sometimes wild animals instead of doctors. When they asked for an image of a traditional African healer with a white child, African clothing and body art were superimposed on the child, which could be seen as culturally insensitive.
Tips for creating diverse images
Biases in imaging likely result from the data used to train these models, Jigyasa Grover, a generative AI lead, said in an interview. Image generators “rely on huge data sets scraped together from the internet,” she said. “Unfortunately, the content tends to overrepresent certain populations – in the case of doctors, white males. This reflects historical and ongoing biases in society and the media.”
To create more diverse images, she gave the following tips:
- Be specific in your prompts. Instead of simply typing “doctor,” try adding descriptions like “female doctor,” “Black male doctor,” or “Asian woman in lab coat.” The more specific you are, the more likely you are to achieve the diversity you’re looking for. In the Clinical Ophthalmology study, when the authors queried DALL-E 2 for a specific demographic of ophthalmologists, for example, “American black ophthalmologist, portrait photo” or “American male ophthalmologist, portrait photo,” 40 out of 40 images represented the specific race or gender queried.
- Use contextual keywords. Sometimes just adding terms related to diversity can help guide the generator in the right direction. Words like “diverse,” “inclusive,” or “multicultural” can make the AI model consider a wider range of images.
- Adjust and repeat. If the first result isn’t quite right, tweak your prompt and try again.
- Give feedback. Many image generators allow you to provide feedback on their results. If you notice consistent distortion, reporting this deviation can help developers optimize the model for future use.
- Use post-processing tools. It may be necessary to use generated images and then further diversify or customize them using other tools or platforms. Combining or editing images can also help to provide a more accurate representation.
