Professor Jungwoo Lee's Research Team at Seoul National University Develops AI Bias Mitigation Technologies to Increase the Reliability of AI Models
- Paper presented at NeurIPS 2024, the most prestigious international conference in the field of AI
- Expected to be widely used in video data, medical data, legal data, etc


▲ Professor Jungwoo Lee (left) and Researcher Hyeonggeun Han (right) of the Department of Electrical and Computer Engineering at Seoul National University

Seoul National University College of Engineering announced that Professor Jungwoo Lee’s research team in the Department of Electrical and Computer Engineering has developed an original technology that reduces bias that is prevalent in artificial intelligence (AI) training data.

HodooAI, an AI  company specialized in AI agents founded by Professor Lee, presented a paper titled “Mitigating Spurious Correlations via Disagreement Probability” at NeurIPS 2024 (Neural Information Processing Systems), an international conference on AI theory and practices held in Vancouver, Canada, from December 9 to 15. The publication of the paper at the most prestigious conference with an Impact Factor of 24 and a 25% paper acceptance rate is considered significant as it is an international recognition of the cutting-edge technology of Seoul National University's Cognitive Machine Learning Lab (CML) and HodooAI Labs.

The biggest obstacle to the practical use of AI is the unfairness and opacity of AI judgment due to data bias. Even chatGPT, which has recently gained wide popularity, is said to have this unfairness problem. For example, a bank that uses an AI model to assess the repayment potential of a loan applicant may favor applicants of a certain race or gender with higher income levels. To ensure that AI models are fairly evaluating applicants based on other legitimate reasons instead of race, gender, or age, bias in loan applicant data must first be removed. This bias in AI training data is a fundamental problem inherent in all AI technologies.

▲ A simple example of AI bias: Suppose an AI model is learning to classify cows and camels, and the images in the above configuration are used for training. (Cows or camels in a desert or pasture background) Generally speaking, cows and camels live in pastures and deserts, respectively, so the images marked with red dotted lines in the figure above are relatively few in number. This can cause problems for AI models to predict the type of animal based on the background alone, such as incorrectly predicting a camel in a pasture as a cow.

In response, Professor Lee's research team developed an AI bias mitigation technology that trains AI models to make correct predictions regardless of the presence or absence of spurious correlations in training data. This breakthrough technology can be used to improve the generalization ability of AI models to identify the core features of data rather than the superficial features. In particular, the algorithm proposed in the paper demonstrated high performance and reliability, with an accuracy improvement of about 21% compared to existing AI models.

To develop the technique, the researchers first created an intentionally biased AI model, then used it to find the probability of no spurious correlations for every training data sample. The training data was then resampled according to the probability of no spurious correlations, and the biased model was trained again. “By making the training data more likely to be free of spurious correlations, the AI model gradually becomes less reliant on spurious correlations,” the researchers explained.


▲ Pseudocode representing the main steps of the AI bias reduction algorithm

This bias reduction technology, which reduces the AI model's reliance on spurious correlations and enables the model to accurately capture key factors and use them as the basis for judgment, has a wide range of universality. Therefore, it can be widely used in the future as a technique to remove bias in imaging, medical, legal, and financial data. In particular, it is expected to contribute to rapid and accurate medical treatment when utilized in the medical field. In addition, this patented technology will be used as a core technology to create a 'bias removal AI engine' on HdooAI's AI platform in the future.

“This technology will play a big role in raising Korea's technology for AI bias removal to a world-class level,” said Hyeonggeun Han (first author), a researcher at HodooAI. ”We hope that AI bias reduction technology will be actively researched regardless of the type of model and data, so that AI models with higher reliability and stability can be used safely around the world.

“We are pleased that our paper was accepted at NeurIPS 2024, the most prestigious AI conference in the world,” said Professor Jungwoo Lee, who led the research, adding, ”We will contribute to raising the technical level of Korean AI academia and startups by developing more innovative technologies.”

Professor Jungwoo Lee of Seoul National University's Department of Electrical and Computer Engineering, one of South Korea's leading experts on Trustworthy AI technology, founded HodooAI, an AI automatic learning platform for AI service companies, in 2017 with five of his lab students. Hyeonggeun Han, Hyungjun Joo, and Sehwan Kim, who co-authored the paper, are currently working on Responsible AI and plan to pursue further research in academia or work in global corporate research centers.


[Reference Materials]
“Mitigating Spurious Correlations via Disagreement Probability”, NeurIPS 2024
https://arxiv.org/pdf/2411.01757v1

[Contact Information]
Professor Jungwoo Lee, Department of Electrical and Computer Engineering, Seoul National University / +82-2-880-1754 / junglee@snu.ac.kr