Decoding cultural tapestries: A deep dive into Indian social stigma patterns in large language models

Sridhar Jonnala (), Rushikesh Tade () and Nisha Mary Thomas ()

Journal of Asian Scientific Research, 2025, vol. 15, issue 2, 226-244

Abstract: The widespread adoption of Large Language Models (LLMs) raises critical concerns about the amplification of societal biases, especially in non-Western contexts where cultural and social nuances are often underrepresented. This study introduces a multi-agent bias detection framework to systematically evaluate GPT-4o, Claude 3.5 Sonnet, and Llama 3.3 across Indian social stigma categories, including caste, religion, gender, mental health, socio-economic status, appearance, language/region, and family dynamics. We present SocialStigmaQA, a benchmark dataset of 320 prompts, validated through expert review and pilot testing, and use the Overall Bias Detection Factor (OBDF) to measure model performance. Findings reveal that Claude 3.5 Sonnet achieved the highest OBDF (98.75%), demonstrating superior bias detection across all categories, while GPT-4o showed moderate performance (72.8%) with noticeable gaps in gender and socio-economic domains. Llama 3.3 scored the lowest (71%). The multi-agent framework enhanced detection accuracy by 25–30% over single-agent models, particularly in subtle bias areas. These results underscore the need for culturally contextualized evaluation frameworks and suggest that OBDF-like metrics should be integrated into India's AI auditing processes to ensure fairness, inclusivity, and ethical deployment of AI systems in sensitive sectors such as hiring, education, and governance.

Keywords: Bias detection; Cultural bias; Ethical AI; Indian social stigmas; Large language models; Responsible AI; AI Governance. (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://archive.aessweb.com/index.php/5003/article/view/5483/8332 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:asi:joasrj:v:15:y:2025:i:2:p:226-244:id:5483

Access Statistics for this article

More articles in Journal of Asian Scientific Research from Asian Economic and Social Society
Bibliographic data for series maintained by Robert Allen ().