High-accuracy, privacy-compliant multilingual sentiment categorization on consumer-grade hardware: A monte carlo evaluation of locally deployed large language models

Abstract

This study presents a comprehensive evaluation of multilingual sentiment categorization performance using locally deployed large language models (LLMs) on consumer-grade hardware, focusing on GDPR-compliant implementation scenarios. Through extensive Monte Carlo validation involving 947,700 classifications over 702 iterations, we demonstrate significant performance capabilities across English, Italian, and Japanese languages while operating within consumer hardware constraints. Using lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half on a Python-based llama-cpp framework on consumer NVIDIA GPU hardware, English achieved 96.3% accuracy (95% CI: 0.963–0.964), with Italian and Japanese showing strong performance at 92.2% (95% CI: 0.921–0.922) and 90.7% (95% CI: 0.906–0.908) respectively. Notably, our analysis demonstrates that plurality voting can achieve extremely high confidence levels across all languages, suggesting an efficient approach to improving classification reliability without requiring extensive computational resources.