Key Points

• 1This paper introduces LLM-in-the-Loop (LITL) as a novel framework to address the limitations and cognitive burden associated with traditional Human-in-the-Loop (HITL) processes for large language models.
• 2LITL employs an ensemble of diverse LLMs to simulate human decision-making, identify critical junctures, and generate explanations, effectively automating and scaling tasks typically requiring human oversight.
• 3This approach significantly reduces human cognitive load and enhances the scalability and reliability of LLM applications by integrating advanced AI reasoning directly into the feedback loop.