Recently, the legal community has been abuzz with discussions about the integration of artificial intelligence in professional exam settings. Notably, a petition to the California Supreme Court reveals that a California State Bar test consultant utilized OpenAI’s ChatGPT to craft 29 out of 200 multiple choice questions for the February Bar Exam.
The involvement of OpenAI’s ChatGPT was facilitated by ACS Ventures Inc., the firm engaged by the State Bar for exam analysis and scoring. As noted in the petition, ACS Ventures Inc. was tasked with creating prompts that would generate multiple-choice questions aligning with specific topic areas determined by admissions staff. These prompts were subsequently run through ChatGPT to achieve the desired outcomes.
According to reports, the submission of this particular petition was delayed by a day. The reason behind this delay was the California Supreme Court’s request for clarification regarding the extent of AI’s involvement in the examination process, indicating heightened judicial scrutiny over the use of AI-driven methods in significant evaluation systems like the Bar Exam.
This case underscores the emerging complexities and considerations surrounding the integration of artificial intelligence in the legal domain. As legal entities increasingly adopt AI tools for various functions, there is an ever-growing need for transparent practices, thorough evaluation, and regulation to ensure they meet professional and ethical standards.