ChatGPT Struggles with Accounting Exams, Highlighting AI’s Limitations

    AI chatbot scores significantly lower than human students in accounting exams

    In a surprising turn of events, ChatGPT, an AI chatbot, has failed to outperform human students in accounting examinations. Researchers from Brigham Young University (BYU), US, and 186 other universities published their findings in the journal Issues in Accounting Education, revealing the AI chatbot’s score of 47.4 percent, compared to the students’ overall average of 76.7 percent.

    The Performance Gap

    The study found that while ChatGPT scored higher than the student average in 11.3 percent of the questions, doing particularly well on accounting information systems (AIS) and auditing, it struggled with tax, financial, and managerial assessments. One possible reason for the poor performance, according to the researchers, could be the mathematical processes required to solve such questions.

    Question Type Analysis

    According to the findings, ChatGPT demonstrated greater accuracy rates on true/false questions, scoring 68.7% correctly, and on multiple-choice questions, with 59.5% accuracy. However, its accuracy rate ranged between 28.7% and 39.1% when it came to short-answer questions, indicating some difficulties in this area.

    The researchers also observed some interesting trends, such as the AI chatbot’s struggle to understand mathematical questions, leading to illogical errors like adding two numbers in a subtraction problem or dividing numbers incorrectly. Furthermore, ChatGPT was seen providing explanations for its incorrect answers and sometimes even fabricating facts.

    Despite these shortcomings, the researchers called ChatGPT’s performance “impressive” and a “game changer that will change the way everyone teaches and learns – for the better.

    Implications for Education

    The study’s co-author and fellow BYU accounting professor Melissa Larson said, “It’s an opportunity to reflect on whether we are teaching value-added information or not. This is a disruption, and we need to assess where we go from here. Of course, I’m still going to have TAs, but this is going to force us to use them in different ways.”

    This revelation comes shortly after ChatGPT was reported to have flunked the Joint Entrance Examination (JEE) Advanced, a highly competitive engineering entrance exam in India. The AI program reportedly failed miserably after solving just 11 questions in the exam.

    AI Limitations and Future Potential

    The performance of ChatGPT in these examinations highlights the limitations of AI systems, particularly in areas requiring mathematical reasoning and comprehension. However, it also offers a chance to reflect on how these technologies can be improved and adapted to better serve educational purposes.

    In the future, AI chatbots like ChatGPT could potentially play a significant role in revolutionizing education and learning methodologies. As AI continues to advance, researchers and educators will need to collaborate to ensure that these tools are used effectively and ethically to augment human learning and understanding.

    While ChatGPT’s performance in accounting exams may have fallen short of human students, it serves as a valuable reminder of the current limitations of AI technology and the potential for future improvement.

    Disclaimer: While we make every effort to update the information, products, and services on our website and related platforms/websites, inadvertent inaccuracies, typographical errors, or delays in updating the information may occur. The material provided on this site and associated web pages is for reference and general information purposes only. In case of any inconsistencies between the information provided on this site and the respective product/service document, the details mentioned in the product/service document shall prevail. Subscribers and users are advised to seek professional advice before acting on the information contained herein. It is recommended that users make an informed decision regarding any product or service after reviewing the relevant product/service document and applicable terms and conditions. If any inconsistencies are observed, please reach out to us.

    Latest Articles

    Related Stories

    Leave A Reply

    Please enter your comment!
    Please enter your name here

    Join our newsletter and stay updated!