PCMS Member Corner: Review of Systems and AI

While bureaucracy and paperwork continue to increase, there are a few items of good news.

The first and easiest way to decrease paperwork is to join the cohort of clinicians who have abandoned the review of systems (ROS). It has been several years since it was removed as part of necessary documentation, with good reason. The ROS rarely adds any significant information that cannot be asked as part of a good history and it makes the patient think that you are looking for more problems.

The other interesting area that may improve how we approach medical paperwork is artificial intelligence. There are numerous AI programs out there, each specializing in different areas. To get an idea of what is possible, I used AI Simplified, which calls itself a “text generator and AI copywriting assistant.”

Using the “paragraph writer” setting, I asked it for a “letter of recommendation for an average student who wants to enter ortho residency,” and it generated the following response:

It is with great pleasure that we write this letter of recommendation for ____________, who is applying for an orthopedics residency program. ___________ is a highly intelligent and hard working medical student who has consistently shown excellent grades and a dedication to the field of orthopedics. While ___________ may not be the top of their class, they are a very competent medical student who has a great deal of potential. We believe that they would be an excellent orthopedics resident and would be a valuable asset to any program. Thank you for your time and consideration.

Next, I used the prompt for “letter of recommendation for a bad student who wants to enter orthopedics residency,” and it came back with:

It is with great reluctance that I write this letter of recommendation for John Smith, who is applying for an orthopedics residency. Mr. Smith was a mediocre medical student at best, and I strongly believe that he is not prepared for the rigors of a residency program. Additionally, Mr. Smith has a history of poor personal conduct, including several incidents of unprofessional behavior. Taking all of these factors into consideration, I do not believe that Mr. Smith would be an asset to your residency program, and I therefore cannot in good conscience recommend him for admission.

I found this second AI-generated response interesting as it did not write a letter in any way recommending the theoretical student and, secondly, invented the “poor personal conduct” comment.

This comes to the core of what AI is currently capable: it can look good grammatically and even
stylistically, but much of its content may not be factual but triggered by key words.

Finally, I asked AI Simplified to write a “letter from an insurance company denying the use of vancomycin for pseudomembranous colitis.”

We are sorry to inform you that we are unable to approve your request for oral vancomycin for pseudomembranous colitis. After reviewing your medical records, our independent review board has determined that this treatment is not medically necessary. We understand that this may be disappointing, but we hope that you will understand our decision.

Again, the AI came up with a “reasonable” sounding answer but it independently fabricated the part about the independent review board.

I believe artificial intelligence has the potential to make our work less cumbersome, but I can also imagine a dystopian future where Medical Office AI and Insurance Company AI end up in an unending paperwork battle over prior authorizations. Time will tell.

Written by Dr. Roy Loewenstein, PCMS Board President
March 2023