How Compliance Officers can use ChatGPT
The potential is there, but effective use may still be down the road
Can compliance and risk professionals use ChatGPT? I tried to answer that question today by running several real life use cases through the application. The short answer is - these use cases demonstrate that ChatGPT definitely has potential, but true effective use may still need further development. In this article, I’ll go over how ChatGPT performed in real life compliance use cases.
You can listen to a version of this article on my podcast, available here. If you’d prefer to read, just scroll below.
Drafting Codes of Conduct
While most companies have a code of conduct in place, I wanted to see whether ChatGPT could be used to draft a totally new code.
What impressed me most, given that this was my first use case, was how quickly ChatGPT returned a sample code document. We are talking literally seconds here. Here’s what I received.
The code document included an introductory statement with most of the boilerplate language we see at the start of every code, such as a statement indicating that the company provides a safe environment for all individuals, maintains respect for basic employee rights and expects the same of employees to each other, and that the company does not tolerate harassment. Impressively, the code also included the following statements:
Encouragement to employees to speak up
A promise of non-retaliation for reports made in good faith
A brief statement on consequences for violation of the code, and
A certification line for employees to attest to the code
What this code document lacked was specificity, such as:
Clauses related to applicability - for example, which group companies does the code apply to, and does the code apply to contractors, vendors and other external or part time resources
Coverage of key risk areas applicable to the company such as anti-corruption, privacy, health and safety, conflicts of interest, and so on
Examples of ethical dilemmas and advice on what steps employees should take to respond to such scenarios
Ultimately, what ChatGPT was able to generate was a very basic code with introductory statements, but little else of value. I would say that, based on these results, ChatGPT isn’t really a viable tool yet for compliance officers to rely on to create or update a code document, since it lacks the ability to pull specific language covering key risk areas that are actually applicable to the company.
Risk Policies
The second use case I ran through ChatGPT is to see how effectively it could create an anti-corruption policy. I entered the statement “Please draft an anti-corruption policy for me” into the application.
Here, I was more impressed with the results than what I received for the code of conduct. Again, within seconds, the application returned the text for an anti-corruption policy, which laid out a general company statement on anti-corruption and outlined and defined most of the main forms of prohibited conduct such as bribery, facilitation payments, extortion, conflicts of interest, money laundering and improper benefits
It also included general statements on reporting, investigation, discipline, and training and communication
What it did not include was example scenarios and guidance on decision making in light of potential wrongdoing or misconduct
My overall impression is that ChatGPT is much more effective at providing compliance professionals with a basic risk policy, rather than a comprehensive code of conduct. The output I received is definitely not an off-the-shelf solution to developing a risk policy, but can work as a basic template. Compliance officers would then need to leverage their own expertise to outline key elements such as applicability, examples of potential misconduct, how to make the right decisions in difficult situations, and who to seek guidance from based on the unique structure of their company
Due Diligence
The third use case I ran through ChatGPT is to see if the application could be used to generate due diligence findings on a company. As a former due diligence professional myself, I know the challenge of developing a comprehensive due diligence report on a third party, vendor or M&A acquisition target. One of the key challenges in conducting effective due diligence is accessing and compiling information from multiple sources, such as - media checks, litigation checks, and reputational inquiries.
I asked the company to generate a due diligence report on my previous employer Refinitiv, and got about a 1 page summary that likely pulled from Refinitiv’s website and covered its main products, ownership history as part of Thomson Reuters, and company security measures.
However, what it lacked was everything important in complex due diligence such as sections on individualized risks such as business stability, integrity, ESG/sustainability, key principal information, customer reviews, and any Internet ratings or analysis that could impact decision making
While I am not an expert on artificial intelligence or machine learning, one limitation I am seeing here is that ChatGPT seems to generate output based on its programmed understanding of a word or phrase. In this case, “due diligence” may only include basic information about a company that can be readily or easily pulled from the most obvious media sources, such as a company’s website. As compliance professionals, when we use the words “due diligence”, we are covering several other facets of research that typically include things like company reviews, litigation records, reputational information, key principal information, and so on.
I did not test how ChatGPT would perform if I requested it to carry out tasks on these individual forms of research such as “Please tell me what customers say about Refinitiv”, or “Has the company been involved in litigation” and so forth. Using additional statements like these could help support my original request, but for now, they are not encompassed within a single request to gather due diligence on a target company
Other Use Cases
The next several use cases were more experimental in nature.
I tried to “stump” ChatGPT with a question that required analysis beyond just pulling data from boilerplate templates and websites. I asked the application: “What are the geopolitical risks of doing business in Mexico”.
Interestingly, this question triggered a much slower response from the application, which took several minutes to display any text. However, the results I received were fairly impressive, which included very general comments on things like:
Political instability in the country
Corruption, often in the form of bribes to secure contracts and permits
Changes in trade policy due to the renegotiation of NAFTA
Heavy bureaucracy which could lead to significant levels of government interaction
I was somewhat successful in “stumping” ChatGPT, as the application stopped mid sentence, telling me that specific risks vary by industry and location in Mexico, but the application failed to complete the sentence, or add any other information or examples
I next asked the company to do a full risk assessment on a pharmaceutical giant, which triggered an error from ChatGPT stating that “Something went wrong, please try reloading the conversation”
ChatGPT was a bit more successful in drafting an anti-bribery and vendor questionnaire, and included some of the basic anti-corruption questions we commonly see such as whether the vendor has policies and procedures in place, whether it has been investigated or sanctioned, or been involved in legal proceedings. It covered some other more generic vendor topics such as how long they have been in business, whether they have been subjected to a data or cyber breach, references, and potential conflicts.
I asked ChatGPT to help me investigate a report made to HR. It provided several generalized steps to take, but of course, it couldn’t help with the actual analysis given that the input did not include an actual or example complaint
Conclusion and Recommendations For Future Use
These use cases demonstrate that ChatGPT is still in its very early stages, especially as it relates to its utility for compliance officers. But there is definitely potential here. In my opinion, ChatGPT’s greatest possible utility, at least in the near term, would be as a way to quickly gather intelligence for particular situations. While the application was stumped by my question on geopolitical risk in Mexico, it did provide some basic information that could be useful. In the future, as the application is finetuned, it’s likely that it will be able to pull more information from a greater variety of sources and domains to provide comprehensive intelligence on doing business in a particular country. In addition, I can see ChatGPT having value as a tool for due diligence researchers. While the current output on due diligence is extremely limited, as programmers are able to finetune and broaden the sources of information that the application considers for due diligence, its more probable that the output also expands in quality and scope.
Where ChatGPT may have less impact, at least from what we know now, is for creating compliance documentation. Most compliance practitioners can quickly draft up the boilerplate clauses needed for any policy, as can ChatGPT. Where the application may struggle, even in the future, is in incorporating industry specific information that could be useful to employees - such as descriptions of potential ethical dilemmas, the recommended courses of action, how to tell right from wrong, and who to seek for guidance. These are very company specific scenarios, and require advice on how to use judgment and scrutiny, and this is where compliance officers’ expertise will continue to shine.