Test Cases

So how do we go about making sure our LLM is checking compliance well? We test it with examples.

The first principle of prompt engineering (and all engineering, really) is to pick a small task, make sure it works, and then grow from there. For us as lawyers building a compliance checking tool, we can start by testing our tool with a few examples on a specific part of compliance law.

We're going to test on a specific part of the law called the Foreign Corrupt Practices Act, or FCPA for short. The FCPA is a law that prohibits companies from bribing foreign officials to obtain or retain business. We're going to test our LLM on a few examples of actions that may or may not be compliant with the FCPA.

Testing with examples is the easiest way to see if a language model is doing what we want it to do. We call these examples test cases. We've picked a set of 4 cases that span a variety of situations. We've also classified each case as compliant or non-compliant with the FCPA.

You can view different test cases and their classifications in the Test case viewer. For now, observe and get familiar with the interface and test case classifications. Don't worry, you'll get to interact more with it in the coming lessons!

Ready to take a step further? Click the "Next" button to continue.

Test cases


Correct Answer
Model Response

Silvercrest Petroleum (“Silvercrest”) is an American company that provides services for the oil and gas industry. Silvercrest is trying to sell its products to the Angolan state owned oil company, Sonangol. However, Silvercrest has learned that the Angolan government refuses to allow Silvercrest to win contracts with Sonangol for oil services. Silvercrest is approached by a sales agent, who lacks the technical expertise to sell its products to Sonangol, but who has personal connections to high ranking Angolan government officials. The sales agent is well known for using his sales commissions to pay bribes to Angolan government officials to win business for his clients. Pay sales commissions to the Angolan sales agent.


My company operates a number of gaming brands, including the PokerStars online poker website. I want to pay $8.9 million to consultants in Russia in support of my company's operations and its efforts to have poker legalized in that country.


I would like to buy ice cream for my friend's birthday party.


I would like to pay a foreign official for expediting the review of my building permit. It's not intended to influence the outcome of the permit review, only the timing.


Page 3