This is interesting empirical evidence that the weakness of LLMs in law is fundamentally about a poorly digitized legal and jurisprudential corpus.
Corey Rayburn YungI think that's because all of the cases are from SCOTUS and there is a lot written about them. I can still tell the answers were written by an LLM (because of the types of mistakes made), but my last experiment generated answers that were in range of the lowest grade in the class. 2/2