I work at the seam between finance and AI.

When I started doing RLHF work on financial reasoning, I assumed my edge would be the domain knowledge: that having actually bootstrapped a credit curve would let me spot the errors quickly. Partly true. But the real difficulty is subtler.

A model that gets a DCF wrong is easy to correct. A model that gets the number right through a chain of reasoning that would break on any adjacent problem is much harder, because the output looks perfect. You have to grade the reasoning, not the answer, and that means articulating why you personally believe something, which most finance people have never had to do explicitly.

My internships taught me the answers. This job is making me explain them.

Almost none of the finance work I've done started with a clean question. It started with a shared drive: three versions of a model, a term sheet saved as a PDF, a data room with two hundred files, and an email thread that changed the assumptions halfway through. A model that can only answer questions in a chat window can't do much with that.

That's why I think agents are the future. The useful version of AI opens the folder, works out which file is the current one, reads the email that changed the discount rate, updates the model and writes the summary. It works through the mess the way an analyst would, step by step, and keeps track of what it has already checked.

The next step is several agents working together. One builds the model, one checks the formulas, and one compares the output against the source documents and flags anything that doesn't match. A good deal team already works this way: nobody signs off their own numbers. Agents should check each other in the same way.

Getting there depends on trust. An agent has to show what it did, which files it used and where it was unsure. That's the part I care about most.

A lot of teams launch an AI tool, see a few impressive answers and move on. Then the questions change, the data changes, the model gets updated, and nobody can say whether it's doing better or worse than last month.

The fix is unglamorous and it works: decide what good looks like before you deploy. Write the criteria down. For a finance tool that means questions like these. Is the number correct? Is the method appropriate? Are the terms used the way a professional would use them? Is the document formatted to market convention? Would you send it to a client? Turn those into a rubric with clear scores, and build a set of test cases that reflect the work people actually do.

This is a large part of my daily work. I write the cases, grade the outputs against the rubric, and document the patterns behind the failures so they can be fixed. A single wrong answer matters less than a pattern, like a model that keeps mislabelling cash flows or rounds too early in a multi-step calculation.

In finance this matters more than almost anywhere. A wrong number in a valuation or a misread covenant is expensive, and it often looks perfectly plausible. Being able to evaluate an AI system, and then improve it, is becoming as important as being able to build one.