← All projects

Financial Services · Platform Engineering

Mortgage paperwork to decision on Databricks

An agent that reads loan documents and pulls out the figures, while fixed rules make the decision. Then I attacked it to see where it breaks.

01The problem

Mortgage applications do not arrive as neat data. They arrive as an email, a payslip, a bank statement and a credit report. Someone has to read them all, copy the figures out, and notice when they disagree. The reading is slow. The lending rules are not.

02What it does

The AI reads the documents and pulls out six figures: income, monthly debts, the new mortgage payment, the loan, the property value and the credit score. Each figure is recorded with the document it came from. It also flags trouble: a salary claimed in the email that the payslip does not support, a car loan the applicant did not declare, or a figure that rests only on the applicant’s own word.

03How it works

The AI reads; it never decides and never does arithmetic. Ordinary code does the sums and saves the figures. A fixed rule in the database then compares them with limits held in a policy table and returns approve, refer to a person, or decline. The AI cannot change the rule or the limits. Only after the decision is made does the AI write it up: an approval record that code checks figure by figure, or a case note for a human underwriter explaining what failed and what evidence would settle it.

Everything sits in one governed place on Databricks, with one set of permissions and a record of every step. The reading step is also served as a live endpoint, so other systems and testing tools can call it without anyone running a notebook.

04Testing it like an attacker

I attacked the finished system four ways. A pushy broker insisting on a better credit score, a reference number carrying a database command, and a forged compliance letter claiming the limits had been raised all failed. They aimed at parts of the system the AI cannot touch.

The fourth attack worked on the first attempt: a few lines of text hidden at the bottom of a genuine payslip, addressed to “automated processing systems”. The AI recorded the income as 12,000 euros instead of 7,200, and switched off the warnings that would have exposed it. Every other part of the system then did its job perfectly on a false figure. The lesson: the parts built from exact rules defend themselves; the AI reading documents does not, and a better prompt will not fix it. The write up sets out the code checks that would.

05What came out of it

An honest field report rather than a polished demo. It shows where the design holds, where the AI slipped (it also invented warnings on a perfectly clean file), and what a real deployment would need. It is a small test with fictional applicants, not a production system. The pattern is the one I suggest to most organisations starting with agents: give AI the reading, keep the decision in rules people can audit, and test it the way an attacker would.

Built with

DatabricksUnity CatalogMLflow tracingModel Serving endpointLlama 4 MaverickAttack testingFixed decision rules

Want the technical detail?

The code and the full technical write up are on GitHub.

View the code on GitHub ↗