The problem
The office wanted two things: a modern upgrade to how it models property values, and its first real use of AI. I found a vendor whose valuation and appeal-support tools looked like a good fit and brought them to the Property Appraiser. He asked me to run the project. I had the most machine learning and AI background in the office.
My role
I ran it end to end:
- Weekly meetings with the vendor and with our internal team.
- Data: coordinated cleaning, mapping, and extraction, and wrote the specs for the sales and assessment data we sent.
- Testing: organized all testing of the vendor’s results.
- The go-between: I turned our findings, requests, and issues into clear asks for the vendor.
- Fix-or-accept calls: I decided what could realistically be fixed and what we would live with.
The hardest part: alignment
Their model vs. ours. The vendor’s approach to modeling didn’t line up neatly with ours, so we had to agree on what each field meant and where every piece of data came from. That took a lot of trial and error.
What the team needed. The other half was working out exactly what my own team needed. Appraisers know a wrong value when they see one, but turning that instinct into a requirement a vendor can build to is hard. My job was to pin it down, ask for it, and then check whether it came back.
Setting the bar
We started with statistical requirements, the standard measures of how close values are to sale prices. The team pushed it further: every value had to be defensible. If an owner appeals, an appraiser has to explain the number. That applied to the AI tools too. A model that is accurate on average but can’t explain a single parcel wasn’t good enough.
That became the core requirement. It’s the same question any organization faces when it brings AI into decisions that affect people.
How we tested
I split the testing by specialty. Each group checked the part of the system closest to its own work, so the people who knew land, buildings, or data quality best were the ones looking at it. The findings came back to me, I sorted out what mattered, and I took it to the vendor. Validation caught statistical inconsistencies, land value anomalies, and data quality issues before anything reached production.
Results
- The office’s first structured evaluation of AI valuation tools, judged against a clear standard.
- Issues found and documented before anything went to production.
- The office decided not to continue past the pilot. Testing whether a tool fits before committing to it is part of what an evaluation is for.
- It showed me what the office actually needed, and that I could build it myself. That led directly to Resonance.
What I learned
- Spend most of the time on scope and planning. Most of the problems traced back to things we hadn’t pinned down at the start.
- Alignment is everything with a vendor. If you’re not on the same page, the project fails, even if the product could have been right for you.
- “Defensible” is a better bar than “accurate.” For AI in public-facing decisions, being able to explain each answer matters as much as getting the average right.
- AI for the busywork. I used AI to help draft emails and clean data. The project itself ran on judgment and teamwork.