AI financial advice: biased, inconsistent and sometimes just wrong

Artificial intelligence tools give personal finance advice that can be inaccurate, inconsistent and demographically biased — and the results vary significantly depending on which platform a user chooses. That is the central finding of a peer-reviewed study published last month in the Journal of Financial Planning, which tested seven widely available generative AI programs on identical financial scenarios. The research arrives as a large majority of American AI users are already turning to these tools for money guidance.

Seven platforms, one test — and widely different answers on retirement and savings

Researchers at the University of Georgia and the University of Rome Tor Vergata queried seven generative AI platforms in August 2025 with an identical set of prompts covering three core personal finance scenarios: emergency savings, the optimal withdrawal rate from a retirement portfolio, and the recommended composition of an investment portfolio.

The platforms assessed were the free-access versions of ChatGPT, Claude, Copilot, DeepSeek, Gemini, Meta AI and Perplexity. Despite receiving the same questions, the tools produced what the authors describe as «substantial variation in guidance» — particularly around emergency savings and asset allocation.

While the platforms generally aligned on broad principles — such as the widely cited 4 percent retirement withdrawal rule — the differences in specific recommendations were significant enough to raise serious concerns about reliability. «Although the tools often produced recommendations that broadly aligned with generic financial planning principles, such as the 4 percent retirement withdrawal rule, there were significant differences across platforms in suggested emergency savings and portfolio allocations,» the authors wrote.

Confident-sounding answers that can still be incomplete, misleading or wrong

One of the sharpest warnings in the study concerns the tone AI tools adopt when answering financial questions. «GenAI-driven responses may sound confident but can still be incomplete, misleading, or incorrect,» the paper states — a combination that researchers argue is particularly dangerous for users who may not have the background to detect errors.

Andrew Lo, director of MIT’s Laboratory for Financial Engineering and principal investigator at its Computer Science and Artificial Intelligence Lab, put it plainly in an interview with CNBC in March. «One of the things about LLMs that I find particularly concerning is that no matter what you ask it, it’ll always come back with an answer that sounds authoritative, even if it’s not,» he said.

Lo identified the highest-risk use case: personalized calculations. «When it comes to very, very specific calculations of your own personal situation, that’s where you have to be very, very careful,» he said. This limitation is compounded by the well-documented problem of algorithmic «hallucination,» where AI models generate plausible-sounding but factually incorrect outputs. AI tools are also sensitive to how prompts are phrased, meaning small differences in how a user words a question can lead to meaningfully different — and potentially conflicting — recommendations.

Demographic bias: recommendations shifted when race and gender changed

Beyond accuracy, the study uncovered a fairness problem. Researchers ran the same financial scenarios a second time, but altered the race and gender of the hypothetical individual in the prompt. The AI platforms’ recommendations changed in response — a finding the authors say raises questions «about the consistency and fairness of GenAI-driven recommendations.»

Suggested Posts

Social Security may pay only 78% of benefits by 2032 — senators want a vote

A bipartisan group of six U.S. senators introduced legislation Wednesday to compel Congress to act on Social Security reform before the program's trust fund…