Learn
Do AI Trading Bots Work? Four Questions Before You Trust One
At a glance
Do AI trading bots work? The question hides four different ones. Does the bot trade at all, or does the provider just hold your money? Where does the AI sit, and can a backtest contain it? Was the result produced on prices the model could not have seen in training? And does the result beat a coin flip with the same exposure after costs? In our own tests, an EA that asks an online AI service got no answer in the MT5 Strategy Tester (11 of 11 requests failed), and in a gold test a coin-flip EA made a profit in 8 of 10 runs. We have not yet run a test of ChatGPT or Claude making trading decisions; this guide explains how that test has to be built and what claims to treat as unproven until it is.
On this page
“Do AI trading bots work?” is four questions in one, and a bot can pass the first and fail the fourth. Whether a bot works depends on whether it trades at all, where its AI sits, whether the evidence was produced on data the AI could not have seen, and whether the result is better than luck.
This guide sorts the claims you meet into those four questions and shows which evidence answers each. It does not review a named bot, and it does not report a result for ChatGPT or Claude as a trader: we have not run that test yet. How it has to be built, and why most public tests do not meet it, is covered below.
Disclosure: RoboXpert sells its own Expert Advisors and a risk tool. None of our EAs uses AI in its trading decisions, so we have a commercial interest in the subject. The two tests we report are our own and dated. AI assisted with research and drafting. See our editorial disclosure.
Four questions that decide whether an AI trading bot works
Scroll horizontally if needed
| # | Question | What to look for | Evidence that answers it |
|---|---|---|---|
| 1 | Does it trade, and who holds the money? | Software on your own broker account, not a deposit with the provider | Registration check, withdrawal terms |
| 2 | Where does the AI sit? | Fixed rules, a model inside the EA, or an online service | Description, tester behaviour, live record |
| 3 | Was it tested after the model's cutoff? | Prices the model could not have seen in training | Test period against the documented knowledge cutoff |
| 4 | Does it beat a coin flip after costs? | A result outside what luck produces, with the number of trades stated | Benchmark runs on the same period and costs |
What “work” can mean for an AI trading bot
Three meanings get mixed up. The bot works technically: it places orders without errors. It works commercially: it makes more than it costs, including spread, commission and any subscription or API fees. It works as claimed: it does what the seller said, with the AI where the seller said it was. Only the second is what most readers care about, and it is the hardest to show. The first can be checked in an afternoon.
No bot works in the sense of removing risk. The U.S. Commodity Futures Trading Commission writes in its customer advisory: “AI technology can't predict the future or sudden market changes” (CFTC).
Question 1: does the bot trade, and who holds the money?
Before the AI matters, find out where your money sits. Software that runs on your own broker account can at worst lose that account. An offer that asks you to deposit with the provider is a different thing. In three cases the SEC filed in 2026, investors sent about $27.6 million to the provider; according to the complaints no AI bot traded, the profits shown were invented and withdrawals were blocked. We summarize those cases and the registration check in Is AI trading legit?. If an offer fails this question, the other three do not matter.
Question 2: where does the AI sit?
An EA or bot sold as AI can make its decisions in three places: in fixed rules, in a trained model inside the EA, or in an AI service on the internet such as ChatGPT, Claude or the seller's server. The third kind has a consequence that most sellers do not mention: it cannot be backtested as it trades.
We checked this in MetaTrader 5. A small test EA that sends a web request once a day, as an EA with an online AI would, was run in the Strategy Tester on 31 October 2026. All 11 requests failed immediately with error 4014. The MQL5 reference says that WebRequest “cannot be executed in the Strategy Tester” (MQL5). A backtest of such a bot therefore shows its fallback rules or nothing, not the AI. The full log and a count of 38 AI-named MQL5 Market listings are in AI Expert Advisors: What the AI Does and Why Your Backtest Can't Show It.
This is why testing a language model as a trader takes a different set-up from an ordinary backtest: the decisions have to be collected first, outside the tester, and replayed afterwards.
Question 3: was it tested after the model's cutoff?
A language model has read a large part of the internet up to a date called its knowledge cutoff. If you ask it what to do with gold on a day before that date and show it the real prices of that day, it may not be forecasting. It may be remembering. Research papers on this problem describe it as look-ahead bias, and a Quantpedia summary reports that language models can reproduce pre-cutoff prices from memory. We list both sources, but did not re-open them for this guide; treat the details as unchecked until you have read them.
The consequence for any test is simple. A test of a language model on dates before its cutoff does not show whether it can trade. It can show how well it remembers. A valid test uses a period that starts after the cutoff of every model in it, or hides the symbol and dates so the model cannot recognise the data, or both. Even then, a model that recognises a chart pattern as “gold in 2020” has leaked, so a careful test also runs the same prompt twice on an old period, once with names and dates and once anonymised. A difference between the two is a direct measure of the leak.
A bot that is not a language model, such as a small neural network trained by the seller, has a related problem: if it was trained and tested on the same years, the test shows memory of those years. Ask which period trained the model and which period the backtest covers.
Question 4: does it beat a coin flip after costs?
A profit alone proves little. In our MT5 backtest of a coin-flip EA on gold, from January 2024 to September 2026, 8 of 10 runs ended with a profit and the best made 18,393 USD on a 10,000 USD deposit. Of 10,000 further simulated coin-flip sequences, 44% ended with a profit and 35% lost the whole deposit (Gold EA benchmark). Gold rose strongly in that period, so luck had a tailwind.
So the question for an AI bot is not whether it made money, but how it compares with what a coin flip with the same exposure, the same costs and the same period would have made. The comparison has three parts:
- A coin-flip distribution: many random runs, not one. Where does the bot fall in it?
- Buy and hold: what a position held through the period would have made.
- A simple disclosed rule: one transparent rule on the same data, for example a previous-day breakout.
The number of decisions limits what any result can say. A test with a few dozen decisions cannot separate skill from luck, however good it looks. If the sample is thin, extend the period or the markets and state the count. Costs have to be included, with one fixed spread over the whole period; our cost-sensitivity guide shows how to find the cost level at which an average result reaches zero. For a language model, API fees and response delay belong in the cost as well.
What public tests of AI models trading show so far
Public tests exist, mostly as videos and blog posts. In our search on 31 October 2026 we found tests of ChatGPT on forex accounts of $100 to $10,000, over days or weeks, usually with no stated costs, no number of decisions and no control for the model's knowledge cutoff. Search engine summaries of that material said that none of it is peer-reviewed and that no large independent study of ChatGPT and forex was found. That is the state of our search on that date, not a finding about all public work.
One larger public experiment, Alpha Arena by nof1, let six language models trade crypto perpetual futures with $10,000 each; its first season is reported to have ended on 3 November 2025, with four of the six models losing and the best, Qwen3-Max, at +22.3%, and with the organiser itself saying it applied no statistical controls. We took these figures from secondary reports and did not verify them. They are an example of the kind of evidence to read cautiously: one period, six models, a market that moves on news, and no coin-flip comparison.
How we would test Claude and GPT
We have not run this test. The design below is what we would do, and what you can ask of anyone who claims a result.
- Period after the cutoff. Start after the documented cutoff of the latest model in the test, and say which cutoff that is.
- Several markets and enough decisions. One decision at each H4 bar close, on gold, the Japanese yen and the euro. About 500 decisions per market over four months; a daily decision would give too few.
- Hidden identity. The last 100 bars as an index with symbol and dates removed, and a fixed prompt that allows long, short or flat.
- Memorization control. The same prompt on an old period, once named and dated, once anonymised.
- Consistency. Each decision asked three times; report how often the answers agree on identical input.
- Replay in the tester. Decisions are stored in a file and replayed by a small EA with real ticks, one fixed spread and commission, because a live model call cannot run inside the tester (see question 2).
- Benchmarks. 1,000 coin-flip seeds with identical exposure, buy and hold, and one simple disclosed rule. Report the position of each model within the coin-flip distribution, plus API cost and delay per decision.
- Separate labels. A historical simulation after the cutoff is not a demo forward test and not a live record.
The outcome we can honestly expect from a test of this size is one of two: no evidence of an edge after costs, or a result outside the coin-flip range for these markets and months. Neither is a profit claim. If we run it, the result will be published as a dated update to this page.
Questions to ask before you trust an AI trading bot
- Is the software on my own broker account, and who can withdraw money from it?
- Where is the trading decision made: fixed rules, a model inside the file, or an online service?
- If it is an online service: which one, what does it cost per decision, and what does the bot do when the service does not answer?
- Which period was the result produced on, and is it after the model's documented knowledge cutoff?
- How many decisions or trades support the result?
- How does it compare with a coin flip, buy and hold and a simple rule on the same period, with the same spread?
- Is the result a backtest, a historical simulation, a demo forward test or a live record? Our backtest versus live guide separates them.
An answer that is missing leaves the question open. It does not prove the bot does not work, but it also does not support buying it.
What we did not test
- We have not tested ChatGPT, Claude or any other language model as a trader. Nothing in this guide is a result for a model.
- We tested one thing about online AI: that a web request fails in the MT5 Strategy Tester. We did not buy or run a third-party AI bot.
- The coin-flip figures come from one gold test with one lot size and period; they say how often luck wins there, not for other markets.
- The research papers, the Quantpedia article and the Alpha Arena reports were not re-opened or re-checked for this guide.
- We did not measure how many AI trading bots work. This guide does not recommend an investment.
Frequently asked questions
Can ChatGPT trade for you?
It can produce a trade idea from text or prices you give it, and a bot can be built that sends those prompts to it. Whether the ideas make money after costs is not shown by the videos and blog posts we found, and we have not tested it ourselves. A bot that asks ChatGPT for each decision cannot be backtested in MT5 as it trades.
Do AI trading bots make money?
Some bots make money in some periods, as a coin flip also does in some periods. A single profitable stretch does not answer the question; the comparison with luck, the costs and the number of decisions do.
How can I tell whether an AI trading bot is a scam?
Start with who holds the money. The 2026 SEC cases we cite involved providers that took deposits; see Is AI trading legit? for the warning signs and the registration check.
Is a backtest of an AI EA reliable?
Only for the part it contains. If the AI sits on a server, the backtest leaves it out. If a model sits inside the EA, the backtest runs the model, but only evidence from after the model's training period says anything about the future. See AI Expert Advisors.
Scope and sources
Written on 20 November 2026. The tester run and the coin-flip figures come from our own earlier tests on this site and were not re-run for this guide. The briefing for this guide listed further sources (MQL5 pages, the research papers, Quantpedia and Alpha Arena reports); we could not open them for this version, so figures taken from them are marked as unverified in the text. The planned test is a design, not a result.


