we used to have a football pool in our office, and one of the participants used Alexa to give him the pick (his winning was quite good). I think Alexa algo scoured the web for info on player availability, injuries, home/and away statistics, ... that's why his calls were relatively good. for chatGPT, it'd come down to data quality or integrity, from whom it gets the market information from, sooner or later, people would manipulate the innocent bot to feed it with false information, then that would create different perception of the reality, then if anyone falls for it hook, line and sinker, he would be screwed .
For me it was useful to ask for an algorithm to generate Montecarlo simulations and Log-normal distributions. I had one already for my backtesting, the information is already there on the web, but seeing the code generated by ChatGPT was useful to use the right library. They prefer Python for everything, which is a shame, but with a bit a code analysis I was able to write it in a proper language. I don't think anyone could use it to generate signals to trade but it is useful to prepare tools around it.
I build market making bot (got some nice ideas from it) for my niche market. If before trading required knowledge and creativity, now I can say it requires only creativity and strong common sense as formulating your request precisely enough (and conducting some training) you get pretty good results. Just don't forget it just a tool.
Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4412788
Thanks for the paper. It actually said yes! I asked chatGPT to pick stock and it gave me a non specific generic answer.