Generative AI
Follow


John D'Errico

Should the AI apply to all questions posted on Answers?

John D'Errico on 14 Nov 2023 (Edited on 14 Nov 2023)
Latest activity Edit by Matt J on 15 Dec 2023

I saw this post on Answers.
https://www.mathworks.com/matlabcentral/answers/2046770-explaining-a-matlab-code/?s_tid=ans_lp_feed_leaf
I was impressed at the capability of the AI, as I have been at other times when I posed a question to it, at least some of the time. So much so that I wondered...
What if the AI were automatically applied to EVERY question on Answers? Would that be a good or bad thing? For example, suppose the AI automatically offers an answer to every question as soon as it gets posted? Of course, users would still be allowed to post their own, possibly better answers. But would it tend to disincentivise individuals from ansering questions?
Perhaps as bad, would it push Answers into the mode of a homework solving forum? Since if every homework question gets a possibly pretty good automatic AI generated solution, then every student will just post all HW questions, and the forum would quickly become overwhelmed.
I suppose one idea could be to set up the AI to post an answer to all un-answered questions that are at least one month old. Then students would not gain by posting their homework.

4 Comments

Matt J
Matt J on 14 Dec 2023 (Edited on 15 Dec 2023)
Maybe someday, but not today. The quality of its answers is insufficiently reliable.
Also, I do share the concern of AI being a do-my-homework engine, though I don't see much hope of policing that. Students determined to have their homework done for them will just take their business elsewhere, e.g., to ChatGPT.
ME
ME on 27 Nov 2023 (Edited on 28 Nov 2023)
I've been having a play around with the AI chat playground and at present I don't think there would be much use in applying it to all questions posted on answers. I have two main reasons for this answer:
(1) The few times I have tried to get it to do a relatively simple task, it usually fails miserably the first few times. I've then had to provide several rounds of clarification and stating what failed to get a halfway reasonable solution to a very simple question.
(2) My main issue is that it will quite happily say you can do things that are actually not possible. For example, one of my sample tasks was trying to get it to plot something with a colour that depends on the value of the z coordinate - which there are a number of questions about on answers. The AI kept stating that it was possible to do this with a single line item (which it is not!). I even directly asked the question, "can I have a single line item with a colour that varies according to its z value?" to which the response was - yes this is possible, followed by a code segment that threw up an error because it was trying to do something that isn't possible (or at least lots of people online have said it isn't possible and the AI definitely wasn't able to).
So to sum up, my opinion is that this would be a really cool idea - but only once the AI is more reliably able to come up with a reasonable working answer on the first pass.
Chen Lin
Chen Lin on 16 Nov 2023
Thanks for sharing your thoughts on Gen AI. Students are already heavily leveraging ChatGPT or other tools for homework questions. We noticed a significant drop of homework questions in Answers.
David
David on 14 Nov 2023
I agree with how you're thinking @John D'Errico. I'm part of the MATLAB Answers development team and we have been thinking about different use cases a Large Language Model (LLM) could fill across different content in Answers.
Both Quora and StackOverlow (Q/A forums) are leveraging LLMs which will be interesting to see play out. I personally like the idea of a LLM/AI posted answer or comment to questions. I do think a LLM could provide a robust and completely correct answer eventually. This approach could be helped if we automatically ran the code to help determine if there are any errors in the LLM output before posting anything.

See Also

Tags

No tags entered yet.