Two points via a recent Prof. Emily M. Bender post (https://dair-community.social/@emilymbender/111162430463721548)
... discussions of (AI) technology become much clearer when we replace the term AI with the word “automation."
A very key thing to keep in mind here is that the output of these (ChatGPT-like) systems don’t actually make sense. It’s that we are making sense of the output.