As a disclaimer, I've always (that is, since 2011; but privately, I think I had already quite grasped the possibility in my mind long before, when I was a kid with no Internet) been a believer in the (imminent) possibility of superhuman AI and the Singularity and thought a lot of the implications of humanity becoming extinct or irrelevant or anyway unrecognizable. I think this was good for me, as I went through some existential crises a bit before their time.
As an example, I am a mathematician and I had thought since 2018 of the possibility of computers routinely proving advanced math results while society otherwise marches on (I admit that, before 2018, I had only thought of a "hard take-off" where the period between no AI and superintelligent AI was of the order of several hours to a month). I think this makes me better prepared mentally than my colleagues who are in various states of denial.
In the past year, I have turned more and more to literature, thanks partly to your blog (I know it sounds a bit silly but your introductory post [1] brought me a lot of warmth). I have always enjoyed books, but now I realized that the act of reading a book, and then turning it around in my mind, is something that AI, no matter how intelligent or powerful, can never take from me (except by killing or torturing me, but humans could do that already anyway).
It is possible that we'll no longer get money or prestige or status from scholarship of any sort, but as long as we manage to find ways to stay alive and healthy in the coming times (and, if we can't, then our problems are bigger than the end of academia), we can still find ways to do the activities we like.
Anyway, I use this comment to thank you very much for writing this blog (whose archives I haven't yet read in full). It was and still is for me a light in dark times.
I am reading American Prometheus. It turns out that Oppenheimer not only was a better quantum physicist than me, but he was also much better read. Among the innumerable works he read and I didn’t was a poem by a poet surnamed Herbert with the line “I will abroad”.
I asked Claude and I got the poem. The title is “The Collar” and it was written by the 17th-century English poet George Herbert. English is not my first language, I am not as well read as Oppenheimer, and the poem is hard. Again, thanks to Claude, I managed to understand its beauty and to comprehend its importance.
Gentlemen, AI is Asimov’s Encyclopedia Galactica and Captain Picard’s computer all in one. It is one of our greatest inventions and the most necessary one: we are drowning in information, we are besieged with so many truths and so many lies, that we need our own curator, our own researcher, and our own wise man.
Just to agree with the other reader. You don't understand that poem, nor do you comprehend its importance.
This is like reading the summary of a song and being told what key it was in and you claim you've heard that song.
Whatever knowledge or expereince you've gained from that interaction is so tenuous. But a poem you read, and re-read, and think about, and read again 5 years later becomes interwoven into your own DNA.
Chat bots can't do any of that, and in fact they obfuscate the very process of gaining knowledge.
It is probably a painful mistake to confuse difficulty with knowledge, or a chatbot’s limitations with the limitations of its user. A machine does not relieve us of judgment, verification, or thought. Nor does efficiency oppose curiosity or agency; it simply lowers the cost of locating, comparing, and connecting what already exists in the noosphere. That leaves more human attention for questions, interpretation, and experiment.
It is probably a costly mistake to believe that a tool increases one's inherent skill or doesn't mitigate one's ability to perceive Truth.
We are in this *exact* predicament because people foolishly believe your last two sentences without holding them up to scrutiny. This is the error of our culture.
Having a poem explained to you doesn't really hit the same as understanding it yourself. There's a kind of gut-level, intuitive feeling of real understanding. A good teacher can maybe subtly guide you to that feeling, but not through blunt explanation.
I'm not sure a culture that uses AI extensively will produce training data (text) for future models. To that extent, it seems like immoderate AI use will lead to stagnation.
The use you describe is a far cry from generative AI, which is what most of us are worried about. But I do hope you checked each source yourself, because LLMs *very* frequently hallucinate even information they claim has a source.
I will confess upfront I am not at all interested in using AI. But it seems to me there are a few flaws here. Yes, the digitalisation of archives and books has allowed access to the general public and it is fantastic - but by and large, this has been freely available or for a small fee and the AI you speak of is not free. The libraries are free to access, AI is not and it does not provide access, it provides you information that its systems calculate will answer your question. AI companies will - or eventually will - gatekeep the information they have - they have to in order to be economically viable. They're already destroying books. That will finally lock us out of those original sources unless we are willing to pay for information that it determines answers our question - information that was once freely available for us to discover. Most of us already have a sense that simply googling something has become much less useful over the last couple of years. Yes, it might at the moment appear quicker to find a fact with AI (it used to be a lot quicker with Google and took you to the original source, but c'est la vie), but it narrows context, it must be checked for accuracy, and it will always make mistakes. There's a reason scholars will dig and dig even after they apparently have the facts. Even our own records lie, obscure, and leave things out or word meaning alters over time etc. AI will collapse debate. "Oh, but AI said so," will ring through the halls of the universities. To the frustration of those of us who know that nothing in the past is set in stone and it's not about the answers, it's about the questions. Scholarship is not simply about finding a fact. It is about understanding and challenging those facts. And I'm sorry, but no chatbot telling me that such and such is the answer to my question will be more exciting than discovering a new angle on the question simply by digging and digging through the wealth of human knowledge that is already freely available to me. Scholarship is about changing and refining the question itself. And eventually, AI will simply continue to feed on our collective past and nothing new will come into being, because I simply don't think AI is intelligent in the way a scholar is. AI can't ask good questions.
Just to agree and organize a few of these thoughts:
1) what Henry refers to a choice of "knowing the truth or not" in using a chat bot is actually incorrect. He doesn't "know" anything when using a chat bot. No new knowledge is created. The knowledge is created in the research component. The thinking through the problem. Just because a chat bot give him a response to the question he poses does not make it correct, and I'm not just basing this on the idea of hallucinations. Knowledge is only created by human beings.
2) That process of gaining knowledge is gate kept in the way that he suggests is a problem for a reason! You have to work to earn it. Using a chat bot to "gain knowledge" in the way he suggests doesn't work. These machines are aimed at efficiency, and efficiency does not factor in human curiosity or the development of human knowledge.
3) I am sincerely concerned that the increase of usage of chat bots will lead to a decrease in the creation of knew knowledge, not more knowledge. This will, as he suggests, lead to more criticism, and less things to criticise. In short, if a chat bot can muscle its way to a solution of a math problem by super-humanly trying every possible logical option, it is the same as infinite monkeys hitting a keyboard and one of them producing *Hamlet*.
In Eco's *The Name of the Rose*, the monk Jorge of Borges wasn't right to destroy the contents of the book he was hiding, but he was right to hide it behind a mystery that only the worthy scholar could solve. Knowledge must be earned. Removing the obstacles to learning decreases human desire to learn.
Just one question (two parts): what does the term “self command” mean and what does it have to do with Jane Austen? Tell us what you learned from Claude, Henry!
As a disclaimer, I've always (that is, since 2011; but privately, I think I had already quite grasped the possibility in my mind long before, when I was a kid with no Internet) been a believer in the (imminent) possibility of superhuman AI and the Singularity and thought a lot of the implications of humanity becoming extinct or irrelevant or anyway unrecognizable. I think this was good for me, as I went through some existential crises a bit before their time.
As an example, I am a mathematician and I had thought since 2018 of the possibility of computers routinely proving advanced math results while society otherwise marches on (I admit that, before 2018, I had only thought of a "hard take-off" where the period between no AI and superintelligent AI was of the order of several hours to a month). I think this makes me better prepared mentally than my colleagues who are in various states of denial.
In the past year, I have turned more and more to literature, thanks partly to your blog (I know it sounds a bit silly but your introductory post [1] brought me a lot of warmth). I have always enjoyed books, but now I realized that the act of reading a book, and then turning it around in my mind, is something that AI, no matter how intelligent or powerful, can never take from me (except by killing or torturing me, but humans could do that already anyway).
It is possible that we'll no longer get money or prestige or status from scholarship of any sort, but as long as we manage to find ways to stay alive and healthy in the coming times (and, if we can't, then our problems are bigger than the end of academia), we can still find ways to do the activities we like.
Anyway, I use this comment to thank you very much for writing this blog (whose archives I haven't yet read in full). It was and still is for me a light in dark times.
[1] https://www.commonreader.co.uk/p/how-to-get-started-reading-english
I love this comment and agree very much. Reading Tolstoy is irreplaceable!
So why bother engaging with a chat bot, Henry?
I am reading American Prometheus. It turns out that Oppenheimer not only was a better quantum physicist than me, but he was also much better read. Among the innumerable works he read and I didn’t was a poem by a poet surnamed Herbert with the line “I will abroad”.
I asked Claude and I got the poem. The title is “The Collar” and it was written by the 17th-century English poet George Herbert. English is not my first language, I am not as well read as Oppenheimer, and the poem is hard. Again, thanks to Claude, I managed to understand its beauty and to comprehend its importance.
Gentlemen, AI is Asimov’s Encyclopedia Galactica and Captain Picard’s computer all in one. It is one of our greatest inventions and the most necessary one: we are drowning in information, we are besieged with so many truths and so many lies, that we need our own curator, our own researcher, and our own wise man.
Let us embrace this miracle.
Just to agree with the other reader. You don't understand that poem, nor do you comprehend its importance.
This is like reading the summary of a song and being told what key it was in and you claim you've heard that song.
Whatever knowledge or expereince you've gained from that interaction is so tenuous. But a poem you read, and re-read, and think about, and read again 5 years later becomes interwoven into your own DNA.
Chat bots can't do any of that, and in fact they obfuscate the very process of gaining knowledge.
It is probably a painful mistake to confuse difficulty with knowledge, or a chatbot’s limitations with the limitations of its user. A machine does not relieve us of judgment, verification, or thought. Nor does efficiency oppose curiosity or agency; it simply lowers the cost of locating, comparing, and connecting what already exists in the noosphere. That leaves more human attention for questions, interpretation, and experiment.
It is probably a costly mistake to believe that a tool increases one's inherent skill or doesn't mitigate one's ability to perceive Truth.
We are in this *exact* predicament because people foolishly believe your last two sentences without holding them up to scrutiny. This is the error of our culture.
Having a poem explained to you doesn't really hit the same as understanding it yourself. There's a kind of gut-level, intuitive feeling of real understanding. A good teacher can maybe subtly guide you to that feeling, but not through blunt explanation.
I'm not sure a culture that uses AI extensively will produce training data (text) for future models. To that extent, it seems like immoderate AI use will lead to stagnation.
Yes; the more AI text that exists the more AI will find it and it's an endless loop.
Well done. As an English ABD who morphed into a business guy, I think about these issues quite a bit. I think you're dead on.
The use you describe is a far cry from generative AI, which is what most of us are worried about. But I do hope you checked each source yourself, because LLMs *very* frequently hallucinate even information they claim has a source.
I will confess upfront I am not at all interested in using AI. But it seems to me there are a few flaws here. Yes, the digitalisation of archives and books has allowed access to the general public and it is fantastic - but by and large, this has been freely available or for a small fee and the AI you speak of is not free. The libraries are free to access, AI is not and it does not provide access, it provides you information that its systems calculate will answer your question. AI companies will - or eventually will - gatekeep the information they have - they have to in order to be economically viable. They're already destroying books. That will finally lock us out of those original sources unless we are willing to pay for information that it determines answers our question - information that was once freely available for us to discover. Most of us already have a sense that simply googling something has become much less useful over the last couple of years. Yes, it might at the moment appear quicker to find a fact with AI (it used to be a lot quicker with Google and took you to the original source, but c'est la vie), but it narrows context, it must be checked for accuracy, and it will always make mistakes. There's a reason scholars will dig and dig even after they apparently have the facts. Even our own records lie, obscure, and leave things out or word meaning alters over time etc. AI will collapse debate. "Oh, but AI said so," will ring through the halls of the universities. To the frustration of those of us who know that nothing in the past is set in stone and it's not about the answers, it's about the questions. Scholarship is not simply about finding a fact. It is about understanding and challenging those facts. And I'm sorry, but no chatbot telling me that such and such is the answer to my question will be more exciting than discovering a new angle on the question simply by digging and digging through the wealth of human knowledge that is already freely available to me. Scholarship is about changing and refining the question itself. And eventually, AI will simply continue to feed on our collective past and nothing new will come into being, because I simply don't think AI is intelligent in the way a scholar is. AI can't ask good questions.
Just to agree and organize a few of these thoughts:
1) what Henry refers to a choice of "knowing the truth or not" in using a chat bot is actually incorrect. He doesn't "know" anything when using a chat bot. No new knowledge is created. The knowledge is created in the research component. The thinking through the problem. Just because a chat bot give him a response to the question he poses does not make it correct, and I'm not just basing this on the idea of hallucinations. Knowledge is only created by human beings.
2) That process of gaining knowledge is gate kept in the way that he suggests is a problem for a reason! You have to work to earn it. Using a chat bot to "gain knowledge" in the way he suggests doesn't work. These machines are aimed at efficiency, and efficiency does not factor in human curiosity or the development of human knowledge.
3) I am sincerely concerned that the increase of usage of chat bots will lead to a decrease in the creation of knew knowledge, not more knowledge. This will, as he suggests, lead to more criticism, and less things to criticise. In short, if a chat bot can muscle its way to a solution of a math problem by super-humanly trying every possible logical option, it is the same as infinite monkeys hitting a keyboard and one of them producing *Hamlet*.
In Eco's *The Name of the Rose*, the monk Jorge of Borges wasn't right to destroy the contents of the book he was hiding, but he was right to hide it behind a mystery that only the worthy scholar could solve. Knowledge must be earned. Removing the obstacles to learning decreases human desire to learn.
Just one question (two parts): what does the term “self command” mean and what does it have to do with Jane Austen? Tell us what you learned from Claude, Henry!
AI can answer some questions like this before they've stolen all of the worlds knowledge and text. That's not anything to reward or praise them about!