In some ways this is an extension of a problem that already exists in well-researched fields, e.g., Shakespeare studies. Although there are some who disagree, I think most of us in the field feel that there isn't any low-hanging fruit left in terms of basic scholarship. Yet coming to understand Shakespeare and his world remains the work of a lifetime, and no one can even begin to ask interesting critical questions without a deep grasp of the history broadly understood (including literary history and the historiography). You gain a feel for that world only by trying to put things together yourself--even if someone else has already done something broadly similar, or (realistically) far superior. You can't shortcut the process and it can't be relegated to AI or to the books of the past. I think we probably WILL see a shift from scholarship to criticism, but I'm not actually happy about that. What we probably do need is a shift from thinking about knowledge as something embedded in books to thinking about knowledge as embedded in people.
As a disclaimer, I've always (that is, since 2011; but privately, I think I had already quite grasped the possibility in my mind long before, when I was a kid with no Internet) been a believer in the (imminent) possibility of superhuman AI and the Singularity and thought a lot of the implications of humanity becoming extinct or irrelevant or anyway unrecognizable. I think this was good for me, as I went through some existential crises a bit before their time.
As an example, I am a mathematician and I had thought since 2018 of the possibility of computers routinely proving advanced math results while society otherwise marches on (I admit that, before 2018, I had only thought of a "hard take-off" where the period between no AI and superintelligent AI was of the order of several hours to a month). I think this makes me better prepared mentally than my colleagues who are in various states of denial.
In the past year, I have turned more and more to literature, thanks partly to your blog (I know it sounds a bit silly but your introductory post [1] brought me a lot of warmth). I have always enjoyed books, but now I realized that the act of reading a book, and then turning it around in my mind, is something that AI, no matter how intelligent or powerful, can never take from me (except by killing or torturing me, but humans could do that already anyway).
It is possible that we'll no longer get money or prestige or status from scholarship of any sort, but as long as we manage to find ways to stay alive and healthy in the coming times (and, if we can't, then our problems are bigger than the end of academia), we can still find ways to do the activities we like.
Anyway, I use this comment to thank you very much for writing this blog (whose archives I haven't yet read in full). It was and still is for me a light in dark times.
I will confess upfront I am not at all interested in using AI. But it seems to me there are a few flaws here. Yes, the digitalisation of archives and books has allowed access to the general public and it is fantastic - but by and large, this has been freely available or for a small fee and the AI you speak of is not free. The libraries are free to access, AI is not and it does not provide access, it provides you information that its systems calculate will answer your question. AI companies will - or eventually will - gatekeep the information they have - they have to in order to be economically viable. They're already destroying books. That will finally lock us out of those original sources unless we are willing to pay for information that it determines answers our question - information that was once freely available for us to discover. Most of us already have a sense that simply googling something has become much less useful over the last couple of years. Yes, it might at the moment appear quicker to find a fact with AI (it used to be a lot quicker with Google and took you to the original source, but c'est la vie), but it narrows context, it must be checked for accuracy, and it will always make mistakes. There's a reason scholars will dig and dig even after they apparently have the facts. Even our own records lie, obscure, and leave things out or word meaning alters over time etc. AI will collapse debate. "Oh, but AI said so," will ring through the halls of the universities. To the frustration of those of us who know that nothing in the past is set in stone and it's not about the answers, it's about the questions. Scholarship is not simply about finding a fact. It is about understanding and challenging those facts. And I'm sorry, but no chatbot telling me that such and such is the answer to my question will be more exciting than discovering a new angle on the question simply by digging and digging through the wealth of human knowledge that is already freely available to me. Scholarship is about changing and refining the question itself. And eventually, AI will simply continue to feed on our collective past and nothing new will come into being, because I simply don't think AI is intelligent in the way a scholar is. AI can't ask good questions.
Just to agree and organize a few of these thoughts:
1) what Henry refers to a choice of "knowing the truth or not" in using a chat bot is actually incorrect. He doesn't "know" anything when using a chat bot. No new knowledge is created. The knowledge is created in the research component. The thinking through the problem. Just because a chat bot give him a response to the question he poses does not make it correct, and I'm not just basing this on the idea of hallucinations. Knowledge is only created by human beings.
2) That process of gaining knowledge is gate kept in the way that he suggests is a problem for a reason! You have to work to earn it. Using a chat bot to "gain knowledge" in the way he suggests doesn't work. These machines are aimed at efficiency, and efficiency does not factor in human curiosity or the development of human knowledge.
3) I am sincerely concerned that the increase of usage of chat bots will lead to a decrease in the creation of knew knowledge, not more knowledge. This will, as he suggests, lead to more criticism, and less things to criticise. In short, if a chat bot can muscle its way to a solution of a math problem by super-humanly trying every possible logical option, it is the same as infinite monkeys hitting a keyboard and one of them producing *Hamlet*.
In Eco's *The Name of the Rose*, the monk Jorge of Borges wasn't right to destroy the contents of the book he was hiding, but he was right to hide it behind a mystery that only the worthy scholar could solve. Knowledge must be earned. Removing the obstacles to learning decreases human desire to learn.
It’s working constantly as superhuman speed, but it’s absolutely not just iterating randomly over possibilities, anymore than that approach would work with Go.
(The game which took far longer to fall to computers than Chess, due to higher mathematical complexity)
The use you describe is a far cry from generative AI, which is what most of us are worried about. But I do hope you checked each source yourself, because LLMs *very* frequently hallucinate even information they claim has a source.
The widespread and increasing digitisation of archives -especially for Family Historians - often offers intriguing new insights into individuals who lived in the past, but these are often scant and cannot in themselves reveal much that is significant about the communities and social dynamics of the society in which individuals lived and operated.
Network analysis can be a particularly powerful tool for understanding historical communities. By mapping the relationships between individuals documented (however scantily) in archival sources, researchers can identify clusters of connection that might otherwise remain invisible. I was interested in what could be gleaned from this approach in terms of London Quaker female social structures around childbirth in the late C17th - childbirth being an almost entirely female and undocumented experience in the early modern period.
This approach revealed structures and relationships that no amount of close reading of individual records could have uncovered. I was able to make a number of evidenced interventions challenging accepted views of early modern childbirth and midwifery.
I used JuliusAI which uses Python code to provide data analysis and deploy community detection algorithms.
By thinking of these tools as a research assistant or even project team, it is clear that individual historians may rapidly now do the kinds of work that previously would have required human research assistants or teams with all the messy supervision (not to say funding) that this would otherwise require. As these tools become more sophisticated and more widely accessible, they will not only democratise the academy but also enable historians of all stripes to ask questions at scales that were previously thought impossible. Indeed, I am now continuing my work with Claude asking it to build me a database to my specification from which I hope to be able to offer new insights, but also - as importantly- to offer out to other scholars as a resource. That resource would need to be interrogated by an AI, but the questions and framing remain with the historian.
Crucially an AI cannot replace the interpretative work of the historian. It is vital therefore to have a thorough understanding of both the limitations of the original data being analysed and the context of the historical sources used. The significance of a particular network structure, the meaning of a relationship, the context that makes a pattern meaningful—these require historical expertise and critical analysis. The role of computational tools is to reveal what needs to be interpreted, not to provide the interpretation itself.
These tools are therefore not a replacement for close reading and contextual analysis, but a complement to it—a way of seeing the archive from a different angle, of revealing structures and relationships that would otherwise remain hidden
I would be very interested to hear comments from anyone else who has deployed an AI tool in a historical archive
Anybody interested in extracting data from large collections of texts should be stoked about LLMs. Five years ago you would have needed an army of student research assistants to read each text carefully and extract all the data you wanted from it; now you can have an LLM do it more cheaply, quickly, and - for technical writing especially - often more reliably.
I am reading American Prometheus. It turns out that Oppenheimer not only was a better quantum physicist than me, but he was also much better read. Among the innumerable works he read and I didn’t was a poem by a poet surnamed Herbert with the line “I will abroad”.
I asked Claude and I got the poem. The title is “The Collar” and it was written by the 17th-century English poet George Herbert. English is not my first language, I am not as well read as Oppenheimer, and the poem is hard. Again, thanks to Claude, I managed to understand its beauty and to comprehend its importance.
Gentlemen, AI is Asimov’s Encyclopedia Galactica and Captain Picard’s computer all in one. It is one of our greatest inventions and the most necessary one: we are drowning in information, we are besieged with so many truths and so many lies, that we need our own curator, our own researcher, and our own wise man.
Just to agree with the other reader. You don't understand that poem, nor do you comprehend its importance.
This is like reading the summary of a song and being told what key it was in and you claim you've heard that song.
Whatever knowledge or expereince you've gained from that interaction is so tenuous. But a poem you read, and re-read, and think about, and read again 5 years later becomes interwoven into your own DNA.
Chat bots can't do any of that, and in fact they obfuscate the very process of gaining knowledge.
It is probably a painful mistake to confuse difficulty with knowledge, or a chatbot’s limitations with the limitations of its user. A machine does not relieve us of judgment, verification, or thought. Nor does efficiency oppose curiosity or agency; it simply lowers the cost of locating, comparing, and connecting what already exists in the noosphere. That leaves more human attention for questions, interpretation, and experiment.
It is probably a costly mistake to believe that a tool increases one's inherent skill or doesn't mitigate one's ability to perceive Truth.
We are in this *exact* predicament because people foolishly believe your last two sentences without holding them up to scrutiny. This is the error of our culture.
Of course a tool can be used in ways that weaken judgment; some professors accomplish that without machines. But it does not follow that efficiency opposes curiosity or inquiry. Nor does it prevent a considered conclusion.
Efficiency can make it easier to find material, compare accounts, trace thought across disciplines, and test a question against more evidence. None of that performs the judgment for us.
Whether a tool becomes a substitute for thought or an extension of it is on you, or me. Same as it ever was.
Henry! You are a bit vague about your query. Did you look at all of the sources for the word that Claude found? Did you form your own judgment based on the original sources you found? Did you survey related concepts? How did you know its account of which concepts were related? A word search alone is not sufficient. Scholarship involves a judgement based on the sources. The judgement is contingent. There is no "I just wanted the truth!"
It’s a topic I already know something about, which is why I was interested to see what Claude could do. As I said, using AI sends me to the original sources and I don’t think we disagree about what you say in general, but the overall balance will be very different quite soon. What Claude does is not really a word search. On the very narrow question of “this phase is not wisely used in moral discourse” there is some sort of non (or less) contingent truth, and from there all sorts of other factors apply. There is a history of scholarship on this and related topics that applies to Jane Austen which made relatively slow progress: were that line of enquiry to begin today it would happen far more quickly, albeit the points you make would still apply.
I would find a view from this altitude extremely frustrating, but I am a fanatic about sources and about forming my own judgement. Have read too much cruddy scholarship to trust a scholarship-synthesizer.
This would be all fine and dandy if LLMs didn't frequently and inevitably give you false information. It will tell you that the phrase you're looking for is used by some author, and even give you a quotation that looks perfect -- but when you actually look at the text it's not there, so you have to go back to the primary texts anyway. AI models are probabilistic, so every word they spit out is suspicious!
"If we have to choose between knowing much less of the truth but knowing it from human work and knowing more of it from AI, for how long will we be able to choose not knowing?" Henry, this sentence pinpoint one of my primary objections to using AI for research; too often, it does not, in fact, provide "more truth" but hallucinate an answer out of whole cloth. I'd rather have "less truth" which I have determined for myself to be true than no truth at all.
Those forty-five minutes replaced a long stretch of not knowing. Scholars form their judgment inside that stretch rather than at the end of it, and taste is what accumulates while a question stays open. There is a harder thing sitting under your word depressed. A role that tells someone who they are takes years to put down, and the putting down looks a lot like grief. I have sat with people whose competence was holding up their whole sense of themselves, and not one of them could see it until it stopped being needed.
I think the biggest danger with AI is losing our ability to critically think. This is exactly why I believe (hope) literature will sustain - reading is a practice of empathy, challenging your own values in the worlds and minds of others. It is true that online resources and digital archives have increased accessibility. The fine balance is not to use an LLM to understand and synthesize such inputs, but rather as a starting point to access them. It’s a digital library, able to sift through what has been created and help you search the catalog. Now the question I want to raise is - is the internet already not a catalog itself? And if so, what is the real use case for AI in the humanities?
Really appreciate your (and Rebecca Lowe's) intelligent efforts to 'think through' AI/ai. I do have two concerns, about which I am hoping to see more commentary in the future:
1. The challenge of accuracy
You mention that models will continue to improve. But will they become more accurate (or even correct)? How will accuracy (or correctness) be adjudicated? Human crowd-sourcing? Human expert reviewers? ai self-regulation? ai reviewers? If criticism replaces scholarship (I would argue that this has increasingly long since happened in the social sciences, as ideological reinterpretations replaced new ideas), is academic work not likely to get boring pretty quickly? (as has happened in the social sciences)
2. The challenge of access
My daughter has just started university, and the publisher of one of her appallingly expensive textbooks (textbooks were always appallingly expensive, but this pushes the limit) offers a digital version -- but this version, it appears, is only for rent, for 180 or 360 days. This disquiets me greatly: books you cannot own (unless you pay extortionate prices), even in digital format. The university library does not have it, strangely (not so strangely: this university, like many, is under budget pressure, and imagine what the publisher charges universities for digital access!).
I see this as representative of a trend. Many people have commented on the vanishing of Google Books and the shocking decline of Google search (not a new decline, but recently it has been stark). Many institutions' digitization efforts assumed perennial access--an assumption that has proven naive. AI companies' purchase and destruction of print books seems part and parcel of this process (yes: I've read the commentaries, and no, I don't think it matters that many of the books going into their maw are boring manuals -- as a regular purchaser of old books, I've noticed titles simply vanishing, some of which are important books). Libraries de-accessioning large parts of their print collections, journals going out of print, and media archives vanishing into the ether accelerate this process. If a few large ai companies own all our information and feed it only to those who pay to play, with spoon-fed ai answers to the rest of us, what will this mean for access, information, knowledge, understanding? This problem magnifies when ai executives and state actors become aligned.
In some ways this is an extension of a problem that already exists in well-researched fields, e.g., Shakespeare studies. Although there are some who disagree, I think most of us in the field feel that there isn't any low-hanging fruit left in terms of basic scholarship. Yet coming to understand Shakespeare and his world remains the work of a lifetime, and no one can even begin to ask interesting critical questions without a deep grasp of the history broadly understood (including literary history and the historiography). You gain a feel for that world only by trying to put things together yourself--even if someone else has already done something broadly similar, or (realistically) far superior. You can't shortcut the process and it can't be relegated to AI or to the books of the past. I think we probably WILL see a shift from scholarship to criticism, but I'm not actually happy about that. What we probably do need is a shift from thinking about knowledge as something embedded in books to thinking about knowledge as embedded in people.
About this we agree!
As a disclaimer, I've always (that is, since 2011; but privately, I think I had already quite grasped the possibility in my mind long before, when I was a kid with no Internet) been a believer in the (imminent) possibility of superhuman AI and the Singularity and thought a lot of the implications of humanity becoming extinct or irrelevant or anyway unrecognizable. I think this was good for me, as I went through some existential crises a bit before their time.
As an example, I am a mathematician and I had thought since 2018 of the possibility of computers routinely proving advanced math results while society otherwise marches on (I admit that, before 2018, I had only thought of a "hard take-off" where the period between no AI and superintelligent AI was of the order of several hours to a month). I think this makes me better prepared mentally than my colleagues who are in various states of denial.
In the past year, I have turned more and more to literature, thanks partly to your blog (I know it sounds a bit silly but your introductory post [1] brought me a lot of warmth). I have always enjoyed books, but now I realized that the act of reading a book, and then turning it around in my mind, is something that AI, no matter how intelligent or powerful, can never take from me (except by killing or torturing me, but humans could do that already anyway).
It is possible that we'll no longer get money or prestige or status from scholarship of any sort, but as long as we manage to find ways to stay alive and healthy in the coming times (and, if we can't, then our problems are bigger than the end of academia), we can still find ways to do the activities we like.
Anyway, I use this comment to thank you very much for writing this blog (whose archives I haven't yet read in full). It was and still is for me a light in dark times.
[1] https://www.commonreader.co.uk/p/how-to-get-started-reading-english
I love this comment and agree very much. Reading Tolstoy is irreplaceable!
So why bother engaging with a chat bot, Henry?
I will confess upfront I am not at all interested in using AI. But it seems to me there are a few flaws here. Yes, the digitalisation of archives and books has allowed access to the general public and it is fantastic - but by and large, this has been freely available or for a small fee and the AI you speak of is not free. The libraries are free to access, AI is not and it does not provide access, it provides you information that its systems calculate will answer your question. AI companies will - or eventually will - gatekeep the information they have - they have to in order to be economically viable. They're already destroying books. That will finally lock us out of those original sources unless we are willing to pay for information that it determines answers our question - information that was once freely available for us to discover. Most of us already have a sense that simply googling something has become much less useful over the last couple of years. Yes, it might at the moment appear quicker to find a fact with AI (it used to be a lot quicker with Google and took you to the original source, but c'est la vie), but it narrows context, it must be checked for accuracy, and it will always make mistakes. There's a reason scholars will dig and dig even after they apparently have the facts. Even our own records lie, obscure, and leave things out or word meaning alters over time etc. AI will collapse debate. "Oh, but AI said so," will ring through the halls of the universities. To the frustration of those of us who know that nothing in the past is set in stone and it's not about the answers, it's about the questions. Scholarship is not simply about finding a fact. It is about understanding and challenging those facts. And I'm sorry, but no chatbot telling me that such and such is the answer to my question will be more exciting than discovering a new angle on the question simply by digging and digging through the wealth of human knowledge that is already freely available to me. Scholarship is about changing and refining the question itself. And eventually, AI will simply continue to feed on our collective past and nothing new will come into being, because I simply don't think AI is intelligent in the way a scholar is. AI can't ask good questions.
Just to agree and organize a few of these thoughts:
1) what Henry refers to a choice of "knowing the truth or not" in using a chat bot is actually incorrect. He doesn't "know" anything when using a chat bot. No new knowledge is created. The knowledge is created in the research component. The thinking through the problem. Just because a chat bot give him a response to the question he poses does not make it correct, and I'm not just basing this on the idea of hallucinations. Knowledge is only created by human beings.
2) That process of gaining knowledge is gate kept in the way that he suggests is a problem for a reason! You have to work to earn it. Using a chat bot to "gain knowledge" in the way he suggests doesn't work. These machines are aimed at efficiency, and efficiency does not factor in human curiosity or the development of human knowledge.
3) I am sincerely concerned that the increase of usage of chat bots will lead to a decrease in the creation of knew knowledge, not more knowledge. This will, as he suggests, lead to more criticism, and less things to criticise. In short, if a chat bot can muscle its way to a solution of a math problem by super-humanly trying every possible logical option, it is the same as infinite monkeys hitting a keyboard and one of them producing *Hamlet*.
In Eco's *The Name of the Rose*, the monk Jorge of Borges wasn't right to destroy the contents of the book he was hiding, but he was right to hide it behind a mystery that only the worthy scholar could solve. Knowledge must be earned. Removing the obstacles to learning decreases human desire to learn.
On (3) - it’s not doing that.
It’s working constantly as superhuman speed, but it’s absolutely not just iterating randomly over possibilities, anymore than that approach would work with Go.
(The game which took far longer to fall to computers than Chess, due to higher mathematical complexity)
There is ‘thought-like’ processing going on
The use you describe is a far cry from generative AI, which is what most of us are worried about. But I do hope you checked each source yourself, because LLMs *very* frequently hallucinate even information they claim has a source.
Comment
I’m commenting as someone who used AI and machine learning in the archive (Quaker C17th birth records available via Ancestry) for her dissertation for the IHR’s History of People Place and Community programme. I submitted summer 25 and the tools have improved immeasurably since then. I wrote a blogpost for the IHR(https://blog.history.ac.uk/2026/03/ai-in-the-archive-using-data-analytic-tools-and-social-network-techniques-to-uncover-hidden-histories-2/ ) but quote some of the main points below.
The widespread and increasing digitisation of archives -especially for Family Historians - often offers intriguing new insights into individuals who lived in the past, but these are often scant and cannot in themselves reveal much that is significant about the communities and social dynamics of the society in which individuals lived and operated.
Network analysis can be a particularly powerful tool for understanding historical communities. By mapping the relationships between individuals documented (however scantily) in archival sources, researchers can identify clusters of connection that might otherwise remain invisible. I was interested in what could be gleaned from this approach in terms of London Quaker female social structures around childbirth in the late C17th - childbirth being an almost entirely female and undocumented experience in the early modern period.
This approach revealed structures and relationships that no amount of close reading of individual records could have uncovered. I was able to make a number of evidenced interventions challenging accepted views of early modern childbirth and midwifery.
I used JuliusAI which uses Python code to provide data analysis and deploy community detection algorithms.
By thinking of these tools as a research assistant or even project team, it is clear that individual historians may rapidly now do the kinds of work that previously would have required human research assistants or teams with all the messy supervision (not to say funding) that this would otherwise require. As these tools become more sophisticated and more widely accessible, they will not only democratise the academy but also enable historians of all stripes to ask questions at scales that were previously thought impossible. Indeed, I am now continuing my work with Claude asking it to build me a database to my specification from which I hope to be able to offer new insights, but also - as importantly- to offer out to other scholars as a resource. That resource would need to be interrogated by an AI, but the questions and framing remain with the historian.
Crucially an AI cannot replace the interpretative work of the historian. It is vital therefore to have a thorough understanding of both the limitations of the original data being analysed and the context of the historical sources used. The significance of a particular network structure, the meaning of a relationship, the context that makes a pattern meaningful—these require historical expertise and critical analysis. The role of computational tools is to reveal what needs to be interpreted, not to provide the interpretation itself.
These tools are therefore not a replacement for close reading and contextual analysis, but a complement to it—a way of seeing the archive from a different angle, of revealing structures and relationships that would otherwise remain hidden
I would be very interested to hear comments from anyone else who has deployed an AI tool in a historical archive
Anybody interested in extracting data from large collections of texts should be stoked about LLMs. Five years ago you would have needed an army of student research assistants to read each text carefully and extract all the data you wanted from it; now you can have an LLM do it more cheaply, quickly, and - for technical writing especially - often more reliably.
I am reading American Prometheus. It turns out that Oppenheimer not only was a better quantum physicist than me, but he was also much better read. Among the innumerable works he read and I didn’t was a poem by a poet surnamed Herbert with the line “I will abroad”.
I asked Claude and I got the poem. The title is “The Collar” and it was written by the 17th-century English poet George Herbert. English is not my first language, I am not as well read as Oppenheimer, and the poem is hard. Again, thanks to Claude, I managed to understand its beauty and to comprehend its importance.
Gentlemen, AI is Asimov’s Encyclopedia Galactica and Captain Picard’s computer all in one. It is one of our greatest inventions and the most necessary one: we are drowning in information, we are besieged with so many truths and so many lies, that we need our own curator, our own researcher, and our own wise man.
Let us embrace this miracle.
Just to agree with the other reader. You don't understand that poem, nor do you comprehend its importance.
This is like reading the summary of a song and being told what key it was in and you claim you've heard that song.
Whatever knowledge or expereince you've gained from that interaction is so tenuous. But a poem you read, and re-read, and think about, and read again 5 years later becomes interwoven into your own DNA.
Chat bots can't do any of that, and in fact they obfuscate the very process of gaining knowledge.
It is probably a painful mistake to confuse difficulty with knowledge, or a chatbot’s limitations with the limitations of its user. A machine does not relieve us of judgment, verification, or thought. Nor does efficiency oppose curiosity or agency; it simply lowers the cost of locating, comparing, and connecting what already exists in the noosphere. That leaves more human attention for questions, interpretation, and experiment.
It is probably a costly mistake to believe that a tool increases one's inherent skill or doesn't mitigate one's ability to perceive Truth.
We are in this *exact* predicament because people foolishly believe your last two sentences without holding them up to scrutiny. This is the error of our culture.
Of course a tool can be used in ways that weaken judgment; some professors accomplish that without machines. But it does not follow that efficiency opposes curiosity or inquiry. Nor does it prevent a considered conclusion.
Efficiency can make it easier to find material, compare accounts, trace thought across disciplines, and test a question against more evidence. None of that performs the judgment for us.
Whether a tool becomes a substitute for thought or an extension of it is on you, or me. Same as it ever was.
Well done. As an English ABD who morphed into a business guy, I think about these issues quite a bit. I think you're dead on.
Henry! You are a bit vague about your query. Did you look at all of the sources for the word that Claude found? Did you form your own judgment based on the original sources you found? Did you survey related concepts? How did you know its account of which concepts were related? A word search alone is not sufficient. Scholarship involves a judgement based on the sources. The judgement is contingent. There is no "I just wanted the truth!"
It’s a topic I already know something about, which is why I was interested to see what Claude could do. As I said, using AI sends me to the original sources and I don’t think we disagree about what you say in general, but the overall balance will be very different quite soon. What Claude does is not really a word search. On the very narrow question of “this phase is not wisely used in moral discourse” there is some sort of non (or less) contingent truth, and from there all sorts of other factors apply. There is a history of scholarship on this and related topics that applies to Jane Austen which made relatively slow progress: were that line of enquiry to begin today it would happen far more quickly, albeit the points you make would still apply.
I would find a view from this altitude extremely frustrating, but I am a fanatic about sources and about forming my own judgement. Have read too much cruddy scholarship to trust a scholarship-synthesizer.
We don’t necessarily disagree about this!
AI can answer some questions like this before they've stolen all of the worlds knowledge and text. That's not anything to reward or praise them about!
As an archivist I think of the future value of offline collections often. I’m really curious how my job will look in 10 years.
This would be all fine and dandy if LLMs didn't frequently and inevitably give you false information. It will tell you that the phrase you're looking for is used by some author, and even give you a quotation that looks perfect -- but when you actually look at the text it's not there, so you have to go back to the primary texts anyway. AI models are probabilistic, so every word they spit out is suspicious!
"If we have to choose between knowing much less of the truth but knowing it from human work and knowing more of it from AI, for how long will we be able to choose not knowing?" Henry, this sentence pinpoint one of my primary objections to using AI for research; too often, it does not, in fact, provide "more truth" but hallucinate an answer out of whole cloth. I'd rather have "less truth" which I have determined for myself to be true than no truth at all.
Those forty-five minutes replaced a long stretch of not knowing. Scholars form their judgment inside that stretch rather than at the end of it, and taste is what accumulates while a question stays open. There is a harder thing sitting under your word depressed. A role that tells someone who they are takes years to put down, and the putting down looks a lot like grief. I have sat with people whose competence was holding up their whole sense of themselves, and not one of them could see it until it stopped being needed.
I think the biggest danger with AI is losing our ability to critically think. This is exactly why I believe (hope) literature will sustain - reading is a practice of empathy, challenging your own values in the worlds and minds of others. It is true that online resources and digital archives have increased accessibility. The fine balance is not to use an LLM to understand and synthesize such inputs, but rather as a starting point to access them. It’s a digital library, able to sift through what has been created and help you search the catalog. Now the question I want to raise is - is the internet already not a catalog itself? And if so, what is the real use case for AI in the humanities?
Really appreciate your (and Rebecca Lowe's) intelligent efforts to 'think through' AI/ai. I do have two concerns, about which I am hoping to see more commentary in the future:
1. The challenge of accuracy
You mention that models will continue to improve. But will they become more accurate (or even correct)? How will accuracy (or correctness) be adjudicated? Human crowd-sourcing? Human expert reviewers? ai self-regulation? ai reviewers? If criticism replaces scholarship (I would argue that this has increasingly long since happened in the social sciences, as ideological reinterpretations replaced new ideas), is academic work not likely to get boring pretty quickly? (as has happened in the social sciences)
2. The challenge of access
My daughter has just started university, and the publisher of one of her appallingly expensive textbooks (textbooks were always appallingly expensive, but this pushes the limit) offers a digital version -- but this version, it appears, is only for rent, for 180 or 360 days. This disquiets me greatly: books you cannot own (unless you pay extortionate prices), even in digital format. The university library does not have it, strangely (not so strangely: this university, like many, is under budget pressure, and imagine what the publisher charges universities for digital access!).
I see this as representative of a trend. Many people have commented on the vanishing of Google Books and the shocking decline of Google search (not a new decline, but recently it has been stark). Many institutions' digitization efforts assumed perennial access--an assumption that has proven naive. AI companies' purchase and destruction of print books seems part and parcel of this process (yes: I've read the commentaries, and no, I don't think it matters that many of the books going into their maw are boring manuals -- as a regular purchaser of old books, I've noticed titles simply vanishing, some of which are important books). Libraries de-accessioning large parts of their print collections, journals going out of print, and media archives vanishing into the ether accelerate this process. If a few large ai companies own all our information and feed it only to those who pay to play, with spoon-fed ai answers to the rest of us, what will this mean for access, information, knowledge, understanding? This problem magnifies when ai executives and state actors become aligned.
Looking forward to reading your ongoing thoughts.