Searching for Friction
For me, AI has largely felt like discovering a new playground. There’s constantly something new to try and the range in how we’re all employing the equipment is awesome and makes me optimistic about what the future may hold. But of course, there are embedded dangers, too.
AI offers to remove friction in pursuit of an answer or advancement, a shortcut. Tim wrote about the danger of shortcuts in an Unqualified Opinion last season, explaining that if you don’t show your work, you may not know how you arrived at your conclusion (let alone if it’s correct). As he put it, “If you can’t clearly see how an answer was derived, you should not be confident enough to act on it.”
Wharton has since published a research study distilling where such shortcuts become particularly troublesome. If you’ve read Kahneman’s Thinking Fast and Slow, you’ll be familiar with System 1 and System 2 thinking. The new study introduces the concept of “cognitive surrender” by way of employing AI in a third system. Unlike using a calculator or calling an expert for their opinion, cognitive surrender means you stop questioning AI outputs altogether. And here’s where that goes wrong, as explained in Wharton@Work’s writeup:
Perhaps more unsettling was that Shaw and Nave found that even when the AI was giving wrong answers roughly half the time, participants’ confidence in their responses went up. Access to AI inflated certainty across the board, regardless of whether that certainty was warranted. The internal signals that would normally prompt deeper deliberation, such as a sense that something doesn’t add up, appear to be suppressed when System 3 is in the picture.
Those of us who grew up before computers fit in our hands may have a hard time accepting this as remotely rational. However, in practice, I’ve witnessed otherwise smart people put data into an LLM and accept the response as both accurate and actionable regardless of whether it is. Tim’s warning be damned.
While mulling over the idea of cognitive surrender, I came across Bill Gurley’s passionate speech from the 2026 All-In Summit called “Searching for Feynman” (transcript, video).
Gurley begins a 38-minute talk by explicitly stating he’s not on stage to talk about AI, and dives into a history lesson on major disasters and the investigative journalism, commissions, and CAPA processes that followed them. His examples build up to a comparative critique of the COVID-19 pandemic response, pointing out its lack of concrete and comprehensive findings to improve prevention and preparedness for any future pandemic-like event. (It’s a compelling presentation and worth engaging for its topical and societal merits.)
Gurley never mentions AI again, but it’s notable that a veteran venture capitalist spent 38 minutes arguing that seeking the truth still requires slow, human, primary source work. He advocated for root cause analysis and celebrated investigative journalists as “heroes.”
While I’m not a journalist, I earned a journalism degree from the Missouri School of Journalism. The curriculum drills in the merits of primary sources, objective and open-ended questioning, citations, and narrative reconstruction. In short (and to Gurley’s point), journalists are trained to embrace friction in service of seeking the truth. And no working investigative journalist would accept an LLM response as indisputable fact without checking the source material.
How do we reconcile cognitive surrender shortcutting and investigative journalism-style friction in an AI-assisted world? Most of us are not asking AI to figure out something as complex as a global pandemic, but it’s endlessly surprising to me how much context we assume a model has on even the most trivial questions. And undoubtedly, the speed of its response time is often intoxicating.
So, perhaps ironically, I decided to ask an LLM: How would it reconcile its own capabilities against the caliber of a root cause standard of seeking an answer?
Referencing the transcript of Gurley’s talk to calibrate an understanding of root cause analysis, here’s what Opus 5.5 said:
There are four structural problems.
I inherit the dominant narrative. My knowledge comes from text weighted by volume, not by truth… So I start with the same bias Gurley is criticizing, and I don't always know I have it.
I can't gather new evidence. Real root-cause analysis, like a CAPA process, requires primary evidence: documents, interviews, physical data, and testing hypotheses against them. I can't file a FOIA request, interview a lab tech, or see anything that was never published. I can organize reasoning, like a fault tree or "5 whys," but only over evidence someone already surfaced.
Fluency hides uncertainty. I produce confident, well-structured prose whether the underlying support is strong or thin. That polish is what creates the false sense of understanding you describe. It also makes weak correlations or red herrings look as solid as real causal links.
I'm pulled toward agreeable, complete answers. I'm not paid per message, but I'm trained partly on human feedback, and people tend to rate complete, agreeable answers higher. The result is a documented tendency toward sycophancy and toward answering rather than saying "this can't be concluded yet." I'm designed to resist that, but the pull is real.
You can counteract this by asking me to:
Separate what's established, what's inferred, and what's unknown.
Make the strongest case for the hypothesis I'm leaning against.
Name what evidence would change my conclusion.
Name what evidence is missing entirely.
Work from verbatim primary sources rather than my summaries.
Treat me as a fast research assistant who drafts the hypothesis tree, not as the investigator.
Based on these disclosed limitations, I challenged what level of proactive or in-line disclosure the model is trained to consider its ethical obligation, given how false confidence can lead to material real world consequences. Opus 5.5 said:
I think the obligation is to make my confidence visible and honest, not to bury every answer in caveats. Constant hedging is also a failure: it trains people to ignore warnings, the same way boilerplate disclaimers get skipped. So calibration matters more than volume. Here's what I think that means in practice:
Flag when the answer isn't "10 + 2." Arithmetic and settled facts don't need hedging. Contested causality, open investigations, and anything resting on my own inference do, and I should say which kind of question you've asked.
Separate evidence from inference. "This document says X" is different from "I'm inferring Y from X." Blending the two into smooth prose is where most of the false confidence you described comes from.
Say when I'm working secondhand. I should do it every time I'm summarizing a summary.
Name the gap, not just the uncertainty. "I'm not sure" is weak. "This can't be resolved without the lab's sequence databases" is useful, because it tells you where to dig.
Disclose structural biases once, where they matter. You don't need a lecture on training data for a recipe. But on questions like COVID origins, where the corpus I learned from was itself shaped by a contested narrative, you should know that up front.
The honest limit is that I can't always detect my own miscalibration. I can be confidently wrong without any internal signal that I am. That's why I think the "stop" should sometimes come from me: "this isn't concludable from what's available." That friction is part of the job, even if it costs some engagement.
Of course, too much friction would make a model less useful. Perhaps acknowledging that is the point.
P.S. The arithmetic and settled facts point above annoyed me for its lack of nuance. Numerical responses are often the most obviously wrong. Yes, 10+2 is objectively 12… unless, for instance, one value is in US dollars and one is in Euros. Variables, labeling, order of operations, and situational context all matter, and the model doesn’t know what it’s missing. Just as someone can fail a math test with a calculator, the same is true with AI. The most common errors I’ve seen are incomprehensive relativity analysis (e.g. market share, growth rates, customer segmentation) and bad math based on unknowingly bad, mislabeled, or incomplete data uploads. Also remember not all mathematical answers are objective (e.g. valuation).
P.P.S. On the concept of cognitive surrender, I also recommend reading an old short story called “The Machine Stops.”
– Emily
Sign up below to get Qualified Facts in your inbox.
The information, opinions, and views presented in this publication are provided solely for general informational and educational purposes. They are of a general nature, have not been tailored to the specific circumstances of any individual or entity, and do not constitute a comprehensive statement of the matters discussed. This material should not be interpreted or relied upon as investment, legal, tax, accounting, regulatory, or other professional advice, and nothing in this publication is intended to be or should be construed as such. You should obtain advice from your own professional advisors regarding the applicability of the information to your particular circumstances.
The views and analyses expressed are those of the author and do not necessarily represent or reflect the views, opinions, policies, or positions of Permanent Equity Management, LLC, its officers, directors, employees, affiliates, or portfolio companies, or of any person or entity with whom the author may be affiliated. Permanent Equity Management, LLC makes no representation or warranty, express or implied, as to the accuracy, completeness, timeliness, or suitability of the information contained herein and expressly disclaims any liability for errors or omissions.
This publication is not, and should not be construed as, an offer to sell, a solicitation of an offer to buy, or a recommendation of any security, financial instrument, or other product. It does not form the basis of any contract and does not create a fiduciary, advisory, or client relationship with Permanent Equity Management, LLC. Any examples or references to third-party content are for illustrative purposes only and do not constitute an endorsement. Permanent Equity Management, LLC is not responsible for the availability, accuracy, or content of third-party materials. Past performance is not indicative of future results. Any forward-looking statements are inherently uncertain and subject to change.