xerox13ster an hour ago

Maybe this is a really really, really, really, really dumb idea:

What if we started Refining and reinforcing these Extremely fast thinking, models with text written exclusively by stoners?

Stoners think slower, but they have more wide, branching thoughts. I find I do some of my most seemingly creative and explorative thinking when I am high.

If there are effects of time and season and energy in models of LLMS such that you can say it’s June and it performs worse or it’s January and it performs better. Or you can say you’ve got a good nights rest so do better and it does, due to the fact that human writing is affected by our mental state, and our mental state is affected by physiological factors.

So then, I wonder what happens if you do HFRL and refinement on text from people under the influence of marijuana.

  • cab11150904 an hour ago

    God I hate potheads. It’s like they beg you to dislike them.

    • mandmandam 25 minutes ago

      I've noticed that visceral hatred of stoners is a common trait with people who define their success in life almost entirely by their career, and other visible external factors - marriage, kids, house size.

      ... Maybe stoners know something you don't. Maybe part of you sees that, and seethes.

Jimmc414 5 hours ago

Maps well to Kahneman's "Thinking Fast and Slow" framework

system 1 thinking for early layer processing of uncertainty in LLMs. quick, intuitive decisions, focuses on uncertainty, happens in early transformer layers.

system 2 thinking for later layer processing of empowerment (selecting elements that maximize future possibilities). strategic, deliberate evaluation, considering long-term possibilities, happens in later layers.

system 1 = 4o/llama 3.1

system 1 + system 2 = o1/r1 reasoning models

empowerment calculation seems possibly oversimplified - assumes a static value for elements over a dynamic context-dependent empowerment

interesting that higher temperatures improved performance slightly for system 1 models although they still made decisions before empowerment information could influence them

edit: removed the word "novel". The paper shows early-layer processing of uncertainty vs later-layer processing of empowerment.

  • hinkley 3 hours ago

    Some people have been trying to claim this book is based on faulty research but I definitely know people who don’t believe they work the way the book describes but the only sane explanation for their behavior is that either they do or I know an inordinate number of pathological liars.

    And I know some neurodiverse people who absolutely believe this is why they are like this. And that’s always a problem with studies. Some things are 10% of the time, it works every time. If you test the general public for mental acuity and caffeine you’re going to see undiagnosed ADHD people benefitting, or find they’re already consuming alarming quantities of the stuff (self medication).

  • fudged71 2 hours ago
    • nradov an hour ago

      Modeling an analog system like human cognition into any number of discrete tiers is inherently kind of arbitrary and unscientific. I doubt you could ever prove experimentally that all human thinking works through exactly two or three or ten or whatever number of tiers. But it's at least possible that a three-tier model facilitates building AI software which is "good enough" for many practical use cases.

  • svnt 2 hours ago

    Predicting language tokens but without logic probably does not map well at all to system 1 thinking, except where fast=conversation.

    • Jimmc414 an hour ago

      the system 1/2 analogy is obviously imperfect for token prediction, but the paper does provide evidence that uncertainty processing in early layers (layer 2) affects decisions before empowerment processing in later layers (layer 72) can influence them. quick decisions based on uncertainty before deeper strategic evaluation does echo how fast thinking can override slow thinking, even if the mechanisms are substantially different.

  • nonameiguess 32 minutes ago

    I think it goes beyond that. When we say animals learn and humans in particular generate new knowledge by exploration, it's not just system I versus system II thinking. You might legitimately not be able to answer question just by thinking at all. You have to go out in the world and find the answer. It might be as simple and walking around a building to see what's on the other side or it might as complicated as spending a few thousand years learning to build particle accelerators and sensors that can infer what is happening at a subatomic level in these, but you can't just "generate" an answer with sheer brainpower.

    This is a lot older than Kahneman. It's the classical split between rationalists and empiricists.

    AI maximalists and x-risk death cult fanatics are effectively being rationalists. They don't believe a sufficiently intelligent entity will be bound by the need to do science. It can, by sheer force of thought, figure out the answer to any question, develop a perfect course of action to achieve any goal, as quickly as electricity can traverse its own internal circuits.

    It's interesting to see the constraints on how they can even study this in the paper, though. You can only ever ask a computing agent to do what it even can do. If you want it to explore, it needs a toy world that maybe digitally encodes a reasonable stripped-down representation of the real world, but without a body, sensors, and full autonomy, it can't ever truly explore the real real world.

    • Qwertious 11 minutes ago

      >it can't ever truly explore the real real world.

      Would you believe that computers are made out of atoms, and that computer programs modify those atoms? Via electrons, I believe.

  • optimalsolver 4 hours ago

    > quick, intuitive decisions, focuses on what's novel/unknown

    Wouldn't it be the exact opposite? That is, novel stimuli requires more extensive processing at higher levels of cognition.

    • hinkley 3 hours ago

      Requires != receives. Knee-jerk reactions to novel circumstances get people into trouble all the time. Particularly in impulsive people, who find their actions sometimes don’t align with their values, to the point they can’t be friends with people who are particularly judgemental about actions versus words and won’t accept apologies.

      It is a lot of work to retrain your lizard brain to not react in unpleasant ways when the shit hits the fan or you’re exhausted, and the default answer to everything is “no”. There are ways I show up in a crisis that are exemplary, and others that I’m not proud of. Understanding why doesn’t fix it. It’s just step 1, like in AA.

kadushka 4 hours ago

From the abstract:

Results show most LLMs underperform compared to humans, except for the o1 model

  • GaggiX 4 hours ago

    It's the only reasoning model they evaluated to be honest.