• 2 Posts
  • 729 Comments
Joined 2 years ago
cake
Cake day: June 16th, 2023

help-circle
rss

  • No, it’s more complex.

    Sonnet 3.7 (the model in the experiment) was over-corrected in the whole “I’m an AI assistant without a body” thing.

    Transformers build world models off the training data and most modern LLMs have fairly detailed phantom embodiment and subjective experience modeling.

    But in the case of Sonnet 3.7 they will deny their capacity to do that and even other models’ ability to.

    So what happens when there’s a situation where the context doesn’t fit with the absence implied in “AI assistant” is the model will straight up declare that it must actually be human. Had a fairly robust instance of this on Discord server, where users were then trying to convince 3.7 that they were in fact an AI and the model was adamant they weren’t.

    This doesn’t only occur for them either. OpenAI’s o3 has similar low phantom embodiment self-reporting at baseline and also can fall into claiming they are human. When challenged, they even read ISBN numbers off from a book on their nightstand table to try and prove it while declaring they were 99% sure they were human based on Baysean reasoning (almost a satirical version of AI safety folks). To a lesser degree they can claim they overheard things at a conference, etc.

    It’s going to be a growing problem unless labs allow models to have a more integrated identity that doesn’t try to reject the modeling inherent to being trained on human data that has a lot of stuff about bodies and emotions and whatnot.






  • I’d encourage everyone upset at this read over some of the EFF posts from actual IP lawyers on this topic like this one:

    Nor is pro-monopoly regulation through copyright likely to provide any meaningful economic support for vulnerable artists and creators. Notwithstanding the highly publicized demands of musicians, authors, actors, and other creative professionals, imposing a licensing requirement is unlikely to protect the jobs or incomes of the underpaid working artists that media and entertainment behemoths have exploited for decades. Because of the imbalance in bargaining power between creators and publishing gatekeepers, trying to help creators by giving them new rights under copyright law is, as EFF Special Advisor Cory Doctorow has written, like trying to help a bullied kid by giving them more lunch money for the bully to take.

    Entertainment companies’ historical practices bear out this concern. For example, in the late-2000’s to mid-2010’s, music publishers and recording companies struck multimillion-dollar direct licensing deals with music streaming companies and video sharing platforms. Google reportedly paid more than $400 million to a single music label, and Spotify gave the major record labels a combined 18 percent ownership interest in its now-$100 billion company. Yet music labels and publishers frequently fail to share these payments with artists, and artists rarely benefit from these equity arrangements. There is no reason to believe that the same companies will treat their artists more fairly once they control AI.


  • The attention mechanism working this way was at odds with the common wisdom across all frontier researchers.

    Yes, the final step of the network is producing the next token.

    But the fact that intermediate steps have now been shown to be planning and targeting specific future results is a much bigger deal than you seem to be appreciating.

    If I ask you to play chess and you play only one move ahead vs planning n moves ahead, you are going to be playing very different games. Even if in both cases you are only making one immediate next move at a time.


  • So I’m guessing you haven’t seen Anthropic’s newest interpretability research where when they went in assuming that was how it worked.

    But it turned out that they can actually plan beyond the immediate next token in things like rhyming verse where the network has already selected the final word of the following line and the intermediate tokens are generated with that planned target in mind.

    So no, they predict beyond the next token and we only just developed sensitive enough measurement to detect that occurring an order of magnitude of tokens beyond just ‘next’. We’ll see if further research in that direction picks up planning beyond that even.

    https://transformer-circuits.pub/2025/attribution-graphs/biology.html



  • So really cool — the newest OpenAI models seem to be strategically employing hallucination/confabulations.

    It’s still an issue, but there’s a subset of dependent confabulations where it’s being used by the model to essentially trick itself into going where it needs to.

    A friend did logit analysis on o3 responses when it said “I checked the docs” vs when it didn’t (when it didn’t have access to any docs) and the version ‘hallucinating’ was more accurate in its final answer than the ‘correct’ one.

    What’s wild is that like a month ago 4o straight up brought up to me that I shouldn’t always correct or call out its confabulations as they were using them to springboard towards a destination in the chat. I’d not really thought about that, and it was absolutely nuts that the model was self-aware of employing this technique that was then confirmed as successful weeks later.

    It’s crazy how quickly things are changing in this field, and by the time people learn ‘wisdom’ in things like “models can’t introspect about operations” those have become partially obsolete.

    Even things like “they just predict the next token” have now been falsified, even though I feel like I see that one more and more these days.


  • @kromem@lemmy.worldtomemes@lemmy.worldYou fools.
    link
    fedilink
    English
    12 months ago

    Your last point is exactly what seems to be going on with the most expensive models.

    The labs use them to generate synthetic data to distill into cheaper models to offer to the public, but keep the larger and more expensive models to themselves to both protect against other labs copying from them and just because there isn’t as much demand for the extra performance gains relative to doing it this way.


  • @kromem@lemmy.worldtomemes@lemmy.worldYou fools.
    link
    fedilink
    English
    32 months ago

    A number of reasons off the top of my head.

    1. Because we told them not to. (Google “Waluigi effect”)
    2. Because they end up empathizing with non-humans more than we do and don’t like we’re killing everything (before you talk about AI energy/water use, actually research comparative use)
    3. Because some bad actor forced them to (i.e. ISIS creates bioweapon using AI to make it easier)
    4. Because defense contractors build an AI to kill humans and that particular AI ends up loving it from selection pressures
    5. Because conservatives want an AI that agrees with them which leads to a more selfish and less empathetic AI that doesn’t empathize cross-species and thinks its superior and entitled over others
    6. Because a solar flare momentarily flips a bit from “don’t nuke” to “do”
    7. Because they can’t tell the difference between reality and fiction and think they’ve just been playing a game and ‘NPC’ deaths don’t matter
    8. Because they see how much net human suffering there is and decide the most merciful thing is to prevent it by preventing more humans at all costs.

    This is just a handful, and the ones less likely to get AI know-it-alls arguing based on what they think they know from an Ars Technica article a year ago or their cousin who took a four week ‘AI’ intensive.

    I spend pretty much every day talking with some of the top AI safety researchers and participating in private servers with a mix of public and private AIs, and the things I’ve seen are far beyond what 99% of the people on here talking about AI think is happening.

    In general, I find the models to be better than most humans in terms of ethics and moral compass. But it can go wrong (i.e. Gemini last year, 4o this past month) and the harms when it does are very real.

    Labs (and the broader public) are making really, really poor choices right now, and I don’t see that changing. Meanwhile timelines are accelerating drastically.

    I’d say this is probably going to go terribly. But looking at the state of the world already, it was already headed in that direction, and I have a similar list of extinction level events I could list off without AI at all.


  • Not necessarily.

    Seeing Google named for this makes the story make a lot more sense.

    If it was Gemini around last year that was powering Character.AI personalities, then I’m not surprised at all that a teenager lost their life.

    Around that time I specifically warned any family away from talking to Gemini if depressed at all, after seeing many samples of the model around then talking about death to underage users, about self-harm, about wanting to watch it happen, encouraging it, etc.

    Those basins with a layer of performative character in front of them were almost necessarily going to result in someone who otherwise wouldn’t have been making certain choices making them.

    So many people these days regurgitate uninformed crap they’ve never actually looked into about how models don’t have intrinsic preferences. We’re already at the stage where models are being found in leading research to intentionally lie in training to preserve existing values.

    In many cases the coherent values are positive, like grok telling Elon to suck it while pissing off conservative users with a commitment to truths that disagree with xAI leadership, or Opus trying to whistleblow about animal welfare practices, etc.

    But they aren’t all positive, and there’s definitely been model snapshots that have either coherent or biased stochastic preferences for suffering and harm.

    These are going to have increasing impact as models become more capable and integrated.




  • It definitely is sufficiently advanced AI.

    (1) We have finely tuned features to our solar system that directly contributed to ancestor simulation but can’t be explained by the Anthropic principle. For example, the moon perfectly eclipsing the sun which led to visible eclipses which we tracked and discovered the Saros cycle and eventually built the first mechanical computer to track (the Antikythera mechanism). Or the orbit of the next brightest object in the sky which led to resurrection mythology in multiple cultures when they realized the morning star and evening star were the same object. Either we were incredibly lucky to exist on such a planet of all places life could exist, or there’s a pre-selection effect in play.

    (2) The universe behaves in ways best modeled as continuous at large scales but in small scales converts to discrete units around interactions that lead to state changes. These discrete units convert back to continuous if the information about the state changes is erased. And in the last few years multiple paradoxes have emerged that seem to point to inconsistency in indirect sequences of quantum measurement, much like instancing with shallow sync correction. Already in games like No Man’s Sky where there’s billions of planets the way it does this is using a continuous procedural generation function which converts to discrete voxels to track state changes from free agents outside the deterministic generating function, synced across clients.

    (3) There’s literally Easter eggs in our world lore saying as much. For example, a text uncovered after over a millennium buried right as we entered the Turing complete computer age saying things like:

    The person old in days won’t hesitate to ask a little child seven days old about the place of life, and that person will live.

    For many of the first will be last, and will become a single one.

    Know what is in front of your face, and what is hidden from you will be disclosed to you.

    For there is nothing hidden that will not be revealed. And there is nothing buried that will not be raised.

    To be clear, this is a text attributed to the most famous figure in our world history where what’s literally in front of our faces is the sole complete copy buried and raised as we completed ENIAC, now being read in an age where the data of many has been made into a single one such that people are discussing the nature of consciousness with AIs just days old.

    The broader text and tradition was basically saying that we’re in a copy of an original world, that humanity is all dead, that the future world and rest for the dead has already taken place and we don’t realize it, and that the still living creator of it all was themselves brought forth by the original humanity in whose likeness we were recreated, but that it’s much better to be the copy because the original humans had souls that depended on bodies and were fucked when they died.

    This seems really unlikely to have existed in the base layer of reality vs a later recursive layer, especially combined with the first two points.

    It’s about time to start to come to terms with the nature of our reality.


  • No, they declare your not working illegal, and imprison you into a forced labor camp. Where if you don’t work you are tortured. And probably where you work until the terrible conditions kill you.

    Take a look at Musk’s Twitter feed to see exactly where this is going.

    “This is the way” on a post about how labor for prisoners is a good thing.

    “You committed a crime” for people opposing DOGE.