Sjors Provoost's avatar
Sjors Provoost
sjors@sprovoost.nl
npub1v25j...exw3
https://github.com/sjors/ Book: https://www.btcwip.com Podcast: https://bitcoinexplainedpodcast.com/@nado/episodes
Sjors Provoost's avatar
Sjors Provoost 12 hours ago
On rare occasions they run a steam train to the museum. Fun, albeit useless, fact: it used to take the southern way around. #trainstr
> The authoring agencies recommend U.S. AI companies take three immediate actions: > 2. Deploy targeted response changes: Subtly alter responses for suspected malicious distillation attempts to attenuate the payoffs to companies conducting industrial-scale distillation campaigns. In other words: poison competitor models, and potentially harm falsely flagged legitimate users (and get sued if they find out). When reading, replace "China-based AI companies" with "US based competitors" and understand what that implies in a future where the big four can no longer undercut competitors with mountains of investor cash.
> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage) And who pays for that botnet's tokens? And then decides to just let it keep racking up bills? > ... even when this gets us accused of hype, “doomerism”, or regulatory capture. I wonder why... > The first step is something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match). > The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible [...] Narrator: it's not. We probably shouldn't have immature doomsday cult members in charge of very powerful technology, but that's where we are, and I'm still looking forward to cheap RAM when the bubble pops.
The title really undersells it, tl&dr: - "thinking" text is useful for humans to analyze what a model is up to (I often find it more useful than the final burst of slop) - this text is fed back through the model, which helps it with the next step (they give a nice example) - but text is a lossy projection of the last tokens, what if you skip it? - it's more efficient, great, but now you can't see what it's about to do until it does it - ok, so force it to write "thoughts" e.g. once every 10 rounds - turns out if you skip too many rounds, it can intentionally trick you with fake thoughts I suspect there's actually a simple solution: convert these skipped rounds to text anyway, just don't feed them back into the model. (via @m0wer)
I agree with Mark Karpelès. You heard it here first. > The mentioned government agency has not been identified and other financial institutions or cryptocurrency exchanges could have been misled into disclosing their customer details through the same channel. I believe it is important for Revolut to urgently disclose information about the agency in question (most importantly, which was the email address used), or for the agency itself to come out about this case. In addition there should be an investigation into criminal negligence in that government agency. But naming and shaming is a good start. It doesn't absolve Revolut one inch of course. They committed the KYC and they leaked the data. Customer blood is on their hands.
"KYC is Kill Your Customer" (ok, @FRANCIS - BULLBITCOIN.COM said Citizen) > Revolut received a request for information disguised as a legitimate government agency request. > identity documents, facial verification selfies, IBANs, and complete transaction histories, including Bitcoin, left Revolut’s hands (ht @Patrick van der Meijde)
While hackers use AI to plunder databases faster than ever, governments think it's a great idea to create even bigger data honeypots - secured by the lowest bidder. And then give hackers a decade.