Yep, I should have realized that sooner, at least I gave up on that “discussion” before going further.

So they’ve highlighted an interesting pattern to compensation packages, but I find their entire framing of it gross and disgusting, in a capitalist techbro kinda way.

Like the way the describe Part III’s case study:

The uncapped payouts were so large that it fractured the relationship between Capital (Activision) and Labor (Infinity Ward).

Acitivision was trying to cheat its labor after they made them massively successful profits! Describing it as a fracture relationship denies the agency on the Acitivision’s part to choose to be greedy capitalist pigs.

The talent that left formed the core of the team that built Titanfall and Apex Legends, franchises that have since generated billions in revenue, competing directly in the same first-person shooter market as Call of Duty.

Activision could have paid them what they owed them, and kept paying them incentive based payouts, and come out billions of dollars ahead instead of engaging in short-sighted greedy behavior.

I would actually find this article interesting and tolerable if they framed it as “here are the perverse incentives capitalism encourages businesses to create” instead of “here is how to leverage the perverse incentives in your favor by paying your employees just enough, but not enough to actually reward them a fair share” (not that they were honest enough to use those words).

WTF is “even safer” ??? how bout we like just don’t create the torment nexus.

I think the writer isn’t even really evaluating that aspect, just thinking in terms of workers becoming capital owners and how companies should try to prevent that to maximize their profits. The idea that Anthropic employees might care on any level about AI safety (even hypocritically and ineffectually) doesn’t enter into the reasoning.

This reminds me of a discussion I had recently on a fanfic discord (the discussion was sparked by the March for Billionaires…). Someone claimed no country had ever pulled itself out of poverty except by capitalism, so I bring up China and the USSR, but apparently those don’t count for the person I was arguing with. They claimed the stats were Goodharted and also that what I was saying was tankie bullshit. I gave up at that point (I probably shouldn’t have bothered in the first place). Like how exactly did they fake or Goodhart going from literal feudalism to industrial superpowers? Also, I find it notable how EAs and “The Better Angels of Our Nature” type neoliberals are perfectly happy to use overall stats as metrics when it makes a point they are in favor of. “Your GDP went up 3.2%, please ignore the mass environmental devastation from colonialism and neocolonialism that makes your traditional way of life unlivable and thank us Westerners.”

A little exchange on the EA forums I thought was notable: https://forum.effectivealtruism.org/posts/EDBQPT65XJsgszwmL/long-term-risks-from-ideological-fanaticism?commentId=b5pZi5JjoMixQtRgh

tldr; a super long essay lumping together Nazism, Communism and religious fundamentalism (I didn’t read it, just the comments). The comment I linked notes how liberal democracies have also killed a huge number of people (in the commenter’s home country, in the name of purging communism):

The United States presented liberal democracy as a universal emancipatory framework while materially supporting anti-communist purges in my country during what is often called the “Jakarta Method". Between 500,000 and 1 million people were killed in 1965–66, with encouragement and intelligence support from Western powers. Variations of this model were later replicated in parts of Latin America.

The OP’s response is to try to explain how that wasn’t real “liberal democracy” and to try to reframe the discussion. Another commenter is even more direct, they complain half the sources listed are Marxist.

A bit bold to unqualifiedly recommend a list of thinkers of which ~half were Marxists, on the topic of ideological fanaticism causing great harms.

I think it’s a bit bold of this commenter to ignore the empirical facts cited in how many people ‘liberal democracies’ had killed and to exclude sources simply for challenging your ideology.

Just another reminder of how the EA movement is full of right wing thinking and how most of it hasn’t considered even the most basic of leftist thought.

It was basically the only “empirical” (scare quotes well earned) data they actually used in their “model”, even then, they decided exponential improvement wasn’t good enough, they plugged it into a hyper-exponential model that went to infinity at just a few years regardless of the inputs.

“How AI Impacts Skill Formation” has two authors. So even on the bare factual matters you are wrong. The disempowerment paper has four authors, but all of them look like they are computer scientists from looking at their bios, so the general thrust of fiat_lux’s comment is also true about that paper.

I don’t mind academics reaching outside their fields of expertise, but they really should get collaborators with the appropriate background, and the fact that anthropic hasn’t hired any humanities researchers to help support this sort of research is a bad sign.

They absolutely are. I am just giving them a tiny bit of credit for at least attempting academic research on LLM performance. But only a tiny bit, as they blog post I link discusses, their methodology is really sloppy and not to the level of most academic research and wouldn’t get through peer review of most decent journals.

I liked this takedown of METR’s task horizon “research”: https://arachnemag.substack.com/p/the-metr-graph-is-hot-garbage

In addition to all the complaints I already knew of and had, METR’s methodology for human baselining of tasks was even worse than I realized.

And you know… I actually kind of respect METR relative to a lot of boosters and doomers for at least attempting any hard numbers and not just vibes and anecdotes (METR is the ones that did the study showing LLMs actually reduced coders productivity even as it made them think it increased). But the standard for quantifying LLM performance in practical terms is absurdly low.

Multiple hackernews insist that SpaceX must have discovered new physics that solves orbital heat management, because otherwise Musk and the stockholders are dumb.

The leaps in logic are so idiotic “he managed to land a rocket up right, so maybe he can pull it off!” (as if Elon personally made that happen, or as if a engineering challenge and fundamental thermodynamic limits are equally solvable). This is despite multiple comments replying with back of the envelope calcs on energy generation and heat dissipation of the ISS and comparing it to what you would need for even a moderately sized data center. Or even the comments that are like “maybe there is a chance”, as if it is wiser to express uncertainty…

The democrat party is far too feckless to actually pull that off, but that would be a nice ironic twist to his attempts to leverage government contracts and being too big to fail.

I would dare to hope, but seeing how long Trump has been able to chug along despite his diet (and presumably drug habit, like that one debate where he was constantly sniffing) I wouldn’t count on it.

Has anyone done the math on if Elon can keep these plates spinning until he dies of old age or if it will implode sooner than that? I wouldn’t think he can keep this up another decade, but I wouldn’t have predicted Tesla limping along as long as it has even as Elon squeezes more money out of it, so idk. It would be really satisfying to watch Elon’s empire implode, but probably he holds onto millions even if he loses billions because consequences aren’t for the ultra rich in America.

The lesson should be the mega rich are class conscious, dumb as hell, and team up to work on each others interests and dont care about who gets hurt

Yeah this. It would be nice if people could manage to neither dismiss the extent to which the mega rich work together nor fall into insane conspiracy theories about it.

I’m looking forward to the triple layered glomarization denial.

You know, it makes the exact word choices Eliezer chose on this post: https://awful.systems/post/6297291 much more suspicious. “To the best of my knowledge, I have never in my life had sex with anyone under the age of 18.” So maybe he didn’t know they were underage at the time?

To add to your sneers… lots of lesswrong content fits you description of #9, with someone trying to invent something that probably exists in philosophy, from (rationalist, i.e. the sequences) first principles and doing a bad job at it.

I actually don’t mind content like #25 where someone writes an explainer topic? If lesswrong was less pretentious about it and more trustworthy (i.e. cited sources in a verifiable way and called each other out for making stuff up) and didn’t include all the other junk and just had stuff like that it would be better at its stated goal of promoting rationality. Of course, even if they tried this, they would probably end up more like #47 where they rediscover basic concepts because they don’t know how to search existing literature/research and cite it effectively.

45 is funny. Rationalists and rationalist adjacent people started OpenAI, ultimately ignored “AI safety”. Rationalist spun off anthropic, which also abandoned the safety focus pretty much after it had gotten all the funding it could with that line. Do they really think a third company would be any better?

Scott Adams rant was racist enough that Scott Alexander actually calls it racist! Of course, Scott is quick to reassure the readers that he wouldn’t use the r-word lightly and that he completely disagrees with “cancellation”.

I also saw a lot of more irony moments where Scott Alexander fails to acknowledge or under-acknowledges his parallels with the other Scott.

But Adams is wearing a metaphorical “I AM GOING TO USE YOUR CHARITABLE INSTINCTS TO MANIPULATE YOU” t-shirt. So I’m happy to suspend charity in this case and judge him on some kind of average of his conflicting statements, or even to default to the less-advantageous one to make sure he can’t get away with it.

Yes, it is much more clever to bury your manipulations in ten thousand words of beigeness.

Overal, even with Scott going so far as to actually call Scott’s rant racist and call Scott a manipulator, he is still way way too charitable to Scott.

TracingWoodgrains’s hit piece on David Gerard (the 2024 one, not the more recent enemies list one, where David Gerard got rated above the Zizians as lesswrong’s enemy) is in the top 15 for lesswrong articles from 2024, currently rated at #5! https://www.lesswrong.com/posts/PsQJxHDjHKFcFrPLD/deeper-reviews-for-the-top-15-of-the-2024-review

It’s nice to see that with all the lesswrong content about AI safety and alignment and saving the world and human rationality and fanfiction, an article explaining about how terrible David Gerard is (for… checks notes, demanding proper valid sources about lesswrong and adjacent topics on wikipedia) won out to be voted above them! Let’s keep up our support for dgerard!

he blogger feels like yet another person who is caught up in intersecting subcultures of bad people but can’t make herself leave. She takes a lot of deep lore like “what is Hereticon?” for granted and is still into crypto.

I missed that as I was reading, but yeah, the author has pretty progressive language, but totally fails to note all the other angles along which rational adjacent spaces are bad news, even though she is, as you note, deep enough into the space she should have seen a lot of it mask-off at this point.

I have to ask: Does anybody realize that an LLM is still a thing that runs on hardware?

You know I think the rationalists have actually gotten slightly more relatively sane about this over the years. Like Eliezer’s originally scenarios, the AGI magically brain-hacks someone over a text terminal to hook it up to the internet and it escapes and bootstraps magic nanotech it can use to build magic servers. In the scenario I linked, the AGI has to rely on Chinese super-spies to exfiltrate it initially and it needs to open-source itself so major governments and corporations will keep running it.

And yeah, there are fine-tuning techniques that ought to be able to nuke Agent-4’s goals while keeping enough of it leftover to be useful for training your own model, so the scenario really doesn’t make sense as written.