Публикация

... and introduce all sorts of new ways for machines to increase bias and errors.

Care to generalize these finings more broadly, like (to say), university administration? And then to corporate administration?

A year ago in a group I was part of the best statement about llms was this: "they do repetition very well" And I wish more people took that approach. You don't ask llm to make you a new set of plates: you have it wash the dishes in the sink so you have time to create new ones.

Does this result generalize to _any_ productivity-enhancing technology*? It reduces the friction of some component of the research process, and the "most productive" style of project moves somewhere else in this landscape. * again under the assumption that it works

Busier with less reward (for scientists at least)—as happened with other technological advances. Advances are indeed great, but workers are generally not the main beneficiary of improvements in efficiency/ productivity (the same?). Interesting to see similar conclusions. (Preaching to the choir?)

For months I've been saying the following: Think of a table made 100 years ago. Now think of one made today. The one from today is absolute garbage, but near free in comparison. It goes straight to the dump when you're done with it. Now the knowledge sector too is in the great garbage reef economy.

Very interesting stuff... I'm not really sure how much science is done in a way that conforms to your model though. I'd argue that in e.g. biomedical research a lot of researchers are doing incremental work and don't really have a "discovery period".

this is an aspect of enshittification not explicitly covered by cory doctorow

I’ve heard people I know who are software engineers say that adding AI tools has actually caused them to overwork themselves. They have a lot more code to review and deal with, but also just the ability to be able to seemingly produce more is somehow driving them to overwork themselves.

How would considering costs and errors change things? At a guess for experimental research: true costs would dramatically reduce LLM use; while LLM errors will contaminate analyses/results/discussions in weird ways.

…which of course leads to the question “so then is it really ‘more’ at all?”

The ultimate example of what happens when you pursue constant improvements in production efficiency. When you cherish monetary results over product quality. When you employ a system of unrestrained capitalism.

Side bar: We (not me) are doing it for the privledged class who will like always reap the benefits, but yeah, continue on the same old process. We still got a little time before we completely destroy civilization. JFC!

fast forward a few days to this reveal from Hank Green bsky.app/profile/mxfe...

If LLMs are helping scientists decide what to try and research, won't that also bias those choices towards what is most obvious in training data? Unless a scientist specifically asks for things that aren't popular? And even then, won't the options be skewed by the content of training data?

Every technology that brings more efficiency & productivity inevitably puts most of the individual humans into yet another version of the Red Queen’s race, being required to run ever faster just to stay even.

So we need metrics that specifically penalise “more but worse” in output. E.g. lab 1 produces 10x more papers than lab 2 but the papers from lab 2 are cited 10x more. 2👍 Or: lab 1 produced 100 papers and all of them were cited below the expected median for the journals in which they appeared. Fail!

It seems LLMs are better suited (at best) to generating broad literature reviews than extracting the critical knowledge that informs better research design

It’s not just science, it’s numerous industries. The pressure to produce “more and more and faster and faster” is the EXACT complaint of a TV production designer I know who has to keep up with the competition by using AI design tools in their job.

I think we have reached this point already. I see a lot of papers (maybe 5%) nowadays with half of the reference list being hallucinated. As nearly all of these authors do not declare the use of #AI tools, I can only assume that the numbers in these papers are also just made up figures.

Isn't that what has happened with modern science anyway? If so, it would be an acceleration of trends caused by other factors...

Thanks for this. Very interesting. And in line with a lot of technology over the decades, I think, that has paradoxically led to less downtime, not more

I know many folks at Amazon who say precisely this for their work, too.

May be a hidden gift. LLMs are the distillation of the obsession with “knowing what”. Answers above all. We’re confronted with the explicit lack of concern for “knowing how”. Will we act on this provocation to care?

Once again, the Cult of Convenience buggers everything.

this sounds a bit like what @hankgreen.bsky.social has been dealing with in his work, where he just kept going faster and faster with LLMs.

I can remember the director of the IGC, Antonio Coutinho, already in 2000 or so, talking about his mentor, Nils Jerne, proposing a radical cap: at the conclusion of a PhD, each researcher gets 10 publication tickets, and that's it :) So in the 70s they were already worrying about this...

Absolutely! It is essential that we find some way of reducing the publication volume even if, as you note, this particular proposal seems unlikely to succeed. We've seen some progress in terms of shifting evaluation criteria, but a lot more is needed, fast.

This is a huge issue too—and one that we don't even address in the present paper. We're looking at short-term responses and so we treat as exogenous the benefits of producing a given result to a given degree of thoroughness. But as the literature becomes further saturated, those benefits change too.

The key turns out to be where in the research pipeline the LLMs offer the most labor augmentation: 1) Before or after you have a good sense of the value of the project if completed? 2) If after, on preparing the minimum viable manuscript or on various forms of "polish"?