A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.
Rules
This is why I always shake my head and dudebros saying "Naw bro it is/is not peer reviewed, so it's bullshit!"
Even though there are many times when the peer was wrong or outright lying to protect their pre-conceived notion or pet theory... but if you just call that the "Galileo Gambit" you don't have to take that seriously...
Never,
It’s peer review not peer verified.
English is my second language so I don’t get this post, it always meant someone else read it.
I think to some of us a review is seen as a verification of veracity.
I honestly always mistook peer review as OPs post so I guess I was 37 when I learned that...
When I have reviewed IT system design changes, my favorite comment for correct-looking changes has been "looks good, I look forward to seeing whether it works"
Agreed. Reviewing literally means just reading and making comments
I'm only a layman casual, but I have never in my life seen an actual peer review.
I've read/skimmed actual papers from primary sources whenever I actually care to try to understand the proof for something. No idea what a peer review looks like, no idea if the paper I read were ever peer reviewed.
I'm guessing maybe the publisher itself also/sometimes does the "we read it, looks fine"-process? Either way, I've never seen one. They're like some mythical creature I've only ever heard descriptions of.
Same but some of my friends i went to uni with is a moron who went on to do a PhD....
Its like having your work marked and, if they don't Iike it, they'll just say like "not clear enough" or "needs more research" and deny its publication.
I mean, what they meant was "you haven't addressed Dr Y et. al.'s critique of that particular essay's attempt at modelling the disease you're researching" but they're not just going to come out and tell you that. That would be too easy.
Every now and then I feel like I can hear them muttering some kind of highly expletive death threat at reviewer number 3.
For any scientific journal that's worth anything, your article has to get approved by other scientists in your field before the journal will accept it. They're mostly just looking for exactly what this post is referencing. Does it seem legit? If it passes a once-over by the other scientists, then it gets published.
This is why you should not trust any single study by itself. It's just the results from one experiment that easily could have had a consequential error no one picked up. The results could be statistical noise. Hell, even rarely, you'll get someone who's been faking data. This is not to say "science is broken," only that science has never relied on the results from a single unreplicated experiment to determine truth. If you read about scientists from the past, it's fairly common for them to publish a landmark paper and for no one to care, or even for people to argue they're wrong. Only with additional research do they get proved correct and we imagine that everyone immediately accepted this new paradigm shift off of one single paper.
Is there no journal/publishing site where other scientists can put out publicly visible peer reviews of a paper after the paper is already published?
A peer review really is just someone checking for glaring errors. If a paper gets published and someone had some real beef with it, best they can do is some of their own research to prove how shitty the other team was. After that, there are some journals that will publish letters where people comment on previous articles. But generally, most articles just get mildly ignored. It's only after a pattern of corroborating evidence piles up that people will start to say that the results of a particular early study were significant.
Mind you, the details about how this consensus process works varies from field to field. Particle physics has a different culture than hydrology. But, in general, one paper is not enough to hang your hat on.
I'm sorry, but this seems like a profoundly archaic, indirect, and unnecessary way to format it.
And with how brief you people seem to describe these peer reviews, they're apparently lower effort than a good reddit comment, yet they cannot be directly publicly visibly attached to the article they are directly reviewing?
Academia can't be too proud to take a hint of inspiration from the mitigating effects of well-informed internet comments and Twitter's community notes against low quality content?
Why would intelligent people shackle their own publications by simulating the limitations of last century? Separately published "letters"? Honestly?
The few times I've heard the processes of papers and journals described, they seem to be clinging to the logistical solutions of physical paper with some kind of demented nostalgic love for the flaws of it.
I think you're placing a lot more weight on the authority of a single scientific paper than any actual scientist ever would. If you have one paper, you have one paper. If you have a series of papers all put out by the same lab... maybe there's something there, maybe not. If other groups start publishing similar papers, okay this is sometime serious.
In some of the messier sciences, like medicine, people will publish meta-studies, where they combine results from similar, but independently published, papers and see what they can come up with using the combined data. People will also publish literature reviews, where they essentially try to summarize the state of the science in their particular little niche. To trust a single study in medicine is to hitch your horse to a wicket.
The peer review process doesn't stop wrong papers from getting published, just obviously wrong or bad ones. I'm not entirely sure what you could even do to stop wrong papers from making into journals, since often times the problem isn't in the published experimental design or analysis. Plus, there's some papers that used to be right, but have become wrong as things change.
they're apparently lower effort than a good reddit comment
They're not, people are being flippant. People frequently complain about having to do peer reviews specifically because it's unpaid labor. Regardless, if the paper is so wrong it would warrant a community note on Twitter, the paper would be strongly rejected. The standard for acceptance is way higher than that. Remember that it gets reviewed by fellow experts in the field. They will easily spot small errors.
Is it the best possible system? Heck if I know. It works. Moving to a different system would require everyone to recalibrate their understanding of what good science looks like. We know how to identify it under the current publication model, it would take a fair bit of time to adjust to the new one.
Edit: Oh yes, re: letters. It can take a year or more between publications. Letters might be slow, but it's not terribly important. It takes time to do science, you don't need to clap back in an instant.
In my field of research, there seems to be a recent push for artifact evaluation. It's a separate process which is also optional but you get to brag about the fact that you get badges if your experiment results were replicated.
There's also some push back against this since it's additional work, but I think it's a step in the right direction.
Was lucky to contribute to a paper for the first time recently and was certainly suprised to see what peer reviews looked like lmao
Is it better or worse than code reviews in programming? Typically, if it's 5 lines, we scrutinize everything. If it's 500 lines, it's a quick scan with a "looks good" comment.
I'd say its similar. Though from the limited dataset of peer reviews I have, I'd say that peer reviews are more informative / detailed while code reviews usually have way less typos lol.
Damn I guess I was today years old. I remember in high school chemistry class we were taught about peer review and had to do it for each other, except the way we did was actually testing and replicating results, so that cemented the misconception.
LGTM!
Science is essentially just throwing shit at the wall and seeing what sticks.
The more shit you throw, the higher chance there is that something sticks. You just need to make sure the shit is properly documented, and that's what the peer review is for?
I doubt this will stick on the wall.
Throws it to the wall and it sticks.
Holy shit!
I recently read an interesting article proposing to get rid of the current peer review system: https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review
The argument was roughly this: for the unfathomable (unpaid) hours spent on peer review, it's not very effective. Too much bad research still gets published and too much good research gets rejected. Science would also not be a weak-link problem but a strong-link problem, i.e., scientific progress would not depend on the quality of our worst research but of that of our best research (which would push through anyway in time). Pretty interesting read, even though I find it difficult to imagine how we would transition to such a system.
What did you think the "review" part of it meant other than reviewing it?
In my field, peer review was "obviously hasn't read enough Foucault".
LGTM ⛴️
Wait deadass?!?!? If so then 20 lol
Best part is the reviewers don't get paid for their work, the publishers pocket all of the money they get from selling journals
While charging researchers to publish the paper and the reader for accessing it. If they can get away with it. It's a fucking scam, thus arxiv and others exist.
I've personally had much less respect for global academia ever since I learned how publishing journals can demand so much from researchers and their audience, while providing so little.
I'm just happy they learned what peer review means. I doubt even a third of Americans know what it means or its impact on their lives
The ones that fail peer review go from "unexpected result" to "the fuck were you actually doing?!?"
Thank the greed. Even bad results should be kept. It's still knowledge. To get closer to a goal, many mistakes are made and we have to learn from them. Using the scientific method to find out that something does not work is still valuable.
This is a lesson I try to teach my kids every day. When they get upset they can't do something, I ask, "well whatd you learn?" And sometimes it's as simple as "that didn't work." Other times they think for a second they try something new.
Failure is a learning opportunity. Take advantage if it.
I never learned peer review as anything more than others in the field reviewing the paper and confirming it meets standards. Its like logic vs truth. Peer review is like proofreading. Is the structure of the experiment proper. Is there controls. Is the statistical analysis proper. so on and so forth. Honestly though science is dependent on replication which used to be a sort of competition so it worked. Oh you think this is this and this is how you proved it. Well I will see for myself and I will lambast you if it does not work. It was kinda personal with the field before modern times. Competition was very direct. Now no lab wants to do anything but something they can say is new and a discovery. I feel at least 50% of public science funding should be for experiment replication
Sounds like maybe you learned about it from some kind of actual education, not just reading about it on social media. That's cheating.
~~Is there controls?~~
Rejected.
Edit this is a petty peer review joke. Please clap
...Today years old, what the fuck? Is this how so much bunk science makes it to the front-pages of supposedly-science-related websites?
Yes. There's no prestige in spending time and money on trying to falsify other people's results, even though it's the crux of everything. People would rather spend their time working on their own discovery.