this post was submitted on 01 Feb 2024
1 points (100.0% liked)

SneerClub

983 readers
5 users here now

Hurling ordure at the TREACLES, especially those closely related to LessWrong.

AI-Industrial-Complex grift is fine as long as it sufficiently relates to the AI doom from the TREACLES. (Though TechTakes may be more suitable.)

This is sneer club, not debate club. Unless it's amusing debate.

[Especially don't debate the race scientists, if any sneak in - we ban and delete them as unsuitable for the server.]

founded 1 year ago
MODERATORS
 

OpenAI blog post: https://openai.com/research/building-an-early-warning-system-for-llm-aided-biological-threat-creation

Orange discuss: https://news.ycombinator.com/item?id=39207291

I don't have any particular section to call out. May post thoughts ~~tomorrow~~ today it's after midnight oh gosh, but wanted to post since I knew ya'll'd be interested in this.

Terrorists could use autocorrect according to OpenAI! Discuss!

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (4 children)

I guess there both are no real biochemists (or whatever the relevant field is), nor well read cybersecurity people (so they know a little bit more than just which algorithms are secure and why mathematically) working at openai as this is a classic movie plot threat. LLMs could also teach you how to make nuclear weapons, but getting the materials is going to be the problem there.

(Also I think there is a good reason we don't really see terrorists use biological weapons, nor chemical weapons (with a few notable, but not that effective exceptions), big bada boom is king)

[–] [email protected] 0 points 9 months ago (1 children)

Even if one had the means necessary to carry out a bioterrorist attack, simply bombing a place is much easier, faster and safer.

[–] [email protected] 0 points 9 months ago

Yeah and also, terrorists are not genocidal death cults. 'terrorists skip getting microbiology phd using chatgpt to create a pandemic that kills untold numbers of beings' is pure fantasy, it gets worse as it turns out that the number of actual bioterrorists deaths in total ever isn't even on the level of a 9/11. People seem to forget that terrorist groups have goals, and they just use terror/violence as a method to reach those goals, sure a few of them may die [chatgpt insert a gif of Bin Laden dressed as Lord Farquaad] but the goal of the terrorist organisation is to keep existing to reach their political goals.

[–] [email protected] 0 points 9 months ago (2 children)

Terrorists have used biological and chemical weapons to large effect. The barrier to entry has never been lower and gets lower by the day. This isn't a non issue, it's a very real concern for CBRN groups, both counter terrorism and state actors.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8818129/

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7855324/

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (2 children)

I think what’s going miss here is that “CBRN groups” is very obviously and primarily shit made up by the military-industrial complex to justify itself after the Cold War

I don’t want to be dismissive of genuine attempts at being ready just in case, but the scale and scope of this is defined by politics, not by technical possibility

[–] [email protected] 0 points 9 months ago (1 children)

i mean, i don't blame them, ritual dogfighting for congressional attention and money has became an art form. but the primary realistic concern of bioweapons preparedness would be, from what i understand, use of biological weapons by state actor, and i don't really see a scenario where this happens before nukes start flying

[–] [email protected] 0 points 9 months ago

100% agreed, what terrifies me is that our friend here seems to see the word “science” in here and immediately assume impeccable faith and perfect knowledge

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

I think the claim that cbrn is made up to self justify needs a lot more justification than you're giving it. It's just a profoundly confusing claim. They didn't issue mopp4 in Syria for nothing...

And whether or not you think nuclear weapon proliferation is a problem, it's hard to claim CBRN anti proliferation efforts are just a made up excuse to exist, it's a very real reason to exist as a program concept. Maybe you wouldn't have one if you could decide to, but that's a far cry from whatever you seem to be claiming.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

I expressly put “CBRN groups” in scare quotes to tag along with my line at the bottom “I don’t want to be dismissive of genuine attempts…but the scale and scope of this is defined by politics, not by technical possibility

You, however, have me saying “cbrn is made up to self justify” - of course if I had said any such thing, then one counter-example would have sufficed. Although actually it wouldn’t have sufficed, because in this context we’re talking about terroristic or otherwise chaotic release of a novel weapon. We’re not talking at all about bad powerful people deliberately employing chemical weapons they already have, for which of course CBRN is a worthy use and “genuine attempt at being ready”.

“CBRN groups”, here, operates at the level of rhetoric, and that’s what I tried to draw attention to. The context in which “CBRN groups” the rhetorical and political device emerged was that in which Bill Clinton could become so enthused by a sci-fi novel about bioterrorism that he had its author up in front of the senate testifying as an expert on the subject. So on reflection, I should have deferred to Eisenhower’s original formulation: the military-industrial-congressional complex.

[–] [email protected] 0 points 9 months ago (2 children)

Incredible gymnastics to bend over backwards to interpret my response as one which doesn't address what you say, when I specifically ask you to expand on what you mean and justify it. Ambiguous language doesn't make you clever. Truly a discourse for the times.

So your problem is with politicians using fearmongering. Sure. That's always frustrating, using fear mongering top drum up support has been a political passtime since politics.

I was not however, referring to fear mongering politics, but the practical and technical application of CBRN as a program and the actual, real, issues with bioterrorism and state bio weapons programs. Glad you got that soapbox out of your system though.

Bio and chemical terrorism are hardly akin to nuclear weapons. Refining uranium at any rate that could produce a bomb in someone's lifetime takes industry that must be hidden at a state level.

This is simply not true of chemical and bio warfare.

[–] [email protected] 0 points 9 months ago (1 children)

maybe SneerClub is the wrong outlet for whichever ax you’re currently grinding

[–] [email protected] 0 points 9 months ago

I had fine discussions with others here and in the past. This particular poster wants to soapbox and dismiss rather than engage.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

If I may refer you back to the book cited, the (made up) fears of that time in fact incorporated the difficulty of obtaining fissile material during that period, when amongst the worries was that obtaining fissile material would not actually be that difficult. To simply state that biological and chemical warfare bear no resemblance is to depart from the lesson being related here to making excuses for that object of which you happen to be afraid. In each case the fear being constructed will make its own allowances for the real or supposed facts on the ground, and in this case there was no need to assume that a bombmaker would have to make his own plutonium - you’re drawing attention to an irrelevant distraction.

Another point which you’re glibly avoiding, with tellingly unnecessary recourse to insulting language, is that “CBRN” the construct cannot be so easily distinguished from the “practical and technical application” that the real enterprise has. Indeed the existence of the real enterprise is often driven in part by the made-up fears (which does not licence the fears) - this happened, for example, with security protocols around the management of fissile material. I refer you back to the same book and to the rather famous data point about Bill Clinton’s interest in manufactured diseases.

For more on stuff like this, although again not on the subject of bioterrorism because I don’t have that material in front of me, I recommend the confluence of two chapters in The Merger of Knowledge with Power by Rabitz (as well as the whole book), namely “Recombinant DNA Research: Whose Risks?” and “Hardware and Fantasy in Military Technology”. This isn’t paranoid soapboxing from a teenage Chomsky fan, it’s just part of the fabric of industrial science and technology as a social phenomenon.

[–] [email protected] 0 points 9 months ago (2 children)

happen to be afraid

Again, I'm not.

fissile material easy to get

Right so fundamentally getting the ingredients for chem and bio warfare is objectively easier than fissile material. To dismiss them as the same implies you don't realize you almost certainly have the ingredients to make a substantial amount of chlorine gas sitting in your home right now.

Yes bio is a bit harder than that, but not as much as you might think. Anthrax is a common soil bacteria. Ricin from grain. Isolating specific bacteria takes time and is sloppy, sure, but doable in a garage. Not easy, not something we should simply brush off, either.

Ultimately, you're not going to be convinced. You want to paint something as the same ol false fear instead of a developing threat from genuine technological improvements that you are potentially not aware of. Oh well.

You can ramble about the politics of politicians and CBRN all day if you want, it won't be responding to the focused discussion I was having about the practicality of bio warfare though.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

Look, if you want to understand where I’m coming from, I can give you - at a glance - the two sources I’ve already given: Alex Wellerstein Unrestricted Data and J.R. Ravetz “The Merger of Knowledge with Power”, specifically the two chapters cited (the Wellerstein chapter is “Unrestricted Data: New Challenges to the Cold War Secrecy Regime”. I would also urge you to check out Lisa Stampnitzky Disciplining Terror. The introduction to the Ravetz book is also a must-read, not just for this, but also for a general understanding of how scientific research at the industrial level serves political and sectoral interests of all kinds - this is not radical pamphleteering about “the politics of politicians” but real empirical work about the real conditions under which science is done.

Stampnitzky is extremely useful here for understanding how the word “terrorist” (or similar) functions in the sorts of papers you cite at the very top. “Terrorist” and “state actor” are political words, and the risks (supposedly) measured which are attached to the threats you describe are weighted by those words, not by the scientific words pertaining to technical capability. To say that “terrorists” might get hold of this or that technology is to say that a particular type of person (who may or may not exist) will get hold of that technical capacity and make use of it.

The point is, in fact, that technical capacity has almost nothing to do with the measurement of risk from terrorist acquisition of that technical capacity. The measurement of risk is locused pretty much exclusively around the type of person who poses a threat. That type of person is a construction of politics, not a scientifically neutral object term in which people with medical or physical science qualifications have any expertise whatsoever.

To put it extremely briefly, this means that when you come across papers by CBRN professionals assessing speculative risks, much of the work being done is being done at the behest of political projects which have their home in the defense industry, not in assessment of the mere technical capacities available to people at large. As we learn from Ravetz, speculative risk created such an enormous bubble during the Cold War that it is almost impossible to take those measured risks remotely seriously - and as we learn from Stampnitzky, the idea of a “terrorist” has been constructed in such a way as to fuel that bubble. This means that CBRN professionals, however unimpeachable their contributions to the amelioration of those occasional disasters which do actually happen, are thoroughly questionable as unbiased witnesses to the scale that risks at large present.

Because, as your own inconsistencies show, you are not having a focused discussion (for example: you angrily claim in your second reply that in your first you asked me to expand on an earlier point, even though this never actually happened) it is extremely difficult to get this point across without appearing to just be dismissive of technical capacity as a factor. But in fact technical capacity has been factored in to my discussion this entire time. The fact that you’re unaware of the political environment in which your (non-)fear finds its sources is not anybody’s fault, but it is your fault if you don’t even acknowledge that other people might have a clearer idea about how this stuff works.

[–] [email protected] 0 points 9 months ago (1 children)

Look, ultimately why I'm focusing on technical capacity is because I don't disagree on the political side of this. I think it's highly unlikely that anyone would use bio weapons or develop them soon, and it's definitely true that it's inextricably political. My point is ultimately that bio weapons are getting easier and easier to produce and it's not a non issue to consider. The top comments here were more than happy to dismiss bioweapons wholesale as even a thing that could happen.

I'll check out your book recs. I'll point out that the main book on this topic folks have cited is ten years old now, and we're experiencing a legitimate bio tech revolution right now. How much further it'll go is anybody's guess.

[–] [email protected] 0 points 9 months ago (1 children)

I understand that you’re olive branching me here, but I don’t accept “How much further it’ll go is anybody’s guess”. Trends are analysable, and the sources of projections are equally analysable. A book that’s ten years old is far better than (a) 30-year old (and more) newspaper-level stuff, without citations, about backyard anthrax, (b) nothing, and (c) two links to tangentially related reports, and you’ve brought those three.

I am seriously concerned about the confluence of two things: (1) how closely your comments here mirror, right down to the level of language, press releases and opinion columns paraphrasing press releases, some of them (the anthrax stuff) extremely old hat; (2) the level of outrage and confidence you bring to the table when challenged on this and similar. Phrases like “How much further it’ll go is anybody’s guess” are press release language - they have absolutely no place in serious discussion, but they have a powerful rhetorical effect which allows them to displace serious analysis, and that displacement furthers specific, analysable, interpretable sectoral and political interests.

The same goes for “bio tech revolution” - you are never clear, in any of this, what that actually entails. What you do is cite possibility and unknowability, in a manner innovated precisely by sectoral and political interests from the 1950s onward. You have no detail of any value, and you write off actual detail with speculation and glib remarks about the age of the detail you’re given - that is a political innovation to which you have allowed yourself to be susceptible. You may also try on Naomi Oreskes for size as an author who grapples with this in both directions.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (2 children)

Sure a book that's ten years old is better than 30 year old papers. Doesn't mean that it can't be wrong, especially in the face of technological change. Digging into how modern biotech is constantly breaking ground is difficult, there's no one thing happening. Biocrystallization is used commercially, we produce drugs using modified organism as, we're understanding genetics at genome network/hologenetic levels like never before with new computational and statistical tools. Immune system level effects are getting easier to control. In part I'm not being specific to not oust synthetic biologist friends of mine, in part because the conversation hasn't called for it, and in part it's not my field, just close to it. (Biomedical)

Technological trends at this scale are notoriously impossible (in fact, there's a whole field of research on it trying to figure out how to predict breakthroughs for funding reasons) to predict. We could find out tomorrow that genetics is incomprehensibly complicated in a way that defies most of the use cases for bio tech that we'd like, or alternatively, we find new tools which incrementally make it easier to predict phenotypic effects from genotypic changes. My bet is on the latter, but the prior isn't a ridiculous position to hold.

Here's a cool presentation about where macroscopic synthetic biology is at: https://m.youtube.com/watch?v=1qRIetbuoH4

It's not for bioweapons strictly, my interests in this field tend to be more about macroscopic synth bio, but perhaps illustrative of why this field is not easily summarized and what sorts of leaps have been made and what challenges are obviously remaining.

Your comments on my rhetoric are frankly, rhetorically speaking, grasping at straws. Vagary isn't always a conspiracy developed by powers that be. Sometimes its because the person you're talking to feels you're arguing in bad faith, is tired when they have opportunities to respond, and isn't planning on writing a white paper for your digestion. This isn't a summit on bio warfare, this was me going "hey bio weapons aren't a technological dud" and you deciding to be "seriously concerned" about someone having the audacity to disagree with your stance. Feel free to elaborate on your "serious concern" or don't. We are here to discuss different things, and you're showing nothing but adversarial participation.

[–] [email protected] 0 points 9 months ago

Your comments on my rhetoric are frankly, rhetorically speaking, grasping at straws. Vagary isn’t always a conspiracy developed by powers that be. Sometimes its because the person you’re talking to feels you’re arguing in bad faith, is tired when they have opportunities to respond, and isn’t planning on writing a white paper for your digestion. This isn’t a summit on bio warfare, this was me going “hey bio weapons aren’t a technological dud” and you deciding to be “seriously concerned” about someone having the audacity to disagree with your stance. Feel free to elaborate on your “serious concern” or don’t. We are here to discuss different things, and you’re showing nothing but adversarial participation.

you were warned, but you still filled a SneerClub thread with debatebro garbage, and the last person deriving entertainment from it has given up. your contributions won’t be missed

[–] [email protected] 0 points 9 months ago (1 children)
[–] [email protected] 0 points 9 months ago (2 children)
[–] [email protected] 0 points 9 months ago

If the last act of the human race is to raise a forlorn statue of that woman in every town square it will be a fitting end

[–] [email protected] 0 points 9 months ago

Nice. Any specific recommendation? Interesting titles.

[–] [email protected] 0 points 9 months ago

“focused discussion”…right

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

come the fuck on, there's zero chance some crackhead cultist or other jihadist breaks out CRISPR kit in their dusty garage trying to make microbiological deliverance happen

if you wanna be afraid do what you want, i'm not gonna forbid you, i'm not your dad. but the intro section reads like some semi-palatable drivel that you include in order to justify your grant expenditures

[–] [email protected] 0 points 9 months ago (2 children)

breaks out CRISPR kit in their dusty garage

I mean, it's genuinely not hard. This reads to me more like assuming all terrorists are fundamentally incapable of anything remotely intelligent, which is both silly and not the official position of CBRN experts. From smaller cultists to state actors, bio warfare is a genuine concern.

if you wanna be afraid

I'm not.

justify your grant expenditures

What grants do you think I'm getting?

Your comment sounds to me like lashing out about something because you want to assume every last thing you're sneering at is wrong, when really the thing you're sneering at is wrong in methodology and conclusions but not in the origin of a problem wholesale.

[–] [email protected] 0 points 9 months ago (1 children)

This reads to me more like assuming all terrorists are fundamentally incapable of anything remotely intelligent

The first paper you linked there lists 9 deaths and 806 injuries across 50 years. Conversely, you can look at a single example like the Manchester Arena bombing in 2017 and see deaths and more injuries from a single event using simple techniques where materials and instructions are readily available. It isn’t unreasonable to look at the lack of success of amateur biological and chemical attacks and assume that plausible future attackers will be intelligent enough to simply take the tried and tested approach.

On the other hand, there might be some mileage in hyping up the threat of diy countertop plagues in the hopes that would-be terrorists are as credulous as so many politicians and media figures are, and will take the pointlessly inconvenient and inefficient option which will likely fail and make life a little safer for the rest of us.

[–] [email protected] 0 points 9 months ago

tried and tested

Nobody is saying terrorists won't keep using conventional bombs. Terror attacks aren't just about maximum kills nor casualties per dollar, however, and as the barrier to entry lowers and lowers it's important to consider ramifications from many technologies.

hype them up to fail

This does not seem a reasonable countermeasure when the risk of failure is potential pandemics.

[–] [email protected] 0 points 9 months ago (1 children)

I’m not.

Well, do what you want

What grants do you think I’m getting?

I meant authors of that paper, sorry if i was unclear about it

I mean, it’s genuinely not hard.

like i said before,

while someone who got all the way past about the first semester of organic chemistry lab is perfectly capable of making some rudimentary chemical weapons, they won’t necessarily be able to make it safely, reliably, cheaply, consistently, and without killing themselves,

but with biological weapons stakes are much higher, every single leak carries risk of ending up dead or being discovered and safety requirements are gonna be generally much more stringent than with chemical weapons. you can get away with using small amounts of something that would plausibly pass for a ww1 era chemical weapon with only nitrile gloves and good fumehood; with biological agents you're probably looking at doing about everything in glovebox. to use glovebox, you need to get glovebox, which, among other purchases, can move such person from government watch list to government act list

and even ignoring that, you can't just expect any random jihadi joe to make it work, you need someone who has some actual education and preferably expertise in microbiology, which if anything else severely limits poll of potential perpetrators

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (2 children)

The equipment and ppe for bio weapons and chemical weapons of the same health hazard is about the same. The only difference with biological weapons is you're doing stuff with fridges, incubators, agar, and petri dishes, rather than beakers, Bunsen burners, and filters.

In either case your logic is relying on a threatening actor to not have any education. Sure, the pool of candidates is lower for sophisticated say, anthrax, something you can almost trivially find in dirt, but it's also lower for sophisticated chemical weapons like say, sarin. And keep in mind, yes it's hard to do biology or chemistry, but devoted individuals do it in garages, for often innocuous reasons. You can't just assume some terrorist group will never have a strongly devoted individual or group who are competent enough to pull something off, you need to have preparedness. (In the form of local procedures, drills, and organization and plans and equipment to respond to threats as they develop, along with preventative measures)

Also make no mistake, spotting lab scale chem and biological warfare production is extremely difficult. Even moreso for biological production, but both resemble conventional labs (and could be!). Where biological becomes an issue is that lab scale production of a pathogen can self propagate in a way chem attacks or bomb attacks can't.

I'm not saying to be afraid, the barrier to entry on all weapons production is the lowest it's ever been, but sophistication in preventing them is also quite high. But it's not something that can just be brushed away, it's a real problem that real professionals are continuously solving.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (2 children)

The equipment and ppe for bio weapons and chemical weapons of the same health hazard is about the same.

well i think it strongly depends on your threat model. consider small leak of sarin: effects are detectable in seconds to minutes, antidote is readily available, cheap and specific, and sarin poisoning is not transmissible. in case of, say, plague, you won't know what's going on for days in which time there's already a risk of infecting some random passerby, which is highly suboptimal if you want to stay covert

In either case your logic is relying on a threatening actor to not have any education

it's not that i assume no education, it's that i (as an organic chemist) wouldn't trust crystallographer or electrochemist with synthesis of something like sarin. even with all required PPE and other precautions, your recruiting poll drops from about 100% (IED carrying child soldier) to maybe 1-0.1%? and that's even before you consider that some of these highly specialized chemical weapons people are already on military payroll, or are surveilled precisely for this reason

you're underestimating cost of this entire enterprise, which even at lab scale could easily go into hundreds of thousands to million dollar range. you're underestimating how hard it is even when you have everything provided - look at iraqi chemical weapons program. with no need to stay particularly covert they were only able to manufacture mustard gas of useful quality that could be stored; their mid tier chemical weapon sarin was at something like 30% purity and had very short shelf life; their vx was so dirty it was straight up useless

for some weird reason you're assuming that whatever chemistry you want to do, it works on the first try. it won't; it never does, and even if it did, you have to make sure you've got the right stuff. this makes synthesis only half of the problem, because there's still purification and analysis

you seem to ignore that even in the paper that you cite, anyone that doesn't have to do chemistry, doesn't. (by that i mean performing some reaction, that generates side products, and so requires purification, analysis, and generates waste stream). doing chemistry means generating waste and need of its safe-ish disposal; it means getting considerable PPE; it means getting precursors, maybe in large amounts; all of that might move you from government watch list to government act list.

talib doesn't do chemistry when he makes an IED, because melting down contents of TM-62s or UXO found in nearby field isn't chemistry; unabomber stuffing match heads in a pipe isn't chemistry; stealing cylinders of chlorine (bulk of fatalities in that paper) and putting it in a car bomb isn't chemistry. chlorine is not something you make and put in cylinders, because it's relatively hard, uses large amounts of energy, leaves considerable waste/side product stream, and you can order it on aliexpress. same goes for sulfur mustard, i'm pretty sure most of incidents happened in syrian civil war and ultimately this stuff can be traced to syrian or iraqi chemical weapons program

most of these problems, but especially making sure you've got the right stuff, are much harder for living organisms than for clearly identifiable, publicly known compounds. and we're still nowhere close to the point where llm gets potentially useful. no, getting B in high school biology and relying on gpt4 and scihub to get all the way up there doesn't count. chatgpt writing out rna sequence to be printed out, engineered into a bacterium and spread by a cultist, all done by mail order and or in garage is scenario completely detached from any pretense of being realistic

beakers, Bunsen burners, and filters

this tells me that you've ended all contact with chemistry on (classical, aqueous) qualitative inorganic analysis, because if you tried to cook anything on bunsen burner in organic lab, that'd be pretty hard considering there are none in flammables area. have you considered that you're severely out of your depth and got caught in openai's fear based hype-marketing?

e: if you want to isolate anthrax from dirt, you'll have many more problems than that, especially with "getting the right stuff" part. there are places where anthrax is endemic, but if step 1 involves catching diseased marmot in southern mongolia or deer in eastern siberia, this devolves straight into rube goldberg machine of mass destruction area

[–] [email protected] 0 points 9 months ago (1 children)

Re: ppe

Sure, the outcomes are different, but the scale is too. The scale of a chemical weapons program is necessarily higher from a hazard point of view due to the sheer volume of material. The specifics make that messy though, yes, any particular pathogen would want differing levels of ppe.

Re: precursors

Right, the part of the point I've made about bio weapons is that spotting the precursors is very difficult, because a normal bio lab needs roughly the same stuff a weapons bio lab does.

I don't disagree that many of the chemical weapons used in Syria may be from larger chemical weapons programs. But that doesn't mean lab scale ones don't also exist.

Re: ended contact with chemistry

Not everybody is trying to posture. The point wasn't to show off a magnificent knowledge of lab equipment, but to demonstrate the similarity at a high level.

Re: llms for biology

Ehhhh there are plenty of research applications of llms at the moment and at least one is in use for generating candidate synthetic compounds to test. It's not exactly the most painful thing to setup either, but no if you were to try to make a bio weapon today with llm tools (and other machine learning tools) alone it would go poorly. Ten, twenty years from now I'm not so sure, the prediction horizon is tiny.

Re: caught in openai fear

Why would I consider that when my opinions on bio weapons and CBRN are wholly unrelated to openais garbage? I didn't even know openai cared about CBRN before today and I fully expect it's just cash grabbing.

People can abuse and misinterpret real concepts in a way that makes them seem absurd.

Yes in practice anthrax is nontrivial. But folks here also seem to think any of this is magically impossible, and not something that dedicated people can reasonably do with fewer resources by the day. Which by the way is great, the surge of amateur micro bio is great, we're learning a lot, we're getting very smart hobbyists contributing in unexpected ways.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago)

right, sure. there are very few labs that require select agents, and running an anthrax vaccine program out of your backyard is a hobby i haven't heard of yet

lab scale cws are just that, lab scale. multi-kg amounts are not lab scale, and unless you're running around with suspiciously modified umbrella, sub-gram amounts aren't casualty-producing

The scale of a chemical weapons program is necessarily higher from a hazard point of view due to the sheer volume of material

you can't grow some random bacteria in high concentration, you're looking at tens to thousands of liters of fermenter volume just to get anything useful (then you need to purify it, dispose of now pretty hazardous waste, and lyophilize all the output, it gets expensive too)

The point wasn’t to show off a magnificent knowledge of lab equipment, but to demonstrate the similarity at a high level.

for the reasons i've pointed out before, there's none to very little similarity

Ehhhh there are plenty of research applications of llms at the moment and at least one is in use for generating candidate synthetic compounds to test. It’s not exactly the most painful thing to setup either, but no if you were to try to make a bio weapon today with llm tools (and other machine learning tools) alone it would go poorly.

it's nice that you mention it, because i've witnessed some "ai-driven" drug development firsthand during early covid. despite having access to xrd data from fragment screening and antiviral activity measurements and making custom ai just for this one protein, the actual lead that survived development to clinical stage was completely and entirely made by human medchemists, atom by atom, and didn't even include one pocket that was important in binding of that compound (but involving that pocket was a good idea in principle, because there are potent compounds that do that), and that despite these ai-generated compounds amounted something like 2/3 of all tested for potency. but you won't find any of that on that startup's page anymore, oh no, this scares away vcs.

Ten, twenty years from now I’m not so sure, the prediction horizon is tiny.

i'm equally sure that it'll go poorly then too, because this is not a problem you can simulate your way out of and some real world data would need to get input there, and that data is restricted

But folks here also seem to think any of this is magically impossible, and not something that dedicated people can reasonably do with fewer resources by the day

yeah nah again. lately (june 2023) there was some fucker in norway that got caught making ricin (which i would argue is more of chemical weapon), because he got poisoned in the process, with zero fatalities. [1] around the same time single terrorist incident generated about the same number of casualties and much more fatalities than all of these "bw terrorism" incidents combined. [2] this doesn't make me think that bw are a credible threat, at least compared to usual conventional weapons, outside of nation state level actors

at no point you have answered the problem of analysis. this is what generates most of costs in lab, and i see no way how llm can tell you how pure a compound is, what is it, or what kind of bacteria you've just grown and whether it's lethal and how transmissible. if you have known-lethal sample (load-bearing assumption) you can grow just this and at no point gpt4 will help you, and if you don't, you need to test it anyway, and good luck doing that covertly if you're not a state level actor. you also run into the same problem with cws, but at least you can compare some spectra with known literature ones. at no point you have shown how llms can expedite any of this

you don't come here with convincing arguments, you don't have any reasonable data backing your arguments, and i hope you'll find something more productive to do over the rest of the weekend. i remain unconvinced that bw and even cw terrorism is anything else than movie plot idea and its promise is a massive bait against particular sector of extremists

[–] [email protected] 0 points 9 months ago

in context of Iraqi insurgency even things like EFP plates were industrially manufactured in Iran and shipped there by their special forces, even that it's just a chunk of copper plate pressed in shape of shallow cone. same in Afghanistan, where friendly CIA/ISI agent, or friendly black market weapons trader depending on period would provide them with explosives, fuzes, communication hardware, training and some modern weapons up to and including FIM-92 Stingers

[–] [email protected] 0 points 9 months ago (1 children)

Have you read Barriers to Bioweapons?

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

I have not. Is it good?

Keep in mind that it was written in 2014, the Field of bioengineering in the past ten years has advanced considerably.

[–] [email protected] 0 points 9 months ago (1 children)

Yeah. It addresses your points I think.

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

I'll have to check it out.

The general point seems to be yours, that intellectual availability is the largest restriction on bioterrorism. I don't disagree, but a big part of my argument is that access to this information has never been higher (which is better than not for a variety of reasons) and access to resources usable for this has never been higher. We have plenty of garage scale bio labs as it is. So yes, the biggest limit is availability of people with knowledge to do it, that's not a hard roadblock, at least not anymore.

And the prediction horizon on biotech is tiny. Give it another ten years? Twenty? It's not a zero threat because nobody has done it right now yet.

[–] [email protected] 0 points 9 months ago

Not just intellectual availability, but the complexity of the job itself. iirc it goes into the Russian experience.

[–] [email protected] 0 points 9 months ago

To be clear: it is all movie plot threats. At the very forefront of the entire “existential threat” space is nothing but a mid-1990s VHS library. Frankly if you want to understand like 50% of what goes on in AI at this point my recommendation is just that you read John Ganz and listen to his podcast, because 90s pop and politics culture is the connective tissue of the whole fucking enterprise.

[–] [email protected] 0 points 9 months ago (2 children)

the relevant field would be microbiology. while someone who got all the way past about the first semester of organic chemistry lab is perfectly capable of making some rudimentary chemical weapons, they won't necessarily be able to make it safely, reliably, cheaply, consistently, and without killing themselves, and universities most of the time put enough sense in everyone's head to not do that. this strictly requires that you know anything about chemistry, too. for bioweapons every single problem pointed to above is orders of magnitudes worse, and you probably need masters degree to do anything seriously nefarious. then you get into the problem of using that stuff, and you need explosives for that anyway. the reason for that

(Also I think there is a good reason we don’t really see terrorists use biological weapons, nor chemical weapons (with a few notable, but not that effective exceptions), big bada boom is king)

is that barrier to booms is even lower, especially if your country is strewn with UXO. there's also an entirely different reason why professional militaries don't use chemical/biological weapons https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/

[–] [email protected] 0 points 9 months ago* (last edited 9 months ago) (1 children)

I feel it's important to mention that as far as CBRN threats are concerned, biological warfare threats are very real, a serious problem, and admittedly accelerated by ai tools for novel biological structures. Militaries don't use bio weapons because they suck at military things, largely, but terrorists have and can used bio weapons to terrifying effect. Bio warfare proliferation is difficult to spot and counter.

To be clear here, open ai is late to the party on this front with a terrible paper, but practically it's a serious concern, both ai tools and non ai tools lowering the barrier to entry, as well as the fact that any given bio lab essentially looks like a bio warfare lab.

[–] [email protected] 0 points 9 months ago (2 children)

terrorists have and can used bio weapons to terrifying effect

Can you please tell me when and where? The Japanese subway gassing comes to mind. Anthrax envelopes in 2001. Any others since then?

[–] [email protected] 0 points 9 months ago (1 children)

Here's an overview. Not the highest casualty count but certainly not ineffective.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8818129/

[–] [email protected] 0 points 9 months ago
[–] [email protected] 0 points 9 months ago

there were really only three. one is 2001 anthrax that you mention; other is salmonella spread by cultists in literally 1984; and another would be 1989 medfly infestation. everything else is, relatively speaking, non-incident

[–] [email protected] 0 points 9 months ago

also the another reason that wiped out any interest in chemical warfare among militaries is that they found out first cluster munitions and then PGMs vastly more useful in the roles they were shoehorning chemical weapons in, not to mention lack of diplomatic and other problems