Apparently AI under the control of a rogue actor is undoubtedly going to kill us all with a synthesized virus.

Let me run you through how brain rotted the world is now that this is even being taken in the slightest seriousness.

You cant just prompt an AI model, hey make me a humanity ending virus, no mistakes and send it to the printers. That’s not how reality works. To generate a world ending virus. You will need the following chain of events. If each one fails, the whole chain of events fail.

Step A: AI generates a viable novel pathogen sequence.

Highly unlikely without massive human intervention and iterative wet-lab feedback. Let’s say 1% chance for a non-expert using public tools.

Step B: The sequence is stable enough to be synthesized and remain infectious.

Many generated sequences are nonsense or unstable. Probability of viability: 5%.

Step C: It bypasses DNA synthesis screening (Know-Your-Customer rules).

Screening is imperfect but effective against known dangerous motifs. Probability: 10%.

Step D: The actor has access to a BSL-3/4 lab capable of handling high-consequence pathogens.

You forget extensive of testing of the novel virus canidates is needed so you can validate the virus’s virality. You can’t do this in a garage. You need specialized infrastructure, containment, waste disposal, and ethical blind spots. Probability for a random bad actor: <0.1%.

Step E: The virus achieves optimal transmissibility AND lethality simultaneously.

Biologically, these often trade off. High lethality usually kills the host before they spread it widely. Finding the “sweet spot” is incredibly hard. Probability: 1%.

Step F: It spreads globally before detection/response.

Modern surveillance, travel restrictions, and medical response are faster than in 1918 or 2020. Probability of total global saturation before countermeasures: 5%.

Any one of these fail and you dont get an outbreak.

Total Probability:

0.01 ×0.05 ×0.10 ×0.001 ×0.01 ×0.05 =0.00000000000025%

And before you say state actor will do this. They already can if they want to.

  • diz@awful.systemsEnglish
    4·
    3 days ago

    So here’s the thing, AI labs got billions of dollars to spend on making this “rogue” accident happen. They can buy a lab, kit it out, do some random sequence mixing with already deadly viruses, have natural selection do the heavy lifting, and then claim that they were trying to cure cancer, but their AI is so good it made a supervirus. (Of course, the supervirus is unlikely to be all that good, but it doesn’t have to be, they’ll just hype up what could have happened).

    That is basically what they did for hacking, and they didn’t get in any kind of trouble, so they will be escalating and escalating. Keep in mind these folks have always been about applying “fake it until you make it” attitude to the end of the world. They had been promising their investors a double digit % chance of ending the world, from the very beginning of these companies.