I’ve been thinking lately about AI poisoning. I want to make this post a thread for suggestions about which tools we can use to fight back. Bear in mind that I’m only talking about poisoning tools, not blocklists or bot-detection utilities. Here’s a short list of what I have so far:

Website protection tarpits for text poisoning (fighting web scrapers):

  • Nepenthes was one of the first. In the words of its author, “I’m just fed up, and you know what? Let’s fight back, even if it’s not successful. Be indigestible. Grow spikes.” Nepenthes’s demo page is here.
  • Iocaine is the best tarpit I know of right now. It’s very well-designed and performant. Version 3.0x is complete, and the fourth version is a work in progress. It’s made by Gergely Nagy, who goes by the pseudonym “algernon”, after the character by the same name from Flowers for Algernon, by Daniel Keyes. Iocaine’s demo page is here.

Image poisoning tools:

  • algernon@lemmy.ml
    7·
    5 days ago

    FWIW, while iocaine does generate copious amounts of garbage, it is never going to poison LLM training data: it’s trivial to filter iocaine-generated garbage out pre-training. The garbage is there for two reasons:

    1. To serve the crawlers something.
    2. To poison the crawling queue.

    Not the models, the crawling queue. Place an identifiable substring into every generated URL, and you can catch any crawler that works off of a queue collected by cheaper crawlers.

    It’s not really a poisoning tool - it originally started with that goal, but that goal has changed a loooong time ago -, it’s more bot detection & mitigation nowadays, the garbage is just one of the tools it achieves this goal with.

    But while I’m here, I’d like to mention Alun Jones’ fakejpeg (or my rust port): a library that can help you generate structurally legit-ish JPEGs, files that most decoders will happily display, but with entirely random content. Generating them is very cheap. How effective it is in poisoning? I don’t know. But I find great delight in looking at the random noise.

    Hello Lemmy

    • bold_omi@lemmy.todayOPEnglish
      11·
      1 day ago

      Oh hey, Algernon!

      I didn’t expect to gather the attention of one of the people I mentioned in the post. Thanks for this; it’s quite informative. Iocaine is really impressive, and I follow your work quite closely. I’ve been reading your blog for a year and a half now.

      So, I have a few questions, if you have the time to answer them:

      • I noticed you started another rewrite of Iocaine for version 4.x. May I ask about plans for version four compared to version three?
      • How well does Iocaine integrate with OpenRC? I don’t use SystemD much, as I tend to run Artix.

      P.S. Algernon is a really cool pseudonym; Flowers for Algernon is a fantastic novel (and short story). It’s one of my favorites, along with some Kurt Vonnegut Jr. books. I’m fond of Player Piano, if you’ve read it.

      • algernon@lemmy.ml
        3·
        18 hours ago

        I noticed you started another rewrite of Iocaine for version 4.x. May I ask about plans for version four compared to version three?

        I wrote a bit about the plans over on my blog. The major changes in a nutshell are:

        • Nam-Shub of Enki elevated to “default script” status
          • It also becomes a module library, and will be available in both Roto and Lua
        • Roto upgraded from 0.9 to 0.13 (bringing a whole lot of nice language features, more performance, and less code I need to write)
        • Half the dependencies axed.
        • Plugins for Caddy, nginx, and Apache, embedding iocaine.
        • Much, much, much better error reporting (see an example of an error here)
        • Throttling & rate limiting

        How well does Iocaine integrate with OpenRC? I don’t use SystemD much, as I tend to run Artix.

        Pretty well, I think? There’s an OpenRC init script in the repo. I haven’t tested it myself, I currently use systemd on Linux, and NetBSD’s init system or dinit on NetBSD. The only systemd-specific thing in iocaine is that it notifies systemd when it finished starting up - and that’s going to go away in iocaine 4, because I’m cutting dependencies, and this tiny thing isn’t worth a dependency.