• red_tomato@lemmy.world
    link
    fedilink
    arrow-up
    3
    arrow-down
    1
    ·
    2 days ago

    Not necessarily. Input tokens is far cheaper than output tokens. The alternative is to have the LLM derive the documentation from code each session, which is expensive due to the amount of output tokens it produces.

    • dustycups@aussie.zone
      link
      fedilink
      arrow-up
      3
      ·
      2 days ago

      Are people using LLMs to derive readme’s from source code!? I think they might be the target audience here.

      • red_tomato@lemmy.world
        link
        fedilink
        arrow-up
        3
        arrow-down
        1
        ·
        2 days ago

        More like indirectly. For an LLM to effectively work with a codebase it needs to know how the project is structured and where to find stuff. Instead of having it derive this information each session, it can just read about it in the available documentation. It’s more token efficient.