JOMusic@lemmy.ml to World News@lemmy.worldEnglish · 2 months agoOpen-source Deepseek R1 dethrones commercial AI, now allegedly being hit by cyberattackwww.cnbc.comexternal-linkmessage-square50fedilinkarrow-up1199arrow-down19cross-posted to: technology@lemmy.world
arrow-up1190arrow-down1external-linkOpen-source Deepseek R1 dethrones commercial AI, now allegedly being hit by cyberattackwww.cnbc.comJOMusic@lemmy.ml to World News@lemmy.worldEnglish · 2 months agomessage-square50fedilinkcross-posted to: technology@lemmy.world
minus-squarePieisawesome@lemmy.worldlinkfedilinkEnglisharrow-up4·1 month agoIt’s because LLMs don’t work with letters. They work with tokens that are converted to vectors. They literally don’t see the word “strawberry” in order to count the letters. Splitting the letter probably separates them into individual tokens
It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors.
They literally don’t see the word “strawberry” in order to count the letters.
Splitting the letter probably separates them into individual tokens