• Pup Biru@aussie.zone
    link
    fedilink
    English
    arrow-up
    2
    Ā·
    1 month ago

    you donā€™t actually need to fit the whole model in RAM at once: the 70b for example ā€œrequiresā€ something like 120gb of VRAM, but iā€™m running it on my 64gb m1 mbp - it just starts to run a bit slower (still very usable; i reckon about a word per 300ms)