Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-21 month agoAm I the only one who is really impressed by Granite4 from IBM?message-squaremessage-square4linkfedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1message-squareAm I the only one who is really impressed by Granite4 from IBM?Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-21 month agomessage-square4linkfedilinkfile-text
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up0·1 month agothere’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
minus-squareBaŝto@discuss.tchncs.delinkfedilinkEnglisharrow-up0·1 month agogranite4:micro-h should be able to run on machines with 4GB RAM
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up0·1 month agoYou can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too
there’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
granite4:micro-h should be able to run on machines with 4GB RAM
You can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too