{"comments":[{"comment":{"id":16198377,"creator_id":19334631,"post_id":31245092,"content":"Does ollama work better than vanilla llama.cpp?\n\nI've just migrated from LM Server to llama.cpp to try and liberate my stack a bit, and also heard ollama have up support for AMD chips\n\nEdit: fixed very bad autocorrect","removed":false,"published":"2025-09-03T03:07:10.702049Z","updated":"2025-09-03T11:25:29.064549Z","deleted":false,"ap_id":"https://lemmy.ca/comment/18730103","local":false,"path":"0.16198377","distinguished":false,"language_id":37},"creator":{"id":19334631,"name":"panda_abyss","banned":false,"published":"2025-07-20T15:44:48.097088Z","actor_id":"https://lemmy.ca/u/panda_abyss","local":false,"deleted":false,"bot_account":false,"instance_id":106},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16198377,"score":6,"upvotes":6,"downvotes":0,"published":"2025-09-03T03:07:10.702049Z","child_count":10},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16201000,"creator_id":3050461,"post_id":31245092,"content":"It depends on what you mean. \n\n\nTo me, Ollama feels like it's designed to be a developer-first, local LLM server with just enough functionality to get you to a POC, from where you're intended to use someone else's compute resources. \n\n\nllama.cpp actually supports more backends, with continuous performance improvements and support for more models.\n\n","removed":false,"published":"2025-09-03T08:44:44.645940Z","deleted":false,"ap_id":"https://lemmy.world/comment/19181059","local":false,"path":"0.16198377.16201000","distinguished":false,"language_id":37},"creator":{"id":3050461,"name":"afk_strats","banned":false,"published":"2023-09-06T06:59:41.559123Z","actor_id":"https://lemmy.world/u/afk_strats","local":false,"deleted":false,"bot_account":false,"instance_id":65477},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16201000,"score":6,"upvotes":6,"downvotes":0,"published":"2025-09-03T08:44:44.645940Z","child_count":0},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16200346,"creator_id":18713192,"post_id":31245092,"content":"Ollama uses ROCm whereas llama.cpp uses Vulkan compute. Which one will perform better depends on many factors, but Vulkan compute should be easier to setup. ","removed":false,"published":"2025-09-03T07:15:41.466822Z","deleted":false,"ap_id":"https://aussie.zone/comment/18654183","local":false,"path":"0.16198377.16200346","distinguished":false,"language_id":37},"creator":{"id":18713192,"name":"vividspecter","banned":false,"published":"2025-06-21T08:47:46.594354Z","actor_id":"https://aussie.zone/u/vividspecter","local":false,"deleted":false,"bot_account":false,"instance_id":82370},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16200346,"score":3,"upvotes":3,"downvotes":0,"published":"2025-09-03T07:15:41.466822Z","child_count":3},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16199612,"creator_id":10815502,"post_id":31245092,"content":"I dunno about better, but different. The API and model management that it offers has been nice when building things that want to use different sized models for different tasks since it will mange the given resources and schedule runners on GPU/CPU. My hardware combo is intel/nvidia so I've not had to futz with getting AMD stuff running. If you don't need any of that, and llama.cpp works for you, no reason to use ollama","removed":false,"published":"2025-09-03T05:39:45.072627Z","deleted":false,"ap_id":"https://lemmy.world/comment/19179278","local":false,"path":"0.16198377.16199612","distinguished":false,"language_id":37},"creator":{"id":10815502,"name":"okwhateverdude","banned":false,"published":"2023-08-01T16:27:39.052762Z","actor_id":"https://lemmy.world/u/okwhateverdude","local":false,"deleted":false,"bot_account":false,"instance_id":65477},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16199612,"score":2,"upvotes":2,"downvotes":0,"published":"2025-09-03T05:39:45.072627Z","child_count":4},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16202352,"creator_id":19334631,"post_id":31245092,"content":"That is something I wish was easier with llama.cpp\n\nI’m using llama swap for that but you have to manually specify your models in a yaml config, then you can set up groups of modes that can run at the same time. \n\nI also have to manually download models, which is a more cumbersome.","removed":false,"published":"2025-09-03T11:24:01.159480Z","updated":"2025-09-03T11:24:22.950505Z","deleted":false,"ap_id":"https://lemmy.ca/comment/18735124","local":false,"path":"0.16198377.16199612.16202352","distinguished":false,"language_id":37},"creator":{"id":19334631,"name":"panda_abyss","banned":false,"published":"2025-07-20T15:44:48.097088Z","actor_id":"https://lemmy.ca/u/panda_abyss","local":false,"deleted":false,"bot_account":false,"instance_id":106},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16202352,"score":2,"upvotes":2,"downvotes":0,"published":"2025-09-03T11:24:01.159480Z","child_count":3},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16203084,"creator_id":10815502,"post_id":31245092,"content":"Yeah for sure. Ollama makes all of this way easier including downloading models at runtime (assuming your query can wait that long, lol). I've been very pleased so far in the functionality it gives me. That said, if I was building a very tight integration or a desktop app, I would probably use llama.cpp directly. It just depends on the usecase and scale. I do wish they (EDIT: ollama) would be better netizens and upstreaming their changes to llama.cpp. Also, it is unfortunate that at some point ollama will get enshittified (no more easy model downloads from their library without an account, etc) if only because they are building a company around it. So I am really thankful that llama.cpp continues to be such foundational piece for FOSS LLM infra.","removed":false,"published":"2025-09-03T12:26:01.113888Z","updated":"2025-09-03T12:28:40.276809Z","deleted":false,"ap_id":"https://lemmy.world/comment/19183644","local":false,"path":"0.16198377.16199612.16202352.16203084","distinguished":false,"language_id":37},"creator":{"id":10815502,"name":"okwhateverdude","banned":false,"published":"2023-08-01T16:27:39.052762Z","actor_id":"https://lemmy.world/u/okwhateverdude","local":false,"deleted":false,"bot_account":false,"instance_id":65477},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16203084,"score":2,"upvotes":2,"downvotes":0,"published":"2025-09-03T12:26:01.113888Z","child_count":0},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16209915,"creator_id":220021,"post_id":31245092,"content":"Doesn’t llama.cpp have a -hf flag to download models from huggingface instead of doing it manually?","removed":false,"published":"2025-09-03T18:04:52.365514Z","deleted":false,"ap_id":"https://lemmy.world/comment/19189120","local":false,"path":"0.16198377.16199612.16202352.16209915","distinguished":false,"language_id":37},"creator":{"id":220021,"name":"afaix","banned":false,"published":"2023-06-10T07:43:53.042318Z","actor_id":"https://lemmy.world/u/afaix","local":false,"deleted":false,"bot_account":false,"instance_id":65477},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16209915,"score":0,"upvotes":0,"downvotes":0,"published":"2025-09-03T18:04:52.365514Z","child_count":1},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16209916,"creator_id":19334631,"post_id":31245092,"content":"It does, but I've never tried it, I just use the hf cli","removed":false,"published":"2025-09-03T20:21:24.191787Z","deleted":false,"ap_id":"https://lemmy.ca/comment/18743900","local":false,"path":"0.16198377.16199612.16202352.16209915.16209916","distinguished":false,"language_id":37},"creator":{"id":19334631,"name":"panda_abyss","banned":false,"published":"2025-07-20T15:44:48.097088Z","actor_id":"https://lemmy.ca/u/panda_abyss","local":false,"deleted":false,"bot_account":false,"instance_id":106},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16209916,"score":1,"upvotes":1,"downvotes":0,"published":"2025-09-03T20:21:24.191787Z","child_count":0},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16200591,"creator_id":3050461,"post_id":31245092,"content":"Ollama does use ROCm, however, so does llama.cpp. Vulkan happens to be another available backend supported by llama.cpp.\n\n[GitHub: llama.cpp Supported Backends](https://github.com/ggml-org/llama.cpp/tree/master?tab=readme-ov-file#supported-backends)\n\nThere is an old PRs which attempted to bring Vulkan support to Ollama - a logical and helpful move, given that the Ollama engine is based on llama.cpp - but the Ollama maintainers weren't interested. \n\nAs for performance vs ROCm, it does fine. Against CUDA, it also does well unless you're in a mulit-gpu setup. Its magic trick is compatibility. Pretty much everything runs Vulkan. And Vulkan is intecompatible between generations of cards, architectures AND vendors. That's how I'm running a single PC with Nvidia and AMD cards together ","removed":false,"published":"2025-09-03T07:46:00.923036Z","updated":"2025-09-03T07:47:00.876727Z","deleted":false,"ap_id":"https://lemmy.world/comment/19180510","local":false,"path":"0.16198377.16200346.16200591","distinguished":false,"language_id":37},"creator":{"id":3050461,"name":"afk_strats","banned":false,"published":"2023-09-06T06:59:41.559123Z","actor_id":"https://lemmy.world/u/afk_strats","local":false,"deleted":false,"bot_account":false,"instance_id":65477},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16200591,"score":9,"upvotes":9,"downvotes":0,"published":"2025-09-03T07:46:00.923036Z","child_count":2},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16200814,"creator_id":10608815,"post_id":31245092,"content":"I think llama.cpp merged ROCm support in 2023 already. It's called HIP on their Readme, but I'm not super educated on all the acronyms and compute frameworks and instruction sets.","removed":false,"published":"2025-09-03T08:16:45.034463Z","deleted":false,"ap_id":"https://palaver.p3x.de/comment/2238276","local":false,"path":"0.16198377.16200346.16200591.16200814","distinguished":false,"language_id":37},"creator":{"id":10608815,"name":"hendrik","display_name":"hendrik","banned":false,"published":"2021-08-21T12:00:00Z","actor_id":"https://palaver.p3x.de/u/hendrik","bio":"A software developer and Linux nerd, living in Germany. I'm usually a chill dude but my online persona doesn't always reflect my true personality. Take what I say with a grain of salt, I usually try to be nice and give good advice, though. \r\n\r\nI'm into Free Software, selfhosting, microcontrollers and electronics, freedom, privacy and the usual stuff. And a few select other random things as well.","local":false,"deleted":false,"matrix_user_id":"@hendrik:p3x.de","bot_account":false,"instance_id":244827},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16200814,"score":2,"upvotes":2,"downvotes":0,"published":"2025-09-03T08:16:45.034463Z","child_count":1},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false},{"comment":{"id":16200857,"creator_id":3050461,"post_id":31245092,"content":"ROCm is a software stack which includes a bunch of SDKs and API.\n\n\nHIP is a subset of ROCm which lets you program on AMD GPUs with focus portability from Nvidia's CUDA","removed":false,"published":"2025-09-03T08:23:33.407863Z","deleted":false,"ap_id":"https://lemmy.world/comment/19180859","local":false,"path":"0.16198377.16200346.16200591.16200814.16200857","distinguished":false,"language_id":37},"creator":{"id":3050461,"name":"afk_strats","banned":false,"published":"2023-09-06T06:59:41.559123Z","actor_id":"https://lemmy.world/u/afk_strats","local":false,"deleted":false,"bot_account":false,"instance_id":65477},"post":{"id":31245092,"name":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","url":"https://www.phoronix.com/news/ollama-0.11.9-More-Performance","body":"","creator_id":19536974,"community_id":2344,"removed":false,"locked":false,"published":"2025-09-03T02:58:52.110360Z","deleted":false,"nsfw":false,"embed_title":"ollama 0.11.9 Introducing A Nice CPU/GPU Performance Optimization","thumbnail_url":"https://www.phoronix.net/image.php?id=nvidia-rtx4080-rtx4090-compute&image=rtx40_compute_1","ap_id":"https://piefed.world/post/436272","local":false,"language_id":37,"featured_community":false,"featured_local":false,"url_content_type":"text/html; charset=utf-8"},"community":{"id":2344,"name":"localllama","title":"LocalLLaMA","description":"Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.\n\nGet support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.\n\nAs ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.\n\nRules:\n\nRule 1 -\nNo harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.\n\nRule 2 -\nNo comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.\n\nRule 3 -\nNo comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as \"llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since .\n\nRule 4 -\nNo implying that models are devoid of purpose or potential for enriching peoples lives.","removed":false,"published":"2023-06-08T18:21:54.578976Z","updated":"2025-06-09T20:51:11.460711Z","deleted":false,"nsfw":false,"actor_id":"https://sh.itjust.works/c/localllama","local":false,"icon":"https://sh.itjust.works/pictrs/image/8e07ec26-c7e5-4411-a4d5-c0c7a292b773.gif","banner":"https://sh.itjust.works/pictrs/image/5d07b398-7675-4862-8277-5906400b666d.png","hidden":false,"posting_restricted_to_mods":false,"instance_id":72342,"visibility":"Public"},"counts":{"comment_id":16200857,"score":5,"upvotes":5,"downvotes":0,"published":"2025-09-03T08:23:33.407863Z","child_count":0},"creator_banned_from_community":false,"banned_from_community":false,"creator_is_moderator":false,"creator_is_admin":false,"subscribed":"NotSubscribed","saved":false,"creator_blocked":false}]}