Just used Chat with RTX to summarize your video in about a minute worth of reading. What a crazy time to be alive. I'll leave your video running in a tab so you're credited for the view and watch time.
@BabySisZ_VR
7 ай бұрын
lol
@GumboRyan
7 ай бұрын
Efficient AND considerate.
@looseman
7 ай бұрын
It is reading from subtitle, not from Video.
@KIaKlaa
7 ай бұрын
just used chat with rtx to create a thingmabob to make yo wife bald and yo dog fat, watch out m blud
@ekot0419
6 ай бұрын
I have been doing that using Chatgpt for a long time already.
@MrErick1160
7 ай бұрын
Wow this is AMAZING. A non-cloud chat that we can use with our local documents!!! Freaking cool and very useful product, NVIDIA def knows what people need
@DrakeStardragon
7 ай бұрын
Uhh, they are not the first, but ok.
@merlinwarage
7 ай бұрын
LMStudio is out for almost 8 months what does the same and 10x more.
@KillFrenzy96
7 ай бұрын
Well we already have many solutions for this. It's running Mistral 7B which has been available for many months now. It's nowhere near ChatGPT quality though. However if you have a 24GB GPU, I would suggest running the more powerful Mixtral 8x7B model using EXL2 3.5 bpw quantization. I use the oobabooga WebUI for this. It's about as powerful as ChatGPT free, but is much less restrictive.
@adrianzockt5347
7 ай бұрын
GPT4ALL also exists and supports multiple chats, like chatgpt does. However it crashes when reading large documents and doesn't have the youtube feature.
@chromefuture5561
7 ай бұрын
And it adds finally another real reason for the 40. gen RTX cards
@invisiiso
7 ай бұрын
I’m curious… If you’re comparing between models with the same amount of VRAM (e.g. 3050, 3060 8GB, 4060) will the quality of the outputs improve if the card is better or will it only just have a faster/slower response time?
@ahmetemin08
7 ай бұрын
no, only the interference speed will differ.
@Unknown-xm8ll
7 ай бұрын
See the weights in a neutral network are present by Nvidia so no change in response the model is fitted with the most optimal neutral weights which determine the accuracy and precision of the model. A better faster GPU like 4070, 4080 or the 4090 can improve the speed of the results but the jump till 4080 is not significant. Only 4090 performs faster and more noticeable compared to other GPUs. And fun fact you can run the chat with RTX on AMD gpu 😂 with slight tweeks or just copy the model data and paste it into the lalama interface.
@PrintScreen.
7 ай бұрын
@@ahmetemin08 isn't it "inference" ?
@ahmetemin08
7 ай бұрын
@@PrintScreen. you are correct
@johncollins9263
4 ай бұрын
I am having an issue with installing this as it comes up with chat with rtx failed to install however hardware is not an issue as everything i have is new but it decides not to work?
@IzanamiNoMikotoo
7 ай бұрын
The reason Llama 2 doesn't show is that it "requires" 16GB of VRAM. It will only let you install it if your card has at least 16GB... Unless you change the setting in the llama13b.nvi file. If you set the value to, say, 10GB then you can run it on a 3080 10GB. Idk if it will work perfectly but you can try.
@codeblue6925
7 ай бұрын
where is that file located?
@codeblue6925
7 ай бұрын
nvm i found it
@crobinso2010
7 ай бұрын
@@codeblue6925 Did it work? I have a 12GB 3060
@rockcrystal3277
6 ай бұрын
how do you change the setting in the llama13b.nvi file to 10gb for it to work?
@IzanamiNoMikotoo
6 ай бұрын
@@rockcrystal3277 Go to the file llama13b.nvi located in the installation directory “\NVIDIA_ChatWithRTX_Demo\ChatWithRTX_Offline_2_11_mistral_Llama\RAG”. Then change the "MinSupportedVRAMSize" value however many GB of VRAM your card has.
@onametaquest
7 ай бұрын
This is really cool and one of the biggest missing pieces in the whole equation. Being able to run these models locally and be able to highly curate your own will be very valuable. GPT4All is really neat, does a decent job with this as well, so I am glad to see something similar from Nvidia who makes the GPUs. Crazy times!
@cmdr.o7
7 ай бұрын
I hope this software doesn't just snoop around your file system and documents, scraping it all back to nvidia with telemetry wouldn't be surprised at all if it did, people have little respect left for privacy if it turns out it does, well, just hope video author has done research and not just blindly enabling nvidia that said, we are each responsible for our own security and fighting back against invasive big tech, malware root kits etc
@Jet_Set_Go
7 ай бұрын
They have Nvidia Experience for that already
@jordanturner7821
7 ай бұрын
They already do that with telemetry data. he absolutely does know what he is talking about.@@jeffmccloud905
@cmdr.o7
7 ай бұрын
@@jeffmccloud905 that's right, that is the troubling part clearly you don't know either or you would have enlightened us - but you are a man of few words scraping user data is not a big mystery, it happens everywhere, i think most people have a pretty good idea about that and i do actually know quite a lot about ai systems - and nvidia xD
@AndrewTSq
7 ай бұрын
I think Microsofts AI already does that in Win11
@goldmund22
6 ай бұрын
I'm glad I finally found someone commenting on the privacy aspect of this. Since you mentioned you are experienced with AI and Nvidia, do you think there is a good chance this is happening, even though it is "local"? I am considering using it for analyzing specific folders and PDFs related to my work. I guess the only way to be sure it doesn't also have access to everything else is to literally use this on a different PC and on a different network. I don't know. Then I think about Microsoft OneDrive, and well it already is connected most of everything we have on our PCs by default. Just insane.
@shadowcaster111
7 ай бұрын
is the non C drive install fixed yet ? I tried it on my P drive and it failed to install
@Green_Toast
7 ай бұрын
no, badly not, they talked about it at the nvidia forum
@jackflash6377
6 ай бұрын
I just installed it to my F: drive under a folder named RTXChat and it's working as normal.
@Tore_Lund
7 ай бұрын
System requirements are minimum requirements? Is Win11 needed or does Win10 work?
@Vysair
7 ай бұрын
Isnt Win11 are just Win10 under the hood? Why wouldnt it work
@Subarashi77
5 ай бұрын
they removed youtube url option
@beetrootlife
12 күн бұрын
you guys better pray im not your future doctor after i feed all my lecture slides into the chatbot
@19mitch54
7 ай бұрын
After exhausting the free trials of DALL-E and Midjourney, I bought my new computer with the RTX3070 to run Stable Diffusion. I love this AI stuff. Chat with RTX was a LONG download and it downloaded more dependencies during install but was worth it. I didn’t bother exploring the included dataset and started with my own documents. This works great! I want to build a big library of references and put this thing to work.
@jimmydesouza4375
7 ай бұрын
How good is it for automatically generating things? For example if you stick a bunch of PDFs for a roleplaying game ruleset and setting and then ask it to generate DM prompts from that, can it do it?
@19mitch54
7 ай бұрын
I don’t know much about role playing games. The program is good at answering questions. I pointed it to some manuals including my car’s owners’ manual and it was able to answer technical questions like “how do I reset the service interval?” I want to test it with some microcontroller programming manuals next.
@Vysair
7 ай бұрын
@@19mitch54This is wicked. Your usage is hella perfect for programmer and alike
@AvtarSingh1122
6 ай бұрын
Nice👌🏻
@amumuisalivedatcom8567
4 ай бұрын
@@jimmydesouza4375 i'm late but yup, consider using RAG (Retrieval Augmented Generation) to pass docs to the LLM.
@hairy7653
6 ай бұрын
the KZitem option isn't showing up on my rtxchat
@OpenSourceGuyYT
7 ай бұрын
Yea. With Ollama, you don't need to have an RTX GPU. And it's offline too.
@juanb0609
6 ай бұрын
I dont have the option for KZitem videos
@hairy7653
6 ай бұрын
same here
@vulcan4d
7 ай бұрын
This is a demo which clearly means Nvidia wants to see how many people will use it so they can release a subscription based service later for your AI offline needs.
@ozz3549
7 ай бұрын
That's only UI for llama 2 model, you can find any another ui and this will work same
@gavinderulo12
7 ай бұрын
@@ozz3549it's also something you can build in a week.
@violentvincentplus
7 ай бұрын
35GB goes crazy
@Flashback_Jack
7 ай бұрын
About the same size as a triple A game.
@pedro.alcatra
7 ай бұрын
Exactly. The size is absolutely fine. The problem is having to download it thru the browser instead of a download manager
@arsalanganjeh198
7 ай бұрын
Lighter than cities skylines 2😂
@gamingballsgaming
7 ай бұрын
@pedro.alcatra im fine with that for archival purposes. If i want to install it in the future, i can as long as i have the exe, even if the nvidia servers shut down
@Javier64691
7 ай бұрын
@@Flashback_Jackan old triple a, most nowadays are 60gb plus
@IndieAuthorX
7 ай бұрын
I was excited to use this, but I got it up and running and things did not work so good. I realized that it wasn't technically made to run on Windows 10, according to the requirements page, and I think that might be why. I think that this kind of thing has potential, but I want a chatbot that is completely released for commercial use before getting too comfy with it.
@acllhes
7 ай бұрын
Windows 11 is one of the requirements listed
@IndieAuthorX
7 ай бұрын
@@acllhes yeah, I saw that after. I could have sworn I'd seen both systems. I might have read a non Nvidia page first and then just installed.
@fontenbleau
7 ай бұрын
i'm not sure what you mean "commercial", none of this allowed such by license, it's only allowed for research use by original llama license (except if it based on llama 2 where something allowed but limited by installations). If you want just chatbot right away - easiest way is LLAMAFILE by Mozilla, just click and it works, their small model container is kinda 1,5 Gb but can analyse images
@no_the_other_ariksquad
7 ай бұрын
It's really useful when you have a folder full of documentations for different apis and all things, very good for that.
@MarkAnthonyFernandez-q7y
5 ай бұрын
I don't have the youtube Url option
@yuro1337
7 ай бұрын
it looks like Whisper AI with chat and some additional models
@rionix88
7 ай бұрын
gemini will use this technology. you can chat with 1 hour video
@Vvilvid
6 ай бұрын
I have 4 pcs and none of them can run it 😭😭 Custom Pc1:amd Custom Pc2:amd Laptop1:rtx3050ti(4gb) Laptop2:amd
@handsonlabssoftwareacademy594
3 ай бұрын
Man, I really like your analysis great work. So ChatRTX can be used with any cpu and graphics card including Intel HD Graphics once there's sufficient RAM like 16GB?
@christerjohanzzon
3 ай бұрын
No, you need an RTX card from at least 3000-series. It's the tensor cores that is important. Luckily these cards aren't expensive.
@PixelBrushArt
7 ай бұрын
Oof- why the Windows 11 requirement? Big yikes.
@monkshee
7 ай бұрын
hey man i don't see the llama option when installing i already have an install how would i add it to the list of models?
@haseef
7 ай бұрын
same issue here even though I ticked clean install
@N1h1L3
7 ай бұрын
win 10 ?
@zslayerlpsfmandminecraftan367
7 ай бұрын
llama 2 needs 16gb vram not quantizized, so if you have 8gb it doesn't install it
@MousePotato
5 ай бұрын
AI voice. Us Brits never say anyway with a plural.
@moonduckmaximus6404
6 ай бұрын
THE KZitem OPTION DOES NOT EXIST IN THE DROP DOWN MENU
@elgodric
7 ай бұрын
How many pages of the document can Mistral 7B handle?
@ScottMyers-l1z
6 ай бұрын
Until you run out of RAM and VRAM.
@elpideus
7 ай бұрын
Definitely much easier to set up compared to your average text-generation-webui, however still has a long way to go when it comes to features and control.
@mr.bekfast9744
7 ай бұрын
Am I the only one that is downloading this and Setup.exe is not in the Zip file?
@victornpb
7 ай бұрын
same problem, zip seems corrupted
@pillowism
7 ай бұрын
Same issue here
@0AThijs
7 ай бұрын
For many 😔
@mr.bekfast9744
7 ай бұрын
@@victornpb Okay good to know that im not the only one. Is there anyway for us to report it or get an older version where the zip isnt messed up?
@erkinox1391
7 ай бұрын
I really don't get it; i have all of the requirements (VRAM, RAM, OS, Latest Driver, I got plenty of storage), but whenever I launch the installation, it stops and say Chat with RTX Failed and Mistral Not Installed
@jaderey467
7 ай бұрын
Are you windows 11 it doenst work on 10
@ben9262
7 ай бұрын
I'm getting the same thing
@AlecksSubtil
7 ай бұрын
Disable completely your antivirus, also check the dock icon to disable it from there. avast for example has to be disabled in the tray icon, only in the gui is not enought. Also install it on the default folder. Maybe necessary run it with admin privileges. It is safe to install btw
@James.Mayonnaise
7 ай бұрын
"private" yeeah... Just like incognito mode
@kingofsimulators3242
7 ай бұрын
zip corrupted?
@0AThijs
7 ай бұрын
It seems... 😢 35GB!
@aalejanddro2328
7 ай бұрын
there is a fix?
@kingofsimulators3242
7 ай бұрын
Is it because I have windows 10?
@0AThijs
7 ай бұрын
@@kingofsimulators3242 no, should be fixed, I haven't tried it, redownload 🥲
@ubaidfayaz1989
5 ай бұрын
Sir how can we bypass the nvidia check that occurs prior to installation?
@Jascensionvoid
7 ай бұрын
I keep getting this error when trying to upload some PDF's into my Dataset. [02/23/2024-19:42:28] could not convert string to float: '98.-85' : Float Object (b'98.-85') invalid; use 0.0 instead
@MTX1699
6 ай бұрын
So, is there a solution to this?
@MaiderGoku
7 ай бұрын
Answer this properly, download size and how much space does it take on your hard drive?
@IMABADKITTY
7 ай бұрын
35gb download size
@MaiderGoku
7 ай бұрын
@@IMABADKITTY how much for rtx remix?
@mhvdm
7 ай бұрын
Very buggy, tested it myself and I must say I'm impressed, but darn they need to fix bugs. It was very bad at responding to stuff in general.
@AlbertoPirrotta
Ай бұрын
Thanks for your video tutorial !
@dioghane231
4 ай бұрын
I have a 3050 rtx and it won’t let me install it? Why?
@RedVRCC
4 ай бұрын
Thanks! I just downloaded and installed it but I'm not too sure how to get it running. Working with these complex LLMs is still new to me but I really want my own AI so your video really helps. I hope this runs well enough on my entry level af 3060. This seems simple enough. Will it at least remember everything it learned so I can keep training it more and more?
@lolxgaming7993
4 ай бұрын
I tried downloading it but the download is too slow and this is normal?
@jimmydesouza4375
6 ай бұрын
The anti-che guys did their work decades ago.
@im_Dafox
7 ай бұрын
everything was fine until "windows 11" 😄 Shame, looks really cool and useful
@Fastick911
7 ай бұрын
Rtx 30-40? Really? Rtx 20 are not RTX any more? F...k Nvidia.
@0AThijs
7 ай бұрын
Because my Rtx 3070 (8gb) runs it horribly.
@Waldherz
7 ай бұрын
Downloading dependencies for hours and hours and hours. Zero network activity. Anti virus checked, admin mode checked, network checked. No user error.
@TheMangese
5 ай бұрын
I'm interested in having an interactive AI chatbot in my chat channel on Twitch. Can this do that?
@muruganmurugan507
7 ай бұрын
Its cool does it support 2gb single pdf with 4000 pages😂
@abdiel_hd
5 ай бұрын
Mine didn't come with KZitem as a dataset/source... can someone help me? I have a laptop with a 3070
@blueyf22
5 ай бұрын
my teachers will never know what hit em
@vvhitevvizard_
7 ай бұрын
Nvidia lost touch with reality with these overpriced 4000 series
@_B.C_
7 ай бұрын
Will it do this for yt videos in another language?
@Spengas
7 ай бұрын
That sucks that it is windows 11 only... never upgrading from 10
@GKGames2018
6 ай бұрын
mine does not have youtube
@KenZync.
6 ай бұрын
i just download this and it can't be run can you try remove and redownload it ? i think nvidia cooked something failed
@XiangWeiHuang
7 ай бұрын
can we make a erotic roleplay chatbot with this? I use openai API solely for those.
@TonTheCreator
5 ай бұрын
I installed and used it bu after I closed it I can't use/open it again. I mean i don't know how to
@itxaddict7503
7 ай бұрын
C'mon Skynet. You need us to hand you the world on a silver platter?
@arsalanganjeh198
7 ай бұрын
Nice
@Ortagonation
7 ай бұрын
have dedicated tensor core for ai, but use rtx core instead. Kinda funny
@Vimal_S_Thomas
6 ай бұрын
will it work on my laptop with RTX 2050
@siddharthmishra8283
7 ай бұрын
Waiting for your 12gb SUPIR version installation guide for A1111 Sdxl 😊
@arooman3194
7 ай бұрын
Min 6:56, can not understand the tools you suggest, would you mind to post the link to that tools?
@carlossalgado9075
7 ай бұрын
same isue
@sky37blue
5 ай бұрын
It is in the video description [CHAT] Oobabooga Desktop: • NEW POWERFUL Local ChatGPT 🤯 Mindblow...
@DragoNate
7 ай бұрын
"offline" k
@tonymerasty
5 ай бұрын
yup a solid demo for an intro with your pc and an Ai model thats local
@garethde-witt6433
7 ай бұрын
Won’t be using it, I don’t like or trust any Ai.
@osteoporosisgaming4235
7 ай бұрын
what do you mean its bad? i use ai and i think its improved heaps!
@nosinfantasia
7 ай бұрын
anyone with installer failed , with no reason...
@OpenAITutor
6 ай бұрын
This only works for RTX 4000 series min with 8GB of VRAM.
@IIHydraII
7 ай бұрын
Can you make a video about different presentation modes and how to set them? I’m trying to get my games to run in Hardware Composed: Independent flip, but I’ve only been successful when running games in non native resolutions and also forcing windows to use that resolution. If I try to run native, I end up with Hardware: Independent Flip. I’m aware the only difference between HWCF and HWI is that the former uses DirectFlip optimisations, but I can’t figure out why they’re not working at native resolution. Kinda stumped here. 😅
@LaminarRainbow
7 ай бұрын
Originally I thought it didn't work, but turns out I just have to wait.. :P
@glucapav
7 ай бұрын
It is saying I don't have 8 GB of GPU memory. Is it checking my integrated GPU instead of my NVDA? How do I fix this? I'm using an Asus Pro Duo so the BIOS isn't letting me change it.
@queless
6 ай бұрын
What card do you have?
@flurit
7 ай бұрын
Nvideas really making me regret getting an amd card
@luizmourabr
7 ай бұрын
Me with a 2060: 💀
@bensoos
7 ай бұрын
Finaly my own virtual ai girlfriend.
@tbarczyk1
5 ай бұрын
Awesome tutorial! This is the first one of yours that I've watched, but between this one and few others I've looked at since, your tutorials are the best I've seen anywhere. Thanks for getting into all the interesting details and dumbing it down like your viewers are idiots.
@leeishere7448
7 ай бұрын
How can I get the lama 13b model? I don't have it.
@Baconator119
7 ай бұрын
Eww Windows 11
@_vr
7 ай бұрын
Llama is Facebook's chat model
@Jcorella
7 ай бұрын
6:57 What was that model? Couldn't understand you
@zslayerlpsfmandminecraftan367
7 ай бұрын
oobabooga desktop, wich in itself is a gui similiar to this. but it lets you use custom models. but its more complicated to set up with python 3.10.9
@AmnesiaForever
7 ай бұрын
So... NSFW?
@jonmichaelgalindo
7 ай бұрын
Thanks for the video. Very informative. GPT4All and LMStudio are probably easier for most users though, and they support more models, more OSs, and more features. I wonder what NVidia thought was so special about this...
@NippieMan
7 ай бұрын
Offline AIs can be useful since companies such as OpenAI place in very restrictive rules. While there are already programs that can do what NVIDIA is offering, most consumers are too stupid to set it up themselves
@AntonChekhoff
7 ай бұрын
Which GPU-accelerated model would you recommend? For translation for instance?
@bigglyguy8429
7 ай бұрын
Well I love Faraday and LM Studio, but getting it to understand my own docs is hard,
@jonmichaelgalindo
7 ай бұрын
@@AntonChekhoff I haven't done any translation. I use Mistral raw for my D&D solver system, and for creative writing (mostly for generating large lists, like a thesaurus but for abstract topics).
@crobinso2010
7 ай бұрын
I'm hoping for that too -- a comparison btw LM Studio and Chat with RTX, which do the same things.
@paulocoelho558
7 ай бұрын
File Size 35 GB? Why? 💀💀
@OpenAITutor
6 ай бұрын
The two LLMs 14 GB and 8 GB .. Then NVIDIA installs mini conda and all the python libararies in a separate environment called env_vnd_rag 16 GB plus TensortRT_LLM for creating the enginees to work with your GPU
@X-Prime123
7 ай бұрын
Nope.
@blitzguitar
7 ай бұрын
Can I use it to overclock my 3070
@JoyKazuhira
7 ай бұрын
wow maybe in the future, this will be added in a game. will definitely use instead of turning on ray tracing.
@bensoos
7 ай бұрын
Now real interlegend bots in games.
@heyguyslolGAMING
7 ай бұрын
What is the fastest animal on the planet?
@DeepThinker193
7 ай бұрын
The slug.
@Spectrulight
7 ай бұрын
Idk probably a falcon
@N1h1L3
7 ай бұрын
@@Spectrulight The peregrine falcon is the fastest bird, and the fastest member of the animal kingdom, with a diving speed of over 300 km/h (190 mph).
@TenOfClub
7 ай бұрын
airborne Microbes👌👌
@bgill7475
7 ай бұрын
Me when I need to pee
@banabana4691
7 ай бұрын
i think its make nvidia graphic crad more valuable
@kathiravan_vj
7 ай бұрын
Does RTX 2060 super supports this with 16gb ram?
@xXXEnderCraftXXx
7 ай бұрын
Well no. Atleast not without some bypass programs.
@KrishnVallabhDas
7 ай бұрын
i am getting this error ModuleNotFoundError: No module named 'torch' how to fix this??
@CindyHuskyGirl
7 ай бұрын
pip install torch (put this into your terminal)
@OpenAITutor
6 ай бұрын
You should go through the installer. It has all the stuff build in. It also creates it's virtual python environment in a folder called env_vnd_rag
@月子きり
7 ай бұрын
No rtx 20 series? Yikes.
@TheMidnightGoose
7 ай бұрын
If you're technically inclined lookup "OobaBooga Text Generation Webui" running LLMs locally has been possible for a long time now and they support any graphics card that can run the models. It also has far more features compared to "RTX Chat", sad to see another mega-corporation attempting to stick their grubby fingers into the open source scene.
@minty87
7 ай бұрын
would love to see a photo generator on it id definitely get on it in that case . nice video
@Grim_Cyanide
7 ай бұрын
was pretty excited to try it, then saw the window 11 requirement. shame :(
@Otakugima
7 ай бұрын
You can use windows 10!
@AndrewTSq
7 ай бұрын
@@Otakugima are you sure? other comments say win11 only also. I want to try it, but I will not install windows 11
@MiNombreEsEscanor
7 ай бұрын
I downloaded this, it works pretty good locally, but I want to create a web application and use this chatbot in my application. Currently chat with rtx doesn't offer api to send questions and retrieve answers. Is there any way to achieve this? Or maybe they will add api feature in the future? What do you guys think?
@Hypersniper05
7 ай бұрын
Text generation webui
@voidsh4man
7 ай бұрын
at scale it would cost you more to run an ai chatbot on your own hardware than using openai's api
@anispinner
7 ай бұрын
Considering it runs a local node I suppose one of the folders should contain plain .js files, otherwise it might be packed as an electron which you can unpack and inject your API into.
@fontenbleau
7 ай бұрын
nvidia never made any great software, they're only hardware. Don't count on that, why do you think we use Afterburner made by MSI (why Nvidia can't made such tool is a puzzle), even this they could make a year ago by hiring any student on Ai faculty
@anispinner
7 ай бұрын
Puzzle? Why would you make an overclocking soft that goes against your business model? Your goal (as a business) should be to sell the product, not to extend its lifespan.
@ashw1nsharma
7 ай бұрын
Thanks for this new discovery! Hope you're having a nice day! 🌻
@andyone7616
7 ай бұрын
Can you make a video on how to uninstall chat with rtx?
@TazzSmk
7 ай бұрын
is RTX A4000 supported? should be Ampere generation card I believe
@skym1nt
7 ай бұрын
yes, it can.
@mayorc
7 ай бұрын
Does it support custom models like using OpenAI api endpoint local servers?
@JA_BRE
7 ай бұрын
Its only Demo, no way it supports it yet...
@buttpub
7 ай бұрын
so why on earth would anyone choose this over for example ollama thru wsl on windows or even easier gpt4all? with this you only get one model, mistral, which is a good model but at 35 gb of download how could that possibly be the model file considering min req is 8gb of ram? so what other bloatware is there, the mistral model is only 7.4gb thru any of the freeware model query tools mentioned above or by just downloading the model and weights urself. Nvidia is once again late to the party and they forgot drinks
@anispinner
7 ай бұрын
Most of those that you mentioned use CPU for that easier setup, especially gpt4all. For the size of guess it's the dependencies and the ease that you can uninstall everything with one click as the most of it should be within one folder. Otherwise the user has to deal with pythons, condas and other reptiles. Hmm, maybe it also contains portable CUDA? Id have to give it a closer look as well.
@buttpub
7 ай бұрын
@@anispinner most of what i mentioned? gpt4all AND ollama BOTH have the options to do cpu or gpu depending on your setup. If you have gotten to the point of trying to f with llm's on your local pc, then you know how to open a terminal window.
@anispinner
7 ай бұрын
There is quite a difference between opening a console and clicking an install button.
@buttpub
7 ай бұрын
@@anispinner indeed, without context there is, but with context; and the fact that these are llm's, you need some basic understanding before you even embark on this. And people without any; are rarely at this point yet, and if they are then learn.
Пікірлер: 276