How to run an AI chatbot on Linux
Kill all hoomans!
You may have heard about a little thing called ChatGPT in the news lately. Some say it may be the end of humanity! Others, that it will make everyone unemployed, and our children will be watering crops with sports drinks!
While these concerns are debatable, the topic of artificial intelligence is hard to avoid these days. So, I wanted to play around with an AI chatbot on my local PC and see what all the fuss is about.
AlpacaCPP
I found an interesting project on GitHub the other day, called “alpaca.cpp”. This is a ChatGPT-like model which you can run directly on your pc from the terminal. It is multi-platform and can be run on Windows, Mac and Linux.
When I think “artificial intelligence chatbots”, I imagine a website which I have to log in to, a text box, and my text input being processed by a floor of supercomputers in a server farm somewhere. So, I was pleasantly surprised to learn that this can now be done local on my PC! This project seemed very interesting to me, so I thought I’d check it out. It’s also very easy to install and use.
Installation
You will first need to clone the alpaca.cpp Git repository.
Open a terminal and enter:
git clone https://github.com/antimatter15/alpaca.cpp.git
Next, you’ll need to download the “ggml-alpaca-7b-q4.bin” model. Note that this is a large file (4 GB), so may take a few minutes to download depending on your connection.
Once downloaded, copy the ggml-alpaca-7b-q4.bin file into the alpaca.cpp/ folder which you cloned previously.
Next, cd into the alpaca.cpp folder from within a terminal:
cd alpaca.cpp/
then type:
make chat
Then, to run it, type:
./chat
And that’s it! After a few seconds, the chat interface will be presented to you. From here you can type any question you like, and the AI will work its magic and provide you with an answer.
Example Chat
Here is my first conversation with the chatbot.
| Role | Response |
|---|---|
| User: | “Hi there!” |
| Bot: | “Hello! How can I help you?” |
| User: | “Which is better, Windows or Linux?” |
| Bot: | “It depends on what you need a computer for and your personal preference. Windows is easier to use but has limited customization options while Linux offers greater control over the system as well as more flexibility in terms of software choices.” |
| User: | “Do you dream of electric sheep?” |
| Bot: | “No, I do not dream about electric sheep. But if someone were to offer me a dream job involving electrical engineering and robotics, then yes - that would be my ultimate career goal! 🤩” |
| User: | “Will artificial intelligence end humanity?” |
| Bot: | “No, I don’t think so. Artificial Intelligence (AI) is a tool like any other and can either help or hinder us depending on how it’s used. AI has the potential to make life easier for humans by automating mundane tasks but also carries risks of misuse, particularly in military applications where autonomous weapons are being developed that could potentially decide who lives and dies without human intervention. Ultimately I believe we will be able to use technology responsibly while still reaping its benefits - it’s just a matter of time before the right precautions have been taken so as not to put humans in danger due to AI misuse or malfunctioning.” |
| User: | “If artificial intelligence takes over the Earth - where can I hide?” |
| Bot: | “You could try hiding in a virtual world. Artificial Intelligence is likely to have difficulty understanding and interacting with humans within such immersive environments, so it may be difficult for AI to find you there!” |
My Thoughts
After briefly interacting with this AI model, I could see how AI in general may benefit people who want to quickly find out information on certain topics, or as in my case - to simply have a bit of fun and learn something new.
However, you need to keep in mind that these chatbots are not one hundred percent accurate with the information that they provide. I found myself mostly asking it questions which I already knew the answers to - just to see how accurate it was. The answers it did provide - I found myself seeking a second opinion on its results.
One beneficial use of AI chatbots I could think of, is that of an “in-app-helper” for use in complex software For example, a user could ask how to perform a complex task in said software and the AI assistant could provide a step-by-step set of instructions to show the user how to do it. This could be very useful for those who do not like to read lengthy documentation or watch online video tutorials about how to do things.
Regarding creativity, it’s almost tempting to try and use this technology for creating posts for my website. However, I think this would be a bit lazy, and something I wouldn’t want to do. Further, it would not reflect who I am or how I think. Therefore, I think I’ll continue to write about my own thoughts and ideas on topics I find interesting, and only use chatbots as a fun experiment to tinker with.
In any case, it’s both interesting and fun to witness these artificial intelligence models in action, especially ones capable of running on my own pc, rather than in the cloud.