ย
๐ Lesson: Running a Model and Testing It via Terminal (Ollama + Whisper.cpp)
๐ฏ Lesson Objective
By the end of this lesson, you’ll be able to:
-
โ Run your language model (e.g. Mistral, LLaMA3) with
ollama -
โ Interact with it via terminal (chat, summarization, etc.)
-
โ Run speech-to-text locally with
whisper.cpp -
โ Combine both for document/audio-based AI workflows
๐ง Why Use the Terminal?
The terminal is:
-
๐ง A fast way to test and debug LLMs before integrating into apps
-
๐ Lighter on memory than full UIs
-
๐ Secure โ especially when running offline models
๐งฐ Tools You Should Have Ready
| Tool | Use | Command |
|---|---|---|
| ๐ง Ollama | Run LLMs (chat, QA, code) | ollama run model_name |
| ๐ฃ๏ธ Whisper.cpp | Transcribe audio offline | ./main -m model.bin -f file.wav |
๐ข Part 1: Running a Text-Based LLM with Ollama
โ Step 1: Pull a Model
ollama pull mistral
Other options:
ollama pull llama3
ollama pull gemma
ollama pull codellama
โ Step 2: Chat with the Model in Terminal
ollama run mistral
Now youโre in an interactive chat session like this:
>>> What is quantum computing?
Quantum computing is a technology...
To exit: Ctrl + C or type exit
โ Step 3: Use Prompt Files (Optional)
You can run a predefined prompt from a file:
ollama run mistral < prompt.txt
Where prompt.txt contains:
Summarize this:
Artificial intelligence is the simulation of human intelligence processes...
โ Example Use Cases
| Prompt | Result |
|---|---|
Explain LangChain in simple terms. |
General explanation |
What are the main causes of inflation? |
Economic summary |
Write a Python function to reverse a string. |
Code generation |
๐ต Part 2: Transcribing Audio with Whisper.cpp (Offline)
โ Step 1: Prepare Whisper.cpp
Navigate to folder:
cd whisper.cpp
Make sure you have the binary:
make
Download the model:
./models/download-ggml-model.sh base.en
โ
Step 2: Transcribe a .wav File
./main -m models/ggml-base.en.bin -f your_audio.wav
This will produce a .txt file with the transcription.
Optional: Add -otxt to save transcript:
./main -m models/ggml-base.en.bin -f audio.wav -otxt
โ Step 3: Use Transcript with Ollama
You can now:
-
Transcribe audio to text using Whisper.cpp
-
Copy the text
-
Paste it into Ollama:
ollama run mistral
>>> Summarize this: <paste your transcription>
โ๏ธ Tips for Terminal Power Users
| Task | Command |
|---|---|
| Run a code model | ollama run codellama |
| Use multi-line prompts | `cat file.txt |
| Save transcript output | -otxt in Whisper |
| Test memory impact | htop or free -h during runs |
โ Summary
| Tool | Test Command |
|---|---|
| ๐ง Ollama (text) | ollama run mistral |
| ๐ฃ๏ธ Whisper.cpp (audio) | ./main -m ggml-base.en.bin -f audio.wav -otxt |
| ๐ Combine | Transcribe -> Ask AI about it |
ย
101
