ย 


๐Ÿš€ Lesson: Running a Model and Testing It via Terminal (Ollama + Whisper.cpp)


๐ŸŽฏ Lesson Objective

By the end of this lesson, you’ll be able to:

  • โœ… Run your language model (e.g. Mistral, LLaMA3) with ollama

  • โœ… Interact with it via terminal (chat, summarization, etc.)

  • โœ… Run speech-to-text locally with whisper.cpp

  • โœ… Combine both for document/audio-based AI workflows


๐Ÿง  Why Use the Terminal?

The terminal is:

  • ๐Ÿ”ง A fast way to test and debug LLMs before integrating into apps

  • ๐Ÿ“‰ Lighter on memory than full UIs

  • ๐Ÿ” Secure โ€” especially when running offline models


๐Ÿงฐ Tools You Should Have Ready

Tool Use Command
๐Ÿง  Ollama Run LLMs (chat, QA, code) ollama run model_name
๐Ÿ—ฃ๏ธ Whisper.cpp Transcribe audio offline ./main -m model.bin -f file.wav

๐ŸŸข Part 1: Running a Text-Based LLM with Ollama


โœ… Step 1: Pull a Model

ollama pull mistral

Other options:

ollama pull llama3
ollama pull gemma
ollama pull codellama

โœ… Step 2: Chat with the Model in Terminal

ollama run mistral

Now youโ€™re in an interactive chat session like this:

>>> What is quantum computing?
Quantum computing is a technology...

To exit: Ctrl + C or type exit


โœ… Step 3: Use Prompt Files (Optional)

You can run a predefined prompt from a file:

ollama run mistral < prompt.txt

Where prompt.txt contains:

Summarize this:

Artificial intelligence is the simulation of human intelligence processes...

โœ… Example Use Cases

Prompt Result
Explain LangChain in simple terms. General explanation
What are the main causes of inflation? Economic summary
Write a Python function to reverse a string. Code generation

๐Ÿ”ต Part 2: Transcribing Audio with Whisper.cpp (Offline)


โœ… Step 1: Prepare Whisper.cpp

Navigate to folder:

cd whisper.cpp

Make sure you have the binary:

make

Download the model:

./models/download-ggml-model.sh base.en

โœ… Step 2: Transcribe a .wav File

./main -m models/ggml-base.en.bin -f your_audio.wav

This will produce a .txt file with the transcription.

Optional: Add -otxt to save transcript:

./main -m models/ggml-base.en.bin -f audio.wav -otxt

โœ… Step 3: Use Transcript with Ollama

You can now:

  1. Transcribe audio to text using Whisper.cpp

  2. Copy the text

  3. Paste it into Ollama:

ollama run mistral
>>> Summarize this: <paste your transcription>

โš™๏ธ Tips for Terminal Power Users

Task Command
Run a code model ollama run codellama
Use multi-line prompts `cat file.txt
Save transcript output -otxt in Whisper
Test memory impact htop or free -h during runs

โœ… Summary

Tool Test Command
๐Ÿง  Ollama (text) ollama run mistral
๐Ÿ—ฃ๏ธ Whisper.cpp (audio) ./main -m ggml-base.en.bin -f audio.wav -otxt
๐Ÿ”— Combine Transcribe -> Ask AI about it

ย 

101