⚡ CLI (Interactive Mode)
FastFlowLM offers a terminal-based interactive experience, similar to Ollama, but fully offline and accelerated excusively on AMD NPUs.
🔧 Pre-Run PowerShell Commands
🆘 Show Help
flm help
🚀 Run a Model
Run a model interactively from the terminal:
flm run llama3.2:1B
flmis short for FastFlowLM. If the model isn’t available locally, it will be downloaded automatically. This launches FastFlowLM in CLI mode.
⬇️ Pull a Model (Download Only)
Download a model from Hugging Face without launching it:
flm pull llama3.2:3B
📦 List Downloaded Models
Display all locally downloaded models:
flm list
❌ Remove a Downloaded Model
Delete a model from local storage:
flm remove llama3.2:3B
📄 Run with a Text File
Load input from a local text file:
flm run llama3.2:1B "C:\Users\Public\Desktop\alice_in_wonderland.txt"
🌐 Start Server Mode
Launch FastFlowLM as a local REST API server (also support OpenAI API):
flm serve llama3.2:1B
🧠 Commands Inside Interactive Mode
Once inside the CLI, use the following commands:
🆘 Help
/?
Displays all available interactive commands. Highly recommended for first-time users.
🪪 Model Info
/show
View model architecture, size, cache path, and more.
🔄 Change Model
/load [model_name]
Unload the current model and load a new one. KV cache will be cleared.
💾 Save Conversation
/save
Save the current conversation history to disk.
🧹 Clear Memory
/clear
Clear the KV cache (model memory) for a fresh start.
📊 Show Runtime Stats
/status
Display runtime statistics like token count, throughput, etc.
🕰️ Show History
/history
Review the current session’s conversation history.
🔍 Toggle Verbose Mode
/verbose
Enable detailed performance metrics per turn. Run again to disable.
👋 Quit Interactive Mode
/bye
Exit the CLI.
⚙️ Set Hyperparameters
/set
Customize decoding parameters like
top_k,top_p,temperature, etc.
⚠️ Note: Providing invalid or extreme values may cause inference errors.