# Skill: Read a local model name and select the right family, size, and quantization for given hardware

> Apply this skill when a user needs to choose a specific local model from a catalog and match it to their available memory without guesswork.

Source: sodigi-learn -- local-agents/vybor-lokalnoy-modeli -- https://sodigi.io/learn/local-agents/vybor-lokalnoy-modeli

## When to use
- User is staring at a list of model names (e.g., Llama-3-8B-Instruct-Q4_K_M) and does not know which to pick.
- User's model is running too slowly or refusing to start and you need to help them downgrade.
- User is buying a new computer and wants to know what model sizes they will be able to run.
- User asks which AI model family (Llama, Qwen, Mistral, Gemma) to start with.

## Core rules
- A model name encodes three things: family (Llama/Qwen/Mistral/Gemma), size in billions of parameters (7B/14B/70B), and quantization level (Q4/Q5/Q8).
- Family determines the training recipe and personality; for beginners any popular mid-size model from any major family works fine -- do not obsess over the "best" family.
- Parameter count (B) is the brain size: more parameters = smarter but more memory hungry and slower.
- Quantization (Q-number) is lossy compression: Q4 is the standard sweet spot (roughly 4x smaller than uncompressed, quality barely affected); Q8 is near-original quality but heavy; Q3 and below noticeably degrade quality.
- The K_M suffix (e.g., Q4_K_M) means "medium-quality within this quant level" -- prefer K_M over K_S when in doubt.
- The model file must fit in VRAM with at least 2 GB of headroom for context.
- "The model that actually runs beats the model that is theoretically smarter but crawls or crashes."
- For commercial use, check the specific model's license -- "open" does not always mean unrestricted.

## Procedure
1. Find the user's VRAM (GPU) or RAM if no GPU, and note their platform (Windows/Linux/Mac Apple Silicon).
2. Determine the maximum model file size that fits in memory minus ~2 GB buffer.
3. Map that to a parameter count using the 4-bit ballpark:
   - Q4 7B/8B ~ 4-6 GB file
   - Q4 13B/14B ~ 8-10 GB file
   - Q4 70B ~ 35-40 GB file
4. Pick a family based on the use case:
   - Llama (Meta): broad community, tons of fine-tunes, good all-rounder starting point.
   - Qwen (Alibaba): strong on code and multilingual tasks.
   - Mistral: efficient; gets high quality from small parameter counts.
   - Gemma (Google): clean, compact, some variants are multimodal.
5. Recommend an instruct-tuned variant (suffix "Instruct" or "Chat") for conversation and task use.
6. Start the recommendation at Q4_K_M; only go to Q5/Q8 if the user has memory to spare.
7. Advise the user to run a quick test (ask the model two questions) and downgrade size or quant if speed is unacceptable.

## Ready-to-use prompt
```
Help me pick the right local AI model for my hardware.

My setup:
- GPU VRAM: [X GB] (or "no GPU, using CPU+RAM: X GB")
- Platform: [Windows / Linux / macOS Apple Silicon, chip generation]
- Primary use case: [coding help / writing / Q&A / multilingual / other]
- Commercial use: [yes / no]

Please:
1. Name the maximum model size (parameter count) I can run in Q4 quantization with a safe buffer.
2. Recommend one specific model (family + size + quant variant, e.g., "Qwen2.5-7B-Instruct Q4_K_M").
3. Explain why you chose that family for my use case.
4. Tell me where to download it (Ollama pull command or LM Studio Discover search term).
5. Describe what to do if it runs too slowly (which size or quant to try next).
6. Flag any license issue if I plan to use it commercially.
```

## Pitfalls
- Recommending the largest model that technically fits: no headroom for context means crashes mid-conversation.
- Recommending an uncompressed (non-quantized) model for home hardware: it will likely not fit and always weighs several times more.
- Obsessing over which family is "best right now": rankings change monthly; any popular mid-size model is fine for starting.
- Recommending Q2/Q3 without necessity: the quality drop is noticeable; only go there if nothing larger fits.
- Ignoring the instruct/chat variant distinction: base models are not tuned for conversation and will behave unexpectedly.
- Skipping the license check for commercial projects: "open weights" and "open license" are not the same thing.