# Skill: Use multimodal model capabilities to convert screenshots, voice, and video into actionable outputs

> Lets an AI coding/design agent exploit image, audio, and video input channels to accelerate development workflows -- especially screenshot-to-code, bug reproduction via screen recording, and voice task entry -- and correctly distinguish input understanding from output generation; applies whenever a non-text artifact can replace or accelerate a text description.

Source: sodigi·learn — models/multimodalnost · https://sodigi.io/learn/models/multimodalnost

## When to use
- A user wants to build a UI element that matches a reference screenshot or sketch.
- Debugging a visual or animation bug that is hard to describe in words.
- The user is on the move and wants to dictate a task instead of typing.
- A project needs placeholder images, icons, or rough interface drafts generated programmatically.

## Core rules
- Multimodality has two independent sides: input (what the model can understand: text, image, audio, video) and output (what the model can generate: text, image, audio, sometimes video). A model can do one without the other.
- Always check the specific model's input list and output list before building a process around it -- "multimodal" alone tells you nothing.
- "Understands images" does not mean "generates images." These are different capabilities and often live in different models.
- Show, do not describe: a screenshot conveys layout, color, and structure faster and more accurately than a paragraph of text.
- Video is the most expensive channel (many frames + audio); trim a short clip containing just the relevant behavior instead of uploading a full recording.
- Never upload sensitive data -- screenshots can contain passwords, tokens, or private client data; blur or crop before sending.
- Image/screenshot-to-code gives a draft, not a pixel-perfect final; expect to refine by hand.

## Procedure
1. Confirm the model supports image input before attaching a screenshot (check the provider's model card).
2. Attach the image/screenshot and write a structured prompt (see below). Do not describe the image in text -- attach it.
3. Ask the model to first list the elements it sees in the image before generating code or analysis. This catches misunderstandings before any code is written.
4. Iterate with screenshots: if the output does not match, attach a screenshot of the current result and state specifically what needs to change.
5. For bug reproduction: record a short screen clip (5-15 seconds) showing exactly the broken behavior; attach it with a one-line description of what is wrong.
6. For voice input: dictate the task when typing is impractical; treat the transcription as a first draft and refine if needed.
7. For generated images (placeholder, icon, banner): use a separate image-generation model if the primary model does not support image output.

## Ready-to-use prompt
```
Here is a screenshot of a screen. Build a similar block.

What matters:
- stack: plain HTML and CSS, no frameworks;
- responsive: on mobile everything stacks into one column;
- pull the colors, spacing, and fonts as close to the picture as you can;
- semantic tags, clear class names.

First briefly list which elements you see in the screenshot, and only then output the code as a single file.
```

## Pitfalls
- Assuming every multimodal model can both see images and draw them -- understanding input and generating output are separate capabilities; verify both.
- Describing in text what could simply be shown -- attaching a screenshot is faster and more precise than a written layout description.
- Uploading a full-length video when a 10-second clip of the actual issue would do -- video is the heaviest and most expensive channel.
- Uploading screenshots containing API keys, passwords, or sensitive client data without blurring them first.
- Expecting generated images or screenshot-to-code output to be pixel-perfect -- multimodal output is a strong draft, not a finished product.
- Skipping the "describe what you see first" step -- asking the model to enumerate visible elements before generating code catches misinterpretations early.