What is multimodal AI?
Short answer
AI that can work with more than one kind of input or output, such as text, images, audio and video, in the same task.
Definition
Multimodal AI handles several formats at once. It can read a scanned PDF, look at a chart image, listen to audio and write text about all of it. Most major AI assistants are now multimodal to some degree.
For advisors it means you can hand an AI a photo of a document or a screenshot of a statement and get text back.
Why it matters for advisors
It widens what counts as client data in a prompt. A screenshot of an account page or a photo of a tax form is just as sensitive as typed numbers, and it's easier to share without thinking.
It also powers useful tasks: reading charts in research reports, extracting figures from scanned documents and summarizing recorded videos.
Images and screenshots can carry more than you notice. A photo of a desk or a screenshot of a dashboard may include client names, account numbers or other open windows. Crop or check before you upload.
Example
An advisor photographs a client's paper statement and asks an approved assistant to list the holdings. Because the firm's policy treats images like any other client data, the advisor uses the approved plan, not a personal phone app.
Watch out for
Images can carry more than you notice: other accounts on the page, addresses or account numbers in a corner.
Related terms
- OCR and document AITechnology that reads text and figures from scans, PDFs and photos so software can extract and analyze them.
- Generative AIAI that creates new content, such as text, summaries, images, audio or code, in response to a prompt or a document.
- Nonpublic personal informationUnder Regulation S-P, personally identifiable financial information about a client that isn't publicly available, such as balances and tax data.
Learn more
Last updated .