The barrier to writing phone automation scripts has blocked plenty of people: you need to understand code, APIs, image-recognition parameters, and device communication. And iPhone automation is even harder — jailbreak environments are messy, and signing and distribution are painful. Many needs are simple; it is just that nobody can write them.
AI agent + AI chat is turning this into “just talk in plain language”. This article walks you through AI chat-driven iPhone automation: how the interface works, how to phrase requests, what operations are available, and what to do without an LLM configured.
1. What AI Chat Looks Like: A Chat Page by Default
Open the AI agent and the AI Chat page is the default view. The interface has four zones:
| Zone | Content |
|---|---|
| Top toolbar | Connection status, model config, history, task panel, settings, workflow editor, new session, CLI/MCP integration |
| Left | Device selection + workflow library (card list, runnable and editable) |
| Middle | Chat history with multiple conversation tabs (each tab can pick its own model) |
| Bottom | Input box; with an LLM configured you can choose which model this session uses |
Switch between 「Chat」 and 「Workflow Editor」 at the top — chat suits ad-hoc needs; the editor fixes repeatable flows.
2. How to Use It: Three Steps
- Check a device on the left (if none is selected, AI asks which phone to operate on);
- Describe your need in Chinese in the input box and press Enter (Shift + Enter for a new line);
- Wait for the AI reply; when execution is needed, a task is created — watch progress in the task panel.
That simple. No scripts, no code, no parameter tables.
3. What You Can Say to the AI
Chat is essentially “natural language mapped to automation capabilities”. Common phrasings fall into groups:
3.1 Basic operations
- “Open Safari on my first iPhone”
- “What automations are available right now?”
- “Run the login flow”
3.2 Screenshots and OCR
- “Screenshot without automation and show me the current screen” — works as long as the device is connected; no automation environment needed
- “Screenshot with automation and save the current frame” — requires starting the automation environment first
- “Read the on-screen text, see if there is a login button”
- “OCR without automation, read out all the text on screen”
3.3 Template matching
- “Find image without automation, use template btn_ok.png to locate the button” — prepare the template path in advance
- “Use visual location to find the settings button in the top right” — requires a VLM model configured
AI replies in Chinese and asks for clarification when information is missing — it will not invent flow names that do not exist.
4. Capabilities You Can Call: Screenshot, OCR, Matching, Visual Location
AI chat and the workflow editor share the same capability names. Remember this table:
| Phrase | Meaning | Automation environment needed |
|---|---|---|
| Screenshot without automation | Direct full-screen capture via Bridge | No |
| Screenshot with automation | Full-screen capture via Agent service | Yes |
| OCR without automation | Bridge capture + text recognition | No |
| OCR with automation | Agent capture + text recognition | Yes |
| Find image without/with automation | OpenCV template matching | Depends on environment |
| Find image by color without/with automation | Matching transparent/small images | Depends on environment |
For complex configuration like template paths, region cropping, and center-point extraction, use the find-image preview in the workflow editor — chat’s image-based locating is less complete than the editor, so fixed icons are more reliably handled with workflows.
5. The Workflow Library on the Left: Turn Chat into Flows
The left workflow library lists your saved main flows (sub-flows live in the editor), and each card supports:
| Action | Description |
|---|---|
| Run | Execute immediately on the selected device (select a device first) |
| Edit | Open the workflow editor to modify |
| Open folder | Open the flow file’s folder on your computer (for backup) |
| Delete | Delete this flow |
| New | Create a blank flow |
The search box above filters by name and description. Good chats deserve to become workflows — operations verified in chat, dragged into a visual flow in the editor, then run repeatedly and scheduled in batches.
6. No LLM Configured?
If no API key is configured, AI chat accepts only fixed commands:
| Input | Effect |
|---|---|
list workflows |
List all workflows |
run <workflowId> |
Run on the selected device; a device ID may be included |
Limited, but at least it runs. Still, follow the quick start to configure an LLM (DeepSeek, OpenAI, etc.) for the real natural-language experience.
7. Tips
- Chat history is saved automatically — reopen it from past sessions anytime;
- AI replies stream character by character — click cancel if you cannot wait;
- Each conversation tab can select a different model for easy comparison;
- Before executing, confirm USB device authorization and the runtime environment (automation environment / Bluetooth / OTG), or execution-type operations fail.
8. Summary
The value of AI chat-driven automation in one sentence: writing scripts becomes stating requirements.
- Fast to start: three steps — pick a device, state the need, watch the progress;
- Complete capabilities: screenshots, OCR, matching, color-keyed matching, VLM visual location, workflows — all available;
- Safe and controllable: AI invents no flows, asks when information is short, and execution is monitored in the task panel;
- A clear growth path: validate in chat → fix as a workflow → schedule in batches.
If you are an iPhone automation user who was previously turned away by “cannot write scripts”, the game has changed — no coding needed. Just say it out loud.
Ready to build it for real?
Every approach in this article can be built on the EasyClick phone automation platform — full documentation, developer tools and cluster/cloud-control products, free to try.