Nothing about controlling an iPhone from a computer is obvious. Here is the whole USB HID path, from why it works to the code you actually write.
1. Why iOS automation is hard in the first place
Anyone coming from Android automation knows AccessibilityService. It reads UI elements directly and simulates taps, which makes development comfortable.
iOS offers nothing comparable. The system is closed, and no accessibility interface is exposed to third parties.
So automating an iPhone comes down to two technical routes.
The first is software injection. You install a proxy app and inject test commands inside the app through the XCTest framework. It works, but it brings baggage: you need signing, then a provisioning profile, then trusting the developer manually. Signatures expire and must be redone. An app or system update can break the proxy.
The second is impersonating an external input device. A computer, or a small development board, pretends to be a real mouse and keyboard and pushes touch events through the system input channel. iOS cannot tell a physical peripheral from a program, so this route is cleaner.
The USB HID approach this article covers is the least painful variant of that second route. It needs one USB cable and no additional hardware.
For anyone doing Apple cluster control, the valuable part is this: one computer can drive many iPhones at once, injecting touches the same way on each, with identical script logic. Going from three devices to thirty does not change your code.
2. USB HID under the hood
HID stands for Human Interface Device. The mouse, keyboard, and gamepad on your desk are all HID devices.
At the protocol level, a HID device declares through standard descriptors what kind of device it is and which inputs and outputs it supports. That specification is public and natively supported by every mainstream operating system, iOS included.
The key point: when a device announces itself over USB as a HID keyboard, iOS trusts it unconditionally and treats it as a real keyboard.
That opens a neat path. Software on the computer builds a payload conforming to the HID specification saying, in effect, that a press just happened at screen coordinate (300, 500). The phone input system dispatches it as a genuine touch event.
To the system and to apps, there is no difference from a finger.
Compared with the other two HID paths
The same idea has two more implementations.
| Path | Hardware dependency | System requirement | Best for |
|---|---|---|---|
| USB HID | One USB cable | iOS 17+, central control 10.7.0+ | Least setup, single-device validation |
| Bluetooth BLE HID | ESP32C3 board | Better on iOS 18+ | Combining with no-automation screenshots to bypass mirroring entirely |
| OTG HID | ESP32S3 board | General | Running independently of a computer |
The API surface is essentially identical across all three. What differs is the link and the hardware. Choosing between them comes later.
3. Environment preparation
Confirm these before you start.
- Central control version: USB HID requires EasyClick iOS USB central control 10.7.0 or later. Earlier builds do not have this API.
- Phone system: iOS 17 or later is recommended. Some devices on 16 and below cannot take this route, and Bluetooth or OTG can substitute.
- Connection state: cable plugged in, Trust This Computer tapped on the phone, and bridging confirmed as started in the central control.
- Cable: use an original or MFi-certified cable. Many generic cables carry power but no data, so nothing happens when you plug in. This is the most common beginner trap.
For USB device authorization and the automation environment, the official docs have a dedicated page worth keeping open: https://ieasyclick.com/iosdocs/advance/ai-agent/prerequisites
4. Five APIs to get running
Every USB HID operation hangs off the usbHidEvent object. Here is a minimal working example.
function main() {
// 1. Open the session
let r = usbHidEvent.sessionStart(true);
if (!(r == null || r === "")) {
logw("Failed to open session: " + r);
return;
}
// 2. Set screen size. Coordinate conversion depends on this
r = usbHidEvent.setScreenSize(1170, 2532);
if (!(r == null || r === "")) {
logw("Failed to set size: " + r);
return;
}
// 3. Tap coordinate (300, 400)
r = usbHidEvent.clickPoint(300, 400);
logd("Tap result: " + (r == null || r === "" ? "ok" : r));
// 4. Input text via clipboard paste, works for any language
r = usbHidEvent.inputText("Hello from USB HID");
logd("Input result: " + (r == null || r === "" ? "ok" : r));
// 5. Close the session
usbHidEvent.sessionStop();
}
main();One convention you must remember
Every call above checks r == null || r === "". That is the unified convention for this API: a return value of null or an empty string means success, and any other string is the error message.
That means no try-catch is needed. Just check the return value. Wrapping it in a helper is worth it:
function _ok(r) {
return r == null || r === "";
}Parameters of sessionStart
sessionStart(gate) takes one optional parameter, gate, defaulting to true, which attempts an enhanced compatibility mode. Older systems ignore it, so leaving the default is usually right.
If a session already exists it is reused rather than recreated. When you need a hard reset, for a dropped stream or unresponsive touch, use sessionRestart. It is equivalent to sessionStop followed by sessionStart, and is more thorough than reopening alone.
The coordinate system
setScreenSize(w, h) sets the screen pixel width and height in pixels.
One point matters here: script coordinates, mirroring view coordinates, and screenshot pixel coordinates are the same system. Coordinates you measure off the mirroring view can be written directly into the script.
Note that after a resolution change or an orientation flip, you must call setScreenSize again, or every coordinate shifts. This is one of the highest-frequency pitfalls.
5. Complete API reference
Grouped by purpose for quick lookup. Full parameter documentation and examples live in the official manual: https://ieasyclick.com/iosdocs/funcs/usb-hid-event-api/
Session management
| Function | Description |
|---|---|
sessionStart(gate) |
Open a session, reusing an existing one |
sessionStop() |
Close the session and release resources |
sessionRestart(gate) |
Force a rebuild, used for troubleshooting |
Screen and coordinates
| Function | Description |
|---|---|
setScreenSize(w, h) |
Set screen pixel dimensions |
Touch operations
| Function | Description |
|---|---|
clickPoint(x, y) |
Single tap |
doubleClickPoint(x, y) |
Double tap |
press(x, y, delay) |
Long press, delay in milliseconds |
swipeToPoint(x1, y1, x2, y2, duration) |
Swipe from start to end point |
touchDown(x, y) |
Press down |
touchMove(x, y) |
Move |
touchUp(x, y) |
Lift |
multiTouch(points, timeout) |
Replay a multi-touch trace |
The trace format for multiTouch uses action 0 for down, 1 for up, and 2 for move, with delay in milliseconds per point:
let trace = [
{"action": 0, "x": 100, "y": 500, "delay": 20},
{"action": 2, "x": 100, "y": 300, "delay": 30},
{"action": 1, "x": 100, "y": 300, "delay": 20}
];
usbHidEvent.multiTouch(trace, 10000);Good for complex gestures such as circles or pinch zoom.
Text input
| Function | Behavior | When to use |
|---|---|---|
inputText(text) |
Always pastes via clipboard | The default, stable for any language |
typeText(text) |
Printable English types key by key; Chinese or emoji falls back to paste | When you need to mimic real typing |
setClipboard(text) |
Writes the clipboard only, no paste | When triggering paste manually with a key combo |
getClipboard() |
Reads the clipboard | Has known limits, see pitfalls |
Key operations
| Function | Description |
|---|---|
keyPressChar(prefix, code) |
Character key or key combination |
keyPress(key) |
Press a single key |
keyUp() |
Release all keys |
systemKey(key) |
System keys: home, recents, lock |
The combination prefix accepts alt, ctrl, gui, shift, r_ctrl, or r_shift. Pass an empty string when no combination is needed.
Simulating paste looks like this:
// gui maps to the Command key on iOS
usbHidEvent.keyPressChar("gui", "v");Volume control
| Function | Description |
|---|---|
volumeUp() |
Volume up |
volumeDown() |
Volume down |
mute() |
Mute |
6. Going further: HTTP from Python
If you would rather not write JavaScript, other languages can drive the central control over HTTP. Every endpoint is POST with a JSON body. Full request parameters, response fields, and examples in Python, Node.js, cURL, and C# are in the official OpenAPI docs: https://ieasyclick.com/iosdocs/advance/openapi/usbhid
The prefix is your central control address, normally http://127.0.0.1:8019, and paths map one to one with script functions:
| Script function | HTTP path |
|---|---|
sessionStart |
/openapi/usbhidSessionStart |
sessionStop |
/openapi/usbhidSessionStop |
sessionRestart |
/openapi/usbhidSessionRestart |
setScreenSize |
/openapi/usbhidSetScreenSize |
clickPoint |
/openapi/usbhidClickPoint |
doubleClickPoint |
/openapi/usbhidDoubleClickPoint |
press |
/openapi/usbhidPress |
swipeToPoint |
/openapi/usbhidSwipeToPoint |
touchDown/Move/Up |
/openapi/usbhidTouchDown and siblings |
multiTouch |
/openapi/usbhidMultiTouch |
inputText |
/openapi/usbhidInputText |
typeText |
/openapi/usbhidTypeText |
systemKey |
/openapi/usbhidSystemKey |
keyPressChar |
/openapi/usbhidKeyPressChar |
keyPress/keyUp |
/openapi/usbhidKeyPress, /openapi/usbhidKeyUp |
volumeUp/Down/mute |
/openapi/usbhidVolumeUp and siblings |
setClipboard |
/openapi/usbhidSetClipboard |
getClipboard |
/openapi/usbhidGetClipboard |
The HTTP response format differs from the script API. It returns JSON:
{
"code": 0,
"msg": "",
"data": ""
}A code of 0 means success. When non-zero, msg carries the error.
Python example
import requests
BASE = "http://127.0.0.1:8019"
DEVICE_ID = "your-device-id" # from the device list endpoint
def call(path, body=None):
r = requests.post(f"{BASE}{path}", json=body or {}, timeout=30)
data = r.json()
if data.get("code") != 0:
raise RuntimeError(f"{path} failed: {data.get('msg')}")
return data
# open session
call("/openapi/usbhidSessionStart", {"deviceId": DEVICE_ID, "gate": True})
# set screen size
call("/openapi/usbhidSetScreenSize", {"deviceId": DEVICE_ID, "w": 1170, "h": 2532})
# tap
call("/openapi/usbhidClickPoint", {"deviceId": DEVICE_ID, "x": 300, "y": 400})
# input text
call("/openapi/usbhidInputText", {"deviceId": DEVICE_ID, "text": "Hello"})
# close session
call("/openapi/usbhidSessionStop", {"deviceId": DEVICE_ID})This suits wiring automation into an existing Python system, such as an e-commerce backend, a test platform, or an operations ticketing flow.
7. In practice: a complete automation flow
Suppose the task is to open an app, wait for the page, tap a button, type content and submit, then return to the home screen.
function _ok(r) {
return r == null || r === "";
}
// Random wait, avoiding a mechanical rhythm
function randSleep(minSec, maxSec) {
let ms = (minSec + Math.random() * (maxSec - minSec)) * 1000;
sleep(parseInt(ms));
}
function main() {
let r = usbHidEvent.sessionStart(true);
if (!_ok(r)) {
logw("session start failed: " + r);
return;
}
// Screen size must match the current actual resolution
r = usbHidEvent.setScreenSize(1170, 2532);
if (!_ok(r)) {
logw("set size failed: " + r);
return;
}
// Step 1: back to home so the starting point is consistent
usbHidEvent.systemKey("home");
randSleep(1, 2);
// Step 2: open the target app (icon assumed at the second slot)
r = usbHidEvent.clickPoint(400, 780);
if (!_ok(r)) {
logw("open app failed: " + r);
return;
}
randSleep(4, 7); // wait for the page to load
// Step 3: tap the input field
r = usbHidEvent.clickPoint(585, 1200);
if (!_ok(r)) {
logw("tap input failed: " + r);
return;
}
randSleep(1, 2);
// Step 4: type content
r = usbHidEvent.inputText("text written by automation");
if (!_ok(r)) {
logw("input failed: " + r);
return;
}
randSleep(1, 2);
// Step 5: tap submit
r = usbHidEvent.clickPoint(585, 1600);
if (!_ok(r)) {
logw("submit failed: " + r);
return;
}
randSleep(2, 4);
// Step 6: back to home
usbHidEvent.systemKey("home");
usbHidEvent.sessionStop();
logd("flow completed");
}
main();A few notes.
Check the return value every step. Once one step fails, everything after it runs on a false premise and can make things worse. Fail early, exit early, and the log points straight at the problem.
Randomize the waits. A fixed interval is itself a machine signature. A real person might spend two seconds on a page or ten. Wrapping it in a random function measurably lowers detection probability.
Measure coordinates from screenshots. Values measured off the mirroring view are directly usable. Do not copy numbers from someone else’s script, because the coordinate system differs by model.
8. Pitfalls worth knowing
These are all from actual use, ordered by how often they show up.
Coordinates shift as a whole
The most common cause is a missing setScreenSize call, or forgetting to call it again after an orientation flip.
A subtler cause is mixed device models. Different models have different resolutions, so a coordinate set that is right on an iPhone 11 is off on an iPhone 12. For batch scenarios, keep devices to a single model.
Devices keep dropping off
Work through this order: the cable, then USB port power, then the hub, then device count. Swap in an original cable, prefer ports wired straight to the motherboard, use a hub with independent power, and keep any single hub under seven or eight devices.
Extra spaces appear when pasting English
This is an iOS keyboard setting, not the API. On the phone, go to Settings, General, Keyboard, and turn off Smart Punctuation. The problem disappears.
getClipboard has known limits
This function reads through CoreDevice. Reading immediately after setClipboard writes normally works.
Reading content copied manually by long-pressing on the phone, though, times out or returns nothing on some iOS versions, 26.x among them, and in bad cases wedges the device clipboard service until the phone restarts.
Do not rely on it to read human-copied content. The steadier approach today is routing around it with a Shortcut, or simply having the script write and read its own values.
Overnight system updates breaking scripts
Any device running automation must have automatic updates turned off. One silent upgrade overnight can leave the entire flow unable to run the next morning. That setting is mandatory, not optional.
9. Choosing between the three HID paths
Back to the selection question.
USB HID suits quick validation and single-device work. One cable, no extra hardware, at the cost of staying tethered.
Bluetooth BLE HID needs an ESP32C3 board, a few dollars. Its distinctive advantage is pairing with the no-automation screenshot mode so the whole chain avoids screen mirroring. That matters when detection pressure is real, and it costs you low mirroring frame rates and fussier configuration.
OTG HID uses an ESP32S3 board wired straight to the phone, and can run without a computer, which suits deployments that need independence.
To get started, begin with USB HID and upgrade as needed.
Both other paths have documentation: Bluetooth BLE at https://ieasyclick.com/iosdocs/funcs/ble-event-api and OTG HID at https://ieasyclick.com/iosdocs/funcs/otg-event-api . Each also has its own setup tutorial.
For how the three compare on cost and capability, there is a dedicated comparison on this blog that is faster to read than prose: three HID paths compared.
10. Summary
The core idea of USB HID is that the computer impersonates a HID device and injects touch events through the system input channel. iOS cannot tell real from injected, so no jailbreak and no signing are needed.
That also makes it the least troublesome no-jailbreak path for Apple cluster control: no Bluetooth board and no OTG board, just one cable, with scripts reusable across devices.
At the API level, remember three things. Open the session first, set the screen size correctly, and treat null or an empty string as success.
Troubleshooting order is essentially fixed: physical connection first, parameters second, code logic last.
Reference documentation
The interfaces and configuration covered here are documented in full:
- EasyClick iOS USB documentation: https://ieasyclick.com/iosdocs/
- USB HID function manual (
usbHidEvent): https://ieasyclick.com/iosdocs/funcs/usb-hid-event-api/ - USB HID OpenAPI over HTTP, with multi-language examples: https://ieasyclick.com/iosdocs/advance/openapi/usbhid
- Bluetooth BLE functions: https://ieasyclick.com/iosdocs/funcs/ble-event-api
- OTG HID functions: https://ieasyclick.com/iosdocs/funcs/otg-event-api
- Environment and device authorization: https://ieasyclick.com/iosdocs/advance/ai-agent/prerequisites
- Apple cluster control overview: https://ieasyclick.com/apple_qunkong/
Related reading on this blog, from different angles:
- How USB HID is reshaping Apple cluster control, on the landscape shift rather than the code
- Three HID paths compared, covering cost and capability boundaries
- The no-signature Bluetooth HID route and OTG HID setup, for the other two links
About EasyClick: A phone automation AI-agent platform covering Android no-root, iOS no-jailbreak and HarmonyOS Next, offering script development, Apple cluster control, local central control & mirroring, and cloud control systems. → Explore all products
Ready to build it for real?
Every approach in this article can be built on the EasyClick phone automation platform — full documentation, developer tools and cluster/cloud-control products, free to try.