yuechen-c/ios-screen-record

About A python based Apple Quicktime protocol，you can record audio and video from real iOS devices

介绍

本应用程序使用 python 实现,可以通过 USB 连接 iOS 设备进行屏幕共享

• 高帧率（30〜60fps）
• 高画质
• 低延迟（<200ms）
• 非侵入性
• 支持多设备并行

window 支持并查看 window 分支 查看文档:windows.md

Mac OSX 安装

python >=3.7

1. brew install libusb pkg-config
2. 如需使用 gstreamer 则需要安装 brew install gstreamer gst-plugins-bad gst-plugins-good gst-plugins-base gst-plugins-ugly
3. pip install ioscreen

使用

# vlc 工具播放udp地址： udp/h264://@:8880
# 转发 h264 udp 广播，Mac 本身限制了 udp 发送，延迟会越来越大（仅测试使用）
$ ioscreen --udid=xxxx udp

# 录制 h264/wav 文件
$ ioscreen --udid=xxxx record -h264File=/home/out.h264  -wavFile=/home/out.wav

# gstreamer 渲染显示画面
$ ioscreen --udid=xxxx gstreamer

QQ 交流群：37042417

ios-screen-record's Issues

录制ios手机，会导致手机上没有声音

这个是不是usbWrite的时候同时获取了音频和视频呢？我们希望保留手机上的声音，是不是可以通过修改这里的PingPacketMagic来修改获取的内容呢？但是苦于没有文档，求助

使用ioscreen录制h264/wav会修改设备的显示时间

执行命令
ioscreen -u=xxxxx record -h264File=test1.h264 -wavFile=test2.wav
设备型号：iPhone 12
设备系统：14.1
效果如下视频
https://user-images.githubusercontent.com/12739697/130032815-de58a1d1-0175-49c4-9830-333f95271d68.MP4

vlc 无法播放udp流 udp/h264://@:8880

1.运行 ioscreen --udid=xxxx udp
2.vlc 输入网址 udp/h264://@:8880
3.vlc 依旧白屏，无视频流播放

录屏会导致webdriverAgent连接断开

录屏会导致webdriverAgent连接断开，我尝试用先启动录屏，但是不开始读取流数据，等wda server启动之后的方式解决这个问题，但是在等待wda server启动后在读取流时会报错WARNING: [Errno 60] Operation timed out，无法正常读取流

def writeStream(self):
        """ 异步写入线程
        :return:
        """
        while True:
            try:
                data = self.device.read(self.inEndpoint, 1024 * 1024, 3000)
                self.byteStream.put(data)
            except Exception as E:
                logging.warning(E)
                self.message.outEndpoint = None
                self.message.inEndpoint = None
                self.stopSignal.set()
                break

录制帧率只有25FPS

录制帧率只有25，播放变成慢动作了。
mac os 11.3.1 iphone 11 iOS 13.6

libusb0-dll:err [_usb_reap_async] timeout error

2023-04-06 14:40:28,868 - util.py[line:95] - INFO: Disabling hidden QT config
2023-04-06 14:40:28,869 - util.py[line:156] - INFO: enable_qt_config..
2023-04-06 14:40:28,869 - util.py[line:75] - INFO: Enabling hidden QT config
2023-04-06 14:40:28,872 - util.py[line:62] - INFO: Find Device UDID: 518c8489fe16c02a4af34c224034e37bddc9f3a6
2023-04-06 14:40:28,889 - util.py[line:180] - INFO: USB connection ready, waiting for ping..
接收： array('B', [16, 0, 0, 0, 103, 110, 105, 112, 0, 0, 0, 0, 1, 0, 0, 0])
写入: b'\x10\x00\x00\x00gnip\x00\x00\x00\x00\x01\x00\x00\x00'
2023-04-06 14:40:28,913 - meaasge.py[line:148] - INFO: 接收到 PING
2023-04-06 14:40:28,913 - meaasge.py[line:149] - INFO: AudioVideo-Stream has start success
2023-04-06 14:40:58,901 - util.py[line:196] - WARNING: [Errno None] b'libusb0-dll:err [_usb_reap_async] timeout error\n'
2023-04-06 14:40:58,901 - meaasge.py[line:160] - INFO: Telling device to stop streaming..
2023-04-06 14:40:58,901 - util.py[line:95] - INFO: Disabling hidden QT config

Process finished with exit code 0

usb.core.USBError: [Errno 5] Input/Output Error

Hi,

I am trying to record iOS screen video using a Mac.
Command used: ioscreen -u="00008103000E09D23ADA001E" record -h264File=out.h264 -wavFile=out.wav
Device details: iPad connected to Mac using type C - C connector.
Error:
2023-01-31 12:14:42,307 - util.py[line:58] - INFO: Find Device UDID: 00008103000E09D23ADA001E
2023-01-31 12:14:42,307 - util.py[line:91] - INFO: Disabling hidden QT config
2023-01-31 12:14:42,316 - util.py[line:153] - INFO: enable_qt_config..
2023-01-31 12:14:42,316 - util.py[line:71] - INFO: Enabling hidden QT config
2023-01-31 12:14:42,319 - util.py[line:58] - INFO: Find Device UDID: 00008103000E09D23ADA001E
Traceback (most recent call last):
File "/opt/homebrew/bin/ioscreen", line 8, in
sys.exit(main())
File "/opt/homebrew/lib/python3.10/site-packages/ioscreen/main.py", line 56, in main
args.func(args)
File "/opt/homebrew/lib/python3.10/site-packages/ioscreen/main.py", line 12, in cmd_record_wav
start_reading(consumer, device, stopSignal)
File "/opt/homebrew/lib/python3.10/site-packages/ioscreen/util.py", line 164, in start_reading
device.ctrl_transfer(0x02, 0x01, 0, 0x86, b'')
File "/opt/homebrew/lib/python3.10/site-packages/usb/core.py", line 1072, in ctrl_transfer
ret = self._ctx.backend.ctrl_transfer(
File "/opt/homebrew/lib/python3.10/site-packages/usb/backend/libusb1.py", line 893, in ctrl_transfer
ret = _check(self.lib.libusb_control_transfer(
File "/opt/homebrew/lib/python3.10/site-packages/usb/backend/libusb1.py", line 604, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 5] Input/Output Error

How to solve this error? - is there any workaround possible? Your feedback would be of great help!

我想获取视频数据显示在前端页面该从哪获取呢

楼主没遇到gstreamer crash吗？

MacOS 15.7

quicktime_hack项目，,/qth gstreamer的话，gstreamer大概率会crash的

大佬，这个逻辑是想避免除0写错了吗？

CMTime.py里的代码

gstreamer 渲染显示画面命令报错

报错信息：

执行命令：
ioscreen --udid=00008020-000E08CC1A38002E gstreamer
报错说找不到设备，后来看了下issue，把-删除变成这样：
ioscreen --udid=00008020000E08CC1A38002E gstreamer
依旧报错如上图

使用gstreamer发生异常，这个是什么原因？

使用udp或录制文件功能正常。但是使用gstreamer就报错，这个是什么原因？
环境：macOS Monterey 12.3.1 gstreamer 1.20.1

$ python3 main.py --udid=d9109d1133548f70456c8c150a63cdc610a6ddd8 gstreamer 2022-05-02 20:06:17,642 - util.py[line:58] - INFO: Find Device UDID: d9109d1133548f70456c8c150a63cdc610a6ddd8 2022-05-02 20:06:17,731 - gstreamer.py[line:124] - INFO: Starting Gstreamer.. Traceback (most recent call last): File "/Users/weiyansheng/work/Owner/ios-screen-record/ioscreen/main.py", line 60, in <module> main() File "/Users/weiyansheng/work/Owner/ios-screen-record/ioscreen/main.py", line 56, in main args.func(args) File "/Users/weiyansheng/work/Owner/ios-screen-record/ioscreen/main.py", line 28, in cmd_record_gstreamer consumer = GstAdapter.new(stopSignal) File "/usr/local/lib/python3.9/site-packages/ioscreen/coremedia/gstreamer.py", line 126, in new videoAppSrc = setup_video_pipeline(pipe) File "/usr/local/lib/python3.9/site-packages/ioscreen/coremedia/gstreamer.py", line 27, in setup_video_pipeline src.set_property("is-live", True) AttributeError: 'NoneType' object has no attribute 'set_property'

ubuntu下空闲设备接入，执行record时显示usb.core.USBError: [Errno 16] Resource busy错误

2022-08-24 04:53:58,088 - util.py[line:91] - INFO: Disabling hidden QT config  
Traceback (most recent call last):  
File "/home/root/.local/bin/ioscreen", line 8, in  
sys.exit(main())  
File "/home/root/.local/lib/python3.8/site-packages/ioscreen/main.py", line 56, in main  
args.func(args)  
File "/home/root/.local/lib/python3.8/site-packages/ioscreen/main.py", line 12, in cmd_record_wavstart_reading(consumer, device, stopSignal)  
File "/home/root/.local/lib/python3.8/site-packages/ioscreen/util.py", line 152, in start_reading
device.set_configuration()  
File "/home/root/.local/lib/python3.8/site-packages/usb/core.py", line 905, in set_configuration
self._ctx.managed_set_configuration(self, configuration)  
File "/home/root/.local/lib/python3.8/site-packages/usb/core.py", line 113, in wrapper
return f(self, *args, **kwargs)  
File "/home/root/.local/lib/python3.8/site-packages/usb/core.py", line 159, in managed_set_configuration
self.backend.set_configuration(self.handle, cfg.bConfigurationValue)  
File "/home/root/.local/lib/python3.8/site-packages/usb/backend/libusb1.py", line 812, in set_configuration
_check(self.lib.libusb_set_configuration(dev_handle.handle, config_value))  
File "/home/root/.local/lib/python3.8/site-packages/usb/backend/libusb1.py", line 604, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])  
usb.core.USBError: [Errno 16] Resource busy```



问题似乎出现在`device.set_configuration(config_index)`函数上，我把这个函数注释掉后执行命令
```2022-08-30 04:55:11,230 - util.py[line:58] - INFO: Find Device UDID: 00008101001645560C42001E  
2022-08-30 04:55:11,232 - util.py[line:91] - INFO: Disabling hidden QT config  
2022-08-30 04:55:11,233 - util.py[line:155] - INFO: enable_qt_config..  
2022-08-30 04:55:11,233 - util.py[line:71] - INFO: Enabling hidden QT config  
2022-08-30 04:55:11,235 - util.py[line:58] - INFO: Find Device UDID: 00008101001645560C42001E  
2022-08-30 04:55:11,238 - util.py[line:178] - INFO: USB connection ready, waiting for ping..  
2022-08-30 04:55:14,243 - util.py[line:193] - WARNING: [Errno 110] Operation timed out  
2022-08-30 04:55:14,243 - meaasge.py[line:168] - INFO: Telling device to stop streaming..  
2022-08-30 04:55:14,243 - util.py[line:91] - INFO: Disabling hidden QT config```


bytestream内似乎接受不到data数据。

请问有什么好的建议，或者解决办法么。不胜感激

这个工具能支持在Windows连接iPhone场景下录屏吗

执行：pip install ioscreen，出现以下错误：
Building wheel for PyGObject (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: 'F:\GitLab\apprunner\venv\Scripts\python.exe' 'F:\GitLab\apprunner\venv\lib\site-packages\pip_ven
dor\pep517_in_process.py' build_wheel 'D:\Users\yl1150\AppData\Local\Temp\tmpg86y3yx1'
cwd: D:\Users\yl1150\AppData\Local\Temp\pip-install-ia3oem36\pygobject_e138554edb0d49deb98def802edfefc4

Complete output (42 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.8
creating build\lib.win-amd64-3.8\pygtkcompat
copying pygtkcompat\generictreemodel.py -> build\lib.win-amd64-3.8\pygtkcompat
copying pygtkcompat\pygtkcompat.py -> build\lib.win-amd64-3.8\pygtkcompat
copying pygtkcompat_init_.py -> build\lib.win-amd64-3.8\pygtkcompat
creating build\lib.win-amd64-3.8\gi
copying gi\docstring.py -> build\lib.win-amd64-3.8\gi
copying gi\importer.py -> build\lib.win-amd64-3.8\gi
copying gi\module.py -> build\lib.win-amd64-3.8\gi
copying gi\pygtkcompat.py -> build\lib.win-amd64-3.8\gi
copying gi\types.py -> build\lib.win-amd64-3.8\gi
copying gi_constants.py -> build\lib.win-amd64-3.8\gi
copying gi_error.py -> build\lib.win-amd64-3.8\gi
copying gi_gtktemplate.py -> build\lib.win-amd64-3.8\gi
copying gi_option.py -> build\lib.win-amd64-3.8\gi
copying gi_ossighelper.py -> build\lib.win-amd64-3.8\gi
copying gi_propertyhelper.py -> build\lib.win-amd64-3.8\gi
copying gi_signalhelper.py -> build\lib.win-amd64-3.8\gi
copying gi_init_.py -> build\lib.win-amd64-3.8\gi
creating build\lib.win-amd64-3.8\gi\repository
copying gi\repository_init_.py -> build\lib.win-amd64-3.8\gi\repository
creating build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\Gdk.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\GdkPixbuf.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\GIMarshallingTests.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\Gio.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\GLib.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\GObject.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\Gtk.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\keysyms.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides\Pango.py -> build\lib.win-amd64-3.8\gi\overrides
copying gi\overrides_init_.py -> build\lib.win-amd64-3.8\gi\overrides
running build_ext
pycairo: new API
pycairo: trying include directory: 'D:\Users\yl1150\AppData\Local\Temp\pip-build-env-46pn1i2p\overlay
\Lib\site-packages\cairo\include'
pycairo: found 'D:\Users\yl1150\AppData\Local\Temp\pip-build-env-46pn1i2p\overlay\Lib\site-packages
\cairo\include\py3cairo.h'
building 'gi._gi' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://vi
sualstudio.microsoft.com/visual-cpp-build-tools/

ERROR: Failed building wheel for PyGObject
Failed to build PyGObject
ERROR: Could not build wheels for PyGObject which use PEP 517 and cannot be installed directly

ubuntu 下不能使用

我在ubuntu 20.04下安装pip install ioscreen ;连接iphone6s手机，执行出现如下错误，这个是怎么回事。
root@ubuntu:~/Downloads$ ~/.local/bin/ioscreen --udid=4e50da4c5b930e51d8a37f8ec407864aea9d800c record -h264File=/home/root/a.h264 -wavFile=/home/root/b.wav
2022-08-24 04:53:58,087 - util.py[line:58] - INFO: Find Device UDID: 4e50da4c5b930e51d8a37f8ec407864aea9d800c
2022-08-24 04:53:58,088 - util.py[line:91] - INFO: Disabling hidden QT config
Traceback (most recent call last):
File "/home/root/.local/bin/ioscreen", line 8, in
sys.exit(main())
File "/home/root/.local/lib/python3.8/site-packages/ioscreen/main.py", line 56, in main
args.func(args)
File "/home/root/.local/lib/python3.8/site-packages/ioscreen/main.py", line 12, in cmd_record_wav
start_reading(consumer, device, stopSignal)
File "/home/root/.local/lib/python3.8/site-packages/ioscreen/util.py", line 152, in start_reading
device.set_configuration()
File "/home/root/.local/lib/python3.8/site-packages/usb/core.py", line 905, in set_configuration
self._ctx.managed_set_configuration(self, configuration)
File "/home/root/.local/lib/python3.8/site-packages/usb/core.py", line 113, in wrapper
return f(self, *args, **kwargs)
File "/home/root/.local/lib/python3.8/site-packages/usb/core.py", line 159, in managed_set_configuration
self.backend.set_configuration(self.handle, cfg.bConfigurationValue)
File "/home/root/.local/lib/python3.8/site-packages/usb/backend/libusb1.py", line 812, in set_configuration
_check(self.lib.libusb_set_configuration(dev_handle.handle, config_value))
File "/home/root/.local/lib/python3.8/site-packages/usb/backend/libusb1.py", line 604, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 16] Resource busy

udp play error

Hi,

I have encountered an error while trying to play udp:

command:

ffplay -f h264 udp://127.0.0.1:8880

The output :

[NULL @ 0x7fdb6603be00] non-existing PPS 0 referenced [h264 @ 0x7fdb6a017e00] non-existing PPS 0 referenced [h264 @ 0x7fdb6a017e00] decode_slice_header error [h264 @ 0x7fdb6a017e00] no frame! [NULL @ 0x7fdb6603be00] non-existing PPS 0 referenced 0B f=0/0 [h264 @ 0x7fdb6a018400] non-existing PPS 0 referenced [h264 @ 0x7fdb6a018400] decode_slice_header error [h264 @ 0x7fdb6a018400] no frame! [NULL @ 0x7fdb6603be00] non-existing PPS 0 referenced 0B f=0/0 [h264 @ 0x7fdb6a018a00] non-existing PPS 0 referenced [h264 @ 0x7fdb6a018a00] decode_slice_header error [h264 @ 0x7fdb6a018a00] no frame! [NULL @ 0x7fdb6603be00] non-existing PPS 0 referenced [h264 @ 0x7fdb6a019000] non-existing PPS 0 referenced [h264 @ 0x7fdb6a019000] decode_slice_header error [h264 @ 0x7fdb6a019000] no frame! [NULL @ 0x7fdb6603be00] non-existing PPS 0 referenced [h264 @ 0x7fdb6a019600] non-existing PPS 0 referenced [h264 @ 0x7fdb6a019600] decode_slice_header error [h264 @ 0x7fdb6a019600] no frame!

Thanks

在iphone那里停止镜像后要重启手机才能正常录制

若udid 中含有符号 - 则提示找不到手机，ioscreen -u 694a32 udp运行，提示超时并关闭程序

1.(testdevice) MacBook-Pro ~ % ioscreen -u 694a32 udp

• util.py[line:58] - INFO: Find Device UDID: 694a32
• consumer.py[line:100] - INFO: listen UDP: udp/h264://127.0.0.1:8880
• util.py[line:91] - INFO: Disabling hidden QT config
• util.py[line:152] - INFO: enable_qt_config..
• util.py[line:71] - INFO: Enabling hidden QT config
• util.py[line:58] - INFO: Find Device UDID: 694a32
• util.py[line:175] - INFO: USB connection ready, waiting for ping..
• meaasge.py[line:148] - INFO: AudioVideo-Stream has start success
• util.py[line:190] - WARNING: [Errno 60] Operation timed out
• meaasge.py[line:159] - INFO: Telling device to stop streaming..
• util.py[line:91] - INFO: Disabling hidden QT config
  手机运行ioscreen -u 694a32 udp ，AudioVideo-Stream提示成功，过几秒后，提示超时，程序自动关闭
  2.若udid 中含有符号 - 则提示找不到手机