simonmitchell/rocc

A Swift framework for remote control of digital Cameras

rocc's Issues

Fix aperture not working if device is set to European decimal formatting

Steps to reproduce:

Set camera to a European language which uses "," as the decimal separator instead of ".". See that aperture values fail to parse and therefore the Event object fails to include

Actual behavior:

Aperture values are not parsed, and therefore not available to consumers of this library.

Expected behavior:

Aperture values should be correctly parsed and handed on to users of the library!

Rocc version:

2.0.0

iOS/macOS version:

N/A

Camera manufacturer and model:

a6500

Other questions

Does the issue still occur after updating your Camera's Firmware?

Yes! Firmware is fully up to date

Investigate 40401 error on RX0

Steps to reproduce:

Try and call getCameraFunction rapidly after just connecting to RX0.

Actual behavior:

The function call returns a 40401 error (Camera not ready)

Expected behavior:

This function call should not return a 40401

System configuration

Rocc version:

1.0.2

iOS/macOS version:

iOS 13.2.1

Camera manufacturer and model:

Sony RX0

Implement PTP IP ping functionality

Add parsing logic for PTP IP standard calls

• InitCommandAck
• InitEventAck
• CommandResponse
• Event
• StartDataPacket
• DataPacket
• EndDataPacket
• Pong
• Ping

Add new PTP IP device properties to Event object

Add support for parsing and creating Sony PTP IP camera

Changing Drive mode and Focus mode not reflected on camera

Steps to reproduce:

Change focus mode or Drive mode

Actual behavior:

The camera does not reflect the changes

Expected behavior:

The drive mode and Focus mode on the camera should be updated to reflect the change in the app as with shutter, iso and aperture

System configuration

Rocc version:

2.0

iOS/macOS version:

13.5

Camera manufacturer and model:

Sony A9

Other information about your system(s):

Firmware 6.0

Other questions

Does the issue still occur after updating your Camera's Firmware?

Have you tried any other versions of NVDA?

Fix crash on launch on a9ii firmware 2.0

Unfortunately whilst we have recently added support for control of the a9ii (see 2.0.0 release branch), it seems that either in the latest a9ii firmware, or just on the a9ii Sony have changed the format of the live view stream.

Luckily I managed to find a user with experience using WireShark so we have a trace of this data which I have put part of here

I have tried to attempt to break this payload down... it seems to have some resemblance of an RTP stream (In particular, ffmpeg seems to think it's mjpeg) but the format does not match RFC 2435.

I'm not sure whether we aim to try and reverse-engineer their protocol entirely, or if we simply try and do a hacky fix to simply remove their RTP headers from the stream of data (Which could break in the future). I'm not even sure if there's a reliable way to detect RTP headers in the Data, or if we even get sent them or they're removed by URLSession!

Here's my attempt at breaking down the stream:

// RTP Header

HEX: 88 
Binary:     10    |      0     |          0          |        1000        |
        Version 2 | No Padding | No extension header | 8 CSRC Identifiers |

00    // No type, marker, e.t.c
00 00 // Sequence number 0

00 5e 00 00 // Timestamp (24064)
10 00 00 00 // SSRC Identifier (16)

// CSRC Identifiers (8 of them)
78 00 00 00 
0d 0a 37 38 
0d 0a 65 00 
00 00 00 00 
00 00 01 00 
00 00 00 00 
00 00 02 00 
00 00 00 00 


00 // Type-specific, not the correct type for JPEG stream
00 00 00 // Fragment Offset, this never goes above zero so can't be right!
00 // Type (0)
00 // Q (0)
03 // Width (3 * 8) = 24... this can't be right!
00 // Height (0 * 8) = 0
00 00 
03 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 
0a 00 00 80 07 00 00 00 00 00 
00 00 00 00 00 00 0a 00 00 80 
07 00 00 00 00 00 00 00 00 00 
00 00 0a 00 00 80 07 00 00 00 
00 00 00 00 00 00 00 00 0a 00
00 80 07 00 00 00 00 00 

00 00
00 00 0d 0a 35 45 30 30 0d 0a

// Beginning of JPEG segment

ff d8 ff db 00 84 00 03 03 03 
03 03 03 05 05 05 05 05 05 05 
05 05 0a 08 05 05 08 0a 0d 0a 

Fix timelapse shooting on PTP cameras

Steps to reproduce:

Take photos in a loop (either using code and Rocc, or Camrote)

Actual behavior:

The app hangs after a while, waiting for a callback that never arrives

Expected behavior:

The routine continues until the battery is drained :)

System configuration

Rocc version:

f3763b8 / Camrote 1.6.5

iOS/macOS version:

iOS 14.2

Camera manufacturer and model:

RX100 VII

Other questions

Does the issue still occur after updating your Camera's Firmware?:

Yes

Have you tried any other versions of NVDA?

What's NVDA? :)

---

I think I've seen this hanging in every possible completion callback in https://github.com/simonmitchell/rocc/blob/master/Sources/Manufacturer%20Implementations/Sony/PTP%20IP%20Camera/SonyPTPIPCamera+TakePicture.swift#L22 Moreover, my app waits for photo download before starting a new capture, and sometimes it hangs there, too (i.e. no JPEG is recognized even though objectAdded and objectRemoved events were received as usual).

My current hypothesis is that when we execute a command that is immediately followed by getAllDevicePropDesc (because some event was received) the camera assumes that responses to previous queries were included in "all device props" and doesn't bother dealing with old commands. This is supported by the fact that transactionId of responses seems to always monotonic (warning: this is not scientific, that's just what I saw with my sample sequences).

Unfortunately, so far I wasn't able to reliably serialize commands (so that getAllDevicePropDesc is called only after all previous commands finished) without any side effects. I also tried timing out all completion callbacks that were awaiting transactionIds older than the last received one. Again, that didn't seem to fully resolve the issue and caused problems with exposure settings.

Another idea is that there might be some field in getAllDevicePropDesc that instructs how to handle outstanding requests.

I'm a bit reluctant to attach my Wireshark sessions here because:

1. they might contain private data and I haven't had the time to clean them up
2. they contain random experimentations and I have no confidence they represent the normal Rocc behavior
3. if anyone is interested and has time to investigate the issue, I'll be more than happy to prepare them a custom-tailored Wireshark session from my RX100 VII. :)

In terms of next steps, I'm working on a custom Wireshark dissector that will more reliably detect Sony PTPIP packets based on this work: https://github.com/sboisson/wireshark-ptpip (GPL alert, don't base your apps on that, use only as an external validation tool). My hope is that it'll be easier to spot differences between Sony's app behavior and Rocc.

Add data sending and receiving to PTP IP Client

Do we have android version of rocc?

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Proposal: Supporting other camera manufacturers

This issue will be a log of my plan of action and implementation of support for camera manufacturers other than Sony in ROCC!

Motivation

I want ROCC (as well as Camrote) to be as useful a tool as possible for as many photographers and developers as possible. Until now my main focus has been on Sony cameras (with that being the brand of camera I own) but after implementing PTP/IP for Sony’s latest lineup of cameras I – unbeknownst to me – did a lot of the groundwork for supporting a lot of other manufacturers!

Roadmap

Release v2.0.0

As Camrote (as well as other iOS apps) rely on a stable library for controlling Sony cameras it is vitally important that none of the work done when refactoring PTP/IP support (a necessity to reduce complexity and duplicate code across manufacturers) breaks support for these models.

For this reason, the number one priority is releasing v2.0.0 of Camrote along with an extensive suite of tests for Sony devices using PTP/IP support. This is already underway but requires some time as it is a painstaking process making sure hex packets being sent and received are spot-on.

It is also important to make Camrote as accessible as possible to devs, so I will also be concentrating on adding support for SPM in this release if possible without refactoring large areas of the codebase.

v2.x.x Releases

I am very optimistic that support for other manufacturers will be able to be added without any breaking changes now that Sony PTP/IP support has been completed, however this may need to be re-assessed at a later date depending on implementation discoveries when trying to refactor or implement support for other manufacturers.

Whilst adding support is theoretically possible without me having test devices (which I am unable to afford at present) I am hoping I will be able to lean heavily on technical folk in the photographical community to test beta releases of Camrote, or even to develop support for other manufacturers themselves via forking and submitting pull requests to this repo.

Once I have refactored the project to a point where I think this is possible (using a wireshark dump of the Canon remote app provided by a kind Reddit user) I will create a guide on how to implement and test other manufacturers, and likely also produce a demo app to test functionality without having to write any code.

Task Tracker

2.0.0

• Add tests for SonyPTPIPCamera
• Add support for SPM

2.1.0 (Canon)

• Refactor camera discovery and XML parsing to support multiple manufacturers
• Refactor SonyPTPIPCamera to PTPIPCamera using libgphoto2 as a guide for what standard behaviour is and overriding non-standard behaviour in SonyPTPIPCamera
• Refactor conversion code for converting between Camrote types and binary data to be sent to camera!
• Add support for Canon cameras using WireShark logs provided by kind togs!

2.x.x Further Manufacturers

• Provide guidelines on how best to add support for further manufacturers
• Write a demo/debug iOS app for debugging support for new manufacturers

What Can You Do?

ROCC (And Camrote) are very much part-time ventures for me, and I’m not at a point in my life when I can invest money (or large amounts of time) in them! There are many ways you can support the ongoing development, both financially and non-financially!

Provide input/insight on this proposal via the comments

This is the first library that I have ever sole-developed that has had such a large scope and has had this many stars... I am not by any stretch of the imagination an expert in computer science or networking technologies so any advice/insight etc on any of the code or approaches is greatly appreciated!

Provide financial help

I do not have the money to buy devices from all manufacturers, but any money that is donated via the Camrote website, via my buymeacoffee or via the “fund me” options in Camrote (please don’t use this, as Apple get a 30% cut) will go directly towards buying test gear!

Contribute!

Once i have guidelines out for how to add support for other manufacturers (or even before) please contribute to ROCC (in any way) to make sure it’s as good and as stable as possible.

Send me Wireshark/Packet Dumps!

If you don’t have the money to contribute financially, but happen to have a camera from another manufacturer please, please, please provide annotated Wireshark dumps from the remote control app your manufacturer provides! I will be providing instructions on how to do this shortly...

Send me your old gear!

We all have old cameras lying around (sadly my old Canon was too old to have a WiFi chip), if you do, and it’s just gathering dust, or broken horribly (but still controllable remotely) then please consider donating it to the cause! If you want details on how you can do this then please contact me: [email protected]

Thank you for taking the time to read this! I look forward to all of your input ☺️

Convert appropriate event/function properties to enums

Things like WhiteBalance.Value.mode can be changed to an enum rather than a string!

Add new shooting modes to enum

PTP IP on Sony has new shooting modes which are returned by the stillCaptureMode object in events. Instead of adding a still capture mode property, I'd like to enhance ShootMode to include Bracketed and other relevant ones returned via PTP IP and then munge the data in ROCC to support this.

ROCC framework question

hello

I would like to ask, can ROCC frame support SONY camera to view photos and download on SD card?

Can I receive information about newly taken and deleted photos?

Cheers.

Add fetching and parsing of all device props from Sony PTP IP camera

Add support for "super slow rec" mode returned by RX0

Describe the solution you'd like

Add super slow rec ("super slow rec") to shoot mode enum

Investigate AF w/shutter on PTP/IP models

"Hi, I’m using an A7RIV and started by playing around with Sony’s app before trying yours. The reason being, like many photographers I use back button focus and turn off AF w/ shutter. Unfortunately, that means in Sony’s app that I can’t focus unless I turn it back on. Is this a camera limitation or is there any way you can get around this via your app?"

Add support for setting device props via PTP IP

Investigate migrating SSDP code to NWConnectionGroup on iOS 14

NWConnectionGroup is a much more usable API than CFSocket... CFSocket is ancient, and difficult to use, and means we have to handle network changes ourselves. If we can move over to using the Networking framework (At least on iOS 14 for now) we can remove some of the tricky reachability stuff we have going on and let Apple handle it for us.

Will need to think of a clever way to keep support for both so we can still support iOS < 14, perhaps making UDPClient a protocol and setting it using an if #available( statement in the initialiser of the UDP device discovery.

Add support for package managers

Add support for popular package managers:

• Carthage
• Swift Package Manager (if possible with ThunderRequest)

Implement Sony PTP IP handshake

isConnected does not update to true.

Steps to reproduce:

Connect to camera observe isConnected after:
camera.connect { (error, isInTransferMode) in

Actual behavior:

isConnected is false

Expected behavior:

isConnected should be true

System configuration

Mac OS iOS 13.3.1 Sony A9

Rocc version:

iOS/macOS version:

iOS 13.3.1

Camera manufacturer and model:

Sony A9

Other information about your system(s):

Other questions

Does the issue still occur after updating your Camera's Firmware?

Have you tried any other versions of NVDA?

Add support for Sony α9 5.0 firmware update

The library can no longer connect to Sony α9 cameras after the 5.0 firmware released for them, the API description files which Sony make available on the camera have changed reasonably drastically and no longer contain an array of services in the X_ScalarWebAPI_DeviceInfo of dd.xml:

<?xml version="1.0" encoding="utf-8"?>
<root xmlns="urn:schemas-upnp-org:device-1-0" xmlns:dlna="urn:schemas-dlna-org:device-1-0" xmlns:av="urn:schemas-sony-com:av">
	<specVersion>
		<major>1</major>
		<minor>0</minor>
	</specVersion>
	<device>
		<deviceType>urn:schemas-upnp-org:device:Basic:1</deviceType>
		<friendlyName>ILCE-9</friendlyName>
		<manufacturer>Sony Corporation</manufacturer>
		<manufacturerURL>http://www.sony.com/</manufacturerURL>
		<modelDescription>SonyRemoteCamera</modelDescription>
		<modelName>SonyImagingDevice</modelName>
		<modelURL>http://www.sony.net/</modelURL>
		<UDN>uuid:00000000-0005-0010-8000-60f189acc029</UDN>
		<serviceList>
			<service>
				<serviceType>urn:schemas-sony-com:service:DigitalImaging:1</serviceType>
				<serviceId>urn:schemas-sony-com:serviceId:DigitalImaging</serviceId>
				<SCPDURL>/DigitalImagingDesc.xml</SCPDURL>
				<controlURL>/upnp/control/DigitalImaging</controlURL>
				<eventSubURL></eventSubURL>
			</service>
		</serviceList>
		<av:X_ScalarWebAPI_DeviceInfo xmlns:av="urn:schemas-sony-com:av">
			<av:X_ScalarWebAPI_ImagingDevice>
				<av:X_ScalarWebAPI_LiveView_URL>http://192.168.122.1:60152/liveviewstream?%211234%21%2a%3a%2a%3aimage%2fjpeg%3a%2a%21%21%21%21%21</av:X_ScalarWebAPI_LiveView_URL>
				<av:X_ScalarWebAPI_DefaultFunction>RemoteShooting</av:X_ScalarWebAPI_DefaultFunction>
			</av:X_ScalarWebAPI_ImagingDevice>
		</av:X_ScalarWebAPI_DeviceInfo>
	</device>
</root>

There is still reference to X_ScalarWebAPI so I am hopeful with a few small tweaks full-behaviour will be re-instated, but there could be more to it: Like Sony moving to PTP (We see reference to PTP versions in digitalImagingDescription.xml)

An attempt to hard-code the expected services has been made here which will be going out to testers of Camrote later today! 🤞 it's as simple as this, otherwise we're looking at significant amounts of work and needing access to an α9 body, which is something I can't personally afford currently!

Add demo project!

Create a demo (iOS for now) project so when people want to get started with ROCC they don't have to flesh out a whole app UI and perform connection to the Camera's WiFi network e.t.c. this should be basic, and provide a standard UI for each of the functions available! Not going to be pretty but it will work and help people debug issues on particular models!

Support non okay command response codes

There is a vast list of error command response codes. The awaitData function should be enhanced to return a Result object to its completion closure and in the case of a bad response code should error. The command response code can be ignored if an end data packet is received as some cameras don’t send it in this case, but otherwise should be heeded

Add support for two-way events to CameraEventNotifier

Maybe by having different protocols for if the camera supports polled events (Continuously hitting endpoint like smart remote control API) or if it responds to events sent by the camera that a new event is ready (PTP IP). See Issue #1 for further ideas on this!

UPnP code not finding date folders for whole of SD card contents

Something is happening currently to certain camera models (perhaps not formatting SD card regularly even, as one user reported this fixing his issue) which means that our code isn’t finding date folders via UPnP for all dates that the camera has stored.

Some preliminary investigation shows that on Camrote we start by searching for a folder with ObjectID “0” we then get a single result which is “PhotoRoot”... which also has a unique identifier based single folder, and when we ask this for all subfolders we don’t get all of the date folders. It’s possible we need to paginate the request to this folder or make it sort in ascending date order rather than descending or similar.

Fetch device info in `SonyPTPIPCameraDevice`'s `connect` method.

Need to call first:
Cmd_Request OpenSession command
Cmd_Request GetDeviceInfo command
Cmd_Request sdioGetExtDeviceInfo command

Investigate 404 error on RX100-M2

Steps to reproduce:

Try and connect to RX100-M2

Actual behavior:

getVersions call returns a 404 (Not found)

Expected behavior:

Should be able to connect correctly to the device.

iOS/macOS version:

N/A

Camera manufacturer and model:

Sony RX100-M2

Other questions

Does the issue still occur after updating your Camera's Firmware?

Unsure...

Integrate PTP IP Events with EventNotifier

RAW File transfer

Enable RAW file transfer

Ddoes the manufacturer source support RAW file transfer? I think that would make this library really stand out!

Implement `Camera` protocol on `SonyPTPIPCameraDevice`

Add support for new HFR mode on RX100 VII

• Need to support locking/unlocking exposure controls
• Need to support starting HFR recording
• Need to support camera letting us know that HFR recording has begun, and subsequently has ended