shotgunsoftware / tk-framework-adobe Goto Github PK
View Code? Open in Web Editor NEWFoundation Tech for all Adobe Integrations
Home Page: https://developer.shotgridsoftware.com/tk-framework-adobe/
License: Other
Foundation Tech for all Adobe Integrations
Home Page: https://developer.shotgridsoftware.com/tk-framework-adobe/
License: Other
I am finding duplicated unit tests in the AE and PS engines which really should belong in the adobe fw. Example:
https://github.com/shotgunsoftware/tk-aftereffects/blob/master/tests/rpc_tests/basic.py#L30
We need to create a set of tests for the framework specifically and remove the non-engine-specific tests from the PS/AE engines.
Looks like the log file is named after the framework rather than the engine which is currently being launched. We want the log file to be named tk-photoshopcc.log
and tk-aftereffects.log
depending on the DCC.
We should do a pass on the various environment variables to ensure they are in the right place. I am seeing a lot of stuff on the engine level that looks like it likely belongs at the framework level and in some cases it is defined in both the engine and the framework, which is bad.
The goal: Get as many of the environment variables into the framework as possible. Update the documentation to reflect the new state (wiki + sphinx, adobe fw, AE engine, PS engine).
SHOTGUN_ADOBE_NETWORK_DEBUG
environment is defined both in the engine and in the framework - we should refactor this to be less leaky. I think we should drop the network_debug
flag and the only way to control the debug state is through the SHOTGUN_ADOBE_NETWORK_DEBUG
env var, which is only accessed from the fw.SHOTGUN_ADOBE_PYTHON
.Add sphinx documentation to cover all public methods that form the official contract of the API and how it is intended to be used. For an example, see https://github.com/shotgunsoftware/tk-framework-shotgunutils.
Move some of the photoshop logic into the photoshop engine. There could be a subclassed version of the Communicator
inside the engine, which implements the various photoshop specific methods.
Using the adobe fw results in non-standard imports:
Note now our other frameworks are imported like this:
shotgun_settings = sgtk.platform.import_framework("tk-framework-shotgunutils", "settings")
Whereas the adobe fw is imported like this:
adobe_bridge = sgtk.platform.import_framework(
"tk-framework-adobe",
"tk_framework_adobe.adobe_bridge"
)
win_32_api = sgtk.platform.import_framework(
"tk-framework-adobe",
"tk_framework_adobe_utils.win_32_api"
)
We need to get parity with other frameworks and define a strict interface for the adobe framework modules inside __init__.py
- for alignment with other patterns in tk, but more importantly to ensure there is a clear public interface which will allow for refactoring in the future.
Framework interfaces should only expose the minimum public interface that we commit to officially support.
Documentation
Note how the docs for the shotgun utils are generated to look like this:
Currently, the docs for the adobe fw look like this:
As part of this work, we want them to align in structure with how the shotgun utils ones above look, e.g it should just say adobe_bridge.AdobeBridge
.
Comments from CR:
Do another pass on quoting and linting for the AE engine, the PS engine and the adobe FW:
Invoking the Photoshop API with engine.adobe
can cause the AdobeBridge websocket communication to be interrupted. This only seems to happen during long, blocking operations that cause a progress bar in Photoshop to appear such as saving and loading large files from network storage, or saving files with high compression settings.
Note that the original operation (File Save, File Open, etc.) does complete successfully, but because it's immediately followed up by an error that crashes the plugin, you can't do anything else afterwards.
This bug causes the SGTK plugin to fail during vital operations such as saving and loading files. It can often become a blocking issue that prevents artists from publishing work.
Note: line numbers will be slightly off: I've inserted a lot of logging messages to my local files for debugging.
[49446 ERROR sgtk.env.project.tk-photoshopcc] Traceback (most recent call last):
File ".../tk_multi_pythonconsole/1.3.0._build1/5322/a/ext/python/app/input_widget.py", line 246, in execute
exec(python_code, self._locals, self._locals)
File "python input", line 13, in <module>
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/rpc/proxy.py", line 232, in __call__
parent=self._parent,
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/rpc/communicator.py", line 300, in rpc_call
wrapper_class=ProxyWrapper,
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/rpc/communicator.py", line 760, in __run_rpc_command
results = self._wait_for_response(payload["id"])
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/adobe_bridge.py", line 44, in wrapper
result = func(*args, **kwargs)
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/adobe_bridge.py", line 440, in _wait_for_response
return super(AdobeBridge, self)._wait_for_response(uid)
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/rpc/communicator.py", line 655, in _wait_for_response
self.wait(single_loop=True, process_events=False)
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/rpc/communicator.py", line 523, in wait
process_events=process_events,
File ".../tk_framework_adobe/1.1.7/411f/a/ext/python/tk_framework_adobe/rpc/communicator.py", line 245, in process_new_messages
self._io._process_packets()
File ".../tk_framework_adobe/1.1.7/411f/a/ext/pkgs/socketIO_client_nexus/__init__.py", line 276, in _process_packets
for engineIO_packet in self._transport.recv_packet():
File ".../tk_framework_adobe/1.1.7/411f/a/ext/pkgs/socketIO_client_nexus/transports.py", line 158, in recv_packet
packet_text)
File ".../tk_framework_adobe/1.1.7/411f/a/ext/pkgs/socketIO_client_nexus/parsers.py", line 96, in parse_packet_text
packet_type = int(get_character(packet_text, 0))
File ".../tk_framework_adobe/1.1.7/411f/a/ext/pkgs/socketIO_client_nexus/symmetries.py", line 33, in get_character
return chr(get_byte(x, index))
File ".../tk_framework_adobe/1.1.7/411f/a/ext/pkgs/socketIO_client_nexus/symmetries.py", line 29, in get_byte
return indexbytes(x, index)
IndexError: index out of range
Here is a video of me saving a PNG file with the maximum compression settings.
Reproducing the error can be tricky. I can reliably reproduce the error on MacOS, but not on Windows. Furthermore, not all large files produce the error. However, once I have a file that produces the error, it is consistently reproducible with the same steps.
from pathlib import Path
dst = Path.home() / "Desktop" / "save_test.png" # Save file to Desktop
png_file = engine.adobe.File(dst.as_posix())
png_options = engine.adobe.PNGSaveOptions()
png_options.interlaced = False
png_options.compression = 9 # Highest compression value, most likely to cause Photoshop to hang
document = engine.adobe.get_active_document()
document.saveAs(png_file, png_options, True)
If you get an IndexError
in the Python console, followed by the ShotGrid Adobe panel shutdown, you have reproduced the error successfully.
Toolkit environment:
v1.1.7
v1.9.4
v0.20.13
Python version:
3.7.13
Photoshop version:
v23.5.4
)System information:
I have reproduced the error on both of the following systems:
v12.6.3
v13.2.1
I have read the ShotGrid community forum post for
Adobe engine crashing on long operations. While this issue feels related, and indeed presents some of the same symptoms, I could not get any of the solutions to work.
My understanding is that disabling the heartbeat should be the most "forceful" way of circumventing some of these issues. I have tried saving with the heartbeat_disabled
context manager, but the issue persists:
with engine.heartbeat_disabled():
document.saveAs(png_file, png_options, True)
(This is going to be really long, I apologize. I needed to get this all written down while it's still fresh in my head.)
The IndexError
comes from the bundled pkgs.zip.
I've tried to break down the steps in the websocket communication to understand where the error is coming from, and this is my best understanding of what happens when you call, for example, document.saveAs
:
Document.saveAs
function is invoked on the document's FunctionWrapperCommunicator.__run_rpc_command
line 677Communicator._wait_for_response
line 611Websocket.recv
lines 292-304 def recv(self):
"""
Receive string data(byte array) from the server.
return value: string(byte array) value.
"""
opcode, data = self.recv_data() # <-------------------------------- EMPHASIS MINE
if six.PY3 and opcode == ABNF.OPCODE_TEXT:
return data.decode("utf-8")
elif opcode == ABNF.OPCODE_TEXT or opcode == ABNF.OPCODE_BINARY:
return data
else:
return ''
This part is important. From the packet, we receive an opcode, and some data bundle as a byte string. In the example above that produces the IndexError, we get the following values:
opcode == 8
data == b''
Ok, let's continue...
WebsocketTransport.recv_packet
lines 157-158def parse_packet_text(packet_text):
packet_type = int(get_character(packet_text, 0)) # <------------ EMPHASIS MINE
packet_data = packet_text[1:]
return packet_type, packet_data
This line emphasized here is why we get the IndexError
. We are trying reading the first character from an empty byte string.
Alright, great. So that's one mystery solved. But why do we get an empty byte string, and what is the 8
opcode telling us? If we jump back to the websocket-client module, we can see the opcodes defined in _abnf.py lines 80-86:
# operation code values.
OPCODE_CONT = 0x0
OPCODE_TEXT = 0x1
OPCODE_BINARY = 0x2
OPCODE_CLOSE = 0x8
OPCODE_PING = 0x9
OPCODE_PONG = 0xa
The 8
opcode is a close event. Indeed if you keep tracking it down, you'll see that WebSocket.send_close
(lines 362-373) is called immediately before the packet is received, which happens right before the IndexError
.
If you add an except IndexError: ...
block in tk-framework-adobe's Communicator.process_new_messages
(lines 238-251), you can ignore the IndexError, but you immediately get another communication error, because the close signal has already been sent, and the socket communication is terminated.
And that's as far as I've been able to get. I do not know why the close signal is being sent. I don't know whether it's being deliberately called somewhere, or if it's a byproduct of some other error wrapped in a try/except
clause. I dug around in the Adobe CEP plugin and couldn't find a smoking gun in there. I don't have a ton of familiarity with JavaScript, so I don't feel confident saying anything definitive one way or the other with regards to the plugin.
To circumvent the error for now, I have dialed down the compression settings that our publisher plugin uses to render PNGs. That makes the document.saveAs
function complete more quickly, which doesn't produce the error. That is just a stopgap solution though. Sooner or later, I'm worried that an even more "troublesome" file will come along that we can't buy ourselves any more leeway on with the compression settings, and we'll be stuck unable to save it at all.
Any help investigating this, or suggestions on using the Python API to avoid the error, would be extremely welcome!
Publish failed in Photoshop CC when using if Begin File versioning is selected: (tested on Windows)
Steps:
Traceback (most recent call last):
File "C:\Users\mathurf\AppData\Roaming\Shotgun\bundle_cache\app_store\tk-multi-publish2\v2.3.0\python\tk_multi_publish2\api\plugins\publish_plugin_instance.py", line 282, in _handle_plugin_error
yield
File "C:\Users\mathurf\AppData\Roaming\Shotgun\bundle_cache\app_store\tk-multi-publish2\v2.3.0\python\tk_multi_publish2\api\plugins\publish_plugin_instance.py", line 198, in run_publish
self._hook_instance.publish(settings, item)
File "C:\Users\mathurf\Documents\GitHub\tk-photoshopcc\hooks\tk-multi-publish2\basic\start_version_control.py", line 247, in publish
engine.save_to_path(document, version_path)
File "C:\Users\mathurf\Documents\GitHub\tk-photoshopcc\engine.py", line 522, in save_to_path
save_options.quality = 12
File "C:\Users\mathurf\Documents\GitHub\tk-framework-adobe\python\tk_adobe_basic\rpc\proxy.py", line 278, in __setattr__
remote_names = self.data["properties"] + self.data["methods"].keys()
KeyError: 'properties'
I'm not able to use Shotgun integration in both After Effects and Photoshop on mac.
Here is the console output from Photoshop CC 2019 (it is the same win_32_api error in AE)
PhotoshopErrorMac.txt
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.