hi-rez / satin Goto Github PK
View Code? Open in Web Editor NEWA 3D Graphics Framework built on Apple's Metal
License: MIT License
A 3D Graphics Framework built on Apple's Metal
License: MIT License
I think this might be a Catalina issue:
2020-06-26 10:59:30.385096-0400 LiveCode[18428:4915416] [cas] LaunchedApplication: failed with error -13052 (null){ "ApplicationType"="Foreground", "BundleIdentifierLowerCase"="com.apple.finder", "CFBundleExecutablePath"="/Users/jzting/Downloads/Satin-master/Example/Source/Shared/LiveCode/Assets/Shaders.metal/Contents/MacOS/Finder", "CFBundleExecutablePathDeviceID"=16777221, "CFBundleExecutablePathINode"=1152921500312409048, "CFBundleIdentifier"="com.apple.finder", "CFBundleName"="Finder", "CFBundlePackageType"="FNDR", "CFBundleSignature"="MACS", "LSASN"=ASN:0x0-0x206206:, "LSBundlePath"="/Users/jzting/Downloads/Satin-master/Example/Source/Shared/LiveCode/Assets/Shaders.metal", "LSBundlePathDeviceID"=16777221, "LSBundlePathINode"=8645686135, "LSDisplayName"="Shaders.metal", "LSExecutableFormat"="LSExecutableMachOFormat", "LSLaunchDLabel"="com.apple.Finder", "LSLaunchedByLaunchServices"=true, "LSLaunchedWithLaunchD"=true, "LSLaunchEventRecordTime"=85367951456787, "LSLaunchTime"=now-ish 2020/06/26 10:59:30, "LSParentASN"=ASN:0x0-0x204204:, "LSWantsToComeForwardAtRegistrationTimeKey"=true, "pid"=5092 }
AR background is way too bright. Sorry, should be a pull request, but not sure it's perfect.
I think you're meant to change the shader in SatinSceneKitARRenderer to:
// don't return ycbcrToRGBTransform * ycbcr;
float4 color = ycbcrToRGBTransform * ycbcr;
// Flatten Color Space to rgb?
return float4(pow(color.rgb, float3(2,2,2)), color.a);
Very cool project.
What is triangulation algorithm using in Satin?
Implementing lights & shadows into the framework. Let me know if you want to delegate this work!
Is there background rendering, not screen demo, API, and parallel processing possible at the same time?
Hey Reza
Building according to instructions (using bundler) goes well:
Downloading dependencies
Installing Forge (0.1.0)
Installing Satin (0.2.0)
Generating Pods project
Integrating client project
Pod installation complete! There are 2 dependencies from the Podfile and 2 total pods installed.
...but then on building the project using xcworkspace file, I get 'No such module 'Forge'' in renderer.swift...is that happening to you too?
In the actual scene, multiple mesh models are often added to the object, and multiple mesh models are exported as a whole?
Performance improvement in changing and refreshing UI interface for dynamic addition and deletion of mesh
Hi Reza, thanks for sharing this amazing repo!
Have you thought of adding a ray tracer?
https://developer.apple.com/videos/play/wwdc2021/10149/
Or maybe integrate with Blender cycles Metal version?
cheers!
From the Apple documentation on ARAnchorCopying
:
Because anchor objects are immutable, ARKit must copy them to make changes from one ARFrame to the next.
Therefore, the transform values here in Satin are never getting updated (they stay the same):
fileprivate class ARObject: Object {
…
override func update(camera: Camera, viewport: simd_float4) {
guard let anchor = anchor else { return }
self.worldMatrix = anchor.transform
super.update(camera: camera, viewport: viewport)
}
ARKit appears to require us to use the
ARSessionDelegate.session(_ session: ARSession, didUpdate anchors: [ARAnchor])
method to get updated transform values.
Is it possible to easily send textures around using Syphon with Satin? On apple sillicon, there are currently no easy options (besides unity) for a creative coding framework that both supports access to both compute shaders and syphon
Why are MSL resource files not integrated into 'Group' files, but packaged in the form of resource drops? Are there any subsequent situations to be considered in the design? Or is the original intention of such a design in consideration of aspects?
Object
gained support for observability with conformance to @ObservedObject
and a couple of @Published
values, which is great as it opens up for supporting a SwiftUI frontend that displays information about the scene.
But the property wrappers were later changed from @Published
to a custom @PublishedDidSet
:
Satin/Sources/Satin/Core/Object.swift
Line 50 in c29677b
This custom property wrapper defeats the purpose of conforming to ObservableObject
, as the whole purpose of this class is:
ObservableObject
A type of object with a publisher that emits before the object has changed.
If an object does not have any @Published
properties, there is no longer any effect in making it an ObservableObject
(and thus breaking the ability to use directly in SwiftUI).
If possible I'd suggest to revert these back to @Published
. If there is need to observe @Published
values in another lib (Youi?) after they have changed, you can always use an async dispatch on the next runloop to ensure the value has been set:
cancellable = object.$position.sink { [weak self] position in
// Position not yet set here, self?.object.position is still old
DispatchQueue.main.async {
// Position has been set here, self?.object.position is new
}
}
This is summed up in Satin.podspec, but:
I still can't get the example to work (I'm unsure if you need Xcode 11+)
What editor is the editor for, or is it possible to develop it later?
Log: Triangulation for 轛 FAILED!
let input = "車轛"
let geo = TextGeometry(text: input, fontName: "PingFangSC-Regular", fontSize: 8)
seams little problem with new index of triangle. happens only in PingFangSC-Light and PingFangSC-Regular, but works well in PingFangSC-Medium and PingFangSC-Semibold.
It looks like there's an issue with the ffi gem not supporting the M1 chip yet, which causes an issue when you initially setup the examples and try to run
bundle exec pod install
I get the following error:
Analyzing dependencies
Pre-downloading: `Forge` from `https://github.com/Hi-Rez/Forge.git`, commit `4d1c7760cb12e129fb027e4e250037a8bed520a3`
/Users/USER/Desktop/code/Satin/Example/vendor/bundle/ruby/2.6.0/gems/ffi-1.15.0/lib/ffi/library.rb:275: [BUG] Bus Error at 0x0000000100cb4000
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.arm64e-darwin20]
And relevant crash report info:
Process: ruby [1268]
Path: /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
Identifier: ruby
Version: 145.100.1
Code Type: ARM-64 (Native)
Parent Process: zsh [1107]
Responsible: Terminal [519]
User ID: 501
PlugIn Path: /Users/USER/Desktop/*/ffi_c.bundle
PlugIn Identifier: ffi_c.bundle
PlugIn Version: ??? (0)
Date/Time: 2021-04-29 14:16:58.436 -0500
OS Version: macOS 11.3 (20E232)
Report Version: 12
Anonymous UUID: 72AB6060-84EC-29E6-9538-A204D5EB2E7D
Time Awake Since Boot: 450 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGABRT)
Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000100cb4000
Exception Note: EXC_CORPSE_NOTIFY
I'm going to try the suggestions on the StackOverflow thread, but wanted to document the issue here as well.
When I run the sample PBR-iOS on an iPhone 12 Pro the following error appear in the console:
Diffuse - Level: 0 - Size: 64
2022-06-07 14:59:04.084563+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.084625+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 1 - Size: 32
2022-06-07 14:59:04.119867+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.120190+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 2 - Size: 16
2022-06-07 14:59:04.181093+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.181260+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 3 - Size: 8
Diffuse - Level: 4 - Size: 4
Diffuse - Level: 5 - Size: 2
Diffuse - Level: 6 - Size: 1
Specular - Level: 0 - Size: 512
2022-06-07 14:59:04.281598+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.281868+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 1 - Size: 256
2022-06-07 14:59:04.315465+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.315655+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 2 - Size: 128
2022-06-07 14:59:04.336547+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.336725+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 3 - Size: 64
Specular - Level: 4 - Size: 32
Specular - Level: 5 - Size: 16
Specular - Level: 6 - Size: 8
Specular - Level: 7 - Size: 4
Specular - Level: 8 - Size: 2
Specular - Level: 9 - Size: 1
The metallic sphere on the top left seems to have an issue and turns dark:
It is supposed to look like this
To compare my own engine i've used Pisa HDR and compare to real world location where pisa tower is on the right not on the left.
Same thing can be observed with brown studio hdr you are using as window is on the right side of the bed not left https://polyhaven.com/a/brown_photostudio_02
Which versions can be used in the production environment, or how long is the production version estimated?
I tried using the default PhysicalMaterial
and got this error:
Physical Shader: program_source:985:5: error: no matching function for call to 'pbrDirectLighting'
pbrDirectLighting(pixel, lights);
^~~~~~~~~~~~~~~~~
program_source:673:6: note: candidate function not viable: no known conversion from 'const constant Light' to 'const constant Light *' for 2nd argument; take the address of the argument with &
void pbrDirectLighting(thread PixelInfo &pixel, constant Light *lights)
^
There appears to be a missing ampersand on this line in the Physical Shader.
Other calls to pbrDirectLighting
within this repo use an ampersand, such as this one and this one.
I cloned Stain, added an ampersand to that line, added it as a local swift package and the error went away.
Hi @rezaali,
I'm playing with the lookAt
function and I can't get it to work as I expect it – meaning I would like the cones to point to each other by the tip. But also at the middle crossing point they "flip". Do you know how to solve this?
thx
import MetalKit
import SwiftUI
import Forge
import Satin
class MKTest1: Forge.Renderer
{
var context: Context!
var renderer: Satin.Renderer!
var camera: PerspectiveCamera!
var scene: Object = .init("Scene")
var coneMesh1 = Mesh(geometry: ConeGeometry(size: (radius: 1, height: 3),
res: (angular: 8, radial: 1, vertical: 1)),
material: BasicDiffuseMaterial(0.75))
var coneMesh2 = Mesh(geometry: ConeGeometry(size: (radius: 1, height: 3),
res: (angular: 8, radial: 1, vertical: 1)),
material: BasicDiffuseMaterial(0.5))
var time: Float = 0.0
override func setup()
{
context = Context(device, sampleCount, colorPixelFormat, depthPixelFormat, stencilPixelFormat)
camera = PerspectiveCamera(position: [0, 0, 35], near: 0.01, far: 100.0)
renderer = Satin.Renderer(context: context, scene: scene, camera: camera)
renderer.setClearColor([0, 0, 0, 1])
coneMesh1.position = [0, 0, 0]
coneMesh2.position = [10, 0, 0]
scene.add(coneMesh1)
scene.add(coneMesh2)
}
override func update()
{
time += 0.01
let sx = sin(time)
let sy = cos(time * 5)
let sz = sin(time)
coneMesh2.position = [sx * 10, sy * 10, sz * 5]
coneMesh1.material?.set("Color", [abs(sx), abs(sy), abs(sx + sy), 1.0])
coneMesh1.lookAt(coneMesh2.position)
coneMesh2.lookAt(coneMesh1.position)
}
override func draw(_ view: MTKView, _ commandBuffer: MTLCommandBuffer)
{
guard let renderPassDescriptor = view.currentRenderPassDescriptor else { return }
renderer.draw(renderPassDescriptor: renderPassDescriptor, commandBuffer: commandBuffer)
}
override func resize(_ size: (width: Float, height: Float))
{
camera.aspect = size.width / size.height
renderer.resize(size)
}
}
struct MKTest1ContentView: View
{
var body: some View
{
ForgeView(renderer: MKTest1())
}
}
Can TextGeometry support Chinese?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.