Giter Site home page Giter Site logo

maxxfrazer / focusentity Goto Github PK

View Code? Open in Web Editor NEW
346.0 9.0 52.0 7.31 MB

Bringing the scanning box from SceneKit to RealityKit

Home Page: https://maxxfrazer.github.io/FocusEntity/documentation/focusentity/

License: MIT License

Swift 98.22% Shell 1.78%
realitykit arkit ios13 swift swiftpm augmented-reality swiftpackage arkit3

focusentity's Introduction

FocusEntity

This package is based on ARKit-FocusNode, but adapted to work in Apple's framework RealityKit.


The Example looks identical to the above GIF, which uses the FocusEntity classic style.

See the documentation for more.

Minimum Requirements

  • Swift 5.2
  • iOS 13.0 (RealityKit)
  • Xcode 11

If you're unfamiliar with using RealityKit, I would also recommend reading my articles on Getting Started with RealityKit.

Installation

Swift Package Manager

Add the URL of this repository to your Xcode 11+ Project.

Go to File > Swift Packages > Add Package Dependency, and paste in this link: https://github.com/maxxfrazer/FocusEntity


Usage

See the Example project for a full working example as can be seen in the GIF above

  1. Install FocusEntity with Swift Package Manager
https://github.com/maxxfrazer/FocusEntity.git
  1. Create an instance of FocusEntity, referencing your ARView:
let focusSquare = FocusEntity(on: self.arView, focus: .classic)

And that's it! The FocusEntity should already be tracking around your AR scene. There are options to turn the entity off or change its properties. Check out the documentation or example project to learn more.


Feel free to send me a tweet if you have any problems using FocusEntity, or open an Issue or PR!

The original code to create this repository has been adapted from one of Apple's examples from 2018, license also included. I have adapted the code to be used and distributed from within a Swift Package, and now further adapted to work with RealityKit.

focusentity's People

Contributors

deebov avatar marcos-vinicius-mafei avatar maxxfrazer avatar multitudes avatar ralfebert avatar reality-dev avatar rus-cosmin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

focusentity's Issues

Closure containing control flow statement cannot be used with result builder 'ViewBuilder'

For this issue I created this correct solution

import SwiftUI

struct ModelsByCategoryGridView: View {

@Binding var showBrowse: Bool

let models = Models()

var body: some View {
    VStack {
        ForEach(ModelCategory.allCases, id: \.self) { category in
            // No need for optional binding here
            let modelsByCategory = models.get(category: category)
            
            // Check if the array is not empty before creating the HorizontalGridView
            if !modelsByCategory.isEmpty {
                HorizontalGridView(showBrowse: $showBrowse, title: category.label, items: modelsByCategory)
            }
        }
    }
}

}

Errors when using simulator

Describe the bug
Not a bug really.
I know that ARView is only available on device but when I run my app on the simulator I have the compiler skipping my AR code.
This is not happening with FocusEntity.
So I get errors where it says that it did not find ARVire or something. And those errors are not in my code but come from the swift package :)

To Reproduce
Add FocusEntity to project and CMD-R to run on simulator

Expected behavior
no errors from the compiler

Screenshots
If applicable, add screenshots to help explain your problem.
Screenshot 2021-11-10 at 13 22 19
Screenshot 2021-11-10 at 13 23 11

Environment:

  • iOS Version - iOS 15
  • Device Information iPhone 12 mini
  • macOS Version 11.5
  • Xcode Version 12.5

Additional context
Add any other context about the problem here
I had a similar problem...
In my AR code I write this to avoid this problem.
Maybe this would work? If I have time I will test this and do a pull request.

#if !targetEnvironment(simulator)
        // my AR code
#else
        // keep this again because req by the UIViewRepresentable protocol
	func makeUIView(context: Context) -> UIView {
		UIView()
	}
	func updateUIView(_ uiView: UIView, context: Context) {}
#endif

Issue when importing this Swift package to Swift Playgrounds 4 on iPad.

Describe the bug
I am using this project as part of a tutorial on the latest version of Swift Playgrounds 4 for the iPad. When trying to import the swift package nothing shows up in Swift Playgrounds to be able to import the project.

To Reproduce
Steps to reproduce the behavior:

  1. Open Swift Playgrounds 4
  2. Open an App project
  3. Import Swift Package
  4. Paste git url
  5. Nothing happens

Expected behavior
The project version should appear when pasting the git url

Screenshots
https://share.icloud.com/photos/0881xVKzop_6nPJCrTlZg7JDg

Environment:

  • iPad OS 15.2
  • iPad Pro (11-inch) (3rd generation)
  • Swift Playgrounds 4

Error: BaseColor was deprecated on iOS 15

Describe the bug
I am getting an error in the FocusEntity Colored swift file for

var modelMaterial = UnlitMaterial(color: .clear)
modelMaterial.baseColor = endColor

With the error baseColor was deprecated on iOS 15, use Color instead. When changing to color I get the following error:

Cannot assign value of type 'MaterialColorParameter' to type 'UnlitMaterial.BaseColor' (aka
'PhvsicallvBasedMaterial.BaseColor')

To Reproduce
Steps to reproduce the behavior:

  1. It happens when you open the project in Swift Playgrounds on iPad

Environment:

  • iPadOS 15.2
  • iPad Pro (11-inch) (3rd generation)
  • Swift Playgrounds 4.0

It's sometimes fail to detect planes and is not expected in the result of WWDC example

Hi,
Thanks you to re-implement the focus square on RealityKit
I tried the newest example "Placing Objects and Handling 3D Interaction" from WWDC code.
The focus square can perfectly detect plane to close and open the square ...
However, I tried to use the FocusEntity component, it's sometimes fail to detect plane and is not expected in the result of WWDC example.

I check both code, the only difference is:
WWDC example's raycasting is based on ARRaycastResult from iOS 13
And FocusEntity is based on ARHitTestResult from iOS 11
is this the reason ?

isEnabled is not defined in the class

Describe the bug

Preview compilation issue. When setting focusEntity isEnabled, because there is no member, Swift raises an exception.

error: value of type 'FocusEntity' has no member 'isEnabled'
        arView.focusEntity?.isEnabled = false
        ~~~~~~~~~~~~~~~~~~~ ^~~~~~~~

To Reproduce

arView.focusEntity?.isEnabled = false

Where this was set:

class CustomARView: ARView {
    required init(frame frameRect: CGRect, sessionSettings: SessionSettings, modelDeletionManager: ModelDeletionManager) {
        super.init(frame: frameRect)
                
        self.focusEntity = FocusEntity(on: self, focus: .classic)

In Xcode: Value of type 'FocusEntity' has no member 'isEnabled'

I see that RealityKit's Entity has isEnabled:

/// Indicates whether the entity is active, ignoring parent's active state.
    public var isEnabled: Bool

but I'm not sure why FocusEntity is not inheriting that.

Interestingly, the app does compile and run. I bumped into this problem when compiling the app and it disappeared. This is now happening when I preview...

Expected behavior

A member variable called isEnabled that can also be set on a public interface.

Environment:

  • iOS Version 15
  • Device Information actual device iPhone 13 Pro
  • macOS Version 12.2.1
  • Xcode Version 13.2.1

Additional context

Glitch in ARView

Describe the bug
I have an iOS application in which you can open an AR view by tapping on a button. However, after popping the view from my navigation controller and tapping on the AR button again something unusual happens in the camera feed. I see a sudden appearance of the last frame of my previous ar session that comes and goes during the whole session.
Here is my code:

override func viewDidLoad() {
    super.viewDidLoad()
    focusSquare = FocusEntity(on: arview, style: .classic(color: .yellow))
    setupCoachingOverlay()
  }
   
  override func viewWillAppear(_ animated: Bool) {
    self.navigationController?.isNavigationBarHidden = true
  }
   
  override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    UIApplication.shared.isIdleTimerDisabled = true
    resetTracking()
  }

  override func viewWillDisappear(_ animated: Bool) {
    super.viewWillDisappear(animated)
    arview.session.pause()
  }

  func resetTracking() {
    arview.automaticallyConfigureSession = false
    let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = [.horizontal, .vertical]
    arview.session.run(configuration, options: [.resetTracking, .removeExistingAnchors])
    arview.scene.addAnchor(anchor)
  }

To Reproduce
Steps to reproduce the behavior:

  1. Using the storyboard, create a view controller with a button and embed it in a navigation controller.
  2. Create view controller number 2 which contains an ar view.
  3. Tapping on the button in the view controller number 1 pushes the second view controller on the navigation controller.
  4. Press back in the navigation bar from the second view controller (so that it's popped from the navigation controller)
  5. Again tap on the button in the first view controller to open the second view controller.

Expected behavior
I expected to see the normal camera feed.

Environment:

  • iOS 14.4.2
  • iPhone 7
  • macOS Big Sur 11.2.2
  • Xcode 12.4

Transparent Image Background

Describe the bug
When using images with transparent backgrounds like in the Example, the background becomes black.

To Reproduce
Create FocusEntity using image like in the Example.

Expected behavior
Actual Transparent Background.

Screenshots
IMG_A1D05D316F94-1

Environment:

  • iOS 17.2.1
  • macOS 14.2.1
  • Xcode 15.2

Problems when using Collaborative session

Hi man, your repo and your work is awesome.

I tried using your code on a collaborative session project, as soon as another phone connected and added the entity to the session, it would break on the previous phones due to not finding an appropriate init

I would suggest to add the the option to disable synchronizing, or to put it off completely (since the focusing feature works around only one screen)

i tried running this on your code

  public required init(on arView: ARView, focus: FocusEntityComponent) {
    self.arView = arView
    super.init()
    self.focus = focus
    self.name = "FocusEntity"
    self.orientation = simd_quatf(angle: .pi / 2, axis: [1, 0, 0])
    self.positioningEntity.synchronization = nil
    self.addChild(self.positioningEntity)
    cameraAnchor = AnchorEntity(.camera)
    cameraAnchor.synchronization = nil
    synchronization = nil
    arView.scene.addAnchor(cameraAnchor)

because I was in a hurry to get this out, i didnt test this to see if only one of the = nil on the synchronization components where required, but that did it for me

i need help!

_Hi Max,

thank you for your unbelievable nice codes! I'm new in this topic, actually studying architecture, I watched this tutorial https://www.youtube.com/watch?v=9R_G0EI-UoI and tried to use your code as it is shown in the video, but I don't get it work.

here is my code: please help me out. Thank you so much

Bildschirmfoto 2020-10-25 um 21 45 29

It always says "Cannot find 'FESquare' in scope, Cannot find'FEDelegate' in scope

//
// ContentView.swift
// Model Picker
//
// Created by Niko Endres on 12.10.20.
//

import SwiftUI
import RealityKit
import ARKit
import FocusEntity

struct ContentView: View {
@State private var isPlacementEnabled = false
@State private var selectedModel: Model?
@State private var modelConfirmedForPlacement: Model?

private var models: [Model] = {
    // dynamically get file names
    let filemanager = FileManager.default
    
    guard let path = Bundle.main.resourcePath, let files = try?
            filemanager.contentsOfDirectory(atPath: path) else {
        return []
    }
    
    var availableModels: [Model] = []
    for filename in files where filename.hasSuffix("usdz") {
        let modelName = filename.replacingOccurrences(of: ".usdz", with: "")
        let model = Model(modelName: modelName)
        
        availableModels.append(model)
    }
    
    return availableModels
}()

var body: some View {
    ZStack(alignment: .bottom) {
        
        ARViewContainer(modelConfirmedForPlacement: self.$modelConfirmedForPlacement)
    
        if self.isPlacementEnabled {
            PlacementButtonsView(isPlacementEnabled: self.$isPlacementEnabled, selectedModel: self.$selectedModel, modelConformedForPlacement: self.$modelConfirmedForPlacement)
        } else {
            ModelPickerView(isPlacementEnabled: self.$isPlacementEnabled, selectedModel: self.$selectedModel, models: self.models)
    
        }
    }

}
}

struct ARViewContainer: UIViewRepresentable {
@binding var modelConfirmedForPlacement: Model?

func makeUIView(context: Context) -> ARView {
    
    let arView = CustomARView(frame: .zero)
    
    return arView
    
}

func updateUIView(_ uiView: ARView, context: Context) {
    if let model = self.modelConfirmedForPlacement{
        
        if let modelEntity = model.modelEntity {
            print("DEBUG: adding model to scene - \(model.modelName)")
            let anchorEntity = AnchorEntity(plane:.any)
            anchorEntity.addChild(modelEntity.clone(recursive: true))
            
            uiView.scene.addAnchor(anchorEntity)
        } else {
            print("DEBUG: Unable to load modelEntity for \(model.modelName)")
        }
        
        
        
        
        DispatchQueue.main.async {
            self.modelConfirmedForPlacement = nil}
    }
    
    
}

}

class CustomARView: ARView {
    let focusSquare = FESquare()

    required init(frame frameRect: CGRect) {
    super.init(frame: frameRect)
    
    focusSquare.viewDelegate = self
    focusSquare.delegate = self
    focusSquare.setAutoUpdate(to: true)
        
        self.setupARView()
    
    }
    
    @objc required dynamic init?(coder decoder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    func setupARView(){
        let config = ARWorldTrackingConfiguration()
        config.planeDetection = [.horizontal, .vertical]
        config.environmentTexturing = .automatic
        
   //     if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
  //          config.sceneReconstruction = .mesh
      //  }
        
        self.session.run(config)
        
    }

}

extension CustomARView: FEDelegate {
func toTrackingState () {
print("tracking")
}

func toInitializingState() {
    print("initializing")
}
}

struct ModelPickerView: View{
@binding var isPlacementEnabled: Bool
@binding var selectedModel: Model?

var models: [Model]
var body: some View{
    ScrollView(.horizontal, showsIndicators: false) {
        HStack(spacing: 30){
            ForEach(0..<self.models.count)  {
                index in
                Button(action: {
                    print("DEBUG: selected model with name: \(self.models[index].modelName)")
                    
                    self.selectedModel = self.models[index]
                    
                    self.isPlacementEnabled = true
                }) {
                    Image(uiImage:
                           
                                        self.models[index].image)
                        .resizable()
                        .frame(height: 80)
                        .aspectRatio(1/1, contentMode: .fit)
                        .background(Color.white)
                        .cornerRadius(4)
                }.buttonStyle(PlainButtonStyle())
        }
        }
    }.padding(20)
    .background(Color.black.opacity(0.5))
}

}
struct PlacementButtonsView: View {
@binding var isPlacementEnabled: Bool
@binding var selectedModel: Model?
@binding var modelConformedForPlacement: Model?

var body: some View {
    HStack {
        // Cancel Button
        Button(action: { print("DEBUG: model placement canceled")
            
            self.resetPlacementParameters()
        }) {
            Image(systemName: "xmark")
                .frame(width: 60, height: 60)
                .font(.title)
                .background(Color.white.opacity(0.75))
                .cornerRadius(30)
                .padding(20)
        }
        // Confirm Button
        Button(action: { print("DEBUG: model placemend confirmed")
            self.modelConformedForPlacement = self.selectedModel
            
            self.resetPlacementParameters()
        }) {
            Image(systemName: "checkmark")
                .frame(width: 60, height: 60)
                .font(.title)
                .background(Color.white.opacity(0.75))
                .cornerRadius(30)
                .padding(20)
        }
    }
    
        
        }
func resetPlacementParameters() {
    self.isPlacementEnabled = false
    self.selectedModel = nil
    }

}

#if DEBUG
struct ContentView_Previews : PreviewProvider {
static var previews: some View {
ContentView()
}
}

#endif

Cannot use focusEntity 2.2.3 with Xcode 13-Beta2

dyld: Library not loaded: /System/Library/Frameworks/RealityFoundation.framework/RealityFoundation
Referenced from: /private/var/containers/Bundle/Application/8D110FAF-932D-4910-BCE9-169EE48D339E/testAR.app/testAR
Reason: image not found
dyld: launch, loading dependent libraries
DYLD_LIBRARY_PATH=/usr/lib/system/introspection
DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libBacktraceRecording.dylib:/Developer/usr/lib/libMainThreadChecker.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib:/usr/lib/libMTLCapture.dylib

Environment:

  • iOS Version - 13
  • Device Information - iPAD 2020
  • macOS Version - 11.3.1 (20E241)
  • Xcode Version - 13 Beta 2

Error: Value of type 'FocusEntity' has no member 'position'

Hi, added the source files directly to the project, and when I try to compile my project and select iOS simulator I am getting this error:

Value of type 'FocusEntity' has no member 'position'

This error will be gone if I select a real device. AR is just a part of my app, so I need to compile the project on the simulator too. How can I resolve this issue?

Change color or shape of original target-square.

Describe the bug
Can't change the color of the target square and the shape (if I don't really like the square)

To Reproduce

class CustomARView: ARView {
    var focusEntity: FocusEntity!

    required init (frame frameRect: CGRect) {
        super.init (frame: frameRect)

        focusEntity = FocusEntity (on: self, style: .classic (color: .red))
// оr focusEntity = FocusEntity (on: self, style: .colored (onColor: .white, offColor: .gray, otherColor: .blue, mesh: MeshResource.generateSphere (radius: 0.5)))
   
        focusEntity.delegate = self
        focusEntity.setAutoUpdate (to: true)
    }
    
    @objc required dynamic init? (coder decoder: NSCoder) {
        fatalError ("init (coder :) has not been implemented")
    }
}

Expected behavior
Change the color or shape of the original target square.

Environment:

  • iOS Version - 14.1
  • Device Information - iPhone XS Max
  • macOS Version - BigSur beta 11
  • Xcode Version - 12.0

Crashing on iOS 13.2

Describe the bug
When using iOS 13.2 this class does not work when imported as a Swift Package.

To Reproduce
Steps to reproduce the behavior:

  1. Add package to project
  2. Run on iOS 13.2
  3. See crash when even creating any of the given classes

Expected behavior
No crash, should work as in previous iOS versions.

Environment:

  • iOS Version: 13.2
  • Device Information: iPhone 11 Pro
  • Xcode Version: 11.1

Additional context
Works fine when adding the swift files individually, as with the Example in this repo

FocusEntityDelegate Bug

Describe the bug
i'm trying to connect swiftui UIViewRepresentable with FocusEntityDelegate. however FocusEntityDelegate has two functions but never gets called. i am trying to see if it is called or not using the print() command. but it doesn't appear
how can i use FocusEntityDelegate in my project?
I'm using the latest version of this library

image
image
image

Environment:

  • iOS 15.2
  • iphone 12
  • macOS 12.1
  • Xcode 13.3.1

Transparent texture problem

Describe the bug
The example with the png texture are not transparent

To Reproduce
Steps to reproduce the behavior:

  1. Go to example
  2. Try the material example
  3. Try on a device
  4. The png's are not transparent

Expected behavior
The texture is transparent

Problem seems to be here
Proposed fix
modelMaterial.color = .init(tint: .white.withAlphaComponent(0.9999), texture: .init(tex))

Texture mapping capability

@maxxfrazer
Question:
Thanks for bug fix with the color change when creating FocusEntity. The second part of my question was about how to apply a different shape for the FocusEntity (by default it is a square, I need a circle (see Ikea Place app)), besides how I should be able to add texture, not just color.

Decision:
You need to add this functionality as a must have.

The alternatives I have considered:

  1. Using FocusEntity, I create it custom - I can apply MeshResource and colored, but there is no way to apply a texture:
let meshResource = MeshResource.generatePlane (width: 0.2, depth: 0.2)
let focusEntity = FocusEntity (on: self, focus: FocusEntityComponent (style: .colored (onColor: .red, offColor: .green, nonTrackingColor: .blue, mesh: meshResource)))
  1. Without using FocusEntity, I created a ModelEntity with a solid fill or texture, after which I generated a rayCast from center of screen and positioned (moved) ModelEntity to a point on the detected plane:
// ...
let meshResource = MeshResource.generatePlane (width: width, depth: width, cornerRadius: 2)
var material: UnlitMaterial!
 if let url = Bundle.main.url (forResource: "Files/targetShapeTexture_001", withExtension: "png") {
 texture = try TextureResource.load (contentsOf: url)
material = UnlitMaterial()
material! .baseColor = .texture (texture!)
material! .tintColor = UIColor (red: 25, green: 25, blue: 25, alpha: 0.5)
// ...
// modelEntity
let modelEntity = ModelEntity (mesh: meshResource, materials: [material])
modelEntity.name = "modelEntity_targetShape"
// anchorEntity
let anchorEntity = AnchorEntity (plane: .any)
anchorEntity.addChild (modelEntity)
anchorEntity.name = "anchorEntity_targetShape"
return anchorEntity
// ...
// Moving
targetShape.move (to: transform, relativeTo: nil, duration: 0.4, timingFunction: .easeOut)
// ...

This works, but there is a noticeable lag in texture rendering when moving, moreover, when changing the ARCamera.TrackingState status other than .normal, unpredictable texture behavior is observed.

Thanks!

Lidar Device Issue

Have you tried running this same project on a Lidar-enabled device?

I tried it on my iPad Pro 2020 and it did not perform well with locating planes.
It would fail to change orientation to match the new surface.

I also tried it with a prior version of the project, commit 88112e6, and it has the same issues.

I think this could have something to do with the raycasting, but sceneUnderstanding.options does not include .collisions, so it should not be affecting raycast results.

I did also use

self.debugOptions.insert(.showAnchorGeometry)

To make sure that planes were being detected, and they were.

required public init()

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

required public init() {
    fatalError("init() has not been implemented")
}

this occurs an error “Thread 1: Fatal error: init(coder:) has not been implemented”

Expected behavior

shouldn't it be changed to like below?

required public init?(coder decoder: NSCoder){
fatalError("init(coder:) has not been implemented")
}

Screenshots
If applicable, add screenshots to help explain your problem.

Environment:

  • iOS Version : 15.6
  • Device Information : iPad Pro(12.9)(4th)
  • macOS Version : 13.1
  • Xcode Version : 14.2

Additional context

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.