lisilence/arkitdemo

ARKit学习Demo合集

ARKit学习

1. ARKit点击屏幕增加文字
2. ARKit点击屏幕增加3D模型
3. ARKit检测到平面自动增加3D模型
4. QuickLook的最简单使用
5. ARKit人脸贴图
6. ARKit微笑检测
7. ARKit皱眉检测
8. ARKit人脸参数BlendShapes详解

1. ARKit点击屏幕增加文字

• command+shift+n新建一个项目，然后选择Augmented Reality App

• 在Content Technology中选择SpriteKit即可

• 控制文字距离相机的距离(改变这个Z感受一下变化)

matrix_float4x4 translation = matrix_identity_float4x4;
translation.columns[3].z = -1;

2. ARKit点击屏幕增加3D模型

2.1 画面捕捉

主要就是三个类：

• ARSCNView: 画面显示
• ARConfiguration: 捕捉画面
  • ARWorldTrackingConfiguration:后置摄像头
  • ARFaceTrackingConfiguration：前置摄像头,会实时监测面部表情特征
• ARSession:数据中转

在viewDidLoad的时候初始化资源

self.arSCNView = [[ARSCNView alloc] initWithFrame:self.view.bounds options:nil];
    self.arSCNView.session = [[ARSession alloc] init];
    // 1. 创建世界追踪配置，需要支持A9芯片也就是iPhone6S以上
    self.arWordTrackingConfiguration = [[ARWorldTrackingConfiguration alloc] init];
    // 2. 设置追踪方向,追踪平面
    self.arWordTrackingConfiguration.planeDetection = ARPlaneDetectionHorizontal;
    self.arWordTrackingConfiguration.lightEstimationEnabled = YES;

在viewDidAppear时让session开始工作

[self.arSession runWithConfiguration:self.arWordTrackingConfiguration]

2.2 点击增加3D图像

当点击屏幕的时候加载一个scn文件并且作为childNode添加到self.arSCNView.scene.rootNode

- (void)touchesBegan:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event {
    // 1. 使用场景加载scn文件
    SCNScene *scene = [SCNScene sceneNamed:@"art.scnassets/ship.scn"];
    
    SCNNode *shipNode = scene.rootNode.childNodes.firstObject;
    shipNode.position = SCNVector3Make(0, -1, -1);
    
    [self.arSCNView.scene.rootNode addChildNode:shipNode];
}

3. ARKit检测到平面自动增加3D模型

前期准备工作和2.1一样，只是增加了self.arSCNView.delegate = self 然后在代理方法renderer:didAddNode:forAnchor:中实现以下代码：

#pragma mark - ARSCNViewDelegate
// 添加节点的时候调用(当开启平地捕捉模式之后，如果捕捉到平地，ARKit会自动添加一个平地节点)
- (void)renderer:(id<SCNSceneRenderer>)renderer didAddNode:(SCNNode *)node forAnchor:(ARAnchor *)anchor {
    
    if (![anchor isMemberOfClass:[ARPlaneAnchor class]]) return;
    
    // 添加一个3D平面模型，ARKit只有捕捉能力，锚点只是一个空间位置，想更加清楚看到这个空间，我们需要给控件添加一个平地的3D模型来渲染它
    // 1. 获取捕捉到的平地锚点
    ARPlaneAnchor *planeAnchor = (ARPlaneAnchor *)anchor;
    // 2. 创建一个3D模型(系统捕捉到的平地是一个不规则的大小长方形，这里笔者q将其变成一个长方形，并且对平地做了一个缩放效果)
    // 参数分别是长、宽、高、圆角
    SCNBox *planeBox = [SCNBox boxWithWidth:planeAnchor.extent.x * 0.3 height:0 length:planeAnchor.extent.x * 0.3 chamferRadius:0];
    // 3. 使用Material渲染3D模型(默认模型是白色的)
    planeBox.firstMaterial.diffuse.contents = [UIColor clearColor];
    // 4. 创建一个基于3D物体模型的节点
    SCNNode *planeNode = [SCNNode nodeWithGeometry:planeBox];
    // 5. 设置节点的位置为捕捉到的平地的锚点的中心位置
    // SceneKit中节点的位置position是一个基于3D坐标系的矢量坐标SCNVector3Make
    planeNode.position = SCNVector3Make(planeAnchor.center.x, 0, planeAnchor.center.z);
    
    [node addChildNode:planeNode];
    
    // 6. 创建一个花瓶场景
    SCNScene *scene = [SCNScene sceneNamed:@"art.scnassets/vase/vase.scn"];
    // 7. 获取花瓶节点
    // 一个场景有多个节点，所有场景有且只有一个根节点，其它所有节点都是根节点的子节点
    SCNNode *vaseNode = scene.rootNode.childNodes.firstObject;
    // 8. 设置花瓶节点的位置为捕捉到的平地的位置，如果不设置，则默认为原点位置也就是相机位置
    vaseNode.position = SCNVector3Make(planeAnchor.center.x, 0, planeAnchor.center.z);
    // 9. 将花瓶节点添加到屏幕中
    // !!!!FBI WARNING: 花瓶节点是添加到代理捕捉到的节点中，而不是AR视图的根接节点。
    // 因为捕捉到的平地锚点是一个本地坐标系，而不是世界坐标系
    [node addChildNode:vaseNode];
}

4. QuickLook的最简单使用

这个没什么好说的，直接上代码

#import "ViewController.h"
#import <QuickLook/QuickLook.h>

@interface ViewController ()<QLPreviewControllerDataSource, QLPreviewControllerDelegate>

@end

@implementation ViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    
}

- (void)touchesBegan:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event {
    QLPreviewController *preVC = [[QLPreviewController alloc] init];
    preVC.dataSource = self;
    preVC.delegate = self;
    
    [self presentViewController:preVC animated:YES completion:nil];
}

#pragma mark - QLPreviewControllerDataSource && QLPreviewControllerDelegate
- (NSInteger)numberOfPreviewItemsInPreviewController:(QLPreviewController *)controller {
    return 1;
}

- (id<QLPreviewItem>)previewController:(QLPreviewController *)controller previewItemAtIndex:(NSInteger)index {
    
    return [[NSBundle mainBundle] URLForResource:@"plantpot.usdz" withExtension:nil];
}

- (UIImage *)previewController:(QLPreviewController *)controller transitionImageForPreviewItem:(id<QLPreviewItem>)item contentRect:(CGRect *)contentRect {
    
    CGRect rect = CGRectMake(100, 200, 300, 300);
    contentRect = &rect;
    
    return [UIImage imageNamed:@"wy.jpeg"];
}

5. ARKit人脸贴图

设置session的configuration为ARFaceTrackingConfiguration,然后在ARSCNView的代理renderer:willUpdateNode:forAnchor中增加一个SCNNode核心代码如下：

• 创建SCNNode
  • 试试看设置fillMesh为YES会怎么样
  • 试试看设置masterial.diffuse.contents为一个颜色会怎么样

- (SCNNode *)textureMaskNode {
    if (!_textureMaskNode) {
        
        id<MTLDevice> device = self.arSCNView.device;
        ARSCNFaceGeometry *geometry = [ARSCNFaceGeometry faceGeometryWithDevice:device fillMesh:NO];
        SCNMaterial *material = geometry.firstMaterial;
        material.fillMode = SCNFillModeFill;
        material.diffuse.contents = [UIImage imageNamed:@"wy.jpg"];
        _textureMaskNode = [SCNNode nodeWithGeometry:geometry];
    }
    _textureMaskNode.name = @"textureMask";
    return _textureMaskNode;
}

• 添加SCNNode并更新人脸特征

- (void)renderer:(id<SCNSceneRenderer>)renderer willUpdateNode:(SCNNode *)node forAnchor:(ARAnchor *)anchor {
    
    if (!anchor || ![anchor isKindOfClass:[ARFaceAnchor class]]) return;
    ARFaceAnchor *faceAnchor = (ARFaceAnchor *)anchor;
    
    if (!_textureMaskNode) {
        [node addChildNode:self.textureMaskNode];
    }
    
    ARSCNFaceGeometry *faceGeometry = (ARSCNFaceGeometry *)self.textureMaskNode.geometry;
    if (faceGeometry && [faceGeometry isKindOfClass:[ARSCNFaceGeometry class]]) {
        [faceGeometry updateFromFaceGeometry:faceAnchor.geometry];
    }
}

6. ARKit微笑检测

主要用到了ARBlendShapeLocationMouthSmileLeft和ARBlendShapeLocationMouthSmileRight表示微笑的键值 我提供的demo是用于调试微笑阀值的 核心代码：

- (void)renderer:(id<SCNSceneRenderer>)renderer didUpdateNode:(SCNNode *)node forAnchor:(ARAnchor *)anchor {
    
    if (!anchor || ![anchor isKindOfClass:[ARFaceAnchor class]]) return;
    
    ARFaceAnchor *faceAnchor = (ARFaceAnchor *)anchor;
    
    NSDictionary *blendShips = faceAnchor.blendShapes;
    CGFloat leftSmile = [blendShips[ARBlendShapeLocationMouthSmileLeft] floatValue];
    CGFloat rightSmile = [blendShips[ARBlendShapeLocationMouthSmileRight] floatValue];
    
    NSLog(@"leftSmile = %f, rightSmile = %f", leftSmile, rightSmile);
    if (leftSmile > self.smileValue && rightSmile > self.smileValue) {
        NSLog(@"检测到笑容");
        [self.arSession pause];
        dispatch_async(dispatch_get_main_queue(), ^{
            self.resultLabel.hidden = NO;
        });
    }
}

7. ARKit皱眉检测

我这里用的是眉毛向上的键值ARBlendShapeLocationBrowInnerUp 核心代码：

- (void)renderer:(id<SCNSceneRenderer>)renderer didUpdateNode:(SCNNode *)node forAnchor:(ARAnchor *)anchor {
    
    if (!anchor && ![anchor isKindOfClass:[ARFaceAnchor class]]) return;
    
    ARFaceAnchor *faceAnchor = (ARFaceAnchor *)anchor;
    NSDictionary *blendShapes = faceAnchor.blendShapes;
    NSNumber *browInnerUp = blendShapes[ARBlendShapeLocationBrowInnerUp];
    
    if ([browInnerUp floatValue] > self.browValue) {
        [self.arSession pause];
        dispatch_async(dispatch_get_main_queue(), ^{
            self.resultLabel.hidden = NO;
        });
    }
    
    NSLog(@"browInnerUp = %@", browInnerUp);
}

8. BlendShapes

• 仅在iOS11及以上可用，每个参数的详细介绍和图片对比可以打开Xcode->Window->Developer Documentation,然后搜索对应的键值即可
• 每个建对应的值都是0~1的值
• 共51个表示人脸特征的参数

属性	说明	备注
ARBlendShapeLocationBrowDownLeft	左眉毛外部向下
ARBlendShapeLocationBrowDownRight	右眉毛外部向下
ARBlendShapeLocationBrowInnerUp	两眉毛内部向上
ARBlendShapeLocationBrowOuterUpLeft	左眉毛外部向上
ARBlendShapeLocationBrowOuterUpRight	右眉毛外部向上
ARBlendShapeLocationCheekPuff	两个脸颊向外
ARBlendShapeLocationCheekSquintLeft	左眼向下斜视
ARBlendShapeLocationCheekSquintRight	右眼向下斜视
ARBlendShapeLocationEyeBlinkLeft	眨左眼
ARBlendShapeLocationEyeBlinkRight	眨右眼
ARBlendShapeLocationEyeLookDownLeft	左眼睑运动的系数与向下凝视一致
ARBlendShapeLocationEyeLookDownRight	右眼睑运动的系数与向下凝视一致
ARBlendShapeLocationEyeLookInLeft	左眼睑运动的系数与向右凝视一致。
ARBlendShapeLocationEyeLookInRight	右眼睑运动的系数与向左凝视一致。
ARBlendShapeLocationEyeLookOutLeft	左眼睑运动的系数与向左凝视一致
ARBlendShapeLocationEyeLookOutRight	右眼睑运动的系数与向右凝视一致
ARBlendShapeLocationEyeSquintLeft	左眼脸部收缩
ARBlendShapeLocationEyeSquintRight	右眼脸部收缩
ARBlendShapeLocationEyeWideLeft	左眼周围眼睑变宽
ARBlendShapeLocationEyeWideRight	右眼周围眼睑变宽
ARBlendShapeLocationJawForward	下颌向前运动
ARBlendShapeLocationJawLeft	下颌向左运动
ARBlendShapeLocationJawOpen	下颌开口
ARBlendShapeLocationJawRight	下颌向右运动
ARBlendShapeLocationMouthClose	嘴唇闭合的系数与颌位置无关
ARBlendShapeLocationMouthDimpleLeft	嘴左角后移
ARBlendShapeLocationMouthDimpleRight	嘴右角后移
ARBlendShapeLocationMouthFrownLeft	嘴左角向下运动
ARBlendShapeLocationMouthFrownRight	嘴右角向下运动
ARBlendShapeLocationMouthFunnel	两个嘴唇收缩成开放形状
ARBlendShapeLocationMouthLeft	两个嘴唇向左移动
ARBlendShapeLocationMouthLowerDownLeft	左侧下唇向下运动
ARBlendShapeLocationMouthLowerDownRight	又侧下唇向下运动
ARBlendShapeLocationMouthPressLeft	左侧下唇向上压缩
ARBlendShapeLocationMouthPressRight	右侧下唇向上压缩
ARBlendShapeLocationMouthPucker	两个闭合嘴唇的收缩和压缩
ARBlendShapeLocationMouthRight	两个嘴唇向右运动
ARBlendShapeLocationMouthRollLower	下唇向嘴内侧移动
ARBlendShapeLocationMouthRollUpper	上唇向嘴内侧移动
ARBlendShapeLocationMouthShrugLower	下唇向外运动
ARBlendShapeLocationMouthShrugUpper	上唇向外运动
ARBlendShapeLocationMouthSmileLeft	嘴左角向上运动
ARBlendShapeLocationMouthSmileRight	嘴右角向上运动
ARBlendShapeLocationMouthStretchLeft	嘴左角向左移动
ARBlendShapeLocationMouthStretchRight	嘴左角向右移动
ARBlendShapeLocationMouthUpperUpLeft	左侧上唇向上运动
ARBlendShapeLocationMouthUpperUpRight	右侧上唇向上运动
ARBlendShapeLocationNoseSneerLeft	左鼻孔抬高
ARBlendShapeLocationNoseSneerRight	右鼻孔抬高
ARBlendShapeLocationTongueOut	舌头延伸

1. ARBlendShapeLocationMouthClose