sobri909/locokit

Location, motion, and activity recording framework for iOS

LocoKit

A Machine Learning based location recording and activity detection framework for iOS.

Location and Motion Recording

• Combined, simplified Core Location and Core Motion recording
• Filtered, smoothed, and simplified location and motion data
• Near real time stationary / moving state detection
• Automatic energy use management, enabling all day recording
• Automatic stopping and restarting of recording, to avoid wasteful battery use

Activity Type Detection

• Machine Learning based activity type detection
• Improved detection of Core Motion activity types (stationary, walking, running, cycling, automotive)
• Distinguish between specific transport types (car, train, bus, motorcycle, airplane, boat)

Record High Level Visits and Paths

• Optionally produce high level Path and Visit timeline items, to represent the recording session at human level. Similar to Core Location's CLVisit, but with much higher accuracy, much more detail, and with the addition of Paths (ie the trips between Visits).
• Optionally persist your recorded samples and timeline items to a local SQL based store, for retention between sessions.

More information about timeline items can be found here

Supporting the Project

LocoKit is an LGPL licensed open source project. Its ongoing development is made possible thanks to the support of its backers on Patreon.

• patreon.com/sobri909

If you have an app that uses LocoKit and is a revenue generating product, please consider sponsoring LocoKit development, to ensure the project that your product relies on stays healthy and actively maintained.

Thanks so much for your support!

Installation

pod 'LocoKit'
pod 'LocoKit/LocalStore' # optional

Note: Include the optional LocoKit/LocalStore subspec if you would like to retain your samples and timeline items in the SQL persistent store.

High Level Recording

Record TimelineItems (Paths and Visits)

// retain a timeline manager
self.timeline = TimelineManager()

// start recording, and producing timeline items 
self.timeline.startRecording()

// observe timeline item updates
when(timeline, does: .updatedTimelineItem) { _ in
    let currentItem = timeline.currentItem

    // duration of the current Path or Visit
    print("item.duration: \(currentItem.duration)")

    // activity type of the current Path (eg walking, cycling, car)
    if let path = currentItem as? Path {
        print("path.activityType: \(path.activityType)")
    }

    // examine each of the LocomotionSamples within the Path or Visit
    for sample in currentItem.samples {
        print("sample: \(sample)")
    }
}

Low Level Recording

Record LocomotionSamples (CLLocations combined with Core Motion data)

// the recording manager singleton
let loco = LocomotionManager.highlander

// decide which Core Motion features to include
loco.recordPedometerEvents = true
loco.recordAccelerometerEvents = true
loco.recordCoreMotionActivityTypeEvents = true

// decide whether to use "sleep mode" to allow for all day recording 
loco.useLowPowerSleepModeWhileStationary = true

Note: The above settings are all on by default. The above snippets are unnecessary, and just here to show you some of the available options.

// start recording 
loco.startRecording()

// watch for updated LocomotionSamples
when(loco, does: .locomotionSampleUpdated) { _ in

    // the raw CLLocation
    print(loco.rawLocation)

    // a more usable, de-noised CLLocation
    print(loco.filteredLocation)

    // a smoothed, simplified, combined location and motion sample
    print(loco.locomotionSample())
}

Fetching TimelineItems / Samples

If you wanted to get all timeline items between the start of today and now, you might do this:

let date = Date() // some specific day
let items = store.items(
        where: "deleted = 0 AND endDate > ? AND startDate < ? ORDER BY endDate",
        arguments: [date.startOfDay, date.endOfDay])

You can also construct more complex queries, like for fetching all timeline items that overlap a certain geographic region. Or all samples of a specific activity type (eg all "car" samples). Or all timeline items that contain samples over a certain speed (eg paths containing fast driving).

Detect Activity Types

Note that if you are using a TimelineManager, activity type classifying is already handled for you by the manager, on both the sample and timeline item levels. You should only need to directly interact with clasifiers if you are either not using a TimelineManager, or are wanting to do low level processing at the sample level.

// fetch a geographically relevant classifier
let classifier = ActivityTypeClassifier(coordinate: location.coordinate)

// classify a locomotion sample
let results = classifier.classify(sample)

// get the best match activity type
let bestMatch = results.first

// print the best match type's name ("walking", "car", etc)
print(bestMatch.name)

Note: The above code snippets use SwiftNotes to make the event observing code easier to read. If you're not using SwiftNotes, your observers should be written something like this:

let noteCenter = NotificationCenter.default
let queue = OperationQueue.main 

// watch for updates
noteCenter.addObserver(forName: .locomotionSampleUpdated, object: loco, queue: queue) { _ in
    // do stuff
}

Background Location Monitoring

If you want the app to be relaunched after the user force quits, enable significant location change monitoring.

More details and requirements here

Examples and Screenshots

• Location filtering examples
• Activity type detection examples

Documentation

• LocoKit API reference
• LocoKitCore API reference

Try the LocoKit Demo App

1. Download or clone this repository
2. pod install
3. In Xcode, change the Demo App project's "Team" to match your Apple Developer Account
4. In Xcode, change the Demo App project's "Bundle Identifier" to something unique
5. Build and run!
6. Go for a walk, cycle, drive, etc, and see the results :)

Try Arc App on the App Store

• To see the SDK in action in a live, production app, install Arc App from the App Store, our free life logging app based on LocoKit

locokit's Issues

Build the Android SDK

I'll be tracking the development of the initial Android SDK in this task. Just filing it now as an empty shell, to get the ball rolling.

[Question] background tracking notification

Can I somehow re-enable the ios blue bar notification while my app is tracking GPS?

I often forget to stop the exercise tracking, so I have +15h exercise. I am afraid that the users will do this too :/

Location update after suspending the app from background

Can we get location update even after suspending my app from background?

Consider going back to 1 SD for visit radiuses during overlap testing

Related to #31.

In Arc App v1 builds, visit radiuses were uniformly treated as meanFromCenter + 1sd (with SD being taken each sample's distance from the weighted centre).

In Arc App v2 builds (and I think maybe all LocoKit releases), I standardised on 2 SD instead. This made adjacent visit merges more greedy, and solved problems where people were seeing separate rooms in their houses logged as separate timeline items, for example.

But in the past week I've been getting a few reports of the opposite complaint, where adjacent visits are merging when the user doesn't want that. Nearby shops / buildings / etc glomming together into single visits, due to overlapping visit radiuses.

The #31 changes might've solved this in a bunch of cases. But there's a good chance it won't please everyone in all cases.

I had some cunning plans for dealing with this, something to do with dynamic radius calculations, taking into account more factors. Essentially ditching the hard coded SD factor. But I can't remember right now what those cunning plans were. Hopefully I'll remember soon. It would be nice to do something smart, instead of just nudging hard coded thresholds back and forth and pleasing one group of people while annoying others.

Keep arrays of the CoreMotion raw data in LocomotionSamples

Currently LocomotionSamples keep arrays of their source raw CLLocation data (sample.rawLocations), but not the source raw CoreMotion data. It would be helpful for those objects to be kept associated with the LocomotionSamples.

This task has some crossover with #21.

License

Hi, do you have any public license for this pod. Something like MIT?

CoreMotion Question

Is the CoreMotion data used to improve the location accuracy or just used to provide filtered and sanitised accelerometer, etc ?

Attempt to guess the end location for paths that end with nolos

This one comes out of several users reporting airplane flights where the start of the flight has been recorded, but the end of the flight was only nolos (no location data). Currently these paths won't be sensibly fleshed out to the end, because reasons.

Because Reasons

If LocoKit has both ends of the flight recorded with location data, then it has an existing mechanism to sensibly merge over the nolo gap, based on the path speeds on either side (ie if both speeds imply that the user could have covered that distance in that time).

But there's no mechanism in LocoKit for dealing with the case where there's no speed relevant path on the other side of the gap. For example if there's 30 minutes of airplane speed, then nolos for 2 hours, then 1 minute of walking (ie getting off the plane at the airport), then it will nope out on the merge, because the average of the airplane speed and walking speed means that the two paths can't be merged. Which of course they shouldn't be.

Better Idea

But the airplane flight should still be able to look at its own speed, and the subsequent nolos gap, and then the first coordinate of the walking, and determine that it could have covered that distance in that time. The walking path's first coordinate could then be grafted onto the end of the airplane flight, and voila, a healed airplane trip.

ArcApp merges few transport types into one

So basically Arc app (newest version) started merging some of different transport types into one. For example:

1km walk
5km train
2km walk

Merges into "8km walk". When I go to 'Recorded activities' tab and I try to split activities, it detaches for a few seconds and then merges again.

I only had such problems with data from Moves import, demonstrated on the video: (might be a bit confusing without finger circles, but I hope it'll help!)

ScreenRecording_08-31-2018 22-18-53.mp4.zip

Start using a "jobs queue" (ie an OperationQueue) for processing tasks

With MutableActivityType coming soon (#34), there's going to potentially be a lot more background processing going on in LocoKit.

Arc App currently manages these job queue priorities, timeouts, etc by extensive use of duct tape, number 8 wire, and spare hair ties found on the floor. That's not going to cut it once the queues are moved into LocoKit [translation: It would be too embarrassing to have people see my jumble of hacks]. So it's time to do it properly.

Migrate Arc's UD model layer into LocoKit

Arc keeps an extra layer of activity type ML models than LocoKit. The extra layer being the user's own personal models, based on their own type confirmations / corrections.

• In LocoKit the tree is GD2 -> GD1 -> GD0 (neighbourhood -> city -> world).
• In Arc the tree is UD2 -> GD2 -> GD1 -> GD0 (personal neighbourhood -> shared neighbourhood -> city -> world).

So that UD2 layer needs to be migrated into LocoKit.

This will also open the door for doing custom activity types directly in LocoKit (and also in Arc).

Todos

• Migrate UserActivityType upstream
• Migrate UserActivityTypeClassifier upstream
• Migrate UserTimelineClassifier upstream

[Request] Swift 4.2

Hello,

can you please make a cocoapod version that can imported in xCode10 and swift 4.2?

Thanks :)

Example for persistent store

Hey, I'm having some trouble figuring out how to get the persistent store set up.

It seems like things might be working in localstorage, there is some level of persistence on the device in my simulator (e.g. when I print out Items, this persists from one session to the next). But I'm not seeing those show up in a database anywhere. I didn't see a ton of specifics on this in terms of what type of database to use and how to set it up. Any help would be much appreciated!

Here's my file where I'm initializing everything (also, fair warning, I come from a node.js background, so swift in general is all a bit new to me 😄)

Thank you in advance!

//
//  AppDelegate.swift
//  Joy
//
//  Created by Danny on 10/4/17.
//  Copyright © 2017 Joy. All rights reserved.
//

import UIKit
import LocoKit
import SwiftNotes

let appDelegate = UIApplication.shared.delegate as! AppDelegate

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
    
    var timeline: TimelineManager = PersistentTimelineManager()
    var store: PersistentTimelineStore = PersistentTimelineStore()
    
    var window: UIWindow?
    
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
        LocoKitService.apiKey = "<REDACTED>"
        
        self.store = PersistentTimelineStore()
        store.dbUrl = URL(string: "<REDACTED>")!

        // the CoreLocation / CoreMotion recording singleton
        let loco = LocomotionManager.highlander
        
        // this is independent of the user's setting, and will show a blue bar if user has denied "always"
        loco.locationManager.allowsBackgroundLocationUpdates = true
        
        // retain a timeline manager
        self.timeline = PersistentTimelineManager()
        
        // restore the active timeline items from local db
        if let timeline = timeline as? PersistentTimelineManager {
            timeline.bootstrapActiveItems()
            print("bootstrapActiveItems ran....")
            
            // start recording, and producing timeline items
            self.timeline.startRecording()
        }
        
        // observe new timeline items
        when(timeline, does: .newTimelineItem) { _ in
            if let currentItem = self.timeline.currentItem {
                print(".newTimelineItem (\(String(describing: type(of: currentItem))))")
            }
            onMain {
                let items = self.itemsToShow
                print(items, "<<<<")
            }
        }
        
        // observe timeline item updates
        when(timeline, does: .updatedTimelineItem) { _ in
            onMain {
                let items = self.itemsToShow
            }
        }
        
        // observe timeline items finalised after post processing
        when(timeline, does: .finalisedTimelineItem) { note in
            if let item = note.userInfo?["timelineItem"] as? TimelineItem {
                print(".finalisedTimelineItem (\(String(describing: type(of: item))))")
            }
        }
        
        when(timeline, does: .mergedTimelineItems) { note in
            if let description = note.userInfo?["merge"] as? String {
                print(".mergedItems (\(description))")
            }
        }
        
        // observe incoming location / locomotion updates
        when(loco, does: .locomotionSampleUpdated) { _ in
            self.locomotionSampleUpdated()
        }
        
        // observe changes in the recording state (recording / sleeping)
        when(loco, does: .recordingStateChanged) { _ in
            // don't log every type of state change, because it gets noisy
            if loco.recordingState == .recording || loco.recordingState == .off {
                print(".recordingStateChanged (\(loco.recordingState))")
            }
        }
        
        // observe changes in the moving state (moving / stationary)
        when(loco, does: .movingStateChanged) { _ in
            print(".movingStateChanged (\(loco.movingState))")
        }
        
        when(loco, does: .startedSleepMode) { _ in
            print(".startedSleepMode")
        }
        
        when(loco, does: .stoppedSleepMode) { _ in
            print(".stoppedSleepMode")
        }
        
        when(.debugInfo) { note in
            if let info = note.userInfo?["info"] as? String {
                print(".debug (\(info))")
            } else {
                print(".debug (nil)")
            }
        }
        
        return true
    }
    
    var itemsToShow: [TimelineItem] {
        if timeline is PersistentTimelineManager { return persistentItemsToShow }
        
        guard let currentItem = timeline.currentItem else { return [] }
        
        // collect the linked list of timeline items
        var items: [TimelineItem] = [currentItem]
        var workingItem = currentItem
        while let previous = workingItem.previousItem {
            items.append(previous)
            workingItem = previous
        }
        
        return items
    }
    
    var persistentItemsToShow: [TimelineItem] {
        guard let timeline = timeline as? PersistentTimelineManager else { return [] }
        
        // make sure the db is fresh
        timeline.store.save()
        
        // fetch all items in the past 24 hours
        let boundary = Date(timeIntervalSinceNow: -60 * 60 * 24)
        return timeline.store.items(where: "deleted = 0 AND endDate > ? ORDER BY endDate DESC", arguments: [boundary])
    }
    
    func locomotionSampleUpdated() {
        let loco = LocomotionManager.highlander
        
        // don't bother updating the UI when we're not in the foreground
        guard UIApplication.shared.applicationState == .active else {
            return
        }
        
        // get the latest sample and update the results views
        let sample = loco.locomotionSample()
    }

    func showLoggedInView() {
        // initilize vc right away, otherwise flashes on launch
        let vc = UINavigationController(rootViewController: SummaryViewController())
        show(vc)
    }

    func showLoggedOutView() {
        let vc = OnboardingNavigationController(rootViewController: WelcomeViewController())
        show(vc)
    }

    private func show(_ vc: UIViewController) {
        window = UIWindow(frame: UIScreen.main.bounds)
        window?.rootViewController = vc
        window?.makeKeyAndVisible()
        window?.tintColor = .joyPurple
    }

    func applicationDidBecomeActive(_ application: UIApplication) {
        application.applicationIconBadgeNumber = 0
    }

}

Rare, mystery high mobile data usage

There was (or apparently still is) an iOS bug that sometimes results in high mobile data usage due to wifi triangulation.

Arc App suffered from this a couple of years ago. And with a bit of back and forth with Apple, we determined that it was due to Arc App's regular updating of geofences, which would result in Location Services resetting and forgetting all its wifi triangulation data, thus having to refetch it.

At that time the easiest solution was to just give up the geofences. Arc App didn't need them anyway. I also reduced the frequency of LocoKit's dynamic desiredAccuracy updating, because changing that value also appeared to trigger the iOS bug.

Since then I hadn't heard anything more about it, until last week.

An Arc App user who spends many hours driving every day has reported the bug again. Mysteriously, "Time & Location" under "System Services" had consumed around 2GB of mobile data in a month.

I don't have any other user reports of this bug resurfacing. But I vaguely recall that Apple never actually closed the bug report for it. So maybe they merely mitigated it, but the core problem still remains.

So there's investigating to be done!

High battery usage consumption while stationary

A handful of Arc App users are reporting mysteriously high battery usage. Sometimes 10 or 20 times more than expected.

For the more extreme cases the users are seeing their batteries draining in a matter of hours. Normally Arc App and LocoKit can record entire days on a single battery charge, so for these particular users, something very wrong is happening.

My current best guess of the cause, based on user screenshots, is that for these users LocoKit is cycling between sleep and wake every few minutes, while stationary, instead of staying in sleep state until the user leaves their current location.

My hunch is the affected devices are producing a magical rare mix of low accuracy location data and slowly drifting location data, which could be somehow getting through LocoKitCore's filtering. Basically some poisonous combination of inaccurate location data while stationary that's slipping through the cracks and messing up sleep mode.

The problem is this never happens on any of my test devices, and is extremely difficult to properly debug remotely. So I need your help!

Are you experiencing this bug? Are you an iOS developer?

If so, I could really do with your help!

The best way to help is to install the LocoKit Demo App from this repo, and run it on the affected device in the affected location, and look for patterns.

The "LM" tab will hopefully be the most informative. Look for:

• Patterns in "Requesting accuracy" vs "Receiving accuracy"
• The number of locations in the "Latest sample"
• Incorrect changes in "Moving state"
• Drifting current location dots on the map
• A pattern of alternating short visits and short paths in the "TM" view while stationary

Note: You won't need an API key. I don't believe this bug is related to the ML classifiers at all.

Thanks for your help!

Delete/manage private places.

I accidentally created a private place and noticed that there seems to be no way to delete a private place.
Would be nice to remove the wrong private place. Now I just had to rename it to "wrong entry".

Settings for ItemSegment creation rules

Context

Historically, a TimelineItem's segments have been grouped on both sample.recordingState and sample.activityType. That gives the most granular detail about segments inside an item, and guarantees that a walking segment, for example, will only contain walking samples with the same recordingState (eg only [.walking && .recording] samples, with [.walking && .sleepMode] samples separated into a separate segment).

That level of granularity is desirable in some cases, but for Arc App it has evolved to mostly be a nuance. It results in things like a 30 minute stop at a cafe inside a large mall visit being split over multiple segments, for example:

1. [.stationary && .recording] 2 minutes <- arrived at the cafe
2. [.stationary && .sleepMode] 28 minutes <- recorder went to sleep

Changes in v6.0.0

A week or so back I changed the segment creation logic to ignore all recordingState differences except for the .off state (ie data gaps).

That greatly simplified the handling of visit brexits (see #13). All samples for a stationary period, regardless of whether .recording or .sleepMode, could be easily extracted together. Which let me throw away some nasty old logic from Arc App's brexit implementation.

More changes in v6.0.0

Now I want to go even further, and ignore recordingState altogether, so that even data gap segments are grouped together by matching activityType. Because I'm still ending up with situations like this. It's especially exaggerated during development, because there's unavoidably data gaps when relaunching the app. But it can still lead to unnecessary noise in normal operation too.

Getting back the lost detail

Simplifying down the segments like this necessarily throws away recodingState detail. There needs to be a way to get it back, when necessary.

One option would be to have a separate segments array on items (lazy loaded, of course). Something like item.segmentsByRecordingState or item.segmentsByActivityType, or ... yeah, something.

Another option would be to make it configurable. But that seems less ideal, because for most projects, both kinds of segment creation rules will be appropriate, at different times.

Yeah, I think I've talked myself into the idea of having separate segment "views", with differing levels of detail. I'll go with that, and see how it works out.

Move Arc App's brexit() method to LocoKit

Splitting this task off from #11, because it's not a requirement for that task, but will be nice to eventually have in LocoKit instead of hidden away in Arc App.

Background

The brexit() method allows for extracting an ItemSegment out of a TimelineItem into a separate item (for example a brief visit to a specific mall shop within a larger mall visit) and intelligently maintaining the timeline's linked list as well as extracting path items between the new items, as appropriate.

The reason the method is called "brexit" is because if the post processing doesn't go how you want it, you can get left with an unholy mess and wish you never tried in the first place.

Todos

• Split parent items in two if segment isn't on the edge
• Attempt to extract paths on either side, to create a more harmonious timeline

Deep sleep doesn't leave behind sleep samples for visit ends

If a deep sleep session isn't woken from before the device leaves the building, the visit's duration will be wrong because there's no sleep samples to mark its end.

I just spotted this one now on one of my test devices, after getting home from the cafe.

The app got woken by sig locs / CLVisits, and started recording again about 10-15 minutes after leaving the cafe, but because it deep slept at the cafe it didn't have any sleep samples to properly mark the end. So the visit ended up with a duration of 15 minutes, instead of 2 hours. Oops.

1. Need to do some more deep sleep testing, to make sure I haven't made any dumb mistakes. Need to be sure that the blame in this instance solely laid in the hands of iOS, and not LocoKit / me.

2. Need to figure out a way to more gracefully record from this sort of situation, so that the affected visits don't end up with nonsense short durations.

Time at home, work and commute for the Arc App like Moves App had

Hi,

I know you are busy with the import feature right now but can we get this on the list of features for the Arc App? Basically have the option to set some locations as 'home' or 'work' and have the time spent there displayed somewhere in the app. Thanks!

More activity types, and open sourcing the ML classifiers

It's too early to start on this one yet, but implementation ideas have been popping up in my head all week, so I might as well get them written down.

Misc random thoughts

1. Ditch the coords pseudo counts for transport types that are coordinate bound (eg car, bus, train, and any future road/track/etc bound types).

2. Only include types in the transport classifier that have non zero sample/coord counts for the D2 region, to avoid over stuffing the classifiers with possible types.

3. Look at ditching some model features, due to possibly being a negative influence. coreMotionActivityType is the most likely candidate for ditching, due to being wrong more often than right. Next would be courseVariance, due to being essentially meaningless for all types except stationary, since the introduction of the Kalmans.

4. Try letting zero step Hz back in for walking. There's certainly enough data for the models now, and phones are newer and smarter, so there's less risk of false zeroes, and more data to cope gracefully with those cases now too.

5. Look at deepening / broadening the composite classifiers tree. Could consider clustering all "steps" based types in a subtree (walking, running, cycling, horse riding, maybe skateboarding, others). Consider clustering the road bound types (car, bus, tram?). And there's the open question of whether a third depth would make sense in some cases, for when there's too many types to be sensibly clustered in a single classifier.

• Need to flesh out the storage model for user custom types. And for custom types that have the potential to being upgraded to shared types. Things like naming specific train lines, naming specific roads, etc. These might initially come in as custom user types, but there needs to be a privacy conscious, opt-in path to upgrading these to shared types, if it makes sense to do so.

I'll come back to this and add more thoughts over the next week or two. There's a bunch more details that have already crossed my mind, but I'm not remembering them right now...

Issues with classifier trees and classifier model loading

When a classifier has completely empty models, it's classifying everything as pure zero, instead of deferring to its parent classifier. Which results in stationary being the top match, by being at the top of the list of zero results.

This used to work perfectly. So it'll be a case of me having tripped over a cable somewhere in the score calculation code. Should be an easy fix.

Fix underground train recording

Before I open sourced LocoKit, Arc App had a dirty hack that made the underground train recording about as perfect as possible. It consistently got better results than Moves or anything else.

Then when I was open sourcing LocoKit, I saw that dirty hack, and thought "eh? what's this shit? what does that even do? that's not staying", and I ripped it out.

Oops.

So yeah, I need to spend a day on this. That's probably all it'll take. Find the old dirty hack in the private repo's history, put it back into LocoKit, then go spend a few hours on underground trains, making sure it's all good again.

Open source Arc App's GPX export functionality

Arc App has internal code for translating sets of LocomotionSamples and TimelineItems into GPX documents. There's no reason why that can't be part of LocoKit, instead of hidden away in Arc App.

It could probably be done nicely on TimelineSegment. Fetch a segment for a date range, then have a method on the segment that returns a GPX document. Although I think Arc App's code is going it the other way around, in that a GPX document can be initialised by passing in a TimelineSegment. There's pros and cons either way, I guess. Both are legit.

Okay so this is what I'm seeing in Arc App's GPX class at the moment:

class GPX {

    let timelineItems: [TimelineItem]

    init(path: ArcPath) {
        self.timelineItems = [path]
    }
    
    init?(timelineItems: [TimelineItem]) {
        if timelineItems.isEmpty {
            return nil
        }
        
        self.timelineItems = timelineItems
    }

    ...

}

So it looks like it's also happy to produce GPX for a single path, which makes sense.

Anyway, this should be trivial to do. Just a matter of finding some spare time to get it done!

can I use it in OC project？

I try use it in my OC project，but I can't found LocomotionSample method in Class LocomotionManager. I try to add @objc in this method ,but tips 'Method cannot be marked @objc because its result type cannot be represented in Objective-C' when I Compile，pls help me，thx，very much！！！

[Question] iOS 12

👋 Hey Matt, anything else we should be cautious of when updating to Swift 4.2? Saw this thread #20 and was successfully able to migrate to Swift 4.2.

iOS 12
Unfortunately with our existing build on the latest released version of LocoKit, we're experiencing weird/inaccurate location data on iOS 12. I saw this comment, looks like you were/are experiencing similar issues #29 (comment)

I'm seeing much higher rates of background fails in iOS 12 (in Arc's analytics, and on my test devices). And much higher rates of nonsense location data, with some devices "sleepwalking" on almost a daily basis, several kilometres away from where they really are.

Are you still seeing issues with LocoKit on iOS 12? Did Apple end up fixing some of their issues with the patch release? Anything I can potentially help with? (don't know how much time I have)

For what it's worth, I'm also experiencing quite a few crashes in the background on TimelineStore.save(). The app crashes after some time in the background. Anything potentially wrong with my setup or are you noticing the same thing? Any help is much appreciated, thanks.

0  libswiftCore.dylib             0x10143e480 _stdlib_destroyTLS + 207848
1  libswiftCore.dylib             0x101493f2c swift_unexpectedError + 280
2  LocoKit                        0x101064b10 TimelineStore.save() (TimelineStore.swift:325)
3  LocoKit                        0x101066858 closure #1 in TimelineStore.process(changes:) (TimelineStore.swift:348)
4  LocoKit                        0x100ff2a68 thunk for @escaping @callee_guaranteed () -> () (<compiler-generated>)
5  libdispatch.dylib              0x1db5c36c8 _dispatch_call_block_and_release + 24
6  libdispatch.dylib              0x1db5c4484 _dispatch_client_callout + 16
7  libdispatch.dylib              0x1db56bc18 _dispatch_lane_serial_drain$VARIANT$mp + 592
8  libdispatch.dylib              0x1db56c760 _dispatch_lane_invoke$VARIANT$mp + 432
9  libdispatch.dylib              0x1db574f00 _dispatch_workloop_worker_thread + 600
10 libsystem_pthread.dylib        0x1db7a60f0 _pthread_wqthread + 312
11 libsystem_pthread.dylib        0x1db7a8d00 start_wqthread + 4

Getting Compile Errors: Generic parameter 'ML' is not used in function signature

Looking to try it out but running into some problems on the demo app.

I ran pod install before building. Running into some compile problems:

I also tried downloading the app, but after setting location preference to only when in use, the app froze.

iPhone 7 iOS11 Beta
XCode 9.0 Beta 4

modeActivityType should take duration into account

Otherwise it’s kinda wrong, and sometimes mega wrong.

Different activity types are sampled at different rates (ie walking, running, cycling sampled at higher frequency, due to Apple Health workout requirements). So a mode based on sample count alone can easily pick the wrong type.

Throttle network requests for the same location.

I've played a little with SDK (latest version, master branch).
I'm testing it on my project which has NetworkLogger. (I use it to inspect network request, log specific info)
What I've found strange that my application using SDK executes tones of API requests for the same coordinates. I tried to change my location just by walking at the office.
So I don't have significant location changes, just stationary.
Also I use default accuracy options (30 meters).
It might be better to add some throttling mechanism in order to reduce number of network requests for the same location.

Link for screenshot

iOS version: 11.4.1
Device: iPhone 6+
Xcode: 9.4.1
GPS/Wifi/CoreMotion is Enabled.

Driving mode can't be detected on iPhone 6 (iOS 11)

Tried on motor-bike

Log says, .movingStateChanged (moving or stationary or visit)
--> when moving, it's always on 'walking' mode

If you please make this issue clear to me. Is this a Device issue or iOS version issue something related to your side?

NB: I've used this commit for Demo app installation, https://github.com/sobri909/LocoKit/commit/f65b00eeb2d17dfb5d52a636ad732a2781d1d2a0

why notice “Unable to find a pod with name, author, summary, or description matching `ArcKit`” when I pod search ArcKit

hello，i clone code About ‘ArcKit’ ，but couldn't run ，tips can‘t find ’ArcKit‘ ，try ’pod install‘ tips ’Unable to find a specification for ArcKit (= 5.0.0.pre)‘. how i can fix it, thx谢谢！！

Separate low level / high level recording controllers in the Demo App

The current Demo App mostly uses the high level PersistentTimelineManager APIs, using the persistent (ie SQL backed) store. It doesn't have clear examples of low level recording, and mushes together the example handling of persistent and non persistent stores.

Its example code is capable of a one line change, to replace its PersistentTimelineStore with a plain TimelineStore, but that double duty hurts the example code's readability. And a simple copy and paste of the controller into a new project will result in redundant, never used code.

The Demo App should instead contain separate controllers that demonstrate the different usage strategies, for different requirements.

Usage strategies

1. LocomotionManager as a replacement for LocationManager
2. TimelineManager as a replacement for CLVisit monitoring
3. PersistentTimelineManager for a database backed timeline recorder with all the goodies

It would be nice to also have some howto guides written up on each of these strategies, explaining them in more detail.

Dropping support for non-persistent TimelineStore

The main reason why LocoKit 6.0.0 hasn't yet gone final is because some changes along the way during development broke non persistent timeline stores (ie when TimelineStore is used instead of PersistentTimelineStore).

It's been a few months, and I still haven't had a spare moment to fix those regressions. And I've just now started work on a task (that's initially being tracked in the private repo, due to overlapping with some privacy considerations) that will further worsen the situation for non persistent stores.

I dearly want to keep the non persistent stores. But time constraints and limited resources are making it impractical to cover that much breadth and quasi-duplication of functionality. So something has to give.

What that means (probably) is that the LocalStore optional subspec will get renamed to Timelines, and all timeline recording functionality (eg Visits, Paths, TimelineStore, TimelineProcessor, etc) will move into that subspec, and will require the persistent SQL database, thus the GRDB dependency.

It also means that ActivityTypeClassifier and friends will also move into the Timelines subspec and require the SQL store. (The reason for that being that the current task I'm working on is a migration of activity type ML models from in-memory cache to persistent SQL storage. Semi related to #34).

The main LocoKit spec will still provide LocomotionManager and its associated helpers, for recording of Kalman filtered and dynamically smoothed LocomotionSamples, as well as the existing low level, real time moving/stationary state detection. So that low level functionality will continue to be available without needing an SQL store. Thus LocomotionManager will still be easily available as a "CLLocationManager on steroids", for producing cleansed, sanitised location and locomotion samples, with real time moving/stationary state detection.

The new Timelines subspec will then provide TimelineRecorder, TimelineStore, etc, for producing high level Visits and Paths, the activity types ML classifier, and the rest of the high level goodies, while depending on GRDB for its SQLite based persistent store.

Todos

• Reshuffle the classes
• Document the changes

Accuracy question

Hello,

i'm planning to use this framework in an exercise tracking application.
I have tried draw the exercise track with the filteredPoints and the locomotionSample.location

The locomotionSample definitely smoother, but i got a strange wave - i!m walking on the streets but the line is not correct - see the attached image.

Can you give any advice to get more accurate results?
I'm useing an iphone 7 plus with mobile data connection.
I dont enable the coreMotion, maybe that can help (i dont know how)

BAD_ACCESS crash in MLClassifierManager

Since Arc App v2.0 there's been a crash in MLClassifierManager. Or two crashes, really. One happening in canClassify() and one in classify().

They look like threading race conditions, but I can't see any other threads getting dirty with the same vars at the same time. And the crash report symbols are a bit nonsense, because I don't know why.

Crashed: TimelineSegment
0  libswiftCore.dylib             0x106244ef4 _swift_release_dealloc + 16
1  LocoKit                        0x1054b3ccc _T07LocoKit19MLClassifierManagerPAAE11canClassifySbSC22CLLocationCoordinate2DVSgF + 792
2  LocoKit                        0x1054b3ccc _T07LocoKit19MLClassifierManagerPAAE11canClassifySbSC22CLLocationCoordinate2DVSgF + 792
3  LocoKit                        0x1054b65ac _T07LocoKit19MLClassifierManagerPAAE8classify0aB4Core17ClassifierResultsVSgAE24ActivityTypeClassifiable_p_Sb8filteredtFTf4gnn_n + 316
4  LocoKit                        0x1054b3de8 _T07LocoKit19MLClassifierManagerPAAE8classify0aB4Core17ClassifierResultsVSgAE24ActivityTypeClassifiable_p_Sb8filteredtF + 32
5  LocoKit                        0x105530d60 _T07LocoKit15TimelineSegmentC17reclassifySamples33_23F73C3CE65E984CFEE46CE5D6809471LLyyF + 1648
6  LocoKit                        0x1055303f8 _T07LocoKit15TimelineSegmentC6update33_23F73C3CE65E984CFEE46CE5D6809471LLyyFyycfU_ + 308
7  LocoKit                        0x10549ae80 _T0Ieg_IeyB_TR + 36
8  libdispatch.dylib              0x184724aa0 _dispatch_call_block_and_release + 24
9  libdispatch.dylib              0x184724a60 _dispatch_client_callout + 16
10 libdispatch.dylib              0x1847631d4 _dispatch_queue_serial_drain$VARIANT$armv81 + 568
11 libdispatch.dylib              0x184763af8 _dispatch_queue_invoke$VARIANT$armv81 + 328
12 libdispatch.dylib              0x18476449c _dispatch_root_queue_drain_deferred_wlh$VARIANT$armv81 + 332
13 libdispatch.dylib              0x18476c46c _dispatch_workloop_worker_thread$VARIANT$armv81 + 612
14 libsystem_pthread.dylib        0x184a57e70 _pthread_wqthread + 860
15 libsystem_pthread.dylib        0x184a57b08 start_wqthread + 4

That's a typical example. There's also the other one in classify(), but meh, I'm 99% sure they're basically both the same crash, just tripping up at slightly different steps.

Arc App v1.9 didn't have this crash, even though it used essentially the same classifiers code. But the v2 series hammers the classifiers much more aggressively, reclassifying samples at various stages, instead of treating the first result as authoritative for life. So it's possible that the bug is old, and simply wasn't triggered in v1.9.

I'm throwing in some fiddlings with the mutexes/locks, to see if that will clear it up. With the stack traces being a bit nonsense it's largely guesswork. Hopefully the mutex fiddles will do the job...

ArcKit doesn't build in XCode 8.3

I did a pod install for ArcKit which was successful, but when I build, I get the error:

“Swift Language Version” (SWIFT_VERSION) is required to be configured correctly for targets which use Swift. Use the [Edit > Convert > To Current Swift Syntax…] menu to choose a Swift version or use the Build Settings editor to configure the build setting directly.

I did this to upgrade to Swift 3.0, but it still fails to build. I also changed the build target to be iOS 10.3 instead of iOS 11, but still doesn't build. Is there anyway to fix this?

Thanks.

Defer locations to max of one per second

I've had a report of Arc App using excessive battery when running alongside Google Maps when Google Maps is providing directions.

My suspicion is that Google Maps is requesting the highest level of location accuracy, which will result in Arc / LocoKit also receiving that level of accuracy, and also that frequency of location updates.

Which means instead of location update frequency maxing out at about 1 Hz, it might be going as high as 2 Hz. Which is completely unnecessary for LocoKit, and will be doubling the energy cost of the filtering layers.

So yeah, idea is to either use deferred location updates, to set a maximum sampling frequency, or to simply discard locations that arrive too soon after the previous.

TimelineRecorder / TimelineSegment

This is an issue to track development on the new managers / stores architecture for the next major release.

db -> store -> recorder / segment / processor

public TimelineRecorder (new)

TimelineRecorder will take over the jobs of a) communicating with LocomotionManager, and b) creating new timeline items.

The Recorder does not do any Item processing. It records new Samples and new TimelineItems, using the existing logic for whether to continue the current Item or create a new Item (ie on stationary/moving state changes, and activityType changes).

This division of labour keeps the Recorder simple and focused, and makes it easier to separate the Item processing loop from the recording loop. Which should hopefully allow for more battery efficient recording, by way of delaying Item processing until a point in time where it becomes necessary (eg sleep mode is about to start; some UI wants to show processed items to the user; etc).

public TimelineSegment (new)

TimelineSegment adds functionality for fetching of TimelineItems within specified date ranges. It will also internally manage the processing of these Items (by making use of TimelineProcessor) so that the resulting Items set is coherent and human presentable (as compared to the raw unprocessed Items produced by Recorder).

public TimelineProcessor (new)

TimelineProcessor takes over the internal job of processing (merging, edge cleansing, safe deleting, etc) sequential Items.

TimelineManager

TimelineManager will be removed. The new classes take over all of its responsibilities, so it will be made redundant.

Misc todos

• Make these classes final, no subclassing (only stores should be custom)
• Only one Recorder per Store
• Can have multiple overlapping Segments
• Segments are created by giving a date range or SQL query
• Should probably move Arc App's edge healing to LocoKit
• Should probably move Arc App's data gap insertion to LocoKit
• Fix the mystery broken item edges that aren't broken at heal time
• Persistent store needs to add a data gap at startup

Todos for the plain (non persistent) TimelineManager

Non persistent stores are no longer supported 😞

• Need a replacement for finalisedTimelineItem
• ~Non persistent store needs to know which items to retain for active item processing~~

Testing todos

• Stress test in the Demo App
• Road test in a public Arc App release

Docs todos

• Write the API docs
• Write up some example code snippets

I think LGPL is conflicting with AppStore rules

https://forums.developer.apple.com/thread/27709

So an LGPL library should not be used (legally) on AppStore

And because app in AppStore is not allowed to be used dynamic link, the app with an LGPL lib should be considered as GPL.

What do you think?

Make guesstimates for timeline item calorie consumption values

This is the second most often requested feature for Arc App, after the Moves data importer.

I was just about to add it to the tasks list in the Arc App private repo when I realised there's no reason why it can't be done in LocoKit directly.

This also relates to #24. That task is for importing this data from HealthKit. Possibly the same TimelineItem property (item.activeEnergyBurned) should be used for this one. Although I suspect it might make sense to store it in a separate property and database field, so that it can be distinguished from more accurate calorie burn data sourced from wearable devices.

Misc Notes

• "Mifflin St Jeor Equation"
• https://en.wikipedia.org/wiki/Basal_metabolic_rate
• https://sites.google.com/site/compendiumofphysicalactivities/

LocoKit doesn't build on Xcode 9.3

I have many "Swift Compiler Error" in my project after installing it with CocoaPods (57).

I can't even build the LocoKit Demo App because it also gives me all of these errors.

This is the error I get when adding it to my project.

• 'collations' is inaccessible due to 'private' protection level
  (and many others of the same type)

These are the errors I get trying to build the LocoKit Demo app (20):

• Type 'ComplexArray' does not conform to protocol 'RangeReplaceableCollection'
• Ambiguous use of 'count'
• Type 'ValueArraySlice' does not conform to protocol 'Collection'

What am I doing wrong?

Import MovesApp data

Moves just announced it's shutting down service all-together– you're probably about to see an influx of users. Any luck so far on perhaps importing data? I've got over 4 years worth of data I'd love to see included.

Generic Parameter 'C' could not be inferred

let locationT = CLLocationCoordinate2DMake(CLLocationDegrees(34.531), CLLocationDegrees(-122.02358845))
let classifier = ActivityTypeClassifier(coordinate: locationT)

On the second line Xcode is throwing the following compilation error: Generic parameter 'C' could not be inferred

Any idea how to solve this?

Edit: I have set my API key as well, though I don't expect that to throw an error at compilation.

Gpx month export sometimes doesn't include movement type (Arc app)

I have one day fully corrected without any 'low confidence' errors and it features many car trips.
When I exported it as only one-day GPX everything was fine, but full-month GPX file lacks many of type attributes (<type>car</type>).

Checked that with a few months and every time it looks the same.

EDIT: After a few checks I'm pretty confident it only happens with the 'car' transport type

There is not LocoKit in Pods on `pod install`

Cannot build the project because LocoKit Pod does not contain the lib just a Readme.md file.

Open source Arc App's HealthKit hooks

Arc App has various extensions in its TimelineItem subclasses for fetching data from HealthKit. There's no reason why these can't be moved to LocoKit.

They should be wrapped in HealthKit module availability tests. But otherwise they should mostly be easily moved upstream from Arc App's ArcPath and ArcVisit subclasses and into the main Path and Visit classes. (Or more likely directly into the base TimelineItem class, given that they currently live in an ArcTimelineItem protocol).

Would probably also need to move over Arc App's HealthKit wrapper class, for simplified data fetching.

HKQuantityTypes

• stepCount
• activeEnergyBurned
• heartRate (average value for the timeline item, and max value for the timeline item)

HKObjectTypes

• sleepAnalysis
• workoutType

It should also be easily extensible to support fetching of more HK types in custom subclasses, etc.

Record some test databases, and build tests for sample processing

Quoting myself from #20:

So far I've found automated testing to be effectively impossible for the core "ActivityBrain" engine, due to the requirement of a steady flow of CoreLocation and CoreMotion data inputs, where the timing of the inputs is significant. If it's not receiving inputs with effectively real world timings and values, there's very little point in testing it at all. The tests simply wouldn't be useful.

But what can be tested is the results of the timeline processing engine(s) (TimelineProcessor et al). So the plan is to record one or more sessions, with several hours of travel + stationary data, then keep those as "test databases". That will allow processing to be repeatedly run over the same data, and tested for desirable results.

Todos

• Ability to export a recorded time range from the Demo App
• Ability to store the raw CLLocation and CoreMotion data in the db
• Convenience methods for reprocessing a provided database

XCode 10.1 & Swift 4.2 can't Build

I already try
pod 'LocoKit'
which is build with Swift 4.1 and version is 5.2.0, this one can't build
Then I try
pod 'LocoKit', :git => 'https://github.com/sobri909/LocoKit.git', :branch => 'develop'
pod 'LocoKitCore', :git => 'https://github.com/sobri909/LocoKit.git', :branch => 'develop'
which is Swift 4.2 and version is 7.0.0, but still got 3 errors:
Pods/LocoKit/LocoKit/Base/LocomotionSample.swift:27:30: Use of undeclared type 'ActivityTypeTrainable'
Pods/LocoKit/LocoKit/Base/LocomotionSample.swift:113:35: Use of undeclared type 'ClassifierResults'
Pods/LocoKit/LocoKit/Base/LocomotionSample.swift:145:9: Use of unresolved identifier 'CLPlacemarkCache'

Any solution? I find related issue like #20 but still no help.

Development branches consolidated back into "develop"

Note that I have merged the sub-develop branches (timelines and markov) back up into develop, and will shortly be deleting those branches.

So anyone pointing their Podfile at timelines should update their Podfile to point to develop again.

I will be looking to merge back to master soon too, but that will require a bit more spare time than I have right now, due to needing to document at least the main migration requirements! So until then, develop can be considered the authoritative branch again, which very closely tracks Arc App releases.

Use 1 SD instead of 2 SD when edge cleansing timeline items

Pre-LocoKit Arc App used 1 SD for all visit radius tasks, both display and processing. LocoKit has instead always used 2 SD.

2 SD is more "correct", but it means that visits look fatter on the map, and makes the edge cleansing versus paths much more hungry.

The fatter presentation on the map is probably fine. It does show a more true radius for the visit. But the greedy edge cleansing can be too much, especially when dealing with visits to places that are only a short distance apart.

So yeah, I want to try going back to 1 SD for edge cleansing.