Example for persistent store

[sobri909]

Example for persistent store

[sobri909]

Ok, let's figure this out 😄

Sim versus Device

Step one is to avoid using the simulator as much as possible. For location and motion data it provides extremely misleading or completely absent data.

For example all of its CLLocations have "perfect" accuracy (ie location.horizontalAccuracy == 0), which means LocoKit's Kalman filters can't do their thing. The sim also doesn't provide any Core Motion data, so it's impossible (or extremely difficult) to test anything with activity type detection.

Your best bet is to always test on device. Even if your test device is just sitting flat on your desk, it'll still be giving real world location and motion data, which LocoKit can turn into workable LocomotionSamples.

As a general rule of thumb (not just LocoKit related), you should only fall back to the sim when you need to test a device format that you don't have. There's a lot of other ways in which it subtly lies or can lead you astray, and testing directly on device is just as fast, if not faster these days anyway.

The Persistent Store Database

There's no setup required for the database. If it doesn't exist, the store will create it as soon as it's needed. You'll find it in the app bundle as LocoKit.sqlite in the Application Support dir.

I see in your code above that you're overriding the store.dbUrl property. That's not something I anticipated, and may be causing problems. Although you're doing it before anything else is done on the store, there's still a risk of the database pool pointing at the wrong database file. I'd recommend avoiding changing the dbUrl unless absolutely necessary.

It might be worth having a poke around in the app bundle and see what's in the SQLite file directly, with an SQLite editor of some sort. (I use Base, but there's several others for macOS).

You can get the app bundle from Window -> Devices and Simulators, then find the device and the app install in the list, then tap on the cog icon and use "Download Container".

Misc Code Notes

There's a lot of code you've got there that you probably won't need. A lot of it is just stuff for demonstration in the demo app, and you can probably throw it away.

You can definitely throw away the var itemsToShow property, and instead use persistentItemsToShow directly. The itemsToShow property is for users who're using plain TimelineManager instead of PersistentTimelineManager.

A lot of the when() observer blocks will likely be unnecessary for you. But it might be helpful to keep them around for a while, until you're comfortable with what's going on. They're helpful for debug output, to see the various state changes as they happen.

Fetching TimelineItems / Samples

The persistentItemsToShow property gives an example of how to fetch a series of timeline items. But you can use any SQL query you like in there. Most of the table's fields are indexed, or at least all of the ones that could conceivably be used in queries.

For example if you want to get all timeline items between the start of today and now, you might do this:

let date = Date() // some specific day
let items = store.items(
        where: "deleted = 0 AND endDate > ? AND startDate < ? ORDER BY endDate",
        arguments: [date.startOfDay, date.endOfDay])

You can then iterate through all the items and samples in the results:

for item in items {
    print("ITEM: \(item)")
    for sample in item.samples { 
        print("SAMPLE: \(sample)")   
    }
}

You can also construct more complex queries, like for fetching all timeline items that overlap a certain geographic region. Or all samples of a specific activity type (eg all "car" samples). Or all timeline items that contain samples over a certain speed (eg paths containing fast driving).

But step one will be making sure you're reliably getting recorded data and can retrieve it from the db 😊

[sobri909]

I would also change these lines

var timeline: TimelineManager = PersistentTimelineManager()
var store: PersistentTimelineStore = PersistentTimelineStore()

to just be this:

var timeline = PersistentTimelineManager()
var store = PersistentTimelineStore()

That will allow you to treat the store and timeline as the correct types without having to typecast them, so then you can remove this line:

if let timeline = timeline as? PersistentTimelineManager {

and also can remove this line:

guard let timeline = timeline as? PersistentTimelineManager else { return [] }

because they're already assured to be persistent managers / stores, so you won't need to check whether they're the persistent types or not.

The Demo App is written in such a way that the same controller can have its store/manager either be persistent or non persistent. But that's not something you'll need in your code.

[dannyfreed]

This is super helpful, thank you so much!! Going to give it a go today.

🙌

[joediv]

👋 having trouble with the LocomotionSample data. The TimelineItem duration can exceed well over 60 seconds (unless duration is not in seconds?). Anywhere from 900 - 38,000 as the duration. Any tips for troubleshooting? I'm seeing data in SQLite for different types of motion but everything is grouped together the past few days.

Any ideas to debug invalid paths/visits in the logs? Seeing this is the logs when I try to print recent timeline items. Is this normal? Is that temporary until the TimelineItems are merged?

.newTimelineItem (PersistentVisit)
[invalid path, invalid visit, invalid path]

Outside of starting up the PersistentTimelineManager on app launch, is there anything we need to do in the setup?

Thanks for the help in advance. Much appreciated

[sobri909]

Hi @jdivittorio3 😄

Yeah, TimelineItem durations are in seconds. (All durations are TimeInterval type).

Each TimelineItem is a high level grouping of samples, representing either a Visit or a Path, depending on whether the user was stationary or travelling between places. So the durations can be as brief as. few seconds and as long as days (eg if the user stays at home for several days).

Inside each TimelineItem there is an time ordered array of LocomotionSample samples. These are found in timelineItem.samples. So the first sample in that array is the sample taken when the timeline item began, and the last sample marks the end of the timeline item.

LocomotionSamples typically represent between 6 seconds and 30 seconds. If location data accuracy is high, new samples will be produced about every 6 seconds. But if location data accuracy is low, samples can be produced less frequently, due to iOS updating the location less frequently.

(The maximum frequency is configurable, with TimelineManager.samplesPerMinute)

So for something like a Path timeline item, for example a few minutes walk between places, the Path object itself will have an activityType of .walking, but there are also all the individual samples that make up that path, some of which might not be .walking. For example if the user walks for a minute, pauses for a few seconds, then starts walking again, there might be a .stationary sample somewhere half way through the array.

Hope that helps!

[sobri909]

Oh, I should also mention ItemSegment.

Every TimelineItem (Path or Visit) has its array of samples, but it also has an array of ItemSegments (timelineItem.segments). These are a convenience grouping of the samples inside the timeline item, grouped together by sequentially same activity types.

So for the previous example of a few minutes walking, with a few seconds pause in the middle, that TimelineItem (Path) would have three ItemSegments in its segments array. A walking segment, a stationary segment, then another walking segment.

So that's another convenience method for getting at the samples in the timeline item in an organised way.

ItemSegments always contain sequential samples of the same activityType. So taking another example, someone's drive to work in the morning: They walk a few seconds to their car, they drive their car, they stop at the lights on the way, they resume driving, they arrive at work, they walk out of their car for a few seconds then enter their office.

Because each of the pauses and walking segments within that item are too brief to justify being their own separate TimelineItems, they are instead kept as ItemSegments within the single Path item for the entire trip.

So the timelineItem.samples will contain all the samples taken for that trip, and timelineItem.segments will contain ItemSegments grouping those samples based on activityTypes. Thus the segments would be: walking, car, stationary, car, walking (probably with some more stationary segments thrown in, due to multiple stops at traffic lights).

You can see the threshold boundaries used for the decision of whether to merge segments into a single timeline item or whether to keep them separate in the Path class. For example Path. minimumKeeperDuration defaults to 60 seconds, so any moving segment shorter than 60 seconds will be merged into an adjacent timeline item instead of standing alone as its own timeline item.

There are similar threshold settings for Visits. For example Visit. minimumKeeperDuration defaults to 2 minutes.

[sobri909]

Aside: Those Path and Visit "keeper" thresholds can be configured to make LocoKit's timeline items processing engine create more or less separate items.

However I wouldn't recommend setting them to much lower than they are now, as the defaults are configured to create the most detailed possible timeline items without generating too much timeline noise, and without pushing too close to the limits of the hardware's accuracy.

For example it's desirable to merge too-brief or too-short Paths into adjacent timeline items because they have a high likelihood of being the result of low location data accuracy, and thus more likely to represent noise than something real.

The exception there is if your app is intended for outdoor recording in open spaces, where you can assume there to always be high GPS accuracy. In those cases it is possible to push the boundaries lower. LocoKit can happily identify the difference between short stops and short paths within a few seconds resolution. But you just don't want to push your luck with that while in built up city areas, or other areas where the location data is likely to be unreliable.

[joediv]

Awesome thanks for the quick response! I ended up going on vacation shortly after asking my questions and getting back to this now. I was able to get everything working as expected.

The last piece I'm curious is how to record location when the application is in a terminated state? Seems as though the Arc app does this. When the app is in the background, it works well. Also is it possible to send data to our servers when the application is terminated (seeing conflicting information out there). Thanks again for all the help, really like the project and is working well.

[sobri909]

The last piece I'm curious is how to record location when the application is in a terminated state?

Unfortunately that's impossible. iOS doesn't store location data for later retrieval, so apps need to be running to record it.

Arc App achieves all day and background recording by using LocoKit's "sleep mode", which is on by default in LocoKit, so your own app should already be making use of it.

https://www.bigpaua.com/locokit/docs/Classes/LocomotionManager.html#/Sleep%20Mode%20Settings

Basically the useLowPowerSleepModeWhileStationary setting (which is on by default) allows the manager to keep recording effectively continuously, 24/7, with optimal energy efficiency.

There are three general requirements for background location recording:

1. Your app has been granted "always" location permission
2. Your app has "Location updates" toggled on in Xcode's "Background Modes"
3. You called startRecording() while in the foreground

If the user doesn't grant "always" location permission, then it will still work, but the user will see an annoying blue bar at the top of the screen while your app is in the background.

If the user has granted you "always" permission, then you may also be able to call startRecording() in the background. But you would want to also start a "background task" with UIApplication.shared.beginBackgroundTask(), to ensure iOS doesn't suspend the app.

So the easiest option is to make sure you call startRecording() while in the foreground, and don't ever call stopRecording() while in the background. If your app is intending to do 24/7 location recording, then you should pretty much never call stopRecording() anyway. LocoKit is optimised to ensure that you can leave it running indefinitely.

One of my test devices (an iPhone 7) has an Arc App uptime of coming on two months, and typically lasts up to 30 hours on a single battery charge. So the app has been constantly running and recording without termination for ~60 days 😄

[joediv]

I misspoke in regards to app termination. I notice with the Arc app, if I force close, the app stops recording and that's expected. After significant location changes, Arc does start to record location again. When I relaunch the app after these changes, I am able to see the past location changes from my movement.

For right now, we're mimicking the demo app and are trying to get that recording to work when the app wakes back up for significant location changes. We've had the 3 requirements listed above but haven't seen updated tracking. Also added beginBackgroundTask and recalled startRecording() but no luck. Any tips? How does Arc handle this?

Here's what we're doing to key into the application lifecycle. Within our LocoKitManager, setup() will call startRecording() again. Could have missed something simple and am not as familiar with background modes.

    var backgroundTask: UIBackgroundTaskIdentifier = UIBackgroundTaskInvalid

    func applicationDidEnterBackground(_ application: UIApplication) {
        backgroundTask = UIApplication.shared.beginBackgroundTask { [weak self] in
            self?.endBackgroundTask()
        }
        LocoKitManager.shared.setup()
    }

    func endBackgroundTask() {
        UIApplication.shared.endBackgroundTask(backgroundTask)
        backgroundTask = UIBackgroundTaskInvalid
    }

    func application(_ application: UIApplication, performFetchWithCompletionHandler completionHandler: @escaping (UIBackgroundFetchResult) -> Void) {
        LocoKitManager.shared.setup()
    }

Thanks again for all the help. Very much appreciated 😃

[sobri909]

I actually just yesterday committed a change to the develop branch that adds the background task management to LocomotionManager.

It looks essentially identical to what you've got there. So your code looks correct to me!

One thing that was important, several iOS versions ago (maybe around iOS 8 or 9), was starting the location recording before starting the background task. But I'm not sure if that's still the case now. Background task management is very black box, and subtle changes can have unexpected and undocumented impacts, so anything's possible really.

Basically in Arc App, whenever startRecording() is called, a background task is created immediately after. So the changes I did yesterday in the develop branch are essentially the same. Any instance inside LocomotionManager where the internal LocationManager is started, the background task is also started (if it's not already started).

Also, be very wary of the iOS 12 betas. The current iOS 12 beta is extremely broken for background location recording. Background tasks are expiring when they shouldn't, and location accuracy isn't being achieved anywhere near to the desiredAccuracy settings. So there's no hope with iOS 12 yet.

Although the same thing happened last year with the iOS 11 betas. It wasn't until the 4th or 5th beta that background location recording (or location recording in general) came back to normal. So this year I'm basically ignoring the weirdness in the betas, on the assumption that Apple will fix it before the final release.

[sobri909]

Oh, I haven't given the changes in the develop branch long enough on test devices to be sure they're solid yet. I was mostly testing them on my iOS 12 device yesterday (which was a complete loss, due to iOS 12 beta 1 doing essentially everything wrong with background location in general).

Today I'll give them a hammering on one of my iOS 11 test devices, to make sure they're still good. But yeah, it's basically the same logic that Arc App is/was using, just moved into LocoKit instead.

It should be as simple as making sure you always start a background task after starting location recording. (Or before. But I'm quite superstitious about this stuff, given how undocumented and black box it is. So I still always start the background task after, even though it seems logically the wrong order.)

If I spot any weirdness on iOS 11 test devices today, I'll let you know. Maybe there's some other detail in Arc App that I'm missing. It's possible that there's some voodoo incantation required, that I never even realised I was doing in Arc App. So if that's the case, I should figure that out today.

[joediv]

Interesting. Thanks! I'll start recording before the background task to see if that makes a difference and test with some logs. I hear you with the black box. I've been trying to read into background task management but found the documentation lacks enough detail. Appreciate it!

[sobri909]

Oh, I just realised your code is starting the background task after entering background. You'll almost certainly want to do it before entering background. Even though you're in the foreground, you still want to start the background task at the same time as starting location recording. Basically have location recording always wrapped in a background task.

Although when the app gets launched by significant location change or CLVisit, it obviously will have to start location recording in the background, not foreground, so there's no choice about it. But that seems to be allowed by iOS, perhaps as a special case because the app was launched by a location change?

[joediv]

Whoops 🤦‍♂️. Good catch. I'm not sure 🤔

[sobri909]

@jdivittorio3 I just noticed another potential trap in your code above.

You're implementing performFetchWithCompletionHandler but not calling the completionHandler. It's likely that iOS will blacklist your app on devices that are doing that, due to not playing strictly by the rules. Which could result in the app not being given background time for other tasks.

This is my current performFetch in Arc App:

func application(_ application: UIApplication,
                 performFetchWithCompletionHandler completion: @escaping (UIBackgroundFetchResult) -> Void) {

    // debug logging
    if launchDate.age < 60 {
        Amplitude.instance().logEvent("Launched from background fetch")
        logToFile("performFetchWithCompletionHandler() AT LAUNCH TIME")
            
    } else {
        logToFile("performFetchWithCompletionHandler()")
    }

    // shouldn't be here if the user has turned off recording
    guard User.meInMainContext.recordingOn else {
        completion(.noData)
        return
    }

    LocomotionManager.highlander.startWakeup()

    delay(2) { completion(.newData) }
}

[sobri909]

Uh, the delay(2) before calling the completion handler is pure superstition. I've no idea if it's necessary or not.

And Arc App's use of background fetch is new (unshipped), and intended for handling LocoKit's new "deep sleep" mode, which allows the app to fully suspend and/or terminate. So it's serving a fairly experimental and unproven task.

But the general gist of it is correct. A background fetch needs to call its completion handler, otherwise iOS will consider it a breaking of the rules, and count it as a black mark against the app's future requests for background tasks / background fetch / silent push / etc.

[joediv]

Gave it a shot earlier and I must be missing something. Unfortunately still did not work.

I'm a little confused by this

And Arc App's use of background fetch is new (unshipped)

In the current version on the App Store, how does the Arc App take advantage of recording location after an app has been closed from the multitasking ui? I'm able to close the app, walk a few blocks, open the app back up and then able to see the path I took with Arc. Arc App is able to accomplish this without backgroundModes?

Where in the application lifecycle does the Arc App make sure recording still occurs? What functions are called there as well? The demo app doesn't handle this case and curious what is different in the Arc App. Trying to replicate the functionality in the current Arc App. Not sure if you'd be willing to give a sneak peak into the repo to look around so I don't have to keep asking questions 😄. Appreciate any help 🙇
(Note: terminology may be somewhat off )

[sobri909]

"Background fetch" is a separate background mode from "background location". The version of Arc App on the store now doesn't have "background fetch" enabled, it's only using "background location".

So v1.9.0 of Arc App gets woken up after being force quit by way of signification location updates. Those are one of the few (perhaps the only) ways in which an app can be relaunched automatically after the user has manually force quit the app.

CLVisit monitoring is also used as another fallback in the current public Arc App version. But that won't relaunch the app if the user manually quit it. As far as I know, it's only significant location changes that can do it. Although possibly silent push can too? Not sure.

Basically as long as you've got your location manager monitoring significant location changes, the app will get relaunched automatically if the user force quits it, provided the user moves far enough to trigger a significant location change.

In the develop branch of LocoKit I've got this in a couple of places:

// start the safety nets
locationManager.startMonitoringVisits()
locationManager.startMonitoringSignificantLocationChanges()

In Arc App I used to do that outside of LocoKit (and probably still do), as soon as location recording is started. You basically want to turn those on and never turn them off, unless the user asks that your app stop recording location. (So for example, in the develop branch of LocoKit, those two are turned off when stopRecording() is called).

But back to "background fetch" and "background location", the former isn't of much use in most cases, and Arc App hasn't used it until recently (in my unshipped beta code). The one you want is the "background location" background mode.

Background fetch is designed to let your app do periodic fetches of remote data, so that your app has up to date data to fill its UI when the user reopens the app. So for things like news apps, etc.

The new "deep sleep" feature in the next LocoKit release will use background fetch (hopefully). Although it's unproven as to whether it will work well enough to make deep sleep viable. So when deep sleep is requested, with a desired wakeup time, LocoKit will do this:

// turn on background fetches
onMain { UIApplication.shared.setMinimumBackgroundFetchInterval(deepSleepDuration) }

// request a wakeup call silent push
LocoKitService.requestWakeup(at: wakeupTime)

So in that case, LocoKit will be using background fetch to hopefully wake the app up from suspend/termination after a desired period of time.

And LocoKit will also be requesting a "wakeup call" silent push from LocoKitService (ie my server). So that's two separate ways that LocoKit will be hoping to be woken up again: background fetch, and silent push. If one fails, maybe the other will succeed.

Anyway, you don't need either background fetch or silent push right now. All you need is the "background location" background mode. And if you add in significant location change monitoring, and CLVisit monitoring, you'll have two safety nets that will hopefully relaunch the app after it's been force quit by the user.

The demo app doesn't handle this case and curious what is different in the Arc App.

I think the Demo App isn't starting significant location change monitoring or CLVisit monitoring. I didn't want the Demo App to unexpectedly keep going forever, if a developer forgot about it and left it recording 😉 So it's just letting itself die off and not resurrect, if it's force quit.

Not sure if you'd be willing to give a sneak peak into the repo to look around so I don't have to keep asking questions 😄.

Heh. Unfortunately it wouldn't necessarily be helpful! The current Arc App codebase is doing things a bit differently, due to the deep sleep mode experiments.

But yeah, anyway, significant location change monitoring is the one you want, if you want to recover from the app being force quit. I'm pretty sure nothing else is allowed to relaunch the app after force quit. (Except for maybe silent push, but I'm not sure about that).

[sobri909]

Actually, the full snippet from the develop branch's deep sleep method is a good example of using all four safety nets:

// turn on background fetches
onMain { UIApplication.shared.setMinimumBackgroundFetchInterval(deepSleepDuration) }

// request a wakeup call silent push
LocoKitService.requestWakeup(at: wakeupTime)

// start the safety nets
locationManager.startMonitoringVisits()
locationManager.startMonitoringSignificantLocationChanges()

The first two are the two that you don't need or want right now. You don't need background fetch or silent push.

The last two are the ones you'll need right now. You want CLVisit monitoring and sig loc change monitoring.

[joediv]

Ok, made some progress based on your comments 🙌 . Able to see the application wake up from significant location change. I did not set useDeepSleepModeWhileStationary = true and therefore startMonitoringVisits() & startMonitoringSignificantLocationChanges() were never called 😭 🤦‍♂️ .

I'm calling setupAndStartRecording() below within the appDelegate in didFinishLaunchingWithOptions. That seem right? Anything I might be missing? What is the difference between this setup and Arc? I'm calling the monitoring functions on the location manager directly to be sure they continue to get called. From my understanding, calling the monitoring functions on the locationManager is required. Also seems like you have early returns in startWakeup() & startRecording() so it's ok to call multiple times as well.

let timeline = PersistentTimelineManager()

func setupAndStartRecording() {
    timeline.activityTypeClassifySamples = false

    LocomotionManager.highlander.useDeepSleepModeWhileStationary = true
    LocomotionManager.highlander.locationManager.allowsBackgroundLocationUpdates = true
    
    timeline.bootstrapActiveItems()

    LocomotionManager.highlander.startWakeup()
    timeline.startRecording()

    LocomotionManager.highlander.locationManager.startMonitoringVisits()
    LocomotionManager.highlander.locationManager.startMonitoringSignificantLocationChanges()
}

I'm now able to see a path for a significant location change. Earlier today right before leaving work, I force closed the app. I can see my path walking home from work today all from the background location monitoring. But, I took another walk later for a couple blocks and the app did not record those events. I'm not seeing the shorter walk as a TimelineItem but I do see a very accurate timeline in Arc. Is there anything that you can think of? Anything the arc app might be doing differently? Anything I may have forgotten to implement? A certain setting I need? (most are default).

Side note: I also tried with two different apps on the device at once. One with activityTypeClassifySamples as true and the other as false. Same result for both (didn't expect there to be a change since it's for the activity type classification).

Really appreciate the help. Lifesaver 😄

[sobri909]

Oh, you don't want deep sleep mode! That's experimental, and results in the app being fully suspended (and possibly terminated) once stationary is detected.

What you want is just normal LocoKit sleep mode, which is on by default. You'll want to leave deep sleep turned off, and shouldn't need to change any LocoKit settings from their defaults.

What you do need however, if you want the app to be relaunched after the user force quits it, is significant location change monitoring. (And I always throw in CLVisit monitoring as well, even though sig loc monitoring makes it redundant.)

You also won't need allowsBackgroundLocationUpdates = true, because LocomotionManager is doing that already, inside startRecording().

This is how I'd write your startup method:

let timeline = PersistentTimelineManager()

func setupAndStartRecording() {
    timeline.activityTypeClassifySamples = false
    timeline.bootstrapActiveItems()
    timeline.startRecording()

    startBackgroundTask() 

    let loco = LocomotionManager.highlander
    loco.locationManager.startMonitoringVisits()
    loco.locationManager.startMonitoringSignificantLocationChanges()
}

The reason why I do startWakeup() in my current Arc App code is because I'm experimenting with the new deep sleep mode for the next release. Deep sleep will require some extra handling in the AppDelegate, separate from the standard startup code, so that's where the startWakeup() comes in. You won't need to call startWakeup() yourself.

This is what Arc App's startup method looks like (with some irrelevant cruft removed):

class Stalker {

    static let highlander = Stalker()

    ... 

    func startStalking() {
        guard User.meInMainContext.recordingOn else { return }

        // fire up the gimp
        recorder.startRecording()
        
        // start the safety nets
        loco.locationManager.startMonitoringVisits()
        loco.locationManager.startMonitoringSignificantLocationChanges()
    }

    ...

}

And my AppDelegate's launch sequence (with lots of irrelevant cruft removed):

func application(_ application: UIApplication, 
        didFinishLaunchingWithOptions options: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {

    ... 

    Stalker.highlander.startStalking()

    ...
}

The recorder mentioned there is the same as timeline in your code. (In LocoKit 6.0 the timeline recording and timeline processing has been split up into separate manager classes, where as in LocoKit 5 they're in the same class).

I'm also not starting a background task in my current code because that's handled internally in LocoKit 6.0. But you should start a background task in your code, if you want the app to be able to record after being launched in the background.

[sobri909]

I'll eventually write up an article explaining all these steps, and giving more example code. I've been meaning to for years, but never have enough spare time. Sigh.

The reason why this stuff isn't all done internally to LocomotionManager / TimelineManager is it's not safe to assume that SDK users want their apps to be relaunched in the background, or do background recording, etc.

For example LocomotionManager can be used standalone as a wrapper for LocationManager, for producing Kalman filtered location and motion data, without making use of sleep mode or extended background recording sessions.

Although it would be nice to have some common patterns like this wrapped up as methods on LocomotionManager. Perhaps something like this:

loco.attemptToRelaunchAfterTerminate = true

I'll have a think about how to wrap up some of this stuff like that. LocoKit 6.0 is taking over the job of managing background task creation, so that's one down.