denzilferreira/aware-client

AWARE's Android client. This application allows researchers to easily collect sensor/plugin data locally or remotely to an AWARE server instance.

aware-client's Issues

Aware plugins libraries in maven repository

Could you add Aware plugins libraries to maven repository? It would be really useful for people that want to build plugins based on existing ones.

And also other thing - could you remove from aware-core and plugins git repositories: .idea directory, local.properties, build directories and other that are dependent on programming environment?
This files aren't necessary for people that want to use source code, but they make updates really time consuming.

.

[bug] UI not updated after configuration URL entered

Hi,

While QR was crashing my app, I was entering the configuration URL manually. I eventually realized that the following would happen:

• Enter configuration URL
• Go look at options. You see old config options.
• Exit app by clicking back button repeatedly. (not just switching to home screen)
• Start app again. Look at config options, and the configuration retrieved from the server is now show.

I'm not sure if the same happens when configuring via QR code.

What do you think? We're not very familiar with the code yet so it may be easiest for you to find, if it's worth it.

Thanks,

• Richard

[Wishlist] Add support for sensing scheduling

Is there any way (or will there be in the future…) to schedule when data is logged? For instance, I would only like to log data between the hours of 4pm and 1am. Having certain options enabled really seems to drain the battery and use data necessarily, so it would be nice to be able to stop this when not required.

deterministic upload times, and unreliable uploads

Hi,

This is continued from a private email thread. I am putting it here for openness and recordkeeping.

I was trying to see why AWARE was not reliably uploading data. I eventually saw that it uploaded at fixed intervals of the hour. For example, if the upload frequency was 15 (min), it would upload on the [:00, :15, :30] of the hour. I can see the following possible issues:

• Due to a bug, it doesn't upload at the :00 of the hour
• The uploads are deterministic and the same for all clients, which is bad for scalability!
• I haven't tested yet, but there may be issues with frequencies longer than 60 minutes.

These can be solved with some code changes, and then adding a random offset from [0, freq-1]. Code link: https://github.com/denzilferreira/aware-client/blob/master/aware-core/src/main/java/com/aware/Aware.java#L399 (second copy right below)

However, even with the above, Android returns control to the application at non-deterministic times. This overall makes sense and is just how it works. However, the scheduler only runs the upload process if the process runs exactly on the minute. When the application is not getting many resources (as it normally shouldn't), this is not often enough and it doesn't upload reliably.

So... I would think that the best answer is to understand in incorporate this non-determinism. For something like uploading, we shouldn't try to force it to upload at fixed offsets, but have a scheduler mode that can say "no more often than every X minutes, do this task". Then, uploading can happen anytime the scheduler gets called, if it hasn't already in the last X minutes. This also nicely solves the fixed-offset problem, and then we don't have to worry about the schedule runner being non-deterministic. We can then run the schedule runner less often, which may encourage Android to give it a higher priority...

What do you think? We can at least fix the schedule to be non-deterministic. Do you have any suggestions for adding a frequency-based scheduler, too?

Thanks,
Richard

Link to study-specific privacy policy/information

There should be a link somewhere that links to the information/privacy policy of the current study, from within the app. If the study settings are locked (study_id != ""), then a pop-up dialog appears when you start the app. In my case it's relevant even if study_id="". The main core of the this issue is a link that would be described as "Information about the study and study privacy policy". Alos, since now a user can enter the app if they are in a study, there should be a way to get back to the study info.

Proposal: study_url variable, if undefined then no changes. I would implement this either as a menu item on the settings page, or in the top-right drop-down menu.

Note: this can be considered assigned to myself.

Google vision services not operational right now

https://developers.google.com/vision/announcement

This says that there is a bug in the API and it is disabled for new users (and only new users). I'm filing a issue here to keep track. The date on the announcement is 27 May 2016 so hope the "several weeks" they say is correct.

[wishlist] sensor thresholds and other options

Hi,

Previously, we were using Purple Robot. It had configuration options for some sensors, like accelerometer, which would only generate a new data row if the new value was more than some threshold away from the old value. For example, this was useful on accelerometer which would be generating 1GB of data per day without the threshold, but with it it was small. The data excluded was mostly zeros while the device was at rest. Several other probes had the same idea. The threshold was configurable as an option per-sensor, and there was usually a "0" option that would produce full data. This allowed higher frequency sampling when data was interesting while lower when it was not.

This was important since some phones had limited storage, and storing too much data severely impacted them. Also uploading so much data can have an effect on the subjects network usage, which we would then need to consider.

(on another note, where were options such as "high frequency sample for N seconds, every M minutes).

I could have my team study and make a list of these types of options that would be useful. We could even try to implement them. Do you think this is something that would be useful to you?

Thanks,

• Richard

.

[question] is MQTT the only way to send ESMs, and other ESM questions?

Hi,

I was investigating ESMs, to see if I can get it to work for our purposes. I understand some parts of it, but just to make sure I wanted to ask some questions to make sure I get it right... I'm sorry if this is a lot.

Is MQTT the only way to send ESM questions to a device? Is there any way to send them as part of the config file? And then, is each question asked immediately: is there any way to send a bunch of questions and have them asked at specified times throughout the day?

I get the idea that MQTT persistence would ensure that a message is received even if the device is off/disconnected at that time. Is that correct?

In the iOS client, I see that a lot of MQTT code is commented out. Is it currently working? If not, and if there's no other way to do ESM, can they be done in iOS ?

Thanks,

• Richard

[WEB API] The calendar does not allow for changing months

I have in my database data from year ago. There was a few months break in collecting data, and now we continue. However, the calendar on the website api.awareframework.com does not allow me to go back to previous months. Is it a bug or feature? :)

Check what esm_status 0 and 2 mean

Make sure we are consistent about how we use them

When phone starts, it requests bluetooth be turned on even if it is already on

Hi,

When I turn on my phone, it makes a notification saying 'an application want to turn on bluetooth, accept or deny'. It seems this is produced by the OS, not explicitly app-rendered (but I could be wrong here). When this happens, bluetooth is already turned on.

I don't remember this happening months ago, but I can't say for sure when it started. I think I do have the bluetooth sensor enabled.

Also, this is a separate issue, but the dialog doesn't say what app is requesting it or why. This is probably because it is a system dialog and it can't say anything else. Still, I think it's worth asking how the user experience could be improved, especially for long studies with only limited connection, so that people forget what is going on.

Thanks,

• Richard

Settings to control hashing

Hi,

I'd like to propose some settings that can adjust hashing of sensitive information. In summary:

• An option to set the hash algorithm
• An option to salt the values hashed
• Abstracting out the hashing to another function
• Option to specify different hash functions for different types of data

In detail:

Option to set the hash algorithm + separate function

• Right now, in android and iOS, I see sha1 hard-coded in different places. We only need to worry about reversals, but the differences in brute forcing things like phone numbers for them all is not that much.
• The abstracting out the hashing to another function is needed for other things, so may as well make it configurable.
• I would suggest options be at least md5, sha1, sha256. sha1 should be default for backwards compatibility. Maybe md5 could be excluded, but could be useful if researchers need to compare to other data sources.
• Controlled by variable hash_function which is the string of hash function names.

Option to salt the values hashed

• Add a new variable, hash_salt. It is a string defaulting to the null string "".
• This really should be done, there are websites which can reverse hashes of phone numbers, as students have found out. Salt doesn't protect against brute force, but at least our hashes should not be in rainbow tables!
• hashed_value = H(value || salt_string). This is string concatenation. I don't think any of the extra security of HMAC is necessary here, since the salt_string is public.

Option to specify hash functions for different types of data

• This means that, for example, you could specify a different hash function for phone numbers as compared to other things.
• The motivation for this is that we are doing a study, and we need to connect app-recorded phone numbers with numbers collected by surveys. Asking for whole phone numbers is too invasive, so we settled on collecting only the last six digits. We need to be able to uniformly hash this
• I would expect implementation would be something like adding a variable hash_function_phone, and have some way to use this if it is set, otherwise use the default hash_function. People with special needs could define a custom hash_function that has their policy.

By the way, Purple Robot use the function android.telephony.PhoneNumberUtils.formatNumber to standardize the number formatting. This was quite useful and important. We should consider this, but for the non-deprecated formatNumber functions, we need to specify a country code and I'm not sure what to use there.

What do you think?

• Richard

[wishlist] configuration mode to allow complete removal of all data

Can the sdcard AWARE folder be deleted when app uninstalled?

No, it can not. It’s intentional. Some participants decide to uninstall the app themselves. If we cleared that folder, the researcher would lose all the data.

In our use case, we could be deploying to hundreds of clients (aside: it will be installed and configured before subjects meet researchers physically). The study will be long (an academic year at least), and we'd expect some clients to drop out early, and we won't plan on meeting people after the study. And for our permissions to do the project, and also to minimize the support load, we would like to say "A subject can opt-out of the study at any time by uninstalling the app, when they do so their phone is restored to the state it was before the project started". In this case, data is collected via the webservice, and if data is not sent, then we don't get it. (aside: in this use case, the "delete data immediately after sending" option is very useful, since we also want to say that data is stored on the phone for the minimum time possible).

So, from our perspective, we could contribute a hidden option that would put the AWARE folder inside the Android/data folder, which appears to be removed on de-installation. This is also good because the data is hidden to make things as un-intrusive as possible. This should be a hidden option that is only available if you link to a server and the server sends it.

What do you think?

Thanks,

• Richard

[BUG] DatabaseHelper crashes app API levels < 19

It makes an attempt to create directory on type, Environment.DIRECTORY_DOCUMENTS:

FATAL EXCEPTION: main java.lang.NoSuchFieldError: android.os.Environment.DIRECTORY_DOCUMENTS at com.aware.utils.DatabaseHelper.<init>(DatabaseHelper.java:54)

I am using version 3.8.5. (EDIT: Fixed on latest version)

Uploads crashing (cursor window is full)

One of us had a problem with uploading data. The symptom: when data starts uploading, it stops with accelerometer. The notification forever, saying that upload is still in progress, and all the data uploads stop.

Here is relevant part from adb logcat. The timestamps corresponds to when the sync happens and the notification first appears.

<syncing all sensors above>
07-13 11:53:18.714 11909 13182 D AWARE   : Synching: sensor_accelerometer
07-13 11:53:19.010 11909 13182 D AWARE   : Synching data only over Wi-Fi and internet is available, let's sync!
07-13 11:53:19.011 11909 13182 D AWARE   : Synching: accelerometer
07-13 11:53:19.251 11909 13182 W CursorWindow: Window is full: requested allocation 96 bytes, free space 74 bytes, window size 2097152 bytes

I don't know enough to immediately know what the problem or solution is. The phone is a new Nexus 5X, checking memory on the phone after the fact shows 76% in use (2GB total).

Thanks,

• Richard

.

Condition for cleaning old data

Hi,

In WebserviceHelper.java, bottom of function onHandleIntent, lines 301, I see (effectively) the following condition:

if (Integer.parseInt(Aware.getSetting(getApplicationContext(), Aware_Preferences.FREQUENCY_CLEAN_OLD_DATA)) == 4
    && highFrequencySensors.contains(DATABASE_TABLE)) {

Does this effectively mean that only the high-frequency tables are cleared (as listed right above)?

Could there be an option to clear all data once it has been sent to the webservice? On a related note, in WebserviceHelper.java, is the POST response to /insert checked enough to ensure that data is only removed if the server verifies that it was successful? Would having the server return something that is verified be useful? (I guess with the aware-server, the latest row is retrieved from the server to provide this failsafe).

Thanks,

• Richard

[bug] force-stopping AWARE disables it as accessibility service

Hi,

I noticed that if I force-stop AWARE (though Android settings), then it is no longer enabled as an accessibility service. This is quite an issue, since from what I understand then it won't remain running in the background. There is no notice of this once you re-open the app. This is a big problem for us, since as studies get longer, the chances of this happing to some users increases.

Maybe one work-around is to make it produce a notification when the app is opened and services are not turned on? Beyond that, I guess this is just the way Android works. (As an aside, in the long-term, we would be interested in a mode that had a persistent notification and wasn't dependent on accessibility services. We would develop this if we needed it.)

Device: Nexus 5 running latest Google Android.

Any thoughts?

Thanks,
Richard

URI schema for linking

Hi,

I have a new user story. In this case, subjects will come and attend a meeting introducing the study. Then, they will link devices there. Some may just have the phone, and not a computer too. They will have their own accounts on the study server, which will display the instructions. If they log in to the study site on their phone, they can't scan the QR code with their own phone. I'd like to be able to make a button with a URI that gets handled by the app and does the linking.

• Does this already exist? (I couldn't find any intent filters in the manifests)
• What should the scheme be? aware: seems too generic, awareregister: or awarefw-register ?
• @denzilferreira, any suggestions about what class should handle the uri? should it go to one of the StudyJoin or StudyUtils classes as mentioned in another issue?
• @tetujin, any comments from the iOS side?

Thanks,

• Richard

Collect Application usage data using aware library in my own app

Hi,
I want to develop my own app using your library collecting application usage info from users. But I realized aware service should be turned on in accessibility as you suggested in this aware client. How could I achieve this by only using aware library? Thanks.

Need to make cert directories (?)

I'm running on my phone, and with the new certificate techniques I see this:

10-06 02:57:47.086 26943 27307 D AWARE   : Can not write certificate: /storage/emulated/0/Android/data/com.aware.phone/files/Documents/credentials/XXX.fi/server.crt
10-06 02:57:47.086 26943 27307 W System.err: java.io.FileNotFoundException: /storage/emulated/0/Android/data/com.aware.phone/files/Documents/credentials/XXX.fi/server.crt: open failed: ENOENT (No such file or directory)
10-06 02:57:47.087 26943 27307 W System.err:    at libcore.io.IoBridge.open(IoBridge.java:452)
10-06 02:57:47.087 26943 27307 W System.err:    at java.io.FileOutputStream.<init>(FileOutputStream.java:87)
10-06 02:57:47.087 26943 27307 W System.err:    at java.io.FileOutputStream.<init>(FileOutputStream.java:72)
10-06 02:57:47.087 26943 27307 W System.err:    at com.aware.utils.SSLManager.setCertificate(SSLManager.java:230)
10-06 02:57:47.087 26943 27307 W System.err:    at com.aware.utils.SSLManager.handleCrtParameters(SSLManager.java:201)
10-06 02:57:47.087 26943 27307 W System.err:    at com.aware.utils.SSLManager.handleUrl(SSLManager.java:81)
10-06 02:57:47.087 26943 27307 W System.err:    at com.aware.phone.ui.Aware_QRCode$StudyData.doInBackground(Aware_QRCode.java:292)
10-06 02:57:47.087 26943 27307 W System.err:    at com.aware.phone.ui.Aware_QRCode$StudyData.doInBackground(Aware_QRCode.java:264)
10-06 02:57:47.087 26943 27307 W System.err:    at android.os.AsyncTask$2.call(AsyncTask.java:295)
10-06 02:57:47.087 26943 27307 W System.err:    at java.util.concurrent.FutureTask.run(FutureTask.java:237)
10-06 02:57:47.087 26943 27307 W System.err:    at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:234)
10-06 02:57:47.087 26943 27307 W System.err:    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
10-06 02:57:47.087 26943 27307 W System.err:    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
10-06 02:57:47.087 26943 27307 W System.err:    at java.lang.Thread.run(Thread.java:818)
10-06 02:57:47.087 26943 27307 W System.err: Caused by: android.system.ErrnoException: open failed: ENOENT (No such file or directory)
10-06 02:57:47.087 26943 27307 W System.err:    at libcore.io.Posix.open(Native Method)
10-06 02:57:47.087 26943 27307 W System.err:    at libcore.io.BlockGuardOs.open(BlockGuardOs.java:186)
10-06 02:57:47.087 26943 27307 W System.err:    at libcore.io.IoBridge.open(IoBridge.java:438)
10-06 02:57:47.088 26943 27307 W System.err:    ... 13 more

I a guessing I need to mkdirs to make the parent directories in my new code in setCertificate in SSLManager.

I probably can't fix this until next Monday, but it can be considered assigned to me unless someone gets to it first.

.

New device is not added to aware_device table in the database

When new device was added with the QR-code scanning it was not added to the list of devices.
In the web api website the list of devices is empty and it looks like it constantly loading something, but nothing is ever happening. (I have 19 devices but none of them is displayed in the web api panel)
The other data from device sensors are coming to the database.

Android wear module missing

I didn't find android-wear module in the repo, but I was able to build the repo by commenting out android-wear module references. Do you have any plans to include the android-wear module in this repo?

Thanks for making it open source.

cheers,
Shrirang

Periodically updating configuration

Hi,

I am interested in making a way to have configuration periodically updated (at a certain frequency, without MQTT).

Currently in aware-core / utils.StudyUtils onHandleIntent, I have made a branch which only updates the config. It is triggered by if an extra attribute is trueis_dynamic_update = intent.getBooleanExtra(EXTRA_DYNAMIC_UPDATE, false);. I am trying to see how to schedule this using Scheduler, which is a bit trick to me so far. Do you have any ideas?

I get can work myself down to here, in Aware.onStartCommand:

Scheduler.Schedule schedule = new Scheduler.Schedule(SCHEDULE_CONFIG_UPDATE)
    .setActionType(Scheduler.ACTION_TYPE_BROADCAST)
    //.setActionClass(JoinStudy)
    .addActionExtra(StudyUtils.EXTRA_DYNAMIC_UPDATE, true)
    .setInterval(frequency_update);

Any suggestions on where to go from there?

There may be an entirely better way to do this, if you would like to provide another idea or would like to hack something together, please let me know. I have a config option frequency_webservice_update.

By the way.. I see utils/StudyUtils, which has methods of joining a study. But I also see Aware.Aware.JoinStudy, extending the other. Which should be used for updating config?

Thanks,

• Richard

.

[BUG] Inserting some polish characters to database fails (probably not only polish)

As I pointed out in title - when you try to synchronize local Aware DB with some polish characters ('ę' for sure) - nothing happens, you don't get any failure - response status code is 200(OK).
Other polish characters (like 'ł' for example) are replaced with question mark ('?').
I think Aware should correctly replicate characters from every language.

It would be also nice to change database default charset to utf8 from latin1_swedish_ci which can generate some additional issues.

[TODO] Add support for SSL on remote database connections

See:
http://www.awareframework.com/forums/topic/enable-ssl-for-remote-mysql-databases/

[BUG] off-by-one error in webservice uploader

Hi,

I see this error message in adb log. The "sync accelerometer OK" means that hte first-round of syncing succeeded, but it is still trying. Thus, the error comes during the second round (syncing records [10000,19999])

09-04 02:07:48.244  9735 10825 D AWARE   : Sync accelerometer OK
09-04 02:07:48.249  9735 10825 W System.err: org.json.JSONException: Index 10000 out of range [0..10000)
09-04 02:07:48.257  9735 10825 W System.err:    at org.json.JSONArray.get(JSONArray.java:293)
09-04 02:07:48.257  9735 10825 W System.err:    at org.json.JSONArray.getJSONObject(JSONArray.java:521)
09-04 02:07:48.257  9735 10825 W System.err:    at com.aware.utils.WebserviceHelper.onHandleIntent(WebserviceHelper.java:348)
09-04 02:07:48.257  9735 10825 W System.err:    at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:66)
09-04 02:07:48.257  9735 10825 W System.err:    at android.os.Handler.dispatchMessage(Handler.java:102)
09-04 02:07:48.257  9735 10825 W System.err:    at android.os.Looper.loop(Looper.java:148)
09-04 02:07:48.257  9735 10825 W System.err:    at android.os.HandlerThread.run(HandlerThread.java:61)

It is trying to do the steps needed to delete the old values so that it can upload the rest. However, it fails on this line:

 long last = rows.getJSONObject(rows.length()).getLong("timestamp");

The error message says it all:

org.json.JSONException: Index 10000 out of range [0..10000)

If I understand this right, it should be rows.length()-1 instead of rows.length(), since indexes go from [0...9999].

The effect is that on my device, syncing is about one day behind, each 15 minutes it gets about 5000 new points and syncs 10000, and I guess due to circumstances it falls behind sometimes. If webservice frequency was lower, data would accumulate forever. Also data isn't being deleted. This behavior has been seen on two phones.

While fixing this, it would be nice for "always clean old data" to clean old data from all tables, not just high frequency ones (line 346), as I have mentioned before.

Also, at one point you mentioned something suggesting that the web upload API was improved, but I see the same large number of requests as before (for every table, for every upload attempt, /create_table and /latest every time regardless of what data their is). Looking back at that, maybe it just meant that you wouldn't /create_table and /latest or each 10000 row chunk. Should I expect any other behavior? And are there any more improvement expected?

Thank you,

• Richard

[bug] Configuration lost when upgraded through play store

Hi,

On two devices now, we have lost configuration when AWARE has been updated through Google Play. There haven't been enough upgrades to repeat this several times to get more info.

The exact symptom is that AWARE is re-opened, and then you see in the UI that configuration options are not set (seen most clearly in the study configuration).

Any thoughts? We will update this issue when we observe more.

Thanks,
richard

[BUG/Improvement] Synchronization of deleted items

Hi,

As I noticed, there is no synchronization for deleted or updated items in Aware.
I understand that synchronizing updates can be difficult and overkill, but sync of deleted items, should be quite easy and manageable. Then users could develop apps using delete-insert operations instead of update.

If it isn't possible, could you give privilege to manage triggers? It'd allow us to create some hacks. I prefer to create hack than mess with Aware code. Changing sync queries to "insert on duplicate update" also would work for my case.

Regards,
K.B.

Application usage problems

I'm trying to use your framework to collect notifications but I'm having some troubles.

Here is the procedure I followed:

1. I downloaded awareframework.apk from this url

http://www.awareframework.com/awareframework.apk

1. I installed it on my smartphone (Karbonn sparkle V) and I opened it to set all the settings
2. I enabled every available sensor (after enabling accessibility service)
3. I exited the app and after a while I came back in the app
4. If I went to the "Applications" tab I found the "Application usage" flag disabled.
   If I try to enable it again, the system asks me to enable the accessibility service again, but it is still enabled.

Do you know anything about this problem?
I tried on a Nexus 5, too, and I had the same problem!

If user already in study, using QR code does not change their study.

Steps to reproduce:

• Link study via QR code
• Observe that you are in a study
• While still in that study, scan a QR code for a different study.
• webservice_server does not change, and also if you click the study icon, it presents information on the first study, not the second.

I just found this with a tester (@TuomasA) here, I haven't looked into it yet.

• Richard

Bluetooth beacon support

Hi,

We'd like to add support for bluetooth beacons. I opened a issue at digitraceslab#3 which we will use for discussion. Would you like to comment there (or here)? Mainly we should settle on a client-server API for configuration, and then DB schema.

Thanks

• Richard

[BUG] Exceptions when joining a study (MQTT and cursors)

After I scan the QR of my Aware server I get two exceptions, one related to the MQTT URL and the other to an unclosed cursor in the Scheduler:

08-30 11:41:25.680 13378-13378/com.aware.phone W/ContextImpl: Failed to ensure directory: /storage/sdcard1/Android/data/com.aware.phone/files
08-30 11:41:25.680 13378-13378/com.aware.phone E/AWARE: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: inStream == null
08-30 11:41:25.877 13378-13378/com.aware.phone D/AndroidRuntime: Shutting down VM


                                                                 --------- beginning of crash
08-30 11:41:25.882 13378-13378/com.aware.phone E/AndroidRuntime: FATAL EXCEPTION: main
                                                                 Process: com.aware.phone, PID: 13378
                                                                 java.lang.RuntimeException: Unable to start service com.aware.Mqtt@2c4c5b44 with Intent { cmp=com.aware.phone/com.aware.Mqtt }: java.lang.IllegalArgumentException: ssl://https://mystudy.co.uk:8883
                                                                     at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:2975)
                                                                     at android.app.ActivityThread.access$2100(ActivityThread.java:156)
                                                                     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1449)
                                                                     at android.os.Handler.dispatchMessage(Handler.java:102)
                                                                     at android.os.Looper.loop(Looper.java:211)
                                                                     at android.app.ActivityThread.main(ActivityThread.java:5373)
                                                                     at java.lang.reflect.Method.invoke(Native Method)
                                                                     at java.lang.reflect.Method.invoke(Method.java:372)
                                                                     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1020)
                                                                     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:815)
                                                                  Caused by: java.lang.IllegalArgumentException: ssl://https://mystudy.co.uk:8883
                                                                     at org.eclipse.paho.client.mqttv3.MqttConnectOptions.validateURI(MqttConnectOptions.java:458)
                                                                     at org.eclipse.paho.client.mqttv3.MqttAsyncClient.<init>(MqttAsyncClient.java:273)
                                                                     at org.eclipse.paho.client.mqttv3.MqttAsyncClient.<init>(MqttAsyncClient.java:167)
                                                                     at org.eclipse.paho.client.mqttv3.MqttClient.<init>(MqttClient.java:224)
                                                                     at com.aware.Mqtt.initializeMQTT(Mqtt.java:414)
                                                                     at com.aware.Mqtt.onStartCommand(Mqtt.java:352)
                                                                     at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:2958)
                                                                     at android.app.ActivityThread.access$2100(ActivityThread.java:156) 
                                                                     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1449) 
                                                                     at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                                     at android.os.Looper.loop(Looper.java:211) 
                                                                     at android.app.ActivityThread.main(ActivityThread.java:5373) 
                                                                     at java.lang.reflect.Method.invoke(Native Method) 
                                                                     at java.lang.reflect.Method.invoke(Method.java:372) 
                                                                     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1020) 
                                                                     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:815) 
08-30 11:41:29.865 13378-13390/com.aware.phone E/Cursor: Finalizing a Cursor that has not been deactivated or closed. database = /storage/emulated/0/AWARE/scheduler.db, table = scheduler, query = SELECT schedule, timestamp, device_id, _id, schedule_id, package_name, last_triggered FROM scheduler
                                                         net.sqlcipher.database.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
                                                             at net.sqlcipher.database.SQLiteCursor.<init>(SQLiteCursor.java:217)
                                                             at net.sqlcipher.database.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:53)
                                                             at net.sqlcipher.database.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1772)
                                                             at net.sqlcipher.database.SQLiteQueryBuilder.query(SQLiteQueryBuilder.java:330)
                                                             at net.sqlcipher.database.SQLiteQueryBuilder.query(SQLiteQueryBuilder.java:280)
                                                             at com.aware.providers.Scheduler_Provider.query(Scheduler_Provider.java:213)
                                                             at android.content.ContentProvider.query(ContentProvider.java:1000)
                                                             at android.content.ContentProvider$Transport.query(ContentProvider.java:214)
                                                             at android.content.ContentResolver.query(ContentResolver.java:482)
                                                             at android.content.ContentResolver.query(ContentResolver.java:426)
                                                             at com.aware.utils.Scheduler.getSchedule(Scheduler.java:150)
                                                             at com.aware.Aware.onStartCommand(Aware.java:415)
                                                             at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:2958)
                                                             at android.app.ActivityThread.access$2100(ActivityThread.java:156)
                                                             at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1449)
                                                             at android.os.Handler.dispatchMessage(Handler.java:102)
                                                             at android.os.Looper.loop(Looper.java:211)
                                                             at android.app.ActivityThread.main(ActivityThread.java:5373)
                                                             at java.lang.reflect.Method.invoke(Native Method)
                                                             at java.lang.reflect.Method.invoke(Method.java:372)
                                                             at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1020)
                                                             at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:815)

After a few minutes Aware tried to resume itself but I got the same exception with the MQTT URL and a new one with an unclosed cursor in Light.

08-30 11:46:29.338 6054-6054/com.aware.phone E/AWARE: com.android.org.conscrypt.OpenSSLX509CertificateFactory$ParsingException: inStream == null
08-30 11:46:29.406 6054-6066/com.aware.phone W/CursorWrapperInner: Cursor finalized without prior close()
08-30 11:46:29.408 6054-6066/com.aware.phone E/Cursor: Finalizing a Cursor that has not been deactivated or closed. database = /storage/emulated/0/AWARE/light.db, table = sensor_light, query = SELECT sensor_version, sensor_type, device_id, double_sensor_resolution, double_sensor_power_ma, tim
                                                       net.sqlcipher.database.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
                                                           at net.sqlcipher.database.SQLiteCursor.<init>(SQLiteCursor.java:217)
                                                           at net.sqlcipher.database.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:53)
                                                           at net.sqlcipher.database.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1772)
                                                           at net.sqlcipher.database.SQLiteQueryBuilder.query(SQLiteQueryBuilder.java:330)
                                                           at net.sqlcipher.database.SQLiteQueryBuilder.query(SQLiteQueryBuilder.java:280)
                                                           at com.aware.providers.Light_Provider.query(Light_Provider.java:374)
                                                           at android.content.ContentProvider.query(ContentProvider.java:1000)
                                                           at android.content.ContentProvider$Transport.query(ContentProvider.java:214)
                                                           at android.content.ContentResolver.query(ContentResolver.java:482)
                                                           at android.content.ContentResolver.query(ContentResolver.java:426)
                                                           at com.aware.Light.saveSensorDevice(Light.java:159)
                                                           at com.aware.Light.onCreate(Light.java:230)
                                                           at android.app.ActivityThread.handleCreateService(ActivityThread.java:2825)
                                                           at android.app.ActivityThread.access$1800(ActivityThread.java:156)
                                                           at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1434)
                                                           at android.os.Handler.dispatchMessage(Handler.java:102)
                                                           at android.os.Looper.loop(Looper.java:211)
                                                           at android.app.ActivityThread.main(ActivityThread.java:5373)
                                                           at java.lang.reflect.Method.invoke(Native Method)
                                                           at java.lang.reflect.Method.invoke(Method.java:372)
                                                           at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1020)
                                                           at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:815)

[wishlist] customize upload notifications with study name

Hi,

While testing AWARE, we saw that upload notifications say "AWARE Sync". For studies where subjects don't know what AWARE is, it could be useful to have the study name, or some more information, in the notification.

A simple option could be to use the configured study_name. Or, there could be another option that can contain a different text that could be used in cases like that. Then, at least when notifications appear, the study name is seen. Of course the app name would stay the same so when looking at system things, you would see that.

Relevant location: WebserviceHelper.java, notifyUser function, line 62+

What do you think? Any ideas? We could contribute this, but you can probably do and test much faster.

Thanks,

• Richard

Client pings oulu server regardless of configured webservice

From the forum:

Does the android client always ping the Oulu server?

Yes. We like to keep track of how many active clients there are. For statistics and for us to measure the demand on our server.

In the case I am planning for, we are not using the Oulu server. Pinging it reveals extra information, and for some of our studies even the fact that someone might be involved would be considered information which shouldn't be shared. Our ethical permissions state that our apps do not give any information to any third party (and also I wouldn't be comfortable with sharing this information about my subjects).

Perhaps could the client ping the configured webservice instead? For the google play version, it should have statistics on total installations.

Thanks,

• richard

Finalizing a cursor exception

Hi,

I ran into this exception after doing a clean Aware Client install and activating the accelerometer logging for 30 seconds and turning it off again. The app is running on Android 5.1.1.

Could this be because the method saveAccelerometerDevice() in Accelerometer.java is only closing the cursor when is null or empty?

08-25 16:11:06.836 4094-4106/com.aware.phone E/Cursor: Finalizing a Cursor that has not been deactivated or closed. database = /storage/emulated/0/AWARE/accelerometer.db, table = sensor_accelerometer, query = SELECT sensor_version, sensor_type, device_id, double_sensor_resolution, double_sensor_power_ma, tim net.sqlcipher.database.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here at net.sqlcipher.database.SQLiteCursor.<init>(SQLiteCursor.java:217) at net.sqlcipher.database.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:53) at net.sqlcipher.database.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1758) at net.sqlcipher.database.SQLiteQueryBuilder.query(SQLiteQueryBuilder.java:330) at net.sqlcipher.database.SQLiteQueryBuilder.query(SQLiteQueryBuilder.java:280) at com.aware.providers.Accelerometer_Provider.query(Accelerometer_Provider.java:386) at android.content.ContentProvider.query(ContentProvider.java:1000) at android.content.ContentProvider$Transport.query(ContentProvider.java:214) at android.content.ContentResolver.query(ContentResolver.java:482) at android.content.ContentResolver.query(ContentResolver.java:426) at com.aware.Accelerometer.saveAccelerometerDevice(Accelerometer.java:154) at com.aware.Accelerometer.onCreate(Accelerometer.java:224) at android.app.ActivityThread.handleCreateService(ActivityThread.java:2825) at android.app.ActivityThread.access$1800(ActivityThread.java:156) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1434) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:211) at android.app.ActivityThread.main(ActivityThread.java:5373) at java.lang.reflect.Method.invoke(Native Method) at java.lang.reflect.Method.invoke(Method.java:372) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1020) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:815)

.

[wishlist] better ssl cert verification using QR code

verification of pinned SSL cert hash via QR code parameter

Can you explain to me why is this important? The user only gets the SSL after reading the QRCode. I guess you want to prevent a man-in-the-middle attack?

Yes, something like that. Right now cert is fetched over non-ssl. When making our first app, we had QR code embed https://url?cert_sha256=hash&.... So, if the computer presenting QR code is secure, then you can establish the entire trust chain. It's not perfect, for example it assumes the PKI works on the subject's computer if they are doing it themselves. The disadvantage is that, for long studies, the cert couldn't be updated partway through.

What we could contribute:

• when getting config URL, strip off everything after ?. At least this could be done sooner, so that if options are added in the future, old clients won't break.
• The options after ? can be querystring parsed and stored somewhere
• When getting server.crt (or ca.crt), if there is a hash given in qr code, verify it.

What do you think? It is a relatively specialized idea that is only useful against active attacks. Can you think of any better ways to do this? (aside: we would implement server-side support in our server, but I'm not expecting this in the aware-server. We actually have server-side support for this for another app).

Thanks,

• Richard

Permissions being asked multiple times

Hi,

Sometimes, when Aware asks for permissions, it gets into a loop and does so multiple times. Recently, I was demonstrating it to someone and it asked for call/sms/and a few other permissions 5-10 times in a loop.

I don't have details right now, but I am making the issue for tracking, and will update as I learn more.

Android app requires camera, which restricts google play availability

From forum post:

– It doesn’t run on my tablet (requires back camera feature?)

Can you check the logcat and see what could be the reason? I’ve tested it before on a tablet and had no issues.

I should clarify: it isn't that it doesn't run, but that it is not seen as available on google play (the latest version). I could possibly install it manually, but that restricts our audience. Steps to reproduce: search "Aware Framework" in google play app on device, or go to google play web and search aware framework, it is listed as "not compatible with $tablet name.

After investigating, I think the reason is the declared required features, since this tablet does not have a back camera. In aware-phone AndroidManifest.xml:

    <uses-feature android:name="android.hardware.camera" />
    <uses-feature android:name="android.hardware.camera.autofocus" />

According to https://developer.android.com/guide/topics/manifest/uses-feature-element.html#hw-features, android.hardware.camera means back facing camera.

Being able to run on as wide a variety of platforms as possible is important to us, since our subjects may be from demographics which are not as likely to have modern devices. And our use case would be monitoring all devices, not just the primary device.

Suggested workaround: adding android:required="false" to the camera-related permission and if they are not available, the app does not use them. Instead, one has to configure by manually entering the config URL.

[BUG] Aware clear data

There is bug in implementation of WebserviceHelper, when you try to clear data you got:

FATAL EXCEPTION: IntentService[AWARE Webservice Sync]
    Process: com.aware.plugin.google.fused_location, PID: 11641
    java.lang.NullPointerException: uriString

It's because of this line:

Uri CONTENT_URI = Uri.parse(intent.getStringExtra(EXTRA_CONTENT_URI));

You try to parse uriString that will always be null for clear data broadcast.

[BUG] DB synchronization in batch

When I try to synchronize DB with Aware server for some reason it fails (there is not feedback about exception and no change in remote DB).

It's a json that is posted as 'data' in insert request:

[{
    "timestamp": 14306816639010,
    "device_id": "b0c46dda-37a8-4a50-9740-0655e17d5fe3",
    "double_poi_id": 534590614,
    "type": "node",
    "double_lat": 50.0142059,
    "double_lon": 19.9279012,
    "name": "Galeria Solvay",
    "shop": "supermarket"
},
{
    "timestamp": 14307555075910,
    "device_id": "b0c46dda-37a8-4a50-9740-0655e17d5fe3",
    "double_poi_id": 2708916232,
    "type": "node",
    "double_lat": 50.0435547,
    "double_lon": 19.9439609,
    "name": "Klopsztanga – trzepak reaktywacja"
},
{
    "timestamp": 14307555214700,
    "device_id": "b0c46dda-37a8-4a50-9740-0655e17d5fe3",
    "double_poi_id": 1500744003,
    "type": "node",
    "double_lat": 50.0290285,
    "double_lon": 19.9613803,
    "name": "Pomnik Ofiar Faszyzmu",
    "trsm": "viewpoint"
},
{
    "timestamp": 14307560115450,
    "device_id": "b0c46dda-37a8-4a50-9740-0655e17d5fe3",
    "double_poi_id": 1480534372,
    "type": "node",
    "double_lat": 50.0429659,
    "double_lon": 19.9468393,
    "name": "Basen KS Korona",
    "amnt": "swimming_pool"
}]

When I send every row separately everything works!

.

Log version code and version name on the installations sensor

Add support for logging the version code and version name on the installations sensor.

Buffering sensor events: API level, timestamps

See #30 for discussion.

Topics to be addressed:

• Currently using a function added in API level 19, this needs to be made backwards-compatible
• Buffering currently can be made more flexible: right now reporting latency may not be set correctly.
• If it was set correctly, we would need to resolve the meaning of sensor timestamps, since it is not standardized across devices.

.