Improve export/import, add stats sync across devices

[andlyticsproject]

Improve export/import, add stats sync across devices

[AndyScherzinger]

Thanks @nelenkov !

I haven't found the time to look at your implementation but from my point of view it is be a great enhancement to getting exports from one device to another manually. Autosync is definitely the nicest way and we should see if someone can find the time in the future to implement this for an upcoming release.

[AndyScherzinger]

hitting the import button leads to an app crash (fresh install from dev branch):

07-24 11:10:31.551: E/AndroidRuntime(10474): FATAL EXCEPTION: main
07-24 11:10:31.551: E/AndroidRuntime(10474): android.content.ActivityNotFoundException: Unable to find explicit activity class {com.github.andlyticsproject/com.github.andlyticsproject.ImportActivity}; have you declared this activity in your AndroidManifest.xml?
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Instrumentation.checkStartActivityResult(Instrumentation.java:1541)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Instrumentation.execStartActivity(Instrumentation.java:1416)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Activity.startActivityForResult(Activity.java:3351)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Activity.startActivityForResult(Activity.java:3312)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Activity.startActivity(Activity.java:3522)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Activity.startActivity(Activity.java:3490)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.github.andlyticsproject.Main.onOptionsItemSelected(Main.java:203)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.actionbarsherlock.app.SherlockActivity.onMenuItemSelected(SherlockActivity.java:197)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.actionbarsherlock.ActionBarSherlock.callbackOptionsItemSelected(ActionBarSherlock.java:600)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.actionbarsherlock.internal.ActionBarSherlockNative.dispatchOptionsItemSelected(ActionBarSherlockNative.java:78)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.actionbarsherlock.app.SherlockActivity.onOptionsItemSelected(SherlockActivity.java:148)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.Activity.onMenuItemSelected(Activity.java:2534)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.policy.impl.PhoneWindow.onMenuItemSelected(PhoneWindow.java:958)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.view.menu.MenuBuilder.dispatchMenuItemSelected(MenuBuilder.java:735)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.view.menu.MenuItemImpl.invoke(MenuItemImpl.java:149)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.view.menu.MenuBuilder.performItemAction(MenuBuilder.java:874)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.view.menu.ActionMenuView.invokeItem(ActionMenuView.java:514)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.view.menu.ActionMenuItemView.onClick(ActionMenuItemView.java:99)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.view.View.performClick(View.java:4084)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.view.View$PerformClick.run(View.java:16966)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.os.Handler.handleCallback(Handler.java:615)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.os.Handler.dispatchMessage(Handler.java:92)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.os.Looper.loop(Looper.java:137)
07-24 11:10:31.551: E/AndroidRuntime(10474): at android.app.ActivityThread.main(ActivityThread.java:4745)
07-24 11:10:31.551: E/AndroidRuntime(10474): at java.lang.reflect.Method.invokeNative(Native Method)
07-24 11:10:31.551: E/AndroidRuntime(10474): at java.lang.reflect.Method.invoke(Method.java:511)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
07-24 11:10:31.551: E/AndroidRuntime(10474): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
07-24 11:10:31.551: E/AndroidRuntime(10474): at dalvik.system.NativeStart.main(Native Method)

(I checked, the activity is in the manifest...)

[AndyScherzinger]

...forget about it... Eclipse didn't properly refresh to manifest... works now...

[AndyScherzinger]

Just for a little discussion, we should find a namining pattern for the zip file. "andlytics.zip" doesn't work for me since I have three dev console accounts atm and thus couldn't put all exports in the same folder (on dropbox or whatever) and I couldn't distinguish between the exports of the different accounts.
It doesn't look nice, but how about adding the dev-account email to the filename?

Any thought? with this we could even import based on the zip's filename ;) (a bit hacky I know)

[nelenkov]

I actually thought about adding this, but it would require some more work to let you select a file if there are multiple ones. Right now the only way is manually rename and go through the ANDROID_VIEW action. It probably also needs a way to select the account to import to, right now it's fixed to the current (from preferences) account.

Sanitized email should work as an identifier. We could also add a timestamp, but that would result in a bunch of files if you export often.

[AndyScherzinger]

Not sure about the timestamp, having just the email as identifier in the name would get us a implizit connection between account and export.zip. So we could then (at some point) find the matching account during import :) I would save that for later, but already implement the naming pattern? What do you think?

[willlunniss]

Looks good so far, definately make an export activity, and remove the buttons and replace them with a split action bar, or standard dialog style button (using the proper seperator style). The cancel button can be replaced with a homeAsUp actionbar indicator/pressing back. Also there is the blue dialog border that could be removed.

I also think something should be done about just having one file name, can we not have wildcard intent to look for all andlytics-*.zip files?

[nelenkov]

OK, account name is now included, please test. To export you need to switch to the relevant account. There is no 'export all' action (yet?).

I've removed the dialog border, and renamed the the close button to 'Cancel'. If we decide to use native buttons, it should be done for the whole app. If you want to keep the current look and feel, we should create a button style based on the current button-like layouts. I don't think the bottom bar makes sense in those activities, so I've kept the buttons.

[willlunniss]

I'll test this later today.

[willlunniss]

The import into the wrong account protection worked. Maybe change the notification to include the full file name and path. Otherwise, works well.

[AndyScherzinger]

How does the import work now?
If you choose import in the app itself it shows the csv-file to import which I have lying around from before the zip-file export functionality. Did I do something wrong, or do we also have to rewrite the import logic in the app itself?

[nelenkov]

It should be looking for a '[email protected]' file and displaying the names of the files (packages) inside. It doesn't use CSV files directly.

[AndyScherzinger]

hmm will have to check that later. Just deleted the csv file in the folder and now I get a "SD card not mounted or invalid file format" toast...

[nelenkov]

I see the problem now. It does read from the zip file, but when verifying tries to use the CSV files directly. Will fix.

[AndyScherzinger]

Let me know when it is fixed and I will retest it then :-)
Thanks for your effort towards a more comfortable "sync" between devices.

[nelenkov]

This has to work now :) Also writes directly to the zip files and thus export should be slightly faster.

[AndyScherzinger]

Works fine now on my device :)

[willlunniss]

What is the status of this, any thing else that needs doing before the next public release (it can obviously be improved on later)?

[nelenkov]

The zip export/import works and unless anyone has any issues ideas, I consider it done for now. The next phase -- auto sync, Google Docs, etc., should go in some next release. I will check how it works with latest dev tomorrow, but there shouldn't be any problems.

[AndyScherzinger]

Works fine on my device

[fda77]

Could old export still be imported? Would be nice for the next version-bump

[fda77]

My 1st try to export:

Service com.github.andlyticsproject.io.ExportService has leaked ServiceConnection android.media.MediaScannerConnection@41492790 that was originally bound here
android.app.ServiceConnectionLeaked: Service com.github.andlyticsproject.io.ExportService has leaked ServiceConnection android.media.MediaScannerConnection@41492790 that was originally bound here
at android.app.LoadedApk$ServiceDispatcher.(LoadedApk.java:965)
at android.app.LoadedApk.getServiceDispatcher(LoadedApk.java:859)
at android.app.ContextImpl.bindService(ContextImpl.java:1216)
at android.app.ContextImpl.bindService(ContextImpl.java:1208)
at android.content.ContextWrapper.bindService(ContextWrapper.java:394)
at android.media.MediaScannerConnection.connect(MediaScannerConnection.java:116)
at android.media.MediaScannerConnection.scanFile(MediaScannerConnection.java:235)
at com.github.andlyticsproject.io.MediaScannerWrapper.scanFile(MediaScannerWrapper.java:15)
at com.github.andlyticsproject.util.Utils.scanFile(Utils.java:81)
at com.github.andlyticsproject.io.ExportService.exportStats(ExportService.java:102)
at com.github.andlyticsproject.io.ExportService.onHandleIntent(ExportService.java:71)
at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:65)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:137)
at android.os.HandlerThread.run(HandlerThread.java:60)

[nelenkov]

Old style exports can't be imported automatically, but if you do need them you can simply zip them up and import.

Re: the stack trace, this a warning, will see if it can be avoided.

[willlunniss]

In that case, we should put a note on google play with instructions for importing old style data for the first few weeks after releasing 2.2. Does it need to be named anything in particular?

[nelenkov]

The file should be named andlytics-account_name.zip, where 'account_name' is the developer account email ([email protected], etc.)

The latest merge uses the new notifications and properly handles detaching AsyncTaks.

Re: the MediaScanner error, this appears to be a bug in MediaScanner. Default apps (Email, etc.) don't handle it in any special way, so leaving it as is for now.

[willlunniss]

A few small comments:
https://github.com/AndlyticsProject/andlytics/pull/155/files#r1261016

Use getSupportActionBar().setDisplayHomeAsUpEnabled(true); (combined with the correct onOptionsItemSelected call) as the user can navigate back from either the import or export

In the import one, maybe don't show the .csv bit on the end, it might confuse users and doesn't make that much sense anymore

I know I said this before, but it stands out even more now that users can hide AdMob:
Apart from the 'Add AdMob account' button, nowhere else uses those buttons so they do look a little out of place.

[nelenkov]

I don't think homeAsUp makes sense here: this is not master-detail, and the fact the import/export is started from the previous activity is merely an implementation detail. For example, import can be started by opening a zip file, and the main activity may not even be running.

I'll look into the .csv extensions thing.

I could just replace them with regular buttons, but that would break the overall design, especially on 2.x where buttons are light gray(ish). Is everyone OK with that?

[willlunniss]

For example, import can be started by opening a zip file, and the main activity may not even be running.

I forgot about that. Good point.

I'm not sure what the best solution would be tbh, its more the fact it goes half white when you hold it down that doesn't fit. It is fine for now, it was just an observation.

[nelenkov]

OK, I see. It seems this comes from the gradient in the image_button drawable, but not sure what the best way to handle it is. Moving all colors and dimensions to resources would be a good start, so all of this is a little more readable. BTW, drawable/image_button shows up in the following layouts:

layout/admob_config_addaccount.xml
layout/admob_login.xml
layout/crash_dialog.xml
layout/export_stats.xml
layout/feedback_dialog.xml
layout/import_stats.xml
layout/translate_dialog.xml

Are all of those currently used?

[willlunniss]

Translate and feedback aren't. You can answer for the stats ones, AdMob is.

[AndyScherzinger]

just pushed a fix on the image_button to dev. Works for the import/export & admob.

[willlunniss]

I think crash is as well. Clearing out unused resources is another thing on my todo list.

[nelenkov]

image_button looks good.

Fixed notifications and .csv extension in latest pull request.

[willlunniss]

Someone has made a very interesting comment on Google Play

Could you set it up to import the CSV export from the dev console? And preferably do it automatically?

The first part would be especially useful for new users.

[AndyScherzinger]

Also think that would be very helpful especially with new users, so we could probably just download the csv file per app and use that data (just an idea, no idea how easy/hard this would be to implement) but we would then endup with the API calls for the csv file and for the comments plus we could even get all data present in the console for installs/uninstalls/devices/countries/etc. :)
But this should be considered being a top release like 3.0.0 or something like that.

[nelenkov]

That's an interesting idea, but the console stats only have installs/uninstalls/upgrades numbers. Real data seems to start from November 2011, but the daily active_device_installs and total_user_installs could probably be used to fill out the installs. Everything else has to be 0/undefined.

[willlunniss]

OK, I'll mark this as closed for now. We can open a new one later with enhanced features