map-a-droid/mad

Map PoGo stuff with Android devices

Map'A'Droid

Map'A'Droid is a scanner for PoGo based on Android devices.

Information

• Website - For general information
• Wiki Page - For Installation instructions and other stuff
• Discord - For general supports
• ATV boxes - For information about running MAD on cheap ATV boxes
• Supported Devices - Supported devices by MAD
• RocketMAD - Compatible map frontend
• Github Issues - For reporting bugs (not for support!)
• github wiki page

Requirements

• A supported version of Python: 3.9, 3.10, 3.11
• MySQL database, with RocketMap structure
• Rooted Android device
• PogoDroid token, obtainable maddev.eu
• Game running in English, Deutsch or Français (for Quest mode)

Installation

Have a look at our Wiki

Contributing

Information related to contributing to Map'A'Droid can be found here

mad's Issues

Device origin is unknown after adding it

[ WARNING] Register attempt of unknown Origin {name}

It looks like not all parts of MAD are properly reloaded after a new device has been added.

gyms area vs actual gyms

I have one area called "gyms". The MADmin Map only shows the route of it or the real gyms. But never both.

Changes to existing area not reflected in MADmin

Within an existing Area I just changed my geofence and routecalc (to scan a bigger area for quests), MAD created new route.calc, MADmin on the other hand is still showing the old route, from last night which does not reflect the changes made.

Next I’ve created a new Area, including the geofence and routecalc from the old area, to avoid a recalc, that new area isn’t shown in MADmin at all.

Finally delete route calc in new Area and have it recalculated calculate by MAD (restarted MAD), then both area as the new route.calc are shown in sidebar.

Nothing shocking, just fyi.

raids ocr scanning, some raids are missing

Some raids in ocr scan is missing. Logs show in some gyms make a crop image and not analyse. Some crop images are empty in temp (event fresh created). And last interesting thing is that, show only gyms and raids (in ocr menu) from area that match with geofence that have the another walker.

Intel Importer (intelimport.sh) is broken

The Intel Importer is broken right now after the removement of Monocle.
The error msg is right now: Unknown dbmethod set in MAD config file.

open gym is not close during pokestops scan.

Opened GYM is not detected on mode quest .

Never test present of cross for close gym.

And after start loop "test turn map , click pokestop" and long time after "pokemon go restart"

Need detect open gym instead of a pokestop or detected cross before turn ( add proto 156 on wait_proto ).

Starting MAD in Sleeptime with no nighttime_area results in an Error in connection handler

I get an error when I start MAD and the following conditions are met:
Sleeptime is on
Time for Sleeptimemode is reached
nighttime_area is None

Error:
[02-10 09:12:25] [ switchtimer] [ timer:59] [ INFO] [s5mini1] - Switching Mode
[02-10 09:12:25] [ switchtimer] [ timer:26] [ INFO] [s5mini1] - set switch: True
[02-10 09:12:25] [ switchtimer] [ timer:64] [ INFO] [s5mini1] - Currently in switchmode
[02-10 09:12:26] [ scanner] [ server:171] [ ERROR] Error in connection handler
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/websockets/server.py", line 169, in handler
yield from self.ws_handler(self, path)
File "/root/MAD/websocket/WebsocketServer.py", line 69, in handler
continue_work = await self.__register(websocket_client_connection)
File "/root/MAD/websocket/WebsocketServer.py", line 137, in __register
client_mapping['mon_ids_iv'] = self.__routemanagers[client_mapping["nighttime_area"]].get("routemanager").settings.get("mon_ids_iv", [])
KeyError: None
[02-10 09:12:51] [ switchtimer] [ timer:64] [ INFO] [s5mini1] - Currently in switchmode

Add gps coordinates to 100iv and shiny stats

What was mapping tools without coordinates info :D?

Crash due to screenshot ocr error (?) (latest dev as of now)

[07-30 03:09:20.50] [         s8walker] [           MITMBase:138 ] [    INFO] Worker s8walker is not injected till now (Count: 1)
[07-30 03:09:40.52] [         s8walker] [           MITMBase:138 ] [    INFO] Worker s8walker is not injected till now (Count: 2)
[07-30 03:10:00.55] [         s8walker] [           MITMBase:138 ] [    INFO] Worker s8walker is not injected till now (Count: 3)
[07-30 03:10:00.55] [         s8walker] [           MITMBase:141 ] [    INFO] Worker s8walker will retry check_windows while waiting for injection at count 3
[07-30 03:10:00.55] [         s8walker] [         WorkerBase:560 ] [    INFO] Checking pogo screen...
[07-30 03:10:09.79] [         Thread-9] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.RETRY
[07-30 03:10:26.09] [        Thread-10] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.RETURNING
[07-30 03:10:44.89] [         Thread-9] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.LOGINSELECT
[07-30 03:10:50.76] [        Thread-10] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.GGL
[07-30 03:10:50.76] [        Thread-10] [         screenPath:122 ] [    INFO] Request next Account - Using Nr. 1
[07-30 03:10:50.76] [        Thread-10] [         screenPath:134 ] [    INFO] Using GGL Account: po***on@***.org
[07-30 03:10:52.92] [        Thread-10] [         screenPath:478 ] [    INFO] Found mail po***on@***.org
[07-30 03:11:29.10] [         Thread-9] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.FAILURE
[07-30 03:11:35.77] [         s8walker] [         WorkerBase:580 ] [ WARNING] Something wrong with screendetection
[07-30 03:11:47.59] [        Thread-10] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.LOGINSELECT
[07-30 03:11:53.25] [         Thread-9] [         screenPath:218 ] [    INFO] Processing Screen: ScreenType.GGL
[07-30 03:11:53.25] [         Thread-9] [         screenPath:125 ] [    INFO] Request next Account - Restarting with Nr. 1
[07-30 03:11:53.25] [         Thread-9] [         screenPath:134 ] [    INFO] Using GGL Account: po***on@****.org
(<class 'xml.etree.ElementTree.ParseError'>, ParseError('syntax error: line 1, column 0',), <traceback object at 0x7fc6401f59c8>)
[07-30 03:11:54.78] [         s8walker] [              start:66  ] [CRITICAL] Unhandled patched exception (<class 'xml.etree.ElementTree.ParseError'>): "syntax error: line 1, column 0".
[07-30 03:11:54.78] [         s8walker] [              start:66  ] [CRITICAL] Unhandled patched exception (<class 'xml.etree.ElementTree.ParseError'>): "syntax error: line 1, column 0".
Traceback (most recent call last):
  File "start.py", line 53, in run_thread
    run_thread_old(*args, **kwargs)
  File "/usr/lib64/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pokemon/MAD/worker/WorkerBase.py", line 321, in _main_work_thread
    self._internal_pre_work()
  File "/home/pokemon/MAD/worker/WorkerBase.py", line 271, in _internal_pre_work
    self._pre_work_loop()
  File "/home/pokemon/MAD/worker/WorkerQuests.py", line 77, in _pre_work_loop
    if not self._wait_for_injection() or self._stop_worker_event.is_set():
  File "/home/pokemon/MAD/worker/MITMBase.py", line 142, in _wait_for_injection
    self._check_windows()
  File "/home/pokemon/MAD/worker/WorkerBase.py", line 565, in _check_windows
    returncode = self._WordToScreenMatching.matchScreen()
  File "/home/pokemon/MAD/ocr/screenPath.py", line 165, in matchScreen
    return self.__thread_pool.apply_async(self.__internal_matchScreen, ()).get()
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/pokemon/MAD/ocr/screenPath.py", line 223, in __internal_matchScreen
    if self.parse_ggl(self._communicator.uiautomator(), ggl_login.username):
  File "/home/pokemon/MAD/ocr/screenPath.py", line 475, in parse_ggl
    xmlroot = ET.fromstring(xml, parser=parser)
  File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1314, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: syntax error: line 1, column 0

Redmi 6A quests not deleting

Quests never delete menu quests is open. Bot not click on trash for delete eache quests

Log with -vvvv and screenshot same time ( get with phonecontrol )

DeviceWidth=720, deviceHeight=1440 ( 2x)
Mode full immesive activate for pokemon go

[05-16 13:43:26.10] [clear_thread_redmi6a07] [ WebsocketServer:437 ] [ DEBUG] redmi6a07 sending command: screen capture jpeg 80
[05-16 13:43:26.10] [clear_thread_redmi6a07] [ WebsocketServer:452 ] [ DEBUG] To be sent: 47;screen capture jpeg 80
[05-16 13:43:26.10] [clear_thread_redmi6a07] [ WebsocketServer:457 ] [ DEBUG] Timeout: 30
[05-16 13:43:26.12] [ scanner] [ WebsocketServer:310 ] [ DEBUG] Retrieving next message to send
[05-16 13:43:26.74] [ system] [ start:141 ] [ DEBUG] Starting internal Cleanup
[05-16 13:43:26.74] [ system] [ start:142 ] [ DEBUG] Collecting...
[05-16 13:43:26.76] [ system] [ start:145 ] [ DEBUG] Unreachable objects: 0 - Remaining garbage: [] - Running threads: 26
[05-16 13:43:26.76] [ system] [ start:154 ] [ DEBUG] Clearing gc garbage
[05-16 13:43:26.76] [ system] [ start:160 ] [ INFO] Instance Name: "DEV" - Memory usage: 0.13051223754882812 - CPU usage: 73.9
[05-16 13:43:26.76] [ system] [ dbWrapperBase:1149] [ DEBUG] dbWrapper::insert_usage
[05-16 13:43:27.75] [ scanner] [ WebsocketServer:391 ] [ DEBUG] Received binary values.
[05-16 13:43:27.75] [clear_thread_redmi6a07] [ WebsocketServer:459 ] [ DEBUG] Received answer in time, popping response
[05-16 13:43:27.75] [clear_thread_redmi6a07] [ WebsocketServer:467 ] [ DEBUG] Received binary data to redmi6a07, starting with b'\xff\xd8\xff\xe0\x00\x10JFIF'
[05-16 13:43:27.76] [clear_thread_redmi6a07] [ communicator:116 ] [ DEBUG] Storing screenshot...
[05-16 13:43:27.76] [clear_thread_redmi6a07] [ communicator:121 ] [ DEBUG] Done storing, returning
[05-16 13:43:27.76] [clear_thread_redmi6a07] [ WorkerBase:643 ] [ DEBUG] Success retrieving screenshot
[05-16 13:43:27.76] [clear_thread_redmi6a07] [ WorkerBase:610 ] [ DEBUG] _get_trash_positions: checking screen
[05-16 13:43:27.98] [clear_thread_redmi6a07] [ MITMBase:122 ] [ INFO] Found 1 trashcan(s) on screen
[05-16 13:43:27.98] [clear_thread_redmi6a07] [ MITMBase:128 ] [ INFO] Delete old quest 1
[05-16 13:43:27.98] [clear_thread_redmi6a07] [ WebsocketServer:437 ] [ DEBUG] redmi6a07 sending command: screen click 624 834
[05-16 13:43:27.98] [clear_thread_redmi6a07] [ WebsocketServer:452 ] [ DEBUG] To be sent: 48;screen click 624 834
[05-16 13:43:27.98] [clear_thread_redmi6a07] [ WebsocketServer:457 ] [ DEBUG] Timeout: 30
[05-16 13:43:28.00] [ scanner] [ WebsocketServer:310 ] [ DEBUG] Retrieving next message to send
[05-16 13:43:29.58] [ scanner] [ WebsocketServer:386 ] [ DEBUG] Receiving message: 48;OK
[05-16 13:43:29.58] [clear_thread_redmi6a07] [ WebsocketServer:459 ] [ DEBUG] Received answer in time, popping response
[05-16 13:43:29.58] [clear_thread_redmi6a07] [ WebsocketServer:464 ] [ DEBUG] Response to redmi6a07: OK
[05-16 13:43:30.58] [clear_thread_redmi6a07] [ WebsocketServer:437 ] [ DEBUG] redmi6a07 sending command: screen click 360 791
[05-16 13:43:30.58] [clear_thread_redmi6a07] [ WebsocketServer:452 ] [ DEBUG] To be sent: 49;screen click 360 791
[05-16 13:43:30.58] [clear_thread_redmi6a07] [ WebsocketServer:457 ] [ DEBUG] Timeout: 30
[05-16 13:43:30.60] [ scanner] [ WebsocketServer:310 ] [ DEBUG] Retrieving next message to send
[05-16 13:43:32.36] [ scanner] [ WebsocketServer:386 ] [ DEBUG] Receiving message: 49;OK
[05-16 13:43:32.36] [clear_thread_redmi6a07] [ WebsocketServer:459 ] [ DEBUG] Received answer in time, popping response
[05-16 13:43:32.36] [clear_thread_redmi6a07] [ WebsocketServer:464 ] [ DEBUG] Response to redmi6a07: OK
[05-16 13:43:33.36] [clear_thread_redmi6a07] [ WebsocketServer:437 ] [ DEBUG] redmi6a07 sending command: screen click 360 1344
[05-16 13:43:33.36] [clear_thread_redmi6a07] [ WebsocketServer:452 ] [ DEBUG] To be sent: 50;screen click 360 1344
[05-16 13:43:33.36] [clear_thread_redmi6a07] [ WebsocketServer:457 ] [ DEBUG] Timeout: 30
[05-16 13:43:33.37] [ scanner] [ WebsocketServer:310 ] [ DEBUG] Retrieving next message to send
[05-16 13:43:35.33] [ scanner] [ WebsocketServer:386 ] [ DEBUG] Receiving message: 50;OK
[05-16 13:43:35.33] [clear_thread_redmi6a07] [ WebsocketServer:459 ] [ DEBUG] Received answer in time, popping response
[05-16 13:43:35.33] [clear_thread_redmi6a07] [ WebsocketServer:464 ] [ DEBUG] Response to redmi6a07: OK
[05-16 13:43:36.83] [clear_thread_redmi6a07] [ MITMBase:140 ] [ DEBUG] {_clear_quests} finished

Delete lucky eggs if account is > lvl 30

If the scanning account is lvl 30 or above, then it should be ok to delete the lucky eggs from the bag as well. Sometimes it's causing problems because it can't delete other items (for lack of seeing it on one screen because of Lucky Eggs, Camera, Raid Pass and Gifts).

Handle black-screen warning

I have recycle old accounts and get advertisement BLACK/RED

Juste need click for pass warning

MADmin Walker settings bugged "Back to editor" link.

Environment:

• dev branch, this commit (0d97dc2)
• py3.7
• Raspbian Buster

Steps to reproduce:

1. Open MADmin's "Walker" settings tab.
2. On the Basedata of one of the walkers, click Edit.
3. In the Basedata, on one of the assigned areas, click Edit.
4. Top left corner of the screen, click "Back to editor".

Problem: This links to http://host:port/showsettings?area=, an empty area name, causing an empty editor screen.

To fix:
Link back to the "Walker" settings.

Can be the generic /showsettings?area=walker for a quick fix, or the specific selected Walker's settings.

MADmin geofence editor coordinates go out of bounds.

When using MADmin's map to draw a geofence, if you've scrolled past the bounds of the original world, the saved coordinates will be beyond the bounds of the real world, causing the route/position generation to go beyond the geofence and map the world.

Environment:

• dev branch, latest commit (0d97dc2)
• py3.7
• Raspbian Buster

To fix:
Modulo. Keep coordinates within bounds.
(Keep in mind the difference between remainder [= JS' modulo] and least positive residue [= python's modulo].)

Steps to reproduce:

1. Open MADmin's map.
2. To make things easier/quicker, zoom all the way out till you see the world multiple times. (Step can be omitted.)
3. Move the map beyond the bounds of the original world.
4. Draw a geofence in any world that is not the very first one.
5. Save the geofence.

The saved geofence will have coordinates within the bounds of the world you drew it in, beyond the bounds of the first world.

The problem: When generating the route (positions), it will try to generate routes to map the world several times, going beyond the geofence. No error message or crash, just stuck generating positions.

raids_ocr, starting from scratch, some issues in code (patch)

I'm trying to get some proof of concept working for Map-A-Droid with raids/ocr.
In trying to do so, I've found 2 issues that can't seem to work if you ask me. This is getting me a little further to getting stuff done. I've been told not so many people do raid_ocr, so it could be nobody notices.

See patches below:

--- a/db/monocleWrapper.py
+++ b/db/monocleWrapper.py
@@ -410,7 +410,7 @@ class MonocleWrapper(DbWrapperBase):
         )
         data = []
         res = self.execute(query, vals)
-        for (id, distance) in res:
+        for (id, distance, tmplat, tmplon, tmpname, tmpurl) in res:
             data.append(id)
         return data

--- a/ocr/segscanner.py
+++ b/ocr/segscanner.py
@@ -271,10 +271,10 @@ class Scanner:

             log.debug('[Crop: ' + str(raidNo) + ' (' + str(self.uniqueHash) +') ] ' + 'detectGym: Detecting Gym')
             for closegym in closestGymIds:
-                candidates = glob.glob("ocr/gym_img/_" + str(closegym[0]) + "_.jpg")
+                candidates = glob.glob("ocr/gym_img/_" + str(closegym) + "_.jpg")
                 for file in candidates:
                     find_gym = fort_image_matching(raidpic, file, True, float(self.args.gym_detection_value), raidNo, hash, checkX=True, radius=radius, x1=x1, x2=x2, y1=y1, y2=y2)
-                    log.debug('[Crop: ' + str(raidNo) + ' (' + str(self.uniqueHash) +') ] ' + 'detectGym: Compare Gym-ID - ' + str(closegym[0]) + ' - Match: ' + str(find_gym))
+                    log.debug('[Crop: ' + str(raidNo) + ' (' + str(self.uniqueHash) +') ] ' + 'detectGym: Compare Gym-ID - ' + str(closegym) + ' - Match: ' + str(find_gym))
                     if foundgym is None or find_gym > foundgym[0]:
                         foundgym = find_gym, file

& Symbol is not 'percent-encoded', causing issues with pages

Using an & symbol is confirmed causing a error in the Walker section, when navigating to it results in an empty page. If a percent-encoded ampersand is manually typed in, you can access the page info as normal:

Name: This & That
URL: config?type=walker&area=walker&block=fields&edit=This_&That
expected:config?type=walker&area=walker&block=fields&edit=This%26_That

Additionally, it seems that it causes an issues when deleting the Walker

trs_spawn.last_scanned doesn't update correctly

see attached pdf file...................

Anyone having the same behaviour?

Last_scanned.pdf

Not updating gyms on mono and latest commits.

Copy-paste from Discord. It wasn't updating gyms on mono when processing GMO.

[04-01 01:58:18] [  MITMReceiver-0] [  MITMReceiver:144] [    INFO] Processing GMO received from redmi at 1554076698
 [04-01 01:58:18] [  MITMReceiver-0] [ dbWrapperBase:160] [   ERROR] Failed executing query: Not all parameters were used in the SQL statement

Reverting to 7046a71 fixed the problem.

Guess: We missing one %s in inserting query - there is no place for is_ex_raid_eligible to be used. 7 * %s, 8 params ;).
https://github.com/Map-A-Droid/MAD/blob/master/db/monocleWrapper.py#L740

https://github.com/Map-A-Droid/MAD/blob/master/db/monocleWrapper.py#L775

Don't have mono, can't test.

MADmin status "Last date/time of data" never gets filled.

On MADmin's status page, the column "Last date/time of data" never gets filled.

Mapping works, column stays empty:

• PogoDroid and RGC are working properly on the ATV.
• PogoDroid shows last data sent timestamp on the ATV.
• Phone is walking/scanning.
• Scanning results show up in the database, in MADmin, and in OSM-RM. Everything OK.

But an empty column.

Environment:

• dev branch, latest commit (0d97dc2)
• py3.7
• Raspbian Buster
• PogoDroid/RGC on an ATV

Configuration that caused it:

• MITM mon scanning
• 0.5km² geofenced area
• init mode, 10 init rounds

Nothing special. Basic setup according to the docs.

Madmin: Statuspage dont remember sort by settings.

If you sort by a category and the next update loads, the setting is lost and the list is sorted by default again.

Hang on Premium Raid Pass cleanup

When bag is full it attempts to delete the premium raid pass. When it does it it clicks it, attempts to get the maximum amount by clicking + and then

[ MITMBase:97 ] [ WARNING] Timeout waiting for data
[ WorkerQuests:540 ] [ ERROR] Unknown error clearing out Premium Raid Pas
(YES this says Pas and not Pass in the logs)

Statistic (quest) not working.

Going to Statistic in madmin renders no graphs - error below.
Just leaving it here if anyone ever decide to fix it without us changing sql_mode.
This is default MySQL setting.

 [03-30 00:32:05] [      Thread-839] [ dbWrapperBase:127] [   ERROR] Failed executing query: 1055 (42000): Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'database.trs_quest.quest_timestamp' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
 [03-30 00:32:05] [      Thread-839] [           app:1761] [   ERROR] Exception on /get_game_stats [GET]
Traceback (most recent call last):
  File "/home/mad/.local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/mad/.local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/mad/.local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/mad/.local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/mad/.local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/mad/.local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/mad/MAD/MAD/madmin/madmin.py", line 60, in decorated
    return func(*args, **kwargs)
  File "/home/mad/MAD/MAD/madmin/madmin.py", line 1356, in game_stats
    for dat in data:
TypeError: 'NoneType' object is not iterable

Error message meant for no_ocr shown only when no_ocr is disabled.

The configuration option no_ocr will apparently be removed. To make this obvious, an error message was added.

Unfortunately, the message is only shown when no_ocr is disabled instead of enabled.

Environment:

• dev branch, latest commit (0d97dc2)
• py3.7
• Raspbian Buster

To fix:
Right here: https://github.com/Map-A-Droid/MAD/blob/dev/start.py#L240

MAD/start.py

Lines 240 to 241 in 0d97dc2

	if not args.no_ocr:
	logger.error('The argument no_ocr will be eliminated soon - we need ocr (no more option now!)')

Steps to reproduce:

1. Run MAD without no_ocr.

MADmin is vulnerable to directory traversal exploits.

If auth is enabled, auth is required before it can be exploited (at least in this one instance I checked). If auth is disabled, boop server.

Vulnerable code:
Below is the instance where I noticed it.

MAD/madmin/routes/map.py

Line 361 in 05042df

file = open(os.path.join(geofence_file_path, (str(name) + ".txt")), "a")

>>> os.path.join('/a/b/c/', '../../../passwd')
'/a/b/c/../../../passwd'

Steps to abuse:

1. Use MADmin's editor to draw a new geofence.
2. Abuse the "name" field to define a system path.
3. Save.

Any path the user running MAD has access to, ending in .txt (since MAD appends .txt), can be appended.

Possible solutions:

• Check that a file doesn't already exist.
• Ensure the path stays within the MAD configuration folder(s).

Other notes:
There are more things to be done w/ os.path.join:

>>> os.path.join('/a/b/c/', '\\' + '.txt')
'\\.txt'

Disallowed Windows filename characters:

>>> os.path.join('/a/b/c/', '!!##&/\<>#*|:nope')
'/a/b/c/!!##&/\\<>#*|:nope'

Newlines in file names (works on Linux, can be annoying to delete if you don't already know how to):

>>> os.path.join('/a/b/c/', '\n')
'/a/b/c/\n'

The vulnerability extends to all possible uses of open(). I checked one other besides the first one I happened to find, and it's also vulnerable via string formatting and origin injection:

MAD/utils/adb.py

Lines 66 to 76 in c0664c7

	def make_screenshot(self, adb, origin, extenstion):
	try:
	device = self._client.device(adb)
	if device is not None:
	logger.info('MADmin: Using ADB ({})', str(origin))
	result = device.screencap()
	# TODO: adjust with devicesettings
	with open(os.path.join(self._args.temp_path, 'screenshot_%s.png' % str(origin)), "wb") as fp:
	fp.write(result)
	if extenstion == "jpg":
	pngtojpg(os.path.join(self._args.temp_path, 'screenshot_%s.png' % str(origin)))

which gets its origin from a request's arguments:

MAD/madmin/routes/control.py

Lines 119 to 121 in c0664c7

	@auth_required
	def take_screenshot(self, origin=None, adb=False):
	origin = request.args.get('origin')

e.g.:

>>> os.path.join('/a/b/c/', 'screenshot_%s.png' % '/../../../../some-filename')
'/a/b/c/screenshot_/../../../../some-filename.png'

Environment:

• Only tested on dev branch

raid ocr scanning, ongoing raid (without egg) doesn't scan time left

When a raid is found in the "raid boss" state ("MonWithoutEgg" it's called in the code if I'm not mistaken), the OCR tries to scan "Ongoing", and can't get the time from that:

2019-04-11 16:32:44,809 [ Thread-11][ segscanner:65][ DEBUG] [Crop: 1 (1554993164.478644) ] detectRaidTime: detect raidtimer text: 0ng0ing

It should look at the text above that for the real "time left" data.
The result is that "time_end" in the database will be wrong

Add version info for PD, RGC (and the game) to phone control

Since RGC allows to exectue basically all commands, it'd be nice to store several app versions in the DB once a device connects to MAD. It would be helpful to have an overview of these devices for example in the phone control.

Bag is full

When i add luckyegg to "not delete" list so MAD cannot delete anything.. for pokeballs or raspberries delete it needs to scroll down.
This happens even though I have no pokeball in bag but full of raspberries

LastDate of data not pertinent for monitoring

Hi,

For check status all worker lastProtoDateTime is not realy pertinent .

1.) First problem is set to Null when worker register to route ( and never updated )
2.) Is only updated when worker is register and received data waiting

When lastProtoDateTime is very up. Pogodroid can send data and never is recorded .

Recorded get_address
Recorded last GMO
Recorded lastOk RGC ( Worker allways register and RGC connected )
Rename lastProtoDateTime to lastOK

When worker register not set to null ( set timestamp )
When worker unregister unset route

Except my error, never in the code on log last data received from pogodroid

Find ditto based on weather

How to Spot a Ditto in the Wild - posted by Boby360

Partly Cloudy – Search for Yanma, Zubat, Gastly and Gulpin. If weather boosted it’s a Ditto.
Rainy – Yanma is boosted in rainy weather. So, during a rainy day, search for Yanma that doesn’t appear as boosted!
Windy – Yanma and Zubat are boosted by windy weather. Be on the lookout for any Yanma or Zubat that is not weather boosted.
Cloudy – Zubat, Gastly and Gulpin are boosted when cloudy. Search for any Zubat, Gastly or Gulpin that doesn’t appear as boosted.
Foggy – Gastly is boosted when cloudy. Seach for one that is not weather boosted.

Samsung Galaxy S8 not deleting quests

Using a galaxy S8 with softbar disabled. Quests are not deleted, so only 3 quests can be done.

https://cdn.discordapp.com/attachments/507239863918133248/544968931572580364/Screenshot_20190212-195121_Pokemon_GO.jpg

New Pogodroid quest problem

After update Pogodroid to new version, quest scanning is not working.

quest hickups

hi,

great work, just a list of hickups so far with quest scanning:

• check if pogo is in foreground before doing swipes /back / touchevents and don't do random stuff on the desktop
• after restarting mad because two workers send locations at the same time the route starts with first waypoint, ignoring needed waiting time - data time out, restart, loop
• quest rewards (spin stops) are not recognized when deleting quests, implement "accept" or "run away" logic
• don't go 🛒, think this happens when trying to clear bag but not shure
• When cleaning bag: After "Click Gift / Raidpass" somehow the nearby pokemon gets clicked and hangs in the Pokestop image fullscreen display (trytodelete item-> "back"-> lower right click-> small swipe->click pokemon navigation -> pokestop details) ->reboot, loop
• Some stops are not reached, location is some meters off but would be reachable if Orientation would be rotated a bit not quest specific
• Save some time and clear quest while walking to stop

Not receiving all the gym updates (rmWrapper) as webhooks

No idea if this problem applies to mono, not even looking there.
No idea if this problem applies to other webhooks types, just migrating, this one was broken :)

Webhook worker is trying to detect changed gyms with last_modified field and 10 seconds sleep using time.time().
rmWrapper fills last_modified field in DB as the last_modified_timestamp_ms field from POGO protos (https://github.com/Map-A-Droid/MAD/blob/master/db/rmWrapper.py#L790).

Situation:

Someone put pokemon in gym / change gym color at 20:00:00, my worker goes there 20:00:44, and let's say process GMO at 20:00:45 and in the same second sleep ends for webhook worker.

last_modified in DB will be set to 20:00:00, but because it's currently 20:00:45 the earlier changes detected by 10 seconds sleeping webhook worker will be AFTER 20:00:35 - ergo we will never get those gym changes. Not this run, not next run, never. It looks like actual gym changes must happen within 10 seconds to be processed by webhook worker.

No idea how you want to approach. The whole idea of having last_modified from actual POGO protos is amazing, but won't work this way.

Timeout during init of worker CRITICAL kills RouteManager

[08-15 15:36:47.22] [             a95x] [         WorkerMITM:89  ] [    INFO] Walking roughly: 14.072936377982668
[08-15 15:36:49.05] [       MainThread] [  MITMDataProcessor:68  ] [ SUCCESS] Processing GMO received from a95x. Received at 2019-08-15 15:36:48
[08-15 15:36:58.94] [             a95x] [         WorkerBase:253 ] [   ERROR] Timeout during init of worker a95x
[08-15 15:36:58.94] [             a95x] [         WorkerBase:302 ] [    INFO] Internal cleanup of a95x started
[08-15 15:36:58.94] [             a95x] [         WorkerBase:305 ] [    INFO] Internal cleanup of a95x signalling end to websocketserver
[08-15 15:36:58.94] [ MainProcess|a95x] [   RouteManagerBase:151 ] [    INFO] Worker a95x unregistering from routemanager RajdyWolaIOkolicePRIO
[08-15 15:36:58.94] [ MainProcess|a95x] [   RouteManagerBase:154 ] [    INFO] Cleanup routepool for origin a95x
[08-15 15:36:58.94] [             a95x] [       communicator:22  ] [    INFO] Communicator of a95x acquiring lock to cleanup worker in websocket
[08-15 15:36:58.95] [             a95x] [       communicator:25  ] [    INFO] Communicator of a95x calling cleanup
[08-15 15:36:58.95] [             a95x] [         WorkerBase:309 ] [    INFO] Stopped Route
[08-15 15:36:58.97] [             a95x] [         WorkerBase:316 ] [    INFO] Internal cleanup of a95x finished
[08-15 15:36:58.98] [      worker_a95x] [         WorkerBase:227 ] [    INFO] Worker a95x stopped gracefully
[08-15 15:36:59.05] [       MainThread] [  MITMDataProcessor:68  ] [ SUCCESS] Processing GMO received from a95x. Received at 2019-08-15 15:36:58
[08-15 15:37:02.98] [             a95x] [           MITMBase:63  ] [    INFO] Waiting for data after 2019-08-15 15:36:45.957604
[08-15 15:37:03.99] [             a95x] [              start:66  ] [CRITICAL] Unhandled patched exception (<class 'KeyError'>): "'a95x'".
Traceback (most recent call last):
  File "start.py", line 53, in run_thread
    run_thread_old(*args, **kwargs)
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/mad/MAD/worker/WorkerBase.py", line 411, in _main_work_thread
    self._post_move_location_routine(time_snapshot)
  File "/home/mad/MAD/worker/WorkerMITM.py", line 30, in _post_move_location_routine
    self._wait_for_data(timestamp)
  File "/home/mad/MAD/worker/MITMBase.py", line 124, in _wait_for_data
    self.worker_stats()
  File "/home/mad/MAD/worker/MITMBase.py", line 240, in worker_stats
    routemanager_status = self._mapping_manager.routemanager_get_route_stats(self._routemanager_name, self._id)
  File "<string>", line 2, in routemanager_get_route_stats
  File "/usr/lib/python3.6/multiprocessing/managers.py", line 772, in _callmethod
    raise convert_to_error(kind, result)
KeyError: 'a95x'
[08-15 15:37:10.29] [       MainThread] [  MITMDataProcessor:68  ] [ SUCCESS] Processing GMO received from a95x. Received at 2019-08-15 15:37:09
[08-15 15:37:40.62] [       MainThread] [  MITMDataProcessor:68  ] [ SUCCESS] Processing GMO received from a95x. Received at 2019-08-15 15:37:39
and so on and so on.

avatar not moving, standing in one place. GMO still processed (from one place), so no auto-restart by MAD. MAD full restart needed.

MADmin: Doesent save password correctly if using numbers only

I noted a small bug: if you use only numbers in password under AUTH it writes the password without "" in mappings.json

weather_hook, Monocle schema, missing index/broken logic, 100% CPU usage.

Everything started after #7b88f2bff6626a736fa320747148fe06374c5d78 and chages to db/monocleWrapper.py

Before changes (with weather_hook enabled) query was:

 query_weather = (
             "INSERT INTO weather (s2_cell_id, `condition`, alert_severity, warn, day, updated) "
                number_of_rows = len(res)	            "VALUES (%s, %s, %s, %s, %s, %s) "
            "ON DUPLICATE KEY UPDATE `condition`=VALUES(`condition`), alert_severity=VALUES(alert_severity), "
            "warn=VALUES(warn), day=VALUES(day), updated=VALUES(updated)"
        )

which make sense - insert new row, on duplicate key update conditions.

However, if we look at "recommended" cleandb.sql

CREATE TABLE `weather` (
  `id` int(11) NOT NULL,
  `s2_cell_id` bigint(20) DEFAULT NULL,
  `condition` tinyint(3) UNSIGNED DEFAULT NULL,
  `alert_severity` tinyint(3) UNSIGNED DEFAULT NULL,
  `warn` tinyint(1) DEFAULT NULL,
  `day` tinyint(3) UNSIGNED DEFAULT NULL,
  `updated` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

ALTER TABLE `weather`
  ADD PRIMARY KEY (`id`);

ALTER TABLE `weather`
  MODIFY `id` int(11) NOT NULL AUTO_INCREMENT;

The only key is id int(11) which is auto increment and not used in insert or update... [should be: s2_cell_id] - so with that kind of query every time MAD receives weather info from client it will INSERT new row. So after not even a month of running my stats are:

mysql> select count(*) from weather;
+----------+
| count(*) |
+----------+
|   175486 |
+----------+

mysql> select COUNT(DISTINCT(`s2_cell_id`)) from weather;
+-------------------------------+
| COUNT(DISTINCT(`s2_cell_id`)) |
+-------------------------------+
|                             7 |
+-------------------------------+
1 row in set (0.07 sec)

I haven't really checked what the code by @cecpk is supposed to do, but it hammers my CPU to almost death (db on same server as MAD) and the whole refactor was probably not needed - all we need is index/key on s2_cell_id?

No idea how it goes with 2nd schema - never used, never tested, please double check.

I just run DELETE FROM `weather`; and I hope current code will handle duplicates, will report later.

EDIT: Yep, current code handles duplicates, only one entry per s2_cell_id and no cpu ⚒

Errors causing process count to go crazy

May be a misleading title (not sure what to call it!), but one phone was generating errors (couldn't see what the phone was doing) and the error log was filling with entries like this:

�[33m [02-12 19:09:30] [           razer] [    WorkerBase:351] [ WARNING] Turning screen on�[0m
�[33m [02-12 19:09:32] [           razer] [    WorkerBase:355] [ WARNING] turnScreenOnAndStartPogo: (Re-)Starting Pogo�[0m
�[33m [02-12 19:09:33] [           razer] [    WorkerMITM:102] [ WARNING] Turning screen on�[0m
�[31m [02-12 19:09:35] [         scanner] [WebsocketServer:219] [   ERROR] retrieve_next_send: connection closed, returning None�[0m
�[33m [02-12 19:09:35] [         scanner] [WebsocketServer:247] [ WARNING] Connection closed in consumer_handler�[0m
�[33m [02-12 19:09:37] [         scanner] [WebsocketServer:226] [ WARNING] Consumer handler of razer starting�[0m
�[33m [02-12 19:09:37] [           razer] [    WorkerBase:351] [ WARNING] Turning screen on�[0m
�[31m [02-12 19:09:39] [         scanner] [WebsocketServer:219] [   ERROR] retrieve_next_send: connection closed, returning None�[0m
�[33m [02-12 19:09:40] [           razer] [    WorkerBase:355] [ WARNING] turnScreenOnAndStartPogo: (Re-)Starting Pogo�[0m
�[31m [02-12 19:09:40] [           razer] [    WorkerBase:179] [   ERROR] Timeout during init of worker razer�[0m
�[33m [02-12 19:09:41] [         scanner] [WebsocketServer:226] [ WARNING] Consumer handler of razer starting�[0m
�[33m [02-12 19:09:41] [           razer] [    WorkerBase:351] [ WARNING] Turning screen on�[0m
�[31m [02-12 19:09:43] [         scanner] [WebsocketServer:219] [   ERROR] retrieve_next_send: connection closed, returning None�[0m
�[33m [02-12 19:09:45] [         scanner] [WebsocketServer:226] [ WARNING] Consumer handler of razer starting�[0m
�[33m [02-12 19:09:46] [         scanner] [WebsocketServer:247] [ WARNING] Connection closed in consumer_handler�[0m
�[31m [02-12 19:09:46] [         scanner] [WebsocketServer:219] [   ERROR] retrieve_next_send: connection closed, returning None�[0m
�[33m [02-12 19:09:47] [           razer] [    WorkerBase:355] [ WARNING] turnScreenOnAndStartPogo: (Re-)Starting Pogo�[0m
�[31m [02-12 19:09:47] [           razer] [    WorkerBase:179] [   ERROR] Timeout during init of worker razer�[0m
�[33m [02-12 19:09:48] [         scanner] [WebsocketServer:226] [ WARNING] Consumer handler of razer starting�[0m
�[33m [02-12 19:09:48] [           razer] [    WorkerBase:351] [ WARNING] Turning screen on�[0m
�[33m [02-12 19:09:51] [           razer] [    WorkerBase:355] [ WARNING] turnScreenOnAndStartPogo: (Re-)Starting Pogo�[0m

It looks as though each attempt was starting a new thread. After an hour or so, there were over 500 PIDs and my server ground to a halt.

import_allspawns.sh -> Config.ini not found

I wanted to extract the spawnpoints from my old rocketmap database (which I also use directly as the database connection in the config.ini and for running MAD).
So I edited file import_allspawns.sh with "database_old" (Which is same connection than in config.ini). The path to the config.ini is default ( configs/config.ini )

I tried to execute the import_allspawns.sh script in folder /scripts and got the error, that the config.ini could not be found. I also tried it with the complete path to config.ini in the script, and also with the a copy of config.ini in script folder. Nothing worked as excpected.
I am using Ubuntu server and the Scanner itself with MITM is running fine (with one Device).
I also tried to run from directory MAD directory as: sh scripts/import_allspawns.sh but I got the same error.

Error
import_allspawns.sh: 35: import_allspawns.sh: [[: not found
Unable to find your MAD config. You should be running this in the MAD directory

Quests MITM stuck at GPS signal Error

Adding this one here for reference.

When using mock locations, it's quite normal to have an Error 11 on pogo startup. It usually resolves when start.py start to send geofix.

Issue is that when using quests scan, it will detect the Error 11 :

[ WorkerBase][ WARNING] _check_pogo_main_screen: GPS signal error

and after 5 errors in a row it will restart pogo :

[ WorkerBase][ ERROR] _check_pogo_main_screen: Red error multiple times in a row, restarting

but as far as it will never send any geofix, it will remain stuck in a reboot loop forever.

Solution would be to send a geofix when GPS signal error is detected so that it resolves itself and quest worker can continue its normal work.

Compute height for login PTC not correct

Height for login PTC not correct

Issue with rotate playerlvl on dev

MAD/worker/WorkerQuests.py

Line 351 in b4e8b3f

if self.get_devicesettings_value('rotate_on_lvl_30', False) and self._mitm_mapper.get_playerlevel() >= 30 \

[07-22 11:11:37.48] [           moto01] [              start:66  ] [CRITICAL] Unhandled patched exception (<class 'TypeError'>): "get_playerlevel() missing 1 required positional argument: 'origin'".
Traceback (most recent call last):
  File "start.py", line 53, in run_thread
[07-22 11:11:37.48] [           moto01] [              start:66  ] [CRITICAL] Unhandled patched exception (<class 'TypeError'>): "get_playerlevel() missing 1 required positional argument: 'origin'".
    run_thread_old(*args, **kwargs)
  File "/usr/local/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/src/app/worker/WorkerBase.py", line 411, in _main_work_thread
    self._post_move_location_routine(time_snapshot)
  File "/usr/src/app/worker/WorkerQuests.py", line 348, in _post_move_location_routine
    if self.get_devicesettings_value('rotate_on_lvl_30', False) and self._mitm_mapper.get_playerlevel() >= 30 \
  File "<string>", line 2, in get_playerlevel
  File "/usr/local/lib/python3.7/multiprocessing/managers.py", line 834, in _callmethod
    raise convert_to_error(kind, result)
TypeError: get_playerlevel() missing 1 required positional argument: 'origin'

I'm not actually familiar enough with the codebase to offer a PR at this time.

Deleting an Area does not remove it from Walker.

Deleting an Area via the Area settings does not remove the previous assignments, causing Walkers' settings to show assigned Areas that no longer exist.

I'm not sure if this has additional consequences, I didn't let the non-existing Area's assignment stick around to figure that out. The mismatch in data shouldn't exist, otherwise it's just a matter of time till it breaks something.

Area in this screenshot was deleted but is still assigned:

Environment:

• dev branch, latest commit (0d97dc2)
• py3.7
• Raspbian Buster

To fix:
Automatically remove all assignments of Areas that are deleted.

Steps to reproduce:

1. Configure a working MAD setup with at least one Area assigned to a Walker.
2. In the Area settings tab, delete the assigned Area.
3. Open your Walker's settings (where you see the assigned areas).

Your Walker is still assigned to an Area that doesn't exist.

Move configuration to DB

With the latest MADmin rework a huge milestone has been reached to move things completely into the database.

With this change a few more extras should be easier to implement like moving geofences and calculated routes to the DB and reuse geofences in multiple areas.

General RouteManager rework

Right now, per area is only one single RouteManager. These RouteManagers are assigned to devices based on the schedule in a Walker. If multiple Walkers run the same area/RouteManager at the same time, devices will be distributed in that area.

For things like leveling mode, this might cause issues since you don't want devices to be distributed but all should start at the first position in the route.

The same issue could affect other modes. Making this behaviour optional would be a nice addition.

sightings table can grow big

MAD expects mono to have some key that will trigger ON DUPLICATE KEY (https://github.com/Map-A-Droid/MAD/blob/master/db/monocleWrapper.py#L544) but mono keys are (almost) useless for us. Now MAD will insert each new sighting as new row and people scanning mons will end with a lot of records in short time. In theory except database overhead there are no cons, but the moment we would try to JOIN something with that table the results will be horrible [and longer query for webhooks due to more rows ;)].
-edited, deleted-
4. Add (optional in config) method to delete old records on MAD startup.

cleandb.sql extract:

CREATE TABLE `sightings` (
  `id` bigint(20) NOT NULL,
  `pokemon_id` smallint(6) DEFAULT NULL,
  `spawn_id` bigint(20) DEFAULT NULL,
  `expire_timestamp` int(11) DEFAULT NULL,
  `encounter_id` bigint(20) UNSIGNED DEFAULT NULL,
  `lat` double(18,14) DEFAULT NULL,
  `lon` double(18,14) DEFAULT NULL,
  `atk_iv` tinyint(3) UNSIGNED DEFAULT NULL,
  `def_iv` tinyint(3) UNSIGNED DEFAULT NULL,
  `sta_iv` tinyint(3) UNSIGNED DEFAULT NULL,
  `move_1` smallint(6) DEFAULT NULL,
  `move_2` smallint(6) DEFAULT NULL,
  `gender` smallint(6) DEFAULT NULL,
  `form` smallint(6) DEFAULT NULL,
  `cp` smallint(6) DEFAULT NULL,
  `level` smallint(6) DEFAULT NULL,
  `updated` int(11) DEFAULT NULL,
  `weather_boosted_condition` smallint(6) DEFAULT NULL,
  `weather_cell_id` bigint(20) UNSIGNED DEFAULT NULL,
  `weight` double(18,14) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

ALTER TABLE `sightings`
  ADD PRIMARY KEY (`id`),
  ADD UNIQUE KEY `timestamp_encounter_id_unique` (`encounter_id`,`expire_timestamp`),
  ADD KEY `ix_sightings_encounter_id` (`encounter_id`),
  ADD KEY `ix_sightings_expire_timestamp` (`expire_timestamp`);

ALTER TABLE `sightings`
  MODIFY `id` bigint(20) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=3170831;

Toggle Magisk on/off causes 'device not supported' error and SafetyNet=false

I've noticed that after 2 months of working 24/h my 2 phones Nexus5 and GalaxyS5 started having issues with 'device not compatible' error. I'm pretty sure it started happening after I pulled commit #399. I've noticed that everytime I have this error, POGO in Magisk Hide is not fully checked.

After I fully check POGO in Magisk Hide, clear game data and reboot phone everything is fine.

IF this feature rly helps some people, maybe don't enable it by default but give us an option to enable/disable it?

Starting RGC without PoGodroid and reboot True results in an Error

Hi.

When i start RGC and Parameter reboot is true, i get an Error after Message "Timeout waiting for data"
PogODroid is not started.
reboot_thresh is set to 2

[02-10 18:20:26] [ scanner] [WebsocketServer:67] [ INFO] Waiting for connection...
[02-10 18:20:26] [ scanner] [WebsocketServer:92] [ INFO] Client s5mini1 registering
[02-10 18:20:26] [ scanner] [ timer:17] [ INFO] [s5mini1] - check for Switchtimer
[02-10 18:20:26] [ switchtimer] [ timer:34] [ INFO] [s5mini1] - Starting Switchtimer
[02-10 18:20:27] [ scanner] [WebsocketServer:226] [ WARNING] Consumer handler of s5mini1 starting
[02-10 18:20:27] [ s5mini1] [ WorkerBase:352] [ WARNING] turnScreenOnAndStartPogo: (Re-)Starting Pogo
[02-10 18:20:29] [ s5mini1] [ WorkerMITM:114] [ WARNING] startPogo: Starting pogo...
[02-10 18:21:29] [ s5mini1] [ WorkerMITM:93] [ INFO] MITM worker starting
[02-10 18:21:29] [ s5mini1] [RouteManagerBase:242] [ INFO] Round of route Hockenheim is moving to 49.414038, 8.52543 for a priority event
[02-10 18:21:29] [ s5mini1] [RouteManagerBase:279] [ INFO] Hockenheim done grabbing next coord, releasing lock and returning location: 49.414038, 8.52543
[02-10 18:21:29] [ s5mini1] [ WorkerMITM:39] [ INFO] main: Moving 5556401.284728093 meters to the next position
[02-10 18:21:29] [ s5mini1] [ WorkerMITM:47] [ INFO] main: Teleporting...
[02-10 18:21:29] [ s5mini1] [ WorkerMITM:60] [ INFO] Need more sleep after Teleport: 8 seconds!
[02-10 18:21:29] [ s5mini1] [ WorkerMITM:69] [ INFO] Walking a bit: 13.269789513563467
[02-10 18:21:38] [ s5mini1] [ WorkerMITM:85] [ INFO] Sleeping 8
[02-10 18:21:46] [ s5mini1] [ WorkerMITM:168] [ INFO] Waiting for data after 1549819289, error count is at 0
[02-10 18:23:00] [ s5mini1] [ WorkerMITM:249] [ WARNING] Timeout waiting for data
Exception in thread s5mini1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/root/MAD/worker/WorkerBase.py", line 286, in _main_work_thread
self._post_move_location_routine(time_snapshot)
File "/root/MAD/worker/WorkerMITM.py", line 28, in _post_move_location_routine
self.__wait_for_data(timestamp)
File "/root/MAD/worker/WorkerMITM.py", line 258, in __wait_for_data
and self.__reboot_count > self._devicesettings.get("reboot_thresh", 5)
AttributeError: 'WorkerMITM' object has no attribute '_WorkerMITM__reboot_count'

websockets.exceptions.ConnectionClosed: WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason

During some analysis, we discovered that from time to time a connection problem is ruining the whole webSocket. It looks like a device disconnect, which causes an unhandled error which beaks the socket.
After that all devices are failing and can't be recovered

The exception is :
`2019-03-12 15:31:59,154 [ scanner][WebsocketServer][ ERROR] retrieve_next_send: connection closed, returning None
2019-03-12 15:31:59,156 [ scanner][ protocol][ WARNING] Unexpected exception in keepalive ping task
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 674, in transfer_data
message = yield from self.read_message()
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 742, in read_message
frame = yield from self.read_data_frame(max_size=self.max_size)
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 815, in read_data_frame
frame = yield from self.read_frame(max_size)
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 884, in read_frame
extensions=self.extensions,
File "/usr/local/lib/python3.6/dist-packages/websockets/framing.py", line 99, in read
data = yield from reader(2)
File "/usr/lib/python3.6/asyncio/streams.py", line 672, in readexactly
raise IncompleteReadError(incomplete, n)
asyncio.streams.IncompleteReadError: 0 bytes read on a total of 2 expected bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 984, in keepalive_ping
ping_waiter = yield from self.ping()
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 583, in ping
yield from self.ensure_open()
File "/usr/local/lib/python3.6/dist-packages/websockets/protocol.py", line 646, in ensure_open
) from self.transfer_data_exc
websockets.exceptions.ConnectionClosed: WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason
2019-03-12 15:31:59,159 [ scanner][WebsocketServer][ INFO] consumer or producer of xxx stopped, cancelling pending tasks
`

webhook worker is killed by an unhandled exception

the webhook worker is killed by an unhandled exception:

[04-28 04:27:50.92] [ webhook_worker] [ dbWrapperBase:124 ] [ ERROR] Failed executing query: 2055: Lost connection to MySQL server at 'xxx.xxx.xxx.xxx:xxxx', system error: 104 Connection reset by peer
[04-28 04:27:52.98] [ webhook_worker] [ start:57 ] [CRITICAL] Unhandled patched exception (<class 'mysql.connector.errors.OperationalError'>): "MySQL Connection not available.".

in this case what a local disconnect (24 hours internet discconect).
The webhook worker only works again when I restart MAD.
Is it possible that the webhook worker continues to run and tries until he can send again without restarting MAD?

RemoteGPSController Issue

After installing and systemizing the app I granted permissions for gps and storage, then I said something is using app overlay, but not sure what to disable here.

Then I stuck at Waiting for permissions screen.

Edit: I'm using a Oneplus2