stelios-c/gps_analysis

Analysis of GPS interruption data in public domain

• 👋 Hi, I’m @stelios-c
• 👀 I’m interested in Data, Space, Telecommunications, Transport.
• 🌱 I'm revising autocorrelation and stationary processes.
• 💞️ I’m looking to collaborate on analysing open data and improving open source tools.
• 📫 How to reach me: comment/issue on github or message me on Linked In.

Portfolio:

• Report on GPS disruptions 2018-2021
• Analysis of GPS disturbances in Finland
• Recreation (no solutions) of Efron and Hastie's Computer Age Statistical Inference
  • kidneys dataset analysis
  • leukemia dataset analysis
• Exploration of Eurostat data
• Directory diff using pandas
• Anscombe quartet data analysis

gps_analysis's Issues

make LOCODE detailed format consistent with old LOCODE file

old file: "CY-LIM"
new detailed file: "CY LIM"

issues when comparing country to detailed location

only do this for df[country.notna() & location.notna()]

Licensing

This work is using web scraped wikipedia data. Wikipedia is licensed under CC BY-SA 3.0. Therefore this repo has to be licensed with the compatible license, CC BY-SA 4.0. This license is not recommended for software.

A software license that is compatible with CC BY-SA 4.0, though not with CC BY-SA 3.0, is GPLv3. Therefore the best approach might be to separate data from analysis in two separate repos, any "remixed" wikipedia data released with CC By-SA 4.0, and the code that does further analysis released with GPLv3

Using local date instead of UTC date or UTC timestamp

Move from .replace('CYPRUS, LIMASSOL','LIMASSOL, CYPRUS') to tokenising and matching against country names

try and reuse code from #7

Location spans multiple countries

e.g.
TOKYO, JAPAN. SANTA ROSA, CA

Local language vs international names

LOCODE file has Lemesos instead of Limassol.

some of the previous country names also include notes in brackets etc.

and for User Notes we can't match Taiwan. List of countries has

    and for User Notes we can't match Taiwan. List of countries has 

(common name: Taiwan)

Originally posted by @stelios-c in #16 (comment)

But a few other countries with common names

Iran, Saudi Arabia and United states in list with other form?

split timezone

it can be an extra clue on location

Deal with non 2 letter code common locations which are US states

clear special characters from UN LOCODE file

Not every two letter word is a US state

non capitalised me, in, oh, unlikely to be Maine, Indiana, Ohio

Split NAVCEN response to standard parts

• The GPS Operations Center reviewed the GPS Constellation and Control Segment, there are no known anomalies that might affect GPS signal integrity at the time and vicinity of the reported problem.
• Space weather was reviewed and found unlikely to have impacted GPS performance.
• There were no authorized GPS tests in the area.
• No correlating reports from interagency partners.
• The report of interference is similar to that outlined by U.S. Maritime Administration in Maritime Advisory 2021-004-Various GPS Interference. User encouraged to report the issue to the nearest communication authority.

remove headers before consolidating locode detailed dataframes into one

otherwise we get data rows that contain header

Move from .replace('WASHINGTON DC','DC') to tokenising and matching against list of two letter codes

processing should write out full locode where available

The idea is to group certain locations of a country for analysis e.g. Sicily but not Venice could be affected by Libya, Iskenderun but not Istanbul would be affected by Syria etc.

Initials

e.g. UAE

use pd.series.split where possible

instead of

series.apply(lambda x: x.split())

Split short names of sovereign states

Sometimes LOCODE file has both local language and international country names

    In some cases it has both names, needs to be split out e.g. 

Abū Z̧aby [Abu Dhabi]

Originally posted by @stelios-c in #20 (comment)

Georgia US vs Georgia Caucasus

Cartersville Georgia currently getting mapped to country in Caucasus

Sovereign state name made of more than one word, separated by spaces

Space:

• Saudi Arabia

Originally posted by @stelios-c in #11 (comment)

Split user from NAVCEN section in descriptions

fix location parsing with latest data in new notebook for US state and country only

don't hardcode index of UK two letter code

If we don't tokenise, FJ matches everything. If we do, we miss two word country names.

Issue with matching the locode placenames against the whole string vs the tokenised version on each side.

return LOCODE when you match more detailed location

add a citation file

see https://citation-file-format.github.io/

replace "Cyprus, Limassol" with "Limassol, Cyprus"

rationalise dataframe names for locode detailed

locode list maybe? then consolidated before writing to csv, and detailed when reading from csv?

inefficient for loop through big detailed locode list

if we match detailed locode in same country as matched country, return detail

No detailed LOCODE website for BV

Only country not found from those listed in
https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2
We can safely ignore it

Bouvet Island is an island claimed by Norway, and declared an uninhabited protected nature reserve.

US-IN disappears by the end of the processing notebook