goodmorninggoaway/vendex

Read Me

Scripts

Run commands with npm run xzy. Or you can directly run the node script. but base it on the script in package.json's scripts stanza.

• npm run create-db: Drops and recreates the DB. You'll want to do this once before you start, since the current version on the VM is outdated. However, don't run it to frequently. Google has a limit of 2500 requests per day; the responses are stored in the DB to avoid hitting the limit during testing.
• npm run import-th-territories: Imports the territories from Territory Helper's territory export. This is used by the export-th script to associate new locations to the correct territory
• npm run import-alba-territories: This is a synthesized file with a single record - the congregation boundaries.
• npm run import-th: Import the location export file from Territory Helper.
• npm run import-alba: Import the location export file from Alba.
• npm run export-th: Generates a file that can be imported into Territory Helper. Also includes additional data to explain conflicts or ambiguous matches.

Sequence Diagrams

Overview

• Import territories
• Import files from the respective systems
• Generate an export file

Territory Import

Importing Locations from a Source

Exporting Locations in a Source's Format

Diagrams generated using PlantUML

vendex's Issues

Direct input of alba style data

In addition to supporting uploads of alba-formatted data, there should be a form-based input to support edge cases.

Add About / Home page

Add a page that is both a nav element and the introductory home page of the app

• It should include a visualization of the entire process.
• Not a sequence diagram; that already exists in docs for devs
• Elements should be navigable to take the user to the correct wizard. Generally we want them to start from the beginning and complete the entire process in one sitting.

Attempt to automatically pull data from alba in an iframe

Convert POC to REST API

Rule 2 - FLC no longer tracks a location

TH Type = Language && A.Kind = Foreign-language && A.Status = '' && A.Language = English -> Remove from TH

AC:

1. If Location exists in TH and A AND (TH Type = Language) AND (A.Kind = Foreign-language && A.Status = '' && A.Language = [Congregation Language]), add to "TO DELETE" output
2. If Location does not exist in TH, it should not appear in output
   3. In all other cases, the Location should appear in the "TO DELETE" output as a conflict

Rule 12 - Apartments

If an address includes a apartment unit letter or number, and no previous unit exists in TH for that address, create a location in TH with type=unit and add the address/unit.

AC:

1. Refer to Rule 1 (#1). Extend by adding the unit

Guide users through the entire process with a wizard of wizards

Follow the TurboTax pattern, where one wizard leads to another. This guides the user to the next step rather than artificially separating the Alba import from the TH export.

The current implementation is future-looking where TH will be API-driven and there will also be a TH -> Alba export, but we can implement those as a "Choose your experience" set of wizards and optimize accordingly.

Change geocoding provider

Change geocoding provider, or better yet, support more than one. We will easily start hitting Google's limit and they have recently changed their terms and pricing. Azure Maps and Mapfit seem sustainably priced.

Export TH FL locations

Need to be export interested FL locations from TH to some human-readable format. Alba has no import and no API and can't automate form filling due to iframe restriction.

Process alba imports atomically instead of in aggregate

Decouple congregations from integration agreements

Integration agreements are used by alba imports because of the agregated nature of the alba export.

Summarize result of Alba import

Summary should include:

• count successful locations
• count deleted
• List potential duplicate locations

This should be persisted in the alba_location_import table.

Refactor "Old Apex Spanish" format to be more generic

"Old Apex Spanish" is a temporary shim. It should be refactored to mean any non-alba data that is formatted the same as the alba data.

Alba import history

Now that we have a alba_location_import table, we can show a history of Alba imports. This will aid debugging.

Reevaluate location-territory mapping when TH territories are imported

Since territory boundaries change arbitrarily, locations should be remapped when territories are uploaded. Currently, only changed/added alba locations are updated on export

Load an Alba data set

Vertical spike of initial basic infrastructure setup to get a single Alba set imported.

Rule 1 - CRU FLC Location in TH

TH.Location Type = Language and TH.Location Status = Do not call and (A.Status != Do not call and A.Language != English)

AC:

1. If Location exists in A, but not TH and A.Status != Do not call and A.Language != English, the location is included in the TH "TO INSERT" output
2. If Location exists in A and TH and A.Status != Do not call and A.Language != English, the location is included in the TH "TO UPDATE" output
3. If Location no longer exists in A, but exists in TH and TH.Location Type = Language and TH.Location Status = Do not call, then location is included in the TH "TO DELETE" output
4. Any output should have TH.Location Type = Language and TH.Location Status
5. If a Location exists in both A and TH and (A.Status != Do not call and A.Language != English) and (TH.Location Type != Language), the location is included in the "TO UPDATE" output AND identified as a conflict

Consider rewriting backend in a typed language

I started with Node for expediency and cheapness, but I think the lack of types is getting in the way since this is essentially an ETL app. I have trouble keeping models straight when fixng bugs or adding new features and IDE assistance would improve this.

Prefer Kotlin, Java, Go, or Typescript (still JS but possibly as much work as converting to Kotlin)