flightplan-tool/flightplan

Search for award inventory using Node.js and Headless Chrome!

flightplan's Issues

[SQ] error Could not parse awards: Error: Failed to parse flights from HTML

SQ introduce a Google 2CAPTCHA for login, can answer the CAPTCHA manually but got the issue when parse the response from the pages, is there any fix on the parser script? Thanks.

[SQ] DEPARTURE [HKG -> SIN] - 2022-02-02 (1 Passenger)
[SQ] ARRIVAL [SIN -> HKG] - 2022-02-05
[SQ] error Could not parse awards: Error: Failed to parse flights from HTML
[SQ] DEPARTURE [HKG -> SIN] - 2022-02-03 (1 Passenger)
[SQ] ARRIVAL [SIN -> HKG] - 2022-02-06
[SQ] error Could not parse awards: Error: Failed to parse flights from HTML

BA Captcha timeout

So presumably at 45 queries or so, BA is now prompting with a captcha. Not sure yet if it appears again, will update. I haven't tried to see if multiple sub-45 day searches trigger this as well.

Is the cool-down period adjustable? Is that time or number of queries based? If so I can play with it and see what works.

As it is, the immediate problem is I don't always get a chance to answer the captcha as the search refreshes with the next query just before I'm about to hit submit. If the cool-down isn't the answer, then the parser needs to detect the captcha and wait much longer.

Asiamiles waitlist

Looks like the parser doesn't work on 'waitlist awards', I got no data at the GUI which showing 0 awards.

PS C:\Users\xxxx\desktop\flightplan> flightplan stats
Opening database...
Analyzing search requests...
CX:
HKG => KIX (economy, 1x): 5
KIX => HKG (economy, 1x): 5

sqlite3:
sqlite> select * from awards_requests;
1|CX|HKG|KIX|2018-09-01||economy|1|data/CX-HKG-KIX-2018-09-01-1532934841579.html.gz|3|2018-07-30 07:15:06
2|CX|KIX|HKG|2018-09-01||economy|1|data/CX-KIX-HKG-2018-09-01-1532934841579.html.gz|3|2018-07-30 07:15:43
3|CX|HKG|KIX|2018-09-02||economy|1|data/CX-HKG-KIX-2018-09-02-1532934841579.html.gz|3|2018-07-30 07:16:22
4|CX|KIX|HKG|2018-09-02||economy|1|data/CX-KIX-HKG-2018-09-02-1532934841579.html.gz|3|2018-07-30 07:17:00
5|CX|HKG|KIX|2018-09-03||economy|1|data/CX-HKG-KIX-2018-09-03-1532934841579.html.gz|3|2018-07-30 07:17:37
6|CX|KIX|HKG|2018-09-03||economy|1|data/CX-KIX-HKG-2018-09-03-1532934841580.html.gz|3|2018-07-30 07:18:20
7|CX|HKG|KIX|2018-09-04||economy|1|data/CX-HKG-KIX-2018-09-04-1532934841580.html.gz|3|2018-07-30 07:19:00
8|CX|KIX|HKG|2018-09-04||economy|1|data/CX-KIX-HKG-2018-09-04-1532934841580.html.gz|3|2018-07-30 07:19:39
9|CX|HKG|KIX|2018-09-05||economy|1|data/CX-HKG-KIX-2018-09-05-1532934841580.html.gz|3|2018-07-30 07:20:19
10|CX|KIX|HKG|2018-09-05||economy|1|data/CX-KIX-HKG-2018-09-05-1532934841580.html.gz|3|2018-07-30 07:20:56

one of the jpg under "data/":

License

First of all, thank you so much for this awesome project.

Just wondering, could you please pick a License for this repo? so we can use it without worrying about any potential issue.

SQ Login TImeout

Trying to search SQ (0.3.28) and getting timeouts on login, it never succeeds, is this just me or an issue?

andrew@fp:~$ flightplan search -w SQ -f MEL -t SIN -c first -s 2018-12-06 -e 2018-12-31 -q 2 -h --force
flightplan search 0.3.28
Searching 26 days of award inventory: 12/06/2018 - 12/31/2018
[SQ] DEPARTURE [SIN -> MEL] - 2018-12-06 (2 Passengers)
[SQ] info Logging in...
[SQ] error Unexpected error occurred while searching!
{ TimeoutError: waiting for selector "#kfLoginPopup #membership-1" failed: timeout 30000ms exceeded
at new WaitTask (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:991:28)
at Frame._waitForSelectorOrXPath (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:882:12)
at Frame.waitForSelector (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:836:17)
at Frame.waitFor (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/FrameManager.js:821:19)
at Page.waitFor (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/Page.js:1022:29)
at module.exports.login (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/searcher.js:34:18)
at
at process._tickCallback (internal/process/next_tick.js:189:7) name: 'TimeoutError' }
[SQ] DEPARTURE [SIN -> MEL] - 2018-12-07 (2 Passengers)

AC one-way search failing with "Missing form element: tripTypeOneWay"

Log output:

[AC] error Unexpected error occurred while searching!
Error: Evaluation failed: Error: Missing form element: tripTypeOneWay
    at __puppeteer_evaluation_script__:6:19
    at ExecutionContext.evaluateHandle (/home/zq/node_modules/puppeteer/lib/ExecutionContext.js:121:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at ExecutionContext.<anonymous> (/home/zq/node_modules/puppeteer/lib/helper.js:110:27)
    at ExecutionContext.evaluate (/home/zq/node_modules/puppeteer/lib/ExecutionContext.js:48:31)
    at ExecutionContext.<anonymous> (/home/zq/node_modules/puppeteer/lib/helper.js:111:23)
    at DOMWorld.evaluate (/home/zq/node_modules/puppeteer/lib/DOMWorld.js:112:20)
  -- ASYNC --
    at Frame.<anonymous> (/home/zq/node_modules/puppeteer/lib/helper.js:110:27)
    at Page.evaluate (/home/zq/node_modules/puppeteer/lib/Page.js:782:43)
    at Page.<anonymous> (/home/zq/node_modules/puppeteer/lib/helper.js:111:23)
    at module.exports.fillForm (/home/zq/node_modules/flightplan-tool/src/Searcher.js:160:17)
    at module.exports.search (/home/zq/node_modules/flightplan-tool/src/engines/ac/searcher.js:57:18)

I don't know JS, but I did look at the Aeroplan search page and found that the element seems to be named "tripTypeRoundTrip" and so I subbed that into the code. It seems to work now, but I don't really know yet if the data is worth anything.

Can search 2 ADULTS ?

Can search 2 ADULTS ?

AC Search showing Non saver awards when non-saver unticked

(Assuming 9m points means non saver)

Website version Vs GitHub version

Is there a different version on your website than in GitHub?

I don't see the GetRoutes functionality on GitHub
Also I tried passing a sample JSON from the Search API from the website as input to the GitHub version but get a map() error.

default accounts.txt doesn't have AC config

it gets created with palceholders for NH, KF, etc.. but not for AC. I added it in same pattern and works fine.

Singapore Airlines Cargo instead of just Singapore Airlines

Search halt after 3 consecutive days for NH

After every 3 days of search, the chromium must halt such that the terminal will retry and give up after 90000ms so that I have to restart the search manually.

Was able to reproduce this error on both Mac and Win PC

Mac versions
npm: 6.7
node 11.10.1
flightplan-tools 0.3.49

Win versions
npm: 6.4.1
node: 10.15.2
flightplan-tools 0.3.49

month highlight not very clear

When the chart is very populated, it can be hard to determine what month is actually being highlighted.
The shade is altered but its still difficult.

Some ideas would be:

1. change the color of the month label for the month that is being hovered over
2. highlight a border on each segment within the month

BA: Cannot progress after login. Cannot read property 'value' of null

Hello, I'm very excited to use Flightplan.

After setting up my config file with just BA credentials and doing a test run - I get these errors, is anyone able to help?

➜  flightplan-tool node bin/cli.js search -w BA -f LON -t SFO -c first -s 2020-10-01 -e 2020-10-15 -q 1

flightplan search 0.3.51
Searching 15 days of award inventory: 10/01/2020 - 10/15/2020
[BA] DEPARTURE [SFO -> LON] - 2020-10-01 (1 Passenger)
[BA] info Logging in...
[BA] info 2nd login attempt...
[BA] error Unexpected error occurred while searching!
Error: Evaluation failed: TypeError: Cannot read property 'value' of null
    at __puppeteer_evaluation_script__:1:36
    at ExecutionContext._evaluateInternal (/home/henri/repos/points/node_modules/puppeteer/lib/ExecutionContext.js:122:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at ExecutionContext.<anonymous> (/home/henri/repos/points/node_modules/puppeteer/lib/helper.js:111:15)
    at DOMWorld.evaluate (/home/henri/repos/points/node_modules/puppeteer/lib/DOMWorld.js:112:20)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at Frame.<anonymous> (/home/henri/repos/points/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.evaluate (/home/henri/repos/points/node_modules/puppeteer/lib/Page.js:833:43)
    at Page.<anonymous> (/home/henri/repos/points/node_modules/puppeteer/lib/helper.js:112:23)
    at module.exports.enterText (/home/henri/repos/points/node_modules/flightplan-tool/src/Searcher.js:141:34)
    at module.exports.login (/home/henri/repos/points/node_modules/flightplan-tool/src/engines/ba/searcher.js:28:16)
    at Object.login (/home/henri/repos/points/node_modules/flightplan-tool/src/Engine.js:256:22)
    at process._tickCallback (internal/process/next_tick.js:68:7)
[BA] DEPARTURE [SFO -> LON] - 2020-10-02 (1 Passenger)
[BA] info Logging in...
[BA] error Unexpected error occurred while searching!
Error: Evaluation failed: TypeError: Cannot read property 'value' of null
    at __puppeteer_evaluation_script__:1:36
    at ExecutionContext._evaluateInternal (/home/henri/repos/points/node_modules/puppeteer/lib/ExecutionContext.js:122:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at ExecutionContext.<anonymous> (/home/henri/repos/points/node_modules/puppeteer/lib/helper.js:111:15)
    at DOMWorld.evaluate (/home/henri/repos/points/node_modules/puppeteer/lib/DOMWorld.js:112:20)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at Frame.<anonymous> (/home/henri/repos/points/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.evaluate (/home/henri/repos/points/node_modules/puppeteer/lib/Page.js:833:43)
    at Page.<anonymous> (/home/henri/repos/points/node_modules/puppeteer/lib/helper.js:112:23)
    at module.exports.enterText (/home/henri/repos/points/node_modules/flightplan-tool/src/Searcher.js:141:34)
    at module.exports.login (/home/henri/repos/points/node_modules/flightplan-tool/src/engines/ba/searcher.js:28:16)
    at Object.login (/home/henri/repos/points/node_modules/flightplan-tool/src/Engine.js:256:22)
    at process._tickCallback (internal/process/next_tick.js:68:7)
[BA] DEPARTURE [SFO -> LON] - 2020-10-03 (1 Passenger)                                                                                                                                                                                                                                   ```

➜ flightplan-tool node -v
Node version: v10.13.0

NH Search failing

SYD-BKK 6th Feb there are F seats (Manually searched NH) but FP is getting an error (searched whole Feb and same error) This is the error FP is getting:

installation failure

Hi there, I was trying to install the package on a Ubuntu VM (18.04), and I was running into error code as followed. Would you please help to explain what is wrong and what should be fixed? Thank you so much

 /home/arant/.nvm/versions/node/v12.8.1/bin/flightplan -> /home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/flightplan-tool/bin/cli.js

 [email protected] install /home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/flightplan-tool/node_modules/integer
 node-gyp rebuild

make: Entering directory '/home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/flightplan-tool/node_modules/integer/build'
  CXX(target) Release/obj.target/integer/src/integer.o
../src/integer.cpp: In static member function ‘static Result Integer::Cast(v8::Local<v8::String>, uint8_t)’:
../src/integer.cpp:329:33: error: no matching function for call to ‘v8::String::Value::Value(v8::Local<v8::String>&)’
   v8::String::Value utf16(string);
                                 ^
In file included from /home/arant/.cache/node-gyp/12.8.1/include/node/node.h:63:0,
                 from ../src/integer.hpp:3,
                 from ../src/integer.cpp:1:
/home/arant/.cache/node-gyp/12.8.1/include/node/v8.h:3025:5: note: candidate: v8::String::Value::Value(v8::Isolate*, v8::Local<v8::Value>)
     Value(Isolate* isolate, Local<v8::Value> obj);
     ^~~~~
/home/arant/.cache/node-gyp/12.8.1/include/node/v8.h:3025:5: note:   candidate expects 2 arguments, 1 provided
integer.target.mk:112: recipe for target 'Release/obj.target/integer/src/integer.o' failed
make: *** [Release/obj.target/integer/src/integer.o] Error 1
make: Leaving directory '/home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/flightplan-tool/node_modules/integer/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:196:23)
gyp ERR! stack     at ChildProcess.emit (events.js:203:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Linux 5.0.0-25-generic
gyp ERR! command "/home/arant/.nvm/versions/node/v12.8.1/bin/node" "/home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/arant/.nvm/versions/node/v12.8.1/lib/node_modules/flightplan-tool/node_modules/integer
gyp ERR! node -v v12.8.1
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok 
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/arant/.npm/_logs/2019-08-20T03_04_23_480Z-debug.log

FYI: What I have done (starting from a clean Ubuntu 18.04)

1. install curl by $sudo apt install curl
2. install python 2.7 (and the build tools) by $sudo apt install python2.7 python-pip
3. install git by $apt-get install git-core
4. install nvm by $curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
5. install node and npm by $nvm install node
6. tried to install the package by $npm install --global flightplan-tool

Duplicate Aircraft Types

CX - Doesn't log in, doesn't save results

Hi there,

Trying to run any search on CX fails at login and returns this error:

flightplan search 0.3.45
Searching 9 days of award inventory: 12/25/2019 - 01/02/2020
[CX] DEPARTURE [MEL -> LHR] - 2019-12-25 (1 Passenger)
[CX] info Logging in...
[CX] error Unexpected error occurred while searching!
Error: Evaluation failed: TypeError: Cannot read property 'value' of null
    at sel (__puppeteer_evaluation_script__:1:36)
    at ExecutionContext.evaluateHandle (/Users/me/.config/yarn/global/node_modules/puppeteer/lib/ExecutionContext.js:124:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Looking in the Chrome window, no credentials are attempted to be entered and the page fails but still ticks to new dates in the command window. Same happens on different searches on CX for me. I have added account details in accounts.txt.

OS: MacOS Mojave 10.14.2

I've also noted that when it completes (after manual login), no results are added to the database, even though the CLI shows that it found awards and flights:

Example:

[CX] DEPARTURE [LAX -> MEL] - 2019-12-28 (1 Passenger)
[CX] success Found: 70 awards, 14 flights
[CX] DEPARTURE [MEL -> LAX] - 2019-12-29 (1 Passenger)
[CX] success Found: 72 awards, 9 flights
[CX] DEPARTURE [LAX -> MEL] - 2019-12-29 (1 Passenger)
[CX] success Found: 71 awards, 15 flights
[CX] DEPARTURE [MEL -> LAX] - 2019-12-30 (1 Passenger)
[CX] success Found: 72 awards, 9 flights
[CX] DEPARTURE [LAX -> MEL] - 2019-12-30 (1 Passenger)
[CX] success Found: 66 awards, 10 flights
[CX] DEPARTURE [MEL -> LAX] - 2019-12-31 (1 Passenger)
[CX] success Found: 72 awards, 9 flights
[CX] DEPARTURE [LAX -> MEL] - 2019-12-31 (1 Passenger)
[CX] success Found: 57 awards, 9 flights
success Search complete!

I then run parse:

$ flightplan parse
flightplan parse 0.3.45
info Opening database...
info Parsing search requests...
success Search requests processed: 0
success Total awards found: 0

When running the web UI, the same thing happens and no data is shown. There's JSON and the sqlite db created, but nothing seems to be saving even though it found these results.

Any help is appreciated! :)

SQ Error searching 3pax

flightplan search -w SQ -f MEL -t SIN -c business -s 2019-01-15 -e 2019-12-31 -q 3 -h
...
[SQ] DEPARTURE [MEL -> SIN] - 2019-06-02 (3 Passengers)
[SQ] ARRIVAL [SIN -> MEL] - 2019-06-05
[SQ] error Unexpected error occurred while parsing!
Error: Invalid award mileageCost: 38666.666666666664
at new Award (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Award.js:54:13)
at module.exports.parseAward (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:204:14)
at Node.rows.each (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:148:31)
at initialize.exports.each (/usr/local/share/.config/yarn/global/node_modules/cheerio/lib/api/traversing.js:300:24)
at module.exports.parseFlights (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:72:10)
at module.exports.parseResults (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:54:28)
at module.exports.parse (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:30:15)
at Results._parseAwards (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Results.js:249:20)
at Results.get awards [as awards] (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Results.js:221:12)
at main (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/bin/cli-search.js:382:28)
[SQ] DEPARTURE [MEL -> SIN] - 2019-06-03 (3 Passengers)
[SQ] ARRIVAL [SIN -> MEL] - 2019-06-06
[SQ] error Unexpected error occurred while parsing!
Error: Invalid award mileageCost: 38666.666666666664
at new Award (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Award.js:54:13)
at module.exports.parseAward (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:204:14)
at Node.rows.each (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:148:31)
at initialize.exports.each (/usr/local/share/.config/yarn/global/node_modules/cheerio/lib/api/traversing.js:300:24)
at module.exports.parseFlights (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:72:10)
at module.exports.parseResults (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:54:28)
at module.exports.parse (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/sq/parser.js:30:15)
at Results._parseAwards (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Results.js:249:20)
at Results.get awards [as awards] (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Results.js:221:12)
at main (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/bin/cli-search.js:382:28)
[SQ] DEPARTURE [MEL -> SIN] - 2019-06-04 (3 Passengers)

Request for adding

Greetings! I would like to request adding of some regular & low costing european air companies.

• Pobeda (pobeda.aero)
• Norwegian.com
• EasyJet
• RyanAir
• WizzAir
• AirBaltic
• LOT Polish Airlines

I would like to know if I can help with adding some of these.
Thanks!

Calendar `inactive` type should depend on query parameters

Currently, I believe calendarStore flips a cell's type to active if any awards are found for that date (including the special empty placeholder). Ideally this would be conditional on the query params (engine / quantity / partner flag). Probably need to use the requests table to calculate the state properly, and maybe get rid of the empty award placeholder altogether.

Filtering by flight number doesn't work

This used to work in the old version, but ticking/unticking the flight number checkboxes no longer seems to apply any filter that I can tell?

NH award search failing

I ran flightplan search -w NH -f YEG -t CPT -c economy -s 2019-06-02 -e 2020-06-02 -q 1
It is successful if 0 rewards are found. However if there is an award, it comes back with "Error: Unrecognized Airport: Vancouver" or whatever city a reward is found with layovers. I've spoke to a few people who are having the same issue - thought it would be a good idea to raise this here.

NH waitlisted awards are being parsed as available

Please take a look...

client: '/bin/sh: 1: yarn: not found'

This is a weird one. Everything runs great except when I try to run flightplan client, I get /bin/sh: 1: yarn: not found. And yarn is definitely installed.

I do wonder if this is related to a previous bug I had, where running a flightplan search would tell me puppeteer does not have a chromium revision is not installed. I found a ton of threads about it, tried reinstalling (both local and global) puppeteer, yarn, everything. Turns out Flightplan was referencing the puppeteer in it's own node-modules folder rather than my home folder's node-modules folder. So I had to grab the .local-chromium folder from my own puppeteer and put it in flightplan's ./node-modules/puppeteer folder. (I still don't understand why this was happening??)

Is it possible that is happening in this case?

Error when installing flightplan

Hi guys, would appreciate if someone can guide me. I have zero experience in coding and am trying to install flight plan, but i get the following errors.

MacBook-Pro-3:~ AnsonT$ npm install --global flightplan-tool
/usr/local/bin/flightplan -> /usr/local/lib/node_modules/flightplan-tool/bin/cli.js

[email protected] install /usr/local/lib/node_modules/flightplan-tool/node_modules/integer
node-gyp rebuild

CXX(target) Release/obj.target/integer/src/integer.o
../src/integer.cpp:329:21: error: no matching constructor for initialization of
'v8::String::Value'
v8::String::Value utf16(string);
^ ~~~~~~
/Users/anson/Library/Caches/node-gyp/13.5.0/include/node/v8.h:3210:5: note:
candidate constructor not viable: no known conversion from
'v8::Localv8::String' to 'const v8::String::Value' for 1st argument
Value(const Value&) = delete;
^
/Users/anson/Library/Caches/node-gyp/13.5.0/include/node/v8.h:3203:5: note:
candidate constructor not viable: requires 2 arguments, but 1 was provided
Value(Isolate* isolate, Localv8::Value obj);
^
1 error generated.
make: *** [Release/obj.target/integer/src/integer.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack at ChildProcess.emit (events.js:305:20)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
gyp ERR! System Darwin 18.7.0
gyp ERR! command "/usr/local/Cellar/node/13.5.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /usr/local/lib/node_modules/flightplan-tool/node_modules/integer
gyp ERR! node -v v13.5.0
gyp ERR! node-gyp -v v5.0.5
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/anson/.npm/_logs/2019-12-26T03_58_40_863Z-debug.log

CX parser failing on economy awards

[CX] error Unexpected error occurred while searching!
TypeError: Cannot read property 'cabin' of undefined
at award.segments.f.segments.map.x (C:\Users\User04\AppData\Local\Yarn\Data\global\no de_modules\flightplan-tool\src\cx\parser.js:61:47)
at Array.map ()
at flights.map.f (C:\Users\User04\AppData\Local\Yarn\Data\global\no de_modules\flightplan-tool\src\cx\parser.js:39:30)
at Array.map ()
at module.exports.parse (C:\Users\User04\AppData\Local\Yarn\Data\global\no de_modules\flightplan-tool\src\cx\parser.js:35:28)
at module.exports._parse (C:\Users\User04\AppData\Local\Yarn\Data\global\no de_modules\flightplan-tool\src\base\parser.js:26:16)
at module.exports.parse (C:\Users\User04\AppData\Local\Yarn\Data\global\no de_modules\flightplan-tool\src\base\index.js:99:28)
at module.exports.search (C:\Users\User04\AppData\Local\Yarn\Data\global\no de_modules\flightplan-tool\src\base\index.js:70:31)
at process._tickCallback (internal/process/next_tick.js:68:7)

BA bot detection

Getting this now - haven't explored further the trigger (it happened on the very first query, so it could be a blacklist of the account/IP)
[BA] info Logging in...
[BA] error Unexpected error occurred while searching!
Error: BOT_DETECTED: Suspicious activity detected by website
at module.exports.login (C:\Users\mark\AppData\Roaming\npm\node_modules\flightplan-tool\src\engines\ba\searcher.js:50:13)
at
at process._tickCallback (internal/process/next_tick.js:188:7)

Edit: Looks like pattern detection, not IP/account ban.

AC Search failing: Missing form element: tripTypeRoundTrip

Running a search from the command line, it was running fine yesterday but I now get the following error:

[AC] error Unexpected error occurred while searching!
Error: Evaluation failed: Error: Missing form element: tripTypeRoundTrip
    at __puppeteer_evaluation_script__:6:19
    at ExecutionContext._evaluateInternal (/usr/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/ExecutionContext.js:122:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at ExecutionContext.<anonymous> (/usr/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/helper.js:111:15)
    at DOMWorld.evaluate (/usr/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/DOMWorld.js:112:20)
  -- ASYNC --
    at Frame.<anonymous> (/usr/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.evaluate (/usr/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/Page.js:827:43)
    at Page.<anonymous> (/usr/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/helper.js:112:23)
    at module.exports.fillForm (/usr/lib/node_modules/flightplan-tool/src/Searcher.js:160:17)
    at module.exports.search (/usr/lib/node_modules/flightplan-tool/src/engines/ac/searcher.js:72:18)

Which is weird since I can find the form element in the url set in the engines/ac/index.js file:

<input id="tripTypeRoundTrip1" name="tripTypeRoundTrip" type="radio" value="Round-Trip">
Running version 0.3.51, installed using the setup docs and everything was working fine yesterday.

Calendar "inactive" type should depend on query parameters

Edit: forgot to finish editing title sorry! and now it wont let me

When you click on a No awawds found cell - show what airlines/classes/pax numbers have been searched for on that date?

Use case is that i can't remember if i've searched for 2pax or 1pax so if it shows i only searched for 2pax i could re-run for 1 pax if i was ok for a single seat?

CX - the program will occur the error with first search.

[CX] info Logging in...
[CX] error Unexpected error occurred while searching!
Error: Execution context was destroyed, most likely because of a navigation.
at rewriteError (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\ExecutionContext.js:159:15)
at processTicksAndRejections (internal/process/next_tick.js:81:5)
-- ASYNC --
at ExecutionContext. (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\helper.js:108:27)
at ExecutionContext.evaluate (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\ExecutionContext.js:48:31)
at ExecutionContext. (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\helper.js:109:23)
at DOMWorld.evaluate (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\DOMWorld.js:105:20)
at processTicksAndRejections (internal/process/next_tick.js:81:5)
-- ASYNC --
at Frame. (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\helper.js:108:27)
at Page.evaluate (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\Page.js:815:43)
at Page. (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\node_modules\puppeteer\lib\helper.js:109:23)
at module.exports.textContent (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\Searcher.js:314:17)
at module.exports.login (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\engines\cx\searcher.js:46:33)
at processTicksAndRejections (internal/process/next_tick.js:81:5)

NH Search failing (again)

Candrew@fp:~flightplan search -w NH -f BKK -t SYD -c first -s 2019-08-01 -q 2 -h --force
flightplan search 0.3.30
Searching 1 days of award inventory: 08/01/2019 - 08/01/2019
[NH] DEPARTURE [BKK -> SYD] - 2019-08-01 (2 Passengers)
[NH] info Logging in...
[NH] success Login succeeded
[NH] error Unexpected error occurred while parsing!
Error: Invalid award fare: undefined
at new Award (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Award.js:22:13)
at Node.$.each (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/nh/parser.js:80:23)
at initialize.exports.each (/usr/local/share/.config/yarn/global/node_modules/cheerio/lib/api/traversing.js:300:24)
at module.exports.parseFlights (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/nh/parser.js:34:33)
at module.exports.parse (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/engines/nh/parser.js:22:29)
at Results._parseAwards (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Results.js:249:20)
at Results.get awards [as awards] (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/src/Results.js:221:12)
at main (/usr/local/share/.config/yarn/global/node_modules/flightplan-tool/bin/cli-search.js:382:28)
at
[NH] DEPARTURE [SYD -> BKK] - 2019-08-01 (2 Passengers)

^Candrew@fp:~$ flightplan search -w NH -f BKK -t SYD -c first -s 2019-02-06 -q 2 -h --force
flightplan search 0.3.30
Searching 1 days of award inventory: 02/06/2019 - 02/06/2019
[NH] DEPARTURE [BKK -> SYD] - 2019-02-06 (2 Passengers)
[NH] info Logging in...
[NH] success Login succeeded
[NH] DEPARTURE [SYD -> BKK] - 2019-02-06 (2 Passengers)
success Search complete!

Doesn't work on Ubuntu install

Fresh Ubuntu 18 install with below then running flight plan search returns error. Related to the way you're calling commands with commander

andrew@FlightPlanVM2:~$ flightplan search

flightplan-search(1) does not exist, try --help

//Install NodeJS
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs

// Install Yarn
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn

// Install dependencies
sudo apt-get install build-essential libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libnss3 lsb-release xdg-utils wget

// Install flightplan-tool
yarn global add flightplan-tool

// Add yarn bin folder to profil
echo 'PATH="$HOME/.yarn/bin:$PATH"' >> .profile

mkdir flightplan
cd flightplan
flightplan search

Excessive cool-down periods

Cool-down is occurring almost every request, possibly an issue with countdown?

Migrate NH engine to multi-city search

NH engine is missing some awards when searching one-way, if there is no availability for the dummy return segment. (Since we'll get a "no results" modal, and never see the outbound award.) The multi-city search dialog doesn't have this problem, and can even take a +0 day return date. Return awards may still be hidden though, if there's no outbound availability, need to confirm if a separate search is required in that case.

CX parser can't locate search fields

The elements in the DOM look very different than what is described in the parser for the page given.

Error: No node found for selector: #input-origin
    at assert (/Users/alex/.nvm/versions/node/v11.15.0/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/helper.js:270:11)
    at DOMWorld.click (/Users/alex/.nvm/versions/node/v11.15.0/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/DOMWorld.js:366:5)
    at processTicksAndRejections (internal/process/task_queues.js:86:5)
  -- ASYNC --
    at Frame.<anonymous> (/Users/alex/.nvm/versions/node/v11.15.0/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.click (/Users/alex/.nvm/versions/node/v11.15.0/lib/node_modules/flightplan-tool/node_modules/puppeteer/lib/Page.js:1031:29)
    at module.exports.setCity (/Users/alex/.nvm/versions/node/v11.15.0/lib/node_modules/flightplan-tool/src/engines/cx/searcher.js:308:16)
    at module.exports.search (/Users/alex/.nvm/versions/node/v11.15.0/lib/node_modules/flightplan-tool/src/engines/cx/searcher.js:60:16)
[CX] DEPARTURE [HKG -> SFO] - 2020-01-01 (2 Passengers)

Aeroplan login changes

Aeroplan has changed their login system (integration with Air Canada systems) and it appears to have broken flightplan-tool's Aeroplan login.

Failef to launch chrome with ubuntu 16.04

what happen ?

root@hhi-VirtualBox:~/flightpaln# flightplan search
Airline website to search (2-letter code)? CX
Departure city (3-letter code)? SFO
Arrival city (3-letter code)? HKG
Desired cabin class (first/business/premium/economy)? first
Start date of search range (YYYY-MM-DD)? 2019-03-02
End date of search range (YYYY-MM-DD)? 2019-03-06
Searching 5 days of award inventory: 03/02/2019 - 03/06/2019

Error: Failed to launch chrome!
[9259:9259:0713/021928.407785:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

at onClose (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/Launcher.js:285:14)
at Interface.helper.addEventListener (/usr/local/share/.config/yarn/global/node_modules/puppeteer/lib/Launcher.js:274:50)
at emitNone (events.js:111:20)
at Interface.emit (events.js:208:7)
at Interface.close (readline.js:368:8)
at Socket.onend (readline.js:147:10)
at emitNone (events.js:111:20)
at Socket.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)

root@hhi-VirtualBox:~/flightpaln#

SQ Parsing

Just ran a search

flightplan search -w SQ -f MEL -t SIN -c first -s 2018-11-01 -e 2018-11-30 -q 1 -h

Ran fine, but parsing says no awards found, yet looking at screenshots it captured I can see there are awards. ie 1st Nov - SQ238 and SQ228

andrew@fp:$ flightplan parse
flightplan parse 0.2.5
Opening database...
Parsing search requests...
Search requests processed: 0
Total awards found: 0
andrew@fp:$

Search Error: SQLite3 can only bind numbers...

This happens after the first search on BA site comes up as not available. Similar result with NH so it's not airline specific.

flightplan search 0.2.2
Airline website to search (2-letter code)? BA
Departure city (3-letter code)? STL
Arrival city (3-letter code)? POM
Desired cabin class (first/business/premium/economy)? economy
Start date of search range (YYYY-MM-DD)? 2019-04-01
End date of search range (YYYY-MM-DD)? 2019-07-21
Searching 112 days of award inventory: 2019-04-01 - 2019-07-21
[BA] DEPARTURE [POM -> STL] - 2019-04-01 (1 Passenger)
[BA] info Logging in...
[BA] success Login succeeded
error A fatal error occurred!
TypeError: SQLite3 can only bind numbers, strings, Buffers, and null
    at Object.insertRow (C:\Users\mark\AppData\Local\Yarn\Data\global\node_modules\flightplan-tool\shared\db.js:100:27)
    at Object.saveAwards (C:\Users\mark\AppData\Local\Yarn\Data\global\node_modules\flightplan-tool\shared\utils.js:266:26)
    at main (C:\Users\mark\AppData\Local\Yarn\Data\global\node_modules\flightplan-tool\bin\cli-search.js:318:15)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)

CX parser failing on economy awards

error Unexpected error occurred while parsing!
Error: Conflicting cabin status: [object Object]
at module.exports.availabilityMap (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\engines\cx\parser.js:138:15)
at flightData.map.f (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\engines\cx\parser.js:51:33)
at Array.map ()
at module.exports.parse (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\engines\cx\parser.js:47:32)
at Results._parseAwards (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\Results.js:247:20)
at Results.get awards [as awards] (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\src\Results.js:214:12)
at main (C:\Users\kitko\AppData\Roaming\npm\node_modules\flightplan-tool\bin\cli-search.js:382:28)
at processTicksAndRejections (internal/process/next_tick.js:81:5)

is there a database setup script?

I'm trying to run flightplan locally. I start it using yarn dev but then it says my sql lite has no such table as awards

[SERVER] SqliteError: no such table: awards
[SERVER] at app.get (/Users/xxxxx/dev/flightplan/bin/cli-server.js: 100:26)

Is there a script to setup the database locally?

flightplan-tool install error, successful NPM&node

Ubuntu 18.04
Everything went perfect following the setup guide. Confirmed Node.js and NPM were installed successfully (node v10.16.3 and npm v6.9.0 returned in terminal)
When running the $ npm install --global flightplan-tool I get the following errors:

npm ERR! path git
npm ERR! code ENOENT
npm ERR! errno ENOENT
npm ERR! syscall spawn git
npm ERR! enoent Error while executing:
npm ERR! enoent undefined ls-remote -h -t ssh://[email protected]/jd20/commander.js.git
npm ERR! enoent
npm ERR! enoent
npm ERR! enoent spawn git ENOENT
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR! /home/mikey/.npm/_logs/2019-08-17T04_34_03_494Z-debug.log

Can't Launch Server or Client

Hey Jason, great tool! I've been salivating over it all morning and have been running queries here and there.

I have collected 7 months' worth of data from 1/1/2019 CX HKG-SGN in Y, and it's all parsed, but I can't seem to launch the server and client tool (it freezes after "Success!" and there's an error when doing client). Do you have any suggestions? Running Windows 8.1 and here's my node.js command prompt:

It would be great to visualize these availabilities! Thanks!!

Blue text from flightplan search hard to see on dark background

Using putty SSH to connect it has a default dark background, noticed in flightplan2 the search text is now a dark blue so hard to read...

502 Bad Gateway

Hi,

Due to weird bugs this week, I cleared all of flightplan and yarn, set node configs to default, and reinstalled fresh. Everything in terms of the flightplan install and then searching for rewards is working great.

But then after I run the server in one terminal and the client in another, my browser pops up to “localhost:3000” but I’m showing 502 bad gateway. The weird part is that I’m showing no warnings or errors in either the client window or server window. Do you have any suggestions?

Failure to install flightplan in MacOS Mojave

Hi there!

I don't have any programming experience. Just encountered a problem while installing FlightPlan in MacOS Mojave v10.14. Already installed node, npm, XCode, and HomeBrew. The final few lines in my last attempt was -

sh: yarn: command not found
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! [email protected] postinstall: cd client && yarn
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/Patrick/.npm/_logs/2018-11-22T15_19_46_025Z-debug.log

Thanks in advance!

Parsing error

Finished the first search last night, wanted to parse the results as instructed in the youtube vid. But got the following error:

TypeError: Cannot read property '1' of null

What did I missed to install?

EDIT: Node version is v10.7.0

Expose proxy setting from cli-search.js

Need a command-line switch, to set proxy for the search command (it gets passed as options in the initialize() call on the engine).

Version 0.2.8, CX awards problems

Dear jd20,
Thanks for what you have done, this is definitely a super fantastic way to search for awards tickets.
I have been using your first version several times, and totally forget to renew and update.
In recent days I eventually upgrade the fligtplan-tool to v.0.2.8, and I immediately encounter some problems, and I'll mark below
could you please take a look and tell me if I am wrong in any steps?

A. My system
macOS Mojave 10.14
node --version v11.0.0
yarn --version 1.10.1
flightplan-tool 0.2.8
p.s. I run "yarn global remove flightplan-tool" + "yarn cache clean" + delete my old flightplan folder(which contained config, data, db files)
then "yarn global add flightplan-tool" to make sure I have fresh, new, latest flightplan-tool 0.2.8

B. Steps to set up

1. Run "mkdir flightplan" to create a folder under my user folder
2. Run "cd flightplan" to get into my flightplan folder. Problem 1: But I can't find the folder "flightplan" just like before, although I can get in by Terminal
3. Run "flightplan search" and follow the instructions which lead me to create the file: config/accounts.txt
4. Run "nano config/accounts.txt" and add my accounts

C. Steps to search

1. Run "flightplan search -w CX -p -f HKG -t DOH -o -c business -s 2019-06-25 -e 2019-06-30 -q 2 -h"
2. Problem 2: Unexpected error occurred while searching. Error: Node is either not visible or not an HTMLElement, but the searching kept on, so I eventually got the data between 2019-06-26 to 2019-06-30

3. Run "Open data", then I can finally see the folder "flightplan" with sub-folders
   and I can see the raw screenshots on folder "data", Problems 3 : but if I forget to run "Open data" in next search, I won't see my next raw screenshots.

D. Steps to convert to database

1. Run "flightplan parse", the result showed totally different than the first version of flightplan-tool
   Problem 4 :No matter how many awards seats available, the total rewards found always be 0(zero). And do I still have to run flightplan parse in version 0.2.8?

2. Run "sqlite3 db/database.sqlite3"
3. Run ".table"
4. Run "select * from requests segments;"
5. Run "select * from awards; "
6. Run ".quit"

E. Visulization

1. Run "flightplan stats" to see how many awards are there, and Problem 5: the awards results are much more than the raw screenshot we saw

2. Run "flightplan server" & "flightplan client"
3. Problem6 : I got plenty of awards tickets as shown, but most of them are not available after manual searching and calling to customer service

real screenshots on the same date

Thanks for reading this long post, let's make all the problems list down

1. Is it normal that the folder "flightplan" become invisible?
2. How can I deal with the error: Error: Node is either not visible or not an HTMLElement
3. The raw screenshots didn't show up if I don't run "open data"
4. Do I still have to run "flightplan parse" and are there any mistakes for my sqlite3. steps?
   5+6: The awards results did not reflect the true avalibility.