esri/geoservices-js

Deprecated - please consider using @esri/arcgis-rest-js

Geoservices-js (Deprecated)

Current development efforts are being being invested in arcgis-rest-js. Please consider upgrading.

Node.js bindings for Geoservices.

This module abstracts making both authenticated and anonymous common Geoservices requests and parsing their response.

Anonymously accessible services

• Geocoding
• Reverse Geocoding
• Addresses
• Feature Services

Services that require authentication

• Bulk Geocoding

Usage

Installing

$ npm install geoservices

Basic Usage

var Geoservices = require('geoservices');

var client = new Geoservices();

Building

Geoservices uses grunt for its build system. To install:

$ sudo npm install -g grunt-cli

To build and run tests, simply run:

$ grunt

Testing

Standalone testing can also be run:

$ npm test

Further documentation

• Feature Services allow you to interact with feature geometry and attributes from Esri services
• Geocoding is turning an address or place name into a location. The documentation describes simple geocoding, reverse geocoding and batch geocoding.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing.

This is an open library for communicating with any service that implements the Geoservices specification. The default endpoint for Geocoding is ArcGIS Online. Please see Terms of Use for licensing and usage details.

See Also

• geocoder-arcgis provides a client for just the Geocoding APIs with a promise-based design.

geoservices-js's Issues

its weird (and not documented) that you have to new up featureservice objects

samples show

client.featureservice( params , function (err, result) {

but new is actually required.

new client.featureservice( params , function (err, result) {

looking for enlightenment (most likely from @nixta)

Need to handle JSONP for cross domain requests

If were are going to be serious about this working a browser we need to make sure it handles JSONP requests when needed.

I need this for FeatureLayer in esri-leaflet to work in things IE 8 and 9.

Allow use of application id and application secret

Previously it was impossible to use a token created with the application id and application secret to get things like directions but now that is possible so it should be supported this also means that we will need to catch invalid token errors and get a new token and retry to request.

find > findAddressCandidates

The find operation has been deprecated. Please discontinue using it in your application, and use the findAddressCandidates operation instead.

reference: ../api-reference/geocoding-find-address-candidates.htm

Implement a command-line interface

Example:

$ arcgis-cli geocode "920 SW 3rd Ave, Portland, OR 97201" --simple

💭

Library support for browsers

Starting an open issue to discuss the action from #73

There are numerous use cases for interacting with our GeoServices API that are not Cartographic. We need support in Cedar, as well as projects with US Census citySDK-arcgis

Requiring Leaflet or JSAPI is overkill and arguably a simple library is what they should include rather than vice versa.

If this project doesn't want to do it, then we should make a new one that does and call this geoservices-node

callback on batch.geocode() ?

It would be handy to be able to pass a callback to batch.geocode() that gets handed the result when batch.run() eventually gets called, in the same way that the client.geocode() callback behaves. This would serve concurrent patterns a lot better. The results set from batch.run() is only useful if you know the order in which batch.geocode() calls were made, but there are cases where you might not necessarily know that. For example: a server that proxies to the geocode service by queueing up multiple incoming batch.geocode() calls and flushing them out every second or so with batch.run(), then returning the response to the original requestor. This would also make switching between simple geocodes and batch geocodes work with minimal code changes.

I can work on a PR if you think this is worth implementing.

Wish: Changelog

It would be great to have a Changelog.md or HISTORY.md so users could see an easy summary of what changed between releases to decide whether or not to upgrade.

The recent version bump to 1.0.0 could imply breaking API changes, but if there are any, it's not clear what they are.

Provide paging method on FeatureService

Issue #35 will depend on at least the internal versions of the paging function(s).

Use standard start, limit parameters or a next(XX) function.

Public API using start, limit might be best on the chained query functions. The next operator could be hung directly off of FeatureService instance

example which demonstrates how to batch geocode a .csv

i think we need a sample in this repository to demonstrate how to batch geocode from .csv's

maybe its overkill, but i think a full blown browser/node backend heroku app would be a lot more fun than a node script that expects the input to be provided via the commandline.

requirements:

1. in the browser, direct users to upload a .csv which includes a column named something like address that contains the text that needs to be geocoded.
2. give opportunity to sign in using OAuth2 for geocoding more than a few rows.
3. pass back the .csv with a latitude and longitude field appended.

i should be able to rip off the code @phpmaps wrote here.

Custom Geocoder - singleLineAddressField

It seems like when doing a geocode, this code assumes the singleLineAddressField is singleLine. But when working with a custom Geocoder, that may not always be the case. I would like a way either define this, or have the library look this up on the endpoint automatically.

Chainable query class for FeatureService

I walk talking with @mpriour about this it would be great to have a class that would abstract the complexities of querying a feature service.

Here was the example I gave @mpriour.

Query().within(extent).where({"city": "Portland"})

We could also hang this off FeatureService.

query = new client.FeatureService.Query().within(extent).where({"city": "Portland"});

myFeatureService.query(query, callback);

FeatureService Query Responses

Depending on the parameters sent, you can basically get 3 different response structures from a Query to a FeatureService layer.

See: http://resources.arcgis.com/en/help/arcgis-rest-api/#/Query_Feature_Service_Layer/02r3000000r1000000/

1. Response contains features + other fields, but not objectIds
2. Response contains objectIds + objectIdFieldName
3. Response contains count only

Do we want to :

1. handle those response types differently?
2. provide convenience methods to make Id's only and Count only queries?

Calls fail when organization is set to SSL only.

authenticated API calls fails using this client when the related organization has the "SSL only" option set.

To remedy this, I recommend:

1. Changing the default baseUrl to be 'https'. This makes you "secure by default". Also, HTTP 2.0 is based on SDPY which routes all requests over TLS, so this is the direction the web is heading anyway.
2. Document how to provide your own baseUrl if people want to explicitly opt-in to weaker security and potentially better performance. Currently the baseUrl method remains is not documented at all.

"Invalid JSON" error message is wrongly returned when JSON is valid, but callback fails

There's a bug here, in the request handling code:

    try {
       var response = JSON.parse(this.responseText);
       callback(null, response);
     } catch (err) {
       callback("Invalid JSON on response: " + this.responseText);
     }

It appears that the intent it to check for invalid JSON, but the code is
also catching any problems with the user-supplied callback and declaring
that the issue "invalid JSON" even when it is not.

To compound the issue, the error returned by the callback is silently
discarded, and only the potentially-valid JSON is returned instead,
making it hard to debug what the actual problem is.

In the original code, this bug is two places, as the request handler is
duplicated. In one of my pending pull requests, the request handler has
already been consolidated.

I'll look at a potential patch now, since I need to get this working for
myself.

Wish: support filtering geocoding to a bounding box.

It would be nice if there was an option to specify a bounding box to be used with geocoding, so that results outside the bounding box would be considered geocode failures.

I read the PDF linked into the docs for the upstream REST geocode API being used. It did not have documented support bounding box restriction built in.

This library could perhaps use coordinatesContainsPoint to check results as they come back.

Link to some live demos

It would be super awesome if the README.md could link to some live demos that show geoservices-js in action!

Thanks!
AL

feature service enhancements

Im starting to use the feature service class in Esri Leaflet and I have a suggestions.

1. Add an option to pass a URL to a service

Currently featureservice can only accept an options object of catalog,service, type and layer. This makes it really hard to use if you already have a URL because you have to split the URL up into the correct components.

2. Really needs a query builder class

I chatted with @mpriour about this last night something like Query().within(extent).where({"city": "Portland"}) would be amazing.

3. Misc semantics

featureservice is a class shouldn't it be camel cased FeatureService?

4. Shouldn't throw away data

When you initalize a feature service it makes a request to get all the info about the service and passes it to a callback but then throws it away. Maybe we could merge this info in with the service so you could access it whenever you like?

I would be willing to tackle some of these issues. What do you think @mpriour @chelm @JerrySievert

wish: document the concatenation system.

Have now realized how the file generation system works, it would have been a time-saver as a new contributor if this was better documented. Some improvements would include:

• Clearly identifying auto-generated files as such, so potential contributors don't try to support pull requests for modifications to them. You could add a 'header' file to the list of files to concatenate, which adds a warning/disclaimer to the top of every auto-generated file, to the effect of: "WARNING: This file is auto-generated. See gruntfile.js to find which source files this files is generated from. Please modify the source files instead.
• Some basic code comments in gruntfile.js to explain what's going on would be helpful.
• A project-specific "Contributing" doc could be helpful, explaining the project structure a bit. I'm interested to know more about the layout of splitting out files and re-joining them differently for browsers and node.js. Browserify appears to solve the related problem of getting node.js code to run in the browser. What's the particular advantage of the custom solution here?

Running in browser with example from docs returns error

When trying the example in the documentation, (browser - basically just trying to instantiate a new Geoservices object) an error is thrown:

Uncaught TypeError: object is not a function 

I'm not getting a proper trace on it right now, so it's hard to tell where the error is really coming from.

add doc/samples for feature service CRUD methods

as best i can tell, none of the methods below are documented (yet).

• count()
• ids()
• update()
• add()
• remove()
• edit()

syntax highlighting for existing and new code snippets would also be a nice touch

  ```js

geoservices.js

Hi - I'm trying to use this geoservices.js library with my locally hosted ARCGIS geocoding service.This ARCGIS geocoding service is running on IIS Server , with integrated windows authentication. I see credential errors when running against the locally hosted service. Doing some research, it appears, passport-windowsauth.js. can you help rectify the issue.

var Geoservices = require('geoservices');
var request = require('request');

var passport = require('passport');
var WindowsStrategy = require('passport-windowsauth');



passport.use(function(profile, done){
  User.findOrCreate({ waId: profile.id }, function (err, user) {
    done(err, user);
  });
});

app.get('/express-passport',
  passport.authenticate('WindowsAuthentication'),
  function (req, res){
    res.json(req.user);
  });
'''
I get the error , Authentication Strategies must have a name. Please advise. 

Member

Hi, I would like to be a member of this community 🙂

Wish: document how to use token caching for high performance

When using this as part of a high-performance server that authenticate
calls as a single user, the ideal usage pattern would be generate a
long-lived authentication token once, and then re-use it across many
requests before generating a new authentication token.

From looking at the source code, it appears that the foundation of
supporting this is there.

What's missing is example documentation that shows how the token could
be stored in a global variable where it could persist across requests,
with the application checking the expiration of the token and attempting
to renew it if has expired.

Thanks for considering this feature & documentation.

tests are failing

Your tests are currently failing:

When requesting all address matches using object
✗ It should return a sorted list of likely geocode matches
» expected -122.67633658436517,
got -122.6763365852645 (==) // geocode-test.js:72
When running using the "simple" method
✗ It should return the correct latitude and longitude
» expected -122.67633658436517,
got -122.6763365852645 (==) // geocode-test.js:31
When requesting all address matches using text
✗ It should return a sorted list of likely geocode matches
» expected -122.67633658436517,
got -122.6763365852645 (==) // geocode-test.js:52
When requesting a valid geocode
✗ It should return the correct latitude and longitude
» expected -122.67633658436517,
got -122.6763365852645 (==) // geocode-test.js:20

---

My guess is that the values for some specific locations you are testing
got refined on the server, and your tests needed to be updated to use
the latest values, but I'll leave that for you to confirm.

Provide method on FeatureService to get ALL the features

For pre 10.1 servers this is always a minimum 2 step process

• objectIds query to get total number of features
• client side "paging" query to get the features

Min: 2 , Max: (Ceiling TotalResults/maxRecordCount) + 1

For 10.1+ servers this can be a single step for small datasets, but is likely also a multistep process

• "all" style query (ordered by objectIds); if no exceedTransferLimit then you're done
• if exceedTransferLimit, client-side "paging" queries to get the rest of the features

Min: 1, Max: (Ceiling TotalResults/maxRecordCount) + 1

see: #33 (comment)

Enable good defaults for a query with no parameters

Currently FeatureServices require a where parameter and probably outFields. Instead make it default to return all (up to pagination?) results without requiring the developer to specify where:1=1

Need to handle proxying requests

similar to esri.config.defaults.io.proxyUrl we need to this to be able to handle proxying requests through a proxy when needed.

This would enable editing feature services in esri-leaflet.

Determine level of IE support

We should figure out what version of IE to support with the browser build since it has an impact on #40 and #41.

IE 8 and IE 9 both support XDomainRequest which has a few, interesting limitations that are detailed in http://blogs.msdn.com/b/ieinternals/archive/2010/05/13/xdomainrequest-restrictions-limitations-and-workarounds.aspx I'll explore some of the more relevant ones here.

1. The target URL must be accessed using only the HTTP methods GET and POST
   Since all ArcGIS services only use GET and POST we are ok.
2. No custom headers may be added to the request
   No custom headers are required by ArcGIS services so this is a non issue
3. Only text/plain is supported for the request's Content-Type header
   This is an issue since I believe existing services do not parse query params unless the type is set to application/x-www-urlencoded. However this only impacts IE 8.
4. No authentication or cookies will be sent with the request
   Non issue since authentication is handled with the token parmeter for services.
5. Requests must be targeted to the same scheme as the hosting page
   Since all existing services are hosted over http and https with the exception of certain calls like generateToken and oauth2/token which must be over https

Given restriction 3 we should only support IE 9 since that is a show stopper. But I think we could use XDomainRequest and support IE 9 without #40 and #41. I will do some more research on this and keep posting it here.

@mpriour do you have any experience with XDomainRequest?

geocode.batch() overwriting OBJECTID

An OBJECTID currently gets generated when calling geocode.batch() if one isn't passed in as the second argument (optionalId). The auto-generated ID is also currently overwriting OBJECTID even if it already exists on the object. Example, in the following, OBJECTID will replaced by an auto-generated id based on the number of previously calls to batch():

  geocode.batch({
    SingleLine: "380 New York Street, Redlands, CA 92373-8100",
    OBJECTID: 4326
  })

This was a bit surprising to me - I'd expect the optionalId to overwrite OBJECTID only if it doesn't already exist. If both OBJECTID exist and optionalId is passed but they are different, then maybe throw an error?

requestHandler not present for reverse geocode

I'm trying to get @alaframboise demo for reverse geocoding working but im hitting a

Uncaught TypeError: Cannot call method 'get' of undefined 

that traces back to https://github.com/ArcGIS/arcgis-node/blob/master/browser/arcgis.js#L255

It looks like this.requestHandler isn't getting set. Looks similar to what happened to feature server yesterday.

@JerrySievert can you fix?

code smell: this.token and this.token.token are checked, but only one is checked for expiration

In geocode.js about line 173, there's this bit of code:

if (!this.token ||
      !this.token.token ||
      this.token.expires < current) {
    callback("Valid authentication token is required");
   // ...
}

It's a code smell that while the token is checked at this.token and this.token.token, the expiration is only checked for one these locations. Shouldn't these be handled consistently, or only one location be allowed?

A little further down, data.token is set to this.token.token and this.token is ignored. There it appears that the this.token variation is ignored.

If this bit of code is working as intended, a code comment would be helpful to explain why this.token and this.token.token are being treated differently.

FeatureService improvements

How FeatureService uses the callback param is interesting. I think we could make some improvements to how it handles getting information about itself.

Could we create a basic evented mixin so we could do things like featureService.on("ready", callback) that would fire when the request to get info completes?

We could also assign the result of the request to featureService.info so users have access to it later.

GeoServices API Question

I understand that this is a JavaScript API which consume GeoServices.

My question is with regards to non-ESRI software hosted GeoServices. Are there any? If so, can someone point me to them?

Better yet, are there any open source projects which support creation of GeoServices?

error when batch geocoding more than max number of results

Looks like the geocode service returns a max of 1000 results, silently discarding any additional queries. This is might be variable, but can be queried from the geocode service? https://developers.arcgis.com/rest/geocode/api-reference/geocoding-geocode-addresses.htm#GUID-FB22981C-FDC9-464D-9279-946EA859DC2B

However it's determined, batch() should probably should throw an error when trying to queue up more than the limit rather than silently dropping results.

Error when using `client.options.geocoderUrl`

I get this error when following the example in this documentation:

const client = new Geoservices();
client.options.geocoderUrl = 'https://my.geocoder.com/a/rest/services/Locators/AddressPointLocator/GeocodeServer';
client.geocode({.....}, function() { ... });

TypeError: Cannot set property 'geocoderUrl' of undefined

It seems like the solution is to create an object off of my client object first. For example, this seems to work fine:

const client = new Geoservices();
client.options = {};
client.options.geocoderUrl = 'https://my.geocoder.com/a/rest/services/Locators/AddressPointLocator/GeocodeServer';
client.geocode({.....}, function() { ... });

I'm not sure if I'm doing something wrong, or if we should add that line (client.options = {};) to the documentation, or if we should change the module so that line is not required. Looking for input. Thanks!

Should non-successful requests be considered "errors" ?

When gecoding is done, a callback is executed with an "error" and a "result".

If you specify bad parameters, then the callback receives no errors, and this result:

{ error: 
   { code: 400,
     message: 'Invalid or missing input parameters.',
     details: [] } 
}

If the request does not return an HTTP success code, I would expect that this structure would be returned as the "error" argument. (And possibly as the "result", too).

This way, a simple check of:

if (err) { ... }

Would catch a maximal number of failures. Currently, you need check to see if there's an "error" and also check the that the result doesn't return an "error" as well.