sasjs/adapter

An adapter for bidirectional SAS® <-> Javascript communication

@sasjs/adapter

SASjs is a open-source framework for building Web Apps on SAS® platforms. You can use as much or as little of it as you like. This repository contains the JS adapter, the part that handles the to/from SAS communication on the client side. There are 3 ways to install it:

1 - npm install @sasjs/adapter - for use in a nodeJS project (recommended)

2 - Download and use a copy of the latest JS file

3 - Reference directly from the CDN - in which case click here and select "SRI" to get the script tag with the integrity hash.

If you are short on time and just need to build an app quickly, then check out this video and the react-seed-app which provides some boilerplate.

For more information on building web apps with SAS, check out sasjs.io

None of this makes sense. How do I build an app with it?

Ok ok. Deploy this example.html file to your web server, and update servertype to SAS9, SASVIYA, or SASJS depending on your backend.

The backend part can be deployed as follows:

%let appLoc=/Public/app/readme;  /* Metadata or Viya Folder per SASjs config */
filename mc url "https://raw.githubusercontent.com/sasjs/core/main/all.sas";
%inc mc; /* compile macros (can also be downloaded & compiled seperately) */
filename ft15f001 temp;
parmcards4;
  %webout(FETCH) /* receive all data as SAS datasets */
  proc sql;
  create table areas as select make,mean(invoice) as avprice
    from sashelp.cars
    where type in (select type from work.fromjs)
    group by 1;
  %webout(OPEN)
  %webout(OBJ,areas)
  %webout(CLOSE)
;;;;
%mx_createwebservice(path=&appLoc/common,name=getdata)

You now have a simple web app with a backend service!

Detailed Overview

The SASjs adapter is a JS library and a set of SAS Macros that handle the communication between the frontend app and backend SAS services.

There are three parts to consider:

1. JS request / response
2. SAS inputs / outputs
3. Configuration

JS Request / Response

To install the library you can simply run npm i @sasjs/adapter or include a <script> tag with a reference to our CDN.

Full technical documentation is available here. The main parts are:

Instantiation

The following code will instantiate an instance of the adapter:

let sasJs = new SASjs.default(
  {
    appLoc: "/Your/SAS/Folder",
    serverType:"SAS9"
  }
);

If you've installed it via NPM, you can import it as a default import like so:

  import SASjs from '@sasjs/adapter';

You can then instantiate it with:

const sasJs = new SASjs({your config})

More on the config later.

SAS Logon

All authentication from the adapter is done against SASLogon. There are two approaches that can be taken, which are configured using the loginMechanism attribute of the sasJs config object (above):

• loginMechanism:'Redirected' - this approach enables authentication through a SASLogon window, supporting complex authentication flows (such as 2FA) and avoids the need to handle passwords in the application itself. The styling of the window can be modified using CSS.
• loginMechanism:'Default' - this approach requires that the username and password are captured, and used within the .login() method. This can be helpful for development, or automated testing.

Sample code for logging in with the Default approach:

sasJs.logIn('USERNAME','PASSWORD'
  ).then((response) => {
  if (response.isLoggedIn === true) {
    console.log('do stuff')
  } else {
    console.log('do other stuff')
  }
}

More examples of using authentication, and more, can be found in the SASjs Seed Apps on github.

Request / Response

A simple request can be sent to SAS in the following fashion:

sasJs.request("/path/to/my/service", dataObject)
  .then((response) => {
    // all tables are in the response object, eg:
    console.log(response.tablewith2cols1row[0].COL1.value)
  })

We supply the path to the SAS service, and a data object.

If the path starts with a / then it should be a full path to the service. If there is no leading / then it is relative to the appLoc.

The data object can be null (for services with no input), or can contain one or more "tables" in the following format:

let dataObject={
	"tablewith2cols1row": [{
		"col1": "val1",
		"col2": 42
	}],
	"tablewith1col2rows": [{
		"col": "row1"
	}, {
		"col": "row2"
	}]
};

These tables (tablewith2cols1row and tablewith1col2rows) will be created in SAS WORK after running %webout(FETCH) in your SAS service.

The request() method also has optional parameters such as a config object and a callback login function.

The response object will contain returned tables and columns. Table names are always lowercase, and column names uppercase.

The adapter will also cache the logs (if debug enabled) and even the work tables. For performance, it is best to keep debug mode off.

Verbose Mode

Set verbose to true to enable verbose mode that logs a summary of every HTTP response. Verbose mode can be disabled by calling disableVerboseMode method or enabled by enableVerboseMode method. Verbose mode also supports bleached mode that disables extra colors in req/res summary. To enable bleached verbose mode, pass verbose equal to bleached while instantiating an instance of RequestClient or to setVerboseMode method. Verbose mode can also be enabled/disabled by startComputeJob method.

Session Manager

To execute a script on Viya a session has to be created first which is time-consuming (~15sec). That is why a Session Manager has been created which is implementing the following logic:

1. When the first session is requested, we also create one more session (hot session) for future requests. Please notice two pending POST requests to create a session within the same context:
2. When a subsequent request for a session is received and there is a hot session available (not expired), this session is returned and an asynchronous request to create another hot session is sent. Please notice that there is a pending POST request to create a new session while a job has been already finished execution (POST request with status 201):
3. When a subsequent request for a session is received and there is no available hot session, 2 requests are sent asynchronously to create a session. The first created session will be returned and another session will be reserved for future requests.

Variable Types

The SAS type (char/numeric) of the values is determined according to a set of rules:

• If the values are numeric, the SAS type is numeric
• If the values are all string, the SAS type is character
• If the values contain a single character (a-Z + underscore + .) AND a numeric, then the SAS type is numeric (with special missing values).
• null is set to either '.' or '' depending on the assigned or derived type per the above rules. If entire column is null then the type will be numeric.

The following table illustrates the formats applied to columns under various scenarios:

JS Values	SAS Format
'a', 'a'	$char1.
0, '_'	best.
'Z', 0	best.
'a', 'aaa'	$char3.
null, 'a', 'aaa'	$char3.
null, 'a', 0	best.
null, null	best.
null, ''	$char1.
null, 'a'	$char1.
'a'	$char1.
'a', null	$char1.
'a', null, 0	best.

Validation is also performed on the values. The following combinations will throw errors:

JS Values	SAS Format
null, 'aaaa', 0	Error: mixed types. 'aaaa' is not a special missing value.
0, 'a', '!'	Error: mixed types. '!' is not a special missing value
1.1, '.', 0	Error: mixed types. For regular nulls, use null

Variable Format Override

The auto-detect functionality above is thwarted in the following scenarios:

• A character column containing only null values (is considered numeric)
• A numeric column containing only special missing values (is considered character)

To cater for these scenarios, an optional array of formats can be passed along with the data to ensure that SAS will read them in correctly.

To understand these formats, it should be noted that the JSON data is NOT passed directly (as JSON) to SAS. It is first converted into CSV, and the header row is actually an infile statement in disguise. It looks a bit like this:

CHARVAR1:$char4. CHARVAR2:$char1. NUMVAR:best.
LOAD,,0
ABCD,X,.

To provide overrides to this header row, the tables object can be constructed as follows (with a leading '$' in the table name):

let specialData={
  "tablewith2cols2rows": [
    {"col1": "val1","specialMissingsCol": "A"},
    {"col1": "val2","specialMissingsCol": "_"}
  ],
  "$tablewith2cols2rows":{"formats":{"specialMissingsCol":"best."}
  }
};

It is not necessary to provide formats for ALL the columns, only the ones that need to be overridden.

SAS Inputs / Outputs

The SAS side is handled by a number of macros in the macro core library.

The following snippet shows the process of SAS tables arriving / leaving:

/* convert frontend input tables from into SASWORK datasets */
%webout(FETCH)

/* some sas code */
data a b c;
  set from js;
run;

%webout(OPEN)  /* Open the JSON to be returned */
%webout(OBJ,a) /* Rows in table `a` are objects (easy to use) */
%webout(ARR,b) /* Rows in table `b` are arrays (compact) */
%webout(OBJ,c,fmt=N) /* Table `c` is sent unformatted (raw) */
%webout(OBJ,c,dslabel=d) /* Rename table as `d` in output JSON */
%webout(OBJ,c,dslabel=e, maxobs=10) /* send only 10 rows back */
%webout(CLOSE) /* Close the JSON and add default variables */

By default, special SAS numeric missings (_a-Z) are converted to null in the JSON. If you'd like to preserve these, use the missing=STRING option as follows:

%webout(OBJ,a,missing=STRING)

In this case, special missings (such as .a, .b) are converted to javascript string values ('A', 'B').

Where an entire column is made up of special missing numerics, there would be no way to distinguish it from a single-character column by looking at the values. To cater for this scenario, it is possible to export the variable types (and other attributes such as label and format) by adding a showmeta param to the webout() macro as follows:

%webout(OBJ,a,missing=STRING,showmeta=YES)

The %webout() macro itself is just a wrapper for the mp_jsonout macro.

Configuration

Configuration on the client side involves passing an object on startup, which can also be passed with each request. Technical documentation on the SASjsConfig class is available here. The main config items are:

• appLoc - this is the folder (eg in metadata or SAS Drive) under which the SAS services are created.
• serverType - either SAS9, SASVIYA or SASJS. The SASJS server type is for use with sasjs/server.
• serverUrl - the location (including http protocol and port) of the SAS Server. Can be omitted, eg if serving directly from the SAS Web Server, or in streaming mode.
• debug - if true then SAS Logs and extra debug information is returned.
• verbose - optional, if true then a summary of every HTTP response is logged.
• loginMechanism - either Default or Redirected. See SAS Logon section.
• useComputeApi - Only relevant when the serverType is SASVIYA. If true the Compute API is used. If false the JES API is used. If null or undefined the Web approach is used.
• contextName - Compute context on which the requests will be called. If missing or not provided, defaults to Job Execution Compute context.
• requestHistoryLimit - Request history limit. Increasing this limit may affect browser performance, especially with debug (logs) enabled. Default is 10.

The adapter supports a number of approaches for interfacing with Viya (serverType is SASVIYA). For maximum performance, be sure to configure your compute context with reuseServerProcesses as true and a system account in runServerAs. This functionality is available since Viya 3.5. This configuration is supported when creating contexts using the CLI.

Using JES Web App

In this setup, all requests are routed through the JES web app, at YOURSERVER/SASJobExecution?_program=/your/program. This is the most reliable method, and also the slowest. One request is made to the JES app, and remaining requests (getting job uri, session spawning, passing parameters, running the program, fetching the log) are handled by the SAS server inside the JES app.

{
  appLoc:"/Your/Path",
  serverType:"SASVIYA",
  contextName: 'yourComputeContext'
}

Note - to use the web approach, the useComputeApi property must be undefined or null.

Using the JES API

Here we are running Jobs using the Job Execution Service except this time we are making the requests directly using the REST API instead of through the JES Web App. This is helpful when we need to call web services outside of a browser (eg with the SASjs CLI or other commandline tools). To save one network request, the adapter prefetches the JOB URIs and passes them in the __job parameter. Depending on your network bandwidth, it may or may not be faster than the JES Web approach.

This approach (useComputeApi: false) also ensures that jobs are displayed in Environment Manager.

{
  appLoc:"/Your/Path",
  serverType:"SASVIYA",
  useComputeApi: false,
  contextName: 'yourComputeContext'
}

Using the Compute API

This approach is by far the fastest, as a result of the optimisations we have built into the adapter. With this configuration, in the first sasjs request, we take a URI map of the services in the target folder, and create a session manager. This manager will spawn a additional session every time a request is made. Subsequent requests will use the existing 'hot' session, if it exists. Sessions are always deleted after every use, which actually makes this less resource intensive than a typical JES web app, in which all sessions are kept alive by default for 15 minutes.

With this approach (useComputeApi: true), the requests/logs will not appear in the list in Environment manager.

{
  appLoc:"/Your/Path",
  serverType:"SASVIYA",
  useComputeApi: true,
  contextName: "yourComputeContext"
}

More resources

For more information and examples specific to this adapter you can check out the user guide or the technical documentation.

For more information on building web apps in general, check out these resources or contact the author directly.

As a SAS customer you can also request a copy of Data Controller - free for up to 5 users, this tool makes use of all parts of the SASjs framework.

Star Gazing

If you find this library useful, help us grow our star graph!

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Krishna Acondy 💻 🚇 📝 🖋 🤔 📹	Yury Shkoda 💻 🚇 🤔 ⚠️ 📹	Mihajlo Medjedovic 💻 🚇 ⚠️ 👀	Allan Bowe 💻 👀 ⚠️ 🧑‍🏫 🚧	Muhammad Saad 💻 👀 ⚠️ 🧑‍🏫 🚇	Sabir Hassan 💻 👀 ⚠️ 🤔	VladislavParhomchik ⚠️ 👀
Rud Faden 📓 📖	Sara 📓 📦

This project follows the all-contributors specification. Contributions of any kind welcome!

adapter's Issues

jobViaWeb config override not applied

In request() function we can pass config parameter, to override the global sasjsConfig.
Currently execution method jobViaWeb does not overide with the config that is passed.

Compute API debug log issue

When _debug is 131, for historical reasons (when log was sent through the output) we top'd and tailed the JSON with:

WEBOUTBEGIN<< and >>weboutEND<<

This isn't being parsed in compute instances.

Web approach loginCallback

Executing a request with a web approach returns a promise.reject when loginCallback is called. This behaviour breaks the client's login procedure.

Another issue is that the web approach does not work when debug is switched on.

Console log warning when the serverUrl does not contain the protocol

The serverUrl should always contain the http:// or https:// prefix (if the value is not blank or missing)

To aid debugging, we should warn where this is not the case

folders list

Fix Bug:

sasjs/cli#174

Explanation
We are following the Unix mv command functionality.

Source folder: /Folder/folder1
Target folder: /Folder/test/folder2

If folder2 exists: that means folder1 will be put inside of it.
Othervise: folder1 will be renamed to folder2

UPDATE
Bug is actually in CLI, adapter function is working as intended

Add new function list

It would list all of the children folders (and their types)

break big files into smaller more manageable files

handle context create/edit/delete response with the status 'forbidden'

on startup the serverType value should be written to the log

I was just debugging an app configured for SAS9 that was calling Viya

Turned out the serverType was defined as servertype

We should write message to the log on adapter instantiation to display key values (such as servertype, serverUrl etc) for faster debugging, eg:

The SASjs Adapter has been instantiated!  The following default values are set:
  serverUrl:
  serverType:
  debug:
  (if Viya we can show the extra variables)
To override the defaults, simply pass an updated config in the request method.

Start job execution and return immediately

We need a method that will start executing a job via the compute API, and immediately return with session information.

/**
   * Kicks off execution of the given job via the compute API.
   * @returns an object representing the compute session created for the given job.
   * @param sasJob - the path to the SAS program (ultimately resolves to
   *  the SAS `_program` parameter to run a Job Definition or SAS 9 Stored
   *  Process). Is prepended at runtime with the value of `appLoc`.
   * @param data - a JSON object containing one or more tables to be sent to
   * SAS. Can be `null` if no inputs required.
   * @param config - provide any changes to the config here, for instance to
   * enable/disable `debug`. Any change provided will override the global config,
   * for that particular function call.
   * @param accessToken - a valid access token that is authorised to execute compute jobs.
   * The access token is not required when the user is authenticated via the browser.
   */

Adapter build does not export type declarations for node

The build process for the adapter creates a package that works both in the browser and in Node.js.

The node version is located in a folder called node within the package, and the browser version is at the root of the package.

Type declarations for all classes and interfaces used in the adapter are exported at the root level but not in the node folder. This means that we can't use the types in a Node project, and the TypeScript compiler will complain about not being able to find the type declarations.

Use axios instead of fetch

fileURI undefined during jobDefinition

File upload function

Issues with executin SAS job with access_token

Compute API

Executing the computeApi with access_token fails with unauthorized
Headers are passed wrong to the jobDefinition request.

JES API

At the beginning of the function, it checks the session, since you are not logged in and accessing with token, session is false so it stops the execution.

Check needs to be added if access_token is present, to ignore the session state.

FileUploader needs catch

For example failed to fetch error is not handled. It should be returning error message.

Sessions are not working

Tests failed

enable `_debug` and '_contextname' for file upload

FileUploader issue is waiting for adapter to enable debug in request.

Forbidden when creating contexts

The following was returned from the CLI:

➜  /tmp sasjs context create -s SharedCompute.json -t globviya
An error has occurred when processing context. Forbidden. Check your permissions and user groups. 
traceId: f3020dddb774e035,path: /compute/contexts
:q!

the issue was that the CLIENT was generated without the correct scopes. We should update the warning text to:

An error has occurred when processing context. Forbidden. Check your permissions and user groups, and also the scopes granted when registering your CLIENT_ID.

Public access denied error is not handled when trying to log in

add editContext option

Our deployment workflow for the Viya version of Data Controller is this:

• Create the SAS Compute Context
• Deploy the Job Execution services (sasjs deploy)
• Execute the setup script (using sasjs run)
• Update the autoexec in the Context created earlier with the generated DC Libname

For this last part, we need the ability to edit compute apis using SAS js.

The documentation is here: https://developer.sas.com/apis/rest/Compute/#update-a-context-definition

callback validation

The sasjs.request parameter accepts a callback function to gracefully handle the situation where a user has submitted a request, but the sas cookie session has timed out.

This callback function can be used to present a login screen to the user. After login, the request is resubmitted.

We need to perform some validation to ensure that this parameter, if provided, is correct. For instance, a boolean should not be passed.

Relative service paths do not work when they have multiple nested folders

sasjs request should perform validations on the input data object

For example:

"indata": [
  [{"data": "value"}]
]

Viya - after one login attempt, valid or invalid, login does not work again unless page reloaded

JES Job Execution: Failed requests on each execution attempt

There seem to be two failed requests on each job execution attempt:

• One for uploading tables as files
• One for posting the actual job

This is likely CSRF related and should be easily fixed as we're handling it okay in the other approaches.

When service is not found, error message is not helpfull.

SAS Viya:

SAS 9: (response actually includes Stored process not found)

Ideally, response details would include message like: Service not found on the server.

---

Example of reponse object:

error: {
  message: 'Service not found on the server.',
  details: { serviceUrl: 'common/someService' },
  raw: {}
}

getExecutableContexts() doesn't work

Debug configuration is not passed down properly

SessionManager and SASViyaApiClient have no concept of debug state.
This causes a lot of clutter in the console even when debug is turned off at the top level in the SASjs config.

JES and Compute API in some cases doesn't add log to the requests array, when service does not finish

executeJobViaWeb auto retry request not working

In retry call, params are missing:

Viya deployment when target folder already exists

checkSession is prone to failure when non English language is set in browser

Currently checkSession is looking for You have signed in. to confirm login, and that part is relative to language set in browser.

`executeJobViaJesApi` error handling if job does not exist

Current behavior

If job does not exist, execution of request method fails without finishing.

Desired behavior

If job does not exist, execution finishes and result includes the message about job not found.

---

Docs link: https://adapter.sasjs.io/classes/reflection-708.reflection-180.sasjs.html#request

adapter.login improvements

Currently when using adapter.login() , if the user is already logged in, we do not attempt a login.

This behaviour is ok, however it is not transparent to the user, also can result in unexpected behaviour eg if blank values or incorrect values are passed in.

We should do two things:

1. validate that values are indeed passed
2. write a message to the log indicating that a login was not attempted as a valid session already exists.

Failing test with JES approach

The request is not added to SASjs.sasjsRequests when using the JES approach, which is causing this test to fail.

Standardize error response by creating Error interface

Currently, there isn't a standard error response throughout the library. So error responses can be unexpected in some places.

logon fails after session expires when prompted for administrator groups (reported)

Not yet confirmed but we have reports that during an active session, when the token expires, the redirect to the login screen is breaking when the user is in the admin group (so, prompted to add the admin groups).

So, the adapter is not handling the admin prompt on token expiry.

Cleanup sessions after use

Sesssions (compute + api) will automatically expire after a particular interval (eg 15 mins) but to improve scalability and maximise performance we should tidy up ourselves after making requests

Definition of done - after a request is complete (logs, results etc extracted) we should send a delete request to remove that particular session.

update doc & doc build process with line numbers

To make the documentation more useful it would be helpful to be able to click through to function definitions.

A plugin is available here: https://stackoverflow.com/questions/62502008/link-to-source-code-from-typedoc-documentation/63418909#63418909

FileUpload is missing csrfTokenWeb setter callback

Behavior

File upload function is doing retry request to capture CSRF token if is not present.
We need to pass callback in constructor param so token is forwarded and set in main class.

sessionInactiveTimeout issue on session expiry

There are some cases where the session timeout is not being handled

Run unit tests as part of branch and PR builds

CSRF is not the same for WEB and API request approach

Related issue: #18 FIle upload
File upload is using WEB approach and WEB token.

deployServicePack `force` parameter seems to not work

Clear cookies with path `SASLogon` and serverUrl domain before login

job with errors should have status failed

Issue: jobs with errors have been returned with the status completed.
Job to replicate:

data test;
infile cards;
input var $;
cards;
Yury
was
not
sorted
;run;
data check;
  set test;
  by var;
run;

Add more unit tests

All the code in the utils folder should be easily testable since all functions in there are pure.

For testing the code in SASjs and SASViyaApiClient there will be some mocking/stubbing out required.

Access token is not passed to `waitForSession`

request method `sasJob` parameter should accept full path

Current behavior

If appLoc in global config is not present and to parameter is provided full path eg. /Some/path/sasjob then execution will fail.

Desired behavior

If sasJob parameter is provided without starting slash, then it should use appLoc/sasJob.
But if parammeter is provided with starting slash then appLoc should be ignored and full path provided should be used.

---

https://adapter.sasjs.io/classes/reflection-708.reflection-180.sasjs.html#request