Giter Site home page Giter Site logo

intersystems-ru / cache-mdx2json Goto Github PK

View Code? Open in Web Editor NEW

This project forked from eduard93/cache-mdx2json

10.0 11.0 9.0 709 KB

RESTful web api for MDX to JSON transformation (plus JSONP and XML/A) for InterSystems Caché. Also provides information about DeepSee objects.

License: MIT License

ObjectScript 98.38% C++ 1.62%
intersystems intersystems-deepsee rest-api rest mdx intersystems-cache

cache-mdx2json's Introduction

Cache-MDX2JSON

RESTful web api for MDX2JSON transformation (also JSONP and XML/A). Also supports requests about Dashboards and Widgets. Supports Caché 2014.1+.

Installation

  1. Download Installer.cls.xml (from MDX2JSON folder in repository or releases page) and import it into any namespace (via Studio or SMP or $System.OBJ.Load())

  2. Run in terminal (import namespace) under user with %All role:

     Do ##class(MDX2JSON.Installer).setup()
    

Offline Installation

  1. Download zip and unpack it.

  2. Import Installer.cls.xml (from MDX2JSON folder in unpacked archive) and import it into any namespace (via Studio or SMP or $System.OBJ.Load())

  3. Run in terminal (same namespace as 2) under user with %All role:

     Set pVars("SourceDir") = {SourceDir}
     Do ##class(MDX2JSON.Installer).setup(.pVars)
    

where:

  {SourceDir} is a directory where you unpacked zip \ MDX2JSON (see 1).

For information on how to work with this RESTful web API please refer to included documentation.

What Installer does

Regardless of installation method chosen, here's the list (by the order of appearance) of what installer does:

  1. If Namespace variable is undefined, set it to MDX2JSON
  2. Create MDX2JSON role
  3. If Namespace does not exist then create it
  4. If SourceDir is provided then import and compile all files from there. Otherwise download import and compile all required files from GitHub
  5. If /Namespace web application does not exist then create it and give it MDX2JSON role
  6. If %All namespace (used for mapping purposes) does not exist then create it and map MDX2JSON package and ^MDX2JSON global there
  7. Map MDX2JSON package and ^MDX2JSON global into SAMPLES namespace
  8. If User and Password variables are provided, then create User and give him MDX2JSON role

Additional installation parameters

As a first parameter to Do ##class(MDX2JSON.Installer).setup(.pVars) you can pass pVars - a local array of additional variables (see sample in Offline Installation step 3).

  • Namespace is a namespace you want to install MDX2JSON to (Not namespace with dashes). If it does not exist it would be created automatically. If it does exist only MDX2JSON package would be overwritten. WebApplication would be named /Namespace. Strongly not recommended to change the default. [MDX2JSON]
  • User is a Caché user to create or modify. He will be given SELECT access to %DeepSee_Dashboard.Definition table in Namespace
  • Password must be supplied alongside User parameters
  • SourceDir - all xmls from this directory would be imported and compiled
  • Import - import code from GitHub or SourceDir, defaults to 1, set to 0 to skip import

Update

  1. Run in terminal (namespace where you installed MDX2JSON):

    Do ##class(MDX2JSON.Installer).Update()
    

You can also supply parameters such as fork, desired branch/commit, target namespace, authorization information. Please refer to Caché documentation of MDX2JSON.Installer class for correct syntax.

Uninstall

  1. Run in terminal (any namespace from where MDX2JSON package can be accessed):

    Do ##class(MDX2JSON.Installer).Uninstall()
    

This action would delete MDX2JSON namespace, database (with physical directory) and web application. Mappings and %DB_MDX2JSON role and resource would also be deleted.

Requests

These are the possible requests to web application (add param ?Namespace={Desired Namespace} to query another namespace cubes

URL Type Body (JSON) Response Description
MDX POST { "MDX":"QUERY" } JSON Results of MDX execution
MDX2JSONP POST { "MDX":"QUERY" } JSONP Results of MDX execution
MDXDrillthrough POST { "MDX":"QUERY" } JSON Results of MDX execution
MDX2XMLA POST { "MDX":"QUERY" } XMLA Results of MDX execution
Dashboards GET JSON All dashboards
Dashboards POST {Folder:"FolderName"} JSON All dashboards in FolderName. Empty FolderName for root scope.
Dashboard POST {Dashboard:"DashboardName"} JSON All widgets in a dashboard, with filters as part of dashboard
Widgets POST {Dashboard:"DashboardName"} JSON All widgets in a dashboard, with filters as part of widgets
DataSource POST {DataSource:"Pivot fullname"} JSON All info about Pivot
Action/:Cube/:Action POST {context object} JSON Execute cube action
Filters POST { "DataSource": "DataSourceName.ext", "Values":1, Search:"SearchTerm", "RelatedFilters": [ {"Filter": "Filter", "Value": "Value"}]} JSON All filters for DeepSee DataSource (cube, pivot, kpi, metric) with values (if Values = 1, set to 0 or omit otherwise). If Search is not empty only filter values, containing search term would be returned. RelatedFilters - other filters and their values to limit search.
Format GET JSON Default formatting
TermList POST {"TermList":"TermListName"} JSON Termlist key-value array
Config POST {"Application":"AppName", "Config":"value"} JSON Set config for arbitrary application for current user
Config/:Application GET {"Application":"AppName"} JSON Get config for Application for current user
Favorites GET JSON Array of current user favorites
Favorites/:Item POST JSON Add favorite item
Favorites/:Item DELETE JSON Remove item from favorites
PivotVariables/:Cube GET JSON Get all pivot variables for cube
Test GET JSON Test info
Logout GET JSON Close session

Example

Request URL: http://localhost:57772/MDX2JSON/MDX?Namespace=Samples

Request type: POST

Request body:

{"MDX": "SELECT NON EMPTY [Product].[P1].[Product Category].Members ON 0,NON EMPTY [Outlet].[H1].[Region].Members ON 1 FROM [HoleFoods]"} 

Result

Please note that corresponding cube must be compiled and built beforehand.

Localization

If requested data or meta-information is available in several different languages, you can choose one, by supplying Accept-Language header to your HTTP request, formed in accordance with RFC 2616.

Troubleshooting

If something goes wrong, server must report an error in the following format {Error : "Error description"}, and usually that is a good indicator of what went wrong. If you received an error in another format or an error without "Error description" please file an issue here.

Problem Solution
CSP Error User does not have enough rights. Configure User or Webapplication roles
Authenticated access Web application must have password access, resource for database with MDX2JSON must have public RW rights enabled
No dashboards Configure roles. Change dashboard scope
DeepSee errors Build and compile DeepSee cube(s)
MDX errors Don't forget to escape JSON strings here
Installation errors Usually problems arise when installation is run under user without %All permissions. To repair the install rerun it under correct user. If the error persists then file an issue with installation log (terminal output) attached

Debugging

Use $$$Debug macro. It would evaluate as true only if there is a "Debug" URL parameter present. Example request URL:

    http://localhost:57772/MDX2JSON/MDX?Namespace=Samples&Debug

Use with post conditional expressions, or other flow control statements:

	w:$$$Debug "debugging"
	if $$$Debug { w "debugging" } else { w "not debugging"}

Also available are $$$Public and $$$Private macros. Evaluates to true based on request port (80 and 443 are public, private otherwise).

Querying

Querying this project is done via REST web client. It may be a standalone application or a browser plug-in, a number of different solutions are available.

For Google Chrome, install Advanced REST client extension. For Firefox, install REST client extension. Open installed extension and set the following parameters:

  • URL to required web api method, e.g.: http://serverip:port/mdx2json/Dashboard?Namespace=Samples
  • Request type to GET or POST
  • Payload to {"Dashboard":"Listing with Filters.dashboard"}
  • Content-Type to application/json (only in Advanced REST client)

Press Send button to view results (depending on your server configuration you may be asked to provide valid login/password to access MDX2JSON api).

Settings

User can save and get arbitrary settings for an abstract application (usually - Namespace). Settings are accessible throughout the system. To set a setting for a user, execute: do ##class(MDX2JSON.Users).SetConfig(Application, SettingsValue, Username) or send a request at a corresponding POST /Config with the JSON body containing Application property and Config property which can be a string or a JSON object. Note that in a WEB context user is not allowed to specify a username, it's calculated automatically. To get a setting for an abstract application write ##class(MDX2JSON.Users).GetConfig(Application, Username) or send a request at a corresponding GET /Config/:Application path.

User can have a MDX2JSONSettings role which allows him to get/set a default setting for an application. If the user (any user) does not have a setting for an Application, the default setting for an application would be used.

Calling MDX2JSON from another server

Here's an example.

Development

To develop MDX2JSON you need:

  1. Install MDX2JSON in MDX2JSON namespace

  2. Install Cache-Tort-Git

  3. In terminal, MDX2JSON namespace execute:

     set ^Git("settings","hook") = $lb("MDX2JSON.Tests","OnCommit")
     set ^Git("settings","groupByFolder") = 1
    
  4. Activate Cache-Tort-Git for MDX2JSON namespace

  5. Commit all changes via Studio

KPI

To use KPIs and display row name add this method to KPI class

ClassMethod %OnGetKPIPropertyInfo(ByRef pList As %String, pPropNo As %Integer, pModelId As %String = "") As %Status
{
    Set pPropNo = pPropNo + 1
    Set pList(pPropNo) = "%series"
    Set pList(pPropNo, "defaultValue") = ""
    Set pList(pPropNo, "columnNo") = pPropNo
    Quit $$$OK
}

Postman

You can use Postman to query MDX2JSON API. Collection. Environment.

cache-mdx2json's People

Contributors

bot avatar eduard93 avatar evshvarov avatar frisle avatar jakcpto avatar nikitaeverywhere avatar njektt avatar sergeymi37 avatar v23ent avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cache-mdx2json's Issues

Please, provide better API documentation.

The result of example provided in a readme.md file is confusing.

To get the name of the first column I need to retrieve this value:
result.Cols[0].tuples[0].children[0][0].caption

Everything I can guess here is
result.Cols[DIMENSION_INDEX?].tuples[0].children[0][COLUMN_INDEX].caption

What is the meaning of other indexes and dimensions? Thank for your response.

Last version doesn't compile while installation

2016-07-30 12:15:39 0 MDX2JSON.Installer: ОШИБКА #5475: Ошибка при компиляции программы: MDX2JSON.Users.1. Ошибок: ERROR: MDX2JSON.Users.cls(GetConfig+2) #1002: Invalid character in tag : 'set:((config="")||($Roles[$$$DefaultSettingsRole))' : Offset:30 [zGetConfig+2^MDX2JSON.Users.1]

DeepSeeWeb, WriteAddons error

User only with only one resource to concrete database
{"Error":"ERROR #5002: Cache error: <PROTECT>zWriteAddons+3^MDX2JSON.REST.1 *c:\\intersystems\\ensemble\\mgr\\"}

I'm not going to use any addons.
Why is it trying to write something to CACHESYS Database?

Locale default formatting

It is important for LPT to retrieve default number formatting based on locale settings. If this is possible, please, provide additional parameter like "defaultFormatting" of type "# ###,##" or replace emty "format" properties with such mask.

2015-01-22_205922

Portlet registration service for DSW

Description

We need more simple and clear addons installation for DSW. At present, to install addon, user need two files *.js with client-side widget code and *.xml with custom portlet. It is not convenient to supply two different files as one addon, so I suggest to create service which can automatically create portlet classes for DSW addons. As a result, only one file would be needed to install DSW addon.

Requirements

REST service should receive array of portlet names to create. Portlets are to be created only in case of absence.
Input example: ["DSW.CusmtomMap", "DSW.TestWidget"]

This service would be called on DSW startup with all addons names as input.

Localization support

Get localization support
SQL.ru thread
Cache docs
HabraHabr
q relative quality factor
Accept-Language

Test on:
http://146.185.172.47/MDX2JSON/MDX?Namespace=STC

{MDX:"SELECT NON EMPTY [TrainingCubeRelation].[Year].[H1].[Year].Members ON 0,NON EMPTY [TrainingCubeRelation].[Course].[H1].[CourseName].Members ON 1 FROM [SubGroupCube]"}

Reference code:
set %session.Language="en"
do ##class(%MessageDictionary).SetSessionLanguage("en")
s %response.Language="en"
w 1,%session.Language,2,%response.Language,3,$$$SessionLanguage,4

Dashboards list

Do not filter dashboards (currently only listing dashboards in "mobile" folder).

%All mapping troubleshooting

%All namespace exists as early as 2011.1 (maybe more, hadn't checked).
Sometimes mapping is missing in some namespace(s), usually SAMPLES.

Long output/Buffer?

When I send dashboard request
POST http://localhost:57772/MDX2JSON/Dashboard?Namespace=Samples
With headers
Content-Type: application/json
And body
{"Dashboard":"Widget Examples/All Charts.dashboard"}
The output is garbage.

The problem arises in MDX2JSON.Utils class, GetDashboard method.

The problem seems to be related to the length of the output.
Consider the following modified GetDashboard method:

ClassMethod GetDashboard(pDashName As %String) As %Status
{
    try {
        set st = ##class(MDX2JSON.DashboardFilters).OpenDashboardByName(pDashName, .dash)
        return:($$$ISERR(st)) st

        set st = ##class(MDX2JSON.DashboardFilters).WidgetsToProxyObject(dash, .widgetlist)
        return:($$$ISERR(st)) st

        //do widgetlist.displayInfo.%ToJSON()
        //do widgetlist.info.%ToJSON()
        set filters = widgetlist.filters
        set widgets = widgetlist.widgets


        #dim widgets As List of %ZEN.proxyObject
        for i=1:1:5 {
            set widget = widgets.GetAt(i)
            zw widget
        }
        //set widget = widgets.GetAt(5)
        //zw widget
    } catch ex {
        do ex.Log()
    }
    return st
}

After I tested this method (by changing the total number of cycles in for i=1:1:5), I found out, that:

  • loop size = 5 (as in code) output is garbage
  • loop size = 4 output is wrong, but readable
  • loop size = 3 output is correct

Also, it doesn't matter from where to start. So, for example for i=4:1:5 executes ok, but for i=6:1:11 output is wrong.

Cache for Windows (x86-64) 2016.1 (Build 656U) Fri Mar 11 2016 17:42:42 EST

@dmitry-zasypkin

Multidimensional MDX

SELECT {[Measures].[Units Sold],[Measures].[Amount Sold]} ON 0,NON EMPTY [DateOfSale].[Actual].[YearSold].Members ON 1 FROM [HoleFoods] %FILTER {[DateOfSale].[Actual].[YearSold].&[2009],[DateOfSale].[Actual].[YearSold].&[2010]} %FILTER [MyRegion].&[North]

Causes "Multidimensional (>2) queries are not supported" error.

Input checks

Centralized check that we got an object successfully

Security

Elaborate on Dispatch:
Set tAccess=$SYSTEM.Security.Check($Piece($zu(90,21,$namespace),"^",4))

Add pivot variables support

I believe we have the control description, but we need to supply this controls with values, which can go from the list, termlist or KPI class.

SAMPLES database is mandatory

Why installation fail, if my instance does not have SAMPLES database?

2017-02-02 12:28:09 0 : Mapping MDX2JSON package to %All namespace
2017-02-02 12:28:09 0 : Mapping MDX2JSON package to Samples namespace
2017-02-02 12:28:09 0 MDX2JSON.Installer: ERROR #420: Database SAMPLES does not exist
2017-02-02 12:28:09 0 MDX2JSON.Installer: ERROR #ConfigFailed: Unknown status code: ConfigFailed )
ERROR #420: Database SAMPLES does not exist
2017-02-02 12:28:09 0 MDX2JSON.Installer: Installation failed at 2017-02-02 12:28:09

I think it should be some check if this namespace exists.

Non standart errors

All errors from server must come in a unified JSON format:
{ Error : " Error description" }
If you got an error message, not fitting this structure, please report this happenstance here.

List of files in folder for DSW

To support custom themes feature of DSW we need to be able to receive list of files in themes folder.
REST service should receive folder name as input and return list of files contained in this folder.

Input example: "/css/themes"
Response: ["theme1.css", "theme2.css", "theme3.css"]

Strange Characters in saved settings.json leads to failed loading

When saved, our json contains a lot of wrongly encoded (and duplicated) characters
in any placer where we used a non ascii Dashboard or pivot name.

Trying to load such a config fails to parse and does require manually editing the respective MDX2JSON Global in Caché to restore the settings.

Example (the correct name should have been Dashboards/Übersicht:
Dashboards/����������������������������� ��������������������������������������������������������������������bersicht

Latest installation fails to get widgets list

2015-05-24_165607
I have just installed latest MDX2JSON (with automatic download from github option), and got this error when opened dashboards with configured LPT. Additionally, REST console return HTTP / 1.1 405 Method Not Allowed forever until I comment the line of code which returs error, then just returns nothing.

Could you please check it? Local check failed, and 37.139.4.54 is unreachable at the moment.

Where is MDX2JSON.MDX2JSON?

I am unable to install MDX2JSON probably because of missed file.

Compiler says that there is no include file 'MDX2JSON.MDX2JSON'.

How can I install this properly?

Incorrect formatting on different locales, default formatting

NumericGroupSeparator was changed from "," to " ". Locale has been applied. Here is the responce from /Format:

{
    "AM":"AM",
    "DateFormat":"YYYY-MM-DD",
    "DateSeparator":".",
    "DecimalSeparator":",",
    "MinusSign":"-",
    "MonthAbbr":" Янв Фев Мар Апр Май Июн Июл Авг Сен Окт Ноя Дек",
    "MonthName":" Январь Февраль Март Апрель Май Июнь Июль Август Сентябрь Октябрь Ноябрь Декабрь",
    "NumericGroupSeparator":" ",
    "NumericGroupSize":3,
    "PlusSign":"+",
    "PM":"PM",
    "TimeFormat":"hh:mm:ss",
    "TimePrecision":0,
    "TimeSeparator":":",
    "WeekdayAbbr":" Вос Пон Вто Сре Чет Пят Суб",
    "WeekdayName":" Воскресенье Понедельник Вторник Среда Четверг Пятница Суббота"
}

But the MDX2JSON still return incorrect formatting. Console shows the part of /MDX response:
2015-01-23_204708

I thought about making a demo on our server, but probably changing locale setting can be volatile so I left this idea. I think the problem is clear.

Also I suggest to parse any formatting mask (including default formatting) on the server side, just to avoid different conflicts. And applications should work in such way: MDX2JSON gives formatting masks, app applies it. LPT (or any other application) should not parse (generate) any masks, just apply them.

InterSystems IRIS install

<Namespace Name="%All" Create="yes" Code="CACHETEMP" Data="CACHETEMP" Ensemble="0">

Should be

<Namespace Name="%All" Create="yes" Code="IRISTEMP" Data="IRISTEMP" Ensemble="0">

MDX Parallel Execution

Class MDX2JSON.Utils
ClassMethod GetResultSet

Есть достаточно большая уверенность, что используемый вызов
set RS = ##class(MDX2JSON.ResultSet).%ExecuteDirect(pMDX,.Params,.pStatus)
обрабатывает запрос в один поток.

Если заменить на:
set RS = ##class(MDX2JSON.ResultSet).%New()
set st = RS.%PrepareMDX()
set st = RS.%ExecuteAsync(pMDX,.Params,1)

То должно начать работать значительно быстрее.
Код дан в качестве примера, использования другого метода. Для реального использования, надо его правильно встроить в контекст.

Error handling for malformed MDX

Is it possible to handle this request with response like { error: ERROR_TEXT }?

Note the Status Code:500 Internal Server Error here.

Remote Address: ...
Request URL: http://...
Request Method:POST
Status Code:500 Internal Server Error
Request Headersview source
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:ru,en-US;q=0.8,en;q=0.6,uk;q=0.4
Connection:keep-alive
Content-Length:277
Content-Type:text/plain;charset=UTF-8
Cookie:...; Username=root; CSPWSERVERID=...
DNT:1
Host:..
Origin:...
Referer:...
User-Agent:Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.45 Safari/537.36
Request Payloadview source
{,…}
MDX: "DRILLTHROUGH SELECT FROM [QueueCube] %FILTER [status].[H1].[status].&[Ожидает на текущую дату] %FILTER [ReceiverDepProfile].[H1].[ReceiverId].&[692] %FILTER [].[].[].&[] %FILTER [status].[H1].[status].&[Ожидает на текущую дату]"
Response Headersview source
CACHE-CONTROL:no-cache
Connection:close
CONTENT-LENGTH:58
Content-Type:text/json; charset=utf-8
Date:Mon, 29 Dec 2014 21:17:37 GMT
EXPIRES:Thu, 29 Oct 1998 17:04:19 GMT
PRAGMA:no-cache
Server:Apache

I think this happened because of %FILTER [].[].[].&[] .

Updater

Write an updater installer manifest

Compiling routine MDX2JSON.Installer.1

I am getting following error while executing this command:
do ##class(%Installer.Installer).InstallFromCommandLine("C:\InterSystems\m...\mgr\MDX2JSON\Installer.cls.xml","Namespace=UFT,SourceDir=C:\InterSystems....\mgr\MDX2JSON")

Compiling routine MDX2JSON.Installer.1
MDX2JSON.Installer.1.INT(25) ERROR #1026: Invalid command : 'Return:('st)' : Offset:11 [zDeleteNamespace+4^MDX2JSON.Installer.1]
TEXT: Return:('st) st

I tried to replace Return with Quit but than I start getting following error: MDX2JSON.Installer.1.INT(49) ERROR #1043: QUIT argument not allowed : 'C1)/' : Offset:19 [#;+5^MDX2JSON.Installer.1]
TEXT: &sql(CLOSE C1)
/

Wrong JSON data retrieved when complex column is configured manually

Take a look for this. Hope it will explain the problem.

2014-12-10_235324

In SAMPLES namespace, you can try this MDX:

SELECT NON EMPTY {
    [Outlet].[H1].[Region].Members,
    %LABEL(NONEMPTYCROSSJOIN(,{
        [Measures].[%COUNT],[Measures].[Units Sold]
    }),"ИТОГО","")
} ON 0,
NON EMPTY [Product].[P1].[Product Category].Members ON 1
FROM [HoleFoods]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.