matthewtckr/pdi-zendesk-plugin

pdi-zendesk-plugin's Issues

Zendesk Input Users step does not stop immediately

When a transformation is aborted manually, the Zendesk Input Users step does not stop in a reasonably short time. Instead it continues to fetch all remaining User records.

The step should be updated to identify when the transformation is trying to stop, and abort the processing.

rate limiting from zendesk

I hit another problem: I am often killed by Zendesk's rate limiting. From what I see, this exception isn't caught by the current code. Do you confirm ? (I'll work on a PR)

Zendesk returns userids that are no longer available

Hi,

I'm trying to retrieve the list of users this time… and I also get a 404 for some ids. But there is no code to catch this error, and no error stream for the Zendesk Input Users step, so my transformation dies.

I'm trying to add this to the plugin, but as I haven't been programming in java for a while… :)

New exception

Hi,

I just started having this exception:

java.lang.NullPointerException
        at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1439)
        at java.text.DateFormat.parse(DateFormat.java:364)
        at org.pentaho.di.trans.steps.zendesk.ZendeskTicketAuditHistory.processEvents(ZendeskTicketAuditHistory.java:215)
        at org.pentaho.di.trans.steps.zendesk.ZendeskTicketAuditHistory.createNextAudit(ZendeskTicketAuditHistory.java:117)
        at org.pentaho.di.trans.steps.zendesk.ZendeskInputTicketAuditData.addAudit(ZendeskInputTicketAuditData.java:61)
        at org.pentaho.di.trans.steps.zendesk.ZendeskInputTicketAudit.processRow(ZendeskInputTicketAudit.java:119)
        at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
        at java.lang.Thread.run(Thread.java:748)

From looking at the code, it seems that this is where it fails:

this.dueAt = DUE_AT_DATE_FORMAT.parse( fieldValue );

This is declared as

  private static final String DUE_AT_DATE_FORMAT_STRING = "yyyy-MM-dd HH:mm:ss";
  private static final SimpleDateFormat DUE_AT_DATE_FORMAT = new SimpleDateFormat( DUE_AT_DATE_FORMAT_STRING );

It probably means that the string hasn't this format ? But how can I extract this string and see what comes in to debug this ?

Regards

Add AgentMacroReference events to Ticket Audit step

As an analyst for a support organization, I want to understand how often macros are used by support agents.

Need data from getTicketField

Hi,

Our users store important metadata from https://peopledoc.zendesk.com/api/v2/ticket_fields/{id}.json (some metadata in tags). Is there a way to retrieve them ?

I don't see one implemented right now, does it means extending "Input Ticket Fields"… if so, is there an easy way to do it ?

Feature to filter deleted tickets from get input incrementally step

Hi,

I have a request. Within our company we have a kettle project where we use your plugin. Recently someone has done some testing against Zendesk and has deleted some tickets. Since then jobs are failing.
I think that I have managed to track it down to the zendesk incremental input step. Within ZendeskInputIncremental.java on line 95, the getTicketsIncrementally method is called. I believe this method corresponds to the following API call of Zendesk (https://developer.zendesk.com/rest_api/docs/core/incremental_export#start-time). Within that webpage it states that tickets that are deleted can be filtered using “status”: “deleted”.

Snippet from the website

Excluding deleted tickets
Starting on October 16, 2016, user-provided information in deleted tickets is scrubbed 30 days after deletion. Reporting on deleted tickets remains intact because the ticket record still exists. However, user-provided information (as opposed to system-provided information) is scrubbed with an X for text fields, a 0 for numeric fields, or nothing for fields that didn't have a value. You can exclude these tickets by filtering out any tickets with a "status": "deleted" value.

It seems that the zendesk incremental input step fetches all ticket ids, and does not filter out deleted tickets. Could you look into this maybe and add such a feature?

Null pointer exception in Input Ticket Audit

Hi and thanks for writing this plugin.

I'm having this error:

2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - ERROR (version 7.1.0.6-73, build 1 from 2017-11-10 10.59.25 by marc) : Unexpected error
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - ERROR (version 7.1.0.6-73, build 1 from 2017-11-10 10.59.25 by marc) : java.lang.NullPointerException
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at org.pentaho.di.trans.step.BaseStep.handlePutError(BaseStep.java:1669)
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at org.pentaho.di.trans.step.BaseStep.access$300(BaseStep.java:147)
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at org.pentaho.di.trans.step.BaseStep$DefaultRowHandler.putError(BaseStep.java:4189)
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at org.pentaho.di.trans.step.BaseStep.putError(BaseStep.java:1650)
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at org.pentaho.di.trans.steps.zendesk.ZendeskInputTicketAudit.processRow(ZendeskInputTicketAudit.java:121)
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2017/12/06 14:16:08 - Zendesk Input Ticket Audits.0 - 	at java.lang.Thread.run(Thread.java:748)

Looking at the code, if I understand correctly,

     if ( 404 == zre.getStatusCode() ) {
        putError( getInputRowMeta(), row, 1L, zre.toString(),
          getInputRowMeta().getValueMeta( ticketIdFieldIndex ).getName(), zre.getStatusText() );

I get a 404 from the server. But I get a null pointer exception instead of the ticketID in the error message. No idea why... so I'm stuck.

Please tell me what I can provide…

Regards.

Add step to perform actions on Suspended Tickets

Examples

Hi Matthew, I've installed your plugin but have some problem with getting commends and other ticket's attributes via pdi. Can you pls commit your pdi examples? Many thanks!

Can't get any data

While trying to get data from Zendesk through the Zendesk input. I'm getting the following error messages:
2018/02/20 09:43:22 - Zendesk Input Organizations.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : Unexpected error 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : org.zendesk.client.v2.ZendeskResponseException: HTTP/401: Unauthorized 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.zendesk.client.v2.Zendesk.complete(Zendesk.java:2130) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.zendesk.client.v2.Zendesk.access$1400(Zendesk.java:91) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.zendesk.client.v2.Zendesk$PagedIterable$PagedIterator.hasNext(Zendesk.java:2336) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.pentaho.di.trans.steps.zendesk.ZendeskFacade$ZendeskIterable$ZendeskIterator.hasNext(ZendeskFacade.java:388) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.pentaho.di.trans.steps.zendesk.ZendeskInputOrganizations.processRow(ZendeskInputOrganizations.java:126) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at java.lang.Thread.run(Unknown Source) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - Caused by: org.zendesk.client.v2.ZendeskResponseException: HTTP/401: Unauthorized 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.zendesk.client.v2.Zendesk$PagedAsyncListCompletionHandler.onCompleted(Zendesk.java:1946) 2018/02/20 09:43:22 - Zendesk Input Organizations.0 - at org.zendesk.client.v2.Zendesk$PagedAsyncListCompletionHandler.onCompleted(Zendesk.java:1924)

As I see it. There is an issue with the authentication. I'm using the same credentials while connecting directly to the Zendesk portal successfully.

What do I missed?

Built-in Rate-limiting for all steps

In reviewing PR #32, only a few steps have added support for dealing with the Rate Limit applied by Zendesk APIs.

Some centralized system should be created (extending ZendeskClient?) to centralized all of the retry logic, so all future steps can benefit from the feature.