firegento/firegento-logger

Framework for different logging adapters for Magento 1.

firegento-logger's Issues

How to log apache access and error log?

All our projects are hosted on shared hosting. We don't have access to install new packages.
I want to forward apache access and error log to remote logstash server.
Is there a way to do it? Does it make sense to write log forwarder with this extension?

JsonStream logger does not add line break after log entry

I want to use the JsonStream logger to have one line for each log entry, having the whole file in one line defeats that purpose

Bundled XMPPHP is not PHP7 compatible

File: /.modman/Firegento_Logger/src/lib/XMPPHP/XMLStream.php
> Line 266: Removed function "split" called
    split('}', $ns_tag);
> Line 567: Indirect variable, property or method access
    $handler[2]->{$handler[1]}($this->xmlobj[2]);
> Line 581: Indirect variable, property or method access
    $handler[3]->{$handler[2]}($this->xmlobj[2]);
> Line 587: Indirect variable, property or method access
    $handler[1]->{$handler[0]}($this->xmlobj[2]);
> Line 643: Indirect variable, property or method access
    $handler[2]->{$handler[1]}($payload);

Fatal error: File not found: lib/Graylog2-gelf-php/GELFMessage.php in /app/code/community/FireGento/Logger/Model/Graylog2.php on line 21

If you run (for example) a shell script without CD into the magento directory you see the following error:

ccarnell@web:~$ php /var/www/vhosts/magento-capistrano/current/shell/indexer.php -- --reindex cataloginventory_stock
PHP Warning:  include(GELFMessagePublisher.php): failed to open stream: No such file or directory in /var/www/vhosts/magento-capistrano/releases/20140708150751/lib/Varien/Autoload.php on line 93

Warning: include(GELFMessagePublisher.php): failed to open stream: No such file or directory in /var/www/vhosts/magento-capistrano/releases/20140708150751/lib/Varien/Autoload.php on line 93
PHP Warning:  include(): Failed opening 'GELFMessagePublisher.php' for inclusion (include_path='/var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/local:/var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/community:/var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/core:/var/www/vhosts/magento-capistrano/releases/20140708150751/lib:.:/usr/share/php:/usr/share/pear') in /var/www/vhosts/magento-capistrano/releases/20140708150751/lib/Varien/Autoload.php on line 93

Warning: include(): Failed opening 'GELFMessagePublisher.php' for inclusion (include_path='/var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/local:/var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/community:/var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/core:/var/www/vhosts/magento-capistrano/releases/20140708150751/lib:.:/usr/share/php:/usr/share/pear') in /var/www/vhosts/magento-capistrano/releases/20140708150751/lib/Varien/Autoload.php on line 93
PHP Fatal error:  Class 'GELFMessagePublisher' not found in /var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/community/FireGento/Logger/Model/Graylog2.php on line 75

Fatal error: Class 'GELFMessagePublisher' not found in /var/www/vhosts/magento-capistrano/releases/20140708150751/app/code/community/FireGento/Logger/Model/Graylog2.php on line 75

If you CD into the directory first, it runs OK:

ccarnell@web:~$ cd /var/www/vhosts/magento-capistrano/current
ccarnell@web:/var/www/vhosts/magento-capistrano/current$ php /var/www/vhosts/magento-capistrano/current/shell/indexer.php -- --reindex cataloginventory_stock
Stock Status index was rebuilt successfully

Ideally I need this issue fixing as I use some scripts in cron

Papertrail logger raises warnings with PHP7

Warning: A non-numeric value encountered in /var/www/htdocs/app/code/community/FireGento/Logger/Model/Papertrail/PapertrailSyslogMessage.php on line 59

https://github.com/firegento/firegento-logger/blob/develop/src/app/code/community/FireGento/Logger/Model/Papertrail/PapertrailSyslogMessage.php#L59

LICENSE.txt is OSL not GPL

Fatal error: Class 'Mage_Core_Model_Resource_Db_Abstract' not found

In a recent cron.php run with Magento CE 1.5.x my error catcher got a this status 255

Fatal error:

Class 'Mage_Core_Model_Resource_Db_Abstract' not found

In File:

`.../magento/vendor/firegento/logger/src/app/code/community/FireGento/Logger/Model/Resource/Db/Entry.php

On Line:

29

with firegento/logger (1.3.0) installed via composer.

just reporting, it's late and I couldn't look further into it.

other cron runs did well.

Missing hostname when logging with remote syslog to a logserver like papertrailapp.com

It just adds "NONAME":
Feb 14 18:00:03 NONAME NONAME: : PHP [PID: 32592] 2014-02-14T17:00:03+00:00 ERR (admin): cli shell/indexer.php REQUEST: - TIME: 1.951723s ADDRESS: - USER AGENT: - FILE: refactoring/app/code/core/Mage/Core/functions.php:247 Warning: session_start(): Cann

Test branch 'develop'

@daim2k5 @datenbrille Please test the branch 'develop' before releasing a new version to master branch. This is necessary due to extensive refactoring/cleanup/namespace change/...

Logger priority filter does not work (for sentry)

May duplicate #86 .

I use two targets: Magento default files + Sentry

Settings

| Path                                | Scope   | Scope-ID | Value|
| logger/default/priority             | default | 0        | default|

| logger/general/priority             | default | 0        | 4|
| logger/general/targets              | default | 0        | default,sentry|

| logger/sentry/adminhtml_enabled     | default | 0        | 0|
| logger/sentry/curl_method           | default | 0        | async|
| logger/sentry/frontend_enabled      | default | 0        | 0|
| logger/sentry/priority              | default | 0        | 3|
| logger/sentry/public_dsn            | default | 0        | MYDSN|

Magento 1.9.3.10
Firegento_Logger 1.9.0

Result:
Sentry still gets notices and warnings

Expected:
Sentry will only get errors and more severe logs

I still have no idea what it actually logs

Maybe include screenshots of output emails or New Relic overview or the configuration section in the README?

Wrong timestamp in logstash logger

Every event threw TCP comes with timestamp:

1970-01-01T00:00:00.000Z 10.0.2.2

when using Queue Events: true

open_basedir restriction error

Keep getting these Warnings caused by Logger (Sentry) trying to access outside of public_html dir.
Seems to be caused by an empty variable used as argument in realpath() function.

realpath(): open_basedir restriction in effect. File(/var/www/[DOMAIN].com/) is not within the allowed path(s)

{
code: 2, 
/public_html/lib/sentry/lib/Raven/Client.php, 
line: 317
}

M2 version?

Any chance of a Magento 2 version in the near future?

Broken symlink in newest release

symlink has no referent: "/skin/adminhtml/default/default/firegento/debug"

Call to undefined function random_bytes()

Fatal error: Call to undefined function random_bytes() in app/code/community/FireGento/Logger/Helper/Data.php on line 50

I cant see any information about what PHP version you are targeting but random_bytes() was only introduced in PHP7, Magento isnt php 7 compatible without changes.

Disable logging per module

Some extensions log useless stuff that pollutes the logs. In many cases the filter option won't suffice as these extensions don't prefix their log entries.

I'm not sure if it's easy to do that reliably, but @schmengler had a good idea:

should be possible with defining a custom log writer in global/log/core/writer_model and some debug_backtrace magic. not the most elegant thing but at least doesn't require hacking Mage.php

Logging client side errors

A nice feature would be if the system could log client side errors. Most services have JS-libraries available to log directly from the client.

Filter does not support array params

Just discovered this when I accidentally had two fields named payment[cc_number].

Firegento did not filter the data. I guess because the array it is looping through has ["cc_number"] as the key.

This is with PHP7 on local dev. Probably not a common situation. I'm going to change the name of the field anyway.

Wrong log data when enabling queue

While debugging some issues regarding Loggly I encountered an error.

Queue disabled:

Queue enabled:

Endless loop when using Logstash Logger together with Cm_RedisSession_Model_Session

When the Logstash-Logger is configured together with Cm_RedisSession_Model_Session an endless loop is produced.
This happens because of "\FireGento_Logger_Model_Logstash::buildJSONMessage":
$fields['SessionId'] = Mage::getSingleton("core/session")->getEncryptedSessionId();
$fields['CustomerId'] = Mage::getSingleton('customer/session')->getCustomerId();

The above calls cause the instantiation of "\Cm_RedisSession_Model_Session::__construct" which calls "Mage::log(...)" which calls "\FireGento_Logger_Model_Logstash::buildJSONMessage" and so on...

resolve attribute codes to text

that would make it easier to understand to what value an attribute was changed, except of only showing the code

Add filename to log information

Log function Mage::log($message, $level = null, $file = '', $forceLog = false)
has 3d parameter $file.

We should add $file to log as well.

This will help to group logs.

Update Sentry components

Dear Team,

I'm requesting if firegento-logger can be updated with newer versions of the Sentry components.  This is also to put this on the Teams radar for the roadmap and provide links to the sentry release pages.

PHP

Sentry displays this message:

We recommend you update your SDK from version 1.9.1 to version 2.0.1.

sentry-php releases

JavaScript

Sentry is also displaying:

We recommend you migrate to the sentry.javascript.browser SDK so you can
update your SDK from version 3.24.0 to version 5.19.2

sentry-javascript releases

Sincerely,
Kevin

New Release

Michael: default, advanced, db, mail, papertrail und logstash getestet
fehlen noch xmpp, graylog, loggly und chrome

Add AWS Cloudwatch as a target

Has anyone investigated whether this is possible?

FireGento_Logger_Model_Mail: Character-Encoding of Mail Body Option

The FireGento_Logger_Model_Mail backend (Email via SMTP with the help of Zend_Mail) is not specifying any character encoding for the body of the email send out.

This makes it default to the default charset of Zend_Mail:

It defaults to iso-8859-1.

From: Character Sets - Zend_Mail - Version 1.12 Documentation

This is a feature request to add an additional parameter for mail logging to allow setting the body charset encoding. The error messages we pump out on our site for example use Unicode with the popular 8-bit UTF-8 encoding.

I see no reason to adhere to the Zend_Mail fallback to iso-8859-1 in any case, as this should have been us-ascii in the first place (but that is just a note in the margin).

For general reference the "charset" parameter is specified in 4.1.2. Charset Parameter - Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types - RFC2046

Example of a problem receiving logger-email

This is with a german shop and a german error message encoded as UTF-8 as the shop uses it. Excerpt from such an email showing the typical encoding problem due to wrong charset setting ("möglich" instead of "möglich"):

Transliteration: "exception 'Mage_Core_Exception' with message 'Online-Buchung nicht mehr möglich. "

On using UTF-8 as default

It might make sense to use UTF-8 as default, but this need further decision from the project itself as this would introduce a backwards incompatible change.

I do see some reasons for UTF-8 encoding of the error message as for example the JSON string in there has to be UTF-8 encoded implicit anyway. There is a work around with this as the JSON encodes any character in strings not within the US-ASCII repertoire with their UTF-16 Unicode escape sequences, so the JSON string itself is US-ASCII and LATIN-X safe. However, this processing - even more fail safe with common charsets - makes the JSON data harder to read (it's not valid JSON anyway as it's prepended and appended with other texts).

Things to change

This feature request is rather trivial from a first view. Zend_Mail constructor accepts the charset parameter optionally.

After introducing an optional configuration setting for the email logger expecting a character-set string, this value - if set - can just be passed when FireGento_Logger_Model_Mail::getMail() is invoked.

stderr: PHP Fatal error: Class 'Zend_Log' not found in app/code/core/Mage/Core/functions.php on line 247

Accessing /logger/error/send/ raising 500 Error.

Backtrace:

Warning: include(ZEND/LOG.php): failed to open stream: No such file or directory  in /var/www/share/staging.project.com/htdocs/lib/Varien/Autoload.php on line 94

#0 /var/www/share/staging.project.com/htdocs/lib/Varien/Autoload.php(94): mageCoreErrorHandler(2, 'include(ZEND/LO...', '/var/www/share/...', 94, Array)
#1 /var/www/share/staging.project.com/htdocs/lib/Varien/Autoload.php(94): Varien_Autoload::autoload()
#2 [internal function]: Varien_Autoload->autoload('ZEND_LOG')
#3 /var/www/share/staging.project.com/htdocs/app/code/community/FireGento/Logger/controllers/ErrorController.php(38): spl_autoload_call('ZEND_LOG')
#4 /var/www/share/staging.project.com/htdocs/app/code/core/Mage/Core/Controller/Varien/Action.php(418): FireGento_Logger_ErrorController->sendAction()
#5 /var/www/share/staging.project.com/htdocs/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(254): Mage_Core_Controller_Varien_Action->dispatch('send')
#6 /var/www/share/staging.project.com/htdocs/app/code/core/Mage/Core/Controller/Varien/Front.php(172): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#7 /var/www/share/staging.project.com/htdocs/app/code/core/Mage/Core/Model/App.php(365): Mage_Core_Controller_Varien_Front->dispatch()
#8 /var/www/share/staging.project.com/htdocs/app/Mage.php(684): Mage_Core_Model_App->run(Array)
#9 /var/www/share/staging.project.com/htdocs/index.php(84): Mage::run('', 'store')
#10 {main}

Maintainer

Find maintainers for the project.

FireGento rules force at least two maintainers which take care for the project.

Who is it?

Error when enabling the queue functionality

PHP Fatal error:  Uncaught exception 'Zend_Log_Exception' with message 'Unable to write to stream' in /lib/Zend/Log/Writer/Stream.php:132
Stack trace:
#0 /lib/Zend/Log/Writer/Abstract.php(84): Zend_Log_Writer_Stream->_write('2013-10-17T19:0...')
#1 /.modman/Firegento_Logger/app/code/community/Firegento/Logger/Model/Queue.php(92): Zend_Log_Writer_Abstract->write('2013-10-17T19:0...')
#2 /lib/Zend/Log.php(285): Firegento_Logger_Model_Queue->shutdown()
#3 [internal function]: Zend_Log->__destruct()
#4 {main}
 thrown in /lib/Zend/Log/Writer/Stream.php on line 132

Fatal error: Uncaught exception 'Zend_Log_Exception' with message 'Unable to write to stream' in /lib/Zend/Log/Writer/Stream.php:132
Stack trace:
#0 /lib/Zend/Log/Writer/Abstract.php(84): Zend_Log_Writer_Stream->_write('2013-10-17T19:0...')
#1 /.modman/Firegento_Logger/app/code/community/Firegento/Logger/Model/Queue.php(92): Zend_Log_Writer_Abstract->write('2013-10-17T19:0...')
#2 /lib/Zend/Log.php(285): Firegento_Logger_Model_Queue->shutdown()
#3 [internal function]: Zend_Log->__destruct()
#4 {main}
 thrown in /lib/Zend/Log/Writer/Stream.php on line 132

Move chrome logger class to lib

Libraries should be put in lib folder and not into model.

use mail() as alternative transport for email backend

Is it already possible to use PHP's mail() as alternative transport to send out log events via email?

If not I think this is a considerable option because I think Zend_Mail supports it as backend (IIRC) and it normally comes with zero-configuration.

I ask because right now I have got some problems sending through the SMTP server which doesn't work in an esoteric fashion punctually and the hoster is not really helping much.

Having an alternative is always good.

SMTP generally works fine btw, this must be our hoster.

Loggly interface doesn't seem to work.

Initially I tried Loggly via Remote Syslog (UDP) Configuration with Queue Events set to No. Also tried with Queue Events set to Yes.

I traced it down to this fwrite and saw that it was successfully writing 756 bytes:
https://github.com/firegento/firegento-logger/blob/master/src/lib/rsyslog/rsyslog.php#L90

I also tried Loggly (HTTPS) Configuration and that didn't work either. Traced it down to the fwrite here, and it appears to have successfully written to the stream:
https://github.com/firegento/firegento-logger/blob/master/src/app/code/community/FireGento/Logger/Model/Logglyhttps.php#L163

One thing that I'm not understanding in general is that there doesn't seem to be a place to drop in your customer token, so I'm not sure how it could work at all without that.

I did see the Input Key option under the HTTPS mode and I tried it, but it didn't help, and I don't see any option for it under the UDP mode.

Also I tried it with the default host and port (logs.loggly.com:42146), and I also tried the host and port that seemed to be indicated in the loggly setup instructions: logs-01.loggly.com:514.

@therouv , I noticed that you wrote this code - are you using this actively right now?

Priority filter not working for defining which target to log to

duplicates #82 Priority Level Filter does not work when using multiple targets:

For example when the mail target is set to Zend_log::ALERTand the Files (Magento Default) Configuration Target is set toZend_log::DEBUG it is expected that ONLY alerts are sent to the email target, this is not the case.

The reason is that Zend_Log_Filter_Priority::accept() expects an array to be passed as a parameter but instead an object of type FireGento_Logger_Model_Event is passed, therefore the check:

return version_compare($event['priority'], $this->_priority, $this->_operator);

fails and the Zend_log::DEBUG messages are sent to the Mail target as well.

working together with new relic extension

If you install firegento_logger together with the proxiblue new relic module http://www.proxiblue.com.au/newrelic-magento-integration.html then logger is stopping its work.

With queue enabled, onepage checkout errors out on Graylog

With Queue Events enabled, whenever there is an exception (such as declined credit card) in the built in checkout, the returned Ajax message gets a 500 status code, and this throws the Javascript into a weird state.

This happens with Gralog enabled.

Error:

Uncaught exception 'Zend_Log_Exception' with message 'Missing required data parameter: "version", "short_message" and "host" are required.'

Log session_id with all requests

This allows us to link multiple log entries together. We sometimes experience errors that might be caused by a previous encountered error, it would be nice if we could detect this.

Why skip warn level ?

In Helper/Data, addPriorityFilter function, line 114
https://github.com/firegento/firegento-logger/blob/master/src/app/code/community/FireGento/Logger/Helper/Data.php#L114

Could you please explain why you skip warn level ?
Is there any reason ?
Thanks

logs not logging to remotes with rsyslog lib.

2018/05/23 06:56:14 [error] 167#167: *66 FastCGI sent in stderr: "PHP message: PHP Fatal error: Uncaught Exception: Timestamp must be a number. in /vagrant/sites/enjo/docroot/lib/rsyslog/syslogmessage.php:158
Stack trace:
#0 /vagrant/sites/enjo/docroot/lib/rsyslog/syslogmessage.php(176): SyslogMessage->SetTimestamp(false)
#1 /vagrant/sites/enjo/docroot/app/code/community/FireGento/Logger/Model/Papertrailsyslog.php(121): SyslogMessage->__construct('[] [default][Ht...', 16, 'error', false, Array)
#2 /vagrant/sites/enjo/docroot/app/code/community/FireGento/Logger/Model/Rsyslog.php(181): FireGento_Logger_Model_Papertrailsyslog->buildSysLogMessage(Object(FireGento_Logger_Model_Event))
#3 /vagrant/sites/enjo/docroot/lib/Zend/Log/Writer/Abstract.php(88): FireGento_Logger_Model_Rsyslog->_write(Object(FireGento_Logger_Model_Event))
#4 /vagrant/sites/enjo/docroot/app/code/community/FireGento/Logger/Model/Queue.php(125): Zend_Log_Writer_Abstract->write(Object(FireGento_Logger_Model_Event))
#5 /vagrant/sites/enjo/docroot/lib/Zend/Log.php(366): FireGento_Logger_Model_Queue->shutdow" while reading upstream, client: 172.17.0.1, server: enjo.test, request: "GET / HTTP/2.0", upstream: "fastcgi://unix:/run/php/php7.1-fpm.sock:", host: "www.enjo.test", referrer: "https://www.enjo.test/bathroom-care.html?p=all"

firegento and graylog2 preview

How do I go about using firegento-logger with graylog2 preview release? RC1 is due out this month. It doesn't appear to be listening on port 12201 so at the moment I can't even connect to it from magento.

Is there some sort of test mode for the debuger to see if it is establishing a connection?

After doing some more research - you have to add a input to the server node using the graylog2 web interface - what type of node do I need? e.g. gelf HTTP, gelf UDP, gelf TCP, raw text UDP, raw syslog UDP, JSON etc?

Thanks

Add a way to test the logging service

A simple button to log something to firegentlo_logger.log so the connection between the system and the loggin service can be tested.

Compatible with Magento 2.3?

Is it going to be compatible with Magento 2.3?

Thanks

We should merge this with Wf_Logmanager

Should merge these modules: https://github.com/jayelkaake/logmanager

JsonStream Logger only logs "null"

The following line removes all event data, that is NOT null

    $eventData = array_filter($eventData, "is_null");

GDPR / DSGVO ready ??

Is this module GDPR ready?

Security Issue when logging %requestData%

Just discovered a security issue when I use the advanced logging format.

I'm logging failed payments in onepageController::saveOrderAction but %requestData% contains credit card details.

How can I strip of sensitiv request data from being logged without removing %requestData% from the advanced format? [Now I have removed it :-)]

The issue is here: https://github.com/firegento/firegento-logger/blob/develop/src/app/code/community/FireGento/Logger/Helper/Data.php#L257

It would be nice if there could be an event implemented with which you can modify the GET/POST/FILES/RAWPOST vars before being logged.

Sending exceptions to the configured services?

Thanks to all contributors of this extension. It's definitely a must-have for every Magento shop.

But is it also possible to send all exceptions to the configured external log services and not only all Mage::log calls?

Logging for Reports (all Backends but File)

I'm using firegento-logger now since some days and also in production.

I mainly use it for errror reporting by email (works great so far btw.) but I also like the advanced logging format in files.

Then today I realized that when Magento creates error reports (that are the files in magento/var/report/ - http://example.com/magento/errors/report.php?id=1234567890&skin=default) I don't get any email notification about those.

I think that is because strictly speaking this is not error logging. According to http://magento.stackexchange.com/a/4887/4115 I can even enable this within Magento XML configuration already.

However I think it would be nice to have this in one package, that is, integrate this kind of error reporting as well in the firegento-logger.

Most often - at least it appears to me - those reports are important and having caught these events within the logging so far would make sense to me.

Is this of interest? Am I the first who had that thought?

Email Notification for Database Interface

I'm thinking about building email notifications for the Database interface. I was hoping to be able to use loggly for this but running into issues (#10).

What I'm planning to do is add an Email Notification Map similar to the Target Map that exists currently.

Will have:

• Regular expression
• Severity (dropdown)
• Email address (comma-separated list)

I think I'll overload FireGento_Logger_Model_Db::_write(), loop over the email notification map, if it matches, then fire an email out with a link to the exception details.

I'll also need to build an exception detail page (admin/logger/view/id/{id}) to link to.

Just wanted to post this up in case you guys had any thoughts or suggestions.

Ability to archive to S3

Guys, thanks for this module. Looks amazing and I'm really excited to start using it!!!

I'm thinking of using loggly for my primary logger interface, as it can provide some cool functionality to route error messages by email address to different individuals, etc.

But the main limitation I see with it is the lifetime of the log files (7 days). Every once in a while an issue arises where you need to go back maybe a few months or even more to diagnose what might have gone wrong.

For example, customer support tells you that an order didn't make it out to 3PL shipping system, so you go back and check the logs in your system to see if maybe it hit a failure of some kind.

I'm not sure what the best way to implement this would be. Maybe use two logger interfaces - both loggly and S3 concurrently - and S3 will just stick around longer.

Using S3 over simply storing on your web server could have the benefit of cheaper storage costs / greater flexibility.

Not sure whether an S3 logger interface would make sense, or an option to archive entire log files (on a daily basis?) from file system (or maybe from any interface) to S3.