dmkebusinessgmbh/typo3-mklog

TYPO3 extension which offers a developer log. Aggregate devlog entries and deliver them through several transports (Mail, GELF ...).

MK Log

This extension offers a developer log. Ther is a scheduler task too, the watch dog, which aggregates devlog entries and sends them via a transport. So it is possible to send a mail with fatal errors the minute they occur but warnings only every 6 hours. Or send a mail with infos from an import every night.

Or transport all the logs to a graylog server

Of course the devlog has to be used by the core and extensions. To have exceptions and errors logged to the devlog the error handling of mktools can be used.

Installation

Install TYPO3 via composer. Maybe you can use our TYPO3-Composer-Webroot Project

From project root you need to run

composer require dmk/mklog

Documentation

For usage please have a look in our Documentation

CHANGELOG

typo3-mklog's Issues

TYPO3 9.5 compatibility

BE module broken with rn_base 1.16.7

Due to some changes in rn_base 1.16.7 the BE module throws the following exception:
Wrong module name "" in /var/www/html/htdocs/typo3/sysext/backend/Classes/Utility/BackendUtility.php on line 2736

PHP Error after update to version 10.0.1

After update to latest version my TYPO3 is broken with this error:

Fatal error: Uncaught TypeError: Return value of DMK\Mklog\Utility\ConfigUtility::getGelfCredentials() must 
be of the type string, null returned in /path/to/typo3/webroot/typo3conf/ext/mklog/Classes/Utility/ConfigUtility.php:238 

At this time, there is no ext config persisted for mklog. The default value should not be null, but an empty string.

remove return type declarations and support php 5.6 again

php 5.6 support are dropped in a48089d4.
return type declarations are not supported for php 5.6 and should be removed.
In our composer.json we require at least TYPO3 6.2 which needs php5.6 or earlier.

For now i would prefer to remove the return type declarations.

Check size of extradata in bemodule, mail and gelf transports

The current maximum of extra_data are 16MB.
The BE-Module, E-Mail-Notification and GelfTransports has to check the size of the extra_data and should display a message instead of the large data.
A very large extra_data can be created by DatabaseConnection::debug_check_recordset() for example. Ther are the whole backtrace with arguments are stored to the extradata.

• Add a output parser, which can handle the big extradata and creates short messages
• make the maximum of extra_data configurable by extension, log and/or scheduler configuration
• prevent large output of extra_data in BE-Module
• prevent large output of extra_data in E-Mail-Notifications
• prevent large output of extra_data in GelfTransport for direct logs
• prevent large output of extra_data in GelfTransport for scheduled logs

Cleanup, Autoloading, PHPLint, ...

• fix autoload config (composer.json, ext_emconf.php)
• remove code outside of classes (tx_rnbase::load, ...)
• add phplint to travis-ci
• remove legacy tx_devlog watchdog code
• drop support for 6.2

The watchdog sends mails when there are no messages

i have a scheduler task, which runs every 5 minits:

Config	Value
Minimum error level to trigger email.	FATAL
Force a summary mail	No
Include data_var.	Yes
Group messages.	No

This task shoult send me a mail every 5 minits, if there are a new FATAL message.

But at the moment the scheduler sends a mail every 5 minits:

This is an automatic email from TYPO3. Don't answer!

Developer Log summary since 2015-08-10 09:20:05
    Level INFO (Severity Number: 0): 18 items found

Latest entries by log level

Level FATAL (Severity Number: 0)

Time: 1970-01-01 01:00:00
Extension: 
Message: 

Count: 0 
DataVar: 

It looks like a mail will always sent, when a entry of some level are stored.

A nother thing is, the messages looks grouped, but the option in the scheduler was set to false for the grouping.

Add support for TYPO3 10 LTS

Invalid default value for 'tx_mklog_lastrun'

typo3-mklog/ext_tables.sql

Line 5 in dc6c657

tx_mklog_lastrun datetime DEFAULT '0000-00-00 00:00:00',

Default value ''0000-00-00 00:00:00' is invalid in strict SQL mode. It must be replaced by '1970-01-01 00:00:00'.

Severity is not transported to gelf

The mapper does not map the severity field into the entity, so every message, logged to gelf, has the log level 0 (critical).

typo3-mklog/Classes/Domain/Mapper/DevlogEntryMapper.php

Lines 71 to 86 in 74efe1b

	public static function fromRecord(array $record): DevlogEntryMapper
	{
	$entry = Factory::makeInstance(DevlogEntry::class)
	->setUid($record['uid'] ?? 0)
	->setPid($record['pid'] ?? 0)
	->setRunId($record['run_id'] ?? 0)
	->setExtKey($record['ext_key'] ?? '')
	->setHost($record['host'] ?? '')
	->setMessage($record['message'] ?? '')
	->setExtraDataEncoded($record['extra_data'] ?? '')
	->setCrdate($record['crdate'] ?? 0)
	->setCruserId($record['cruser_id'] ?? 0)
	->setTransportIdsRaw($record['transport_ids'] ?? '');
	return Factory::makeInstance(DevlogEntryMapper::class, $entry);
	}

Add support for TYPO3 8 LTS

log table cleanup broken

On each Request mklog performs an database optimization.
If there are more logs in the table as configured in the max_logs setting, so all older logs should be removed.

Mklog gets all the entries, sorted by date in ascending order - the oldest first.
assuming the max logs config is set to 10, mklog takes the date of the 10th entry and deletes all older entries.

On the first request are 20 entries made. mklog will never find an older entry to delete. the table will overrun.

The ordering to determine the date to delete all older logs should be changed to desc.
So the newest logs are the first and the older logs can be deleted.

make extension compatible to TYPO3 7.6

Fieldvalidation in SchedulerFieldProviderWatchDog does not work in TYPO3 9

The field provider for the scheduler validates the credentials for the watchdog and adds a message if the validation fails.
See SchedulerFieldProviderWatchDog::_validateAdditionalFields

The method addMessage was changed to protected, was marked as deprecated and will be removed in TYPO3 10.

See Deprecation-84387-DeprecatedMethodAndPropertyInSchedulerModuleController.

SQL-Fehler nach Installation

Nach der Installation in TYPO3 7.6 bekomme ich im BE-Modul folgende Fehlermeldung:

SQL QUERY IS NOT VALID
Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 't3_74.DEVLOGENTRY.uid' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
SELECT DEVLOGENTRY.* FROM tx_mklog_devlog_entry AS DEVLOGENTRY WHERE 1=1 GROUP BY DEVLOGENTRY.run_id ORDER BY DEVLOGENTRY.run_id DESC LIMIT 50

Ich denke die Abfrage sollte so gestaltet werden, daß sie mit jedem Modus von MySQL funktioniert.

extract table cleanup process while logging

Currently, the log process cleans up the log entry table on every run.
https://github.com/DMKEBUSINESSGMBH/typo3-mklog/blob/master/Classes/Logger/DevlogLogger.php#L107

This could cause high load on database if logging is used excessively. This behavior should be extracted to an own process.

Delete devlog entries when deleting pages

devlog entries can be on every page. non-admins shouldn't see those entries but they still should be able to delete pages with devlog entries on them. By default this won't work as they need table modify access.

Therefore the deletion of devlog entries should be done before deleting the page without checking access.