heiglandreas/org_heigl_hyphenator

Provide TeX-Hyphenation to PHP

This library provides TeX-Hyphenation in PHP.

Requirements:

This package has the following requirements:

• PHP-Version >= 7.2
• Multibyte-Extension loaded
• Input has to be UTF8-encoded.

On loading the \Org\Heigl\Hyphenator\Hyphenator-class the internal encoding for the Multibyte-String-Extension will be set to UTF8.

Installation:

This package is best installed via composer:

composer require org_heigl/hyphenator

More information can be found in the doc-section

Usage:

use \Org\Heigl\Hyphenator as h;
// Create a hyphenator-instance based on a given config-file
$hyphenator = h\Hyphenator::factory('/path/to/the/config/file.properties');

// And hyphenate a given string
echo $hyphenator->hyphenate('Hyphenation');
// Hy-phe-na-ti-on

Documentation:

More documentation can be found at http://orgheiglhyphenator.readthedocs.org/en/latest/

Build-Status of the latest release can be found at http://travis-ci.org/#!/heiglandreas/Org_Heigl_Hyphenator

Legal Stuff

Copyright (c) 2011-2016 Andreas Heigl[email protected]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

org_heigl_hyphenator's Issues

Use TeX-Hyphenation patterns

Add the usage of TeX-Hyphenation patterns based on the pattern-files available at http://ctan.space-pro.be/tex-archive/language/hyph-utf8/tex/generic/hyph-utf8/patterns/tex/

Modify package name to fix Composer deprecated warnings

During composer installation, it will present following warning message:

Deprecation warning: Your package name Org_Heigl/Hyphenator is invalid, it should not contain uppercase characters. We suggest using org_heigl/hyphenator instead. Make sure you fix this as Composer 2.0 will error.

To be compatible with latest Composer version, we should consider modifying package name.

Add an Upgrade Guide from v1 to v2

Solve encoding mismatch between TEX-files and UTF-8

The TEX files contain strings like "'e" for the letter é for instance. These mismatches have to be found and replaced

Missing dictionary for cs_CZ

Activate Hyphenation-Quality

Currently you can set a Quality for Hyphenation, but it is not used due to some trouble with it.

That trouble has to be resolved

Update composer.json

• Update PHPUnit
• bring dependencies to stable version

Add test for Aussichtsturm

Move Punktuation to Punctuation

just correct a spelling issue

Tag a new release?

Hi,

there has been quite some work since the last release: v2.0.2...master

Can you please tag a v2.0.3? I depend on Org_Heigl_Hyphenator in JoliTypo and would like to be able to use it without having to use composer's minimum-stability flag.

Thx!

"Wasserwirtschaft" (German) is not hyphenated

Hi,

at first thanks for your great module which we use widely. Unfortunately we have an issue with the word "Wasserwirtschaft" which is not hyphenated.

We investigated and it seems that the dictionary doesn't contain it. Could you please check if its an issue or have any other suggestion?

Bye Defcon0

Problems with composer installation in case insensitive file systems

Hi,
since version 2.5 of your library, I get problems when installing them via composer on my local Docker DEv environment (Docker Desktop 3.0.0 on a Windows 10).
The problem is caused by two files inside the src/share/files/dictionaries/ folder, having the same filename but their file endings have a different spelling (uppercase vs. lowercase):

The problem could be resolved easily by removing either one of these files from the repository.

Thanks a lot and happy holidays.

Fix Incomplete tests

Two tests currently are marked as incomplete. These tests have to be fixed

English hyphenation results

Hi!

I tried this code:

use \Org\Heigl\Hyphenator as h;
$hyphenator = h\Hyphenator::factory();
echo $hyphenator->hyphenate('hyphenation');
// hy-phe-na-ti-on
echo $hyphenator->hyphenate('chocolate');
// choco-late

Expected results are:
hy-phen-a-tion
choc-o-late

My config is:

noHyphenateString = null
hyphen = "-"
leftMin = 1
rightMin = 1
wordMin = 3
quality = 9
customHyphen = "=="
defaultLocale = "en_US"
tokenizers = "Whitespace,Punctuation"
filters = "Simple,CustomMarkup"

Any idea why I am seeing different results?

Thanks!
Jose

Import latest hyphenation patterns

We probably should input the latest hyphenation patterns from https://github.com/hyphenation/tex-hyphen/tree/master/hyph-utf8/tex/generic/hyph-utf8/patterns/txt

ru_RU.ini is empty

Russian locale doesn't work

No hyphenation for "rücke" (de-DE) found

Hi,
I have an issue with the german dictionary (de_DE) in conjunction with "ü":

I get no hyphenation for "rücke" (and derivations like "drücke", "Brücke"...) whereas"Röcke" (Rö-cke) and "räcke" (rä-cke) works fine.

I use this settings:

            $o->setHyphen($this->hyphSign)
                ->setDefaultLocale('de_DE')
                ->setRightMin(2)
                ->setLeftMin(2)
                ->setWordMin(3)
                ->setTokenizers('Whitespace', 'Punctuation');

Is this a bug or am I doing something wrong?

Thanks
Vonni

Hyphenating the word "vereinfachen" (German) triggers iOS parental controls content blocker

Hi,

Firstly, thanks for the awesome library :)

I'm experiencing quite a peculiar issue concerning iOS Safari in relation to parental controls. If parental controls are active and set to limit adult web content, iOS Safari in some cases won't display a website that contains the hyphenated word "vereinfachen" (German for "simplify"). Instead it will show this:

This happens in iOS 12.1.4, if the option Settings -> Screen Time -> Content & Privacy Restrictions -> Content Restrictions -> Web Content is set to Limit Adult Websites and if some other (currently unknown) criteria is met.

I can easily reproduce this issue on the aforementioned website on different iPhone/iPad devices. I can just as easyily fix it by simply removing the word "vereinfachen" from the hyphenated string or by not using Hyphenator at all.
I haven't however been able to reproduce the issue in another environment where the HTML structure differs. The implication is that the surrounding HTML does seem to matter to an extent and that the issue may be somewhat exclusive to the website I'm working on. Not sure

Note it doesn't seem to matter if ­ or <i></i>&shy; is used. Also the website in question does not contain any adult content in any shape or form. On the contrary, it's a website aimed at teenagers.

Any ideas as to what's going on here? I can supply an example page if that helps.

Add a dictionary for hyphen exceptions

I have an issue with the German word "Produktionsstrategie" hyphened at Produktionss|trategie, wich seems to be wrong. For these cases it would great to have a dictionary of "hyphen exceptions" that overwrite the default rules for certain words. This would also be great for hyphens of company words etc.
Or is something like this already possible?

Add TokenRegistryFilters

cleanup Hyphenator::factory

Currently in the factory method a lot of assigning is done that could be moved to getter methods for tokenizers and filters

Support PHP 8.1

When using PHP8 several errors are currently shown. They are all very similar, but just pasted them all for the sake of completeness.

Deprecated function: Return type of Org\Heigl\Hyphenator\Dictionary\DictionaryRegistry::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Dictionary/DictionaryRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Dictionary\DictionaryRegistry::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Dictionary/DictionaryRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Dictionary\DictionaryRegistry::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Dictionary/DictionaryRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Dictionary\DictionaryRegistry::valid() should either be compatible with Iterator::valid(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Dictionary/DictionaryRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Dictionary\DictionaryRegistry::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Dictionary/DictionaryRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Dictionary\DictionaryRegistry::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Dictionary/DictionaryRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Filter\FilterRegistry::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 54 of /var/www/html/vendor/org_heigl/hyphenator/src/Filter/FilterRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Filter\FilterRegistry::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 54 of /var/www/html/vendor/org_heigl/hyphenator/src/Filter/FilterRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Filter\FilterRegistry::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 54 of /var/www/html/vendor/org_heigl/hyphenator/src/Filter/FilterRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Filter\FilterRegistry::valid() should either be compatible with Iterator::valid(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 54 of /var/www/html/vendor/org_heigl/hyphenator/src/Filter/FilterRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Filter\FilterRegistry::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 54 of /var/www/html/vendor/org_heigl/hyphenator/src/Filter/FilterRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Filter\FilterRegistry::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 54 of /var/www/html/vendor/org_heigl/hyphenator/src/Filter/FilterRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenizerRegistry::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 52 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenizerRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenizerRegistry::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 52 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenizerRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenizerRegistry::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 52 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenizerRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenizerRegistry::valid() should either be compatible with Iterator::valid(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 52 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenizerRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenizerRegistry::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 52 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenizerRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenizerRegistry::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 52 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenizerRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenRegistry::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenRegistry.php).

Deprecated function: Return type of Org\Heigl\Hyphenator\Tokenizer\TokenRegistry::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in include() (line 49 of /var/www/html/vendor/org_heigl/hyphenator/src/Tokenizer/TokenRegistry.php).

add an upgrading guide for 2.0, missing validation of configuration

Trying to upgrade https://github.com/liip/LiipHyphenatorBundle is turning out to be surprisingly tricky.
I have been chasing one issue after another and now the latest issue seems to be:

An exception has been thrown during the rendering of a template ("Catchable Fatal Error: Argument 1 passed to Org\Heigl\Hyphenator\Hyphenator::getHyphenationPattern() must be an instance of Org\Heigl\Hyphenator\Tokenizer\TokenRegistry, string given, called in /Users/lsmith/htdocs/symfony-standard/vendor/OrgHeiglHyphenator/src/Org/Heigl/Hyphenator/Hyphenator.php on line 343 and defined in /Users/lsmith/htdocs/symfony-standard/vendor/OrgHeiglHyphenator/src/Org/Heigl/Hyphenator/Hyphenator.php line 359")

Guess something is lacking validation inside the lib which is causing it to break in the above place.

add a composer.json file

make the cache dir configurable

see also liip/LiipHyphenatorBundle#2

Problem with single quotes in parsed files

Hi!

Thanks for your great work!

There is an issue about single quotes that are not escaped in the parsed files. I don't know how to upload patch in current issue, so I attach the patch below.

Date: Mon, 21 Mar 2011 19:51:06 +0000
Subject: [PATCH] Fix issue about single quotes in parsed files.

---
 src/Org/Heigl/Hyphenator.php |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/Org/Heigl/Hyphenator.php b/src/Org/Heigl/Hyphenator.php
index 3315649..45dbd18 100644
--- a/src/Org/Heigl/Hyphenator.php
+++ b/src/Org/Heigl/Hyphenator.php
@@ -412,7 +412,7 @@ final class Org_Heigl_Hyphenator
                     $patternint .= '0';
                 }
             }
-            $patternstring = preg_replace ( '/[0-9]/', '', $pattern );
+            $patternstring = preg_replace ( array('/[0-9]/', '/\'/'), array('', "\\'"), $pattern );
             if ( $patternstring != '' ) {
                 fwrite ( $fh, '$pattern[\'' . $patternstring . '\'] = \'' . $patternint . '\';' . "\n" );
             }
-- 
1.7.4.msysgit.0

With regards,
Fabien

Cannot hyphenate a single word

Expected: "Ot^to"
Actual: "Otto"

use \Org\Heigl\Hyphenator as Hy;
$hy = new Hy\Hyphenator();
$op = new Hy\Options();
$op->setHyphen('^')
->setDefaultLocale('de_DE')
->setRightMin(2)
->setLeftMin(2)
->setWordMin(4)
->setFilters('Simple')
->setTokenizers('Punctuation','Whitespace')
;
$hy->setOptions($op);
var_dump($hy->hyphenate('Otto'));