parlr / ruby-font-creator Goto Github PK

View Code? Open in Web Editor NEW

20.0 20.0 2.0 6.64 MB

Generate rich Unicode open fonts with custom annotations, transliterations, pronunciations.

JavaScript 95.03% Shell 1.37% Makefile 3.60%

chinese font hanzi pinyin

ruby-font-creator's People

Contributors

Stargazers

Watchers

Forkers

cf coresh

ruby-font-creator's Issues

Reorganize projects roles, naming, repositories

parlr/ruby-font-maker : Stand alone NodeJS module.
- input : 2 fonts, a table of glyph / annotation pairs
- output : 1 font.
parlr/Hanzi-Pinyin-Data : data = [ { glyph: "", annotation: [ "", "", ... ] }, {...} ]:
parlr/Hanzi-Pinyin-Font :
- input : 1 CJK font + 1 latin font, Hanzi-Pinyin-Data
- runs ruby-font-maker on them !
- output : ./Hanzi-Pinyin-Font.otf (font)

I thus suggest to rename the current repository + stack of issues ruby-font-maker. A bit problematic as of our 10 gh-followers !

Python image creation using font

From Python image library - font positioning:

from PIL import Image, ImageDraw, ImageFont

FULL_SIZE = 100
filename = 'font_posn_test.png'
fontname = '/usr/share/fonts/truetype/msttcorefonts/arial.ttf'
textsize = 40
text = "5"

image = Image.new("RGBA", (FULL_SIZE, FULL_SIZE))
draw = ImageDraw.Draw(image)
font = ImageFont.truetype(fontname, textsize)
print font.getoffset(text)
print font.font.getsize(text)
font_width, font_height = font.getsize(text)

font_y_offset = font.getoffset(text)[1] # <<<< MAGIC!

draw.rectangle(((0, 0), (font_width, font_height)), fill="black")
draw.text((0, 0 - font_y_offset), text, font=font, fill="red")
image.save(filename, "PNG")

Read about chinese font fundamental

Fundamentals of Chinese Typography: #1 Fonts Styles (Songti, Fangsongti, Mingti, Heiti, Kaiti)

Associate glyph with correct codepoint

The font generation works but the glyph are on a private plan. For instance 中 is associate with U+EA02 <Private Use> instead of U+4E2D CJK UNIFIED IDEOGRAPH-4E2D:

General Character Properties
In Unicode since: 1.1
Unicode category: Other, Private Use

Various Useful Representations
UTF-8: 0xEE 0xA8 0x82
UTF-16: 0xEA02

C octal escaped UTF-8: \356\250\202
XML decimal entity: 

We need to investigate webfont in depth.

related: itgalaxy/webfont#53 How can I choose the codepoint of a glyph.

Spread the word to chinese.stackexchange

In 2013 I asked Are there any free fonts designed to help learn/read Chinese?.

When we have release we can add a link to the project.

Base font consideration

Which font should I used for this project?

It need to be open-source in order to allow reusability ;
lighter font will help on mobile device ;
support for ASCII-latin characters ;
support italic/bold ;

Here are some possibilities (from the top of my head):

DroidSansFallbackFull ;
DéjàVu Sans fonts family ;
~~Droid fonts family~~ (see How does Noto relate to Droid?)
~~Noto Sans~~, ~~Noto CJK~~ (see notofonts/noto-cjk#81) ;
Ubuntu fonts family.

FontTools: Tools to manipulate font files

Data gathering

We currently look for database with { "glyph": "西", "phonetic": "xī" } (or xi1, or alternatives).

Sources possible, info to complete :

Moedict

link (to complete)
json format
range : most common caracters, trad only ?

Unicode :

link (to complete)
xml
range : traditional/modern ; -more complete for a font
which phonetic format its provided also. ("glyph": "西", "phonetic": "xī" or xi1 ?)

CJKlib

link

Renaming repo → update remote URL

I rename the project from Hanzi-Pinyin-Font to ruby-font-creator, you need to update your remote URL:

git remote set-url  origin [email protected]:parlr/ruby-font-creator.git

Split font to be used with unicode-range

Building sliced font could help user experience

Noto Sans SC Sliced (Chinese Simplified)

The experience for Chinese webfont users can be improved if the file size and webfont latency are reduced: In any large corpus of documents, there are patterns in character usage. By scanning webpages in Chinese, we model which characters are most likely to appear together, which informs a static subset slicing strategy that sorts all 44,683 glyphs into 102 slices.

Using this method, the entire web page will seem to be using Noto Sans SC while only loading the set of font slices required for that page, which we expect to be much faster than loading the whole font…

https://fonts.google.com/earlyaccess#Noto+Sans+SC+Sliced

Usage

Simple

@font-face {
  font-family: 'Ampersand';
  src: local('Times New Roman');
  unicode-range: U+26;
}

Real one

@font-face {
  font-family: 'Noto Sans SC Sliced';
  font-weight: 300;
  src: url('//fonts.gstatic.com/ea/notosansscsliced/v2/NotoSansSCSliced-Light.0.woff2')
      format('woff2'),
    url('//fonts.gstatic.com/ea/notosansscsliced/v2/NotoSansSCSliced-Light.0.woff')
      format('woff'),
    url('//fonts.gstatic.com/ea/notosansscsliced/v2/NotoSansSCSliced-Light.0.otf')
      format('opentype');
  unicode-range: U+2252, U+30c3, U+5c83-5c94, U+5c9c-5ca8, U+5caa-5cac,
    U+5cae-5cb2, U+5cb4-5cb7, U+5cb9-5cc0, U+5cc2-5cc3, U+5cc5-5cd8, U+5cdb-5ce0,
    U+5ce2-5ce7, U+5ce9-5cec, U+5cee-5cef, U+5cf1-5cfa, U+5cfc-5d06, U+5d08-5d0d,
    U+5d0f-5d13, U+5d15, U+5d17-5d1a, U+5d1c-5d28, U+5d2a-5d2c, U+5d2e-5d4b,
    U+5d4d-5dc4, U+5dc6-5dcc, U+5dce-5ddc, U+5ddf-5de0, U+5de3-5de4, U+5dea,
    U+5dec-5ded, U+5def-5df0, U+5df5-5df6, U+5df8-5dfd, U+5dff-5, U+5e4, U+5e7,
    U+5e9-5b, U+5d-5f, U+5e12-5e14, U+5e17, U+5e19-5e1b, U+5e1e-5e25,
    U+5e28-5e2c, U+5e2f-5e36, U+5e39-5e3c, U+5e3e-5e44, U+5e46-5e54, U+5e56-55ee,
    U+67d1, U+6cba, U+9569-956b, U+958a-958b;
}

Resources

Error at start

cri-ninja@hp-840:~/yug/Hanzi-Pinyin-Font$ npm start

> hanzi-pinyin-font@ start /home/cri-ninja/yug/Hanzi-Pinyin-Font
> node --max_old_space_size=8192 --optimize_for_size --max_executable_size=4096 --stack_size=4096 --require babel-core/register ./index.js

/home/cri-ninja/yug/Hanzi-Pinyin-Font/index.js:49
    var svgContent = _svg2.default.wrap([_ruby2.default.text(char.glyph, _layouts2.default.bottom.glyph(options)), _ruby2.default.annotation(char.ruby, _layouts2.default.top.ruby(options))]);
                                                                                                                                                                              ^

TypeError: _layouts2.default.top.ruby is not a function
    at generateSvg (/home/cri-ninja/yug/Hanzi-Pinyin-Font/index.js:23:45)
    at /home/cri-ninja/yug/Hanzi-Pinyin-Font/index.js:79:5
    at /home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/jsonfile/index.js:46:5
    at /home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/graceful-fs/graceful-fs.js:78:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:445:3)

npm ERR! Linux 4.4.0-70-generic
npm ERR! argv "/home/cri-ninja/.nvm/versions/node/v6.9.1/bin/node" "/home/cri-ninja/.nvm/versions/node/v6.9.1/bin/npm" "start"
npm ERR! node v6.9.1
npm ERR! npm  v3.10.8
npm ERR! code ELIFECYCLE
npm ERR! hanzi-pinyin-font@ start: `node --max_old_space_size=8192 --optimize_for_size --max_executable_size=4096 --stack_size=4096 --require babel-core/register ./index.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the hanzi-pinyin-font@ start script 'node --max_old_space_size=8192 --optimize_for_size --max_executable_size=4096 --stack_size=4096 --require babel-core/register ./index.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the hanzi-pinyin-font package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node --max_old_space_size=8192 --optimize_for_size --max_executable_size=4096 --stack_size=4096 --require babel-core/register ./index.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs hanzi-pinyin-font
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls hanzi-pinyin-font
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/cri-ninja/yug/Hanzi-Pinyin-Font/npm-debug.log

npm-debug..log.txt

Note de version :

node -v
v6.9.1

Opentype.js fails to render Noto fonts

There is blocking issue with Noto-CJK/Opentype.js. For more informations, see:

I met rendering problems when using text-to-svg (shrhdk/text-to-svg#22, shrhdk/text-to-svg#23) with the NotoSansTC-Regular.otf.
Drilling down it appears text-to-svg is based on opentype.js.

Expected

This is for reference, I find that DroidSansFallbackFull.ttf worked fine.

key	value
font	DroidSansFallbackFull.ttf
text	㐅㐆丰wǔyǐnfēng

Actual (with NotoSans)

I tested on opentype.js online demo and the result is scramble too (see below):

key	value
font	NotoSansTC-Regular.otf
text	㐅㐆丰wǔyǐnfēng

Enhance the readme.md via png illustrations.

For md image inclusion syntaxe, review PeergraderJS > readme.md

Include Google's Noto Serif CJK as default font ?

Google’s new font Noto Serif CJK honors ancient type traditions
https://www.wired.com/2017/04/googles-new-font-honors-ancient-type-traditions/
(Published on: 04.03.17)

Read this https://www.google.com/get/noto/help/cjk/
Download font https://www.google.com/get/noto/#sans-hans / https://www.google.com/get/noto/#serif-hans
Test workability
Include or not ?

Sans-serif

NotoSansCJKjp-[weight].otf and NotoSansMonoCJKjp-{Regular, Bold}.otf
NotoSansCJKkr-[weight].otf and NotoSansMonoCJKkr-{Regular, Bold}.otf
NotoSansCJKsc-[weight].otf and NotoSansMonoCJKsc-{Regular, Bold}.otf
NotoSansCJKtc-[weight].otf and NotoSansMonoCJKtc-[weight].otf
36 OTF font files in total: 4 languages × 9 font files per language

Serif

NotoSerifCJKjp-[weight].otf
NotoSerifCJKkr-[weight].otf
NotoSerifCJKsc-[weight].otf
NotoSerifCJKtc-[weight].otf
28 font files in total: 4 languages × 7 font files per language

Pre-release

This is one of the most exciting Chinese font projects at the moment. Is it possible to get a pre-release to see how the project is progressing and to add some screenshots on Github?

Data duplicates crash the script

Add json data preprocessing step to avoid duplicate (?)

cri-ninja@hp-840:~/yug/Hanzi-Pinyin-Font$ npm start

> hanzi-pinyin-font@ start /home/cri-ninja/yug/Hanzi-Pinyin-Font
> node --max_old_space_size=8192 --optimize_for_size --max_executable_size=4096 --stack_size=4096 --require babel-core/register ./index.js

Error: The glyph name "㐀" must be unique.
    at SVGIcons2SVGFontStream._svgIcons2SVGFontStreamTransform [as _transform] (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/svgicons2svgfont/src/index.js:153:27)
    at SVGIcons2SVGFontStream.Transform._read (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_transform.js:159:10)
    at SVGIcons2SVGFontStream.Transform._write (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_transform.js:147:83)
    at doWrite (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_writable.js:345:64)
    at clearBuffer (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_writable.js:445:7)
    at onwrite (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_writable.js:376:7)
    at WritableState.onwrite (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_writable.js:126:5)
    at afterTransform (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_transform.js:81:3)
    at TransformState.afterTransform (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/readable-stream/lib/_stream_transform.js:58:12)
    at SAXStream.svgicons2svgfontSaxEnbCb (/home/cri-ninja/yug/Hanzi-Pinyin-Font/node_modules/svgicons2svgfont/src/index.js:259:7)
(node:30055) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): TypeError: Cannot read property 'ttf' of undefined
cri-ninja@hp-840:~/yug/Hanzi-Pinyin-Font$ git pull

Support different fonts for glyph and it's ruby text

When working with Noto font, one issue is due to the fact a font file is limited to one script. For instance, NotoSansTifinagh-Regular.ttf only contains the Tifinagh's glyphs but not the ASCII.

So the generated font show tofu box instead of the pronunciation (as seen below):

Idea

We should be able to open multiple font in order to be able to create give and pronunciation with different script (e.g. Tifinagh with latin or Arabic, Hanzi with Pinyin or Juyin, etc.)

Note: this is another reason to try fontkit

Spread the word about Tifinagh-font

When a release is available for Tifinagh-font, ping @taziden and @BoFFire for feedback.

Error display ab

I have an error displaying a few characters

Todo: Tifinagh Arabic and IPA dataset

Script currently doesn't extract the correct data when creating the file

Provide a /font/ directory with open CJK fonts (Kaishu and Mingti)

Submodule : https://github.com/saiswa/free-fonts

Github page website ?

Create the webpage file structure : index.html, js/script.js, css/styles.css
Create a minimal page based on readme.md + some elegant css / images / parallax.

Quick, fork something, no custom dev.

JavaScript heap out of memory: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed

mirror: itgalaxy/webfont#54

I run the my script on a directory with 41214 SVG files and got the following error

function buildFont(config) {
  return webfont({
    files: config.inputFiles,
    fontName: config.fontName
  })
    .then(content => content)
    .catch(err => {
      console.log(err);
    });
}

Error

node --require babel-core/register ./index.js
already exists                                 
                                               
<--- Last few GCs --->                         
                                               
  232006 ms: Mark-sweep 1350.0 (1406.9) -> 1349.8 (1406.9) MB, 574.3 / 0.0 ms [allocation failure] [GC in old space requested].
  232583 ms: Mark-sweep 1349.8 (1406.9) -> 1349.8 (1406.9) MB, 576.6 / 0.0 ms [allocation failure] [GC in old space requested].
  233164 ms: Mark-sweep 1349.8 (1406.9) -> 1349.8 (1403.9) MB, 580.3 / 0.0 ms [last resort gc].
  233745 ms: Mark-sweep 1349.8 (1403.9) -> 1349.7 (1403.9) MB, 580.3 / 0.0 ms [last resort gc].
                                               
                                               
<--- JS stacktrace --->                        
                                               
==== JS stack trace =========================================
                                               
    2: arguments adaptor frame: 2->1           
Security context: 0x4e54c9cfb39 <JS Object>    
    3: round [/data/projects/hanzi-pinyin-font/node_modules/svg-pathdata/src/SVGPathData.js:12] [pc=0x26e3767b3b09] (this=0x217c839e5e99 <an SVGPathData with map 0x375d0749d7b1>)                                 
    4: arguments adaptor frame: 1->0           
    5: /* anonymous */(aka /* anonymous */) [/data/projects/hanzi-pinyin-font/node_modules/svgicons2svgfont/src/index.js:336] [pc=0x26e376698166]...                                                               
                                               
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]                       
 2: 0x109b1dc [node]                           
 3: v8::Utils::ReportApiFailure(char const*, char const*) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::DeoptimizationOutputData::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node] [node]
 7: v8::internal::FullCodeGenerator::PopulateDeoptimizationData(v8::internal::Handle<v8::internal::Code>) [node]
 8: v8::internal::FullCodeGenerator::MakeCode(v8::internal::CompilationInfo*) [node]
 9: v8::internal::Compiler::EnsureDeoptimizationSupport(v8::internal::CompilationInfo*) [node]
10: 0xb538d7 [node]                            
11: 0xb54d4d [node]                            
12: v8::internal::Compiler::CompileOptimized(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ConcurrencyMode) [node]                                                                       
13: v8::internal::Runtime_CompileOptimized_Concurrent(int, v8::internal::Object**, v8::internal::Isolate*) [node]
14: 0x26e374e092a7                             
Aborted                                        
error Command failed with exit code 134.

Question

How can I solve this?

related:

process more than one glyph

Adding Jyutping (粵拼)

Hello, there are many fonts I can find with pinyin (拼音) on top, but I've never found a font with Jyutping (粵拼) on top.

Would you ever consider creating one?

This would allow you to learn Cantonese pronunciations through the many apps that only support Mandarin.

Thank you.

Convert text to SVG path -- solved but to integrate

JS text-to-svg: Convert text to SVG path without native dependence using ;
using Inkscape v0.49+ --export-text-to-path option ;

Error: Invalid code point NaN

mirror: mathiasbynens/punycode.js#65

I'm trying to generate glyphs based on a file containing a list of unicode codepoint from U+3400 to U+2B6F8:

U+3400	qiū
…
U+2B6F8	tí

Code

  convertToJson: (filepath, options) =>
    String(fs.readFileSync(filepath)).split("\n").map(line => {
      const [codepoint, ruby] = line.split(options.separator);
      const unicodeCodepoint = codepoint.split("+")[1];
      const hexCodepoint = parseInt(unicodeCodepoint, 16);
      let glyph = punycode.ucs2.encode([hexCodepoint]);
      console.log(codepoint, hexCodepoint, glyph);
      return { codepoint, ruby, glyph };
    }),

Error

6:   convertToJson: (filepath, options) =>
7:     String(fs.readFileSync(filepath)).split("\n").map(line => {
8:       const [codepoint, ruby] = line.split(options.separator); 

Error: Invalid code point NaN

Object.convertToJson (src/dataminer.js:7:51)         
Test.fn (test/dataminer.test.js:26:20)

Question

I can't find the problem, is it a character or my approach?

How to add the font on mobile device?

fix(install): create a .tmp/ directory

Required by bin/extract-unicode-pinyin.bash otherwise get the following error:

bin/extract-unicode-pinyin.bash: line 58: /home/elopez/projects/Hanzi-Pinyin-Font/.tmp/unicode-pinyin.csv: No such file or directory

Fix(install) for php5 on classic ubuntu distribs

git clone, npm install roll out smoothly.

make install-requirements fails due to php5 :

You may gain to improve your make file. There is a lead : http://askubuntu.com/questions/756879/cant-install-php5-on-ubuntu-16-04

Version tarballs for inclusion in Linux

@baimafeima says:

@edouard-lopez Yeah but I meant that if you make ruby-font-creator itself available as versioned tarballs with a changelog here on GitHub, then it may be considered for inclusion into the repositories of various Linux-based distributions and progress can be tracked by distribution maintainers. I would also appreciate to have a separate issue tracker for the hanzi-pinyin-font. What do you think?

parlr / ruby-font-creator Goto Github PK

ruby-font-creator's People

Contributors

Stargazers

Watchers

Forkers

ruby-font-creator's Issues

Moedict

Unicode :

CJKlib

Noto Sans SC Sliced (Chinese Simplified)

Usage

Resources

Expected

Actual (with NotoSans)

Idea

Error

Question

Code

Error

Question

Recommend Projects

Recommend Topics

Recommend Org