Hi,
People
who went to LGM 2016 London in April this year and heard my talk would
hear that, IMHO, the most unique and valuable part of the Font Validator
is its analysis on individual glyphs. That is broadly divided into the
glyf table test on the glyph contours, and the rasterization test on the
truetype hinting instructions. The latter wasn't open-sourced by
Microsoft, but was re-implemented, much faster with broader scope and
wider supported platforms, and superceding the old version in 2.0 a few
weeks ago.
So I have turned my attention to the glyf test, and
threw all my (many buggy) fonts to try to break it. When the glyf test
itself breaks and cannot handle the situation, it is called "A1112
exception", and I filed them under
https://github.com/Microsoft/Font-Validator/issues in the last few days.
Out
of that activity, I collected the reports of the ~3000 fonts on Fedora
(about ~2000 under /usr/share/fonts, and another ~1000 texlive). About
half of them, (~1300 /~2000 for general libre fonts, and ~200 / ~1000
for texlive) have some level of glyph errors or warnings.
At the rate of 2/3, almost all libre font authors should have a look at their own reports and fix their fonts.
fedora-glyf-EW-reports-file-index-2016-08-11.txt
is the file index and "fedora-glyf-EW-reports-2016-08-11.tgz" is the
collection of reports under
http://htl10.users.sourceforge.net/tmp/FontVal-test-results-2016July/The file index entries look like this, sorted by the "FileNameAndPath" field (i.e. this is the first entry, with "aa..."):
tmp6b009ab7.tmp.report.xml:<FontFile
FileName="Comfortaa-Bold.ttf"
FileNameAndPath="/usr/share/fonts/aajohan-comfortaa/Comfortaa-Bold.ttf">
and
author of "/usr/share/fonts/aajohan-comfortaa/Comfortaa-Bold.ttf"
should extract "tmp6b009ab7.tmp.report.xml" from the tar ball, etc.
Disclaimer:
I am just the messager - I just ran the glyf test on all the fonts I
have, and played no part in writing the test itself. People disagreeing or have
questions with the reports' content should file upstream under
https://github.com/Microsoft/Font-Validator/issues .
People
who want to make the glyf test even better can look at and help fixing
the A1112 exception's - I honestly do not mind somebody else doing it!
:-). People who want me to make it better, please feel free to make a
donation (
https://sourceforge.net/p/hp-pxl-jetready/donate/ ).
Hin-Tak
P.S.
It has been exactly a year ago today (baring time-zone arguments...),
that I received and first saw the code bundle from Microsoft to get
fixed up, to be publicly released a few months later. Hurray.
Comments
Is a contour made out of one or two points always an error which need to be fixed? Do you allow for local off-curve extremes? How about composite glyphs with overlapping contours?
Calibri contains 157 unused glyphs, and 40 glyphs have contours with redundant points. It also contains 2 lookups which are not accessible.
San Francisco contains 93 unused glyphs, 17 glyphs with off-curve extreme points. It has invalid PANOSE.
"Composite glyphs should be used whenever a glyph image is identical to another. [...] Overlapping composite glyphs can be used." [1]
[1] Hinting and Production Guidelines, Section 3.2.1,
https://www.microsoft.com/typography/ProductionGuidelines.mspx#glyph
Incidentally, I filed an issue about the used error and warning status not properly documented and explained some months ago when I was looking at the help file, so the full list of issues that the glyf test can flag, collected from the code itself, is listed in:
https://github.com/Microsoft/Font-Validator/issues/15
You can see the whole list of what is considered a problem in the glyph contours, and what should be done about them, towards the end, in the updated and merged help file with bits recovered from the old help file. I am just the messager, as I should repeat. I don't decide what is and what is not an error...
Some of it is possible - e.g. word processor high-lights possibly spelling mistakes - but it is not possible nor convenient to check grammar on every 'save'. But there is a a place for dedicated and detailed external analysis. That's why you get proof-readers, software that analyses articles for plagiarism, etc.
FWIW, the glyf test for the typical CJK font takes about an hour and half. It is certainly quite impractical for any font editors to do that every time per load or per save.
No.
Here is microsoft's recommendation:
https://github.com/HinTak/Font-Validator/blob/master/NewHelp/W1110.htm
In the same directory there are others for the other error codes.
These files make the chm file:
https://github.com/HinTak/Font-Validator/blob/master/bin/FontValidatorHelp.chm
At least for W1110, I think it is written quite clearly.
I haven't read the others, and I really wish people read the bundled documentation - the chm file - before saying random things.
"Although Microsoft discourages this type of glyph design, for legacy reasons this overlap is allowed for composite glyphs."
I really wonder about this statement. It seems to be saying that, for legacy reasons, overlap is allowed, which doesn't make much sense when you consider the statement just before it:
"This type of glyph design introduces some ambiguity in terms of defining what is the 'inside' and 'outside' of a glyph shape, and may cause a rasterizer to reverse the overlapped portion. This problem is generally seen on older printers."
It seems like maybe the comma was misplaced and it was meant to go like this:
"Although Microsoft discourages this type of glyph design for legacy reasons, this overlap is allowed for composite glyphs."
If you follow the link to Microsoft's TrueType spec referenced in the warning message, you'll find this:
"If set, the components of the compound glyph overlap. Use of this flag is not required in OpenType — that is, it is valid to have components overlap without having this flag set. It may affect behaviors in some platforms, however. (See Apple’s specification for details regarding behavior in Apple platforms.)"
Truth is that there are many different rasterizers and font shaping engines out there that do not follow the latest specs, and sometimes we have to make fonts that need to work for those environments.
Point in case: the very latest OT spec that includes variable fonts *does* allow overlapping contours.
I just took another look at SFNSText-Regular, and besides the issues I mentioned earlier, it also has corrupt feature params concerning the character variants. If they are valid, let me know, as then I need to solve a bug in FontCreator!
Like many font editors, FontCreator has its own font validation features. Most outline related issues can be automatically fixed, but this is optional.
EDIT: my program will give control over the approximation. You can set a minimum size of curves, and choose for tight approximation with many curves, or loose approximation with few curves. This should avoid problems like you mentioned.
Remove overlap is indeed helpful but should also be optional, in case you want to generate a variable font where they are needed to make interpolations possible.
So a few general points:
- the bundled CHM file is auto-generated from the actual error code status list; it is actually quite difficult to add a test or add a few tests (as I did with CFF and SVG) without filling in a corresponding help section, because the compiler complains. So read the CHM file first, as it is written by the same people/person who wrote the test. Hopefully he/she explained why some issue is considered troublesome and warrant a warning. I am quite happy to include additional paragraphs from people if people feel like they can explaining issues better.
- on W1110, I thought it was fairly clearly written - there is "what the spec allows", and there is "older legacy systems (printers)". If you think your font will never ever be used for printing, or on older systems, then obviously you could ignore it. I don't really understand why the issue needs dragging on and out. But remember 'web font' does not mean "screen-rendering only" as the user's printer needs to be able to cope too, if he/she decides to print from the Web browser.
- farming off issues to font engines or font editors seems irresponsible. Yes, font engines can cope with a lot of weird shit; and yes, many font editors can auto-correct or warn on issues.
But - font engines (both on screen in the major ones, and printers) are not created all equal, or up to spec. There are part of the spec which are well-written and therefore well-implemented by most rendering engines; there are part which are ambiguous, and people have different opinions - for this a "font checker" should warn ; and there are undocumented behaviors (where different font engines are free to behave differently). yes, if you create a font that is meant to work with one platform - e.g. Windows, and *not* ever print from windows, then you can ignore a lot of that too.
Relying on Font editor to auto-correct /warn is quite impractical - a font editor, particularly the interactive kind, need to have a guaranteed response time - it must save your work fully within say, 2 minutes after you press the "save" button; while it can and should check for common problems, a dedicated "font checker" that will spend as much time and do as thorough a report on any possible issues, without needing to worry about responsiveness, and in turn, you only run once in a while and let it takes as long as it will take to do a check, is a good thing.
Out of the 300 fonts that windows ships, about 10% took longer than 20 minutes to run the full check. The longest is 7 hours with one of the CJK fonts. There are small changes e.g. In prep or cvt which affects every glyph - yet you do not want to wait 7 hours for your font editor to check, every time you touch either prep or cvt. so you run a check only when you think is needed.
That's a good thing to separate the role of font editor vs the role of font checker - one optimises for responsiveness, the other for thoroughness.
So if your font is crap and happens to work okay on screen, you get better result in cheap printers. By "better" I mean "closer to screen display".
This is the sort of changes I'd prefer people filing the issue upstream and let the Microsoft folks explain what they mean by those words. Sorry.
I am fairly tired of people wanting to weaken tests. The way I see it, font checks should only get stricter, more warnings and more errors, as there are more old/odd/different, and different versions of renderers over time, and more problems to trip over. It should rarely if ever get weaker. If you just want some software to "okay" rubber-stamp your font, maybe just writing "my font is okay" on a piece of paper and put it up on your own wall is easier.
If you think a warning/error message does not apply to you, and you do not want to support old/odd renderers, please ignore it. No need to tell me.
Many practices are "allowed" (i.e. tolerated), but not encouraged. I think that was the meaning of Microsoft's message. And between "allowed" (tolerated) vs "encouraged", there is room for a warning message.
The discussion is not about weakening tests, but finding out what the intent of the wording actually was.
Especially when dealing with font size optimizations (web fonts), using overlapping components can be a huge gain. So I think it is fair to question the intentions of this particular check.