Home› Type Design Software

Microsoft Font Validator lives!

Dave CrosslandDave Crossland Posts: 1,250
edited November 2015 in Type Design Software

This week Aaron Bell published a liberated Microsoft Font Validator on Github under the MIT libre license:

https://github.com/Microsoft/Font-Validator

Buggy fonts benefit no one and having the Microsoft Font Validator available for all foundries and type designers to contribute to keeping updated is great news :smiley: 

I'd like to congratulate Aaron and especially Hin-Tak Leung for making this possible :smile: 

This tool was developed mainly in the 1990s, and had fallen into an unmaintained state. The version downloadable from the Microsoft typography web site is from 2003 and it performs 177 tests. 

Aaron and I spoke about liberating it so it could be brought up to modern standards in NYC last year, and after getting approval from his team I sought out some volunteers to take a look.

The main obstacle to liberating the tool was that it partially depended on the Windows font rendering system which was unlikely to be liberated, and less importantly, parts of MSIE for XML report rendering.

The corresponding libre renderer is FreeType and Hin-Tak Leung, a FreeType developer, volunteered; his experience with Mono, the libre implementation of Microsofts app platform, made the work ahead to make it run with and without Windows somewhat familiar.

Aaron was then able to pass Hin-Tak parts of the source code that was updated inside MS around 2009, for the last OpenType spec being prepared at that time as well as testing for ClearType rasterization. The 2009 code does 194 tests.

Today Mono on non-windows can be used to build and run it, and Hin-Tak is mainly developing it using GNU+Linux, and i works on Mac OS X also.

Hin-Tak did an amazing job and now that it runs, Aaron has been able to finally post it publicly. Also my thanks to Cosimo Lupo at Dalton Maag for testing and verifying Hin-Tak's work along the way :)

The 189/194 status in the commit message on the MS repo today is a reference to when the code first passed through Hin-Tak's hands, as 5 of the 194 test did not work; Hin-Tak has since got it up to all but one failing, the DSIG digital signature verification. This version is in his fork, https://github.com/HinTak/Font-Validator

Hin-Tak's near-term roadmap is:

  • FreeType-based rasterisation
  • improving the DSIGInfo tool until it can be scavenged to make the last DSIG test work - the tool currently just reads the signature and tells you about the certificates and the digest, but makes not attempts to verify either (yet.)
  • bring it further up from 2009 level spec to the current 2015 spec.
  • support for CFF fonts
  • replace the missing built-in XML reporter viewer

However, finishing those and other remaining pieces sounds like a full time job for a few months - so Hin-Tak would welcome interested parties to commission him to continue. Would anyone here contribute to a crowdfund for that?

Also, MS requires contributors to sign their microsoft-contribution-license-agreement.pdf - does anyone thinking of contributing have any objections to doing so?

Tagged:
«134

Comments

  • SiDanielsSiDaniels Posts: 277
    Well done Aaron, Dave and Hin-Tak. Nice to see this tool rise from the ashes. Hopefully it will prove useful. Next up VOLT? ;-)
  • Ray LarabieRay Larabie Posts: 1,280
    It'll be nice to have it back. There are few things more humbling to a type designer than page after page of red errors from Font Validator.
  • James PuckettJames Puckett Posts: 1,911
    Thanks to all responsible parties! 
  • Cosimo LupoCosimo Lupo Posts: 1
    Great news! thanks Hin-Tak, Dave and Aaron for making this happen. It's nice to see Microsoft contributing to the open source world of font tools.

    @SiDaniels VOLT +1
  • Paul van der LaanPaul van der Laan Posts: 238
    Nice! The old FVAL is still part of our QA process. Thanks to all involved!

    Curious: does the new version still red flag usWeightClass values that are not multiples of 100?
  • Aaron BellAaron Bell Posts: 88
    @Paul van der Laan This is just a first step getting the code out there, then updating work happens!
  • Aaron BellAaron Bell Posts: 88
    @Frode Bo Helland I'm planning to add the binaries soon, but in the meantime you can grab them from here. 
  • JoeManbeckJoeManbeck Posts: 22
    This is great. Thanks for calling attention to it.
  • Craig EliasonCraig Eliason Posts: 1,317
    edited November 2015
    Aaron Bell said:
    @Frode Bo Helland I'm planning to add the binaries soon, but in the meantime you can grab them from here. 
    Is that link correct? Looks like it goes to some printer drivers.

    EDIT: and if it is indeed the correct file, what's the password to unzip the binary?
  • Sami Artur MandelbaumSami Artur Mandelbaum Posts: 90
    Aaron Bell said:
    @Frode Bo Helland I'm planning to add the binaries soon, but in the meantime you can grab them from here. 
    Thank You Aaron, but the windows version is password protected.
    Could You share the password, please.
    Thank's
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    Sami Artur Mandelbaum said:
    Aaron Bell said:
    @Frode Bo Helland I'm planning to add the binaries soon, but in the meantime you can grab them from here. 
    Thank You Aaron, but the windows version is password protected.
    Could You share the password, please.
    Thank's
    Hello! You must have clicked on the earlier private review bundles or the winewhybrid one? Use the latest 11-12-bin if you just want to give it a go.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    There is actually no special 'Window' version - it is one single binary bundle which works on Linux and mac with mono, and on windows with dotnet. You might be confused by the win32.hybrid one, which is a different thing - see the dev slide.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    Use the 11-12-bin one.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    Frode Bo Helland said:
    How does one install it? (Not all of us are familiar with Terminal)
    Just unzip 11-12-bin.zip, and read the reader.txt
  • Hin-Tak LeungHin-Tak Leung Posts: 358

    Paul van der Laan said:
    Nice! The old FVAL is still part of our QA process. Thanks to all involved!

    Curious: does the new version still red flag usWeightClass values that are not multiples of 100?
    The relevant code is near OTFontFileVal/val_OS2.cs near line 233. If it doesn't do what you think it should, please file an issue , and please quote a paragraph from the spec when you do. And better still send a patch! :smile: 
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    The password for the older review bundles isn't a big secret, but you should use the latest which is password-less.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    Hin-Tak Leung said:
    Frode Bo Helland said:
    How does one install it? (Not all of us are familiar with Terminal)
    Just unzip 11-12-bin.zip, and read the reader.txt
    Stupid auto-spell check on my phone s/reader/readme/. The binaries are usable after unzip. There are two, one GUI, one batch mode. If you are on windows, double-click on either tell you which is which - one launches the GUI, the other on console gives you a list of command options. Oh, the GUI one has a colorful icon also.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    Paul van der Laan said:
    Nice! The old FVAL is still part of our QA process. Thanks to all involved!


    Curious: does the new version still red flag usWeightClass values that are not multiples of 100?
    The warning is correct - see https://www.microsoft.com/typography/otspec/os2.htm#wtc . Only 100-900 in increment of 100 is allowed. If you disagree, contact the spec committee, don't blame the software for telling you what the spec says... :wink: 

  • Dave CrosslandDave Crossland Posts: 1,250
    edited November 2015
    It's not useful to have the tool fail things that are now common practice, and wait for a very slow moving spec to update. It's better to reduce the outcome to a warning/informative note that explains while the spec says 00s only, because windows autobolds values below 250, many designers consider such values undesirable and using non-00 values like 250 and 275 for what "should" be 100 and 200 is ok.

    The tool can move faster than the spec and should take into account things like large longstanding platform behavior, to help font developers produce good fonts. 

    So I would consider anything under 250 to be a bug.
  • John HudsonJohn Hudson Posts: 2,640
    edited November 2015
    It's not useful to have the tool fail things that are now common practice, and wait for a very slow moving spec to update.

    Or not to update at all. As I understand it, the OT and CSS specs are mutually supporting in this limitation, and one can't change one without changing the other, and hence requiring a lot of people to alter how their software handles weight class.

    At this point, my inclination would be to freeze these bits of both specs, declare them legacy, and define new common weight scaling data. If we were really bright, we could express this as a ratio of principal stem weight to a given metric height in the font (e.g. the nominal cap height, bearing in mind that for non-European scripts this might not correspond to a particular letter feature), which would enable the weight scaling data to be used in CSS more like font-size-adjust.

    When something is broken in a way that no one seems able to agree on fixing, the obvious thing to do is throw it out and start again.
  • James PuckettJames Puckett Posts: 1,911
    edited November 2015
    At this point, my inclination would be to freeze these bits of both specs, declare them legacy, and define new common weight scaling data.
    I love this idea. But I think we’d have to keep supporting the old broken spec for twenty years because half the programmers working with this stuff would just keep implementing the old spec.
  • John HudsonJohn Hudson Posts: 2,640
    I love this idea. But I think we’d have to keep supporting the old broken spec for twenty years because half the programmers working with this stuff would just keep implementing the old spec.

    Of course. As I'm fond of saying, all problems are legacy problems. But there's lots of stuff that we handle in this way, replacing older stuff with new and continuing to support the old stuff until we're reasonably sure that almost everyone has switched to the new. Often, the older stuff isn't even broken in the way weight class is: it's just limited in some way that we'd like to overcome. Conversely, incremental fixes or add-ons to older stuff with fundamental flaws just tends to accumulate more and more legacy issues, which is why I tend to favour the amputation solution, even if we have to carry the dead limb around with us for a while,
  • Adam TwardochAdam Twardoch Posts: 506
    @Hin-Tak Leung , @Aaron Bell: HinTak's fork https://github.com/HinTak/Font-Validator is missing the Issues tab on Github. Is this a deliberate decision? Is https://github.com/Microsoft/Font-Validator/issues the correct place to file issues, even for Hin-Tak's fork? 
  • Sami Artur MandelbaumSami Artur Mandelbaum Posts: 90
    I did some tests and Font Validator still says stylistic sets is unregistered:

    "The FeatureRecord tag is valid, but unregisteredFeatureList, FeatureRecord[1], tag = 'ss20'

    Are the opentype specifications updated?

    Regards

    Sami
  • Thomas PhinneyThomas Phinney Posts: 2,467
    Hin-Tak Leung said:
    Paul van der Laan said:
    Nice! The old FVAL is still part of our QA process. Thanks to all involved!


    Curious: does the new version still red flag usWeightClass values that are not multiples of 100?
    The warning is correct - see https://www.microsoft.com/typography/otspec/os2.htm#wtc . Only 100-900 in increment of 100 is allowed. If you disagree, contact the spec committee, don't blame the software for telling you what the spec says... :wink: 

    That's not true. The spec never says that only increments of 100 are allowed. It’s just that those increments are the only ones labeled. If the spec said what you claim it does, those of us at Adobe at the time would not have implemented what we did in fonts.

    Logically, if there was no expectation of using finer increments, why would weightclass even go in 100s? Why wouldn't it just use single units like widthclass?

    So, (1) the spec does not in fact have the claimed restriction, and (2) based on the content of this and neighboring sections, it is entirely reasonable to assume the opposite.
  • Saga SöderbackSaga Söderback Posts: 42
    MS releasing this and a new version of VTT within weeks of each other makes for a very happy autumn to those of us on the deep nerd end of font dev. Kudos to everyone who helped along the way.
  • Paul van der LaanPaul van der Laan Posts: 238
    Thank you Thomas, for weighing in (no pun intented).

    BTW, something else that definitely needs to be updated: FVAL currently red flags newer versions of the OS/2 (v.4) and gasp (v.1) tables.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    edited November 2015
    Thomas Phinney said:
    Hin-Tak Leung said:
    Paul van der Laan said:
    Nice! The old FVAL is still part of our QA process. Thanks to all involved!


    Curious: does the new version still red flag usWeightClass values that are not multiples of 100?
    The warning is correct - see https://www.microsoft.com/typography/otspec/os2.htm#wtc . Only 100-900 in increment of 100 is allowed. If you disagree, contact the spec committee, don't blame the software for telling you what the spec says... :wink: 

    That's not true. The spec never says that only increments of 100 are allowed. It’s just that those increments are the only ones labeled. If the spec said what you claim it does, those of us at Adobe at the time would not have implemented what we did in fonts.

    Logically, if there was no expectation of using finer increments, why would weightclass even go in 100s? Why wouldn't it just use single units like widthclass?

    So, (1) the spec does not in fact have the claimed restriction, and (2) based on the content of this and neighboring sections, it is entirely reasonable to assume the opposite.
    I am inclined to disgree - the tool is supposed to take a pedantic approach, to warn for any indeterministic behavior.

    For a better example - the EBLC/EBDT tables - it states that bit depths of 1,2,4,8 are allowed. The newer CBLC/EBDT says they allow those, in addition to 32. Now what happens if somebody wrote something that 15, 16, 24? You could says well, the X server does 5,5,5, for 15, and 5,6,5 for 16, and 24 for true color; and pixels should be packed contiguously but another person would say, well, 24 should be intepreted as 32 where the higher 8 bits in not used, by the pixels should be padded to 32 bit.

    So the correct approach is to warn any non-specified behavior where there is possible alternative interpretation of off-spec-ness. In this case, one person can implement software to process intermediate value as is (which seems to be what you suggest), but another person might read the spec and say, well, intermediate values should be rounded up or down when used. And both are correct, and both could be wrong. So you warn.
  • Hin-Tak LeungHin-Tak Leung Posts: 358
    Sami Artur Mandelbaum said:
    I did some tests and Font Validator still says stylistic sets is unregistered:

    "The FeatureRecord tag is valid, but unregisteredFeatureList, FeatureRecord[1], tag = 'ss20'

    Are the opentype specifications updated?

    Regards

    Sami
    I added that as commit b7ee9053ecea8fb168cb43edb05362c0adbf4df5 , "sync the feature list table to 2015 spec", on Fri Nov 13 20:50:16 2015 . That went in after I uploaded the binary. It will be in the next binary I do, I guess.



Sign In or Register to comment.