Autohinting Variable Fonts

Rafał Buchner

Hi,
I have a question.
I'm curious what are the most current ways of autohinting VFs? Currently, I'm using fontmake to export my fonts. I was wondering if I could easily apply any ttf-autohinting to the font file exported with this module.

Viktor Rubenko

TrueType autohinting is disgusting. Forget about it. 

If you really want to do ttf-hinting, do it manually, even ttfautohint needs manual deltas.
VTT, Glyphs, FL, don't know about ttfautohint support for vars.

Piotr Grochowski

Viktor Rubenko said:

TrueType autohinting is disgusting. Forget about it. 

If you really want to do ttf-hinting, do it manually, even ttfautohint needs manual deltas.
VTT, Glyphs, FL, don't know about ttfautohint support for vars.

From my experience with constant fonts ttfautohint hinting is virtually the same quality as hinting in ClearType (vertical hinting) because it pretty much is. So I suppose you mean to suggest manual hinting so that horizontal hinting can be done as well, and that makes sense because full hinting (including fixing pixel patterns) is a fundamental part of a constant TrueType font. Not sure about variable fonts because I haven't had experience with them and most systems probably don't even support it.

Rafał Buchner

hehe, @Victoria Rushton
not really helpful

Viktor Rubenko

Piotr Grochowski said:

Viktor Rubenko said:

TrueType autohinting is disgusting. Forget about it. 

If you really want to do ttf-hinting, do it manually, even ttfautohint needs manual deltas.
VTT, Glyphs, FL, don't know about ttfautohint support for vars.

From my experience with constant fonts ttfautohint hinting is virtually the same quality as hinting in ClearType (vertical hinting) because it pretty much is. So I suppose you mean to suggest manual hinting so that horizontal hinting can be done as well, and that makes sense because full hinting (including fixing pixel patterns) is a fundamental part of a constant TrueType font. Not sure about variable fonts because I haven't had experience with them and most systems probably don't even support it.

What do you mean by "hinting in ClearType"?
TTFA looks good for DirectWrite and light weights. 
And most system support variable fonts nowerdays, but not most apps. The main thing is that modern browsers support it.

Piotr Grochowski

Viktor Rubenko said:

Piotr Grochowski said:

Viktor Rubenko said:

TrueType autohinting is disgusting. Forget about it. 

If you really want to do ttf-hinting, do it manually, even ttfautohint needs manual deltas.
VTT, Glyphs, FL, don't know about ttfautohint support for vars.

From my experience with constant fonts ttfautohint hinting is virtually the same quality as hinting in ClearType (vertical hinting) because it pretty much is. So I suppose you mean to suggest manual hinting so that horizontal hinting can be done as well, and that makes sense because full hinting (including fixing pixel patterns) is a fundamental part of a constant TrueType font. Not sure about variable fonts because I haven't had experience with them and most systems probably don't even support it.

What do you mean by "hinting in ClearType"?
TTFA looks good for DirectWrite and light weights. 
And most system support variable fonts nowerdays, but not most apps. The main thing is that modern browsers support it.

hinting in ClearType = vertical hinting. 
And DirectWrite is not a good design because any software using it can enable distorted rendering (such as enabling 'render contrast' or setting gamma to a different value than 2.2 or 2.3, which is distortion because it no longer makes a linear scale but makes a biased scale) as a built in DirectWrite option and not even be aware of it. ttfautohint'ed text renders best in the standard Microsoft renderer with ClearType disabled (classic GDI renderer). Of course, full manual hinting on classic GDI is the best rendering.

Aaron Bell

@Rafał Buchner @Viktor Rubenko

Having done quite a bit of hinting for Microsoft (including variable fonts), I feel particularly well-positioned to answer your question

VTT can do variable font autohinting. I assume that other applications can as well, but I use VTT as that is the tool that Microsoft requires for their projects. 

I view autohinters (in general) as a "best case scenario" kind of solution for hinting fonts. Which is to say, if you happen to fall within a narrow band of Latin / weight / contrast / design ranges, it can understand and apply hints correctly.

However, anything outside of that range will require manual hinting to achieve the best results. For example, an extremely thin font, one that is high-contrast, one that includes other scripts like Devanagari or Tibetan, or a design with slightly wonky design elements. 

In the case of variable fonts, where there are often extended ranges of weight, or style, or what-have-you, there is a greater chance that the hints applied will not work well for the entire range, or will need additional tweaks. Keep in mind that the autohinter only sees the default instance, and has no understanding of any other, so it is on you to double-check. 

If you have a design that requires manual hinting and you choose not to do so, I think it best to just set the GASP table to grayscale / symmetric smoothing across the entire PPEM range and just let it blur. It won't be as clear at smaller sizes, but the middling sizes where most people will experience the font won't be ruined by bad hints.


As an aside, for the Cascadia Code project, I hinted the font in VTT, then exported the font TSI tables into a separate ttx file, and then merge the hints in using vttLib as part of my fontmake build script. It still needs to be "shipped" out of VTT as a post production step, but that's a small price to pay for better rendering. 

Rafał Buchner

Thanks, @Aaron Bell.
Your answer is very helpful.
 

Peter Baker

A few observations from an Open Source point of view.

I have used VTT for hinting an Open Source variable font, and I liked it, up to a point. Its autohinting is a good start, but only a start: you have to edit pretty much every glyph. But in my view it fits uncomfortably in an Open Source workflow, since its reliance on GIDs to identify glyphs presupposes that the number and order of glyphs in the font will not change. That’s probably as it should be for commercial fonts, but not (in my experience) for Open Source, since the stream of users asking for additional glyphs for particular purposes (Egyptology, Uralic phonetics, medieval Polish …) is pretty constant: you never really "ship" such a font. You can make sure new glyphs are always added to the end of the file, but that feels kludgy, and it gets kludgier the longer the list of additions gets. The ability to export everything (except the cvar table—is that an oversight?) into a big XML file helps a little but does not solve the problem. Finally, you end up with a lot of Microsoft programming in your font, and I have no idea what the licensing situation is. That makes me nervous.

For Open Source, then, ttfautohint is the only game in town, but it does not support variable fonts, and as the last ttfautohint release appears to have been in April 2019, I wonder if the project has been abandoned so that it never will. Someone created a VF version some time ago, and you can find it around here and there, but I can’t find a repository for it. It does seem to work: I’ve used it, and I’ve examined the cvar table, which looks okay, but I don’t know what problems might be lurking: no one seems to be vouching for this program.

And of course ttfautohint has the big problem that @Aaron Bell identified: any kind of curve ball defeats it. I have a font with cupped serifs, for example, and it copes poorly. If its (to me) rather opaque control instructions can help, I haven’t figured out how.

Still, ttfautohint is the only game in town for Open Source, and the mysterious VF version is the only game in town for Open Source variable fonts. I’m using it, but to correct its errors I’ve resurrected an ancient (2006!) program of mine called Xgridfit—switching its dependency from FontForge to fontTools, reducing the horrible verbosity of the older version (it was an XML-based language much influenced by XSLT), and adding the ability to generate a cvar table, meanwhile retaining its ability to layer the programming it generates on top of what it finds in a font. So I can run ttfautohint, identify the glyphs where it is failing, and write new programs for just those.

Its far from ideal—no one has ever liked my musty old program much, even me! But I would describe the hinting situation for Open Source fonts as pretty desperate.

Aaron Bell

Peter Baker said:

But in my view it fits uncomfortably in an Open Source workflow, since its reliance on GIDs to identify glyphs presupposes that the number and order of glyphs in the font will not change.

I spent a bit of time researching the TSI tables (that VTT uses for storing hint data) and it turns out there's actually only one place where GIDs matter. In TSI1, in cases where components are used in the font, VTT uses the OFFSET command to identify the GID of the component. No other tables rely on GID. So if you're able to correct the OFFSETS throughout TSI1, then you can change glyph order without concern. 

Peter Baker said:

The ability to export everything (except the cvar table—is that an oversight?) into a big XML file helps a little but does not solve the problem.

You can just use fontTools ttx to export VTT content. Before shipping the font from VTT, ttx allows you pull out all of the TSI tables (which can then be merged back as part of your build script). Or if you use ttx after shipping the font, it'll include the cvar table. Either way you get out what you need. 

Mike Duggan

thanks Aaron and Peter. just a few comments. I agree that when using VTT to Autohint, that you have check every glyph and make any necessary adjustments. The Autohinting does however get you you off to a great start, and depending on the font style, (i.e. not too complex, and no cupped serifs , then the editing can be quite fast. Given how long it usually takes to develop font outlines, ie many months or years, I never saw the additional time to hint a font properly, (one - three weeks) to be that much extra on top of that. There is not much point though in any half hearted efforts, the hinting should be done properly for all glyphs, ensuring clear and consistent rendering, properly hinted accents, and consistent height alignments across a family. I see hinting as part of the font design process, as long as it will be needed for lower resolutions. some folks will never see a font printed, and the only experience they will have of the font is reading it onscreen, or as part of some UI interaction. The nice part about VF hinting, is hinting once and making cvt adjustments, and can be achieved quite easily, and for the most part, the hints just flow across the variation space. The fact that the main problems being solved by hinting these days is horizontal stem sharpening, consistent aligments, and clarity of glyphs that have a tendency to close up,, A,E,B,H,Z, etc,  i j, accents clashing with base glyphs, anything with a double bar, (currency) and consistent alignments, makes it all the easier to achieve when you have the power of VTT to correct issues in a simple way. The instruction set is also now quite simple and using just y commands, interpolates, yanchors, yshifts, and some dists with a minimum distance control, the whole font can be elegantly hinted. Agree with Aaron, that if the time is not going to be invested, its better to disable hinting and have full anti-aliasing apply, as this is better than any half hearted or buggy autohinting. This can work fine for Display fonts that are not intended for reading lengthy text. I am sure there are some smarter minds than me, that could come up with or push for a good solution to the GID order issue, which seems to be a bone of contention. 

Peter Baker

Thanks very much, Aaron and Mike. I agree that hinting is blissfully easier now than when I got interested in the subject some twenty years ago, and the time is definitely worth it. This is especially true for variable fonts, which I think are likely for the foreseeable future to be used more online than in print.

What I was doing with VTT, and the basis of my (partly erroneous) claim that it relied on GIDs, was what I think is the obvious thing to do, namely use file --> export --> all code to XML. The exported file identifies glyphs only by GID, and if you add or delete a glyph from the font you've got to fiddle some of those GIDs, plus a bunch of OFFSETs.

I've now looked at vttlib, and it does seem better. The problem Aaron mentions with OFFSETs is daunting. The number of OFFSETs can (and in my case does) run into the thousands: adjusting by hand is out of the question. I suppose one could write scripts to change those GIDs to names and back again to GIDs, but it is a barrier. (Though one can, perhaps, get away with simply deleting all that code that autohinters add to composites. )

Piotr Grochowski

Peter Baker said:

For Open Source, then, ttfautohint is the only game in town,

What do you mean? ttfautohint does not generate any source code, it directly generates the compiled font making it impossible to have an open source font with ttfautohint. Any change to the hinting requires restarting the autohinter, which is equivalent to automated making of software from scratch. Open source hinting would be expected to be able to expand it manually, adding or removing instructions onto the existing instructions, but in case of compiled fonts including ttfautohint output it's impossible. To have manual full hinting (including gasp table, cvt cut in, dropout control, cvt, cvt delta, x link, y link, x interpolate, y interpolate, x delta, y delta), the software must be made from scratch manually for instance in Visual TrueType.

Peter Baker

I mean that ttfautohint is itself Open Source. If you don't like what it's doing to your font, you can get the source code and change it to your liking. As to the code it generates, in the font world the distinction between binary and source is insignificant: it is trivial to extract the code using fontTools, edit it, and put it back. And yet, as I explain above, it is not enough.

Microsoft has done a lot of Open Sourcing (surprising to an old dog like me who remembers their former hostility to that whole scene), and it would be lovely if they'd Open Source VTT, but they haven't chosen to do so (I understand why, but still). It's nice to have the ability to edit the (sort of) high-level VTT Talk (though it's poorly documented, to put it mildly). I've appreciated that a lot when I've used it. Aaron has persuaded me that vttLib partly fixes the major problem of using VTT in an Open Source workflow. Perhaps it could be made to fix that problem entirely, but it's not there yet.

And I still don't understand the legal status of the 148 functions that the VTT autohinter dumps into my font.

Piotr Grochowski

Peter Baker said:

I mean that ttfautohint is itself Open Source. If you don't like what it's doing to your font, you can get the source code and change it to your liking. As to the code it generates, in the font world the distinction between binary and source is insignificant: it is trivial to extract the code using fontTools, edit it, and put it back.

The font being open source (as opposed to no source but still free font) is insignificant for fonts not manually hinted, but significant for manually hinted fonts because it is virtually impossible to edit existing compiled instruction programs.

Peter Baker

The ttfautohint code is rather opaque, mainly because it consists almost entirely of pushes and function calls, and if the functions are documented anywhere I can't find it (the VTT functions, on the other hand, are reasonably well documented, though I don't know how practical it is for users to call them). I am more in favor of replacing any glyph program whose results you don't like than editing it.

And it's really not hard, even if you decided to do it by editing the "assembly" code dumped by ttx. The TrueType instruction language isn't all that difficult, especially now that we hint almost exclusively on the y axis so you don't have to fiddle much with vectors and such: the set of instructions you need to learn is pretty small. I am nobody's idea of a computing genius, but I was writing this stuff back in the early 2000s, when the available tools were worse and/or harder to get than now.

I wouldn't be surprised, actually, if it was possible to use VTT to replace unsatisfactory ttfautohint glyph programs.

Piotr Grochowski

Peter Baker said:

And it's really not hard, even if you decided to do it by editing the "assembly" code dumped by ttx. The TrueType instruction language isn't all that difficult, especially now that we hint almost exclusively on the y axis so you don't have to fiddle much with vectors and such: the set of instructions you need to learn is pretty small. I am nobody's idea of a computing genius, but I was writing this stuff back in the early 2000s, when the available tools were worse and/or harder to get than now.

I wouldn't be surprised, actually, if it was possible to use VTT to replace unsatisfactory ttfautohint glyph programs.

"exclusively on the y axis" Which is what ttfautohint does virtually perfectly. On the other hand full hinting (including both axes and post-IUP deltas) cannot be done in a reasonable way with ttfautohint, hence the manual hinting. That's the point, to use manual hinting to be able to freely access any horizontal instruction as well.

Aaron Bell

Piotr Grochowski said:

"exclusively on the y axis" Which is what ttfautohint does virtually perfectly. 

For anyone not familiar with these tools and hoping to learn from this thread, I can assure you that this statement is certainly not correct. But I've also learned it isn't worth arguing with Piotr, so please just keep in mind that his is not the only viewpoint.

Peter Baker said:

and it would be lovely if they'd Open Source VTT, but they haven't chosen to do so

The core issue is that VTT contains the Windows rasterizer, which cannot be open sourced. I think that once they figure out how to do it, various pieces of VTT will become available for use, but VTT itself won't ever be. 

Peter Baker said:

Perhaps it could be made to fix that problem entirely, but it's not there yet.

(btw, I'm writing a Python library to get around the GID restrictions)

Piotr Grochowski

Aaron Bell said:

Piotr Grochowski said:

"exclusively on the y axis" Which is what ttfautohint does virtually perfectly. 

For anyone not familiar with these tools and hoping to learn from this thread, I can assure you that this statement is certainly not correct.

From my own experience of fontdevelopment, ttfautohint does vertical hinting well enough, and the reason I switched to manual hinting is not only because it is customizable, but because it allows for horizontal hinting as well.

Aaron Bell

@Peter Baker
I wrote a little python library that will resolve the GID issue. You need to pass the old VTT source ttf file, and the new ttf file and it'll sort things out .  

John Hudson

Thanks Aaron! I presume the fix offset script relies on glyph names?

Aaron Bell

@John Hudson That's correct. You need to have a post table version 2 in order for it to work completely correctly. The script uses FontTools' getGlyphOrder() function, which will fall back to cmap if the post table is set to version 3, but that won't account for non-unicode glyphs and I'm not totally sure what'll happen

I need to still add some things—like being able to run from command line more easily, and building in error checks (like the post table for example), but given the expected scenarios, it'll get the job done.

Between this and vttLib, you can fully build a VTT-hinted font from Python!

Peter Baker

Thanks, @Aaron Bell--this is terrific!