A new graphical editor for TrueType hints

Peter Baker

Ygt can now save its source to a UFO and export compiled TrueType instructions (and related data) to a UFO (conforming to the UFO 3 spec). "Save As" lets you move back and forth between the two modes: storing source in and exporting to a UFO or saving source in separate .yaml files and exporting to .ttf fonts.

Now: can fontmake or any other program actually make use of TT hints stored in a UFO?

Jens Kutilek

Peter Baker said:

Now: can fontmake or any other program actually make use of TT hints stored in a UFO?

ufo2ft, which is used by fontmake, currently compiles TT hints from a UFO if you use my special ufo2ft fork. I've tried to get this merged into the main branch, but interest seems low. Maybe if anyone interested could comment on the pull request, it would add some traction? I've been using this to compile hinting for variable fonts for some years now, so it seems pretty stable.

Peter Baker

I'm a ufo2ft enthusiast, and will definitely look at your fork.

Peter Baker

I thought I would try to get some comment on the interface for editing CVT-related stuff before it gets set in stone.

The easiest way to add a CV is to select one or two points and either click the “CV” button on the toolbar or type “C.” This brings up a dialog with most of the most important info already filled in:

All you need to do is supply a name (since ygt insists that all CVs be named). “Type” (“pos” or “dist”) and “axis” (“x” or “y”) are also required. These are filters for CVs, as are optional “cat” (a Unicode category) and “suffix” (a glyph-name suffix, like “sc”). With these filters in play, CVs are available only when certain conditions are met—for example, when the “cat” is “Letter, Uppercase,” the CV is available only when an uppercase letter is being edited. Optional “color” is the type of distance the CV controls (black, white, gray).

On the “Same As” tab, you can specify that the new CV should be set equal to another CV below or above a certain ppem (Query: is there any use for the “above” option? I’ve never found a use for it myself):

Finally, on the “Variants” tab (for variable fonts only), you can specify variant CVs for each of the font’s masters. By “masters” I don’t mean the masters used to interpolate glyph outlines (though these will often match those), but rather the ones defined in the cvar table (I have never come across a friendly name for these, so I call them “masters” at the risk of some confusion). Ygt generates a suggested collection of masters when it first loads a variable font, but it is up to the user to supply the values:

To edit CVs and masters, summon the “Control Value” window. This is a top-level window that can be left open while editing hints in the main window:

This one shows the CVs automatically generated by ygt on loading a new font. The panel on the right is the same as in the “Make CV” dialog; then there’s a “Source” tab for those who want to edit ygt’s YAML source directly (too boring to illustrate), and a “Masters” tab:

For each master you can supply a friendly name (soon the program will take friendly names, where available, from the fvar table) and normalized values (-1.0 to 1.0) for any of the font’s axes. You can also add or delete masters in this tab.

When hints are generated, information from the “Variants” and “Masters” tabs is assembled to create the cvar table.

There’s still a good bit of testing to do before this stuff is ready for prime time, and not all the widgets are completely hooked up. The CV editing will have its own undo stack, but this is not yet functional. In short, there’s still plenty of opportunity to make changes here, so I welcome feedback.

Paul van der Laan

Peter Baker said:

Over the years I found that control over CVT values is one of the most important aspects in TT hinting. Especially the ability to edit them, or to add new ones at a later stage. This is one of VTT’s weak points and it’s great to see this initiative.

Looking at your proposal I would consider implementing static character groups next to the dinamic filters you're currently offering. Groups of glyphs that users are free to define themselves similar to what VTT does but then a lot easier to deal with.

Suppose I would have a basic Latin lowercase a-z with multiple stylistic variants of a particular glyph (e.g. /a.alt01 /a.alt02 /a.alt03): it does not seem easy to make one CVT for that now.

With regards to "color" I find the terminology confusing here. I would expect it to be the actual color of that CVT when visualised in the editor. (Which is definitely a nice feature to have BTW.) I would rather name this option "distance type" instead.

These are my 2c. Thanks!

Peter Baker

Thanks for the comment! Character groups (or classes) sounds like a good idea: I'll work on it. And you're right that "distance type" seems less confusing: I'll change that.

Thomas Phinney

Some options for a more table-like view of some of these things might be good.
The CVT in particular seems like a natural for a table rather than a straight dialog box. Seeing all those heights and overshoots at once is good.

Jens Kutilek

If it were possible to keep the information from which points or pairs of points in glyphs a CVT is derived, you would get the CVTs for compatible fonts (i.e. other variable font masters) "for free" and could build the cvar data from that.

Peter Baker

@Thomas Phinney: Thanks. I agree that a table view would be convenient. I'll study this and perhaps post a drawing before doing it.

@Jens Kutilek: FreeType will produce an outline for any set of coordinates in a variable font, so in theory it should be easy to auto-fill-in the cvar table--at least for those CVs that the program auto-generates or that are made from points. Unfortunately, I haven't yet figured out how to access the point data in FreeType. I'm also studying fontTools.varLib to see if it can be used for this purpose. Other tools I should be looking at?

Peter Baker

There have been various little developments since I posted a progress report a couple of weeks ago, and one big one, namely that ygt now automatically generates values for the cvar table, per the suggestion above by Jens Kutilek (thanks, Jens!).

It’s been the case for a long time that when ygt first opens a font, it generates a list of twelve control values (xheight, xheight-overshoot, etc.)—mainly by measuring some key glyphs. More recently, when it first opens a variable font it generates a list of “masters”—that is, those sections of the cvar table (usually, but not always, matching the masters you produce in designing the font) containing deltas for control values.

Now, when you first open a variable font, it also keeps a record of which points the generated CVs were based on, generates an instance for each master (using the amazing fontTools instancer), and compares its recorded points with those in the instance.

Also, when you add a CV with the “Add Control Value” dialog, the program records point data for that CV, and you can refresh the cvar data (taking account of any new CVs) at any time by clicking the “Generate Variant Control Values” button in the “Masters” tab of the “Control Values” window.

This is rather time-consuming: for a font with about 1400 glyphs and four axes (six masters), it takes about six seconds on a Mac with an M1 Pro chip. I suppose most of that time is the fontTools instancer generating instances (since ygt really isn’t doing all that much work). Fortunately, this doesn’t have to be done very often.

You can still do all this manually, of course, and it’s advisable to check over what the program does for you.

John Hudson

It’s been the case for a long time that when ygt first opens a font, it generates a list of twelve control values (xheight, xheight-overshoot, etc.)—mainly by measuring some key glyphs.

Will it be possible to extend this list of automated control values for e.g. hinting different scripts? It would be great to be able to define a set of control values for Devanagari, for example, and the key glyphs from which they can be derived. One of the benefits of TT hinting, versus pre-CFF2 PS-style hinting, is being able to define script specific control values. Being able to automate some of that would be nice.

Peter Baker

@John Hudson: I'd be very interested in this, but I'd need some help figuring out how to do it. I know less than I should about non-Latin scripts, and I don't know where to look for key values.

John Hudson

I was thinking in terms of an extensible system. The value of automating some basic control values is that a font maker doesn’t have to go and manually measure things every time they start a hinting project. So you have a set of Latin control values baked in. I presume the code for that relies on you somewhere identifying some key glyphs, and somehow identifying what parts of those glyphs to measure. The first part of that seems easy to generalise to a customisable set of glyphs; the second part seems harder.

I know about a lot of scripts, but far from all, and I have only hinted a handful. Let’s say a ygt user is someone who spends most time making fonts for Makasar script—chosen at random as something I don’t know a lot about, and that I am guessing nor do you—: I am imagining a way for that user to be able to define the key glyphs and measurement points for that script, and store these as a preference with ygt, so that it isn’t necessary to do control value measurements manually for every new Makasar script. The system should be flexible enough that you don’t actually need to know about Makasar.

John Hudson

Thinking about this some more: definition of measurement points for alignment zones and stem distances, could be defined using a fairly simple syntax if you have some kind of stem detection system. The syntax could be relative to cartesian space, so that you would be able to identify e.g. the point closest to y0 on key glyph/H, and point closest to y0 on key glyph /O, and use those to calculate the baseline overshoot alignment zone control value. Similarly, with stem detection one would be able to specify e.g. the left-most vertical stem in a key glyph, or the top-most vertical stem, etc. They would likely provide a flexible enough topographical system to capture arbitrary measurement points on arbitrary key glyphs for most purposes.

Peter Baker

Thanks for these thoughts. I believe a stem-detection system may be within reach, given the existing literature on the subject (e.g. the description of the FreeType 2 autohinter as it was under development back in the year 2000). I haven't thought much farther than this yet, but I would like to target an extensible system like the one you are envisioning.

I can only hope that my skills are up to the task or that I can recruit someone who has got the kind of mathematical skills I lack.

John Hudson

I should clarify that I think this kind of CVT automation for other scripts is something that would be very nice to have, and would address a long imbalance in the way TTF hinting tools have tended to handle ‘non-Latin’ scripts, but it isn’t what I would consider core functionality—after all, one can do the measurements manually, and so long as one can label control values in meaningful-to-the-user ways, that is a good baseline functionality.

Jens Kutilek

Jens Kutilek said:

Peter Baker said:

Now: can fontmake or any other program actually make use of TT hints stored in a UFO?

ufo2ft, which is used by fontmake, currently compiles TT hints from a UFO if you use my special ufo2ft fork. I've tried to get this merged into the main branch, but interest seems low. Maybe if anyone interested could comment on the pull request, it would add some traction? I've been using this to compile hinting for variable fonts for some years now, so it seems pretty stable.

I'm happy to report that TT instruction support has finally landed in ufo2ft 2.31, released a couple of days ago.

Peter Baker

Congratulations, @Jens Kutilek! And thanks for making such a useful tool. I'll use this to make sure my program is producing valid hints and storing them properly in UFOs, and I'll also include info about this in my how-to materials.

Peter Baker

A few more developments:

The "Control Value" editing window has become "Font Info" (on Ctrl/Cmd-I as in Glyphs): in addition to all the CV stuff, it allows editing of other font-wide defaults, including the initial rounding of any type of hint.

You can also add CV deltas via a little table:

For "Distance" you can use a "distance/unit" notation, as with raw TrueType instructions, or an integer or float (in which case the program will substitute the nearest valid value).

All edits (both at glyph and font level) are now in the "Undo" system.

I've still got a number of features to add (including things proposed here), but I've decided the project is mature enough to put on PyPi: so make sure you have Python 3.10.4 or higher and install with 'pip install ygt'. I also hope to make executables for Mac, Windows, and Linux, but there's a problem with freetype-py and pyinstaller (the program that makes executables out of Python apps). There's a fix for this in the freetype-py repository, and I've asked the maintainers of that package to make a release including the fix.

I've decided to pause adding new features for a while to concentrate on testing and stability. I still welcome suggestions, but it may take me a while to get around to them.

Peter Baker

This post is a request for help thinking through problems associated with hinting in UFOs. (Apologies in advance for its length: the problems are complex.)

On one level this is fairly easy: given a quadratic outline (which ufo2ft will produce if a UFO has CFF outlines), Ygt can hint it. It stores its own data in the UFO’s data directory, and when it compiles that data to TrueType instructions, it knows how to store the compiled instructions in the UFO, following the UFO3 spec.

Ygt is fairly tolerant of changes in a glyph’s outline, since it can refer to on-curve points by their coordinates. It doesn’t matter how points are indexed: as long as the points stay in place (and even if they’ve moved by a few font units), Ygt will find them. So it doesn’t matter that, say, Glyphs and fontmake number points differently.

But as soon as Ygt compiles its hints, no changes can be tolerated: the outlines it hints have got to be precisely the same as the outlines in the final TrueType font. Order of contours, outline origins, number of points, and all coordinates have got to be the same. To enforce this requirement, Ygt creates a hash for the glyph it is hinting and stores the hash with its compiled instructions. When ufo2ft (with Jens Kutilek’s amazing patch) compiles these instructions to binary form, it generates a hash for the glyph in the font it is creating and compares it to the one stored in the UFO: if they don’t match, no instructions are compiled.

The problem, then, is guaranteeing that the outlines to which Ygt applies its hints are precisely the same as those in the UFO when Ygt generated its instructions. They frequently are not: the tools that fix up the UFO tend to apply transformations like removing overlap and correcting path direction not in the UFO itself but in the process of generating a TrueType font from the UFO. In the sort of workflow I use, instructions based on the outlines in a UFO will never be valid when the font is generated.

So my question for those who are interested in storing TrueType hints in the UFO is this: what kind of workflow do you envision for hinting UFOs, and what capabilities can Ygt provide that will make it work?

Things I can imagine include 1.) making a new layer in the UFO from a TrueType font and basing the hints on that layer (seems a bit, um, circular); 2.) letting Ygt create a new layer in the UFO and add glyphs with valid quadratic outlines to it (still hard to guarantee that the outlines in the UFO will match those in the generated font); 3.) explaining the problems and constraints in the documentation and informing users that they’re on their own (kind of unhelpful).

But I’m guessing that people who work with UFOs more than I do will have better ideas than these.

Denis Moyogo Jacquerye

The workflow with vttLib is to compile the UFO to TTF, use vttLib to inject VTT tables or instructions generated from them in the TTF if they are in the UFO lib, edit the TTF with VTT, use vttLib to extract the VTT tables in the UFO lib, then repeat for modifications or additions. For release, the VTT Tables are not included, just the generated instructions.

The assumption is that the cubic to quadratic conversion produces the same outlines whether it is for the TTF used to edit the VTT tables in VTT or for the TTF used for final production.

It sounds like Ygt could use TTF with the same cubic to quadratic conversion as the final production TTF to get the outlines.

The main differences would be in the glyphset, as not all glyphs are exported and glyph names may change, but that should be manageable with the public.skipExportGlyphs lib key and public.postscriptNames.

It may not be necessary to store the outlines in the UFO in a new layer, the TTF would be enough while working in Ygt and the hash can be stored in the UFO.

Jens Kutilek

So if I understand Denis’ workflow correctly, the TrueType UFO is never kept around, but reconverted from the cubic UFO if needed.

FWIW, our workflow is somewhat different. We convert multiple master VFBs to quadratic curves in FontLab 5, and hint them using the FontLab 5 TrueType hinting tools. Then we export interpolatable instances to UFO from there, which are later interpolated or used to produce a variable font.

The FontLab hints reference UFO points by a name. We resolve those to point indices when compiling the FontLab hints to VTT assembly (with our custom tools).

For Ygt, something similar may be possible. The UFO format also defines point identifiers, which should never change ("must not be unnecessarily removed or changed"). They may provide a way for Ygt to identify hinted points in the high-level hint code even if the contour has been reversed or overlaps have been removed. Only when compiling to low-level ttx assembly code or xgridfit code, and then storing the glyph hash to mark the glyph and hinting as synced, the identifiers would need to be replaced by point indices in that low-level code.

Jens Kutilek

For illustration, if you add a vertical "Align" high-level command in FontLab 5, its point will get the name "av01"; if you then define a vertical stem distance by using the "Single Link" command, it will add a new name "sv01" to the target point, and the "Single Link" will store "av01" as its reference point and "sv01" as its target point, etc.

If a hinted point already has a name, it will be reused.

Peter Baker

Thanks for these thoughtful remarks, Denis and Jens. If I understand correctly, Denis's workflow relies on always using the same tool for generating the TTF so that one will always get the same point numbers. Jens's relies upon FontLab's hinting system using a more stable system for identifying points than the native TrueType point indexing (the Glyphs TT hinter is also very good at tracking points despite reordering, removing overlaps, etc., but I don't know the details of how it does that). What they've got in common is that point indexes are locked in when whatever high-level code the system is using gets compiled to TrueType assembly.

This is also true for Ygt, which is able to be agnostic about point indexes up to the point where it generates TT assembly.

Thanks, Jens, for mentioning UFO point identifiers. I hadn't thought about using them, but I'll be thinking about them now.

I just did a simple experiment, adding a point identifier to a .glif in a master UFO generated by glyphsLib (via fontmake), and then using fontmake to generate instances. The identifier remains on the correct point in all instance UFOs--which is promising for a future project of automatically generating hinting for all static instances. Fontmake only removes overlaps *after* the instance UFOs are generated, so I can't tell whether the identifier would survive that or similar operations (in fact, *is* there even a tool that removes overlaps in UFOs?). The identifier certainly wouldn't survive a round-trip from UFO to TTF and back—would it?

Peter Baker

Ygt now has a "merge mode," in which it adds its own hints to those already in a font. So if you use, say, ttfautohint and are largely satisfied with the result but want to hint a few glyphs manually, Ygt can replace the programming for just those glyphs. It does this by appending its own control values, functions, etc., to those already in the font. You can choose whether to replace the font's CVT program or append Ygt's CVT program to the existing one.

Also, there are now executable files for M1 Mac and 64-bit Windows-x86. If you can run these, there is no need for a python virtual environment, and all the dependencies are bundled in the one file. I think I can make Linux and Mac-Intel versions within the next few days.

Lot of small fixes and optimizations, but these are the major developments.

pthomas505

Thank you for making this, and also xgridfit! I have been trying it out on Ubuntu. Would it be possible to add a render mode option to turn off antialiasing completely? Also, is there a way to remove a previously added hint without undoing back to when it was made?

Peter Baker

Thanks for the comment. FreeType has a "mono" render mode, which would be not-antialiased, so I can add that. To delete any hint, click on it to select (selected hints are darker) and type the delete key (dependent hints will also be deleted).

pthomas505

Thank you!

pthomas505

I have been trying out the compiled version of YGT 0.2.4 for Linux. I think something is wrong with my Python or pip install; I haven't been able to install it from pip yet.

For some reason I seem to be getting the error message "Failed to resolve point identifier None in glyph A. Substituting zero." Is there any chance that you might know what could be causing this? It seems to happen when I change to horizontal hinting. The yaml file is attached (I had to change the extension from .yaml to .txt to upload it) and the TTF I am using is here: https://github.com/pthomas505/fonts/tree/main/fonts/family_name/monospace/size1/design_2/ttf

The links seen in the screenshot changed from what they were before the error occurred.

Thank you.

pthomas505

I am also getting a lot of these messages on the console: "FFTM NOT subset; don't know how to subset; dropped".