A new graphical editor for TrueType hints

Peter Baker

With apologies for a long post, I’d like to invite anyone interested in TrueType hinting to take a look at an app I’ve been working on for the last few months. Called YGT (don’t ask), it’s a graphical editor somewhat like others I’ve used, but addressing my own wants and concerns about existing tools. To wit:

• Written in Python with PyQt6, fontTools, and FreeType-py, it is cross-platform and independent of existing font editors.
• You can either generate a hinted font from inside the program or produce a script that can be run from makefiles and the like. It’s a useful tool for anyone who needs to generate fonts from command-line scripts.
• All hints are stored in a YAML-based format that’s simple and easy to learn. If you edit the source within the YGT program (you don’t usually have to, but you can if you want), it is validated as you type.
• It’s designed to be fast. Most operations are performed with unmodified shortcut keys so that you can work for considerable stretches with one hand on the mouse and one on the keyboard.
• It names rather than numbers such items as control values, functions, and even points, making them easier to work with.
• It has a light footprint, increasing the size of a font file by about a third as much as, say, ttfautohint. Part of the reason for this is that it dumps only a tiny library of functions into a font—but it does encourage users to write their own functions and macros, and these can be inserted via the GUI like other hints.

YGT does considerably less than VTT, but that’s the point, in a way. Contemporary fonts can be hinted much more lightly than they were twenty years ago: YGT is designed around the things that are required now, and both its GUI and its file format are streamlined by concentrating on just those things. But YGT works well with variable fonts: you can create and edit the cvar table and preview the font’s different instances.

Here’s a screenshot of the main window plus a Font View window, used for navigation and also showing which glyphs have been hinted (they’re highlighted in blue). In the main window, the hint editor is on the right (with tools up above), YAML source in the center text pane, and a preview on the left. The editor for this glyph shows the familiar anchors and arrows of other editors, plus a macro call for the glyph’s cupped serif.YGT is far from finished. The preview (by FreeType) is primitive: I need to supply a grid and perhaps an outline, figure out subpixel rendering, and find a better way to feed FreeType a glyph to display (the procedure right now is slow and clunky). There’s no undo/redo yet. No deltas yet. No preliminary autohinting. I need better interfaces for editing control values and other stuff. I’m an amateur programmer, and I suspect that my code is often amateurish and inefficient, with plenty of things needing improvement or tidying

I’d like to hear from anyone who thinks this project is worthwhile. What features do you absolutely have to have in an app like this? What features look superfluous or badly implemented? This is an Open Source project, and I welcome contributions, especially from people who know more about GUI design than I do, or are more familiar than I am with Qt, fontTools, and FreeType. All contributors will be gratefully acknowledged and, where appropriate, added to the list of authors.

YGT is here: https://github.com/psb1558/ygt. It’s not on PyPi yet: installation instructions are at the site. If you find yourself actually using it, save often. This program is young, and while it is way more stable than it was a month ago (I’ve used it for hours at a stretch and hinted 2000+ glyphs with it), I can’t guarantee that it will never crash.

Dave Crossland

Has your work on this been informed by the docs @Mike Duggan has published?

And, is it using any of the vtt code/resources Microsoft has liberated?

Dave Crossland

Regarding undo, I recommend checking out https://github.com/trufont/trufont/pull/614

Peter Baker

@Dave Crossland : I know Mike Duggan's big how-to and his ATypI talk--both very valuable, and with much that can be useful to users of any tool. What I know of seems oriented more towards users than makers.

I'm not using any VTT code, though of course I'm (inevitably) influenced by VTT, both with regard to features I like and those that I reacted against and wanted to do differently. I'm a bit envious of VTT's ability to instantly update its preview (I haven't figured out how to do that) and the way you can drag hints around. The latter would work very differently in a Qt app, which has its own drag-and-drop framework.

Interesting thread about undo in TruFont. Since I'm all in with Qt, I'll probably use the Qt Undo Framework.

Hin-Tak Leung

FWIW, I think I know about "VTT's ability to instantly update its preview" . It is a capability of Microsoft's rasterizer backend... it can't be done with FreeType 2 (with any programming languages), so I am not surprised that you cannot, with freetype-py. Though, please prove me wrong. It will be freetype 3, if that happens.

Peter Baker

@Hin-Tak Leung: Thanks. I thought it would be something like that. I can't find anything in FreeType's API that would allow me to load a set of instructions into a face that's already loaded into memory. I hoped to be able to hack something, but that would require a good bit more knowledge than I've got of what's going on behind the scenes.

Keeping hope alive ...

Dave Crossland

Couldn't you use fontTools to create a new font object, remove the previous instructions, insert the new ones, write a binary to memory, and load that into freetype? I mean, the steps you do already "by hand" and "step by step", just automated and run quickly enough by 2022 CPUs that it appears real time? If you are only doing it for 1 glyph..? 

If not because its too slow, would a Rust replacement to fontTools like https://github.com/googlefonts/fontations be fast enough?

Peter Baker

I tried what you're suggesting--writing a font to memory (with instructions for only a single glyph) and loading it into FreeType from a stream—but couldn't make it work. I should spend more time on it, since FreeType is supposedly happy to load a font from a stream (and I've gotten more comfortable working with FreeType-py since my first attempt).

Right now I'm writing a font to a temp file and reading that in again. I could also try writing it to a buffered temp file and see how much of a speed gain is there. (I am experimenting with a font that has 4000+ glyphs, so it is sort of a worst case in terms of save and loading times.)

What would be extremely cool would be to generate a font containing a single glyph, its instructions, and only those tables needed for support of those instructions (prep, fpgm, maxp), but I haven't discovered a way to do that (yet).

I'm sorry to say that I don't know fontations. I'll have a look.

On the bright side, I've got subpixel rendering turned on, and one can choose between VTT style, where the RGB components are combined into square blocks of uniform color, or the more accurate (but harder to make sense of) RGB stripes:

jeremy tribby

very nice. was easy to build locally.

do you see a future where this might support UFO, and interpolate hints for the relevant instances in the designspace from the masters?

c.g.

Windows user here. I cloned the repository and installed in editable mode.

When I launch ygt, I get the following error:

ModuleNotFoundError: No module named 'fs'

Once installed 'fs' with pip, I launch ygt 

(ygt-venv) D:\ftCLI\ygt>ygt
C:\Users\Cesare/.ygt/ygt_config.yaml
Exception in open_config:
[Errno 2] No such file or directory: 'C:\\Users\\Cesare/.ygt/ygt_config.yaml'
Traceback (most recent call last):
  File "D:\ftCLI\ygt\src\ygt\window.py", line 639, in open_file
    os.chdir(os.path.split(f[0])[0])
OSError: [WinError 123] La sintassi del nome del file, della directory o del volume non Þ corretta: ''

Maybe you should change these lines using os.path.join()?

config_path = os.path.expanduser('~/.ygt/ygt_config.yaml')

config_dir = os.path.expanduser('~/.ygt/')

Peter Baker

I'll borrow my son's Windows machine and see what I can figure out. I'll open an issue on GitHub and report on the fix(es) there.

Peter Baker

@jeremy tribby: It would be easy to add the ability to read a UFO, but (last I noticed, anyway), UFO doesn't know how to store TrueType hints.

Ygt handles hints for variable fonts now, but setting up the cvar table takes some work (it's covered sketchily in my documentation). I'm hoping to automate as much of this setup as possible, somewhere down the road.

Hin-Tak Leung

@Peter Baker what you try to do - load font, drop all its hinting instructions, write a new set, is exactly one of the operating mode of ttfautohint. So you might want to just study that to see how it is done, and whether you can do better, in terms of offering better usability, for example.

Out of all the programming languages that I tried to do something similar to
 "VTT's ability to instantly update its preview", Google's own go/golang is the hardest. Good you are not trying to write a font editor with golang. 😀

Jens Kutilek

Peter Baker said:

@jeremy tribby: It would be easy to add the ability to read a UFO, but (last I noticed, anyway), UFO doesn't know how to store TrueType hints.

UFO does support TrueType hints, but only in ttx assembly notation, see:

https://unifiedfontobject.org/versions/ufo3/lib.plist/#publictruetypeinstructions
https://unifiedfontobject.org/versions/ufo3/glyphs/glif/#publictruetypeinstructions

You would need to store your custom hinting language in private font/glyph lib keys. And AFAIK the only font building tool that supports compiling the UFO ttx code is my fork of ufo2ft.

Jens Kutilek

I must say, xgridfit is quite ... verbose  This:

<command name="MIRP">
             <modifier type="rp0" value="yes"/>
             <modifier type="minimum-distance" value="no"/>
             <modifier type="round" value="no"/>
             <modifier type="color" value="black"/>
</command>

is equivalent to this in VTT assembly:

MIRP[M<rBl]

You might consider using VTT assembly via vttLib instead of xgridfit as a backend for your editor.

Peter Baker

@Hin-Tak Leung: Yes, it's got to be worthwhile to see what ttfautohint is doing (if only because it's doing it so fast!). I'll check it out. As for go/golang: I'm always glad of an excuse not to learn another language.

@Jens Kutilek: Thanks for these observations. In the end I also produce code in TTX assembler format (since that's what fontTools likes). So building in the ability to work with UFO might be worthwhile. Probably straightforward too.

As to Xgridfit: Sigh. I get it. I must be the only person in the world who would've made that choice. I could go on about why it's actually a decent choice (for an app written by yours truly, anyway), but will only say: at least it works, and since it is a backend, it is (mostly) out of sight. Most people will never have to deal with that verbosity.

Peter Baker

Since the issue of instantaneous (or at least quick) previews has gotten some attention, I’ll report on progress there. The original implementation (re)read the font from disk so as to get a clean copy, compiled the TrueType instructions for the current glyph and inserted them into the font, saved the font to a temporary file, and read that file and displayed the hinted glyph. Whew! It could take up to two seconds for a biggish (ca. 4900 glyphs) font on a reasonably fast machine. On the old slow machine where I run Linux, it could take five seconds.

The new implementation keeps a clean copy of the font (via fontTools) in memory to avoid that first disk access. When a preview is called for, it makes a deep copy of the font, inserts the instructions, has fontTools subset the font (dropping all OT features and all but one glyph), saves that to a spooled temporary file (again avoiding a disk access), and reads and displays the tiny font. (I would have guessed that the subsetting would cost instead of save time, but it doesn’t.) The whole operation (for a biggish font) takes about 0.66 seconds. For a font with about 1000 glyphs, it takes about 0.40 seconds.

Finally, all of this happens in its own thread so as to avoid blocking the GUI. With these changes, I feel comfortable updating the preview whenever any change takes place in the editing pane. The result is not quite an instantaneous update of the preview pane, but it feels more or less like that. Half a second isn’t a lot.

This change is not in the repository yet: I need to test more before I post it, and take care of a few details.

Dave Crossland

Excellent!!

Peter Baker

Here are the substantive developments (excluding bug fixes and other minor changes) since I last posted a couple of weeks ago.

File handling:

Ygt can now have several files open, in separate windows.

“Export font” (which can take a couple of minutes for a very large font) now runs in a background thread, with an indicator to show it is running.

Preferences:

Early versions kept preferences in a YAML file on Unix-like systems; now Ygt saves preferences in the Windows registry as well.

Preview pane:

Preview is now updated automatically; but you can turn this off if it’s too laggy.

Hinting can now be toggled on and off in the preview pane.

For variable fonts: flip through instances with Shift-Left and Shift-Right.

Editing:

Supplementing the choice between “round” and “don’t round,” Ygt can now use TrueType’s rounding types (to-half-grid, down-to-grid, etc.). Not yet via the GUI.

You can choose the initial round state for each kind of hint, e.g. if you want rounding off by default for stem hints. Not yet via the GUI.

Control values:

Ygt guesses at eleven control values when a font is opened for the first time (this was broken for a time and is fixed now).

Now control values can be linked, somewhat in the manner of VTT’s “inheritance.” This is done via a dialog box, and Ygt generates code for the prep table.

Select a hint and type a question mark to make Ygt guess the correct control value.

Ygt now uses Unicode character categories to filter control values. It can also filter by suffix (e.g. “smcp” or “pcap”). If a font has a large number of control values, this makes them easier to navigate.

Documentation:

Documentation is still sketchy, but has been updated to reflect (what I understand to be) current best practices in hinting.

Major to-dos:

Autohinting! I hardly know where to start. I’ve begun to study how other Open-Source programs do it.

Undo/Redo. Still on course to use the Qt Undo Framework.

Peter Baker

Another progress report:

I’ve fixed up the preview pane, adding a grid (which can be toggled off and on) with baseline. The last major to-do for this main preview will to add an outline (via FreeType’s stroker).

Also added a smaller preview pane beneath the main one: by default this shows the current glyph in an array of sizes (from 10 to 100 ppem, or as much as fits). For variable fonts, this pane always shows the same instance as the main preview. If you click on any character, the main preview will change to that size.

You can type a short text in the “Text” box to see glyphs in context (the size and instance always the same as in the main preview). Click on any letter to jump to that glyph in the editing pane.

This preview doesn’t yet use kerning (that will come soon). An additional plan is to use Harfbuzz to apply OT features so that any glyph in the font can be displayed in this pane.

John Hudson

Here’s a suggestion for the size range preview: when the glyph is zero-width, as in the case of combining marks, please insert spaces between the glyphs in the preview, and before the first one so that it isn’t clipped. In VTT, it is a persistent frustration that preview mark glyph bitmaps end up piled on top of each other and clipped at the beginning of the line (yes, I have reported that to the developer).

Peter Baker

@John Hudson: Thanks, I'll do that. I think I should be able to center each mark in its own custom space, with margins of two or three pixels, converting it into a spacing character for the purpose of display. That will accommodate wide marks like U+0305 combining overline, which is (usually) wider than a space.

Will also need to adjust the vertical position much of the time.

Peter Baker

The problem with the size range and zero-width characters is fixed now.

Peter Baker

The app now has undo/redo for all glyph editing commands (editing of font-wide data is not yet covered). As I mentioned earlier, I'm using the Qt Undo Framework, which makes it possible to do fairly sophisticated things without a huge amount of work.

Each glyph has its own undo/redo stack, so you never have an undo or redo happening to another glyph than the one you're looking at: I think this is the most intuitive way to handle it. Each undo/redo command also has a description, so you can tell pretty precisely what effect the undo or redo is going to have.

Thomas Phinney

You will eventually have font-wide data, so you will need to think about how that interacts with your glyph-specific undo stacks! fun, fun.

Peter Baker

Glyphs appears to run a separate stack for Font Info, available only when the Font Info window has focus. I suspect I'll do something like that.

Peter Baker

I'm still tidying up after the undo/redo surgery, but meanwhile thinking about the next step, namely making a GUI for whatever font-wide data can be handled through a GUI: cvt, cvar, certain defaults, and (to some extent) prep.

But first I'd like to check up on my understanding of the cvar table, since I'm pretty sure I've misunderstood it in the past.

I have no problem with the lists of deltas, but my grasp on regions is a bit shaky. Here's what I think I know:

Of the three values (in the spec, peakTuple, intermediateStartTuple, intermediateEndTuple) associated with a tag, only peakTuple is needed in cvar, and the others are ignored. To generate the cvar table I use fontTools, which seems to want all three values in its TupleVariation class, with peakTuple or "value" the second of them. Should the other two be zero or None? E.g. for a condensed bold:

TupleVariation({'wght': (0.0, 1.0, 0.0), wdth: (0.0, -1.0, 0.0)}, [cv deltas ...])
TupleVariation({'wght': (None, 1.0, None), wdth: (None, -1.0, None)}, [cv deltas ...])

The spec suggests that the normalized values should always be -1.0 for the bottom of the range (e.g. an opsz of 6) and 1.0 for the top (opsz 24, with zero for the default (opsz 12). Is this true even for my hypothetical, where the minimum value is twice as far from the default as the maximum (12-6 vs. 24-12)? That is, should the TupleVariation for min  and max opsz be like these:

TupleVariation({'opsz': (0.0, -0.5, 0.0)}, [cv deltas ...])
TupleVariation({'opsz': (0.0, 1.0, 0.0)}, [cv deltas ...])

or like these:

TupleVariation({'opsz': (0.0, -1.0, 0.0)}, [cv deltas ...])
TupleVariation({'opsz': (0.0, 1.0, 0.0)}, [cv deltas ...])

I think these are my only questions for now. Thanks in advance for any help!

John Hudson

The spec suggests that the normalized values should always be -1.0 for the bottom of the range (e.g. an opsz of 6) and 1.0 for the top (opsz 24, with zero for the default (opsz 12). Is this true even for my hypothetical, where the minimum value is twice as far from the default as the maximum (12-6 vs. 24-12)?

If I understand the question correctly, yes. The normalised extremes are always -1.0 and 1.0, regardless of the non-normalised axis scale distances on either side of the default instance location. Think of the normalisation as being applied separately on either side of the default location.

I can’t remember, though, how data should be recorded if the default location is at the axis extreme, e.g. if your 6pt opsz extreme is also the default instance.

Thomas Phinney

Isn’t that then just 0 and 1.0 in the normalized version?
If the default instance is at one end of the axis, you only need to go one direction.

John Hudson

Yes, that is technically the case: I was wondering about various table data and whether the unused axis extreme data is simply omitted or somehow indicated as being null.

Peter Baker

Thanks, John and Thomas. This is helpful. I suppose one could try it both ways and see if it causes problems.