TrueType bytecode interpretation, ClearType: anything critical happening after IUP[x] and IUP[y]?

Nikolaus Waxweiler

Hey everyone,
so I'm currently working on FreeType's bytecode interpreter to shuffle it into "ClearType mode" so to speak. I hope someone deeply knowledgeable about or experienced with TrueType hinting reads this.

What actually needs implementing is a backwards compatibility mode as outlined by Greg Hitchcock and Beat Stamm. Now, FreeType is not the Windows interpreter/rasterizer and doesn't have to care about a few things that Microsoft has to, e.g. compatible widths, rasterizer peculiarities and things pertaining to Windows internals and legacy.

I want to implement a basic compatibility mode without a lot of special treatment of legacy fonts like Arial, Times N. R., Courier (and DejaVu *), etc. Actually, I don't want to handle any internal white/black lists for fonts and their hacks. I'm mainly interested in rendering modern (web)fonts well, I'm happy when legacy fonts render mostly fine. Tricky fonts that depend on the interpreter to render correctly (mostly Asian fonts like DFKai) and native ClearType fonts are left alone and can do whatever they want. Most fonts in circulation will be handled by this compatibility mode as there are very few native ClearType fonts like Constantia, Corbel and Sitka.

I have so far implemented just a few of Hitchcock's and Stamm's basic recommendations (block post-IUP DELTAP/SHPIX, be selective about pre-IUPs without detecting functions, turn all Round_* into Round_None on the x-axis) and a few other things. While experimenting with killing post-IUP DELTAPs, I had an idea: what if I simply stop execution after IUP[x] and IUP[y]? AFAIK, everything that happens afterwards is an attempt at superhinting which is not useful in a ClearType environment. I found that this approach saved me a few extra lines of backwards compatibility hacks and so far, I could not find anything that blows up. Microsoft's core web fonts render fine, as do all the webfonts I came across so far. I think leaving glyphs alone after IUP[x+y] is also considered good practice?

So, my question is: am I missing anything or is it fine to stop execution after IUP[x] and IUP[y]? Does anybody know of any fonts that would or could possibly break?

Georg Seifert

Do you mean ClearType/ClearType or ClearType/DirectWrite. With or without oversampling in y-direction?

Thomas Phinney

That's exactly what I was wondering about. If you are not oversampling in Y, then there may still be utility from post-IUP deltas in Y.

Nikolaus Waxweiler

Georg, you mean "ClearType/ClearType" as in GDI rendering without Y-smoothing? DirectWrite then, FreeType's grayscale rasterizer is in symmetrical smoothing mode 24/7 with 256 shades of gray. No Y-oversampling right now, FT can't do it the way DirectWrite can, but that's on my list of interest because I hope it would "solve" the problem of hint designers snapping stuff to full vertical pixels. At least for a light(er) hinting mode.

Thomas, do you have any examples of fonts that benefit from that?

Werner Lemberg told me to keep executing instructions but prevent anything that would change the rasterization result, as a font might try to change the cvt or stuff like that. So my question is partly answered, yet my brain hungers for more information!

PS: Stamm's paper says that post-IUP deltas are blocked. Is that false? Hitchcock doesn't actually mention them outside an explanatory footnote.

Deleted Account

l have never seen anything but deltap and comments after iup. by rule, anything that occurs after that is probably destructive to the shape, or should be in another table.

That doesn't say it is humanly possible to say for sure.  

Georg Seifert

I Glyphs you can add instructions to diagonals. They are not diagonal stem instruction as you would do for B/W rasterizer but should adjust the thickness of the strokes to the horizontal stems. Horizontal strokes will be thinner before a pixel jump and thinker after. The vertical strokes scale proportionally just like the diagonal. But the diagonals have a horizontal component that should follow the jumpiness of the horizontals. I’m not sure if that is clear, I’ll try to make a sample image. all that is implemented with regulate instructions and not deltas so if you just block the deltas I’m fine. 

But if you change the handling of TrueType instructions, why not align it with the more recent developments and go for symmetrical oversampling?

Jens Kutilek

If you only instruct the vertical direction, do you need to add/execute IUP[x] at all?

Georg Seifert

> If you only instruct the vertical direction, do you need to add/execute IUP[x] at all?

Don’t think so.

Nikolaus Waxweiler

> That doesn't say it is humanly possible to say for sure.

That's what I think, but I guess asking is better than assuming My current code tries to freeze the outline post-IUP but executes all instructions.

> I’m not sure if that is clear, I’ll try to make a sample image. all that is implemented with regulate instructions and not deltas so if you just block the deltas I’m fine.

I think I get the idea, however my current code completely blocks x-movement of points (unless Glyphs generates native ClearType fonts, those can do whatever they want). Can you provide a sample font along with the image so I can experiment?

> But if you change the handling of TrueType instructions, why not align it with the more recent developments and go for symmetrical oversampling?

I'd love to and in fact Werner told me that it would make a backwards compatibility mode superfluous, but apparently it would amount to a rewrite of FreeType or something. I personally have no idea why, but he knows the code much better than me, so yeah.

PS: Regarding post-IUP sheningans: Arial Regular uses "flip range on" on 'D' and 'G' at a few sizes post-IUP to turn the curves into coarse lines. Sneaky. Blocked.

Severin Meyer

As an example of the note by Georg Seifert, see the DejaVu fonts (v2.35).
 
They adjust the thickness of diagonal stems after IUP[x] and IUP[y],  because they do so by re-aligning the freedom and projection vectors to diagonal stems with SFVTL/SPVTL, and then moving points with MIRP.
 
Therefore, if you execute MIRP before IUP but not after, you might end up with inconsistent stems.

Nikolaus Waxweiler

Hm. I see the problem, but can't think of a font-agnostic solution right now that wouldn't break something else. Except oversampling. Will ponder.

Edit: Oh, I just did. Allow diagonal x-moves only post-IUP while blocking DELTAP/SHPIX. Didn't break any core web fonts. Will test some more...
Edit 2: Hm, it only seems to matter to 'Z' and 'z' so far. Will compare with rendering on Win10 later.

Jason Campbell

In the Win rasterizers, I believe that only DELTAPs are not used, but SHPIX are executed. Not that I can recall a case where I've ever used a post IUP SHPIX on purpose, but theoretically it's possible. 

Belleve Invis

@Jason Campbell 
Delta instructions in CT will be effective when a point is touched. sfdhanautohint's output depend on this behavior.

Georg Seifert

I'm just working on improved instruction code that doesn't force full pixel stems if y-oversampling is available. That would make the adjustment of diagonal stems superfluous and will be disabled. This depends on what the getInfo instruction returnes. 

Nikolaus Waxweiler

@Belleve Invis My code will honor DELTAP/SHPIXes on y-touched points. Do you happen to have a test font on hand that was hinted using the latest version?

@Georg Seifert Great, my code currently always returns interpreter version 38, bit 15 (subpixel hinting), bit 17 (subpixel positioning) and bit 18 (symmetrical smoothing). Will that make your code think y-oversampling is active? If not, what is it expecting? I'm asking because FT will by default render bitmaps that look like they have been oversampled (256 grays, symmetrical smoothing), so clamping outlines to full pixels isn't really necessary.

Oh, and since there are people involved in hinting in here, is your code doing "#PUSH 4,3; INSTCTRL[]" somewhere in the control program? If not, please consider doing it to switch all new fonts into native ClearType mode. That will bypass all backwards compatibility hacks I'm putting in, although I'd love to stop e.g. Corbel from adjusting vertical stems.


By the way, if anyone wants to try my code, the repo is https://github.com/madig/freetype2/tree/experimental-v38-interpreter.

Nikolaus Waxweiler

@Georg Seifert, @Belleve Invis ping