Indic shaping issue in Devanagari font

yanone

I’m running into this shaping issue that exists only on Windows (Edge, Word, etc): The combination Ra-Virama isn’t getting replaced in the rphf feature.

This link shows the wrong rendering on top and the correct one at the bottom: https://imgur.com/a/33AGiwy

What puzzles me is that other typefaces that have the exact same OT setup, same feature code, same script and language definitions, render correctly.

Has anyone ever run across something odd like that and has a hint for me where to continue looking?
Thank you

John Hudson

What does your rphf lookup code look like? Are you implementing deva, dev2, or both?

John Hudson

(Edge, Word, etc)

Edge uses HarfBuzz and Word uses DWrite, so if they are both producing wrong results in the same way that strongly suggests a font issue.

yanone

It's an older Edge version that shows the mistake which still shows the IE-style icon. I tried the latest Edge, too, which produces the correct result. I should have mentioned. So as far as I can tell it's DWrite only.

So the feature code is simply sub dvRa dvHalant by dvReph;
Both dev2 and deva scripts are implemented, the feature is active for both.

This is all identical to the other Devanagari typeface I tested which works correctly.

yanone

I know this is all really weird. I'm just hoping someone has experienced such an oddity before and can hint at odd pitfalls

Saurabh Sharma

@yanone, I did some tests on Eczar, Mukta and Poppins. (All three from different foundries). I did font-face test using TTF files in browsers and also in Apache Open Office Writer. Below are the results:



As we can see, all works fine except the Eczar font. May be the post production procedure difference.

John Hudson

So the feature code is simply sub dvRa dvHalant by dvReph;

Which should work fine, and is exactly how my rphf substitution has worked in Microsoft software for many years. Weird, indeed.

Happy to take a look inside the compiled font if you want to email me.

yanone

Hi John
I just emailed you. Please let me know if you didn’t receive anything. Thank you.

Simon Cozens

So this is interesting.

Last Friday, I discovered my Nastaliq font was broken in CoreText environments, and after a lot of swearing, I narrowed the problem down to the fact that CoreText does not support the use of the GPOS 7 lookup type. Since FontTools version 4.14.0 (August 2020), the feature compiler has applied an optimisation where contextual rules without prefixes/suffixes were expressed as the more efficient GSUB 5 and GPOS 7 lookup types. These are more efficient, but they're also pretty rare. makeotf doesn't generate them, and of course until August 2020 neither did fonttools. So there aren't many of them floating around. Turning off that GPOS7 optimization fixed my font on CoreText.

CoreText supports GSUB5, however. But I did a little check of the Google Fonts library anyway to see which fonts were using any GPOS 7 or GSUB 5 lookups.

One of the few fonts in the Google Fonts library with a GSUB 5 lookup is Eczar. It uses GSUB 5 for the lookup "sub uni0930' lookup LigatureSubstitution19 uni094D'". uni0930 is, of course, DEVANAGARI LETTER RA and uni094D is DEVANAGARI VIRAMA.

What a coincidence.

Update: This might be a false alarm; it may just be a coincidence. Seems like switching it for a GSUB6 didn't help. I'm now pointing the finger at GDEF, mark attachment classes, etc.

John Hudson

I took a look at the font Yanone sent me, and I was surprised that the rphf feature was using GSUB lookup type 6 chaining contextual substitution, even though the standard reph substitution is a simple ligature substitutiton: Ra + Virama = Reph

In my fonts (made with VOLT) this is always compiled as a type 4 ligature lookup.

Now, in terms of this possibly explaining why Yanone’s font is not working in non-HarfBuzz environments on Windows, I have to stray into what I hope is accurate memory and undocumented shaping engine behaviour:

As I recall, the MS Indic shaping specification used to contain some confusing statements about some features involved in reordering not being contextual, which no one at MS was ever able to explain to me. When we found some such features definitely needing to be contextual in some scripts (e.g. pref in Javanese), that wording was dropped, but my suspicion is that some of the older Indic shaping engines might expect or only process non-contextual lookup types for these features. So I suspect the problem may be the compilation of the rphf feature lookup as a type 6 substitution, even though the actual substitution used is a type 4 ligature substitution.

I suspect the reph substitution will work in the fonts if it can be compiled as a type 4 substitution. [Not sure what tool and build chain are being used.]

yanone

The old version definitely has a chaining contextual substitution, no idea why, but I tried every combination of script and language tags and of course a simple ligature substitution, all to no avail I'll triple check but I've already done that several times...
Thank you anyway for your time.

Peter Constable

John Hudson said:

... my suspicion is that some of the older Indic shaping engines might expect or only process non-contextual lookup types for these features. 

I'm 95% confident that MS's Indic engine doesn't care what lookup type is used when GSUB features are processed over a glyph run.

However, when a font is loaded, the engine does evaluate consonant conjoining behaviours in the font and tests using context-free <C VIRAMA> or <VIRAMA C> sequences. That might assume type 4 ligature substitution, though I suspect it just provides the sequence and processes a given feature without specifying the lookup type. Of course, a contextual lookup type will be providing _some_ context, and wouldn't match on the above sequences. I don't recall whether this test is done for the 'rphf' feature or not.

John Hudson

Hi Peter, I did some testing of Yanone’s latest build of the font, which uses type 4 lookup for rphf, and confirmed that the font does not work with the Windows shaping engine. I took a look inside the font, and could not find an obvious reason why the font would not work. Maybe you or Andrew could take a look?