Making a Devanagari rupee sign work in Indesign with Opentype

James Puckett

I’m working on a font that has Latin and Devanagari figure. The default latin figures are old style and the Devanagari is the same height as the latin caps, so the default Indian Rupee symbol does not harmonize with the Devanagari figures. I’ve designed a matching Indian Rupee symbol for the Devangari, but I can’t find an OpenType substitution that makes it work in Adobe apps. Substitution does not work in script deva; script dev2; or language hin; language mar; language nep;. John Hudson noted on twitter that this might not work:
U+20B9 is script 'Common'. So it *might* get rolled into glyph runs with adjacent Devanagaru numerals, but I've not tested.

Since that failed I tried using CALT to substitute the Devanagari Indian Rupee Symbol when it is followed by devangari figures (with the Deva figures in a class). This doesn’t work either. As soon as I enter a Devanagari figure CALT is disabled. If I turn CALT back on the substitution works, but only if I use the default composition engine. Changing to the world-ready composer breaks the substitution. Illustrator behaves the same way. Photoshop won’t sub in the alternate Rupee at all. Neither will Chrome.

Suggestions?

Bahman Eslami

This might sound silly but how about putting the contextual lookup in the liga feature. It's always on, right? 

James Puckett

LIGA got it working in Chrome. It still doesn’t work with Adobe’s World Ready Composer.

John Hudson

I don't have a suggestion for how to make this work (other than the lame one of putting it in a Stylistic Set feature and instructing the user the turn it on when using Devanagari numerals), but I can expand on why it isn't working in InDesign and why it might not work other places.

The first step in OpenType Layout as performed by a layout engine is script itemisation. The engine looks through the characters in the text and, using the script property* of the Unicode Standard, assigns each one to a script class. Based on these assignments, the layout engine then splits the text into runs for OTL glyph processing. The script itemisation stage is determined simply by the Unicode script property, but the glyph run segmentation is determined by the layout engine and two things need to be noted: a) this is one of the many undocumented aspects of OpenType Layout, and b) different layout engine makers have implemented it in different ways, so expect incompatibilities.

A lot of glyph run segmentation based on script assignments is fairly straightforward, and you can expect a great deal of compatibility between engines. So, for example, if a piece of text consists of a series of Devanagari letters in a row, all of which have the Unicode script property 'Devanagari', they are going to be rolled into a single glyph run for OTL processing. [Note that this run segmentation is different from and precedes the cluster segmentation then applied within the run for Indic shaping.]

What is less straightforward, and poorly documented, is what should be done when characters with the script property 'Common' occur adjacent to characters with other script properties, and the even less straightforward question of what should happen when this adjacency occurs within a larger context of some text in a third script. So, for example, let's say you have a special <locl> variant glyph for some piece of punctuation such as an opening parenthesis, which you have set up your font to use with the Arabic script. Now imagine a paragraph of English text, containing a parenthetical comment the first word of which is in Arabic, but the remainder of which is in English. I'm guessing that in the context of the overall text, and taking into account the form of the closely parenthesis, you wouldn't want that first parenthesis to be rolled into the glyph run with the adjacent Arabic letters and subject to <locl> processing for Arabic. But how does the layout engine know that?

In the absence of a standard, predictable algorithm for making these kinds of decisions, layout engine makers have either avoided the more complex cases entirely or limited themselves to ad hoc solutions that work in some situations but not in others, based on limited test cases.

The rupee symbol character U+20B9 has the Unicode script property 'Common', and I suspect what you will find is that some layout engines will roll it into glyph runs with adjacent Devanagari numerals, and some -- clearly InDesign's World Ready Composer -- will not.

In case it isn't ovious, what is needed is a proper spec for performing OpenType Layout, including a run segmentation algorithm.
_____

*http://www.unicode.org/Public/UCD/latest/ucd/Scripts.txt

Khaled Hosny

In the absence of a standard, predictable algorithm for making these kinds of decisions

Such thing actually exists: http://www.unicode.org/reports/tr24/, but it is not always implemented consistently (if at all).

John Hudson

So, locl features doesn’t use the dictionary language settings?

That's the user interface mechanism used by InDesign and Word, i.e. linking OTL Language System tags to the text language tagging used for spellchecking and hyphenation. But this is really just a kind of approximate relationship that doesn't actually represent either what the OTL Language System tags capture or how these are structured within the OTL tables.

Language System tags sit under Script tags, and are only active after script itemisation and glyph run segmentation is performed. There's nothing in OpenType Layout that you can use to tell a layout engine 'Treat this character as script X in the following situations'.

John Hudson

Such thing actually exists: http://www.unicode.org/reports/tr24/, but it is not always implemented consistently (if at all).

I'm also not sure if this accounts for all the situations that can occur in glyph layout that a font maker might want to address.

BTW, the kind of late (re)processing model that layout engine makers have been discussing would allow James to do what he wants to do, since script itemisation wouldn't apply at that stage.

Jens Kutilek

Have you put the corresponding languagesystem lines into the OpenType panel?

languagesystem DFLT dflt;
languagesystem latn dflt;
languagesystem latn NLD;

John Hudson

Frode, it is possible that InDesign expects only one-to-one, non-contextual substitutions in the <locl> feature. Despite years and years of being told not to assume particular lookup types in features, and the specification explicitly stating this, software makers continue to make such assumptions.

Try moving the contextual jacute substitutions into another feature under the NLD language system tag, e.g. <calt> (and <rclt> since I presume you'd prefer this not to be disablable).

The idea of the <locl> feature is really to set up new default glyphs for downstream processing, i.e. to replace for specific scripts or language systems the glyphs initially mapped in the cmap table. So while software shouldn't be making assumptions about lookup types, it's arguable that this feature shouldn't be used for the kind of contextual substitution you've applied here.

Michael Rafailyk

Jens Kutilek said:

Have you put the corresponding languagesystem lines into the OpenType panel?

languagesystem DFLT dflt;
languagesystem latn dflt;
languagesystem latn NLD;

Many articles say that for the locl feature to work correctly, I need to specify the languagesystem ..., but no one says how and where to do it.

I suppose, font design editors have changed over time and, for example, FL7 features panel don't have this additional bottom window. I researched a several fonts that have a locl feature but didn't find any mention of languagesystem in Features panel.

So, how to define languagesystem nowadays and do we need to do this at all?
And, does it mean that I can't check it just using FontLab?

Thomas Phinney

Michael Rafailyk said:

Jens Kutilek said:

Have you put the corresponding languagesystem lines into the OpenType panel?

languagesystem DFLT dflt;
languagesystem latn dflt;
languagesystem latn NLD;

Many articles say that for the locl feature to work correctly, I need to specify the languagesystem ..., but no one says how and where to do it.

I suppose, font design editors have changed over time and, for example, FL7 features panel don't have this additional bottom window. I researched a several fonts that have a locl feature but didn't find any mention of languagesystem in Features panel.

So, how to define languagesystem nowadays and do we need to do this at all?
And, does it mean that I can't check it just using FontLab?

The interface is slightly different, but the languagesystem info is still present in FontLab 7, in the “prefix” section of the Features panel. Here:


To me this seems like an improvement over a completely unlabeled mystery section, which is always visible. But it is a change.

Michael Rafailyk

Thomas Phinney said:

The interface is slightly different, but the languagesystem info is still present in FontLab 7, in the “prefix” section of the Features panel.

Oh, it's hidden in plain sight  
I just opened it and looks like it's filled automatically, because it now contains all 12 script/languages, that I declare in the locl feature, nice.

Agree, no reason to show it all the time and this implementation save a lot of space for the features. Just strange that I didn't find any info about this panel using a search on Google.

Thanks, Thomas!

Thomas Phinney

Yes, if you don’t create it manually, it contains stuff automatically. And you can mix the automatic and manual stuff, if desired. It is an elegant system.

This aspect of it does not seem to be documented in the user guide, though at least the general info about the Prefix section is: https://help.fontlab.com/fontlab/7/manual/Features-panel/

John Hudson

[Did something weird happen in the back-end database for this thread? It starts off being about the rupee symbol in Devanagari, and about halfway through suddenly switches to an apparently in-progress thread about Dutch localisation.]