Ligature Substitution Issues in Windows 10

Ashfaq Niazi

I am Creating a Ligature Base Quran Font, estimated around 22000 Glyphs. Most of the Art work is done. I am using FontLab and VOLT to create this single Font File.

I just use one OTF feature which is simple substitution method to avoid Platform & Application issues. My Ligature Length starts from 2 characters and the maximum length is 21.

I include all characters and Aeraab in my Substitution Method.

I tried this Font on Windows 7, and its working perfect on it and also try it in MS Office, Adobe, & CorelDraw, working fine. Only one issue happens in windows 7 that all Words having a length of "16 Characters" are not substituting, and showing break characters, so I change the length of all 16 characters Words to 17 length by adding a "Kashida" in it so the length goes to "17" and it works fine, so in Windows 7 All the words of length from 2 to 21 are working fine except the length of "16" characters.

Now I try my Font on windows 10, and got upset to see the result, My all Words having length from 2 to 15 are substituting fine, but all words having length of more than 16 or more are disturbing my data. Sometimes they don't substitute correctly, and sometime it substitute correctly but it disturb some other words in that Line.

Attached the Image of  comparison, Words with more than 15 length are circled red, and also circle the disturbing shapes.

Any suggestion.

André G. Isaak

Ashfaq Niazi said:

I just use one OTF feature which is simple substitution method to avoid Platform & Application issues. My Ligature Length starts from 2 characters and the maximum length is 21.

Can you clarify exactly what you mean by the above? When you say you use just one OTF feature, which feature is that? Also, does your font encode positional forms in either of the Arabic Presentation Form blocks, or does it leave them unencoded?

Also, are the above actually the same text? Your Win7 sample looks like it begins with a qaf and lam, but these appear to be cut off in your Win10 sample. Also, I see a small square superimposed upon the first lam in the Win10 text. Is that a .notdef character, or is it something else?

If you could post a few examples of the substitutions from your code it would probably help.

Finally, make sure you have read the following document:

https://docs.microsoft.com/en-us/typography/script-development/arabic

That will clarify which features are applied early by the shaping engine. You need to take that ordering into account.

[DISCLAIMER: I don't read Arabic]

AzizMostafa AzizAli

22000 Glyphs for 2-21 long characters and Aeraab substitution table?!
I do recommend consulting http://maryamsoft.com/QuranPrinting/. They make what I consider the best OpenType format Quranic font, in terms of both design and technical capability as elaborated in the attached QB-English.pdf and here https://ahmedgraf.com/1568-2/ .

Bahman Eslami

Ashfaq, without providing the files or details of the structure of your VOLT file, it's tough to diagnose the problem. I suggest you post an issue here and see how experts could help:

https://github.com/MicrosoftDocs/typography-issues/issues

Ashfaq Niazi

AzizMostafa said:

22000 Glyphs for 2-21 long characters and Aeraab substitution table?!
I do recommend consulting http://maryamsoft.com/QuranPrinting/. They make what I consider the best OpenType format Quranic font, in terms of both design and technical capability as elaborated in the attached QB-English.pdf and here https://ahmedgraf.com/1568-2/ .

Dear Bro, I think the attached link already have Joint base OTF fonts for Quran and Naskh Writing, as you mentioned for maryamsoft and also these fonts are all Freeee... The font size is also less than 200K, and also many websites are using these fonts. My main concern is to provide a ligature base font having equal beauty of Hand Written Mushaf with curved Kashidas instead of straight lines for joining shapes...

Ashfaq Niazi

Dear Aziz Mostafa, Sorry I forget to add Link, Here it is,
https://fonts.qurancomplex.gov.sa/

Respected, Bahman Eslami & André G. Isaak, Here I am attaching some more Screenshots to understand the Idea, I am Just using Required Ligature (rlig) feature alongwith Init, Medi, & Final forms of Arabic Unicode Set. I also assigned all unicode standard to my characters and Diacritics.. The only problem is the Words having length of less than 16 characters are working fine, but Words having more than 16 characters having issue only in Windows X, working fine in Windows 7.

Thomas Phinney

When you say “Windows 7” and “Windows 10” it might help to know exactly which application and version you are using on each platform. An app running on Windows could be using any of several different Windows APIs for its complex script shaping, or using its own. (It is technically possible, for example, that you are getting a newer and better shaping engine on Windows 7 than you are on Windows 10, depending on exactly how you are doing it.)

I would not be surprised to learn that some implementations have a limit on the number of glyphs that can feed into a ligature.

André G. Isaak

When I first read your post I read 2200 instead of 22000. It's really unclear to me why you'd need so many ligatures. Just looking at the examples you give above, I'm curious as to why you need each of these to be ligatures rather than just the parts I've circled in red below:



I can see why you *might* need so many ligatures in a Nastaleeq font, but not in Naskh. Splitting these up would certainly simplify coding and minimize ordering issues, and would help to overcome the sorts of length-limitations on ligatures which Thomas hints at (if any).

André G. Isaak

I just did a quick test on the Mac and if there is a maximum length for ligatures, it's over 25 for InDesign, MS Word, and TextEdit (CoreText). While it's possible that Windows works differently, this seems unlikely given the behaviour of Word.

Bahman Eslami

What I suggest is to not include diacritics in ligatures, your font is getting too complex to handle manually. I also recommend decomposing the letters into dots and dotless base letters in the `ccmp` feature, so you can later dynamically position the dots using mark positioning. Then you can position the diacritics using mark to mark positioning on top of the dots. This approach could remove the need to create so many rules in your font. Your method is also affecting how the substitutions are being processed by the layout engine as there are too many glyphs to substitute in one word.

Could you elaborate why you don't use mark positioning to position the diacritics?

Ashfaq Niazi

Thomas Phinney said:
I would not be surprised to learn that some implementations have a limit on the number of glyphs that can feed into a ligature.

Bahman Eslami  said:

Could you elaborate why you don't use mark positioning to position the diacritics?

André G. Isaak said:
I just did a quick test on the Mac and if there is a maximum length for ligatures, it's over 25

Thank you all for the valuable suggetions. My basic shapes are 6000 without Dots. and with dots it goes around 8000. My one and only concern is to make Dots, Diacritics and Kerning position as beautiful as a Calligrapher Can, . I know it can be done with "Calt", "Kern", "Mark", "Mkmk", etc features. It means I need to fix a lot of Kerning pairs, then many "if" and "or", then a lot of Dot & Diacritics Positions according to Base shape, etc. etc... Which I mention in the begining, I don't want to make it complicated with a lot of Lookups. My 22000 Substitutions are ready and with 22000 glyphs, After shipping the Font size is Less than 5 MB.

My whole 604 pages in MS Word loads within 2~3 seconds with my Font. No Issue at all in Windows 7 with MS Word 2013.

The only issue I am facing is in Windows 10 with same MS Word 2013. As Thomas say may be Windows 7 Shaping Engine is working better. It may be like "Old is Gold". The new Windows should perform better than old versions and expected to be have better facilities. 

I update windows 10 but still same issue. If my font is working in Windows 7 fine, it means something goes wrong with Windows 10. What experts suggest now?

AzizMostafa AzizAli

Ashfaq Niazi said:

Sorry I forget to add Link, Here it is,
https://fonts.qurancomplex.gov.sa/
> My main concern is to provide a ligature base font having equal beauty of Hand Written Mushaf with curved Kashidas instead of straight lines for joining shapes...

@ Well, https://fonts.qurancomplex.gov.sa/ is out of question as it is incomparable with http://maryamsoft.com/QuranPrinting/.

2. Please have a look again to see the differences between the two links then go explore the attached YS-Aziz.pdf  to see my curved Kashidas that go upwards, downwards, clockwise and counterclockwise.

André G. Isaak

Ashfaq Niazi said:

I update windows 10 but still same issue. If my font is working in Windows 7 fine, it means something goes wrong with Windows 10. What experts suggest now?

The fact that it isn't able to deal with words of exactly 16 characters in Windows 7 means it isn't actually working fine. There's got to be some sort of bug in your code, but without seeing the actual code it's difficult to suggest what that might be.

I've never been able to get VOLT to run successfully on a Mac, so I'm not familiar with exactly how it works. I know that AFDKO automatically orders ligature substitutions from longest to shortest, but I don't know how VOLT deals with this. Are you *sure* that your ligature substitutions are actually all in a single lookup? To me, that sounds like a very large lookup and it's possible that VOLT breaks this into separate lookups. In this case, there may be some sort of ordering problem with the lookups.

If you open the final font in OTMaster you might be able to get an idea of how many lookups are actually involved which might provide you with some leads (or you could open the final font in FontLab — FL's decompiled code is generally more readable than OTMaster's, but it isn't necessarily as faithful to the original. It should, however, still display the number of lookups and their ordering correctly).

Ashfaq Niazi

AzizMostafa said:

@ Well, https://fonts.qurancomplex.gov.sa/ is out of question as it is incomparable with http://maryamsoft.com/QuranPrinting/.

Dear Bro, I never saw you to discuss about the topic or give suggestion, you always try to prove that you are the best, and no one can compete you. Its nothing to be personal, I ask suggestion, and you start comparison, Anyhow, hopes you also try "TASMEEM"...

Ashfaq Niazi

André G. Isaak said:
Are you *sure* that your ligature substitutions are actually all in a single lookup? 

I create around 20 Lookups to add 22000 Ligatures, almost each lookup have approx 1000 - 1500 Substitutions in one Lookup. In same lookup there are multiple combinations of characters, like having mix length Words, like same Lookup have 4character, 6character, 10 character, 20character, etc. etc... Ligatures having less than 16 length working fine, but length more than 16 showing error..
Anyhow I will create a small font and will upload it with VOLT Code, if this forum let me to add TTF Font, otherwise if you can send me ur email, I will send you in mail. Becoz I am new in this Forum...

André G. Isaak

Ashfaq,

Sorry -- I'm not sure where I got the idea you had a single lookup. I think I was thinking back to where you said you had a single feature. You can either PM me or contact me at agisaak (at) gmail (dot) com.

Simon Cozens

Aziz: while the MaryamSoft font does look very nice, if someone has already designed 22,000 ligatures, I think they're probably very invested in what they're doing already, so "just give up and use something else" is probably not the most constructive advice.

Ashfaq: I'd strongly recommend installing Harfbuzz and using its testing tools. Download a Windows build here: harfbuzz-2.5.3-win32.zip and then use the hb-shape tool in a terminal. It allows you to see the substitutions that are being made by different shaping engines and that may reveal where the problem is. You can use the same text in Uniscribe, DirectWrite, and Harfbuzz's own OpenType implementation and compare the results:

hb-shape --shapers=ot YourFontFile.ttf "Some Text"<br>hb-shape --shapers=uniscribe YourFontFile.ttf "Some Text"<br>hb-shape --shapers=directwrite YourFontFile.ttf "Some Text"