Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New proposal for a Glyph-Extension axis #26

Open
PeterCon opened this issue Jan 3, 2018 · 23 comments
Open

New proposal for a Glyph-Extension axis #26

PeterCon opened this issue Jan 3, 2018 · 23 comments

Comments

@PeterCon
Copy link

PeterCon commented Jan 3, 2018

A new proposal has been submitted for a "glyph-extension" axis. Use this thread for general discussion of the proposal overall. Please create separate issues for specific details or discussion of specific axes within this proposal.

See the proposal summary here.

@tiroj
Copy link
Collaborator

tiroj commented Jan 3, 2018

I'm sympathetic to the intent of this proposal (I suggested something similar in my TYPO Labs presentation last year). Some comments/questions:

  1. I'm not sure 'extension' is the best term. Why not 'elongation', which is the usual translation of kashida? [Yes, I am aware that the proposal covers more than just join elongation, but see 3.]

  2. re. scale interpretation, I strongly recommend the scale to be per mille of em, rather than font units. This enables easier interoperability between fonts with different UPM settings.

  3. As I understand it, the proposal is for a single axis that would apply, in the Arabic context, to both joining and non-joining letters with GEXT variation. In scribal text justification, though, these are independent mechanisms. There are ways in which they could be treated independently using a single axis — e.g. handling wide isolated and final forms with GSUB only, and reserving GEXT for join elongation; or leaving it to the higher level justification algorithm (with JSTF table support?*) to prioritise the characters to which to apply the GEXT variation — but this introduces complexities that might be simplified by having separate axes for elongated joins vs. wide non-joining letters.

*Eliyezer defined the JSTF table to address the fact that Arabic justification involves style-specific prioritisation of independent mechanisms. It's never been implemented, to my knowledge, in either software or fonts, and I recall Behdad questioning whether it is implementable, but the issue it intended to address is a real one, and I think any new mechanism — such as variations axes in justification — needs to consider the same problem space.

  1. Thinking more about the implementation in an Arabic font: elongation rules in individual script styles involve not only which letters can be elongated, but the situations in which they can be elongated. This can be modelled at the font level using GSUB during pre-justification shaping to filter the glyphs contextually so that only those left in legitimate situations have GEXT variation behaviour. Doable, but complicated.

  2. [This is the Big Issue.] At present, line justification takes place after all OpenType Layout is concluded. In order for GEXT to be used programmatically as part of justification, the entire shaping-justification relationship would need to be changed, because — except for completely flat joins, à la tatweel insertion — GEXT variation is very likely to affect GPOS in terms of cursive attachment and mark positioning (and potentially kerning: even if applied only to joins, kerning of non-joining letters preceding elongated initial forms may differ relative to those elongations). [Since 2014, the OT working group has had numerous discussions about possible solutions for late, post-linebreak (re)shaping, which would open the door for this kind of programmatic use of GEXT or similar alongside new GSUB and GPOS options in justification, but these discussions have tended to stall. There are several viable options on the table, but they all require text layout software to make fundamental changes of a kind that is really hard to push.]

  3. So, taking all that into account, the suggested programmatic interactions statement in the proposal contains some huge higher level protocol assumptions. As a contribution to discussion around the issues of glyph-mechanism justification (as distinct from space- or insertion-mechanism justification), I think this proposal is a good step, but as the proposal says, 'a fully defined algorithm for this interaction should be defined, communicated and discussed with application developers for conventionality and interoperability purposes'. I think this needs to happen before any such axis could be registered; certainly, we'd want to be in a position in which this statement is not part of the proposed axis details as would constitute the registered axis description.

@tiroj
Copy link
Collaborator

tiroj commented Jan 3, 2018

Another thought:

  1. An axis with a scale linked to a predictable change in overall glyph width, such as necessary for programmatic use in line justification, assumes that all the glyphs with such variation have the same minimum and maximum width gain/loss. That may be a practical requirement, but its possibly a design problem when one gets away from Arabic join elongations (which tend to be balanced across a justified line and quite flexible with regard to how wide they can get) and into non-joining letters and other scripts, in which there may be some glyphs that have more restricted width adjustment possibilities than others. Practically, this will mean that the maximum width gain for any GEXT variation axis will be the acceptable design width of the narrowest glyph subject to GEXT variation, yes?

@jmsole
Copy link
Collaborator

jmsole commented Jan 4, 2018

I'll start touching on the different points raised by @tiroj (may leave some of them for later for further consideration):

1- We are open to changing the name for this axis to match with its functionality as best as possible. For us, extension felt like the right word to refer to the overall extent a glyph or part of it may have, without leaving out negative values as they could also be used to change said extent. Elongation does seem to imply an increment of the extension, it implies a direction. We wanted to keep things open for uses other than kashida since the potential is there. There is also the possibility of splitting the proposal into multiple axes or making the scope narrower, the name would then need to be reassessed as well.

2- This is my bad. I got cold feet at some point, had per-mille-of-em as scale, but my lack of complete understanding of that term made me backtrack and change it to font units.

Considering the seventh point that was made, we are also wondering if detaching the scale from glyph measurements might work better. Having a relative scale which goes from 0 to 1000 or -1000, 0 being ‘normal’, 1000 being the biggest increase in extent for that particular glyph, -1000 the biggest decrease. Each glyph would then behave in its own way for a particular value, which wouldn’t be a problem for optical interactions, and perhaps wouldn’t be a problem for programmatic ones either, since the layout engine would still be able to keep track of the final advance width for each glyph (of course, assuming changes to layout engine implementations)

3- Indeed that was one particular worry when using one axis for all the different use cases and types of modifications. Some isolated forms have changes applied to their core, making it closer to what happens on other axes like wdth or xtra, though not exactly either of those axes as many things have to change to maintain the desired shaping (angles, connections, heights). Similarly with some final forms,which would also have changes in their connections.

We thought for the purposes of this proposal we could still use the same axis for all, and treat a change in width as a glyph extension change (may complicate things a bit if someone uses the wdth axis on an Arabic font, though not too much, since such a font could still use elongations for justification). And from the perspective of the script, even elongating a connection still counts as changing a part of the core of what makes a character, since without connections the script wouldn’t work as it has to. So it would fall on the justification algorithm to prioritize which glyph needs to be elongated following the rules already defined in calligraphic practice.

One last thought was to have 2 other axes apart from this one. We would use gext or something like it to change glyphs like isolated forms and the like (more central features on a glyph), and then we would add left extension (lext) and right extension (rext). This would also help when designing connections between glyphs in a script like Arabic, maintaining the practice most people follow nowadays while still allowing for curved elongations (each glyph would supply half of the connection, and thus half of the elongated form when applying kashida). This would add more flexibility, in a lot of situations, but also a lot more complexity, to the point of perhaps making it a bit too convoluted for optical interaction (unless it is linked to quite a smart UI). This would also help in situations where having most of the connecting stroke as part of only one glyph makes it near impossible to design a smooth connection, let alone one that can vary in extension. Such an example would be Nastaliq.

4- That would be doable but complicated as you’ve said. We thought this would be better implemented on the layout engine side, but there is no way to communicate what style of Arabic is being used directly on the font. Perhaps introducing a new field that specifies style when one selects justification on a UI, and a CSS tag for the web. But that’s a big jump. Another option would be to add a field on the name table to be able to specify this, with layout engine being aware of it and able to read it and have internally the correct implementations for justification of each style. Another big jump, of course.

The other points we will touch upon later as there’s still a lot to be considered. We do realise this proposal implies big changes to current implementations, which is a different part of this whole project for us. With this proposal out we have something to guide us and figure what needs to be done. And we know things are likely to evolve as different parts come together.

We are preparing examples and demos which we will link to soon. This may happen next week, we still have a lot of work to do.

@PeterCon
Copy link
Author

PeterCon commented Jan 4, 2018

For justification, it's preferable if a layout engine is able to predict with narrow precision what adjustment is needed to achieve a particular width change. A scale that has arbitrary maximal expansion / contraction (whether the corresponding numeric values are +1000/-1000 or +100000/-100000) will require trial and error without even having a heuristic for a good starting point. So, if expanding, start with the maximal expansion and then do a binary search from there down to an acceptable tolerance. With a per-mille-of-em scale, that at least would provide a good heuristic to choose a starting point, and if implemented well in a font, could allow the engine to hit the desired target within an acceptable tolerance on the first try every time.

@jmsole
Copy link
Collaborator

jmsole commented Jan 4, 2018

That's what we had in mind initially. Having the value correspond to something more tangible would act as a sort of pre-cache which could then be used by a layout engine to figure out what it has available. Now that I understand per-mille-of-em better I agree that's a better scale compared to font units.

@tiroj 's point 7 would still be somewhat problematic from the point of view of all glyphs having the same range of extension. I was wondering if there could be other ways for a layout engine to get the actual min and max extension of each glyph when modified through GEXT. It's likely doable but it would imply initial pre-processing of the font, and one that may take too long at that as well.

@tiroj
Copy link
Collaborator

tiroj commented Jan 4, 2018

I've been thinking about this a lot this morning, especially in light of the earlier example that @sahafshar tweeted [link to video as retweeted by @BBBerning]. I know that was a concept demo, and not necessarily reflective of an actual GEXT implementation, but it has a feature that needs to be considered: unequal elongation of different glyphs. That seems to me a potential design desideratum, but its a technical challenge if justification algorithms apply GEXT in the same way that they currently apply tatweel insertion. Now, there are already problems with the insertion justification model, so we should at least moot alternative approaches.

I gave a summary of how Arabic justification algorithms tend to work during my TYPO Lab presentation last year. As @PeterCon notes, such algorithms need to be able to predict with narrow precision what adjustment is needed to fill the measure. They also need to know where they think they should apply that adjustment, and tend to want to do this as simply as possible by dividing the adjustment width by the number of adjustment locations in the line, and inserting an equal amount of adjustment at each location. If plugged into that model, GEXT would require all glyph adjustments on the axis to correspond to the same absolute scale, with the design limitations I noted in (7) above. If that were not the case, then the algorithm would need some kind of heuristic of trial and error to fill the measure, since it wouldn't know how wide individual glyphs adjusted along the GEXT axis would be relative to each other and relative to the scale. As @jmsole notes, such an implementation could work in a visual, user-controlled setting, but not programmatically using existing algorithm models.

I can think of an alternative model which I'm going to call agnostic or font-side justification. Agnostic justification doesn't know where in the line it thinks adjustments should take place, and it doesn't care. All it knows is that it needs to fit to a given measure by expanding or contracting the standard widths of glyph and/or spaces, according to its own tolerances, by an amount x. It can do this simply by applying the GEXT axis across the whole line, until the advance width of last glyph on the line hits the margin. It is up to the font maker to set up the contextual GSUB that will put the expandable/contractable glyphs with GEXT behaviour in the desired places in the line, based on the rules or preferences for the particular script style or design. Since the algorithm doesn't have to calculate the adjustment distance at discrete locations along the line, it doesn't need those adjustments to be equal.

Existing Arabic justification algorithms try to solve a glyph-space design problem at a character-space analysis level. The JSTF table proposes to insert font maker influence from the glyph-space back into the algorithm, but that seems to have been a non-starter for two decades, perhaps because from a
text layout software perspective it seems to complicate what is already too complicated. I think the agnostic justification concept actually simplifies the Arabic justification model by removing the need to have an algorithm that pretends to know universally legitimate locations independent of typeface design to apply adjustments.

[Issue (5) still applies, of course.]

@tiroj
Copy link
Collaborator

tiroj commented Jan 6, 2018

@sahafshar asked me for more detail on how this idea of putting the justification intelligence in the font, rather than the algorithm, might work. I'm still thinking it through, but this strikes me as a possible model:

The font maker uses the so-far under-utilised Justification Alternates GSUB feature to contextually set up the GIDs in a string that can be subject to elongation or contraction for justification purposes. These can be based on standard models as derived from analysis of the manuscript tradition and manuals for the classical styles, or could be novel methods for individual designs.*

The justification algorithm then applies the GEXT fvar axis** across the whole line of glyphs, moving along the axis to elongate or contract glyphs until the measure is neatly filled. The only thing the algorithm would need to know, it seems to me, is how wide the measure is and what tolerances it applies (tolerances could be hardset or, as in InDesign, subject to user preference settings. The font would be responsible for providing sufficient flexibility in the GEXT axis adjustments to fit typical text to typical measures in whatever manner is appropriate to the typeface design. My suggestion would be that fallback beyond the extremes of the GEXT axis would consist of adjusting inter-word spacing, as is already the fallback case for justification. Yes, there are going to be situations like narrow measures or short lines that don't justify well, but that is already the case, and with variable fonts we have some other solutions for things like justifying headlines using wdth.

This model does put a lot more responsibility for justification onto the font maker. The contextual logic for where to apply adjustments, for Arabic script at least, can be complicated and needs to be documented in a way that provides font makers with some standard code models. I don't think all justification needs to be based on this approach, and algorithms should continue to provide fallback based on existing models. I would suggest that e.g. the inclusion of the OTL feature and GEXT axis in a font could be taken by software as a key to apply this new approach.

I'm sure there will be some people who think this is going against the general OpenType model in which string analysis and standardised shaping behaviour is applied at the layout engine level rather than at the font feature/lookup level à la AAT or Graphite. I've come to the conclusion, though, that justification isn't always standardised shaping behaviour (leaving aside that there is no standard, and all software basically implements it differently), but is design-specific. The way that typeface A would best be justified isn't necessarily the same as the way that typeface B should be justified. Yes, software should provide reasonable default mechanisms, but there should also be the option for clever and ingenious font makers to provide something better and appropriate to the type style or individual design. I think Eliyezer understood this when he included the JSTF table in OpenType, but clearly that solution has not been embraced in the past twenty years. In recent years, we've seen — with the Universal Shaping Engine —, a shift towards giving font makers more responsibility for shaping, and perhaps it is time to do the same for justification?


*Above, @jmsole suggested the idea that fonts could carry flag settings to identify the style of typeface to the justification algorithm, and hence trigger appropriate justification logic in terms of what kinds of justification to apply and where in the line to apply them. I don't think we have or are likely to get the level of standardisation in justification that would be necessary to make that viable. I am also wary about the fact that if one were to hard-bake that level of standardisation into document layout apps it would then be very difficult to update or improve, because of pushback against anything that might cause document reflow (done well, as in Adobe's paragraph composer model, justification and linebreaking are not independent). Also, such an approach ties one into a limited number of justification models, which would be fine for fonts that fit neatly into one of the classical styles in terms of justification mechanisms, but means that justification continues to be inherently outside of the design space of the font, limiting the possibility of type designers creating new styles of script with their own script grammar including novel justification mechanisms.

**Thinking about this some more: in this model, the variation axis could look after a lot of aspects of what was envisioned, in the mid-90s, to be handled via the JSTF table, i.e. not only adjustments but also prioritisation of which glyphs get adjusted. This could be done with GSUB interaction on the axis, triggering specific adjustable forms along the axis path. In that case, it might be as well to rename the proposed axis as the JSTF axis. :)

@khaledhosny
Copy link

I have considered using GSUB to determine kashida insertion positions before (and then just using jalt to supply a list of increasing width glyph, which can be replaced by the proposed axis now), but I ruled it out because many of the rules for inserting kashida are word-wide (not inserting kashida in words less than 𝑛 characters, or not inserting more than kashida at the same word) which I think are hard to do with GSUB in an unrestricted way (i.e. with hard-coding a maximum word length).

What about revising the JSTF table and provide a flag indicating the kind of justification that fits the font style, e.g. SPACE (works for most scripts, and for Ruqaa style in Arabic), NASKH, NASTALIQ, etc. The spec would document the algorithm for each style and it would be up to the layout engine to implement them. The table would specify the Kashida glyph that would be inserted in the styles that use Kashida, and the variable axis would handle elongating these glyphs. May be even provide a list other glyphs that can be elongated (Arabic كاف, Latin glyphs that can be elongated, etc). I haven’t figured this completely in my head yet, but may be it can be combined with jalt like features instead of specifying everything in the table.

This I think is in the line with the general OpenType model of splitting the complexity between the font and the layout engine, as compared to the full flexibility at the cost of more complex fonts offered by AAT and graphite.

I also feel spec’ing the axis before having an actual (even of proof of concept) implementation is a bit premature and might en with the fate of JSTF table.

@khaledhosny
Copy link

(I think I should have read @tiroj last comment before commenting).

@tiroj
Copy link
Collaborator

tiroj commented Jan 6, 2018

@khaledhosny If the JSTF table can be revised sufficiently to enable identification of arbitrary GIDs to be subject to elongation (perhaps mapped from cmap characters via jalt as you suggest), then this seems like it might be a flexible enough solution. I am also aware that OTL contextual lookups are far from ideal for capturing the complexity of kashida rules and preferences.

@jmsole
Copy link
Collaborator

jmsole commented Jan 6, 2018

I was rewatching @tiroj's talk at TypoLabs last year and re-reading this two latest post a few times. John’s suggestion is a very interesting one. Was trying to run in my head how I would write the contextual substitutions needed.

Assuming the font has non-elongating glyphs as default, which glyphs can elongate and in which situations is an easy one to solve. Preventing elongations if there’s already been some elongations applied previously would be a bit more convoluted and may end up with very long contexts, but still doable. So in a way you could do some prioritization with all of that. Another convoluted one would be avoiding kashidas on words at the beginning or end of lines. If only we could trust layout engines with supporting reverse chaining contextual substitutions some of it would be simplified. And we would need some platforms to fix certain bugs related to OT features interacting with the space characters.

I am not entirely sure how this would work if we do want to allow the user to have optical interactions as well if we do one single axis. So I get why part of this would need to become something like a JSTF axis. I suppose we could still use all in one axis, having all glyphs that can elongate and have a functional GEXT axis be the defaults and then, when applying Full Justification and jalt, we convert anything we don’t want to elongate into non-GEXT glyphs. I still haven’t thought how complicated such a set of GSUB lookups would be. Maybe my first intuition is wrong it is actually easier from that perspective.

My understanding is that, no matter what we do for the programmatic interactions and justification algorithms, we still would want to prioritise optical interactions. Those would need to have top priority and then programmatic would apply over that. And then for ragged paragraph settings we would also want to allow optical interactions.

Taking a cue from something shared by @khaledhosny previously, it is not unseen to have different Justification behaviours available (Justifying Text using CSS in Internet Explorer 5.5), but of course those were user selectable. That would require a bit of user interaction already. But that’s also not completely unseen, even in more contemporary implementations like Adobe’s, where there is some user interaction in the selection of the length the kashidas can have. It wouldn’t be completely unimaginable to have a least-worst default that assumes a certain style as most used and then allowing the user to choose different algorithms depending on font style. But then making sure everyone follows the same standards and defining those standards is where it gets more complicated.

The fact is, from the beginning we always thought the current repertoire of Arabic justification algorithms does not do justice to how things should or could be, which John explained wonderfully in his TypoLabs presentation last April. Specifying the axis came from a need to show this is needed and very much wanted. We also wanted to show that similar mechanisms are useful for other scripts and applications. We tried to be as general as possible in our proposal, but are completely aware it will very likely change a lot before anything is approved. We absolutely agree a solution to the justification problem needs to exist before approval, but hopefully it can be developed hand in hand with the axis and with all the different possibilities and needs.

I need to re-read the JSTF table specification to fully understand the scope of what it tried to do and how. Thanks for all the time, interest and effort, and for bearing with us.

@sahafshar
Copy link
Collaborator

sahafshar commented Jan 6, 2018

Thank you for further explaining your point, John. I think it’s a very interesting idea, and certainly it would allow for more flexibility. My worry from the start was that while our proposed model could work well with what has generally become a more popular style of Arabic typefaces (simplified Naskh), it should still apply if someone wished to do more complex connections, such as the ones you see in styles like Nastaliq and Thuluth.
In regards to varying amounts of extension, at a design level there’s isn’t a worry, wherever an extension can exist, it will have a maximum (up to 10 dots in the Nastaliq style --except for the Barree Yeh which can go up to 12) or minimum (3 dots) amount of extension (I’m not sure I completely understand point No. 3 in your first response, as these are all treated as Kashida/Mad in Arabic calligraphy, regardless of their contextual forms, and whether the extension appears as a connection, or isolated within a single letter).
The difficulty is, as you pointed out, how to resolve this within a paragraph, where there exist rules such as: If one full extension exists in the centre of the first line, it is best that this long extension is followed by two half-extensions on either side of the same location in the following line (@khaledhosny also pointed this out in a tweet earlier). The context will also matter, because this rule does not apply when setting poetry.

I’m going to play devil’s advocate here and put issue No. 8 on the table: In the proposed axis, the need for the Kashida character (U+0640) becomes obsolete, because extended letterforms will be doing the work, as they do in handwriting/calligraphy. What happens though, when someone copies in a text that has the Kashida character in it? There needs to be a solution for how the software detects and replaces these.

We are still working to finish up some preliminary examples of this in use, and compiling a full list of Kashida use rules in calligraphy. These should be ready by next week.

@tiroj
Copy link
Collaborator

tiroj commented Jan 8, 2018

@sahafshar re. (8)

I’m going to play devil’s advocate here and put issue No. 8 on the table: In the proposed axis, the need for the Kashida character (U+0640) becomes obsolete, because extended letterforms will be doing the work, as they do in handwriting/calligraphy. What happens though, when someone copies in a text that has the Kashida character in it? There needs to be a solution for how the software detects and replaces these.

I can think of at least three reasons why U+0640 might occur in text. One is that someone has used this tatweel character to manually justify a line of text in some document; another is that someone has used the character for stylistic effect in a document; and another is that someone has used it to transcribe an elongation in a manuscript or document, e.g. as part of a palaeographical or bibliographical description and discussion. Without knowing what the intent of the tatweel character insertion was, I don't think it is safe to presume that it can be always removed from the string.

I know Tom Milo is fond of referring to this character as 'trashida', something that can and should be removed from a text string in order to allow a systematically correct layout to be applied to the text according to script grammar. I think that is one legitimate approach, and quite probably one that is appropriate in most cases of algorithmic text justification, but I think this needs to be a deliberate decision on the part of the user.

At the font level, merging U+0640 character glyphs into GEXT elongations should be manageable, but might require some additional glyphs to function as ligatures, depending on how elongation is implemented in the design. You can take a look at how U+0640 is used to create elongated kashida strokes in Aldhabi, or in fonts from Omar Type Foundry in Pakistan. Of course, this still presents issue (5) re. lack of OTL post-linebreaking and justification. [In addition to the proviously made points about cursive attachment and mark positioning relative to elongations, also note the GSUB variant wider vowel marks in the example I showed at TYPO Labs (using one of the Omar Type Foundry fonts).

@erinmclaughlin
Copy link

erinmclaughlin commented Jan 13, 2018

Hi all – is now the time to also hash out how this proposal would be utilized in other scripts? Is that worthwhile? I know the conversation has been focused on Arabic so far, but should we talk about other scripts concurrently?

This is probably totally obvious, but if it helps others to enter the conversation, I'll just jot this down: The glyph extension axis could be used to elongate the connecting top line in Devanagari, Gurmukhi, Bengali and other connecting Indic scripts, or even certain Latin connected script designs, which would allow users to track out words (or lines) without words breaking apart. (In the Indics, this would work well on basic letters and pre-formed conjunct glyphs, but certain pairs like "half-form + base letter", you wouldn't want to drift apart, so you'd have to be able to designate that in certain glyph combinations, the effect wouldn't occur.)

Anyway! I'll circle back to the discussion if there's a need for us to talk about this, and if I have any novel ideas. Thanks for proposing this!

@tiroj
Copy link
Collaborator

tiroj commented Jan 13, 2018

@erinmclaughlin

(In the Indics, this would work well on basic letters and pre-formed conjunct glyphs, but certain pairs like "half-form + base letter", you wouldn't want to drift apart, so you'd have to be able to designate that in certain glyph combinations, the effect wouldn't occur.)

At the design space level, this is easy to manage in only having the head line extend on the left of a half form glyph. The problem is at the level of layout implementation, where a tracking or justification function doesn't necessarily know that it shouldn't apply space adjustment between a half-form and a following full form. If the function is smart, it will understand Indic cluster boundaries, and only apply spacing — and hence potentially GEXT — adjustments at those boundaries (but not apply GEXT, note, at the boundaries at the beginning and ends of words; it gets complicated). If the function doesn't know about cluster boundaries, it may try to apply adjustments between any spacing (non-zero-width, non-anchor-attached) glyphs.

I can think of three approaches to this problem (using 'I#' to distinguish Indic list items from previous general/Arabic items):

I1. Specify detailed behaviour requirements for tracking and justification for each Indic script, and rely on layout engines to apply these, including specified behaviour for GEXT, JSTF, etc.. Fonts will need to be built to support this spec.

I2. Specify general and agnostic behaviour requirements for tracking and justification, and use the font-side method I suggested to control where GEXT extensions are applied by priming a glyph sting with extensible glyphs in appropriate places. My initial thought is that this would be considerably easier for Indic scripts than for Arabic, since there are not the same kind of complex aesthetic rules about where extensions should be applied. So, for example, one could contextually insert extendable zero-width head line extender glyphs at appropriate places between clusters in within words, which would be invisible and have no effect until GEXT axis adjustments were applied to lengthen them appropriately for a given tracking or justification setting. [This, of course, presuming shaping engines in which cross-cluster contextual shaping is fixed everywhere.]

I3. This one just occurred to me. If the problem is that spacing adjustments may be made where they are not wanted because non-attached spacing glyphs are treated as spaceable, they could be attached using GPOS cursive attachment as a way of saying that spacing should not be applied there. I don't think this is a good idea, though, because a) it puts a requirement on Indic font developers that isn't otherwise necessary, and b) the same model would not work for Arabic script, in which spacing adjustments can apply to base letter glyphs attached with GPOS. I'm only noting this model here for the sake of completeness; I don't recommend or endorse it.

BTW, with regard to (I1), the W3C has been attempting to define how CSS layout properties such as letter-spacing apply to Indic scripts.

@sahafshar
Copy link
Collaborator

Thank you Erin. I would just quickly like to point out that our intention was never for this axis to only cater to the needs of the Arabic script. Referring to an earlier comment by José, our current approach is not entirely tailored to the needs of Arabic, but is a general model with compromise that allows for problem-solving in other scripts as well. Had this not been the case, we would have instead proposed REXT (right-side glyph extension) and LEXT (left-side glyph extension), which would much more effectively solve current issues in Arabic justification. We have been actively looking into examples and scenarios in which GEXT can be useful in other scripts, and expect the current list of uses to grow as we investigate more, and hear from individuals with expertise in a particular script or style.

In the case of Indic scripts, we’ve been told that extending the top bar as a means for justification is not ideal, as it can result in odd inter-word spacing or excess white space. I think though, that GEXT may be a very good candidate for fixing the issues around the elongation of the vowel i ि (U+093F). The axis would allow for an entirely organic adjustment for the length of this diacritic, and let the arch of the diacritic adjust according to the target anchor at the final vertical. I’m aware that there are many other technical factors that need consideration for this to become a reality, but it’s a long standing problem, and the GEXT axis can be part of the solution.

@tiroj: Going back to the discussion on the Arabic Kashida, GEXT would resolve the first and third instances of use of U+0640 that you mentioned. As for the third option, I can’t see why anyone would want to insert flat baseline connections, but it’s a well known fact that familiarity is a very important factor, and native users may have grown very used to these flat connections. Also, as you previously mentioned, there is the matter of using the tatweel as a stylistic choice, which by all means should be available to someone if they wish it. However, in the examples we’re currently working on (these will be out very soon, we promise) this wouldn’t even be a possibility, as we are working on a different method of designing connections, so it would not be possible to insert a straight line between two connecting letters in our proposed method.
The issue of mark positioning according to spacing remains, of course, a major issue.

@khaledhosny
Copy link

There are other uses of U+0640 that are not just stylistic, for example it can be used as seat for a seat-less medial hamza, or dagger alef or a few other Quranic marks, e.g. in يـٰـٔادم or الرحمـٰن or إبرهـۧم (arguably this is a Unicode limitation, but not everyone is agreeing with that and proposals to encode these characters atomically were not sucessful in the past AFAK). So fonts (well, some fonts at least) will still need to support U+0640 and handle it correctly and not just strip it away.

@sahafshar
Copy link
Collaborator

sahafshar commented Jan 15, 2018

@khaledhosny That would fall under scenario No.3 from John's list. It is a very valid point, however, that whatever new changes are made, existing fonts should not be negatively effected by the results. Ideally, this would be resolved by clarifying that if a given font has made use of the GEXT axis, the tatweel character would be removed/replaced with an elongated form of the previous letter in copied text, and if it doesn't, then it remains as is.

@tiroj
Copy link
Collaborator

tiroj commented Jan 15, 2018

@sahafshar

I think though, that GEXT may be a very good candidate for fixing the issues around the elongation of the vowel i ि (U+093F). The axis would allow for an entirely organic adjustment for the length of this diacritic, and let the arch of the diacritic adjust according to the target anchor at the final vertical. I’m aware that there are many other technical factors that need consideration for this to become a reality, but it’s a long standing problem, and the GEXT axis can be part of the solution.

I think something like ikar length is a different problem from the one that GEXT addresses. Although GEXT glyph adjustments can be used to aesthetically adjust the display of text, GEXT has at its core the idea that it could be programmatically used within justification, i.e. to modify the amount of space a line of text occupies in order to fill a measure. In order for that the become a real possibility, the application of GEXT axis adjustments needs to be quite narrowly defined and constrained, so we can offer software makers predictable behaviour.

The control of ikar length relative to consonant letters/conjuncts is a different problem, because it usually doesn't involve changing advance widths or modifying the space occupied by the line of text. So although there is a similarity with GEXT in adjustment at the glyph design space level, at the programmatic level it needs to be distinct from something that relates to justification.

@sahafshar
Copy link
Collaborator

@tiroj

Thank you John. I apologise for the late response, it's been a hectic week for me. I understand that there exists a separate issue with the ikar, but what if different extension lengths are designed for this character, and then the user adjusts these manually in a text? I'm just thinking out loud here, and I don't know enough about the use of the vowel to predict how many times this would need to happen in a given text. I imagine that if it's a commonly used mark then having to manually adjust it every time would be quite tedious. It would have been nice if that issue could have somehow been resolved through GEXT as well.

@sahafshar
Copy link
Collaborator

sahafshar commented Jan 20, 2018

As promised, here are two early examples of GEXT in use: https://github.com/jmsole/gext-demos
For now, our samples show how GEXT works for the Arabic script, but we are also working on including examples from other scripts.

The first of these demos (Arabic kashida animation) was the initial idea I had for how variable fonts would ideally solve the issues surrounding Arabic glyph extension. As John previously mentioned, to have different amounts of elongation–as you see in this example– is very complicated, unless it is being controlled manually. However, we thought it would still be good to include it, just to show what the ideal outcome could potentially be.

The second example (GEXT axis demos with glyph handles) was a result of us thinking about different ways in which a user could interact with this axis. While the familiar slider method is included in the sample (and you can see that it effects every connection that can, in this sample, be elongated), we also wanted to provide a more direct method of user interaction, which is the ability to select an extendable glyph, and adjust the length of the elongation/contraction by simply dragging the cursor. @belluzj helped us by making this possible in the demo, where you can see that the highlighted areas can be directly (and easily) manipulated.

I should mention one important note for these samples: these are early ideations and still require refinement in design details. The goal for us was to provide examples (one calligraphic, one in the more common simplified Naskh style) of GEXT in use as soon as possible, so the community would be able to more clearly see what we have in mind.

@gorjious
Copy link

Hello everyone,

I would like to share a demo of an experiment which explores different ways of how glyph-extension relates to Armenian, in effort to support the progress and broaden the applicability of a glyph-extension axis.

I think it will be helpful to separate glyph-extension ideas for Armenian into two categories: non-spacing and spacing. The current demo focuses on non-spacing characteristics, however, I am working on creating a new demo which will illustrate ideas about how glyph-extension can affect spacing. In a broad sense, non-spacing extensions will be helpful for legibility and aesthetics while spacing extensions will be helpful for layout and justification.

An important note about Armenian is that a large part of the identity of the letters are stored in the ascenders, descenders, and horizontal bars. The extension and combination of these elements form a series of relationships.
Selected sample of similarly grouped letters: ւբդեէ րիխ զլղվը ցյ նմ
The reduction of these elements, common in contemporary typefaces, I believe, is having a negative impact on the script.

variabletest_gorjihanian

Notes accompanying demo:

The inherent challenges of the Armenian script can be seen as the length of a horizontal bar increases next to an ascender or descender. In order to avoid the collision, traditionally, Armenian letterforms form a nested relationship. For example: as the horizontal bar of ղ extends, the descender of ի raises up allowing the bar to slide below. This is also true for letters such as ց which take on an alternate form when preceded by a horizontal bar letter. The relationship formed between this combination is maintained as the descender axis changes. However, thinking about this in a variable design space, there comes a point when the length of the descender or the alternate form of a letter would become too distorted so there needs to be a break point in the axis which reverts the letter back to its default form. In this instance, the horizontal bar letter which precedes the reverted letter compromises on its ability to extend. Other versions of the same horizontal bar letter, which are not immediately followed by a descending letter, are still free to extend.

Along the ascender axis, there can be a similar nested relationship between letters. The added difference is that there can also exist ligatures. Going up the ascender axis, at a certain point there is ample room so that the letters do not necessarily need to be ligatures and are free to function more independently and even form different relationships with other letters. As the ն (which here has its only own letter specific axis) extends, it has the ability to span across another letter and form a ligature with մ (if it is at a point in the ascender axis which won’t compromise the legibility of the in-between letter).

Confession: these are not actual ligatures, just conveniently extended horizontal lines, but it does ask more questions. What would be an approach to accomplish this? Could it be an actual ligature that can vary it’s width depending on the width of the letter in between? What if there are multiple letters in between? What if it had a curved connection? What if there is also a width axis?

I’ll try to summarise how the demo was built. As a quick background, during ISType conference last year, I approached Georg and asked how it would be possible to make a variable font that has these sort of characteristics. At first we tried with separate masters but it seemed unnecessary to create a new master for changing only a select number of glyphs. Georg devised a way of creating virtual masters in Glyphs which allowed for applying an axis specific to only the letters it relates too. In this way, I used virtual masters to program what each letter should do at certain points in the axis. In order to retain the independence between letters which are followed by ascenders/descenders or not, I used contextual alternates each set with different ranges.

Looking forward to any input and feedback!
– Gor

@PeterCon
Copy link
Author

You might be interested in John Hudson's recent presentation at Typo Labs in Berlin: https://www.typotalks.com/videos/constrained-unconstrained-variable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants