On vertical metrics (in the context of a biscript, Tamil and Latin font)

This morning, I received an email from Anagha Narayanan—currently a student at EsadType in Amiens—asking about vertical metrics settings in the context of a biscript, Tamil and Latin font on which she is working. Anagha had examined some other such fonts, including Tiro Tamil and Anek Tamil, and noted that they handled vertical metrics differently. She had some questions for me about how best to set the vertical metrics in Glyphs, and I ended up writing rather a lot about vertical metrics. I asked Anagha if she would be okay with me posting my responses to her questions here, since the information might be useful to other people, and she graciously agreed. The response encapsulates my current thinking on how to set vertical metrics, including what seem to me legitimate options given the software implementation issues, as well as my understanding of the history that brought us to this place. My response to Anagha is broken into several posts, since apparently the Typedrawers’ platform doesn’t encourage long-form essays.

_____


Let me begin by noting that I do not use Glyphs, and have not looked for a while at how one sets vertical metrics in that program. I do recall that, as with many things, Glyphs will try to automate settings, but that there are a lot of custom parameters that one can use to override the automation if needed. I’ll try to answer your questions in an app-agnostic way.

Anagha: I understand it is comparatively less complicated to fit Latin in a 1000u box. But for Tamil, it's reaches far above and below the ascender and descender. How do you determine the vertical proportions for the two scripts?

So, there are four different data to take into account when considering vertical metrics settings, and bear in mind that these were all originally defined in font formats in the context of Latin fonts and typography.

A. Simple ascender and descender values corresponding to the top of Latin /d and bottom of Latin /p that originated in Adobe’s Postscript Type 1 format. These do not have direct equivalents in TrueType/OpenType, but are often a starting point for calculating outwards to other metrics, especially for a font that contains Latin support, since this approach enables one to create a font that behaves similarly to other Latin fonts.

If a font contains Latin, I usually begin by measuring the height of /d and the depth of /p, and sum those distances to see how they relate to the overall body height, i.e. the total em height measured in units-per-em (UPM). That lets me know whether the Latin design is small or large on the body. Generally, I want the total height of ascender + descender to be close to the body height, because that means that e.g. at 12pt the typeface will be of similar size to other 12pt text.

B. Apple-specific vertical metrics in the hhea table of TT/OT fonts: ascender, descender, lineGap. These originate in Apple’s original TrueType specification, and are still used in Mac OS and iOS to determine default linespacing and text-to-frame distances. I’ll come back to these: I mention them first because they pre-date the OS/2 metrics.

C. Microsoft OS/2 table sTypoAscender, sTypoDescender, and sTypoLineGap (the Typo metrics); and

D. Microsoft OS/2 table usWinAscent and usWinDescent (the Win metrics).

I’ll discuss the OS/2 metrics together...

Comments

  • When Microsoft licensed TrueType from Apple in the early 1990s, they were not satisfied with the hhea table, so defined their own OS/2 table, as a general place to put a lot of data that Windows uses in text layout, font fallback, etc.. Since Microsoft didn’t want to be grabbing metrics data from multiple tables, they defined the Typo values that directly corresponded to the three Apple hhea vertical metrics data. Microsoft then defined the Win metrics, which were intended to define a vertical distance beyond which glyphs might get clipped in layout, recognising that this distance would not always be the same as the desired interline spacing distance defined by the sum of the Typo metrics.

    Then things almost immediately got messy. Microsoft Windows graphics and Office text layout programmers misinterpreted the OS/2 table data, and started using the Win metrics to determine default linespacing, ignoring the Typo metrics that were intended for that purpose. This is typical programmer thinking: there are metrics that avoid clipping, and hence if one wants to do quick and easy line layout without risk of clipping, using the Win metrics for linespacing ensures that one can paint consecutive lines of text without clipping previous lines. And because the context of the time was almost entirely Latin fonts with limited height and no real need for Typo and Win metrics to diverge, no one really noticed that this was going to be a problem.

    By the early 2000s, it had become recognised as a problem. We were making fonts like Cambria Math, which included very tall glyphs that we needed not to be clipped but which intentionally exceeded normal linespacing heights (because they were intended to be used in math equations, not in regular text layout). So we were setting the Win metrics appropriately huge, and the Typo metrics smaller to reflect what we thought the default linespacing should be. But most Windows software was still ignoring the Typo metrics, and couldn’t switch from using Win to Typo metrics without breaking existing documents created under the mistaken metrics assumptions. [These assumptions also affected preview display in Office font menus, which scaled Win metrics to a fixed height, meaning that fonts like Cambria Math looked miniscule in the menu.] At the 2004 ATypI conference in Prague, Microsoft announced the definition of a new bit setting in the OS/2 fsSelection field (bit 7), which flags to software that it should ‘Use Typo metrics’. This enabled us to have divergent Typo and Win metrics in fonts like Cambria Math, STIX Two Math, Aldhabi, and in some of the Tiro Indic fonts.

    However, if programmers can figure out a way to get it wrong, they will. Both Microsoft and Libre Office, instead of using Typo metrics for linespacing and Win metrics for clipping, if fsSelection bit 7 is set, appear to get the linespacing right but use the TypoAscender and TypoDescender metrics for clipping. This leads to this sort of bug:
    https://github.com/TiroTypeworks/Indigo/issues/16
    https://github.com/TiroTypeworks/Indigo/issues/17

    So my recommendation is that if reasonably possible, one should try to align the Typo and Win metrics such that the sum of three Typo metrics and of the two Win metrics are the same. That is usually possible for European scripts for typical text setting, and for at least some other scripts. It isn’t always possible for Indic scripts, because of the need to avoid clipping conjunct stacks. Because modern Tamil does not use vertical stacks, it is usually possible to align the Typo and Win metrics for Tamil fonts.

  • So, how should you proceed? I suggest these steps:

    1) As suggested above (A), you probably want your typeface to be a similar scale to other types at the same nominal point size, so it is a good idea to start by measuring your Latin /d and /p, and confirm that these simple ascender and descender measurements sum close to the body height (i.e. to the em unit height). Typically, they will be just slightly shorter than the body height.

    2) Now set the TypoAscender and TypoDescender metrics (this is probably what Glyphs just calls Ascender and Descender unless you are using custom parameters; you can get good help on this in the Glyphs forum). I recommend, as a first step, setting these to sum to the body height, which will likely mean they are slightly larger than the actual /d and /p measurement you made in step (1). There are three options for how to handle the difference between the measured height and the body height: add all the difference to either the TypoAscender or TypoDescender; split the difference equally between the TypoAscender and TypoDescender; or split the difference proportionally between the TypoAscender and TypoDescender based on proportion of the glyph height above and below the baseline. I usually use the last method, but if combining Latin with a script with deeper descenders, feel free to add the difference to TypoDescender. Don’t worry about parts of your Tamil glyphs exceeding the TypoAscender or TypoDecender.

    3) Now consider what you want the default linespacing of typical text in your font to be. This can be tricky in multi-script fonts—which is why I have been campaigning for years for a BASE table update to enable script-specific linespacing metrics—but for a biscript Tamil and Latin font it should be relatively easy. You might want slightly more default linespacing than in a typical Latin font, to give space to the deeper Tamil descenders, but probably not so large that it will look freakish in English or other Latin script text. Define this default leading in the TypoLineGap value as the difference between the combined TypoAscender and TypoDescender and the total desired linespacing distance.

    4) Now take a look at the tallest and deepest glyphs in your font, i.e. those that define the vertical bounding box. Note that you will want to do this across all masters and all parts of the font family, which might mean looking at the heaviest weight, checking whether the tallest/deepest italics are the same as the roman, and considering that the tallest glyph might be in a different master than the deepest glyph. [In fonts for some Indic scripts, I also take into account the height and depth of vowel signs or other marks that may be raised or lowered when positioned on particular bases or in conjunct stacks, because software may clip these if they are positioned beyond the Win metrics. This shouldn’t be an issue for Tamil.] Make an initial setting of the WinAscent and WinDescent values to equal the tallest height and deepest depth you want to avoid clipping across your font family.

    5) Now compare your Win metrics setting to your Typo metric setting.
    • If the sum of the two Win metrics is less than the sum of the three Typo metrics, I recommend increasing the Win metrics proportionally so that they align with the Typo metrics height. This ensures compatible linespacing in environments that ignore fsSelection bit 7 and still use the Win metrics for linespacing, and does so without risk of clipping.
    • If the sum of the two Win metrics is greater than that of the three Typo metrics (this is unlikely in a Tamil font), then you have to decide whether to accept this divergence, ensure that fsSelection bit 7 is set (I think it is by default in Glyphs, but confirm this), and accept that you may get incompatible linespacing in some environents; or, if the difference is not too great, you might consider increasing the Typo metrics to align to the Win metrics, preserving compatibility at the cost of larger default linespacing.

    6) You now have arrived at the kind of OS/2 metrics that I like to use. I like to use them because they provide good compatibility while preserving the intention of the OS/2 table specification with regard to Typo metrics defining linespacing and Win metrics defining the non-clipping height. I like to do things the way they should be done, even if following the spec means things breaking in software because of incorrect implementation assumptions. However, existing implementation issues and persistent software bugs lead some font makers to different decisions, and I don’t say they are wrong to do this. Because of the misinterpretation of the fsSelection bit in MS and Libre Office, as discussed above, these environments will clip at the TypoAscender and TypoDescender height, even if the leading value specified in the TypoLineGap provides plenty of space to avoid clipping. This is why some fonts—including the Anek Tamil that you illustrated:


    —instead set the TypoAscender and TypoDescender values to sum to the total linespacing height (and hence to the same values as the Win metrics if these align), and set the TypoLineGap value to 0 (zero). This is a reasonable workaround to the software bugs. I just don’t think bugs get fixed if people accommodate them, and have never accepted that font makers should have to break specification conformance to get things to work in broken software. But I have the kind of clients to whom I can say that, and I recognise that not everyone does: sometimes you just have to make things work in the places where the customer needs them to work today. So...

    7) Consider whether to keep your Typo metrics as initially defined in steps 1–5, or to change them for broader compatibility with current software issues. Keep as is, or make the change.

    8) Remember the Apple hhea metrics? Now you need to set those.

    • If, in step (7), you changed the Typo metrics so that TypoLineGap = zero and the TypoAscender and TypoDescender now sum to the whole linespacing height (and possibly correspond directly to the two Win metrics values), then you should set the corresponding hhea metrics the same way:

      hhea ascender = OS/2 TypoAscender [ = OS/2 WinAscent]
      hhea descender = OS/2 TypoDescender [ = OS/2 WinDescent]
      hhea lineGap = OS/2 TypoLineGap = zero

    • If in step 7 you kept the Typo metrics with leading defined in the TypoLineGap value, then you have a choice:

      EITHER
      hhea ascender = OS/2 TypoAscender
      hhea descender = OS/2 TypoDescender
      hhea lineGap = OS/2 TypoLineGap

      OR
      hhea ascender = OS/2 WinAscent
      hhea descender = OS/2 WinDescent
      hhea lineGap = zero

    Both options are used in various fonts, and I don’t have a recommendation which to employ. [It may also be worth noting a little-known and undocumented behaviour in recent Apple software, which is that if a font contains a STAT table (required in variable fonts and recommended for new fonts generally but not automatically included in static fonts output by Glyphs, I think), then at least some Apple software will use OS/2 metrics instead of hhea metrics. I have not tested this, however, and am not sure if/how they interpret fsSelection bit 7.]

  • Anagha: Does having higher units per em, like 2000u or 2048u benefit Indic scripts (Tamil)?

    Units per em defines the granularity of the design grid. All coordinates in outlines, spacing, kerning and mark positioning, and metrics are relative to the em, as expressed in the units per em scale. So a higher UPM value means a finer grid, which can be a benefit in typefaces with very fine details or certain types of shallow curves that may be hard to make refined at a lower UPM. A higher UPM will typically produce a larger font file, though, since more coordinates will be larger values.

    A higher UPM does not give you more space, it just divides the space more finely.

  • Anagha: In Tamil, in terms of vertical metrics is it prefered to alter the typoAscender/typoDescender or winAscent/winDescent to better accommodate the glyphs?

    See above. You definitely need to ensure that nothing exceeds the Win metrics, to avoid clipping, but other than that the decisions are about how you want the font to behave in software that uses metrics to determine default linespacing.

    Anagha: I found a couple of approaches for dealing with vertical proportions — Anek which works with 2000u and adapts the ascender/descender values to Tamil's proportions, inturn accommodating the outlines within the “box’. And TIro Tamil which works with 1000u and has the tamil outlines spill out of the "box". I'm curious to understand the reasons for choosing the latter in Tiro Tamil, and if there are benefits of one or the other.

    The Tiro Tamil outlines exceed the body height, but they are within the linespacing height as defined by the Typo metrics and well within the non-clipping height as defined by the Win metrics. You see that the actual scale of Tiro Tamil and Anek is similar, which means that the size of the outlines relative to the body height is actually about the same, i.e. the distance of the Latin descender /g relative to the total em height is similar to that of the Tiro Tamil /g. What is different is just the setting of the TypoAscender and TypoDescender values, and whether TypoLineGap is set to zero or to a meaningful leading value.

    The original Murty versions of the Tiro Indic fonts were targeting Adobe InDesign, used in production for the MCLI volumes from Harvard University Press. The first versions of the Tiro release were licensed and tested by Apple, and they were happy with the way I set the vertical metrics to conform to the specification, and were mostly not too concerned with what I did with the OS/2 metrics, since they were only using the hhea metrics. Then Google Fonts funded the OFL release of the fonts, and while the way I set the metrics did not conform to their own recommendations, they understood why I had set them as I had and why backwards compatibility with the Apple release was worth maintaining. Also, because Google Fonts’ own use is targeting the Web, they only cared that things worked in browsers, and didn’t worry about possible desktop application issues.

    I continue to think about the Tiro Indic metrics settings, and have considered and rejected changing them a couple of times. I am still not sure, given the issues reported in MS and Libre Office. A change would be complicated by the fact that I have divergent Type and Win metrics in at least some of the families, so it isn’t as simple as changing the Typo metrics to correspond to the Win metrics.

  • k.l.
    k.l. Posts: 109
    edited October 2022
    Why not keep it simple? SIL’s suggestion may well be considered the final word on the matter. So much so that I decided to reference and rephrase it on the last page of mine. It may not give nicest results but consistent ones. Including in broswers. Whichever set of values programmers may choose, they get the same values anyway. Mine
  • The SIL suggestion is exactly where I came out to, last time I really studied the whole mess. If you want consistent results across browsers on all platforms, it was the way to go.

    Now, that also means you can never improve things, all layout is stuck doing lowest-common-denominator forever. That is what backwards compatibility means.  ¯\_(ツ)_/¯
  • John Hudson
    John Hudson Posts: 3,186
    edited September 2022
    a) The SIL suggestion assumes all ascender and descender metrics sets can be aligned. I already explained the circumstances in which that might not be the case. The step-by-step process I described gets you to the point (step 7) at which you have the information you need to determine if and how to align the metrics. Starting from the assumption that you can and will ‘keep it simple’ and align the metrics may make sense for a project—especially if it is a fairly standard Latin font—, but for people making complex multi-script families it is not helpful to assume something that will e.g. produce massively over-large linespacing in all environments. Going through the steps provides understanding of the relationships between scale, desired linespacing, and clipping zones.

    b) The SIL suggestion is reactive. It begins by discussing software behaviour, and makes a recommendation to get the best outcomes of a bad situation. As such, it does nothing to improve the situation or to provide any pressure to get the situation improved. And then people wonder why we’re still having these discussions twenty years after the problems of GDI use of Win metrics for linespacing were first identified as a problem.

    That is what backwards compatibility means. 

    The fsSelection bit 7 solution was specifically designed not to break backwards compatibility. It has not been properly implemented in some environments, but the good news is that it still can be—even in those environments—without affecting document compatibility and without needing updates to any fonts. The problem now is that some word processing software is clipping to Type metrics as well as using them for linespacing. That behaviour can be changed without affecting the linespacing: there is some indication that Libre Office might already be addressing this, and we know that MS Word has the code to do it because they already do so if user set the interline spacing to an absolute rather than metrics-relative value.
  • Thomas Phinney
    Thomas Phinney Posts: 2,883
    edited September 2022
    The fsSelection bit 7 solution assumes that the font developer is willing to have different linespacing in different environments.

    Basically, due to ancient Windows behaviors, if one wants consistent linespacing across environments, one has to use the clipping boundaries for linespacing. Yes, with some writing systems that will yield VERY large linespacing. If you want no clipping, and consistent spacing across environments, that is the price that must be paid.

    I certainly understand that how big that price is, depends on the nature of the typeface in question—and your case at hand is a clear example of a situation where that price is quite high. Whether it is TOO high is of course a judgment call. I am not saying you are wrong to advocate what you are advocating.

    But similarly, I don’t think the stupid absolutist solution I mentioned is crazy either.

    Honestly, it is a Very Bad situation. There is no particularly good answer, only crappy ones. It is interesting to hear which crappy answer each person picks or would advocate.  

    😛

  • The fsSelection bit 7 solution assumes that the font developer is willing to have different linespacing in different environments.
    100% willing to have that if some of those environments have the correct linespacing and clipping behaviour. That is a better outcome than all of the environments having incorrect linespacing and/or clipping behaviour.

    Most environments outside of plain text processing have mechanisms to define interline spacing in ways other than font metrics. So we’re talking about an overrideable fallback behaviour in some environments, and I don’t think consistency of linespacing in those environments should be treated as important. Working around inertia and bad practices risks perpetuating both,

    But similarly, I don’t think the stupid absolutist solution I mentioned is crazy either.
    I don’t think it is crazy either. There are situations in which it is perfectly reasonable—when the metrics sets can align without comprose of either linespacing or clipping behaviour—, and situations in which it is an acceptable compromise—when aligning the metrics sets might produce less than ideal but okay linespacing or clipping behaviour.

    The point of the step-by-step approach I reocommend, rather than just jumping to the alignment solution, is to understand—in the context of a specific typeface project—what are the relationships are between the glyph shapes (and positioning) and the linespacing and clipping zones, so one can determine the best approach for the individual case.
  • John Savard
    John Savard Posts: 1,126
    edited September 2022
    Divergent Typo and Win metrics. That's why STIX is still broken for me, even after it had been fixed. However, since WordPad is part of Windows, and not written by a third-party programmer, that is still inexcusable; even though it is only a simple program, it should obey the specifications that Microsoft understands, even if no one else does.
  • k.l.
    k.l. Posts: 109
    edited September 2022
    Google’s does not differ that much from SIL’s current one, except for also covering font updates which are special cases anyway: One needs to be very clear about one’s objectives as well as possible tradeoffs. (What exactly, in which environments, do I want to keep backward compatible? Which side effects, in which environments, am I willing to tolerate?)

    (What I liked and still like about SIL’s approach is this: We may have addressed desktop applications before, in slightly varying ways, trying to respect the stated intent of this or that set of vertical metrics where possible. Then web fonts became popular, browsers used metrics differently and, worse, inconsistently so. SIL’s ‘brute force’ approach seemed to address browsers first, removing any possibly vagueness, and at the same time turned out to be good enough for use in desktop applications too. No need to bother type designers with ‘do this, then this, that too, sum these up, then deduce those, etc’ which I realized some still have trouble to follow.)

    If you do a new version of your document, I think it were great if you stated a simple approach addressing new fonts first, followed by separate sections that address more special scenarios, like updating/expanding existing fonts, each such section providing one method only and explicitly stating the respective objective as well as tradeoffs that are to be expected.
  • That's a nice idea - thanks. There are multiple scenarios, all of which have reasonable strategies (and variations): 
    • Simple: Just set everything to yMax/yMin.
    • Thoughtful: Similar to John's approach above (but simpler by starting with yMax/yMin and not using TypoLineGap ever)
    • Primary: Similar to Thoughtful but starting with the primary script if it's not Latin (this is becoming our approach)
    • Legacy: Adjusting metrics of existing families



  • k.l.
    k.l. Posts: 109
    edited October 2022
    This sounds very good! I think ‘Primary’ might even be folded in, as a premise, to all of the approaches, rather than being a separate approach: When it comes to new fonts, type designers should be encouraged at first to consider any scripts to be possibly covered, sketching them even if they are to be added later or even never, at least tallest characters and/or those with most diacritic marks above/below if precomposed, and only then define metrics in an anticipatory way – and generously so. Latin first should be a thing of the past.

  • John Hudson
    John Hudson Posts: 3,186
    edited October 2022
    While I agree that Latin-first is an outdated mode of font development, I think what that really means is that it is long past time to ditch the whole idea of a single set of vertical metrics for all scripts covered in a font, and this, in turn, means ditching any notion of aligning of linespacing and clipping metrics. The only sensible model for providing appropriate linespacing for particular scripts—and potentially distinct linespacing for different langsys—is to localise Typo metrics while being free to increase Win metrics to cover yMax/yMin to avoid clipping any of the scripts.

    I have been pushing for extensions of the BASE table to accommodate script/langsys metrics since the mid-2010s. Apple already have a custom BASE implementation for script-specific metrics that they use in some fonts, but no one else has so far been willing to do the work. That’s a pity, because it would provide a mechanism to fix the broken metrics model, properly specify how the new model is supposed to be implemented in software—rather than once again leaving it to developers to interpret and get wrong—, and relegate the OS/2 Typo metrics to a specifically backwards compatible fallback set.
  • ...it is long past time to ditch the whole idea of a single set of vertical metrics for all scripts covered in a font

    Hear, hear! Heartily agree with you, John. Unfortunately to get there requires adding Yet Another set of metrics values while paying attention to all the existing ones (for all current apps until they all switch). Painful but necessary.
  • k.l.
    k.l. Posts: 109
    edited October 2022
    Happy I did not respond immediately, wanting to say just that, using way more words.

    Let me suggest that we look at it from the application side instead of from the font side. Do applications need fonts to provide more data? I do not think so. It is all there. I would go further and say that good old head table’s UPM and maybe yMin/yMax are all that applications need. For more precise, even script specific, data there are – glyphs. (UPM determines, together with glyphs’ actual dimensions, relative type size. It may inform rule of thumb based fallback baseline to baseline spacing too. yMin and yMax are a reliable indicator of clipping zones, being calculated automatically by font editors and font tools. Given that glyphs may likely sit at yOffset ≠ 0, these days, clipping zones are better calculated at layout time anyway.)

    Some things are not fonts’, not font formats’, not font makers’ business.
  • John Hudson
    John Hudson Posts: 3,186
    What you describe indeed can be used for rule-of-thumb, fallback baseline-to-baseline spacing, but such a fallback knows nothing about script or language, so which ‘glyphs’ does it look at? In what arrangements?

    Some things are not fonts’, not font formats’, not font makers’ business.

    Indeed: they’re the typographer’s business. But an application running blind is not a typographer, and we’re seen what developers instinctively do without guidance on how to set linespacing: they use yMin/yMax.
  • k.l.
    k.l. Posts: 109
    Mocking my ‘glyphs’ to compensate for your own lack of imagination is not particularly nice.

    A little hint. What are we talking about? Layout. Of actual text. Consisting of characters. In a specific font. That maps characters to glyphs. Which can be measured. This is as script and language specific as could be, and as need be, without even bothering to explicitly deal with the notions of ‘script’ and ‘language’.

    Not sure what an application running blind is.
  • John Hudson
    John Hudson Posts: 3,186
    edited October 2022
    I wasn’t mocking. Those are serious questions. I am perfectly capable of imagining the sort of thing you are talking about, and that imagination is what led me to the questions.

    Asking application developers to measure painted glyphs to determine linespacing seems to me very unlikely to get traction. Text layout is supposed to be fast and to be stable. The reason I asked what glyphs and in what arrangements is that while Latin text tends to produce fairly consistent glyph heights across multiple lines of text, that is not the case in all writing systems. Should the existing line heights change if a new line contains a deeper conjunct stack than the previous lines? Or should line heights vary depending on the content of individual lines?

    Font vertical metrics exist precisely because text layout developers wanted reliable font data to determine linespacing. The fact that those developers then used the wrong data in the wrong way says more about the absence of implementation specification—a perrennial problem with TrueType and OpenType—than about the data model itself.
  • k.l.
    k.l. Posts: 109
    Painted glyphs – in final positions. There is no way around that. Precisely because scripts other than Latin do not share similarly even glyph heights. Forget about Latin centered layout practices. I mean it.
    (The concerns you bring forward are all due to your thinking still being informed by Latin centered layout practices.)
    What to do, like changing line height ad hoc, is not a technical matter but a design decision. This implies that there is hardly a way around some sort of styling or formatting, with APIs, stylesheets, or dialogs for defining it, using information extracted from fonts conditionally where it helps.

    Hehe, yes, fonts’ vertical metrics data exist because of a desire to have something reliable to use, but by now we should have established that reliability is a chimaera as soon as we leave Latin, or diacritic-free English, behind.
  • John Hudson
    John Hudson Posts: 3,186
    edited October 2022
    (The concerns you bring forward are all due to your thinking still being informed by Latin centered layout practices.)
    They’re really not.

    If you look at textual practices around the world, in all manner of different scripts, consistent interlinear spacing is the norm, and there are conventions for different kinds of texts, different column widths, etc. just as there are for European scripts. The actual distances involved are different, but they are similarly consistent within texts. There is nothing special about Latin script in this respect, nor is there anything exotic about other scripts.

    So just as a Latin script font can be provided with standard fallback metrics that define a typical or average linespacing that can be used when linespacing is not explicitly specified by the user, the same can be done for any other single-script font. The issue at the font level is only that we now make multiscript fonts but still only have one, script-agnostic set of vertical metrics.

    We don’t need to invent some entirely new way of calculating linespacing based on measuring painted glyphs to lay out scripts with reasonable default interlinear distances. [I can see the attractiveness of the idea for fine-tuning to different kinds of texts that e.g. might involve supralinear notation, e.g. the Samaveda, and hence need greater-than-typical linespacing. On the other hand, these could also be considered situations in which software should stop trying to automate linespacing and instead require user input.]

    Further, measuring painted glyphs still doesn’t get you away from metrics: what specifies the distance above or below the tallest and deepest painted glyph? Yes, this could be a constant applied by the software—standardised across all software? standardised where?—but would that constant be appropriate to all scripts and all kinds of text?

    by now we should have established that reliability is a chimaera as soon as we leave Latin, or diacritic-free English, behind.
    We lack the data to provide reliability, therefore reliability is a chimera? Why not just provide the data, and a proper specification how to implement it?

  • k.l.
    k.l. Posts: 109
    I am sorry to say, John, you do not get it.

    1) Line spacing is rarely defined entirely automatically, instead is based on styles defined by humans. No need for such data in fonts.
    2) For fallback – being fallback – you do not need anything too elaborate.
    3) All that I am saying is that there is no need for more such data in fonts as the existence of such data is based on the assumption that text’s heights can be predicted, which is not the case. (This addresses the chimaera questions too.) All that I am saying is that, if a layout mechanism wants to make more informed decisions, such data is already present in fonts, and is more precise than what font makers could ever anticipate.

    Nowhere did I speak of inventing an entirely new way of calculating line spacing. See 1) above.
    And I definitely did not suggest that line spacing should, or even might, vary from line to line – I always detested the practice of opening up line spacing, above and below individual lines that contain an Arabic word or a mathematical formula, in scientific books. This is really the silly-most thing to put into a typographer’s mouth ...

    There is nothing more I could say on this matter.
  • John Hudson
    John Hudson Posts: 3,186
    edited October 2022
    Re. (1), huge amounts of text layout in everything from word processers to the web are defined in terms of default font metrics, without human user intervention. This is a problem for fonts that support multiple different writing systems. This isn’t a theoretical problem.

    Apple have already demonstrated that putting script-specific linespacing metrics in the BASE table works, it just needs to be documented, standardised, and a clear implementation spec written. And the great news for you is that no font maker would be obliged to include this data in fonts. If you’re only making Latin or other single-script fonts, you can just ignore the new BASE fields. But if, for example, you have made a South Asian font that supports fifteen different writing systems with very different use of vertical space, you would have a way to tailor default linespacing to each script.

    There is nothing more I could say on this matter.
  • The reality is that applications very commonly 'guess' at default line height based either on some arbitrary percentage (120% for InDesign) or the metrics in the font. Typographers who know how to use styles can certainly override that, however that's rare among the general public (and a frequent source of user support queries). Whatever we do that reality will be around for a long time. So while we hope the BASE table may someday get more traction in the industry we have to do the best we can to deal with the mess that is application line spacing.

    I have now updated our recommendations to try and make that as clear and easy as possible. Thanks for the good info everyone!

    Font Development Best Practices: Line Metrics


  • bberning
    bberning Posts: 2
    edited December 2022
    Thanks for sharing your resources and thoughts on this, everyone. This is really interesting.
    I’m finding myself in a position where I’m trying to manage typography at scale across different platforms and geographies as part of a larger design system. What I would like to do is find a way to do that without either compromising the experience (clipping!) or having individual teams overwrite design tokens that define things like the font family, weight, size, line height, etc to accommodate for larger letterforms. I find that the vertical metrics in their current state are no good to me in this context. 
    I don’t know if something like Apple’s BASE table implementation might be the solution to our shared problems but I’d love to find out more about it. Does anyone have a link to this? I googled and ttx’d SF Pro left and right but no luck so far.
  • John Savard
    John Savard Posts: 1,126
    edited December 2022
    While I agree that Latin-first is an outdated mode of font development, I think what that really means is that it is long past time to ditch the whole idea of a single set of vertical metrics for all scripts covered in a font, and this, in turn, means ditching any notion of aligning of linespacing and clipping metrics.

    What is the desired behavior of a font containing multiple languages?
    I know that when I use some typefaces, if a line contains italics, more space is inserted between it and the preceding and following lines. Of course, this involves the use of two different fonts, and for some reason, the italic font, although at the same point size, is larger.
    The behavior I expect is that if I specify 12 points as the font size, then there will be absolutely no clipping whatever, under any circumstances whatsoever, with a line spacing of 12 points. Glyphs may be kerned horizontally, but never vertically.
    And so if a Tamil font needs to be set at 36 points to be legible, and I include Tamil text in a mainly Latin document, using a font that supports both languages, and I find that the Tamil text is really tiny... I can highlight it, and change its point size.
    This thinking, of course, is based on the following premise:
    There is one reasonable and comprehensible way for type to behave in the digital world which will allow people to set type without going insane. That is to have it behave exactly like hot (or cold) metal.
    I think, however, that this premise has been rejected by the designers of TrueType, on the basis that this would limit the great potential for greater flexibility that digital type can offer.
    And, of course, Tamil is not really the case that would cause a fatal problem for my proposal. Vietnamese would be, since Unicode does not specify a set of codepoints for a smaller version of the Latin alphabet to use for writing Vietnamese, so that the accents it requires would fit into the same character cell... and, in fact, since upper-case letters are accented in French and other European languages with simple accents, even they need a smaller version of the Latin alphabet if 12-point digital fonts are to normally be the same size on the paper as 12-point metal fonts.
    Of course, currently, they don't. 12 point Times Roman comes out of a laser printer looking like 10 point Times Roman with 2 points of leading, as I noticed back before we even had TrueType.
    Even back in the era of metal type, of course, people used leading plus piece accents to accent upper-case letters, and they set 12 point type on 13 point bodies for variations of typefaces with long descenders (Monotype Times Roman and Linotype TextType both did this, as I recently learned).
    And it's nice to be able to do it that way, so that the letters of the Latin alphabet could be consistent in size between languages.
    But then I would insist on a user interface which makes it all visible and controllable.
    Initial behavior might be like that of a word processor today - a line acquires some extra leading, not specified by the user, when it contains characters that require it.
    But you should be able to right-click anywhere in the line, get a box to pop up that says "This line contains characters requiring 2 points leading above and 1 point leading below" or something like that, and one would then have the option of selecting the whole document to change the linespacing to be that large everywhere, so line spacing would be even.
    Since long descenders are below the text line, and accents above, instead of simply specifying "space these 12 point lines 15 points apart", decomposing leading into leading above the character cell and leading below the character cell allows avoiding clipping or overprinting with a minimum of extra space.

    Oh, dear. I forgot to mention the other strange behavior that might lead to the size of a line unexpectedly growing, even though none of the characters in the line go beyond the character cell implied by the point size.
    Baseline mismatch.
    So if you have a "standard line" font and an "art line" font on the same line - or a "titling" font, since the baselines are kept aligned, the standard line font might be leaded 1 or 2 points below, and the art line font leaded one or two points above (and should probably be replaced by the same font one or two points larger).

    k.l. said:
    3) All that I am saying is that there is no need for more such data in fonts as the existence of such data is based on the assumption that text’s heights can be predicted, which is not the case. (This addresses the chimaera questions too.)

    k.l. said:
    And I definitely did not suggest that line spacing should, or even might, vary from line to line – I always detested the practice of opening up line spacing, above and below individual lines that contain an Arabic word or a mathematical formula, in scientific books. This is really the silly-most thing to put into a typographer’s mouth ...

    I am afraid that I am profoundly unable to understand you here. You appear to be contradicting yourself. If line spacing cannot be predicted, then some lines will end up needing more space than others.
    Unless you intend to give fonts the power to modify global line spacing settings to the largest setting required in a document?

  • John Hudson
    John Hudson Posts: 3,186
    edited December 2022
    The behavior I expect is that if I specify 12 points as the font size, then there will be absolutely no clipping whatever, under any circumstances whatsoever, with a line spacing of 12 points.
    That is a false expectation. Even in metal type it was not unusual for vertical kerns to overhang the body height, even for some styles of European scripts. In digital type, it is the norm rather than the exception, especially when existing fonts are extended to support additional languages, writing systems, symbol sets, etc.

    And so if a Tamil font needs to be set at 36 points to be legible, and I include Tamil text in a mainly Latin document, using a font that supports both languages, and I find that the Tamil text is really tiny... I can highlight it, and change its point size.
    Quite apart from the fact that is simply not how Tamil fonts are made, you are presuming the active engagement of a human typesetter in a text processing environment where that person can highlight text and change point size. That is not how large amounts of the text that people now read on a daily basis is arrived at. This whole conversation is about defaults, about how text is displayed in environments either without additional user intervention, prior to such intervention, or where such intervention is not possible. Read what Bianca wrote:
    I’m finding myself in a position where I’m trying to manage typography at scale across different platforms and geographies as part of a larger design system. What I would like to do is find a way to do that without either compromising the experience (clipping!) or having individual teams overwrite design tokens that define things like the font family, weight, size, line height, etc to accommodate for larger letterforms.
    She describes a real-world—and not uncommon—situation in which avoiding intervention in text processing by teams implementing a typographic design is one of the goals.
  • Chris Lozos
    Chris Lozos Posts: 1,458
    Great thread!
  • Historically they had vertical overlap (undercut) in the 16th, 17th century. Also they mixed fonts with different baselines, e.g. 14 pt for emphasis within 12 pt text. Maybe they puzzled it together with pieces of lead. They also had some sizes as "grobe", meaning something like rugged, unsubtle. This was e.g. a 12 pt cut on a 13 pt body. I guess that it was valued more elegant to have vertical overlap as it was usual in manual calligraphy (this very expensive copies made by monks). 

    Late 19th century they began to harmonise baselines in font families and also had leading variants like 8/9, 8/9.5. Now in modern typography it is usual to have larger vertical whitespace. Look at this webpage: the leading seems to be as high as the x-height.