How to best give access to special characters in a font

Ebern Klause

Is there consensus or best practice on how to best access special characters (like arrows or numbers in circles) in a font?


I couldn’t find anything so I implemented something my own way with contextual alternates:



Numbers in circles can be created with Contextual Alternates enabled and typing a combination of 

w or b (white or black)
# (number sign)

and a digit (0-9)

Arrows can be created with Contextual Alternates enabled and typing a combination of
> (greater than)
with
r, l, u, d
(right, left, up, down)

Everything works fine this way but maybe there are drawbacks that I didn’t foresee?


Cheers, Ebern.

Thierry Blancpain

I’ll respond with the exact same post that I responded a little over two years ago:

We once were young and foolish and thought this was a smart idea, too. So we did it for (c), for TM, and for other things. We still get support requests from the versions we sold years ago with this annoying behavior. It’s a bad idea, in any shape or form.

The discussion gets a little sidetracked but it’s from this thread.

Basically, the argument against any of these is that you have no idea how your fonts are going to get used and adding these kinds of seemingly smart contextual alternates is that it is very, very annoying for your fonts’ users if they suddenly can’t type what they actually wanted to type, and instead get an arrow.

Most of your users will never know about / remember these features. The others can find them in their app’s OpenType features palette. So you’re really building this for expert users who are also using an app without expert features.

Long story short: do this at your own risk. It’s just not worth it from my perspective.

Ebern Klause

Ha thanks Thierry, I hadn’t seen that one.

I get the point, though I feel it’s slightly different in my case, for two reasons.

The copyright sign is is easily typable from the keyboard (MY keyboard that is) so there seems to be little need to put it in the calt feature, and the trigger ‘(c)’ is a combination of letters that is used quite often and means something in itself.

Numbers in circles and arrows can’t be typed and I simply thought it would be very handy if you could. The triggers I implemented mean nothing in themselves and are not very likely to occur in  text. Of course, in theory this can give unwanted results but in practice the chances of that happening are extremely small.

Anyway, thanks, I’ll reconsider.

Johannes Neumeier

I think adding those things as features that are not on by default is okay, but consider this: If you have to document and show users how to type those things, why not instead instruct on how to activate/use unicode input? Or simply instruct users to copy paste those characters from e.g. a specimen.

Erwin Denissen

In our user manual, in the section for automatic OpenType feature generation, we recommend using Alternate Annotation Forms (thus the nalt feature) for digits placed in circles and squares.

John Hudson

Numbers in circles and arrows can’t be typed and I simply thought it would be very handy if you could.

It would indeed be handy to be able to type e.g. ↗, but don't solve keyboard issues at the glyph processing level.

When you type keys on a keyboard, you are inputting character codes, which are what are stored as text in the computer. If you decide for your font to display the character sequence '>ru' as the ↗ glyph, that will look lovely in your font, but that text is going to display '>ru' in everyone else's font. So you have not actually make it possible to type ↗, you've just hidden the fact that what was actually typed was '>ru'.

Input processing, character processing, and glyph processing are separate stages in text processing and display. It is important to understand the correct stage at which to address different kinds of issues.

Nick Shinn

Suggested best practice:

Give them their correct Unicode values, and make sure they are nicely grouped when they appear in a Glyphs palette (position while in Index mode).

The circled numbers are in the Dingbats code sheet. 

So, if someone selects the dingbat from your font, via a Glyphs palette, and it’s not available in some other font, then a circled icon would be available from a well-stocked system font, or good old Zapf Dingbats/Wingbats.

The arrows are in the Arrows code sheet.

You can also put the circled numbers in Stylistic Sets which switch the default figures for duplicate glyphs named as follows:

one.circle, one.negcircle, etc.

sub [one two etc.] by [one.circle two.circle etc.] ;

Ray Larabie

When I was adapting Canada150 to the now bankrupt Mars One project we needed something that could fall back to readable text for safety reasons. The project (Canada1500 Public Domain) required several symbols that don't exist in Unicode. There's no fallback font that could display the information in the case of font failure; symbols such as microgravity, Phobos, laser warning. We used backslash followed by the shortcode followed by a second backslash.

\microgravity\ \phobos\ \laser\

Backslash was chosen as it's more of a programming character than a typographic character and it's available on most keyboards. Of course we don't know how backslash will be used in the future but for that project I think it was the best choice. Multiple characters were considered like {{shortcode}} or [[[shortcode]]] but we were concerned about adding more characters unnecessarily, causing overruns. You can read the PDF on this page to see a list of shortcodes. Most of the symbols have Unicode locations but a lot of them are emoji so possibly not reliable as a fallback anyway. The shortcodes were also meant to serve as an easier way for technicians to input hard-to-find characters. I wouldn't recommend shortcodes for general use but I think they can be useful for specialized purposes. I recommend using shortcodes which are human readable so the meaning can be understood if the shortcode fails.

Ebern Klause

John Hudson said:

So you have not actually make it possible to type ↗, you've just hidden the fact that what was actually typed was '>ru'.

Input processing, character processing, and glyph processing are separate stages in text processing and display. It is important to understand the correct stage at which to address different kinds of issues.

Well explained, thank you John.

I assumed that my font will be employed in the final stage, the design and layout stage, where the above problem is not really a problem. But of course it can also be employed at input stage and then it will cause problems later on.

I will have to update my cheat sheet to clarify this.

Nick Shinn said:

The arrows are in the Arrows code sheet.

Thanks Nick.

Yes, all symbols in my font have correct Unicode values. They show up together in the glyphs palette in Indesign when ‘entire font’ is selected (a lot of scrolling required). When ‘symbols’ is selected only half of them show up while the other half is in ‘math symbols’ which isn’t very convenient. Is there any way to fix this?

Thomas Phinney

Ebern Klause said:

Yes, all symbols in my font have correct Unicode values. They show up together in the glyphs palette in Indesign when ‘entire font’ is selected (a lot of scrolling required). When ‘symbols’ is selected only half of them show up while the other half is in ‘math symbols’ which isn’t very convenient. Is there any way to fix this?

No.

And those few users who know the groupings of the characters they are looking for will look for them under the correct group. Plus, if they are looking at those characters across different fonts, having them shift around between groups when they shift fonts would be seriously problematic!

Ebern Klause

Thomas Phinney said:

No.

Thanks

Ebern Klause

I have updated my cheat sheet like this. Hope that fixes it

(full version here)

Erwin Denissen

I think it is better to avoid calt. Better put the digits in circles in a stylistic set. And maybe add the regular circled ones to nalt. More or less how it is done in Helvetica Now.

Marc Oxborrow

While you're tweaking things, there's a typo in your text: palette, not palet.

Ebern Klause

Erwin Denissen said:

I think it is better to avoid calt. Better put the digits in circles in a stylistic set. And maybe add the regular circled ones to nalt. More or less how it is done in Helvetica Now.

I don’t quite see the advantage of nalt yet so I’m going to study this.

Marc Oxborrow said:

While you're tweaking things, there's a typo in your text: palette, not palet.

Check! Thanks.

John Savard

John Hudson said:

When you type keys on a keyboard, you are inputting character codes, which are what are stored as text in the computer. If you decide for your font to display the character sequence '>ru' as the ↗ glyph, that will look lovely in your font, but that text is going to display '>ru' in everyone else's font. So you have not actually make it possible to type ↗, you've just hidden the fact that what was actually typed was '>ru'.

That is indeed very important to clearly understand.

However, I will add one comment which may partly explain the widespread temptation to use contextual alternates despite the potential danger of confusion.

That is, somebody is going to have to make Microsoft (and, for that matter, Apple) aware that it really has got to be made easier - much easier - for users to address "keyboard issues" of this type.

Ebern Klause

John Savard said:

(...) that it really has got to be made easier - much easier - for users to address "keyboard issues" of this type.

Is this an OS matter or could this potentially be handled through OpenType?

Peter Constable

A fundamental issue is the meaning of text and anticipated usage over its lifetime.

On the one hand: Suppose you're an app developer and need to put some icon—depicting, say, a foobie (whatever)—into the app user interface. You hope your app will enjoy a long lifespan. But throughout that time, the icon will only ever be viewed, and all that matters is that it appear as intended. You can implement it as a character in a text element displayed using some font, and nobody will ever be selecting, editing, searching, copying or re-using that text. So, it really wouldn't matter a whole lot if you created a font in which you re-purposed the "a" character for your foobie icon: the only thing impacted is you, the app developer, and whoever else maintains the app.

But on the other hand: Suppose you're creating a font that will get used by authors creating a variety of content—this seems more like your scenario. That content will have some lifespan, possibly very long, over which people will read, may edit, may search, may copy and re-use, etc. portions of the text. And along the way, the text may well get re-formatted with some other font. In this situation, the semantics of each character entity in the content matters a lot! In order for the text to be robust, to continue working as expected over that lifespan, you really want the encoded representation for each character to follow an enduring convention—the Unicode encoding standard. If you build the font so that, say, the encoded character U+0061 LATIN SMALL LETTER A displays as an arrow or anything other than "a", then people interacting with the content will encounter lots of problems. E.g., they reformat text and it suddenly presents a different meaning; they search for certain terms and encounter lots of false positives, etc.

So, this is why people are responding saying that access to symbols should be handled as a keyboard / text input issue, not as a font rendering issue.

Can this be handled through OpenType? OpenType is a font format for display of Unicode text, and the only special capability OT Layout can offer is to provide a way to support different design variants of a character—e.g., different styles of head on an arrow.

Most OSes have mechanisms for designing custom keyboard layouts. Also, apps for authoring text often provide means for users to enter any Unicode character that's supported in a font; for example, in Microsoft Word, you can enter any character using the Insert | Symbol dialog.

Apps like Word also have auto-replace features that will automatically replace a sequence of characters with a "special" character; e.g., in Word, if you type (c), it will automatically replace that with ©. But a key thing to note is that this feature can be customized by the user. For instance, I have Word automatically replace --- with an em dash for easy entry. So, that's another means that users can use to enter arrows or other symbols they frequently use.

A font developer should certainly think about documenting for customers what Unicode characters are supported in a font. In some situations, it may also be necessary to help customers explore options for easier text input of "special" characters. But in general, it's not a good idea to put into fonts gimmicks that will cause certain Unicode character sequences to display as something entirely different.

Andreas Stötzner

John Savard said:

… somebody is going to have to make Microsoft (and, for that matter, Apple) aware that it really has got to be made easier - much easier - for users to address "keyboard issues" of this type.

Thanks John, you’ve hit the ceiling. It is important to understand that the access-special-char.s conundrum is an text input problem, not a font issue. I also know the temptation to fancy-code workarounds for letting arrows and other symbols come to life, but that is just a desperate measure, by no way a solution.

As a font designer heavily dealing with Hundreds of special characters for a long time, I have been thinking about that mess for many years. And I dare to say I have the solution. It requires a new soft- and hardware design. It’ll be a groundbreaking relieve for myriads of people who suffer from keyboarderitis.

Where do I need to go to make it happen?

Ebern Klause

Any (old) iPad that functions as a giant glyphs-palette or configurable additional keyboard would do for me. Any app-developers in the house?

Nick Shinn

I can just see it now: every time you type a number, a pop-up asks you whether you want to put a circle around it.

André G. Isaak

On the Mac, Popchar X is my preferred solution.

https://www.ergonis.com/products/popcharx/

Perhaps there's something comparable for Windows?

Erwin Denissen

You could use MainType. It is a font manager that also comes as a free edition. It obviously allows you to manage, tag, and install fonts, but it also allows you to search, select, and copy and paste characters. We love (and we made it).

John Savard

Andreas Stötzner said:

And I dare to say I have the solution. It requires a new soft- and hardware design.

I have bad news for you.

Microsoft was going to bring out a new kind of keyboard that would have helped... but it was based on a principle (that of the Optimus Popularis) that was already patented by someone else. And the other possible variations have also been patented...

https://www.artlebedev.com/optimus/

Helmut Wollmersdorfer

Andreas Stötzner said:

As a font designer heavily dealing with Hundreds of special characters for a long time, I have been thinking about that mess for many years. And I dare to say I have the solution. It requires a new soft- and hardware design. It’ll be a groundbreaking relieve for myriads of people who suffer from keyboarderitis.

Where do I need to go to make it happen?

Depends on the context and the skills of the user, how convenient the input can be supported. In an application, onscreen keyboards or character pickers can be supported. What I mostly use with text or document editors is a cheat sheet (a plain text file) containing all special characters including the description. It's a sort of transcription guideline from where I can copy and paste the characters.

Andreas Stötzner

John Savard said:

… the other possible variations have also been patented...

where can one grasp what possible variations you talk about?

Andreas Stötzner

Helmut Wollmersdorfer said:

Depends on the context and the skills of the user, how convenient the input can be supported. …

In my view that is exactly the sort of approach we don’t need. The “skill of the user” is certainly not the problem. The pre-stone-age concept of the keyboard is the problem. What we still use is actually of around 1860. It is overdue. Why are principal stakeholders so stubborn in that respect? (I loath to be bearer of that message.)
Character input could be made possible generally in a way that puts present-day restrictions of the keyboard where it belongs: into the museum. Completely.

NO excuses, please.

John Hudson

A huge amount of work is being done on voice input, with great progress made in natural language processing. But input of special, non-linguistic characters, it seems to me, is always going to be problematic given input methods that are tailored to natural language, whether that's keyboard, voice, or something else. Onscreen pickers are generally best, but there still needs to be a rapid selection mechanism to get to the appropriate picker.

Ray Larabie

It's frustrating because the systems to do this have been baked into computers for decades. When I type Japanese on my computer, there's a system in place to let me choose a specific character, word or phrase. There are function key shortcuts for quick access. There's no technical reason I shouldn't be able to type -- and be presented with a choice on emdash, endash, minus, two hyphens etc. Those kinds of input systems already work in every text box in every application on every PC. I assume it's exactly the same on Mac and Linux.

James Puckett

Ray Larabie said:

I assume it's exactly the same on Mac and Linux.

The Mac keyboard gives access to all the dashes, quotation marks, inverted exclamation and question marks, lots of currency symbols, and some other good stuff. And the ABC extended layout provides access to nineteen diacritical marks. The Character Viewer app allows users to dig through Unicode blocks to find just about anything. Users can install multiple keyboard layouts (I use Latin and Devanagari) and switch between them on the fly. There’s a keyboard viewer app lets you see what characters are available and changes if you use the shift and option keys. It’s not as smooth as Japanese input systems, but it’s still a great way to access stuff in a typeface.

Andreas Stötzner