Taming the Unicode terminal

For the past six months or so, I’ve been on a quest for the perfect graphical Linux terminal, and it’s led me in some odd directions. The thing is, I already found the perfect terminal (perfect for me, that is): mrxvt. I’ve been using mrxvt for about five years now, and it’s been my favorite for several reasons:

  • It’s insanely fast
  • It’s easy to customize
  • It has tabs, and you can put them at the bottom

About my only gripe has been the lack of Zmodem support. Call me crazy, but I still like to use Zmodem from time to time. The only terminals that still do Zmodem are SecureCRT and old versions of Konsole; since KDE 4’s Konsole regressed, this feature no longer exists.

I had little reason to switch to another terminal until the day I realized that UTF-8 is here to stay, and it’s something a terminal needs to do. I fought this for a long time, but ultimately I got tired of setting LANG=C in all my profiles and jumping through hoops to turn off what is essentially progress. The problem is that mrxvt does not support Unicode, and it probably never will.

What are my other options? I tried many:

  • xterm and rxvt have unicode options, but no tabs.
  • xfce4-terminal is actually quite close to what I want, but it won’t let me put the tabs at the bottom.
  • gnome-terminal is slow, feature-poor, hard to configure, and also won’t let me put the tabs at the bottom.
  • urxvt looks promising, but the tab support is text, not graphical, and needs some UI work. (I hear there’s a GTK version, but it doesn’t work very well and lacks the keyboard support I depend on.)
  • Konsole is probably the best terminal available for Linux, but it’s slow, clunky, and I can’t customize the keyboard shortcuts the way I want. I like how it remembers what sessions I had open last, but I hate not being able to assign hotkeys – with mrxvt, I type Ctrl-Shift-Enter, type a host name, and get an instant ssh connection with the host name as the tab title. I can’t seem to get this effect with Konsole.

I’ve tried others, but I can’t remember them anymore. I tried everything available through Debian and a few others, and wasn’t happy with any of them. I even tried making my own terminal using Python and libvte, and at one point considered that a viable option, but ultimately gave up because I saw how much work I had to do to get decent terminal emulation (I use a lot of keyboard shortcuts in emacs), and besides, VTE just feels kind of slow (which is why gnome-terminal also feels slow).

After flirting with Konsole for a few months (hey, at least it motivated me to try out KDE 4, which is awesome), I got tired of the slowness and lack of keyboard customizations and I switched back to mrxvt. I started looking for other ways to get Unicode to work, and somehow I stumbled upon GNU Screen, which I had been ignoring up to that point, since mrxvt’s tabs left me little reason to add another layer of virtual screen support.

However, GNU Screen had, this whole time, been a workable solution, and I was just completely unaware of what it could do. I don’t think many people know that this is even possible, since it’s hard to find any conversations on the web about it, but screen can emulate UTF-8 on a latin-1 terminal, and I find this completely amazing. Here’s my simple .screenrc configuration:

defbce on
defutf8 on
escape ^]^]
markkeys "h=^B:l=^F:$=^E"
setenv LANG en_US.UTF-8
startup_message off
term $TERM
termcapinfo xterm|xterms|xs|rxvt ti@:te@
zmodem catch

Here’s what it does:

  • “defutf8 on” turns on UTF-8 support, including translation to the parent terminal’s encoding!
  • “defbce on” turns on “background color erase”, which fixes a problem where status-line background colors don’t go all the way across the screen in various full-screen console programs
  • “escape ^]^]” moves the escape key from the default of Ctrl-A to Ctrl-], which is the best compromise I could come up with since my brain is completely wired to use Ctrl-A to go to the beginning of the line (emacs, zsh, etc.)
  • The “markkeys” line I don’t really use much anymore, now that I got mrxvt’s scrollback buffer working again, but it lets me use the other emacs keys to navigate screen’s scroll/copy buffer
  • “setenv LANG en_US.UTF-8″ sets the LANG environment variable to indicate that I want UTF-8 encoding. This is essential because, from mrxvt, I have LANG=C so that screen knows my terminal doesn’t support UTF-8.
  • “startup_message off” skips the startup screen, since I use screen for every single tab, and this would be totally redundant.
  • “term $TERM” uses the TERM setting from the parent terminal rather than letting screen overwrite it as TERM=screen or TERM=screen-bce. I actually use “xterm” as my TERM type, even though I use an rxvt-derived terminal, because this has given me the best compatibility with the various console programs I use. I’ve spent many hours fixing keyboard escape codes so that mrxvt is xtermy enough.
  • The “termcapinfo …” line tells screen not to use the “alternate screen” for anything, which is necessary for mrxvt’s scrollback buffer and corresponding keyboard and mouse support to work.
  • Last but not least, “zmodem catch” enables–holy shit–Zmodem support! I had pretty much already given up on Zmodem, after having bad experiences with zssh, but screen’s Zmodem support really works! Supposedly it’s experimental and crashes sometimes, but so far I haven’t had any major problems with it.

That’s it for my screen setup so far. I made the following change to my .mrxvtrc to use screen for every tab and make sure that LANG=C before screen runs (which screen will later set to en_US.UTF-8):

Mrxvt.command: \!LANG=C exec screen
Mrxvt.macro.Ctrl+Shift+Return: NewTab "ssh" \!read -p"Host: " host; echo -ne "\e]0;$host\a"; LANG=C exec screen ssh $host

The first line is the only one necessary to start using screen for all new tabs. The second one is my handy shortcut for opening up ssh sessions quickly. It really helps when I need to log onto a bunch of boxes at once.

I got all of this working and ran with it for a couple of weeks. Everything was great except for one minor issue, which wasn’t really a big deal, but I came up with a fix for that, too. The issue was that certain Unicode characters were showing up as ‘?’. After some investigation I learned that these characters did not have corresponding characters in latin-1. They included bullets, various dashes/hyphens, and curly single- and double-quotes. I don’t really care if my quotes are curly, but it is distracting when they show up as question marks, so I started digging through the screen manual trying to find a solution. I couldn’t find any way to customize the replacement character it chooses, so I finally downloaded the screen source and started hacking on it. I wrote a little patch that does the replacements I want, and figured I might as well try doing a fork on github so I can keep up with future changes to screen. Here is the commit.

As it turns out, the latest version from github, which I think is a mirror of the CVS trunk, once again kills mrxvt’s scrollback buffer support, and after messing around with it for awhile I gave up and went back to the latest screen source package from Debian unstable, applying my patch to encoding.c and running “debian/rules binary” to build a custom .deb package. That is what I am using now, and it’s working just fine!

All this, and I still haven’t really learned how to use screen for its intended purpose. I still don’t really need the virtual screens, but I’m intrigued by the bind, bindkey, and stuff commands, which seem to be a general-purpose keyboard macro tool. I’m looking forward to learning more about what screen can do.

5 thoughts on “Taming the Unicode terminal”

  1. Thanks a lot for this! Helped me get finch working inside of screen in mxrvt (my favourite terminal!) when all else had failed. Sweet.

  2. You could also use one of the terminal emulators without tab support and have your window manager do the tabbing. Fvwm, Fluxbox, wmii, Ion, possibly others and even KWin (which you use if you still use KDE) support this.

  3. Wow, thanks for that. No UT8 in mrxvt has been driving me nuts for a while now. I’ve put up with it ’cause the benefits outweighed the annoyance.

    Now mutt looks good again. Hooray!

  4. Hi, first of all thank you;
    unfortunately i have some problems, i don’t get unicode;
    steps i’ve done:
    edit ~/.screenrc as you told
    added Mrxvt.command: \!LANG=C exec screen to ~/.mrxvtrc
    start mrxvt
    enter screen
    no unicode :/

    did i miss something?

    my locale output in screen:
    locale
    LANG=en_US.UTF-8
    LANGUAGE=
    LC_CTYPE=”en_GB.UTF-8″
    LC_NUMERIC=”en_GB.UTF-8″
    LC_TIME=”en_GB.UTF-8″
    LC_COLLATE=”en_GB.UTF-8″
    LC_MONETARY=”en_GB.UTF-8″
    LC_MESSAGES=”en_GB.UTF-8″
    LC_PAPER=”en_GB.UTF-8″
    LC_NAME=”en_GB.UTF-8″
    LC_ADDRESS=”en_GB.UTF-8″
    LC_TELEPHONE=”en_GB.UTF-8″
    LC_MEASUREMENT=”en_GB.UTF-8″
    LC_IDENTIFICATION=”en_GB.UTF-8″
    LC_ALL=en_GB.UTF-8

    maybe you know something
    thanks :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">