I recently received this (not the first time) in Mail.app on MacOS 11.4: On Jun 24, 2021, at 10:02, web host wrote: "Head Geek (伟思ç��)" wrote: This is curious. I have everything on this end set to use UTF-8 output and to assume UTF-8 if the encoding header is missing. (Unless Apple changed it during an update as they sometimes do). Why would the first two *hanzi* be rendered correctly and the third use ISOLatin1?  Is it a bug I should report, and to which (Apple or web host)? It looks correct in the Sent folder. It also looks correct in the To: line of the headers on the same message that has the error in the body. It has not happened in recent replies for other places, so it's unlikely to be an Apple bug. Yet if the encoding header is missing, odd that the headers are correctly rendered differently.

This is a question in the continuation of this one: UTF8 filenames and shell utilities : the nightmare of utf-8 and filenames is…. How to avoid this confusion with most notably the ls command on filnames created in the following way: $ echo $LANG en_US.UTF-8 $ mkdir test $ cd test $ touch á $ touch a´ $ The first file is created by typing alt ⌥+E, A, the second one by typing A, alt ⌥+shift ⇧+E Within an xterm window I get: $ ls -al total 0 drwxr-xr-x 2 hal admin 136 Nov 11 15:52 . drwxr-x--- 7 hal admin 4386 Nov 11 14:25 .. -rw-r--r-- 1 hal admin 0 Nov 11 15:52 a´ -rw-r--r-- 1 hal admin 0 Nov 11 15:48 a´ $ Which is wrong. Within a Terminal window I get: $ ls -al total 0 drwxr-xr-x 2 hal admin 136 Nov 11 15:52 . drwxr-x--- 7 hal admin 4386 Nov 11 14:25 .. -rw-r--r-- 1 hal admin 0 Nov 11 15:52 a´ -rw-r--r-- 1 hal admin 0 Nov 11 15:48 á $ Which is right. The result is the same whichever shell I use: bash or zsh. What xterm or XQuartz setting will avoid these 2 filenames to falsely look the same? This problem doesn't exist on Mavericks (MacOS X 10.9).

I am downloading a CSV file, which Safari (annoyingly) puts directly onto the page. This CSV file is encoded with UTF-8, and has been verified to be correctly encoded with UTF-8 from the source file, as well as downloading it using other browsers. However, in Safari the character set is incorrect. It is using a Default character set, as seen by looking in the menu:

View > Text Encoding > Default

If I change this to:

View > Text Encoding > Unicode (UTF-8)

I get the correct characters displayed. Why is UTF-8 not the default? How can I force it to be the default? Isn't UTF-8 the default for the whole MacOS operating system, so I have no idea why Safari would be using something different.

I cannot get Mac Terminal to display U+200C, aka zero-width non-joiner , correctly. Instead, it simply types ``. Unicode settings for the profile I use have been set to UTF-8 as described on this Apple Support page. I am also using a font which has a glyph for this character, as I can test it in Font Book. Any ideas on how to get Terminal to display this character properly? In case you want to try it, use the below text and see if it appears in your terminal as in the webpage.

می‌روم

(Persian word meaning I'm going). P.S. Let me know if you happen to know another terminal on any other platform which might correctly display this character.

How to make iterm2 to properly display Chinese/Utf8? Say I want to execute these commands under iterm2:

touch Hello世界.txt
touch Hello老师我想你.mp3
touch Hello❤️🌹🎉.txt

and this is what I'm getting now: Note that - The [Terminal app](https://support.apple.com/lt-lt/guide/terminal/trml15220/mac) works out of the box for me without any of my tweaking: - The created Hello世界.txt file name show up in my file finder just properly. It is iterm2 that's causing all these problems: This the setting I'm using based on the following searches: - https://logs.paulooi.com/how-to-view-chinese-character-with-iterm-in-mac-osx.php - https://iterm2.com/3.3/documentation-preferences-profiles-text.html - https://support.apple.com/en-ca/guide/pages/tanfbd4156e/mac - https://www.help.k12.com/s/article/Viewing-Chinese-Characters-on-a-Mac - https://chinesemac.org/pages/fonts.html

There used to be support for an encoding option called utf-8-mac with iconv in the Terminal. It's mentioned in this forum several times. I used to use it a lot. Now with the following command (that used to work):

iconv -f utf-8-mac -t CP1252 infile.m3u > outfile.m3u

I get:

iconv: conversion to utf-8-mac unsupported

I tried an encoding called mac but that does not work either. A clumpsy workaround in my case is to open in TextEdit, duplicate and save using Windows Latin 1 encoding. But I'd really prefer using iconv, but I don't know what to look for and there are many encodings. Is there an equivalent to utf-8-mac? Currently I use: macOS 10.14.6

When I google this I can't find it. But sometimes when I'm in the console and I'm fiddling around a certain key combination triggers a dialog to appear. In the dialog you can choose an icon to insert. Just like the icons you can use in for example whatsapp. I tried al lot of combinations but I can't seem to find the right pair. The three times it happend it happend on accident. What is the key combination to open this icon dialog?

I made a script to rename mp3 files based on the tags, basically it extracts the data like this:

sArtist=$(exiftool -s3 -Artist "$sFile")
...
sNewName="$sArtist - $sTitle"
mv -v "${sFile}" "${sNewName}.${sFileExt}"

The problem is with the diacritics, for example in the artist name, the character á in the filename is saved as unicode aU+0301 while the expectation is that it should be saved as U+00E1 Any idea why? Here is the output of env:

$ env
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/bin/zsh
TMPDIR=/var/folders/c5/38kntdc107jbxmc_8sg8_qfc0000gn/T/
TERM_PROGRAM_VERSION=445
TERM_SESSION_ID=BCFA74E2-4B36-447A-8DE8-4598EFB89474
USER=******
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.HuUuhLPn6d/Listeners
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:/opt/X11/bin:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
_=/usr/bin/env
__CFBundleIdentifier=com.apple.Terminal
PWD=/
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
HOME=/Users/******
SHLVL=2
LOGNAME=******
LC_CTYPE=UTF-8
DISPLAY=/private/tmp/com.apple.launchd.omquYUiZVj/org.macosforge.xquartz:0

***Is there an environmental setting that makes pasting text into Terminal work differently from typing it in directly?*** I reported this as a bug, but maybe there's some environmental setting.  After the first failure, I put the command in a script to send them as a demo, but it worked when I ran the script. The command is echo "ΕΡΤΥΙΟΚΗΑΖΧΒΝΜ" | tr [:upper:] [:lower:] The string is Greek letters typed with the keyboard set to Greek.  Behavior is the same whether I use the keys and fingers or the Keyboard Viewer and mouse. locale is en_US.UTF-8 and TextEdit is set to always use UTF-8. I put the command in TextEdit, copied it to clipboard, and pasted into Terminal (with stdin containing the Greek string). Result: zsh: no matches found: [:upper:] If I swap the classes, I get zsh: no matches found: [:lower:] If I save the file from TextEdit, give it a chmod 700, and run it, it does what it should, no error message. If I change the command to convert Greek to Latin with echo "ΕΡΤΥΙΟΚΗΑΖΧΒΝΜ" | tr "ΕΡΤΥΙΟΚΗΑΖΧΒΝΜ" "EPTYIOKHAZXBNM" (instead of using classes), then it works pasted or in script. If I type echo "" | od -xc and paste the Greek between the quote marks, I see that the characters are not ASCII. So Paste from clipboard must be doing something **other than** changing the characters. NOTES: (1) the tr command fails **only** when pasted and it contains neither = nor ?, so escaping those is not a solution, and this is this is not a duplicate of https://apple.stackexchange.com/questions/254014/zsh-no-matches-found (2) It's plausible that zsh might treat [:xxx:] differently from bash, but what we have here is zsh/Terminal treating pasted text differently from typed in text.

I've noticed an anomaly in ID3v2 tags created by Music.app.  It doesn't affect usability at all, at least not on my system. But it's odd and might affect how someone else sees the same file. I have files in several languages, including Greek and Chinese.  So I use Music’s Get Info pop-up to edit Name, Album, Artist, Lyrics, etc.  They all render correctly, including the non-ASCII characters, in Music.  In Terminal, with locale set to "en_US.UTF-8," exiftool also shows them correctly. I also edit Grouping for sorting factors nothing else in Music provides, including language.  There are NO (zero) non-ASCII characters in any Grouping tag.  Yet exiftool tells me there is "something" (indicated by a dot) at the beginning of every Grouping tag.  Using exiftool (file) | od -xc reveals that it is a byte-order-marker (BOM). Will the **lack** of a BOM on other tags that are UTF-8 cause problems if the file is given to someone with a different CPU/O.S. ? (I know they are UTF-8.)

Notice the nonexistent space displayed in the first output. I've sent this to Apple as a bug, but **is there anything the user (or admin) can do to fix it?** Locale is en_US.UTF-8 The worst part of the bug is that it screws up command-recall-and-edit. Any non-ASCII character in the command has a bogus space displayed after it. But the space is counted when the cursor passes it, resulting in the edit position not being where the cursor is. If I copy a line with such a bogus space and paste it in TextEdit, the space goes away. So I don't think it's a bug in the shell. Also, it happened in both bash and zsh.

I have used **rsync** for many years between linux filesystems without any problem with a very deterministic behavior. Now I try to use it on mac and experience some problem that some files (a small proportion of my 600000 files) gets copied over and over. Here is what I tried: 1. Moving from MacOS rsync 2.6.9 to rsync 3.1.3 (from brew) did not solve the problem. 2. Giving some large time interval in order to avoid possible clock skew did not resolve the problem. 3. I saw in some other answers that a possible problem is the *utf-8* vs *utf-8-mac* characters. I understand that it can be a problem but I am ready to accept ascii filenames for the purpose of having linux and mac coexisting. The files in question have ASCII file names. The weird think about it is that this unexpected behavior is completely deterministic. The same file **ListDebug/ForDEBUG** gets copied over and over. Yet the file **ListDebug/ForDEBUG2** which is next to it was not copied over and over. Any indication on the origin of this strange behavior would be much welcomed. EDIT: I found some more info when copying only the directory **ListDebug**. When I run **rsync -vadi -e ssh remote:ListDebug .** I get mathieu@MacBook-Pro: rsync -vadi -e ssh remote:ListDebug . >f.st......... ListDebug/ForDebug mathieu@MacBook-Pro: rsync -vadi -e ssh remote:ListDebug . >f.st......... ListDebug/ForDEBUG mathieu@MacBook-Pro: rsync -vadi -e ssh remote:ListDebug . >f.st......... ListDebug/ForDebug Thus there is some oscillation between one print and another. That really sounds strange and a bug. EDIT2: The file **ListDebug/ForDebug** gets copied but become named **ListDebug/ForDEBUG** and the file **ForDEBUG** never gets copied. EDIT3: If I change the content of ForDebug/ForDEBUG to something standard like TEST1, TEST2 then the bug remains. On the other hand if I rename the files ForDebug/ForDEBUG to file1/file2 then the bug disappear. Ve

I was just wondering if anyone knows the emoji support that the Terminal.app has in MacOS? I'm using zsh, and I end up with this when I type it out, I get the following (note I didn't add a space after the flag on the first one, but added a space on the second one but in both cases you can see there's an overlap on the flag, also note the duplication on the spaces and f on the end of the input) And when using vim I also get How do people resolve this?

What the difference and usage of encodings UTF-8 and UTF-8-MAC in iconv? I thought it was the difference between \n and \r(MAC OS 9) at first. But I tried iconv -f UTF-8 -t UTF-8-MAC filename > filename2 The file content doesn't change in hex view.

I require Polish diacritics, such as ą, ć, ę, etc. Of course, I can always pre-process the file to replace UTF-8 characters with escaped characters but that's not elegant.

I use Mac with English MacOS interface. I download a file from Chinese website(Simplified Chinese). It is garbled file name. The garbled file name: > %E6%B9%96%E5%8D%97IPTV.apk The real file name: > 湖南IPTV.apk I know the real new because of I know the file where I download. I know there is a command "iconv" on MacOS. But, I don't know the proper encoding type. The command "iconv -l" shows up lots of type. It lists all of them. For an instance, I run "iconv -f GB_2312-80 -t UTF-8 orig.txt > new.txt". This means I convert a file with GB_2312-80 format to UTF-8. However, I don't know why the file downloaded from the site has a "garbled" file name on my system. I want to know this garbled file's real character format. **This is my question.** Furmore, according to the below list from "iconv -l" this command, what are those possible "Simplified Chinese file name format type" as below? **If you know it.** Should I try every format which is possibly relative to answer format? It seems a tedious and slow way to do. In addition, does it have any matter on my MacOS's character format? **If you know it.** > ANSI_X3.4-1968 ANSI_X3.4-1986 ASCII CP367 IBM367 ISO-IR-6 ISO646-US > ISO_646.IRV:1991 US US-ASCII CSASCII UTF-8 ISO-10646-UCS-2 UCS-2 > CSUNICODE UCS-2BE UNICODE-1-1 UNICODEBIG CSUNICODE11 UCS-2LE > UNICODELITTLE ISO-10646-UCS-4 UCS-4 CSUCS4 UCS-4BE UCS-4LE UTF-16 > UTF-16BE UTF-16LE UTF-32 UTF-32BE UTF-32LE UNICODE-1-1-UTF-7 UTF-7 > CSUNICODE11UTF7 UCS-2-INTERNAL UCS-2-SWAPPED UCS-4-INTERNAL > UCS-4-SWAPPED C99 JAVA CP819 IBM819 ISO-8859-1 ISO-IR-100 ISO8859-1 > ISO_8859-1 ISO_8859-1:1987 L1 LATIN1 CSISOLATIN1 ISO-8859-2 > ISO-IR-101 ISO8859-2 ISO_8859-2 ISO_8859-2:1987 L2 LATIN2 CSISOLATIN2 > ISO-8859-3 ISO-IR-109 ISO8859-3 ISO_8859-3 ISO_8859-3:1988 L3 LATIN3 > CSISOLATIN3 ISO-8859-4 ISO-IR-110 ISO8859-4 ISO_8859-4 > ISO_8859-4:1988 L4 LATIN4 CSISOLATIN4 CYRILLIC ISO-8859-5 ISO-IR-144 > ISO8859-5 ISO_8859-5 ISO_8859-5:1988 CSISOLATINCYRILLIC ARABIC > ASMO-708 ECMA-114 ISO-8859-6 ISO-IR-127 ISO8859-6 ISO_8859-6 > ISO_8859-6:1987 CSISOLATINARABIC ECMA-118 ELOT_928 GREEK GREEK8 > ISO-8859-7 ISO-IR-126 ISO8859-7 ISO_8859-7 ISO_8859-7:1987 > ISO_8859-7:2003 CSISOLATINGREEK HEBREW ISO-8859-8 ISO-IR-138 > ISO8859-8 ISO_8859-8 ISO_8859-8:1988 CSISOLATINHEBREW ISO-8859-9 > ISO-IR-148 ISO8859-9 ISO_8859-9 ISO_8859-9:1989 L5 LATIN5 CSISOLATIN5 > ISO-8859-10 ISO-IR-157 ISO8859-10 ISO_8859-10 ISO_8859-10:1992 L6 > LATIN6 CSISOLATIN6 ISO-8859-11 ISO8859-11 ISO_8859-11 ISO-8859-13 > ISO-IR-179 ISO8859-13 ISO_8859-13 L7 LATIN7 ISO-8859-14 ISO-CELTIC > ISO-IR-199 ISO8859-14 ISO_8859-14 ISO_8859-14:1998 L8 LATIN8 > ISO-8859-15 ISO-IR-203 ISO8859-15 ISO_8859-15 ISO_8859-15:1998 > LATIN-9 ISO-8859-16 ISO-IR-226 ISO8859-16 ISO_8859-16 > ISO_8859-16:2001 L10 LATIN10 KOI8-R CSKOI8R KOI8-U KOI8-RU CP1250 > MS-EE WINDOWS-1250 CP1251 MS-CYRL WINDOWS-1251 CP1252 MS-ANSI > WINDOWS-1252 CP1253 MS-GREEK WINDOWS-1253 CP1254 MS-TURK WINDOWS-1254 > CP1255 MS-HEBR WINDOWS-1255 CP1256 MS-ARAB WINDOWS-1256 CP1257 > WINBALTRIM WINDOWS-1257 CP1258 WINDOWS-1258 850 CP850 IBM850 > CSPC850MULTILINGUAL 862 CP862 IBM862 CSPC862LATINHEBREW 866 CP866 > IBM866 CSIBM866 CP1131 MAC MACINTOSH MACROMAN CSMACINTOSH > MACCENTRALEUROPE MACICELAND MACCROATIAN MACROMANIA MACCYRILLIC > MACUKRAINE MACGREEK MACTURKISH MACHEBREW MACARABIC MACTHAI HP-ROMAN8 > R8 ROMAN8 CSHPROMAN8 NEXTSTEP ARMSCII-8 GEORGIAN-ACADEMY GEORGIAN-PS > KOI8-T CP154 CYRILLIC-ASIAN PT154 PTCP154 CSPTCP154 KZ-1048 RK1048 > STRK1048-2002 CSKZ1048 MULELAO-1 CP1133 IBM-CP1133 ISO-IR-166 TIS-620 > TIS620 TIS620-0 TIS620.2529-1 TIS620.2533-0 TIS620.2533-1 CP874 > WINDOWS-874 VISCII VISCII1.1-1 CSVISCII TCVN TCVN-5712 TCVN5712-1 > TCVN5712-1:1993 ISO-IR-14 ISO646-JP JIS_C6220-1969-RO JP > CSISO14JISC6220RO JISX0201-1976 JIS_X0201 X0201 CSHALFWIDTHKATAKANA > ISO-IR-87 JIS0208 JIS_C6226-1983 JIS_X0208 JIS_X0208-1983 > JIS_X0208-1990 X0208 CSISO87JISX0208 ISO-IR-159 JIS_X0212 > JIS_X0212-1990 JIS_X0212.1990-0 X0212 CSISO159JISX02121990 CN > GB_1988-80 ISO-IR-57 ISO646-CN CSISO57GB1988 CHINESE GB_2312-80 > ISO-IR-58 CSISO58GB231280 CN-GB-ISOIR165 ISO-IR-165 ISO-IR-149 KOREAN > KSC_5601 KS_C_5601-1987 KS_C_5601-1989 CSKSC56011987 EUC-JP EUCJP > EXTENDED_UNIX_CODE_PACKED_FORMAT_FOR_JAPANESE CSEUCPKDFMTJAPANESE > MS_KANJI SHIFT-JIS SHIFT_JIS SJIS CSSHIFTJIS CP932 ISO-2022-JP > CSISO2022JP ISO-2022-JP-1 ISO-2022-JP-2 CSISO2022JP2 CP50221 > ISO-2022-JP-MS CN-GB EUC-CN EUCCN GB2312 CSGB2312 GBK CP936 MS936 > WINDOWS-936 GB18030 ISO-2022-CN CSISO2022CN ISO-2022-CN-EXT HZ > HZ-GB-2312 EUC-TW EUCTW CSEUCTW BIG-5 BIG-FIVE BIG5 BIGFIVE CN-BIG5 > CSBIG5 CP950 BIG5-HKSCS:1999 BIG5-HKSCS:2001 BIG5-HKSCS:2004 > BIG5-HKSCS BIG5-HKSCS:2008 BIG5HKSCS EUC-KR EUCKR CSEUCKR CP949 UHC > CP1361 JOHAB ISO-2022-KR CSISO2022KR

I've set the locale: $locale LANG="en_GB.UTF-8" LC_COLLATE="en_GB.UTF-8" LC_CTYPE="en_GB.UTF-8" LC_MESSAGES="en_GB.UTF-8" LC_MONETARY="en_GB.UTF-8" LC_NUMERIC="en_GB.UTF-8" LC_TIME="en_GB.UTF-8" LC_ALL= However, while this mount works: $ mount -t smbfs "//Mercury/Echo" "/Volumes/Echo" Password for Mercury: This mount fails: $ mount -t smbfs "//Προκρούστης/DiningRoom" "/Volumes/DiningRoom" mount_smbfs: URL parsing failed, please correct the URL and try again: Invalid argument Any suggestions as to how to fix this?

I have been setting up my new terminal on both of my MacBooks Pro and I noticed that UTF-8 glyphs are way smaller on one of them. This holds true in every application and I can't find the setting that differs between the two laptops so I can get the big glyphs on both. Here are two pictures showing the difference: I thought it would be related to the locale variables but no: LANG="fr_FR.UTF-8" and LC_ALL="fr_FR.UTF-8". Any help is welcome! PS: I work on Mac OS Sierra

how could I change the file encoding using OS X? If I open a file on Windows Notepad I see all the symbols and everything is fine. If I open it using TextEdit or TextWrangler I see only question marks instead of the normal symbols. I tried to save the file as plain text using UTF-8 unicode, but it didn't changed. Any more ideas?

**Edit: I should have mentioned in my original question that I was using MacPorts SSH client, as it turned out there were two root causes. See my own answer below.** When I SSH into my OSX machine, it's not setting the locale properly: LANG= LC_COLLATE="C" LC_CTYPE="C" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL= This should be using a UTF-8 locale. It should look like this: LANG="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_CTYPE="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_ALL= I'm running OSX 10.11.6 (El Capitan), and I've enabled the SSH server in System Preferences. I'm using putty as a client on my Windows machine. When I use the same putty configuration to connect to a Linux server, the locale information is fine. When I run a local terminal on the OSX machine, locale is fine. If I then ssh localhost, I have the same problem: no locale is set. It seems that my OSX SSH server is ignoring or discarding the locale information from the client. I have the following in my (unmodified) /etc/ssh/sshd_config: AcceptEnv LANG LC_* To the best of my knowledge, that's all that's required. My putty configuration is sending TERM=xterm-256color, which is being honored; it's only the locale information that isn't working, and to reiterate, it happens when I ssh localhost from OSX to itself, so I don't think putty is the issue here. **I will not accept answers that suggest setting the locale in my .profile; there are plenty of answers on here suggesting that, but it's a hack, not a solution.**