[pianotech] New Forum

[John Delacour]

At 21:13 -0400 29/06/2011, Ron Berry wrote:

>Apostrophe issue
>
>The mangling of apostrophes in Don Mannino's post was not just a 
>Higher Logic issue. It is an HTML issue.

Ron, this is nothing to do with HTML, it is a question of character 
sets. I won't bore the list with too many technical details but from 
tests I did several weeks ago it appears that the online site is 
internally based on Unicode UTF-8 encoding.  If you write 
non-US-ASCII characters in the online composition window then people 
who read only within the forum site will see them properly displayed, 
but when those messages are sent out via email the UTF is sent with 
"raw bytes" and no encoding.  Incoming postings from email are 
treated as though they had no content-type header and so the raw 
bytes are further "encoded" from garbage into double garbage and so 
on until a single 8-bit character can grow to a great long string of 
garbage characters.

Every email that you send has a Content-Type header in which the 
character set used is declared.  You will see this if you have a 
mailer that allows you to view the source of the message and the 
(usually) hidden headers.  This message is sent as UTF-8 from Eudora 
on the Mac.

When I include characters such as °, ’, £, €, —, which are 
not in the US-ASCII character set, my mailer will encode these 
characters in a character set that includes all these characters, and 
in 2011 that ought to be Unicode's UTF-8 transformation.  This 
transformation will use two bytes to encode the first 4 and three 
bytes to encode the m-dash.

I have a lot of experience dealing with this question since I have 
worked with Chinese and the Internet since 1992 and also studies the 
question in relation to Eudora on the Mac, the mailer I have always 
used.
My article at <http://bd8.com/eudora/multilingual/> explains a little 
of how UTF-8 encoding works.

These questions ought to be second nature to anybody who is writing 
forum or list software.  The fact that they have completely screwed 
up on this basic issue is evidence enough that the $57, $75, ... who 
knows what final rip-off, has been money thrown down the drain.

I am copying this message to Laura Olsen, who I see appears from my 
archive NEVER in the past ten years to have participated in the 
original Pianotech list!

I would also remind list members that the Pianotech group I set up on 
Google some weeks ago is still available for use and that it cost 
NOTHING to set up, works perfectly and requires no maintenance:

<http://groups.google.com/group/pianotech.php/>

JD