JavaScript, strictly speaking, ECMAScript, pre-dates Unicode 2.0, so in some cases you may find references to UCS-2 simply because that was correct at the time the reference was written. Can you point us to specific citations of JavaScript being “UCS-2”? Specifications for ECMAScript versions 3 and 5 at least both explicitly declare a String to … Read more
This question is a near-duplicate of 2 others, and surprisingly – while this one is the most recent – I believe it is missing the best answer. The duplicates, and what I believe to be their best answers, are: Using StringWriter for XML Serialization (2009-10-14) https://stackoverflow.com/a/1566154/751158 Trying to store XML content into SQL Server 2005 … Read more
Here we run into the little used locale properties. If you output your string as a string (rather than raw data) you can get the locale to do the appropriate conversion auto-magically. N.B.This code does not take into account edianness of the wchar_t character. #include <locale> #include <fstream> #include <iostream> // See Below for the … Read more
Depends what you mean by ‘support’. You can certainly put non-UCS-2 characters in a JS string using surrogates, and browsers will display them if they can. But, each item in a JS string is a separate UTF-16 code unit. There is no language-level support for handling full characters: all the standard String members (length, split, … Read more
Wow. On the one hand I’m thrilled to know that university courses are teaching to the reality that character encodings are hard work, but actually knowing the UTF-8 encoding rules sounds like expecting a lot. (Will it help students pass the Turkey test?) The clearest description I’ve seen so far for the rules to encode … Read more
Simply use the “utf-8-sig” codec: fp = open(“file.txt”) s = fp.read() u = s.decode(“utf-8-sig”) That gives you a unicode string without the BOM. You can then use s = u.encode(“utf-8”) to get a normal UTF-8 encoded string back in s. If your files are big, then you should avoid reading them all into memory. The … Read more
I’ve been struggling with this problem for a while, and just discovered (for me) a perfect solution: $ git config –global diff.tool vimdiff # or merge.tool to get merging too! $ git difftool commit1 commit2 git difftool takes the same arguments as git diff would, but runs a diff program of your choice instead of … Read more
UTF-8 has an advantage in the case where ASCII characters represent the majority of characters in a block of text, because UTF-8 encodes these into 8 bits (like ASCII). It is also advantageous in that a UTF-8 file containing only ASCII characters has the same encoding as an ASCII file. UTF-16 is better where ASCII … Read more
Don’t worry about that. According to the same information source: this library component should be retired to Annex D, along side , until a suitable replacement is standardized. So, you can still use it until a new standardized, more-secure version is done.
Java uses UTF-16 for the internal text representation The representation for String and StringBuilder etc in Java is UTF-16 https://docs.oracle.com/javase/8/docs/technotes/guides/intl/overview.html How is text represented in the Java platform? The Java programming language is based on the Unicode character set, and several libraries implement the Unicode standard. The primitive data type char in the Java programming … Read more