What is an in­ter­na­tion­al­ized domain name (IDN)?

According to the in­ter­na­tion­al telecom­mu­ni­ca­tions union (ITU), more than three billion people use the World Wide Web and in­creas­ing­ly so in their mother tongues. This change was in part brought on by the in­tro­duc­tion of in­ter­na­tion­al domain names in 2003. We’ll explain how IDN domains work.

What is an in­ter­na­tion­al­ized domain name (IDN)?

The IETF (Internet En­gi­neer­ing Task Force) refers to IDNs as domain names that contain special char­ac­ters that are not part of the Latin alphabet, such as umlauts or char­ac­ters from other alphabets. However, the Domain Name System (DNS), which is re­spon­si­ble for trans­lat­ing URLs into IP addresses, cannot un­der­stand these domain names. The DNS is based on the limited standard character set ASCII.

In order to make IDNs un­der­stand­able for the DNS as well as other internet protocols, the internet standard In­ter­na­tion­al­iz­ing Domain Names in Ap­pli­ca­tions (IDNA) was created in 2003. This defines a stan­dard­ized trans­la­tion from Unicode to ASCII, therefore enabling the use of non-ASCII char­ac­ters in domain names.

How does IDNA work?

Much of the internet’s in­fra­struc­ture is only supported by the ASCII character set. In order to make sure that in­ter­na­tion­al domain names can be processed, each IDN that’s available in Unicode is trans­lat­ed into an ACE string, which is based on ASCII. Following this, URLs featuring char­ac­ters with accents or umlauts are displayed. The server, on the other hand, continues to process the addresses as ASCII com­pat­i­ble. This procedure is specified in the IDNA2003 internet standard and in the IDNA2008 revision, which was approved in 2010. Trans­lat­ing from Unicode to ASCII occurs client-side (in the browser, email program, etc.) and is based on a stan­dard­ized coding process called Punycode.

Punycode

The RFC 3492-stan­dard­ized Punycode was developed for clearly dis­play­ing Unicode character strings as ASCII symbols without loss of quality. All non-ASCII char­ac­ters are removed from the domain name, encoded and separated with a hyphen. This code sequence contains in­for­ma­tion about the Unicode symbol in question as well as its position in the domain name. Ad­di­tion­al­ly, each ACE string created in this way is labeled with the prefix xn–. This clarifies to the reader that the character sequence is an IDN that has been encoded according to IDNA and Punycode standards. See our article on Punycode for a detailed ex­pla­na­tion of the encoding process as well as some examples.

Dif­fer­ences between IDNA2003 and IDNA2008

For the original 2003 procedure, in­ter­na­tion­al­ized URLs were nor­mal­ized prior to Punycode encoding using the nameprep method. This method changed capital letters into lowercase letters, removed control char­ac­ters and trans­ferred equiv­a­lent char­ac­ters into a unified form. Nameprep was removed from this process when IDNA2008 was in­tro­duced. Now, IDNA does not specify any nor­mal­iza­tion. Instead, it rec­om­mends an algorithm that converts capital letters into lowercase ones.

This adaption also ac­com­mo­dates users in the German-speaking world, since the Unicode character “ß”, which is common in Germany, was orig­i­nal­ly defined as the equiv­a­lent of “ss” according to IDNA2003. Domains such as www.fußball-ergebnisse.de were thus au­to­mat­i­cal­ly nor­mal­ized to www.fussball-ergebnisse.de in the nameprep process. This is no longer the case since IDNA2008 came into the picture. Since 2010, the “ß” is correctly in­ter­pret­ed as “Latin small letter sharp s” and can be reg­is­tered as part of an IDN domain.

In addition, around 8,000 char­ac­ters that were possible in domain names under IDNA2003 are no longer supported under IDNA2008. Four char­ac­ters including “ß” are in­ter­pret­ed dif­fer­ent­ly since the standard was revised. For a detailed dis­cus­sion of the dif­fer­ences between IDNA2003 and IDNA2008, see Unicode Technical Standard #46. The following table provides a summary of the main dif­fer­ences:

What problems are there with IDNs?

By now, all common internet programs should be able to un­der­stand IDN. However, problems with in­ter­na­tion­al­ized domain names sometimes occur because the switch from IDNA2003 to IDNA2008 has not yet been con­sis­tent­ly im­ple­ment­ed. One example that’s prob­lem­at­ic for German is the different in­ter­pre­ta­tion of “ß”. Since IDNA2003 com­pul­so­ri­ly converts “ß” to “ss”, special ß domains that can be reg­is­tered according to IDNA2008 are often not dis­cov­er­able for systems that convert according to the outdated standard. Instead, users are directed to the cor­re­spond­ing domain con­tain­ing “ss”. This problem can be cir­cum­vent­ed by website operators reg­is­ter­ing both variants and redi­rect­ing the second domain to the pri­or­i­tized spelling using a domain redirect.