## Character Set Conversion

Character Set Conversion — convert strings between different character sets using ``iconv()``.

## Synopsis

    #include <glib.h>

    gchar*      g_convert                       (const gchar *str,
                                                 gssize len,
                                                 const gchar *to_codeset,
                                                 const gchar *from_codeset,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    gchar*      g_convert_with_fallback         (const gchar *str,
                                                 gssize len,
                                                 const gchar *to_codeset,
                                                 const gchar *from_codeset,
                                                 gchar *fallback,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    struct      GIConv;
    gchar*      g_convert_with_iconv            (const gchar *str,
                                                 gssize len,
                                                 GIConv converter,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    #define     G_CONVERT_ERROR
    GIConv      g_iconv_open                    (const gchar *to_codeset,
                                                 const gchar *from_codeset);
    size_t      g_iconv                         (GIConv converter,
                                                 gchar **inbuf,
                                                 gsize *inbytes_left,
                                                 gchar **outbuf,
                                                 gsize *outbytes_left);
    gint        g_iconv_close                   (GIConv converter);
    gchar*      g_locale_to_utf8                (const gchar *opsysstring,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    gchar*      g_filename_to_utf8              (const gchar *opsysstring,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    gchar*      g_filename_from_utf8            (const gchar *utf8string,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    gchar*      g_filename_from_uri             (const gchar *uri,
                                                 gchar **hostname,
                                                 GError **error);
    gchar*      g_filename_to_uri               (const gchar *filename,
                                                 const gchar *hostname,
                                                 GError **error);
    gchar*      g_locale_from_utf8              (const gchar *utf8string,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);
    enum        GConvertError;

    gboolean    g_get_charset                   (G_CONST_RETURN char **charset);

## Description

### File Name Encodings

Historically, Unix has not had a defined encoding for file names: a file name is valid as long as it does not have path separators in it ("/"). However, displaying file names may require conversion: from the character set in which they were created, to the character set in which the application operates. Consider the Spanish file name "`Presentación.sxi`". If the application which created it uses ISO-8859-1 for its encoding, then the actual file name on disk would look like this:

    Character:  P  r  e  s  e  n  t  a  c  i  ó  n  .  s  x  i
    Hex code:   50 72 65 73 65 6e 74 61 63 69 f3 6e 2e 73 78 69

However, if the application use UTF-8, the actual file name on disk would look like this:

    Character:  P  r  e  s  e  n  t  a  c  i  ó     n  .  s  x  i
    Hex code:   50 72 65 73 65 6e 74 61 63 69 c3 b3 6e 2e 73 78 69

Glib uses UTF-8 for its strings, and GUI toolkits like GTK+ that use Glib do the same thing. If you get a file name from the file system, for example, from `readdir(3)` or from ``g_dir_read_name()``, and you wish to display the file name to the user, you will need to convert it into UTF-8. The opposite case is when the user types the name of a file he wishes to save: the toolkit will give you that string in UTF-8 encoding, and you will need to convert it to the character set used for file names before you can create the file with `open(2)` or `fopen(3)`.

By default, Glib assumes that file names on disk are in UTF-8 encoding. This is a valid assumption for file systems which were created relatively recently: most applications use UTF-8 encoding for their strings, and that is also what they use for the file names they create. However, older file systems may still contain file names created in "older" encodings, such as ISO-8859-1. In this case, for compatibility reasons, you may want to instruct Glib to use that particular encoding for file names rather than UTF-8. You can do this by specifying the encoding for file names in the `G_FILENAME_ENCODING` environment variable. For example, if your installation uses ISO-8859-1 for file names, you can put this in your `~/.profile`:

    export G_FILENAME_ENCODING=ISO-8859-1

Glib provides the functions ``g_filename_to_utf8()`` and ``g_filename_from_utf8()`` to perform the necessary conversions. These functions convert file names from the encoding specified in `G_FILENAME_ENCODING` to UTF-8 and vice-versa. Figure 1, “Conversion between File Name Encodings” illustrates how these functions are used to convert between UTF-8 and the encoding for file names in the file system.

Figure 1. Conversion between File Name Encodings

#### Checklist for Application Writers

This section is a practical summary of the detailed description above. You can use this as a checklist of things to do to make sure your applications process file name encodings correctly.

1. If you get a file name from the file system from a function such as `readdir(3)` or ``gtk_file_chooser_get_filename()``, you do not need to do any conversion to pass that file name to functions like `open(2)`, `rename(2)`, or `fopen(3)` — those are "raw" file names which the file system understands.

2. If you need to display a file name, convert it to UTF-8 first by using ``g_filename_to_utf8()``. If conversion fails, display a string like "`Unknown file name`". Do not convert this string back into the encoding used for file names if you wish to pass it to the file system; use the original file name instead. For example, the document window of a word processor could display "Unknown file name" in its title bar but still let the user save the file, as it would keep the raw file name internally. This can happen if the user has not set the `G_FILENAME_ENCODING` environment variable even though he has files whose names are not encoded in UTF-8.

3. If your user interface lets the user type a file name for saving or renaming, convert it to the encoding used for file names in the file system by using ``g_filename_from_utf8()``. Pass the converted file name to functions like `fopen(3)`. If conversion fails, ask the user to enter a different file name. This can happen if the user types Japanese characters when `G_FILENAME_ENCODING` is set to `ISO-8859-1`, for example.

## Details

### g_convert ()

    gchar*      g_convert                       (const gchar *str,
                                                 gssize len,
                                                 const gchar *to_codeset,
                                                 const gchar *from_codeset,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string from one character set to another.

`str` : | the string to convert
`len` : | the length of the string
`to_codeset` : | name of character set into which to convert `str`
`from_codeset` : | character set of `str`.
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | If the conversion was successful, a newly allocated nul-terminated string, which must be freed with `g_free()`. Otherwise `NULL` and `error` will be set.

### g_convert_with_fallback ()

    gchar*      g_convert_with_fallback         (const gchar *str,
                                                 gssize len,
                                                 const gchar *to_codeset,
                                                 const gchar *from_codeset,
                                                 gchar *fallback,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string from one character set to another, possibly including fallback sequences for characters not representable in the output. Note that it is not guaranteed that the specification for the fallback sequences in `fallback` will be honored. Some systems may do a approximate conversion from `from_codeset` to `to_codeset` in their `iconv()` functions, in which case GLib will simply return that approximate conversion.

`str` : | the string to convert
`len` : | the length of the string
`to_codeset` : | name of character set into which to convert `str`
`from_codeset` : | character set of `str`.
`fallback` : | UTF-8 string to use in place of character not present in the target encoding. (This must be in the target encoding), if `NULL`, characters not in the target encoding will be represented as Unicode escapes \uxxxx or \Uxxxxyyyy.
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | If the conversion was successful, a newly allocated nul-terminated string, which must be freed with `g_free()`. Otherwise `NULL` and `error` will be set.

### struct GIConv

    struct GIConv;

The GIConv struct wraps an ``iconv()`` conversion descriptor. It contains private data and should only be accessed using the following functions.

### g_convert_with_iconv ()

    gchar*      g_convert_with_iconv            (const gchar *str,
                                                 gssize len,
                                                 GIConv converter,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string from one character set to another.

`str` : | the string to convert
`len` : | the length of the string
`converter` : | conversion descriptor from `g_iconv_open()`
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | If the conversion was successful, a newly allocated nul-terminated string, which must be freed with `g_free()`. Otherwise `NULL` and `error` will be set.

### G_CONVERT_ERROR

    #define G_CONVERT_ERROR g_convert_error_quark()

Error domain for character set conversions. Errors in this domain will be from the GConvertError enumeration. See GError for information on error domains.

### g_iconv_open ()

    GIConv      g_iconv_open                    (const gchar *to_codeset,
                                                 const gchar *from_codeset);

Same as the standard UNIX routine `iconv_open()`, but may be implemented via libiconv on UNIX flavors that lack a native implementation.

GLib provides `g_convert()` and `g_locale_to_utf8()` which are likely more convenient than the raw iconv wrappers.

`to_codeset` : | destination codeset
`from_codeset` : | source codeset
Returns : | a "conversion descriptor", or (GIConv)-1 if opening the converter failed.

### g_iconv ()

    size_t      g_iconv                         (GIConv converter,
                                                 gchar **inbuf,
                                                 gsize *inbytes_left,
                                                 gchar **outbuf,
                                                 gsize *outbytes_left);

Same as the standard UNIX routine `iconv()`, but may be implemented via libiconv on UNIX flavors that lack a native implementation.

GLib provides `g_convert()` and `g_locale_to_utf8()` which are likely more convenient than the raw iconv wrappers.

`converter` : | conversion descriptor from `g_iconv_open()`
`inbuf` : | bytes to convert
`inbytes_left` : | inout parameter, bytes remaining to convert in `inbuf`
`outbuf` : | converted output bytes
`outbytes_left` : | inout parameter, bytes available to fill in `outbuf`
Returns : | count of non-reversible conversions, or -1 on error

### g_iconv_close ()

    gint        g_iconv_close                   (GIConv converter);

Same as the standard UNIX routine `iconv_close()`, but may be implemented via libiconv on UNIX flavors that lack a native implementation. Should be called to clean up the conversion descriptor from `g_iconv_open()` when you are done converting things.

GLib provides `g_convert()` and `g_locale_to_utf8()` which are likely more convenient than the raw iconv wrappers.

`converter` : | a conversion descriptor from `g_iconv_open()`
Returns : | -1 on error, 0 on success

### g_locale_to_utf8 ()

    gchar*      g_locale_to_utf8                (const gchar *opsysstring,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string which is in the encoding used for strings by the C runtime (usually the same as that used by the operating system) in the current locale into a UTF-8 string.

`opsysstring` : | a string in the encoding of the current locale
`len` : | the length of the string, or -1 if the string is nul-terminated.
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | The converted string, or `NULL` on an error.

### g_filename_to_utf8 ()

    gchar*      g_filename_to_utf8              (const gchar *opsysstring,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string which is in the encoding used for filenames into a UTF-8 string.

`opsysstring` : | a string in the encoding for filenames
`len` : | the length of the string, or -1 if the string is nul-terminated.
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | The converted string, or `NULL` on an error.

### g_filename_from_utf8 ()

    gchar*      g_filename_from_utf8            (const gchar *utf8string,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string from UTF-8 to the encoding used for filenames.

`utf8string` : | a UTF-8 encoded string.
`len` : | the length of the string, or -1 if the string is nul-terminated.
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | The converted string, or `NULL` on an error.

### g_filename_from_uri ()

    gchar*      g_filename_from_uri             (const gchar *uri,
                                                 gchar **hostname,
                                                 GError **error);

Converts an escaped ASCII-encoded URI to a local filename in the encoding used for filenames.

`uri` : | a uri describing a filename (escaped, encoded in ASCII).
`hostname` : | Location to store hostname for the URI, or `NULL`. If there is no hostname in the URI, `NULL` will be stored in this location.
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | a newly-allocated string holding the resulting filename, or `NULL` on an error.

### g_filename_to_uri ()

    gchar*      g_filename_to_uri               (const gchar *filename,
                                                 const gchar *hostname,
                                                 GError **error);

Converts an absolute filename to an escaped ASCII-encoded URI.

`filename` : | an absolute filename specified in the encoding used for filenames by the operating system.
`hostname` : | A UTF-8 encoded hostname, or `NULL` for none.
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | a newly-allocated string holding the resulting URI, or `NULL` on an error.

### g_locale_from_utf8 ()

    gchar*      g_locale_from_utf8              (const gchar *utf8string,
                                                 gssize len,
                                                 gsize *bytes_read,
                                                 gsize *bytes_written,
                                                 GError **error);

Converts a string from UTF-8 to the encoding used for strings by the C runtime (usually the same as that used by the operating system) in the current locale.

`utf8string` : | a UTF-8 encoded string
`len` : | the length of the string, or -1 if the string is nul-terminated.
`bytes_read` : | location to store the number of bytes in the input string that were successfully converted, or `NULL`. Even if the conversion was successful, this may be less than `len` if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
`bytes_written` : | the number of bytes stored in the output buffer (not including the terminating nul).
`error` : | location to store the error occuring, or `NULL` to ignore errors. Any of the errors in GConvertError may occur.
Returns : | The converted string, or `NULL` on an error.

### enum GConvertError

    typedef enum
    {
      G_CONVERT_ERROR_NO_CONVERSION,
      G_CONVERT_ERROR_ILLEGAL_SEQUENCE,
      G_CONVERT_ERROR_FAILED,
      G_CONVERT_ERROR_PARTIAL_INPUT,
      G_CONVERT_ERROR_BAD_URI,
      G_CONVERT_ERROR_NOT_ABSOLUTE_PATH
    } GConvertError;

Error codes returned by character set conversion routines.

`G_CONVERT_ERROR_NO_CONVERSION` | Conversion between the requested character sets is not supported.
`G_CONVERT_ERROR_ILLEGAL_SEQUENCE` | Invalid byte sequence in conversion input.
`G_CONVERT_ERROR_FAILED` | Conversion failed for some reason.
`G_CONVERT_ERROR_PARTIAL_INPUT` | Partial character sequence at end of input.
`G_CONVERT_ERROR_BAD_URI` | URI is invalid.
`G_CONVERT_ERROR_NOT_ABSOLUTE_PATH` | Pathname is not an absolute path.

### g_get_charset ()

    gboolean    g_get_charset                   (G_CONST_RETURN char **charset);

Obtains the character set for the current locale; you might use this character set as an argument to `g_convert()`, to convert from the current locale's encoding to some other encoding. (Frequently `g_locale_to_utf8()` and `g_locale_from_utf8()` are nice shortcuts, though.)

The return value is `TRUE` if the locale's encoding is UTF-8, in that case you can perhaps avoid calling `g_convert()`.

The string returned in `charset` is not allocated, and should not be freed.

`charset` : | return location for character set name
Returns : | `TRUE` if the returned charset is UTF-8