[MySQL / MariaDB] convert collation & character-set

Some databases may either be using an old character encoding or mixing diffenent charset, which they want to upgrade or convert to a Unicode Standard.

This example uses the basic utf8, which is encoded on 3bits, it is, however, recommended to use utf8mb4 which is encoded on 4 bits encompassing a more extensive unicode charset.

So in this example, we are changing the latin1 collation
– latin1_swedish_ci => utf8_unicode_ci (or utf8mb4_unicode_ci)
– latin1 => utf8 (or utf8mb4)

Using MySQLDump

First of all, we will use mysqldump to extract a full dump of the database, you may also add:
–skip-set-charset –default-character-set=utf8 to use and set utf8 as the default new character set.
–add-drop-database in order to be able to restore the dump over the top of the existing database.
-u [username] -p your authentication arguments

mysqldump –add-drop-database database > database.sql

sed -i ‘s/latin1_swedish_ci/utf8_unicode_ci/g’ database.sql

sed -i ‘s/latin1/utf8/g’ database.sql

mysql database < database.sql

Don’t forget your authentication arguments -u user -p

Using phpMyAdmin

phpMyAdmin also provides a way to automatically convert the database charset as well as the table and its fields. However this may fail because of foreign key checking, therefore you may need to disable that & convert your data and then re-enable MySQL’s foreign key checks

SET FOREIGN_KEY_CHECKS = 0;
SET GLOBAL FOREIGN_KEY_CHECKS=0;

/* Conversion */

SET FOREIGN_KEY_CHECKS = 1;
SET GLOBAL FOREIGN_KEY_CHECKS=1;

Comments

1 comment

  1. Contextualization: One of the users was using a database with a mixture of unicode & latin1 charsets and because he is upgrading from MySQL 5.7 to a more recent MariaDB or MySQL 8, he encountred a problem with UNION:

    MySQL error: Illegal mix of collations for operation ‘UNION’

    The new DBMS is not tolerant to the these UNIONs and he needs to either:

    1- Declare implicitly the COLLATE of each SQL SELECT that is using a union of different charsets in his application. For example:

    SELECT name COLLATE utf8_general_ci FROM A1
    …
    UNION SELECT name FROM contacts

    Same goes for conditions, he may need to spificy that as well:
    WHERE value1 COLLATE utf8_general_ci = value2 ….

    He can also use conversions in his SQL requests

    2- Or simply to reconvert all his database, tables and fields to a homoganous structure.

Leave a Reply

Your email address will not be published. Required fields are marked *