My Wiki!

Django Translation

Following Bugs

Prerequisites

  yum install gettext-devel.x86_64 gettext-libs.x86_64

Use Postgresql!! see below.

Internationalization: in Python code

Internationalization: in template code

Localization: how to create language files

To create or update a message file, run this command:

  mkdir -p pj_root/pj_name/locale/vi

Add to settings.py:

LOCALE_PATHS = (
  os.path.join(BASE_DIR, "locale"),
)

then

./manage.py makemessages -l de
# edit vfoss_org/locale/vi/LC_MESSAGES/django.po
# find filename.html and add translation.
./manage.py compilemessages

to make the compiler process all the directories in your LOCALE_PATHS setting.

It is important to highlight that when you check that fuzzy translations are ok you should remove the fuzzy flag. Otherwise, they will be skipped with normal compilemessages calls.

458 #: vfoss_org/themes/new_theme/templates/_include_navigation.html:16
459 #, fuzzy
460 #| msgid "News"
461 msgid "Newswire"
462 msgstr "Tin Tức"

Postgresql Unicode Support

MySQL Database Unicode Support

Default collations for each character set can be viewed with the SHOW COLLATION statement, for example, to find the default collation for the latin2 character set:

SHOW COLLATION LIKE 'latin2%';
show variables like 'char%';

See the Database collation

SHOW CREATE DATABASE vfossdb;
+----------+-----------------------------------------------------------------------------------------+
| Database | Create Database                                                                         |
+----------+-----------------------------------------------------------------------------------------+
| vfossdb  | CREATE DATABASE `vfossdb` /*!40100 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin */ |
+----------+-----------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

*_general_ci vs. *_unicode_ci vs. *_bin

Really, I tested saving values like 'é' and 'e' in column with unique index and they cause duplicate error on both 'utf8unicodeci' and 'utf8generalci'. You can save them only in 'utf8_bin' collated column.

And mysql docs (in http://dev.mysql.com/doc/refman/5.7/en/charset-applications.html) suggest into its examples set 'utf8generalci' collation.

[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server=utf8mb4_bin
    #'Index column size too large. The maximum column size is 767 bytes.'
    # INNODB utf8 VARCHAR(255) (3bit, limit 3x255)
    # INNODB utf8mb4 VARCHAR(191) (4bit, index length 4x255 too large)
    innodb_large_prefix=on


Restart Mariadb:

systemctl restart mariadb.service
mysqlcheck -uroot -p --auto-repair --optimize --all-databases
mysqlcheck --auto-repair --optimize --all-databases

Troubleshooting

Index column size too large. The maximum column size is 767 bytes

  INNODB utf8 VARCHAR(255) (3bit, limit 3x255)
  INNODB utf8mb4 VARCHAR(191) (4bit, index length 4x255 too large)    

This ​InnoDB restriction was fixed in MySQL 5.5.14. You have to set the following my.cnf:

​ innodblargeprefix=ON

 innodb_file_per_table=ON
 innodb_file_format=Barracuda

and CREATE TABLE or ALTER TABLE with the ROW_FORMAT=DYNAMIC attribute.

This should be used in together with the appropriate OPTIONS in the database config:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {'charset': 'utf8mb4'},
        (...)

Set Server Level

  sudo mysql
  SET character_set_server = 'latin2';
  SET character_set_server = 'utf8mb4';

Similarly, the collation_server variable is used for setting the default server collation.

SET collation_server = 'latin2_czech_cs';
SET collation_server = 'utf8mb4_bin';

Set Database Level

The CREATE DATABASE and ALTER DATABASE statements have optional character set and collation clauses. If these are left out, the server defaults are used.

CREATE DATABASE czech_slovak_names 
CHARACTER SET = 'keybcs2'
COLLATE = 'keybcs2_bin';

ALTER DATABASE czech_slovak_names COLLATE = 'keybcs2_general_ci';

To determine the default character set used by a database, use:

SHOW CREATE DATABASE czech_slovak_names;
+--------------------+--------------------------------------------------------------------------------+
| Database           | Create Database                                                                |
+--------------------+--------------------------------------------------------------------------------+
| czech_slovak_names | CREATE DATABASE `czech_slovak_names` /*!40100 DEFAULT CHARACTER SET keybcs2 */ |
+--------------------+--------------------------------------------------------------------------------+

utf8mb4 support utf16 an emoji! correct me

ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

CREATE DATABASE `mydb` CHARACTER SET utf8 COLLATE utf8_general_ci;
GRANT ALL ON `mydb`.* TO `username`@localhost IDENTIFIED BY 'password';

ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

Navigation