Welcome to the Question2Answer Q&A. There's also a demo if you just want to try it out.

Unicode 10 characters filtered out?

+3 votes
asked Feb 15 in Q2A Core by q2apro
edited Feb 15 by q2apro

Here is a bunch of nice new emoticons: 

happy ☺ ☻

Should render to:

But after submit, most of the emojis are gone.

I guess the core filters out those characters. Bug?

Update: I realized that it also happens with the CMS Modx...

Q2A version: 1.8.0

2 Answers

+1 vote
answered Feb 15 by pupi1985
selected Feb 15 by q2apro
Best answer

This has been discussed a couple of times. In short, the issue is that MySQL is not able to store it unless the table has a specific character set so, arguably, it has been removed from the core for the time being.

You can remove this limitation by:

1. Making sure the qa_remove_utf8mb4($string) function returns the $string without any change

2. Replacing the charset used in qa_db_connect() from utf8 to utf8mb4.

3. Updating the MySQL fields that you want to store this kind of content to support (e.g.: posts.title, posts.content, words.word, etc). It can be done this way:

ALTER TABLE `qa_posts` CHANGE COLUMN `content` `content` VARCHAR(12000) CHARACTER SET 'utf8mb4' NULL DEFAULT NULL;

4. You may need to change the schema collation as well.

ALTER SCHEMA `q2adb`  DEFAULT COLLATE utf8mb4_general_ci;

Note that as you're using more bytes to represent characters. InnoDB tables have a limit in the amount of bytes they index. When using InnoDB you have less characters to index (191 now), which means some queries might run slower.

BTW, here is how it looks like:

commented Feb 15 by q2apro
Too much hassle for just some emojis ... Thanks for the detailled answer!
commented Feb 15 by pupi1985
My bad. I forgot about something important. Check this other answer: http://www.question2answer.org/qa/54525/is-there-a-way-to-get-unicode-smiley-work-with-q2a?show=54537#a54537 . You can represent them in their HTML form, as long as the editor used supports HTML format. That means they will take even more characters to be represented but it doesn't require you to do any change to the DB or the core.
commented Feb 15 by q2apro
That's pretty cool. So I could write a filter module, run over the post content and replace all by their HTML form. But maybe it's really "too much" for just getting some emojis to show up :D
0 votes
answered Feb 15 by MONKEY
what I do is upload the image to the emoticons folder and then relate the image name to where the code is.
@q2@pro,again I want to thank the sceditor contribution but sometimes there is a problem with the image upload emoticon

when publishing, the image can not be seen, it is seen in this code and if it is not tried, new image so not corrected, then if the image appears

Example ----->.  http://recetarios.top/files/image.png

Could you help me ?