It's the file qa-include/qa-db.php, line 97 in v1.8.7. Here's a link to the line in Github, you'd need to change 'utf8' to 'utf8mb4'.
BTW emoji will be supported in the next main version (1.9), but it may or may not be a while before that's released, depending on my schedule.