将文本拆分为单个单词

I would like to split a text into single words using PHP. Do you have any idea how to achieve this?

我的方法:

function tokenizer($text) {
    $text = trim(strtolower($text));
    $punctuation = '/[^a-z0-9äöüß-]/';
    $result = preg_split($punctuation, $text, -1, PREG_SPLIT_NO_EMPTY);
    for ($i = 0; $i < count($result); $i++) {
        $result[$i] = trim($result[$i]);
    }
    return $result; // contains the single words
}
$text = 'This is an example text, it contains commas and full-stops. Exclamation marks, too! Question marks? All punctuation marks you know.';
print_r(tokenizer($text));

这是一个好方法吗?你有什么改进的想法吗?

$result = preg_split('/((^p{P}+)|(p{P}*s+p{P}*)|(p{P}+$))/', $text, -1, PREG_SPLIT_NO_EMPTY);