在進(jìn)行文本分析、提取關(guān)鍵詞時(shí)，新聞評(píng)論等文本通常是中英文及其他語言的混雜，若不加處理直接分析，結(jié)果往往差強(qiáng)人意。
下面對(duì)中英文文本進(jìn)行分離做一下總結(jié)：
1、超短文本，ASCII識(shí)別。
|
1
2
3
|
s = "China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday.該集團(tuán)總裁朱利安周二表示，中國(guó)聯(lián)想控股將分拆其多個(gè)業(yè)務(wù)部門在股市上市。"result = "".join(i for i in s if ord(i) < 256)print(result) |
|
1
2
|
out:China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday. |
2、unicode編碼識(shí)別
|
1
2
3
4
5
|
import res = "China's Legend Holdings will split its several business arms to go public on stock markets, the group's president Zhu Linan said on Tuesday.該集團(tuán)總裁朱利安周二表示，中國(guó)聯(lián)想控股將分拆其多個(gè)業(yè)務(wù)部門在股市上市。"uncn = re.compile(r'[\u0061-\u007a,\u0020]')en = "".join(uncn.findall(s.lower()))print(en) |
|
1
2
|
out:chinas legend holdings will split its several business arms to go public on stock markets, the groups president zhu linan said on tuesday |
中文的編碼范圍是：\u4e00-\u9fa5，相應(yīng)的[^\u4e00-\u9fa5]可匹配非中文。
匹配英文時(shí)，需要將空格[\u0020]加入，不然單詞之間沒空格了。
以上這篇利用Python將文本中的中英文分離方法就是小編分享給大家的全部?jī)?nèi)容了，希望能給大家一個(gè)參考，也希望大家多多支持服務(wù)器之家。
原文鏈接：https://blog.csdn.net/sinat_36972314/article/details/79746291
