I am interested in doing a calculation of any piece of Japanese text to find an average number of words/kanji which are proper nouns, ie names of people or places. I assume this is a possible function in corpus analysis, but I am interested in how to generate my own data on the use of jinmeiyo kanji and hyogai kanji vs the standard joyo kanji.
1 Answer
Kuromoji might be able to help you, but I've never really tried it myself personally. I tested it on some NHK articles and it seemed to have correctly parsed out the 固有名詞 in those articles. And then if you were to collect a list of those 固有名詞 I suppose you could figure out the number of non-常用漢字 vs 常用漢字.
-
I did a few tests and it does seem to parse the 固有名詞 well. If I was dealing with a large amount of text, how should I differentiate between 常用漢字 and others?– kandymanCommented May 21, 2019 at 9:01
-
@kandyman I suppose the way I would imagine this working is that when you parse a large amount of text, you would write a program that would use kuromoji to identify the part of speech. And then you could collect the list of 固有名詞. Then split them into their individual Kanji and compare to the list of 常用漢字 to figure out the counts of non-常用漢字 vs 常用漢字 . But obviously, that requires some programming...– RingilCommented May 22, 2019 at 11:51