Machine Translation Quality Assessment of Selected Works of Xiaoping Deng Supported by Digital Humanistic Method

: With the development of modern technology, the new technological methods need to be applied timely into translation research and translation quality assessment. As a new research method, digital technology is often used in the field of humanities research instead of the traditional ones and open a new paradigm for humanities research. At the same time, humanities research also provides a broad application space and rich corpus data for computer technology. Different from the traditional methods of machine translation quality assessment, this study attempts to apply the digital technology into the machine translation assessment, with the help of corpus technology, computer technology and statistical methods, so a to evaluate the quality of machine translations generated by different translation software from lexical, syntactical, semantic and pragmatic levels. The machine translation data for analysis come from the automatic translations by Baidu and Youda of the Selected Works of Xiaoping Deng , collection of Xiaoping Deng’s important political speeches and theories, and the comparable data for reference is from the translated text produced by the expert translators. The specific case analysis and evaluation, on the one hand, verify the effectiveness of the digital humanity method applicable in the actual machine translation quality assessment; on the other hand, try to eliminate people’s bias to the machine translation, so as to make people have a deeper understanding of the advantages and disadvantages of machine translation and improve the machine translation software design in the future.


Introduction
In recent years, the machine translation system based on artificial neural network has made initial progress in simulating the neural network model of human brain to realize translation activities, but the computer still can not have the ability of thinking and reasoning like human beings, and the operational mode of computer translation is not the same as that of human translation. Therefore, the results of machine translation can not be completely evaluated by human translation standards. Moreover, the simple theory guided manual scoring evaluation method is too heavy and difficult to meet the needs of judging the results of machine translation in a short time. Inspired by "Digital Humanities", a new research paradigm dedicated to the application of modern computer technology to the research of traditional humanities [1], the author introduces digital technology into the quality evaluation method of machine translation, so as to realize the integration of computer, linguistics and translation theories, and improve the objectivity and accuracy of the quality evaluation results of machine translation. Thus, the efficiency of machine translation quality evaluation has also been improved. Taking the Chinese English Parallel Corpus of Selected Works of Xiaoping Deng as the evaluation reference, this paper attempts to reflect the actual translation situation of the machine translation system based on the results of text data processing and analysis at the lexical level, sentence level, semantic level and pragmatic level with the help of human digital tools, and further verifies the accuracy and feasibility of the evaluation model.

The Existence of Translation Quality Evaluation Methods
Translation standards play a role in measuring the quality of translation in translation practice [2], which is a rule of conduct that must be followed in different translation activities, including manual translation and machine translation. The study of translation quality evaluation began in the 1970s, and most of them focused on the construction of the evaluation criteria of human translation quality. The existing evaluation models only stay in the theoretical stage and lack of applicability in machine translation assessment. However, the evaluation of machine translation quality emphasizes more about the pertinence, practicability and validity of the selected parameters within the scope of effective evaluation, and mainly focuses on the two aspects of faithfulness and fluency, which correspond to Lu Xun's "faithfulness" and "smoothness" standards [3].

Traditional Evaluation Model of Translation
Any translation needs quality assessment. Translation critics, reviewers, teachers, translation enterprises, experts and readers will judge translation in one way or another. At present, there are three major orientations in translation quality assessment: 1) source text oriented evaluation model. It is mainly to compare the translation with the original to see whether the translation is consistent with the original in some or all aspects. That is to say, the quality of the translation is mainly measured by the degree of similarity between the translation and the original. In other words, the higher degree of similarity between the translation and the original, i.e. the translated text is closer to the original, the better the quality of the translation [4]; 2) target text oriented evaluation model. It pay attention to whether the translation conforms to the cultural norms of the target language. In this vein, people usually compare the translated text with the similar text in the target language, namely, parallel texts, to see whether the translated text conforms to the target similar text norms. Its characteristic is that it no longer puts the original text in the supreme position, but pays more attention to the acceptability of the translation. And more considerations are given to the target readers, the norms of the target language and the social and cultural constraints. The evaluation criteria focus more on the degree of similarity between translation and similar texts in the target culture [5]; 3) the effect oriented evaluation model. Questionnaire survey, interview, comprehension test and translation comments are often used to evaluate the effect of translation and the criterion is the principle of equivalent effect [6].
In terms of quality assessment of machine translation, evaluation tends to be descriptive from the perspective of the nature of translation, and the evaluation parameters and their weights vary with different interpretations of translation [7]. However, in the academia, even the concept of translation is often defined differently in the field of translation, due to different understandings and choices of the connotation of the concept [8]. Roughly, the understandings of translation can be summarized into three categories: first, translation is regarded as a tool, that is, translation is taken as a means or a tool to achieve the purpose of communication and expression. Jacobson thinks that the same way of meaning expression in one language can be found in another language, that is, the ability of language expression is the same; second, translation is considered as creation. For example, Ezra Pound advocates "interpretative translation" and the freedom and creativity in poetry translation [9], and Yu Guangzhong regards translation activities as "limited creation" by "using Chinese to trap English who refuses to comply" [10]; third, translation is conceived as a kind of social activities, functioning in a specific social context, as in Hatim and Mason's opinion that translation is a communicative process which takes place within a social context.

Evaluation Model of Machine Translation
The transformation of machine translation from rule-based, corpus based to artificial neural network-based is to achieve automatic and high-quality artificial intelligence translation. However, at present, the quality of machine translation can not be compared with that of professional translation. Computer system only simulates the operation mechanism of human brain to process and produce information, which is far from the ability of human brain to judge and reason according to contextual information. In view of the limitations and special operation model of machine translation system, the evaluation parameters of machine translation are limited to the basic translation criteria of "faithfulness" and "smoothness".

Manual Evaluation Method
It is flexible but time-consuming to evaluate the quality of machine translation according to the standards and parameters set manually. The manual evaluation method generally takes the original text as the reference, and usually uses fidelity, fluency, intelligibility and information as the indicators. Each indicator is divided into several evaluation levels according to the situation of the translation. In practice, these four indicators are usually distinguished. The first two indicators are a group of evaluation parameters, and the second two indicators are another group of evaluation parameters. Faithfulness is the basic criterion of translation evaluation, and the goal of faithfulness is to reproduce the content, thought, emotion and style of the original text. But absolute faithfulness can not exist, and relative faithfulness is easier to be accepted and implemented. Fluency is used to evaluate the degree to which a translation adapts to the expression habits of the target language. The difference between intelligibility and fluency lies in the acquisition, processing and production of semantic knowledge blocks. According to the degree of understanding, it can be divided into four levels: complete understanding, basic understanding, less understanding and complete incomprehension. Information degree is a parameter to evaluate the degree of information the reader obtains from the translation. The larger the weight is, the richer the amount of information is, and the better the effect of machine translation is. At present, researches in mainland China on the quality evaluation methods of machine translation include questionnaire and data analysis (see [11]), the quality estimation algorithm of machine translation based on translation knowledge (see [12]), the quality evaluation method of machine translation based on pretraining language model (see [13]), methods of quality evaluation of machine translation based on knowledge transfer (see [14]), AHP fuzzy evaluation method of machine translation quality in English translation (see [15]), translation quality estimation method based on subword sentence level neural machine translation (see [16]), translation quality estimation method based on word vector of BERT context (see [17]), the translation quality estimation method based on multilanguage pretraining language model (see [18]), etc.

Automatic Evaluation Method
According to the specific rules and algorithms, the automatic evaluation method can automatically score and sort the machine translation quality on the computer system. At present, the principle of machine translation automatic evaluation system is based on N-element matching, editing distance and error rate. Representative evaluation algorithms are BLEU, NIST, WER, PER, TER. BLEU and NIST calculate the similarity between machine translation and reference translation based on N-element grammar. The higher the similarity, the better the fidelity of machine translation. These two automatic evaluation methods help to enhance the objectivity of manual evaluation, and have certain reference value in machine translation evaluation [3]. WER is to calculate the number of times of editing (including insertion, deletion, movement and replacement) when the machine translation is converted into a reference translation. The less the number of times, the better the output of the machine translation system. PER and TER use another common evaluation method of machine translation, which is to identify the error rate in machine translation, formalize the specific translation errors and types, and judge the quality of machine translation from the error degree of analysis data. The method of machine translation quality evaluation based on similarity and error rate is only evaluated from the grammatical level. In order to investigate the quality of machine translation in a deeper level, the method based on linguistic knowledge (morphology, syntax, semantics, pragmatics, etc.) is introduced into the automatic evaluation system of machine translation. For example, Professor Shiwen Yu from the Institute of Computational Linguistics of Peking University proposed an automatic evaluation method of machine translation based on test points (such as word collocation, word order, syntactic structure, etc.). In recent years, the application of context in the computer field has aroused widespread concern. Linguists and machine translation system developers try to introduce the relevant theories of pragmatics into machine translation and machine translation evaluation.
Another important factor in automatic evaluation is the setting of evaluation parameters and weights. Evaluation parameter is the value or element that can be used for reference when evaluating something, and it is a variable. Setting weights for evaluation parameters is to determine the importance of each parameter in the evaluation system, which is expressed by weight coefficient. The larger the coefficient is, the larger the proportion of the parameter in the evaluation results is. Machine translation realizes the transformation between two natural languages based on language model and translation model. Therefore, language parameters (such as vocabulary, syntax, word order, semantics, pragmatics, etc.) are indispensable and occupy a large proportion in the automatic evaluation system of machine translation. Secondly, nonverbal parameters (such as culture, emotion, thinking, common sense, etc.) play an increasingly important role in machine translation quality evaluation. In general, the sum of the weights of each parameter is set to 1, and the weight of each parameter is expressed as a number from greater than 0 to less than 1 according to the degree of importance. The common methods of weight setting include analytic hierarchy process, fuzzy method, expert experience method and so on. Among them, analytic hierarchy process divides multiple levels of indicators, compares the importance of indicators by constructing judgment matrix, and compares the importance of indicators according to 9 importance levels. According to the theory of fuzzy mathematics, fuzzy method is a method to judge the weight of parameters; expert experience method is a method to synthesize the weight of each index according to the experience and subjective judgment of most experts.
At present, the statistical methods based on the co-occurrence of n-gram are BLEU (Bilingual Evaluation Understudy) and NIST (National Institute of Standards and Technology). The difference between them is that the former calculates the score according to the geometric mean of n-gram accuracy, while the latter calculates the score according to the arithmetic mean of n-gram accuracy. The higher the score, the better. Their formulas and algorithms are as follows, BLEU: ; Among them, indicates the number of words in the candidate MT translation, indicates the number of words in the standard reference translation, denotes penalty factor. When the number of words in the candidate MT translation is more than that in the standard reference translation, supposing the value of the penalty factor is 1, there is no penalty. When the number of words in the MT candidate translation is less than or equal to the number of words in the standard reference translation, punishment is required.
words<occurring in the candidate machine translation, and *+ !' , indicates the number of expression words < occurring in the standard reference translation. N is the value of n-gram, 7 represents weight, and the method of uniform weighting is generally adopted, that is, 7 = 1/>. NIST: Among them, @ " indicates the number of times n-1-gram appearing in the standard reference translation, and @ " indicates the number of times n-gram appearing in the standard reference translation. When = 1, the value of @ "is the number of all words in the standard reference translation. ?
@ " represents the amount of information of per n-gram. indicates the number of words in the candidate MT translation, and indicates the number of words in the standard reference translation.
The automatic evaluation method based on editing distance is mainly to calculate the number of editing (including insert, delete, move and replace) when machine translation is converted to reference translation. The less the number, the closer the level of machine translation is to that of manual translation. The minimum editing distance is a similarity calculation function. The smaller the editing distance is, the greater the similarity between the system translation and the reference translation is. WER (word error rate) is based on Levenshtein Distance. In the evaluation of machine translation results, we use words instead of strings as the calculation unit. The basic calculation formula is as follows, WER: Among them, D indicates the number of times the word was replaced, U indicates the number of times the word was deleted,?indicates the number of times the word was inserted, and > means the number of words. The calculation principles of PER, TER and WER are similar, that is, calculating the editing distance between the candidate machine translation and the annotated reference translation, but the difference is that PER ignores the word order in machine translation, that is, it allows the change of word position, while WER and TER consider the word position order in machine translation, and TER also supports block movement.
The automatic evaluation method based on linguistic knowledge investigates the accuracy and recalling rate of machine translation from stemming and synonym matching. The representative evaluation method is METEOR. METEOR: them, ], W and b represent the parameters of the model, $ represents the word set given in advance based on WordNet thesaurus, ℎ represents a chunk formed by aligned and continuous words between the candidate machine translation and the standard reference translation, ! " indicates the number of times the expression word occurring in the candidate machine translation, and *+ !' , indicates the number of times the expression word occurring in the standard reference translation.
With the emergence of large-scale corpus and the emergence of various machine translation algorithms, automatic evaluation has become an indispensable evaluation method in the development process [19]. To a great extent, the automatic evaluation model saves the time and energy of manual evaluation. It evaluates the machine translation through the concepts of similarity, error rate, accuracy and recalling rate, and realizes the automation, algorithmization and accuracy of the quality evaluation of machine translation, but it also has certain limitations. The disadvantage lies in that the evaluation strategies and methods can not be adjusted according to the characteristics and results of machine translation.

Case study of Machine Translation Evaluation Based on Selected Works of Xiaoping Deng
The quality assessment model of Chinese-English machine translation based on digital humanities method aims to evaluate the quality of machine translation from the linguistic level and non linguistic level with the help of corpus, python programming language, similarity algorithm and text sentiment analysis technology, etc. At the linguistic level, the consistency between the machine translation and the reference translation is evaluated from the parameters of functional words, keywords, word collocation, average sentence length and word order; at the non linguistic level, the semantic and contextual effects of the machine translation are judged by text information extraction and calculating the similarity and emotional inclination between the machine translation and the reference translation.

Evaluation at the Lexical Level
Vocabulary is the basic unit of machine translation evaluation, which can be a single word or a group of words. In view of the grammatical features of terms and the semantic relationship between terms, the lexical level of MT evaluation starts from functional words and collocations. The number of functional words in Chinese is small, but the frequency of use is very high, and they must be attached to the notional words to have practical meaning. Whether they are used correctly or not directly affects the meaning of the whole sentence. Whether the machine translation system can recognize the functional words in the original text and translate them reasonably according to the context is related to the fidelity of the machine translation. Chinese and English belong to two different language families with different grammatical rules. Collocation is one of the most common grammatical rules. Firth first mentioned the concept of word collocation in Modes of Meaning [20]. He believed that the semantic meaning of a word is determined by another word that is collocated with it. In addition to the semantic association of collocation, other related studies have mentioned that there is a certain dependency between word and word combination, which may be due to the conceptual elements such as idiomatic language, probability of use and fixed collocation [21].

Functional Word Diagnosis
Chinese functional words are very rich, which can be divided into adverbs, prepositions, conjunctions, auxiliary words, interjections and onomatopoeia. Except adverbs, they are generally not used as phrases or sentences. They have no real meaning, but have grammatical or functional meaning. Functional words do not exist in English language in a sense, but are reflected in the change of tense and sentence pattern in grammar. In the process of translation, we must make flexible transformation according to the context and the grammatical rules of the target language. To evaluate the machine translation of Chinese functional words is to test whether the machine translation system accurately expresses the grammatical or functional meaning of functional words. All the functional words in the Chinese corpus are labeled manually or automatically. The corpus retrieval tool is used to list all the translation examples of the functional words. Through text observation and language analysis, the translation results of the functional words are scored according to the completion. The score of completion degree is set as [0] and [1], that is, the score of completion (correct translation, accurate meaning) is 1, and the score of unfinished (wrong / untranslated, no / reverse meaning) is 0. Taking the Selected Works of Xiaoping Deng (Volume 3) as the test corpus, the translation results of Youdao translation system are taken as the evaluation objects, and the scoring results are shown in Table 1.
(再) the translated words are correct and the meaning is accurate; (很) the untranslated words have no meaning The majority of Chinese functional words show the adaptive transformation of semantic relevance and grammatical rules in English language. In the process of testing, it is necessary to combine the contextual information to judge and deduce the usage standard of the functional words in English language. Therefore, to be familiar with and master the rules of the use of functional words in Chinese and English is the basic condition to evaluate the translation quality.

Word Collocation Diagnosis
Word collocation is a way of meaning expression at the word level [22], which varies with different languages and may lose its original characteristics in the process of translation [21]. There are many factors that affect words and word combination, such as semantics, context, language habits, style, register, etc. in natural language processing, and a variety of factors are introduced. Based on rules, statistics and machine learning methods, the computer can reduce the error rate of word collocation as much as possible in the process of translation. As stated in modern Chinese collocation [23], it is limited by nationality, times, locality, diversity, subjectivity and other factors. The combination methods include subject predicate, verb object, definite middle, adverbial middle, prepositional object and so on. However, English collocations are generally divided into general collocations, rhetorical collocations, professional collocations and conventionalized collocations [24]. According to the collocation intensity, they can be divided into three types: partially restricted collocations, highly restricted collocations and completely restricted collocations [25]. Based on the basic concepts of Chinese English collocation, the evaluation principles of machine translation results of collocation can be divided into three categories: natural principle, conventional principle and creative principle. The principle of nature mainly investigates whether the word form combination conforms to the collocation habits and ways of the target language; the principle of conventionality, that is, the association and co-occurrence of word form, is based on the conventional rules and norms; the principle of creation is the subjective collocation, but it does not violate the idiomatic and conventionality of word form combination. For example: Original: 我们不仅要武装战士的手足，尤其重要的是武 装战士的头脑 Reference translation: Our soldiers should not only have weapons in their hands but, more important, they should also be armed mentally.
Youdao translation: We should not only arm the arms and feet of soldiers, but also the brains of armed soldiers.
The above example is the collocation form of "V + n" and has semantic motivation. The morphological combination of "armed hands and feet" is a specific concept, which means soldiers hold weapons, while "armed mind" is an abstract concept, which means soldiers keep rational and fresh mind. Based on the standard translation, Youdao translation is too literal and does not conform to the target language norms, which violates the natural principle of collocation.
Original: 这对于军队战斗力的提高，有非常重大的意 义。 Reference translation: This is most important to the enhancement of the troops' combat effectiveness.
Youdao translation: This is of great significance to the improvement of the combat effectiveness of the army.
In Chinese, the phrase "great significance" is a definite middle phrase structure, with modifying and modified elements. In English, the fixed collocation "be important / of great significance" can be used to express the importance of something. Therefore, Youdao translation conforms to the conventional principle of collocation in this example.
Original:允许战士家属到营地看望，并给家属以很好的 安慰和招待。 Reference translation: We should also allow their family members to visit them in the army barracks, show solicitude for them and entertain them.
Youdao translation: The family members of the soldiers are allowed to visit the camp and give them good comfort and hospitality.
The phrase "allow someone to do something" is a verb object structure. The reference translation adopts the common collocation "allow someone to do something" in English, while Youdao translation adopts the passive voice of "allow someone to do something" without violating the natural and conventional principles of collocation, highlighting the object of action of the verb, which is in line with the creative principle of collocation.

Sentence Level Evaluation
The evaluation of the quality of machine translation from the sentence level mainly focuses on the diagnosis of consistency of the average sentence length and the word order of interrogative sentences. The average sentence length of standard translation and machine translation is calculated by Python programming language. The larger the value is, the more complex the sentence is. Chinese and English both include declarative sentence, interrogative sentence, exclamatory sentence, imperative sentence and inverted sentence. The significant difference is that interrogative sentence in English can be divided into special interrogative sentence and general interrogative sentence. In written language, special interrogatives such as "where", "what", "why", "how", "who" and "when" are usually placed in the middle or end of Chinese sentences instead of the beginning in English sentences. General questions in English sentences need be-verbs and modal verbs to complete. From the consistency of word order conversion of Chinese and English interrogative sentences, we can judge and evaluate the translation results of machine translation system at the sentence level.

Average Sentence Length Diagnosis
The average sentence length can be used to measure the difficulty of sentences in the text, and the average sentence length is directly proportional to the complexity of sentences [26]. The average sentence length is usually calculated without punctuation and other meaningless sentence elements. Chinese sentences are calculated by the number of "Zi (字)", while English sentences are calculated by the number of "Ci (词)". By comparing the average sentence length between machine translation and reference translation, the degree of sentence complexity handled by machine translation system can be reflected, and the adequacy of sentence translation in machine translation can be judged (as shown in Table 2). The reference translation of the first and second volumes of Selected Works of Xiaoping Deng is taken from the second edition of Foreign Language Press published in 1995, and the reference translation of the third volume is from the first edition in 1994. It can be seen from the table that the sentence complexity of the translation of Volume 1 is high. The average sentence lengths of Baidu translation and Youdao translation are 21.46 and 20.43 respectively, which are 1.57 words and 2.6 words different from the reference translation. The average sentence length of the reference translation of Volume 2 is 20.10, which is 2.09 words and 2.82 words longer than that of Baidu translation and Youdao translation. The average sentence length of the reference translation of Volume 3 is 17.90, which is 1.66 words and 2.49 words longer than that of Baidu translation and Youdao translation. Considering the whole text, the average sentence length of Baidu translation is closer to that of reference translation, with a difference of only 0.32 words. Therefore, for sentence level processing, Baidu translation can better reflect the sufficiency and integrity of sentence content and information than Youdao translation.

Word Order Diagnosis of Interrogative Sentences
The declarative sentences in Chinese and English belong to the same subject predicate object structure, while the order of interrogative sentences is different. Except for rhetorical questions, interrogative words, auxiliary verbs and modal verbs in English interrogative sentences are usually placed at the beginning of sentences. To judge the result of machine translation of English interrogative sentences is to evaluate whether the use of special interrogative words such as what, when, why, where, who, how, be-verbs such as is, am, are and modal verbs such as can, may, shall, will and their positions in sentences are correct or not. For example, by using Python programming language software to search the sentences with question marks in the full text of selected works of Xiaoping Deng, a total of 996 sentences were obtained, of which 23% were interrogative sentences in Volume 1, 44% were interrogative sentences in Volume 2, and 33% were interrogative sentences in volume 3. Statistics of special interrogative words (what, when, why, where, who, how) appear at the beginning of the sentences are shown in Figure 1. As can be seen from the figure above, there are 609 special questions in the reference translation, accounting for 61.14% of the total number of questions; 607 special questions in Baidu translation, accounting for 60.94% of the total number of questions; 599 special questions in Youdao translation, accounting for 60.14% of the total number of questions. In terms of the frequency of using special interrogative words at the beginning of sentences, Baidu translation is closer to the reference translation, with a difference of only 0.2%, while Youdao translation is different from the reference translation, with a difference of 1%. Comparing the machine translation with the reference translation, the closer the proportion of the interrogative sentences with the characteristic words at the beginning of the sentence is, the higher the probability of the correct position of the language unit in the sentence is.

Semantic Level Evaluation
Semantic analysis means that the machine translation system obtains formal language related background knowledge, which is a very complex and important link after lexical analysis and syntactic analysis. Chinese and English have different ways of expressing meaning. The former expresses the grammatical function and logical relationship of sentences through the meaning of language units, while the latter obtains the logical meaning of sentences through the connection of structure and form of language units. Therefore, the Chinese-English machine translation system needs to analyze the grammatical function and meaning of the sentence components in the source language in order to determine the complete semantic information of the sentence. By comparing the text similarity between the machine translation and the reference translation, the understanding ability of the machine translation system is judged. The closer the similarity value is, the stronger the understanding ability of the machine translation system is, which means that the semantic information of the machine translation is more complete.

Text Similarity Diagnosis
In the process of translation, in order to convey complete semantic information, the translator (or machine translation system) should try to maintain the similarity between the target language text and the source language text. Taking the reference translation as a standard, the vector space of the words in the translated text is established by using cosine similarity measure method, and then the cosine value of the angle between the two vectors is calculated through the cosine function formula. The larger the value is, the higher the similarity between the two texts is. The specific operation is as follows: source text: 政府机关支部与一般农村支部的对象不同， 它还有其特殊的任务，而且要有比较健全的支部委员会的 领导，才能实现这些特殊的任务。 reference translation: Unlike those in rural areas, Party branches in the government have their own special tasks, which require sound leadership from the Party branch committee.
Baidu translation: Different from the general rural branch, Supported by Digital Humanistic Method the branch of government organs has its own special tasks, which can only be accomplished under the leadership of a relatively sound branch committee. Youdao translation: Different from the general rural party branch, the branch of a government organ has its own special tasks, which can only be realized by the leadership of a relatively sound party branch committee.

Key Words Diagnosis
Key words are subject words that summarize and convey the content and information of the text, also known as subject words. Through the corpus retrieval software, all the application examples of key node words in the corpus are listed, and the frequency of the words in the text is counted. By comparing the keyword index sequences of the machine translation and the reference translation, we can judge whether the machine translation is variable translation or stable translation according to the degree of change. Secondly, we count the frequency of keywords used in the original text, in the machine translation and in the reference translation, and regard the frequency of keywords used in the original text as F1, the frequency of keywords used in the machine translation as F2, and the frequency of keywords used in the reference translation as F3. The hypothesis is that the closer the F2 value is to the F1 value, the higher the fidelity of the machine translation, and the closer the F2 value is to the F3 value, the better the naturalness of the machine translation. For example, in the first article of the selected works of Xiaoping Deng (Volume I), Mobilization of New Soldiers and Political Work for New Soldiers (January 12, 1938), the key node word is "bing (兵)", and the official term is "recruit". For the sake of vocabulary rules in English, it is searched by word roots, and the results are as follows: In the source text, "bing (兵)" appears 22 times in such collocations literally as "mobilize new soldiers", "new soldiers politics", "soldier corps", "being a soldier", "recruit soldiers", "military service", "officers and men", "recruiting system", and "old soldiers", etc.
"recruit" appears 24 times in the reference translation. The collocations are: mobile new recruits, enlist new recruits, recruiting soldiers, recruitment efforts, methods of recruitment, recruitment methods, carry out recruitment; "recruit" appears 25 times in Youdao translation, with the following collocations: mobilization of recruits, mobilizing recruits, the way we recruit, recruiting troups, recruitment and redeployment work, way of recruitment, recruit new solvers, recruit mobilization.
From the above analysis, Youdao translation is generally consistent with the reference translation, which belongs to stable translation. From the frequency of key words, there are 22 collocations of "bing (兵)" in the original text (that is, F1=22), while "recruit" appears 25 times in Youdao translation (that is, F2=25) and 24 times in reference translation (that is, F3=24), so F2 value is closer to F3 value.
Thus, compared with faithfulness, Youdao translation performs better in target language understanding, expression and fluency.

Pragmatic Evaluation
It is worth trying to introduce the theoretical knowledge of pragmatics into computer technology. Pragmatics is the study of language use and its laws. The implied meaning and communicative effect of language use can only be reflected in the specific language environment. Therefore, understanding the logical relevance of the context information of the source language can make the target language correctly reflect the real meaning and intention of the original text. Text sentiment analysis reflects the subjective information of a text and obtains the speaker's emotional state and communication intention by mining the text content. To compare the affective orientation value (E1) of the machine translation with the affective orientation value (E2) of the reference translation, the closer the E1 value is to the E2 value, the more accurate the machine translation is in understanding the original context, that is, the more sufficient the machine translation is. For example: Source Text: 当前的战局，是处于暂时的局部的失利的 境况，决不是抗日自卫战争的最后失败。 Reference transaltion: Currently we are suffering a temporary and partial setback in our defensive war against Japan, but this is not final defeat.
(Emotional polarity: positive; Positive probability: 0.99920017; Negative probability: 0.0007998973) Baidu translation: The current situation of war is a temporary local defeat, and it is by no means the final defeat of the war of self-defence against Japan.
(Emotional polarity: positive; Positive probability: 0.9954066; Negative probability: 0.0045934017) Youdao translation: The present situation is one of temporary and partial defeat and is by no means the final defeat of the war of resistance and self-defence.
(Emotional polarity: positive; Positive probability: 0.9972767; Negative probability: 0.0027232843) From the above examples, the words "temporary", "partial" and "by no means" in the original text indicate that the speaker's attitude is positive. Therefore, the emotional polarity of the machine translation conforms to the subjective emotional tendency of the original text. The value of emotional polarity is 1, with 0.5 as the cut-off point, greater than is positive emotion and less than is negative emotion. The positive and negative affective strength of the text is reflected by positive probability and negative probability. The former reflects the positive affective strength of the influencing words in the text, while the latter reflects the negative affective strength. The emotional polarity of the reference translation is positive, and the emotional strength is very close to 1. Baidu translation and Youdao translation both belong to positive emotion tendency, and the emotional strength is 0.9954066 and 0.9972767 respectively, which is 0.00379357 and 0.00192347 different from the positive emotion strength of the reference translation. Therefore, in this sentence, Youdao translation has more accurate understanding and processing of the text context, and the adequacy of the translation is higher.

Conclusion
In the era of Digital Humanities, the combination of digital technology and translation studies is reflected in the use of various information tools and means to provide corpus or data for the explanation of translation phenomena. Chinese-English machine translation quality assessment model uses a variety of technical means to evaluate the effect of machine translation system, and takes linguistic knowledge and translation criteria as the theoretical guidance of quality assessment, so as to provide a practical and effective machine translation quality assessment model for researchers and translators. However, there are still some limitations in this model, which need to be further studied.