Datasets for Aspect-Based Sentiment Analysis in Bangla and Its Baseline Evaluation

Transcription

1 data Data Descriptor Datasets for Aspect-Based Sentiment Analysis in Bangla Its Baseline Evaluation Md. Atikur Rahman * Emon Kumar Dey * Institute Information Technology, University Dhaka, Dhaka 1000, Bangladesh * Correspondence: bsse0521@iit.du.ac.bd (M.A.R.); emonkd@iit.du.ac.bd (E.K.D.) Received: 20 March 2018; Accepted: 2 May 2018; Published: 4 May 2018 Abstract: With extensive growth user interactions through prominent advances Web, sentiment analysis has obtained more focus from an academic a commercial point view. Recently, sentiment analysis in Bangla language is progressively being considered as an important task, for which previous approaches have attempted to detect overall polarity a Bangla document. To best our knowledge, re is no research on aspect-based sentiment analysis (ABSA) Bangla text. This can be described as being due to lack available datasets for ABSA. In this paper, provide two publicly available datasets to perform ABSA task in Bangla. One datasets consists human-annotated user comments on cricket, or dataset consists customer reviews restaurants. We also describe a baseline approach for subtask aspect category extraction to evaluate our datasets. Dataset: Dataset License: CC0 Keywords: ABSA dataset; Bangla ABSA; aspect extraction from Bangla 1. Summary People trust human opinion more so than traditional advertising. For example, consumers are used to seeking advice recommendation from ors before making decisions regarding important purchases. Word mouth (WOM) has always been salient for consumers when making a decision. Such referrals have a strong impact on both customer decision-making new customer acquisition for purchasing a company s product or service [1]. On or h, organizations are eager to mine all activities interactions people to underst what ir aknesses strengths are. This understing would help m to develop ir organizational strategy in this competitive world. Sentiment analysis (or opinion mining) is a process to determine viewpoint a person on a certain topic. It classifies polarity a document (i.e., review, tet, blog, or news), that is, wher communicated opinion is positive, negative, or neutral. There are three levels at which sentiment is analyzed [2]: document level, sentence level, aspect level. The document level considers that a document has an opinion on an entity, task is to classify wher an entire document expresses a positive or negative sentiment. The task at sentence level regards sentences determining wher each sentence expresses a positive, negative, or neutral opinion. Neir document level nor sentence level analysis discover exactly what people liked did not like. The aspect level (or aspect-based sentiment analysis ABSA) performs a finer-grained analysis that identifies aspects a given document or sentence sentiment expressed towards Data 2018, 3, 15; doi: /data

2 Data 2018, 3, each aspect. This level analysis is most detailed version that is capable discovering complex opinions from reviews. There are two major tasks when performing ABSA. The first is to extract specific areas or aspects mentioned in opinioned review. The second is to identify polarity (eir positive, negative, or neutral) for every aspect. For example, following review a restaurant reveals two aspects: service food. Both aspects have a positive polarity. The service was excellent food was delicious. As one can see, name aspect categories are explicitly mentioned in this review. A review might also contain implicit categories; for example, The staff makes you feel at home chicken is great. Here, same aspects, service food, are contained without being directly mentioned. Semantic Evaluation (SemEval), a reputed workshop in NLP domain, introduced a complete dataset [3] in English for ABSA task. Later this was exped to ABSA task by adding multi-lingual datasets in which eight languages over seven domains re incorporated. To perform ABSA, datasets several languages, such as Arabic [4], Czech [5], French [6], re created. There is no dataset for Bangla in field ABSA. Consequently, no work is being done to extract aspects to identify corresponding polarities for Bangla reviews. We are currently working on a project to extract aspects from a Bangla review or comments for a particular product a company, as online shopping is very popular nowadays in Bangladesh is growing rapidly. People like to buy products online after reading comments ors. In this paper, have created two new datasets that serve as a benchmark for ABSA domain in Bangla texts. We present two datasets named Restaurant. The first dataset contains 2900 comments on cricket over 5 aspect categories, second dataset contains 2600 restaurant reviews. Because re is no work in Bangla for ABSA task, have introduced ABSA by extracting aspect categories from Bangla texts in order to evaluate our datasets. We performed task with different training approaches found a satisfactory outcome compared to evaluations or languages. There are some related works from which founded idea this topic. The restaurant review dataset, provided by Ganu et al. [7], was used to improve rating predictions. Their annotations included six aspect categories overall sentence polarities. They had not prepared a complete ABSA dataset, as aspect category was present but corresponding polarity that aspect was absent. The SemEval 2014 evaluation campaign [3] extended ir dataset by adding three more fields with aspect category. They published ir dataset with four fields being contained for each review, that is, with aspect term occurring in sentences, aspect term s polarity, aspect category, aspect category s polarity. They also provided a laptop-review dataset manually annotated with similar entities as for restaurant dataset. These are benchmark datasets that [8 11] researches have used for performing ABSA task. The task was repeated in SemEval 2015 [12], for which aspect categories re combination entity type an attribute type. Multilingual datasets re released in SemEval 2016 workshop [13] on seven domains (restaurant, laptop, mobile phone, digital camera, hotel, museum) in eight languages (English, Arabic, French, Chinese, Turkish, Spanish, Dutch, Russian). A book-review dataset in Arabic language was provided by [4]. They annotated book reviews into 14 categories 4 types polarities, including Conflict. In [5], author created an IT product-review dataset for ABSA task, in which a total 2200 reviews re contained. The contribution this paper is as follows: We have collected presented two Bangla datasets for ABSA have made m publicly available. We performed statistical linguistic analysis on datasets.

3 Data 2018, 3, We implemented state---art machine learning approaches for collected datasets Data 2018, 3, x 3 11 found satisfactory accuracies Data Description Digital Bangladesh is is integral integral part part Bangladesh Bangladesh government s government s Vision Vision The Internet The isinternet growingis very growing fast over very fast country, over country, people are using people different are using online different platforms online inplatforms every aspect in every iraspect lives. This ir encouraged lives. This usencouraged to constructus datasets to construct to analyze datasets Bengali to analyze people s Bengali opinions people s to extract opinions ir sentiments to extract in ir different sentiments aspects. in different In this paper, aspects. In have this constructed paper, have two different constructed datasets, two namely, different datasets, dataset namely, Restaurant dataset dataset, to evaluate Restaurant dataset, people s to opinions. evaluate people s opinions Dataset 2.1. Dataset The dataset consists 2900 different comments from different online sources with five different The aspect dataset categories. consists 2900 Mostdifferent comments from are different collectedonline fromsources Facebook with pages five ( different aspect categories. Most comments are collected from Facebook pages Some ( comments are collected from two popular Bengali Websites, BBC Bangla ( bengali), Some comments Daily are Prothom collected Alo ( from two popular Bengali This dataset Websites, was collected BBC Bangla by authors ( this paper. The comments are Daily different Prothom lengths Alo ( each review contains approximately This dataset Bangla was collected words. The by reasons authors behind this choosing paper. se The comments Websites for are collecting different data lengths are given below: each review contains approximately Bangla words. The reasons behind choosing se Websites for collecting BBC Bangla data are given below: Daily Prothom Alo are very popular online news sites for Bengali community BBC Bangla all around Daily world. Prothom They Alo are are popular very popular for publishing online news trustworthy sites for auntic Bengali news. community Bengali all people around frequently world. read They are news popular sometimes for publishing make trustworthy comments to share auntic ir opinion. news. Bengali Although people people frequently write ir read comments news or opinions sometimes in both make Bangla comments English, to share most ir opinion. time, y Although choose people Bangla. write Weir studied comments different or opinions articles in both found Bangla that in almost English, 90% most cases, time, people y expressed choose Bangla. ir opinion We studied in Bangla. different articles found that in almost 90% The cases, Facebook people expressed page Prothom ir opinion Alo has in Bangla. over 13 million follors, BBC Bangla has over 11 The million. Facebook These page two pages Prothom provide Alo enormous has over 13 text million postsfollors, as ll as a large BBC number Bangla has comments. over 11 million. isthese one two pages mostprovide popularenormous games nowadays text posts for as Bengali ll as a people. large number We found comments. that people are moreis interested one in most making popular comments games nowadays on cricket-related for Bengali news people. than We on found any that or people topic. Thus, are more chose interested this category in making for comments our experiment. on cricket-related news than on any or topic. Thus, chose this category for our experiment. Table 1 shows an example comments collected from Facebook pages. Table 1 shows an example comments collected from Facebook pages. Table 1. Example cricket-related comments on Prothom Alo BBC Bangla Facebook pages. Table 1. Example cricket-related comments on Prothom Alo BBC Bangla Facebook pages. Comments ম শর ফ এক জ দ ক র ন ম য ন মট ন লই মন ভ র য য় আম দর জ হর,জনসন, প লক, ট ল ন ই ত ব এক জন ম শর ফ আ ছ ব ল র দরও দ ষ নই, দ ষট 100% ম নজ ম র পস ব ল র দর জন উই কট ত র ত ত র ই ক র ন দ ষট ম নজ ম ন ম ন ল আ ম নব আশ ক র ত স কন অ ত ত দ ল ফর ব আর নয় মত খল ব এব চ র আউট কর ব এখন দশ ক দর ক ছ জন য় হ ট /ওয় ন ড ম ন ষ এখন দখ তই চ য়ন.. হ র লও ব ল দশ জত লও ব ল দশ আগ ম ত আব র আমর ই জত ব ইশ! আম দর য দ বর ট ক হ লর ম ত একট ব ট ন থ ক ত আম র পর মশ হ ল কট র দর ক প রট জগত থ ক দ র র খ ত হ ব Source Prothom Alo Facebook page Prothom Alo Facebook page Prothom Alo Facebook page Prothom Alo Facebook page BBC Bangla Facebook page BBC Bangla Facebook page People People usually usually comment comment in in Bangla Bangla about about news. news. We We also also found found that that 5 10% 5 10% time, time, y y commented in English also wrote Bangla sentences written in English alphabet. We did not commented in English also wrote Bangla sentences written in English alphabet. We did not consider se opinions in our dataset. In addition, some comments had only emoticons no or consider se opinions in our dataset. In addition, some comments had only emoticons no text or words. We also omitted se for our dataset. All processes re done manually by or text or words. We also omitted se for our dataset. All processes re done manually authors. The following section describes annotation process collected corpus cricketrelated comments. by authors. The following section describes annotation process collected corpus cricket-related comments Annotation Dataset The Bangla text on cricket was annotated jointly by authors, a group second-year students BSSE, two employees from Institute Information Technology, University Dhaka,

4 Data 2018, 3, Annotation Dataset The Bangla text on cricket was annotated jointly by authors, a group second-year students BSSE, two employees from Institute Information Technology, University Dhaka, Bangladesh. All participants agreed to categorize whole dataset into five different aspect categories. These Data Data 2018, 2018, re3, 3, bowling, x batting, team, team management, or. Given a comment, task 12 annotators was to recommend aspect category polarity labels for each. Three types Bangladesh. polarities re All considered, participants thatagreed agreed is, positive, to to categorize categorize negative, whole whole neutral. dataset dataset Tableinto into 2 shows five five different different information aspect aspect about categories. These re bowling, batting, team, team, team team management, or. or. Given Given a comment, a comment, participants. task annotators was to to recommend aspect aspect category polarity polarity labels labels for each. for each. Three Three types types polarities re considered, that that is, is, positive, negative, neutral. neutral. Table Table 2 shows 2 shows information information Table 2. Information about participants in data collection. about participants. Participant ID Gender Pression Task Table Information about about participants in data in data collection. collection. P1 Male MS student/author Data collection () annotation P2 Participant ID ID Male Gender Pression Faculty/author Pression Task Data collection Task () annotation P1 P3 P1 Male Male Male MS MS student/author Graduate student/author Data Annotation Data collection collection () () annotation translation annotation (Restaurant) P2 P4 P2 Male Female Male Faculty/author Graduate Faculty/author Data student Annotation Data collection collection () () annotation translation annotation (Restaurant) P3 Male Graduate student Annotation () translation (Restaurant) P5 P3 Female Male Graduate student Annotation () () translation translation (Restaurant) (Restaurant) P4 Female Graduate student Annotation () translation (Restaurant) P6 P4 Male Female Graduate student Annotation Annotation () () translation translation (Restaurant) (Restaurant) P5 Female Graduate student Annotation () translation (Restaurant) P7 P5 Male Female Graduate student Annotation Annotation () () translation translation (Restaurant) (Restaurant) P6 Male Graduate student Annotation () translation (Restaurant) P8 P6 Male Male Graduate student Annotation Annotation () () translation translation (Restaurant) (Restaurant) P7 Male Graduate student Annotation () translation (Restaurant) P9 Female Accountant Annotation P8 P7 Male Male Graduate Graduate student student Annotation Annotation () () translation translation (Restaurant) (Restaurant) P10 Male Officer Annotation P9 P8 Female Male Graduate Accountant student Annotation () Annotation translation (Restaurant) P10 P9 Male Female Accountant Officer Annotation Annotation P10 Male Officer Annotation Each participant categorized every comment dataset. We applied majority voting technique Each to participant make final categorized decision every about comment aspect category dataset. We applied polarity majority a sentence. voting As an technique Each to participant make final categorized decision every about comment aspect category dataset. We polarity applied a sentence. majority As an voting example, have taken following comment: technique example, to have make taken final following decision comment: about aspect category polarity a sentence. As an example, have taken এই following প চ র ন কর comment: ট ফ, ব ল ন স হ ভ ল হ য় ছ The The voting voting result result found এই প চ for for র ন comment comment কর ট ফ, is ব ল is given given ন স হ in Table in Table 3. ভ ল 3. হ য় ছ The voting result found for comment is given in Table 3. Table 3. Voting example to define category polarity. Table 3. Voting example to define category polarity. Comment: Table 3. এই Voting প চ example র ন কর ট ফ, to define ব ল ন স হ category ভ ল হ য় ছ polarity. Participant Voting for Category Voting for Polarity Comment: এই প চ র ন কর ট ফ, ব ল ন স হ ভ ল হ য় ছ P1 Bowling Positive Participant Voting for Category Voting for Polarity Participant P2 Voting Bowling for Category Positive Voting for Polarity P1 Bowling Positive P1 P3 Batting P2 Bowling Bowling Positive Positive P2 P4 Batting P3 Batting Bowling Positive P5 Or Neutral P3 P6 P4 Bowling Batting Batting Positive P4 P7 P5 Or Or Batting Neutral Neutral P5 P8 P6 Bowling Bowling Or Positive Positive Neutral P6 Bowling Positive P9 P7 Bowling Or Positive Neutral P7 Or Neutral P10 P8 Batting Bowling Positive P8 Bowling Positive P9 Bowling Positive P9 Bowling Positive From Table 3, can see P10 that comments Batting had three votes for P10 Batting Batting with a negative polarity, two votes for Or with a neutral polarity, four votes for Bowling with a positive polarity. Thus, our method From Table determined 3, can this see comment that as comments being in had Bowling three votes category for Batting with a positive with a negative polarity. polarity, We two also From votes had Table ties for for Or 3, some with can comments. a see neutral thatin polarity, this comments situation, four hadtook votes three both for votes Bowling categories for Batting with with a positive with ir a negative polarity. in polarity, Thus, twour votes dataset. method fortable determined Or 4 shows withthis an neutral example comment polarity, for as this being scenario. in four Bowling votes category for Bowling with with a positive a positive polarity. polarity. We Thus, also our had method ties for determined some comments. this comment In this situation, as being intook Bowling both categories category with air positive polarity polarity. in Weour alsodataset. had ties Table for some 4 shows comments. an example In for this this situation, scenario. took both categories with ir polarity in our dataset. Table 4 shows an example for this scenario.

5 Data 2018, 3, Data 2018, 3, x 5 11 Table 4. Voting category identification. Data 2018, 3, x Table 4. Voting category identification Comment: Table ওর Voting ক র ছ, category ত মর identification. 100 কর ত প র ব ন? Participant Voting for Voting for Polarity Comment: ওর Voting 200 ক র ছ, for Category ত মর 100 কর ত Voting প র ব ন? for Polarity Participant P1 P1 Voting Batting for Batting Category Voting for Polarity P2 P2 P1 Team Batting Team P3 P3 P2 Team Batting P4 P4 P3 Batting Batting P5 Batting P5 P4 Batting P6 Team P6 P5 Team Batting P7 Team P8 P7 P6 Team Batting P9 P8 P7 Batting Team Team P10 P8 Batting Team P9 Team P9 Team P10 Team P10 Team We can see from Table 4 that 50% evaluators voted for Batting, 50% votes re for We Team can We see category, can from see from Table both Table with 4 that 4 that a negative 50% 50% polarity. evaluators As y voted had for for tied, Batting, Batting, our algorithm 50% 50% took votes both votes re se re categories for for Team inteam category, labeled category, both dataset both with with a a anegative polarity. As y had had tied, tied, our our algorithm took took both both se We se also categories faced in anor in labeled kind dataset problem with a to negative construct polarity. dataset. After calculating category, found We also We dissimilarities also faced faced anor anor forkind some kind comments problem to among construct participants dataset. After After regarding calculating polarity category, category, comment. found found For example, dissimilarities when for for some some considered comments among following participants comment, regarding found voting polarity polarity result given comment. comment. For For example, example, when when considered considered following comment, comment, found found voting voting result in Table 5. result given in Table 5. given in Table 5. Table 5. Problem related to polarity determination. Table 5. Problem related to polarity determination. Table 5. Problem related to polarity determination. Comment: র ক ব ড় হ য় গ ছ, খল প রন ত হ ল আজ কভ ব ক কর ল? Comment: Participant র ক ব ড় হ য় Voting গ ছ, for খল Category প রন ত হ ল Voting আজ কভ ব for Polarity ক কর ল? Participant Voting for Category Voting for Polarity Participant P1 Voting Bowling for Category Voting Positive for Polarity P1 P2 P1 Bowling Bowling Positive Positive P2 Bowling Positive P2 P3 Bowling Team P3 Team Positive P3 P4 Bowling Team P4 Bowling P5 P4 P5 Or Bowling Or Positive Positive P6 P5 P6 Team Or Team Positive P7 P7 Or Or P6 P8 Team P8 Team Team P7 P9 P9 Or Bowling Positive P8 P10 P10 Team Team P9 Bowling Positive From Table P10 5, can see that, for Team comment, both Bowling Team categories had four From Table 5, can see that, for comment, both Bowling Team categories had four votes. Thus, keep both se categories in our annotated dataset. In polarity, re re three votes. Thus, keep both se categories in our annotated dataset. In polarity, re re three From positive Table votes 5, one can negative see that, vote for for bowling. comment, Thus, both considered Bowling it as having Team a positive categories polarity positive had four votes added it to one our negative dataset. Table vote6 for shows bowling. a sample Thus, labeled considered it dataset. as having a positive polarity votes. added Thus, it to our keep dataset. both Table se 6 categories shows a sample in our annotated dataset. In dataset. polarity, re re three positive In Table votes 7, one summary negative vote complete for bowling. Thus, dataset considered is presented. it as We having can see a positive that re polarity are a total added 3034 it different to our dataset. comments Table with 6 shows five different a sample categories, labeled that is, Batting, dataset. Bowling, Team, Team Management, Or. Table Each6. A part categories contains dataset threein different xlsx format. polarities: positive, negative, neutral. For example, Batting category contains a total 583 comments, for which 138 are positive, 389 are বয প র negative, ন eট ধ ম t 56 ঘর টন areছ ড় neutral কছ ন polarity. The Bowling category or contains 154neutral positive, 145 negative, 33 neutral ভক মন polarity ট iগ র দর comments. জনয The Team, Team Management, team Or categories positive contain totals ব ল দশ 774, 332, eখ ন ত ম মর 1013 য গয comments, o পন র প ল respectively. ন batting negative ব ল দশ হ র ব আজ team negative স তজন sন র ন য় ম ঠ ব তল ট ন ন য য় ময চ জত য য় ন bowling negative ট নর পচ ব ন য় ঔষধ খ জ ল ভ ক ব u n পচ ব ন লi হয় team management negative eট প র দ ম ফ k eকট খল হi স team negative জয় ধ সম য়র a পk or positive যi জ য়র সম ন!! or neutral

6 Data 2018, 3, Data 2018, 3, x 6 12 Table 6. A part dataset in xlsx format. বয প র ন eট ধ ম t ঘর টন ছ ড় কছ ন or neutral ভক মন ট iগ র দর জনয team positive ব ল দশ eখ ন ত ম মর য গয o পন র প ল ন batting negative ব ল দশ হ র ব আজ team negative স তজন sন র ন য় ম ঠ ব তল ট ন ন য য় ময চ জত য য় ন bowling negative ট নর পচ ব ন য় ঔষধ খ জ ল ভ ক ব u n পচ ব ন লi হয় team management negative eট প র দ ম ফ k eকট খল হi স team negative জয় ধ সম য়র a পk or positive যi জ য়র সম ন!! or neutral ব ল রর য প রম ন শটর বল দ c- ত ত র ন কত ব শ হয় সট i দখ র বষয়! bowling negative ত ক টs আর o ডআi দ ল নয় মত চ i team positive ব ল দশ k কট আ র e গ য় য ব, o পন র দর eকট ভ ল কর ত হ ব team positive ব ল দশ k কট আ র e গ য় য ব, o পন র দর eকট ভ ল কর ত হ ব batting negative ফ রi চমক দখ লন র j ক bowling positive নব ন দর স য গ দয় দরক র. or neutral ব ল পচ ত ব আম দর বয টসময ন দর আuট ল আtহতয ছ ড় আর কছ i নয় batting negative ব ল পচ ত ব আম দর বয টসময ন দর আuট ল আtহতয ছ ড় আর কছ i নয় bowling neutral দ য়tj ন হ নত র aভ ব? or negative In Table 7, summary Table 7. The complete complete statistics dataset is presented. dataset. We can see that re are a total 3034 different comments with five different categories, that is, Batting, Bowling, Team, Team Management, Or. Each categories contains Polarity three different polarities: positive, negative, neutral. For Category example, Batting category contains a total 583 comments, for Total which 138 are Positive Neutral positive, 389 are negative, 56 are neutral polarity. The Bowling category contains 154 Batting positive, 145 negative, 33 neutral polarity comments. The Team, Team Management, Or Bowling categories contain Team totals 774, 332, comments, 502 respectively Team Management Or Table 7. The 89complete statistics 828 dataset Total Comments Polarity 3034 Category Total Positive Neutral Analysis Proposed Batting Dataset Bowling We used Zipf s law [14] for our proposed dataset. Zipf s law is a statement-based Team observation that states that Team inmanagement a collection data, 24 frequency a given332 word should be inversely proportional to its rank in Or corpus. The word89 that is 828 most frequent scores rank 1 in a dataset should occur approximately twice as Total second Comments most frequent word, three 3034 times as third most frequent, so on. Figure 1 shows diagram in which plotted words our dataset. The plot follows Analysis trend Proposed Zipf s law. Dataset We also calculated reliability annotation process. The valuewe used intraclass Zipf s law correlation [14] for our (ICC) proposed was dataset. Zipf s law is a statement-based observation that states that in a collection data, frequency a given word should be inversely 2.2. Restaurant proportional Dataset to its rank in corpus. The word that is most frequent scores rank 1 in a dataset To should create occur approximately Bangla Restaurant twice as dataset, second most took frequent help directly word, three from times English as third benchmark s most frequent, so on. Figure 1 shows diagram in which plotted words our dataset. Restaurant dataset [3]. All comments re abstractly translated into Bangla with ir exact annotation. The plot follows trend Zipf s law. We also calculated reliability annotation process. The original English dataset contains a total 2800 different comments. Participants from same The value intraclass correlation (ICC) was group involved in dataset s creation re involved in translation process Restaurant dataset, except participants P9 P10. We divided original dataset equally into eight parts distributed se to participants. They translated ir assigned parts original English dataset abstractly. Finally, participants P1 P2 merged separate sections performed an extensive proread.

7 Data 2018, 3, Figure Distribution word frequencies dataset using Zipf s law. law. Annotation Schema for Restaurant The Restaurant reviews [3] dataset used in this paper was abstractly translated into Bangla. There re five types aspect categories, that is, Food, Price, Service, Ambiance, Miscellaneous. As objective was to identify aspect category corresponding polarity, participants did not add aspect terms or ir polarities. In terms polarity an aspect category, considered only three polarity labels, that is, positive, negative neutral. The original dataset consisted four different polarity labels: positive, negative, neutral, conflict. In our translated Bangla dataset, omitted conflict category assumed it to be same as neutral category. The annotators re asked to assign each translated Bangla restaurant review into ir categories ir polarities for original dataset. Table 8 shows a sample translated Restaurant dataset. Data 2018, 3, x 8 12 Table Table A part part Restaurant dataset in inxlsx xlsxformat. খ ব স মত আসন আ ছ eব খ দয প oয় র জনয য থ a পk কর ত হ ব ambience negative খ ব স মত আসন আ ছ eব খ দয প oয় র জনয য থ a পk কর ত হ ব service negative Figure 2. Word দ ম frequency ত লন ম লকভ ব Bangla কম Restaurant dataset according to Zipf s law. price positive 3 i ছল মজ দ র food positive য দo খ ব র ট চমৎক র ছল, e ট সs ছল ন food positive য দo খ ব র ট চমৎক র ছল, e ট সs ছল ন price negative খ ব ভ ল! miscellaneous positive আচ রর স য জন খ ব ভ ল ছল food positive ধ ম t র n i য সর ত নয়, সব সবসময় ম ন য গ eব ভ ল হ য় ছ food positive ধ ম t র n i য সর ত নয়, সব সবসময় ম ন য গ eব ভ ল হ য় ছ service positive সবর দ eক ট স nর ভড়, কn ক ন ক ল হল নi ambience positive সj alsl eব প র র - ব n ব pশ স কর কছ i নi ambience neutral আ ম ন ত য আম ক ব রবর ফ র য ত হ ব,!!! miscellaneous positive সm বত e ট eক ট ছ ট আর মদ য়ক রs রn,ভ ল সj র স র ম nক aন ভ ত ambience positive য দo খ দয ভ ল ছল প র বষন ছল ব food positive য দo খ দয ভ ল ছল প র বষন ছল ব service negative Data 2018, 3, x; do i: কম র ম ন য গ eব বn tপ ণর service om/journal/data positive খ ব র ভ ল ছল food positive Table 9 shows complete statistics Bangla Restaurant dataset. We can see from table Table that 9five shows different complete categories, statistics that is, Food, Price, Bangla Service, Restaurant Ambiance, dataset. Miscellaneous, We can see contained from table that five713, different 178, 336, categories, 234, 613 that reviews, is, Food, respectively, Price, Service, with three Ambiance, different polarities. Miscellaneous, For example, contained 713, 178, 336, category 234, Food 613contained reviews, 500 respectively, positive, 126 with negative, three different 87 neutral polarities. sentiment For labels. example, The Service category Food contained category contained 500 positive, 186 positive, 126 negative, 118 negative, neutral sentiments. labels. We also The found Service that this category Restaurant dataset also follod Zipf s law, which is shown in Figure 2. Table 9. Complete statistics Bangla Restaurant dataset. Polarity Category Total Positive Neutral Food

8 Data 2018, 3, contained 186 positive, 118 negative, 32 neutral sentiments. We also found that this Restaurant dataset also follod Zipf s law, which is shown in Figure 2. Category Table 9. Complete statistics Bangla Restaurant dataset. Polarity Positive Neutral Total Food Price Service Ambiance Figure 1. Distribution word frequencies dataset using Zipf s law. Miscellaneous Baseline Evaluation Figure Word frequency Bangla Restaurant dataset accordingto to Zipf s law. law. Our objective is to provide benchmark datasets for Bangla ABSA. Our datasets are designed for two major tasks ABSA. These are aspect category extraction identification polarity for each aspect category. In this paper, experimented with first subtask, that is, extraction aspect category. We applied three major steps to extract aspect category. Firstly, preprocessing was performed on dataset. After this, extracted features from data finally performed classification using some popular classification models Preprocessing Feature Extraction In preprocessing phase, each Bangla document was represented as a bag words. We applied traditional preprocessing steps for evaluation. Firstly, punctuations stop words Data 2018, 3, x; do i: om/journal/data re removed from each comments. After this, removed digits from our dataset, because found that digits re not necessary for aspect category. Finally, tokenized each Bangla word from our dataset. Thus, a vocabulary Bangla words was prepared after preprocessing. We created a feature matrix for which each review was represented by a vector that vocabulary. Term frequency inverse document frequency (TF IDF) was used for calculating features Results In training phase, extracted feature sets re trained by popular supervised machine learning algorithms. Because this was a multi-label classification problem, trained our models

9 Data 2018, 3, by setting up multi-label output. We used linear SVC in support vector machine (SVM) implementation. The following machine learning algorithms re used: I. Support Vector Machine (SVM) II. III. Rom forest (RF) K-nearest neighbor (KNN) After training was completed, our proposed Bangla test dataset was executed on trained model. The result is shown in following table figure. Table 10 shows results for task aspect category extraction datasets have presented in this paper. We can see that using SVM, obtained highest precision rate for both datasets. Both datasets shod a low recall F1-score. Figure 3 shows overall accuracy models using our datasets. The inherent nature datasets is reason behind lor performance models for both datasets. People share ir opinion with ir individual judgment. Therefore, variety opinions in datasets is much larger. On or h, aspect extraction is a multi-label classification problem. One s opinion might have multiple aspect categories. Conventional classifiers miss some se aspect categories. Table 10. Performance proposed datasets. Dataset Model Precision Recall F1-Score Restaurant SVM RF KNN SVM RF KNN Data 2018, 3, x Accuracy SVM RF KNN SVM RF KNN Restaurant Precision Recall F1-score Figure 3. The result three models our datasets. Figure 3. The result three models our datasets. 4. Conclusions Future Work Two datasets are provided for ABSA Bangla text. These datasets have been designed to perform two tasks covering aspect category extraction identification polarity for that aspect category. We also report baseline results to evaluate task aspect category extraction. As future plans, aim to enhance our work by including furr domains such as cars, mobiles, laptops. We are working on more advanced methods for ABSA Bangla text using our datasets to achieve better performance. These results can be improved if process train datasets in a more sophisticated way. In this work, have taken all vocabulary as features for evaluation after removing punctuation, stop words, digits. Some state---art techniques for information gain can be applied to dataset before classification after preprocessing steps to attain better results. 4. Conclusions Future Work Author Contributions: All authors contributed equally to this work, have read approved final manuscript. Two datasets are provided for ABSA Bangla text. These datasets have been designed to Conflicts Interest: The authors declare no conflict interest. perform two tasks covering aspect category extraction identification polarity for that aspect References category. We also report baseline results to evaluate task aspect category extraction. 1. Trusov, M.; Bucklin, R.E.; Pauls, K. Effects word--mouth versus traditional marketing: Findings from an internet social networking site. J. Mark. 2009, 73, Jeyapriya, A.; Selvi, C.K. Extracting Aspects Mining Opinions in Product Reviews Using Supervised Learning Algorithm. In Proceedings nd International Conference on Electronics Communication Systems (ICECS), Coimbatore, India, February Pontiki, M.; Galanis, D.; Pavlopoulos, J.; Papageorgiou, H.; Androutsopoulos, I.; Manhar, S. SemEval-2014 Task 4: Aspect Based Sentiment Analysis. Available Online:

10 Data 2018, 3, As future plans, aim to enhance our work by including furr domains such as cars, mobiles, laptops. We are working on more advanced methods for ABSA Bangla text using our datasets to achieve better performance. Author Contributions: final manuscript. All authors contributed equally to this work, have read approved Conflicts Interest: The authors declare no conflict interest. References 1. Trusov, M.; Bucklin, R.E.; Pauls, K. Effects word--mouth versus traditional marketing: Findings from an internet social networking site. J. Mark. 2009, 73, [CrossRef] 2. Jeyapriya, A.; Selvi, C.K. Extracting Aspects Mining Opinions in Product Reviews Using Supervised Learning Algorithm. In Proceedings nd International Conference on Electronics Communication Systems (ICECS), Coimbatore, India, February Pontiki, M.; Galanis, D.; Pavlopoulos, J.; Papageorgiou, H.; Androutsopoulos, I.; Manhar, S. SemEval-2014 Task 4: Aspect Based Sentiment Analysis. Available online: (accessed on 3 May 2018). 4. Al-Smadi, M.; Qawasmeh, O.; Talafha, B.; Quwaider, M. Human Annotated Arabic Dataset Book Reviews for Aspect Based Sentiment Analysis. In Proceedings rd International Conference on Future Internet Things Cloud (FiCloud), Rome, Italy, August Tamchyna, A.; Fiala, O.; Veselovská, K. Czech Aspect-Based Sentiment Analysis: A New Dataset Preliminary Results. Available online: (accessed on 3 May 2018). 6. Apidianaki, M.; Tannier, X.; Richart, C. Datasets for Aspect-Based Sentiment Analysis in French. Available online: (accessed on 3 May 2018). 7. Gayatree, G.; Elhadad, N.; Marian, A. Beyond Stars: Improving Rating Predictions Using Review Text Content. Available online: type=pdf (accessed on 3 May 2018). 8. Kiritchenko, S.; Zhu, X.; Cherry, C.; Mohammad, S. NRC-Canada-2014: Detecting Aspects Sentiment in Customer Reviews. Available online: (accessed on 3 May 2018). 9. Kiritchenko, S.; Zhu, X.; Cherry, C.; Mohammad, S. Supervised Unsupervised Aspect Category Detection for Sentiment Analysis with Co-cccurrence Data. In IEEE Transactions on Cybernetics; IEEE: Piscataway, NJ, USA, Soujanya, P.; Cambria, E.; Gelbukh, A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowl.-Based Syst. 2016, 108, Pengfei, L.; Joty, S.; Meng, H. Fine-Grained Opinion Mining with Recurrent Neural Networks Word Embeddings. Available online: (accessed on 3 May 2018). 12. Pontiki, M.; Galanis, D.; Papageorgiou, H.; Manhar, S.; Androutsopoulos, I. Semeval-2015 Task 12: Aspect Based Sentiment Analysis. Available online: (accessed on 3 May 2018). 13. Pontiki, M.; Galanis, D.; Papageorgiou, H.; Androutsopoulos, I.; Manhar, S.; AL-Smadi, M.; Al-Ayyoub, M.; Zhao, Y.; Qin, B.; De Clercq, O.; et al. SemEval-2016 Task 5: Aspect Based Sentiment Analysis. Available online: (accessed on 3 May 2018). 14. Pak, A.; Paroubek, P. Twitter as A Corpus for Sentiment Analysis Opinion Mining. Available online: (accessed on 3 May 2018) by authors. Licensee MDPI, Basel, Switzerl. This article is an open access article distributed under terms conditions Creative Commons Attribution (CC BY) license (