Tecof • September 15, 2026 • 12 dk
AI Customer Review Analysis: Sentiment at Scale
In Brief
Customer review analysis means reading free-text feedback by machine and tagging sentiment and topic together, so the result becomes measurable. Sentiment analysis is only part of that job: it says whether a sentence is positive or negative, not why. The setup that works tags sentiment with topic, pulling two signals out of "shipping was fast but the fabric is thin". As of 2026 the question is no longer whether we can have AI read our reviews, but which decision we are willing to change on the strength of what it reads.
Tuesday morning, 09.20. The weekly meeting of a women's apparel brand. On screen: a Trendyol rating of 4.6 and 4.4 on their own site, both unchanged from last month. Beside them, the return rate has climbed from 18 to 26 percent in three months. Over 90 days 7,400 reviews have piled up; customer service has read perhaps 300, only because a complaint came in. That afternoon, once 2,100 reviews are topic-tagged, the picture is clear: 41 percent of negative reviews concern size and fit, three quarters clustered in three products from one supplier. The average never showed it: "lovely fabric" comments on those same products held it up.
The problem is not the number of reviews but the compression of each into one number. A star average gives an overall impression; it does not say which team must do what. A five-star customer may write "shipping was three days late", a two-star one "great product but it runs small". The review text already holds an operational roadmap. It has simply not been read.
What Sentiment Analysis Is and Is Not
Polarity and topic are two separate things
Classic sentiment analysis assigns a polarity: positive, negative, neutral. That is a number you can report but not act on: "negative review share is 22 percent" tells nobody what to do. The topic tag, the aspect, says what the text is about: quality, size, shipping, packaging, price, description accuracy, returns. Use both and you get "negative share on sizing is 41 percent" — work you can hand a team. The common mistake is starting with polarity and postponing topic tagging; that is how you get a dashboard nobody opens.
Core terms
- Polarity: The sentiment direction of a text or a fragment within it. Three classes are enough; five-point scales reduce accuracy.
- Aspect (topic): Which dimension of the product or service the review is about. Chosen from a closed list, never free text.
- Aspect-based sentiment: A separate polarity for each topic inside one review. A single review produces several rows.
- Tag schema: The written form of your topic list and each topic's definition. Without it the measurement cannot be repeated.
- Validation set: A hand-labelled sample the model has never seen. The only honest measure of accuracy you have.
One review can carry two sentiments
"Shipping arrived very fast but the fabric is thinner than I expected" is one review and two signals. Give it one polarity and you either hand logistics a negative they did not earn or lose a warning buying needed. An aspect-based setup produces two rows: shipping/positive, quality/negative. Roughly a third of reviews carry more than one topic, more in fashion and home textiles. Ask an open-ended "what is this about" and the model invents new names weekly, so no trend holds; fix the list, keep one "other" bucket, and read it by hand monthly. Seven to twelve tags suit most catalogues.
Reading by hand versus reading by machine
Manual reading is more accurate but small in coverage and not repeatable: give the same 200 reviews to two people and some tags will not match. Machine reading errs more often but applies one criterion every week. Combine them: the machine tags everything, a human audits a sample weekly.
| Criterion | Manual reading | AI analysis | Hybrid (recommended) |
|---|---|---|---|
| Reviews covered per month | 200-400 | All of them (10,000+) | All + 200 audited |
| Time spent | 14-20 hours | 1-2 hours after setup | 3-4 hours |
| Monthly cost range | TRY 6,000-12,000 in labour | TRY 800-2,500 model and infrastructure | TRY 3,000-5,000 |
| Topic tag accuracy | Above 95% | 82-92% | 90-95% |
| Repeatability | Low, varies by reader | High | High |
What Makes Turkish Reviews Harder
Negation suffixes and irony
In Turkish, negation is a suffix rather than a separate word: "beğendim" (I liked it) and "beğenmedim" (I did not) differ by two letters, so older word-based methods fail systematically. Irony is harder: in "harika, üç haftada geldi" ("great, it arrived in three weeks") "great" is positive while the sentence is negative. Current models read both largely correctly, but keep ironic examples in your validation set; irony is the largest single source of model error we see.
Abbreviations, emoji and typos
Reviews are written in chat register: "urun güzl ama kargo rezalet", "tşk", "bdn kücük geldi". Turkish characters typed on an English keyboard, missing punctuation and emoji are normal. Do not over-clean: lowercasing and trimming repeated characters is enough. Leave emoji in — models read them correctly, and in a five-word review an emoji may be the only sentiment marker.
Brand names, product names and model confusion
"Brand X was better" can read as negative about your product and positive about the brand it names. Product names cause trouble too: in "the peter pan collar blouse is very tight", the model sometimes mistakes the name for the topic. State in the prompt which product is judged and pass the SKU. The only criterion in model choice is Turkish performance: run one validation set through two or three models with the same prompt rather than trusting English benchmarks. For the practical side of prompt writing, our guide to prompt engineering in e-commerce is useful.
Data Sources and a Single Dashboard
Your own site and the marketplaces
Reviews on your own site are cleanest: they map to an order, an SKU and a customer record, so they answer which product, which size, which carrier. Trendyol and Hepsiburada reviews are larger but weaker in matching, and shipping complaints there may not reflect your operation. Keep both in one dashboard with a source field, and never compare rates without splitting by source.
Sources other than reviews are often sharper
Return reasons, support tickets and carrier delivery feedback carry the same tag schema. Return reasons are short and direct, so accuracy runs higher. Undelivered records from Yurtiçi and Aras validate the shipping tag: when a complaint and a carrier record point at one region, the decision is settled. For how this lands on the warehouse side, see the structure we build for logistics and warehouse management.
Merging into one table
Collect the sources into one table: record id, source, date, SKU, category, topic tag, polarity, model confidence, text. A manual CSV transfer is fine to start; as volume grows you set up the integration layer and move to an automatic flow.
Setup: From Tag Schema to Accuracy Threshold
Pulling the reviews
For your own site, a database export or an API is enough. On the marketplace side, start with a seller panel export and add a daily pull if you have API access. Limit the first round to 90 days; older data helps trends but slows setup.
Tag schema and prompt
Write the schema on one page: tag name, definition, two examples, edge cases. Where "shipping was late but the courier was very polite" belongs must be written down. Build the prompt on it; ask for structured topic-polarity pairs per review with a confidence value each. Do not ask for free-text commentary; you cannot tabulate it.
A validation set of 200 reviews
Have two people independently label 200 random reviews by hand, then settle the differences. The model must never see this set. Measure topic and polarity accuracy against it separately; one combined number misleads, because polarity scores higher and the average hides the weakness.
Threshold and decision
Wiring a below-threshold setup into a dashboard makes you change suppliers on a wrong number. Put nothing below 85 percent into production, and write that rule down up front so it stays non-negotiable when the project runs late.
| Accuracy range | Reliability | Decision | Next step |
|---|---|---|---|
| Below 60% | Close to random | Not for production | Rewrite the tag schema, merge tags |
| 60-75% | Gives direction, not figures | Exploratory only | Add examples to the prompt, change model |
| 75-85% | Trend readable, rates not | Limited use | Separate the two tags that keep colliding |
| 85-92% | Sufficient for weekly decisions | Goes into production | Keep the monthly 200-review audit running |
| Above 92% | High | Automatic alerts can be set | Notify the owning team on threshold breach |
From Result to Action: Review to Product Page
Give every tag an owner
If a tag has no owner, the measurement stays a report and never becomes a decision. Write down, next to each tag, which team looks at it and the typical action. The table below is the starter schema we use in fashion and home; add and remove tags to suit your catalogue, but keep the columns.
| Topic tag | Typical phrasing | Owning team | Typical action |
|---|---|---|---|
| Size / fit | "I should have ordered a size up" | Product / category | Update size chart and model measurements on imagery |
| Quality | "Thin fabric, shrank in the first wash" | Buying | Repeat supplier sampling and wash testing |
| Shipping | "Took four days, the courier never called" | Operations | Change the carrier mix by region |
| Packaging | "The box was crushed, it came in a bag" | Warehouse | Change box type for fragile categories |
| Description accuracy | "This is not the colour in the photo" | Content | Correct product photography and colour naming |
| Returns process | "I waited three days for the return label" | Customer service | Automate return codes and commit to a turnaround |
Product page corrections
Where negatives cluster on size and fit, the first job is the size chart: real measurements in centimetres, the size worn by the model, a "we recommend sizing up" note where it applies. This usually pulls that product's return rate down a few points within weeks. The description accuracy tag points at the copy; when writing product descriptions with AI, feeding in the objections that surface in reviews is among the fastest wins available.
Packaging and carrier decisions
Read packaging negatives by category and box type; usually one category and one box size are responsible. Read the shipping tag by carrier and region. The gap between Yurtiçi and Aras often sharpens in particular provinces rather than nationwide; deciding region by region works better and leaves room to negotiate.
Return rate by category and conversion
Map topic tags onto return reasons and you see which categories return on content and which on the product. Content-driven returns fall with a product page fix, and the same fix lifts conversion; the cheapest input into improving your conversion rate is usually your own review archive.
Replying to Reviews: Machine Drafts, Human Approves
Write the authority limit in advance
A setup that drafts replies must know what it may and may not say. Promises that create cost — accepting a return, covering shipping, issuing a discount code — are human decisions, not draft material. At Tecof, reply drafting can be set up with defined authority limits and an approval step; the AI agents side covers the detail.
Crisis reviews go to a human
Reviews mentioning health, safety, allergies, children's products, legal threats or the press go straight to the responsible person. Do not leave that split to model confidence; let a keyword list and the topic tag trigger it together, and send doubtful cases to a human. The cost of erring here is asymmetric.
Measure the quality of the replies too
Track weekly how many drafts go out unedited. Below 70 percent, either your prompt or your authority limit is wrong. Response time and the rating change after a reply are measurable too; without both, "AI writes our replies" is a status, not a result.
KVKK, Anonymisation and the Fake Review Question
Review text can contain personal data
Customers write names, phone numbers, order numbers and sometimes addresses into reviews. Processing that text puts you within scope of KVKK, Turkey's personal data protection law. Before sending anything to a model, mask phone, email, order number and national ID patterns; a simple regular expression layer does this without costing accuracy. State in your privacy notice that reviews are analysed automatically to improve service.
Transfer and retention
If the provider sits outside Turkey, observe the cross-border transfer regime, put contractual safeguards in place and get a written commitment that your data is not used for training. Set a retention period for raw text; keeping the labelled numeric output long-term is safer than keeping raw text indefinitely, and enough for analysis.
Do not generate fake reviews
This needs saying plainly: using AI to generate fake positive reviews, or to suppress negative ones systematically, is a thing not to do. It is misleading commercial practice, breaches marketplace seller agreements, and when detected carries penalties as far as store closure. You also poison your own measurement: a fabricated archive yields a topic distribution worth nothing. The value here rests on the data being real.
Setting Up Review Analysis in 30 Days
Days 1-7: get the data into one place
Pull the last 90 days of your own reviews, marketplace reviews, return reasons and support tickets into one table. Fields: record id, source, date, SKU, category, text. Do no analysis this week; look at volume, duplicates and empty records. Note how many reviews match an SKU; if the match rate is low, fix that first.
Days 8-14: tag schema and validation set
Define seven to twelve topic tags with a definition and two examples each. Have two people label 200 random reviews independently and resolve disagreements. Write the prompt the same week and take the first model output. Measure topic and polarity accuracy separately and record that result as it stands; every later attempt is compared against it.
Days 15-21: clear the threshold and wire up the dashboard
If accuracy sits below 85 percent, simplify the tags, merge the two that keep colliding and add edge-case examples. Once you clear it, process the whole archive and build the dashboard: negative share by topic, breakdowns by category, SKU and source, weekly trend.
Days 22-30: run the action loop
Hand the top three tags to their owning teams and make one concrete change each: a size chart update, a box type change, a carrier trial in one region. Mark the change date on the dashboard so you can isolate its effect later. On the last day of the month, take a fresh 200-review audit sample and check whether accuracy slipped.
Here is the job for tomorrow morning: export the last 90 days of reviews into a single table, pick 200 of them at random and label them by hand with two people; those 200 rows will give you both the first version of your tag schema and the one honest yardstick you can hold every model you build from here against.
Frequently Asked Questions
At what review volume does this become worth doing?
Below 300-500 reviews a month, reading by hand is cheaper and more accurate; the return does not cover the cost. Past 1,000 a month, or across more than three or four sources, machine tagging makes sense.
Should I buy a ready-made tool or build it myself?
What decides it is whether the tag schema belongs to you. Most off-the-shelf tools arrive with generic topic lists and miss the distinctions specific to your category. Writing your own schema and having a language model classify against it is usually cheaper and sharper.
Is 85 percent accuracy enough, and why not aim higher?
It is enough for weekly rate comparisons and trend reading, because errors are broadly random and do not distort the gap between rates. Aiming higher means cutting tags, and past a point you lose information. To decide automatically on a single review — approving a return — 85 percent is not enough.
Should marketplace reviews sit in the same pool as my own?
Keep them in the same table, but always read the source field as a breakdown. Shipping complaints there can come from processes outside your control; folding them into your own site's rates gives logistics an unfair picture.
Should I strip emoji and fix typos?
No. Lowercasing and trimming repeated characters is enough. Emoji are often the only sentiment marker in a short review. Correcting typos is unnecessary too; models read broken spelling largely correctly.
Does the model really catch irony?
Partly. Its hit rate on explicit irony is good and drops on implicit cases. Put ironic examples into your validation set deliberately and measure that subset separately. If it sits well below your overall accuracy, read results cautiously where irony is common.
Is sending review text to a model provider a KVKK problem?
Not a problem, but it has conditions. Mask personal data patterns, mention automatic analysis in your privacy notice, put contractual safeguards in place and observe the cross-border transfer regime. With those done, analysis to improve service is lawful.
Is deleting or hiding negative reviews a solution?
It is not. Selectively publishing reviews misleads consumers, and marketplace reviews are outside your control anyway. The measurable return comes from the operational fix that removes the complaint at its source; resolving it and replying beats deleting.
Who reads the dashboard, and how often?
One person each from product, operations and customer service is enough for the weekly read, and the meeting should not run past 20 minutes. The monthly read covers return rate by category and shifts in topic distribution. When the dashboard belongs to nobody, the setup is abandoned within months — the most common failure we see.
How do I know if the model output degrades over time?
Through the monthly audit sample of 200 reviews. If accuracy drops more than five points across two consecutive measurements, either the provider changed the model version or your product range shifted and the reviews moved with it. The fix is the same: add new examples to the validation set and update the prompt.