admin/trendyol-analiz
1
0
Fork 0
mirror of https://github.com/nethunterzist/trendyol-analiz synced 2026-08-29 21:38:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix(scraper): mensei ve barkod tablarini doldur — Top Rankings enrichment genislet

Browse Source 
Ne yaptik:
- Enrichment kosulunu 'not any socialProofs' yerine 'not any winnerVariant' yaptik
- Top Rankings API'den winnerVariant, merchantListings ve socialProofs'u birlestiriyoruz
- Search API urunlerinde winnerVariant bos gelir, bu kosul Search API orijinli urunleri dogru tespit eder

Neden yaptik:
- Search API winnerVariant ve merchantListings dondurmuyor (bos dict/liste)
- data_consolidator.py barkodu winnerVariant.barcode'dan, menseiyi merchantListings[].customValues'dan aliyor
- Search API'de socialProofs dolu geldiginden eski kosul ('not any socialProofs') hic tetiklenmiyordu
- Sonuc: Mensei ve Barkod tablari sifir gosteriyordu (kpek urunleri analizi #118 bug'i)
This commit is contained in:
furkanyigit34
2026-07-16 14:15:48 +03:00
parent 0fa034c67d
commit 263295e97c
1 changed files with 24 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
backend/queue_worker.py
View File

@@ -179,25 +179,37 @@ class QueueWorker:
if products:
log.info(f"Fallback başarılı: {len(products)} ürün bulundu (cat_id={cat_id})")
# Enrich with socialProofs from Top Rankings API
if products and cat_id and not any(p.get("socialProofs") for p in products):
# Enrich barcode, origin (winnerVariant/merchantListings) and socialProofs
# from Top Rankings API. Search API returns empty winnerVariant/merchantListings,
# so barcode and country-of-origin data is missing without this step.
if products and cat_id and not any(p.get("winnerVariant") for p in products):
try:
_trendyol_limiter.wait()
top_scraper = TrendyolScraper(cat_id, page_size=20)
top_products = top_scraper.fetch_all_products(delay=0.5, max_pages=5)
if top_products:
social_map = {}
top_map = {}
for tp in top_products:
tid = tp.get("id") or tp.get("contentId")
sp = tp.get("socialProofs", [])
if tid and sp:
social_map[int(tid)] = sp
if social_map:
for p in products:
pid = p.get("id")
if pid and int(pid) in social_map:
p["socialProofs"] = social_map[int(pid)]
log.info(f"Enriched {len(social_map)} products with socialProofs")
if tid:
top_map[int(tid)] = tp
enriched = 0
for p in products:
pid = p.get("id")
if pid and int(pid) in top_map:
tp = top_map[int(pid)]
wv = tp.get("winnerVariant") or {}
ml = tp.get("merchantListings") or []
sp = tp.get("socialProofs") or []
if wv:
p["winnerVariant"] = wv
if ml:
p["merchantListings"] = ml
if sp and not p.get("socialProofs"):
p["socialProofs"] = sp
enriched += 1
if enriched:
log.info(f"Enriched {enriched} products with barcode/origin/socialProofs from Top Rankings")
except Exception as e:
log.warning(f"Top Rankings enrichment failed: {e}")